MiniCPM5-2B vs SmolLM3-3B

MiniCPM5-2B

A tiny dense cousin for local tool calling and long context, with published training data and an active publisher. OpenBMB's own 4-bit builds need about 2 GB, so it runs on a laptop or phone in vLLM, SGLang, llama.cpp, Ollama or Transformers. The main risks are always-on thinking, no safety results and use conditions beyond Apache 2.0.

SmolLM3-3B

A tiny, fully open cousin with documented tool calling and switchable thinking, for local subtasks and routing on laptops and CPUs. It needs about 8 GB in BF16, or under 3 GB in Hugging Face's 4-bit ONNX build for ONNX Runtime. The main risks are a 64K native context, no safety results and no repository changes since September 2025.

MiniCPM5-2B

1. Smallest start is ollama run openbmb/minicpm5-2b (1.6 GB, q4_k_m)
2. On vLLM 0.29.0 or later run vllm serve openbmb/MiniCPM5-2B --max-model-len 131072 --enable-auto-tool-choice --tool-call-parser minicpm5, or tool calls come back as XML text
3. On SGLang 0.5.16 or later add --tool-call-parser minicpm5 --reasoning-parser qwen3; SGLang is OpenBMB's recommended backend for tool calls
4. Tool calls are XML, <function name=...><param name=...>, not Hermes JSON
5. Sample with temperature 1.0, top_p 0.95 and min_p 0.0; llama.cpp's default min_p 0.05 can loop, and repetition_penalty 1.05 helps if it does
6. Expect a <think> block before every answer and leave room in max_tokens; enable_thinking false is accepted by the template but OpenBMB doesn't support it for the 2B
7. Validate JSON yourself or use the runtime's guided decoding, since structured output isn't documented
8. Use the main repository's template; the GGUF, MLX and GPTQ copies predate the 29 September fix

SmolLM3-3B

1. Start it with vllm serve HuggingFaceTB/SmolLM3-3B --enable-auto-tool-choice --tool-call-parser=hermes, or tool calls come back as text
2. Pass chat_template_kwargs with enable_thinking false for short tasks, since thinking is on by default; a /think or /no_think flag in the system prompt overrides the kwarg
3. Put earlier tool calls in the assistant message content, since the template ignores a tool_calls field when it renders history
4. Add the YaRN rope_scaling block (factor 2.0, original 65536) to config.json before sending more than 65,536 tokens
5. Use temperature 0.6 and top_p 0.95, the card's recommended sampling
6. Needs Transformers v4.53.0 or later; for llama.cpp use a GGUF from ggml-org or another converter, since Hugging Face publishes none
7. Validate JSON output yourself or use the runtime's guided decoding