MiMo-V2.6 Distill Qwen 9B vs Qwen3-VL-2B-Instruct

MiMo-V2.6 Distill Qwen 9B

A 9B cousin tuned on MiMo-V2.6's agent work for coding, terminal tasks and visual coding, released as a base for RL research. It needs about 23 GB in BF16, since Xiaomi ships no smaller build. The main risks are no safety results while a seventh of its tuning is offensive security, and nobody at Xiaomi answering reports.

Qwen3-VL-2B-Instruct

The smallest Qwen3-VL cousin, for reading screenshots, documents, charts and video on a laptop or edge box, running in about 2 GB from Alibaba's Q4_K_M GGUF in every major runtime under plain Apache 2.0. Qwen hasn't answered repository questions or published safety work, so keep a human or a check in the loop before it drives a GUI.

MiMo-V2.6 Distill Qwen 9B

1. Serve it as the card does, sglang serve --model-path XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B --reasoning-parser mimo, and add --tool-call-parser mimo for parsed tool calls
2. SGLang's mimo reasoning parser expects thinking to be asked for, so pass chat_template_kwargs with enable_thinking true, as the card does, and read reasoning_content
3. Calls come back as function and parameter blocks inside tool_call, the same shape vLLM's and SGLang's qwen3_coder parsers read
4. The generation config samples at temperature 0.6, top_k 20 and top_p 0.95
5. Images and video go in through Qwen3.5's processor; the template's audio tokens have no audio encoder behind them
6. The MTP head isn't in the files, so skip MTP speculative decoding even though config.json lists a layer
7. It's an SFT checkpoint meant for further RL, and discussions report runaway generation on llama.cpp, so cap max_tokens

Qwen3-VL-2B-Instruct

1. Start it with llama-server -hf Qwen/Qwen3-VL-2B-Instruct-GGUF:Q4_K_M --jinja -c 32768, or ollama run qwen3-vl:2b-instruct
2. In Ollama the plain qwen3-vl:2b tag is the Thinking edition, so name 2b-instruct
3. With the GGUF, load the mmproj vision file too, or images won't be read
4. Parse tool calls as JSON inside <tool_call> tags, or use SGLang's --tool-call-parser qwen
5. Use the card's sampling, temperature=0.7, top_p=0.8, top_k=20, presence_penalty=1.5 for vision and temperature=1.0, top_p=1.0, top_k=40, presence_penalty=2.0 for text
6. Validate JSON output yourself, since structured output isn't documented
7. vLLM needs 0.11.0 or later and Transformers 4.57.0 or later