Gemma 4 26B A4B IT vs MiMo-V2.6 Distill Qwen 9B

Gemma 4 26B A4B IT

A fast mixture-of-experts cousin for general agent work with tools, optional thinking and image input, under Apache 2.0. Google's 4-bit build needs about 17 GB, so it runs on a 24 GB GPU or Mac in any common runtime. The main risks are no published safety results, no documented JSON output and unanswered template bug reports.

MiMo-V2.6 Distill Qwen 9B

A 9B cousin tuned on MiMo-V2.6's agent work for coding, terminal tasks and visual coding, released as a base for RL research. It needs about 23 GB in BF16, since Xiaomi ships no smaller build. The main risks are no safety results while a seventh of its tuning is offensive security, and nobody at Xiaomi answering reports.

Gemma 4 26B A4B IT

1. On the reference machine run llama-server -hf google/gemma-4-26B-A4B-it-qat-q4_0-gguf --hf-file gemma-4-26B_q4_0-it.gguf --jinja -c 65536, and add the mmproj file for images
2. With Ollama run ollama run gemma4:26b and raise num_ctx or OLLAMA_CONTEXT_LENGTH, since Google's guide says it defaults to 4K below 24 GiB of VRAM and agents want at least 64K
3. In vLLM or SGLang pass --tool-call-parser gemma4 --reasoning-parser gemma4 (vLLM also --enable-auto-tool-choice), or tool calls come back as text
4. Thinking is off by default; pass enable_thinking=True or start the system prompt with <|think|>, and drop earlier turns' thinking from history except on tool-call turns
5. Validate tool names and arguments before running them, and check any JSON yourself

MiMo-V2.6 Distill Qwen 9B

1. Serve it as the card does, sglang serve --model-path XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B --reasoning-parser mimo, and add --tool-call-parser mimo for parsed tool calls
2. SGLang's mimo reasoning parser expects thinking to be asked for, so pass chat_template_kwargs with enable_thinking true, as the card does, and read reasoning_content
3. Calls come back as function and parameter blocks inside tool_call, the same shape vLLM's and SGLang's qwen3_coder parsers read
4. The generation config samples at temperature 0.6, top_k 20 and top_p 0.95
5. Images and video go in through Qwen3.5's processor; the template's audio tokens have no audio encoder behind them
6. The MTP head isn't in the files, so skip MTP speculative decoding even though config.json lists a layer
7. It's an SFT checkpoint meant for further RL, and discussions report runaway generation on llama.cpp, so cap max_tokens