Gemma 4 26B A4B IT vs Granite 4.2 3B

Gemma 4 26B A4B IT

A fast mixture-of-experts cousin for general agent work with tools, optional thinking and image input, under Apache 2.0. Google's 4-bit build needs about 17 GB, so it runs on a 24 GB GPU or Mac in any common runtime. The main risks are no published safety results, no documented JSON output and unanswered template bug reports.

Granite 4.2 3B

A small reasoning cousin for tool calls, coding and multilingual chat that runs almost anywhere, from a 2.2 GB Ollama build to vLLM and SGLang, under plain Apache 2.0 with signed files and a dataset disclosure. Thinking turns off or shortens per request. It has no published safety results and no documented JSON mode.

Gemma 4 26B A4B IT

1. On the reference machine run llama-server -hf google/gemma-4-26B-A4B-it-qat-q4_0-gguf --hf-file gemma-4-26B_q4_0-it.gguf --jinja -c 65536, and add the mmproj file for images
2. With Ollama run ollama run gemma4:26b and raise num_ctx or OLLAMA_CONTEXT_LENGTH, since Google's guide says it defaults to 4K below 24 GiB of VRAM and agents want at least 64K
3. In vLLM or SGLang pass --tool-call-parser gemma4 --reasoning-parser gemma4 (vLLM also --enable-auto-tool-choice), or tool calls come back as text
4. Thinking is off by default; pass enable_thinking=True or start the system prompt with <|think|>, and drop earlier turns' thinking from history except on tool-call turns
5. Validate tool names and arguments before running them, and check any JSON yourself

Granite 4.2 3B

1. Start it with ollama run granite4.2:3b; set options.num_ctx in the request, since Ollama's default context is short and the model takes up to 131,072
2. With vLLM 0.32.0 or later, serve ibm-granite/granite-4.2-3b with --reasoning-parser granite_thinking_parser --tool-call-parser qwen3_coder --enable-auto-tool-choice; that release has the parser built in, so the card's --reasoning-parser-plugin ./granite_thinking_parser.py isn't needed
3. With SGLang 0.5.18 or later, --reasoning-parser auto --tool-call-parser auto resolve to IBM's recommended parsers, and the reasoning comes back in reasoning_content
4. Use temperature 1.0 and top_p 0.95 for every mode, as IBM requires
5. Pass enable_thinking false for direct answers or low_effort true for short reasoning, through chat_template_kwargs on an OpenAI-compatible server
6. Earlier turns' thinking is stripped by default (truncate_history_thinking), so pass it false to keep reasoning in multi-turn history
7. For JSON, use the runtime's guided decoding, since IBM documents no JSON mode for 4.2
8. Verify the files with python -m model_signing verify sigstore and IBM's identity, per IBM's signature guide