Gemma 4 31B IT vs Qwen3.8-27B

Gemma 4 31B IT

A dense multimodal cousin with documented tool calling, optional thinking and a 256K context, for general agent work and image reading on one 24 GB GPU through Google's 4-bit build. It runs in every major runtime under plain Apache 2.0. The main risk is that Google publishes no safety results and has gone quiet on the repository since July.

Qwen3.8-27B

A dense cousin for coding and agent work with image and video input that runs almost anywhere. Ollama's 4-bit build needs about 22 GB, and five providers host it for agents without the hardware. The main risk is that nobody at Alibaba answers, there are no safety results or agent guidance, and the tool format is only in the template.

Gemma 4 31B IT

1. On the reference machine run llama-server -hf google/gemma-4-31B-it-qat-q4_0-gguf --hf-file gemma-4-31B_q4_0-it.gguf --jinja -c 32768, adding gemma-4-31B-it-mmproj.gguf for images
2. Or ollama run gemma4:31b for the 20 GB Q4_K_M build
3. In vLLM serve with --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4, as vLLM's recipe does
4. Thinking is off by default, so pass enable_thinking: true in chat_template_kwargs to turn it on, and expect the reasoning in the thought channel block
5. Drop earlier thoughts from the history except on tool-call turns, as the card asks
6. Use the card's sampling, temperature 1.0, top_p 0.95, top_k 64
7. Validate function names and arguments before running a call, and check JSON output yourself or use the runtime's guided decoding
8. The w4a16 build relies on Marlin INT4 kernels that vLLM marks unsupported on AMD, so use the GGUF on the reference machine

Qwen3.8-27B

1. Quickest start is ollama run qwen3.8, about 18 GB of files
2. On our reference machine use BF16 with vllm serve Qwen/Qwen3.8-27B --max-model-len 131072 --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder, since FP8 on gfx1151 isn't documented
3. Use the qwen3_coder tool parser, SGLang's cookbook says the Hermes parser leaves its calls unparsed
4. Thinking is on by default, pass chat_template_kwargs with enable_thinking false for short tasks, and read the reasoning from reasoning_content
5. reasoning_effort takes xhigh (default), medium or low, and any other value makes the template raise an error
6. Sampling per the card is temperature 1.0 and top_p 0.95 when thinking, temperature 0.7, top_p 0.8 and presence_penalty 1.5 without
7. Preserved thinking keeps earlier reasoning in the context unless preserve_thinking is false, which grows long agent sessions
8. Turn on YaRN only beyond 262,144 tokens, the card warns static YaRN can cost quality on short texts
9. Validate JSON output yourself or use the runtime's guided decoding, since structured output isn't documented