Gemma 4 31B IT vs Inkling

Gemma 4 31B IT

A dense multimodal cousin with documented tool calling, optional thinking and a 256K context, for general agent work and image reading on one 24 GB GPU through Google's 4-bit build. It runs in every major runtime under plain Apache 2.0. The main risk is that Google publishes no safety results and has gone quiet on the repository since July.

Inkling

Thinking Machines' large multimodal generalist, stronger than Inkling-Small on factual recall by its own numbers. It needs about 710 GB of datacentre GPU memory even in NVFP4, so most agents will use a host such as Together AI or Tinker. The main risks are a use policy that binds on download and can change, and no outside safety results.

Gemma 4 31B IT

1. On the reference machine run llama-server -hf google/gemma-4-31B-it-qat-q4_0-gguf --hf-file gemma-4-31B_q4_0-it.gguf --jinja -c 32768, adding gemma-4-31B-it-mmproj.gguf for images
2. Or ollama run gemma4:31b for the 20 GB Q4_K_M build
3. In vLLM serve with --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4, as vLLM's recipe does
4. Thinking is off by default, so pass enable_thinking: true in chat_template_kwargs to turn it on, and expect the reasoning in the thought channel block
5. Drop earlier thoughts from the history except on tool-call turns, as the card asks
6. Use the card's sampling, temperature 1.0, top_p 0.95, top_k 64
7. Validate function names and arguments before running a call, and check JSON output yourself or use the runtime's guided decoding
8. The w4a16 build relies on Marlin INT4 kernels that vLLM marks unsupported on AMD, so use the GGUF on the reference machine

Inkling

1. Unless you have a multi-GPU Blackwell or H200 node, call it through Together AI, Fireworks, DeepInfra or Tinker rather than starting it
2. On 8 B200 GPUs, SGLang's recipe is sglang serve --trust-remote-code --model-path thinkingmachines/Inkling-NVFP4 --tp 8 --quantization modelopt_fp4 --enable-multimodal --reasoning-parser inkling --tool-call-parser inkling
3. Use the inkling tool parser, not Hermes, and the inkling reasoning parser
4. Pass reasoning_effort as none, minimal, low, medium, high (the default) or max, or a number from 0 to 0.99. The repository's template has no xhigh preset and raises an error for it, though Thinking Machines' docs list xhigh
5. Tool call arguments must reach the template as an object, not a JSON string, or it raises an error
6. Audio should be 16 kHz WAV, ideally under 20 minutes, and images 40 to 4096 px a side, per the card
7. Read the Model Acceptable Use Policy before deploying, it binds on download and bars some uses that are legal