Gemma 4 E4B IT vs gpt-oss-120b

Gemma 4 E4B IT

A small cousin for light agent work, tool-calling loops, transcription and image reading on a phone or laptop, under Apache 2.0. Google's 4-bit build needs about 7 GB with working space and runs in all the common runtimes. Google's card puts it far below larger Gemma 4 sizes on agent and coding tasks, with no published safety results.

gpt-oss-120b

A reasoning and agent cousin that runs in most runtimes under Apache 2.0 with published safety work. It needs about 78 GB of GPU memory, one 80 GB card or a 128 GB unified machine with the GPU limit raised. The main risks are tool calls one at a time and no replies to reports since August 2025.

Gemma 4 E4B IT

1. On the reference machine run llama-server -hf google/gemma-4-E4B-it-qat-q4_0-gguf --hf-file gemma-4-E4B_q4_0-it.gguf --jinja -c 131072, and add --mmproj with gemma-4-E4B-it-mmproj.gguf for images and audio
2. With Ollama run ollama run gemma4:e4b and raise num_ctx or OLLAMA_CONTEXT_LENGTH, since the default context is small and agents want at least 64K
3. In vLLM or SGLang pass --tool-call-parser gemma4 --reasoning-parser gemma4 (vLLM also --enable-auto-tool-choice), or tool calls come back as text; Google's w4a16 build is the 4-bit route there
4. Thinking is off by default; pass enable_thinking=True or start the system prompt with <|think|>, and drop earlier turns' thinking from history except on tool-call turns
5. Put images before the text and audio after it, keep audio to 30 seconds and video to 60 frames, and validate tool names and arguments before running them

gpt-oss-120b

1. Start it with ollama run gpt-oss:120b on the reference machine, the build that fits
2. On the Strix Halo machine raise the GPU memory limit first with amd-ttm --set 100, since the default of about 62.8 GiB is below the 78 GB it needs
3. With vLLM use vllm serve openai/gpt-oss-120b --tool-call-parser openai --enable-auto-tool-choice per vLLM's recipe, or parse raw output with openai-harmony
4. Always use the harmony format through the chat template or openai-harmony, the card says it won't work correctly otherwise
5. Set Reasoning: low in the system message for short tasks, medium is the default
6. Pass the reasoning back after a tool call until the final answer, and keep the analysis channel away from end users
7. Expect one tool call per turn and loop, don't wait for parallel calls