Gemma 4 E4B IT vs Qwen3-VL-4B-Instruct

Gemma 4 E4B IT

A small cousin for light agent work, tool-calling loops, transcription and image reading on a phone or laptop, under Apache 2.0. Google's 4-bit build needs about 7 GB with working space and runs in all the common runtimes. Google's card puts it far below larger Gemma 4 sizes on agent and coding tasks, with no published safety results.

Qwen3-VL-4B-Instruct

A small vision-language cousin for reading screenshots, documents, charts and video, running in about 3.5 GB from Alibaba's Q4_K_M GGUF in every major runtime under plain Apache 2.0. It has no thinking mode, and Qwen hasn't answered repository questions or published safety work, so add your own guardrails before letting it drive a GUI.

Gemma 4 E4B IT

1. On the reference machine run llama-server -hf google/gemma-4-E4B-it-qat-q4_0-gguf --hf-file gemma-4-E4B_q4_0-it.gguf --jinja -c 131072, and add --mmproj with gemma-4-E4B-it-mmproj.gguf for images and audio
2. With Ollama run ollama run gemma4:e4b and raise num_ctx or OLLAMA_CONTEXT_LENGTH, since the default context is small and agents want at least 64K
3. In vLLM or SGLang pass --tool-call-parser gemma4 --reasoning-parser gemma4 (vLLM also --enable-auto-tool-choice), or tool calls come back as text; Google's w4a16 build is the 4-bit route there
4. Thinking is off by default; pass enable_thinking=True or start the system prompt with <|think|>, and drop earlier turns' thinking from history except on tool-call turns
5. Put images before the text and audio after it, keep audio to 30 seconds and video to 60 frames, and validate tool names and arguments before running them

Qwen3-VL-4B-Instruct

1. Start it with llama-server -hf Qwen/Qwen3-VL-4B-Instruct-GGUF:Q4_K_M --jinja -c 32768, or ollama run qwen3-vl:4b-instruct
2. In Ollama the plain qwen3-vl:4b tag is the Thinking edition, so name 4b-instruct
3. With the GGUF, load the mmproj vision file too, or images won't be read
4. Parse tool calls as JSON inside <tool_call> tags, or use SGLang's --tool-call-parser qwen
5. Use the card's sampling, temperature=0.7, top_p=0.8, top_k=20, presence_penalty=1.5 for vision and temperature=1.0, top_p=1.0, top_k=40, presence_penalty=2.0 for text
6. Validate JSON output yourself, since structured output isn't documented
7. vLLM needs 0.11.0 or later and Transformers 4.57.0 or later