DeepSeek-V4-Flash-0731 vs Gemma 4 31B IT

DeepSeek-V4-Flash-0731

A long-context agent cousin under MIT with documented tool calls, three reasoning effort levels and support in every major runtime but Ollama. It needs about 190 to 200 GB of memory, though six inference providers serve it. The main risks are prompts built from DeepSeek's Python, no published safety results and no answers in its repository.

Gemma 4 31B IT

A dense multimodal cousin with documented tool calling, optional thinking and a 256K context, for general agent work and image reading on one 24 GB GPU through Google's 4-bit build. It runs in every major runtime under plain Apache 2.0. The main risk is that Google publishes no safety results and has gone quiet on the repository since July.

DeepSeek-V4-Flash-0731

1. vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 --trust-remote-code --kv-cache-dtype fp8 --block-size 256 --data-parallel-size 4 --enable-expert-parallel --moe-backend deep_gemm_mega_moe --attention-config '{"use_fp4_indexer_cache": true}' --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"greedy"}' --tokenizer-mode deepseek_v4 --tool-call-parser deepseek_v4 --enable-auto-tool-choice --reasoning-parser deepseek_v4 (the card's 4x GB300 example plus the parser flags from vLLM's recipe)
2. Build prompts with encoding_dsv4.py, deepseek-recipe's V4 encoder or vLLM's --tokenizer-mode deepseek_v4, since the repository has no Jinja template
3. Tool calls come back in a DSML tool_calls block with invoke elements, string parameters carry string="true" and JSON values string="false"
4. Set temperature 1.0 and top_p 0.95 for agent work, and allow up to 384K output tokens at reasoning effort high or max
5. Without the hardware, call it through a Hugging Face inference provider (Together, DeepInfra, Novita, Baseten, Scaleway, Featherless), not DeepSeek's API, where deepseek-v4-flash now means V4.1 Flash
6. LM Studio's library build is a 156.4 GB MXFP4 GGUF and LM Studio states 156 GB minimum system memory

Gemma 4 31B IT

1. On the reference machine run llama-server -hf google/gemma-4-31B-it-qat-q4_0-gguf --hf-file gemma-4-31B_q4_0-it.gguf --jinja -c 32768, adding gemma-4-31B-it-mmproj.gguf for images
2. Or ollama run gemma4:31b for the 20 GB Q4_K_M build
3. In vLLM serve with --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4, as vLLM's recipe does
4. Thinking is off by default, so pass enable_thinking: true in chat_template_kwargs to turn it on, and expect the reasoning in the thought channel block
5. Drop earlier thoughts from the history except on tool-call turns, as the card asks
6. Use the card's sampling, temperature 1.0, top_p 0.95, top_k 64
7. Validate function names and arguments before running a call, and check JSON output yourself or use the runtime's guided decoding
8. The w4a16 build relies on Marlin INT4 kernels that vLLM marks unsupported on AMD, so use the GGUF on the reference machine