NVIDIA Nemotron 3.5 Bolt E4B 4GB vs Qwen3-30B-A3B-Instruct-2507

NVIDIA Nemotron 3.5 Bolt E4B 4GB

A small cousin for voice and device assistants, game characters and robot interfaces, trained for multi-turn tool calls, instruction hierarchy and short spoken replies, with thinking switchable per request. NVIDIA's Q4_K_M GGUF needs about 8 GB, and Ollama and LM Studio carry it. Context is 64K, half the Gemma 4 base, and NVIDIA publishes no safety results.

Qwen3-30B-A3B-Instruct-2507

A quick general-purpose cousin, with 3.3B of 30.5B parameters active per token, for instruction following, tool calling and long documents without a thinking step. It needs about 21 to 23 GB at 4 bits, and several providers host it. The main risk is neglect, with no file changes in over a year, no answers from Alibaba and no safety results.

NVIDIA Nemotron 3.5 Bolt E4B 4GB

1. Quickest start is ollama run nemotron-3.5-bolt:e4b, about 8 GB of files, and set num_ctx to 65536 or less, since NVIDIA supports 64K
2. With NVIDIA's GGUF use llama-server -m Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf -c 65536 -ngl 999 --jinja, and add --reasoning on --reasoning-budget <tokens> to cap thinking
3. With vLLM 0.31.0 or later pass --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4 and the repository's chat_template.jinja
4. Pass enable_thinking explicitly in chat_template_kwargs, since the template defaults to off whatever the card says, and set skip_special_tokens false as the card's examples do
5. Use temperature=1.0 and top_p=0.95 in both modes, as the card says
6. Set conversational_mode true for spoken replies, which appends a brevity instruction to the system prompt
7. Answer a slow tool at once with a pending tool message, then send the result later as a user message starting TOOL RESULT, as the card's asynchronous pattern says
8. Keep persistent rules in the system prompt and send state updates as separate user or system messages in order, per the card's prompting guidelines

Qwen3-30B-A3B-Instruct-2507

1. Quickest start is ollama run qwen3:30b-instruct, 19 GB. Plain qwen3:30b and qwen3:30b-a3b are the Thinking-2507 edition, not this one
2. On our reference machine use BF16 with vllm serve Qwen/Qwen3-30B-A3B-Instruct-2507 --max-model-len 262144 --enable-auto-tool-choice --tool-call-parser hermes, 61 GB of weights fits the GPU share
3. Use the hermes parser in vLLM or qwen25 in SGLang, the template emits JSON inside tool_call tags
4. Don't pass enable_thinking, this edition never thinks and the card says the flag isn't needed
5. Sampling per the card is temperature 0.7, top_p 0.8, top_k 20 and min_p 0, with presence_penalty up to 2 against repetition
6. Leave config.json as it is unless you need more than 262,144 tokens, the 1M mode swaps in config_1m.json and runs one sequence at a time
7. Validate JSON yourself or use the runtime's guided decoding, since structured output isn't documented