Inkling vs Nemotron 3 Nano 30B A3B

Inkling

Thinking Machines' large multimodal generalist, stronger than Inkling-Small on factual recall by its own numbers. It needs about 710 GB of datacentre GPU memory even in NVFP4, so most agents will use a host such as Together AI or Tinker. The main risks are a use policy that binds on download and can change, and no outside safety results.

Nemotron 3 Nano 30B A3B

A fast hybrid Mamba-Transformer cousin with 3.5B active parameters for reasoning and tool calling over very long inputs, in every major local runtime. A 4-bit build needs about 22 to 23 GB. The main risks are that NVIDIA's documented vLLM path runs repository code, no safety results are published and NVIDIA hasn't answered reports since March.

Inkling

1. Unless you have a multi-GPU Blackwell or H200 node, call it through Together AI, Fireworks, DeepInfra or Tinker rather than starting it
2. On 8 B200 GPUs, SGLang's recipe is sglang serve --trust-remote-code --model-path thinkingmachines/Inkling-NVFP4 --tp 8 --quantization modelopt_fp4 --enable-multimodal --reasoning-parser inkling --tool-call-parser inkling
3. Use the inkling tool parser, not Hermes, and the inkling reasoning parser
4. Pass reasoning_effort as none, minimal, low, medium, high (the default) or max, or a number from 0 to 0.99. The repository's template has no xhigh preset and raises an error for it, though Thinking Machines' docs list xhigh
5. Tool call arguments must reach the template as an object, not a JSON string, or it raises an error
6. Audio should be 16 kHz WAV, ideally under 20 minutes, and images 40 to 4096 px a side, per the card
7. Read the Model Acceptable Use Policy before deploying, it binds on download and bars some uses that are legal

Nemotron 3 Nano 30B A3B

1. Start it with ollama run nemotron-3-nano:30b (18 GB MLX on a Mac, 24 GB q4_K_M elsewhere)
2. On the reference machine use BF16 with vLLM, after wget https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/resolve/main/nano_v3_reasoning_parser.py, then vllm serve nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 --max-model-len 262144 --trust-remote-code --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser-plugin nano_v3_reasoning_parser.py --reasoning-parser nano_v3
3. NVFP4 needs an NVIDIA Blackwell GPU, and FP8 isn't documented on gfx1151
4. Without the qwen3_coder tool parser and the nano_v3 reasoning parser, tool calls and reasoning come back as plain text
5. Card sampling is temperature 0.6 and top_p 0.95 for tool calling, temperature 1.0 and top_p 1.0 for reasoning
6. Thinking is on by default, so pass enable_thinking as false in chat_template_kwargs for short tasks
7. Set VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 and raise --max-model-len only when you need more than 262,144 tokens
8. In SGLang the main branch names the reasoning parser nemotron_3, not the card's nano_v3
9. Filter untrusted tool results before they reach the model, as NVIDIA warns prompt injection can lead to code execution
10. Validate JSON output yourself or use the runtime's guided decoding, since none is documented for the model