Ling-3.0-tiny vs Nemotron 3 Super 120B A12B

Ling-3.0-tiny

A small, quick hybrid reasoning cousin for local agent work, tool calls and coding, under MIT. It runs in about 6 GB from inclusionAI's 4-bit GGUF in released Transformers, vLLM, SGLang and llama.cpp, with no host. The main risks are a card that doesn't match the files in places, the vLLM route running repository code, and no model safety results.

Nemotron 3 Super 120B A12B

A long-context agent cousin with trained parallel tool calls, a reasoning switch and a full account of its training data, under a free commercial licence. It needs about 100 GB of GPU memory, a 128 GB unified machine or Blackwell hardware for NVIDIA's NVFP4. The main risks are serving commands that run repository code and no published safety results.

Ling-3.0-tiny

1. Start it with llama-server -hf inclusionAI/Ling-3.0-tiny-GGUF:Q4_K_M --jinja -c 32768, which needs llama.cpp b11772 or a recent release with bailingmoe3
2. With vLLM 0.32.0, vllm serve inclusionAI/Ling-3.0-tiny --trust-remote-code --enable-prefix-caching --mamba-cache-mode align --enable-auto-tool-choice --tool-call-parser ling3 --reasoning-parser ling3, which runs the repository's config code
3. With SGLang 0.5.22, its cookbook uses --reasoning-parser deepseek-r1 --tool-call-parser glm45 for this model, and drop the card's NEXTN flags since tiny has no MTP layer
4. Thinking is on by default, pass chat_template_kwargs with enable_thinking false for short tasks
5. Sampling per the card is temperature 1.0, top_p 0.95 and top_k 20
6. The template keeps earlier reasoning in the prompt, which grows long agent sessions
7. Validate JSON output yourself or use the runtime's guided decoding, since structured output isn't documented
8. For a guardrail, inclusionAI's Ling-3.0-tiny-singprobe scores intent, unsafe output and hallucination per token, but only through SGLang and vLLM integration branches

Nemotron 3 Super 120B A12B

1. On the reference machine start it with ollama run nemotron-3-super, the 87 GB q4_K_M build, the only one that fits (about 104 GB with the margin)
2. Raise the Strix Halo GPU memory limit first with amd-ttm --set 100, since the default of about 62.8 GiB is far below what it needs
3. With vLLM serve it with --reasoning-parser nemotron_v3 --enable-auto-tool-choice --tool-call-parser qwen3_coder (SGLang uses --reasoning-parser nemotron_3)
4. Use temperature=1.0 and top_p=0.95 for every task, as the card says
5. Thinking is on by default. Pass enable_thinking: false or low_effort: true in chat_template_kwargs for short tasks
6. For coding agents pass force_nonempty_content: true in chat_template_kwargs, so a cut-off reasoning trace still returns content
7. Raise context past 262,144 only with the card's flags (VLLM_ALLOW_LONG_MAX_MODEL_LEN=1, --max-model-len 1048576)