Inkling vs Ling-3.0-tiny

Inkling

Thinking Machines' large multimodal generalist, stronger than Inkling-Small on factual recall by its own numbers. It needs about 710 GB of datacentre GPU memory even in NVFP4, so most agents will use a host such as Together AI or Tinker. The main risks are a use policy that binds on download and can change, and no outside safety results.

Ling-3.0-tiny

A small, quick hybrid reasoning cousin for local agent work, tool calls and coding, under MIT. It runs in about 6 GB from inclusionAI's 4-bit GGUF in released Transformers, vLLM, SGLang and llama.cpp, with no host. The main risks are a card that doesn't match the files in places, the vLLM route running repository code, and no model safety results.

Inkling

1. Unless you have a multi-GPU Blackwell or H200 node, call it through Together AI, Fireworks, DeepInfra or Tinker rather than starting it
2. On 8 B200 GPUs, SGLang's recipe is sglang serve --trust-remote-code --model-path thinkingmachines/Inkling-NVFP4 --tp 8 --quantization modelopt_fp4 --enable-multimodal --reasoning-parser inkling --tool-call-parser inkling
3. Use the inkling tool parser, not Hermes, and the inkling reasoning parser
4. Pass reasoning_effort as none, minimal, low, medium, high (the default) or max, or a number from 0 to 0.99. The repository's template has no xhigh preset and raises an error for it, though Thinking Machines' docs list xhigh
5. Tool call arguments must reach the template as an object, not a JSON string, or it raises an error
6. Audio should be 16 kHz WAV, ideally under 20 minutes, and images 40 to 4096 px a side, per the card
7. Read the Model Acceptable Use Policy before deploying, it binds on download and bars some uses that are legal

Ling-3.0-tiny

1. Start it with llama-server -hf inclusionAI/Ling-3.0-tiny-GGUF:Q4_K_M --jinja -c 32768, which needs llama.cpp b11772 or a recent release with bailingmoe3
2. With vLLM 0.32.0, vllm serve inclusionAI/Ling-3.0-tiny --trust-remote-code --enable-prefix-caching --mamba-cache-mode align --enable-auto-tool-choice --tool-call-parser ling3 --reasoning-parser ling3, which runs the repository's config code
3. With SGLang 0.5.22, its cookbook uses --reasoning-parser deepseek-r1 --tool-call-parser glm45 for this model, and drop the card's NEXTN flags since tiny has no MTP layer
4. Thinking is on by default, pass chat_template_kwargs with enable_thinking false for short tasks
5. Sampling per the card is temperature 1.0, top_p 0.95 and top_k 20
6. The template keeps earlier reasoning in the prompt, which grows long agent sessions
7. Validate JSON output yourself or use the runtime's guided decoding, since structured output isn't documented
8. For a guardrail, inclusionAI's Ling-3.0-tiny-singprobe scores intent, unsafe output and hallucination per token, but only through SGLang and vLLM integration branches