Cosmos-Reason2-2B vs Nemotron 3 Ultra 550B A55B

Cosmos-Reason2-2B

A small local cousin for reasoning over video and images in robotics, driving and warehouse scenes, returning points, boxes and timed events as JSON. The 4.9 GB file runs in Transformers or vLLM, though NVIDIA asks for 24 GB of GPU memory. Development has moved to Cosmos 3, and the gated licence is revocable.

Nemotron 3 Ultra 550B A55B

NVIDIA's frontier-scale agent cousin with trained parallel tool calls, up to a 1M-token context and training against prompt injection through tool output. It needs at least four Blackwell GPUs for the 352 GB NVFP4 build, so most agents will use NVIDIA's API or a provider. The main risks are serving commands that run repository code and no published safety results.

Cosmos-Reason2-2B

1. Accept the licence on the Hugging Face card with a signed-in account and set HF_TOKEN before downloading
2. Serve it with vllm serve nvidia/Cosmos-Reason2-2B --max-model-len 16384 --media-io-kwargs '{"video": {"num_frames": -1}}' --reasoning-parser qwen3, keeping max-model-len between 8,192 and 16,384 as NVIDIA advises
3. For local video or images, make a folder for them (mkdir -p media) and pass --allowed-local-media-path "$(pwd)/media", so the server can read only that folder, not everything under the directory you start it from
4. Sample video at 4 fps and allow at least 4,096 output tokens so the reasoning isn't cut off
5. For reasoning, add the card's instruction to answer with a think block and then the answer; leave it out for a direct reply
6. Use NVIDIA's prompt files (prompts/*.yaml in nvidia-cosmos/cosmos-reason2) when you want JSON boxes, events or trajectories, and validate the JSON yourself
7. Check its plans before a robot acts on them, since NVIDIA's own safety card warns of collision and manipulation risks

Nemotron 3 Ultra 550B A55B

1. It doesn't fit the 128 GB reference machine at any precision. Call NVIDIA's API (model nvidia/nemotron-3-ultra-550b-a55b) or a provider instead
2. With vLLM serve the NVFP4 build with --trust-remote-code --reasoning-parser nemotron_v3 --enable-auto-tool-choice --tool-call-parser qwen3_coder on 4 × B200 or more
3. With SGLang use --reasoning-parser nemotron_3, not the BF16 card's nemotron_v3, which SGLang doesn't register
4. On SGLang with tools, pass chat_template_kwargs with enable_thinking true and force_nonempty_content true, as the card says, or reasoning and calls won't both parse
5. Thinking is on by default. Pass enable_thinking: false for short tasks or medium_effort: true to cut reasoning tokens
6. Use temperature=1.0 and top_p=0.95, as the generation config sets