Kimi K3 vs Nemotron 3.5 Lightning 30B A3B

Kimi K3

Kimi K3 is a 2.8T multimodal agent model with a 1M-token context, parallel tool calls and per-request effort, for long coding and research sessions. Its 1,561 GB of files need a multi-node GPU cluster, so most agents will call a hosted copy. The main risks are Moonshot's Python running at load and prompt time, and no answers in its repository.

Nemotron 3.5 Lightning 30B A3B

A small agent cousin, 3B active parameters, with trained parallel tool calls, a thinking switch and up to 1M tokens of context, under a permissive licence. It runs in about 23 GB from the GGUF or Ollama builds, and NVIDIA's NVFP4 build needs NVIDIA GPUs. The main risks are no published safety results and one answered question across two repositories.

Kimi K3

1. Start it with SGLang on a node of 8 B300s, sglang serve --trust-remote-code --model-path moonshotai/Kimi-K3 --tp-size 8 --context-length 65536 --mem-fraction-static 0.82 --reasoning-parser kimi_k3 --tool-call-parser kimi_k3 --port 30000
2. Use the kimi_k3 tool-call and reasoning parsers in vLLM or SGLang, and validate tool-call arguments against the schema, since vLLM's recipe says K3 sometimes emits a format its parser doesn't expect
3. Pass the whole assistant message back every turn, reasoning_content and tool_calls included, or output becomes unstable
4. Thinking is always on; set reasoning_effort to low or high when max is too slow
5. Sampling per the card is temperature 1.0, with top-p 0.95 for single-step tasks and 1.0 for agentic ones
6. Don't hand an ongoing session from another model to K3, and state boundaries in the system prompt, since Moonshot warns it can act on its own when intent is unclear
7. Without the hardware, call it through Moonshot's API or an inference provider listed on its Hugging Face page

Nemotron 3.5 Lightning 30B A3B

1. Quickest start is ollama run nemotron-3.5-lightning, about 19 to 25 GB of files depending on the platform
2. On the reference machine use llama-server -hf ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF:Q4_0 --jinja, since the card's Q4_K_M tag doesn't exist there and NVFP4 needs NVIDIA GPUs
3. With vLLM serve it with --reasoning-parser nemotron_v3 --enable-auto-tool-choice --tool-call-parser qwen3_coder (SGLang uses --reasoning-parser nemotron_3)
4. Use temperature=1.0 and top_p=0.95, as the card says
5. Thinking is on by default. Pass chat_template_kwargs with enable_thinking false for short tasks
6. For coding agents pass force_nonempty_content: true in chat_template_kwargs, which vLLM's nemotron_v3 parser reads so a cut-off reasoning trace still returns content
7. Go past 262,144 tokens only with the card's flags (VLLM_ALLOW_LONG_MAX_MODEL_LEN=1, --max-model-len 1048576) and enough memory