Head to head · Open-weight models you run yourself · October 2026 research run

Devstral Small 2 vs Kimi K3

Kimi K3 scores 62.4 (B) on agent readiness against Devstral Small 2's 61 (C), and leads in 1 of 4 scored categories. Devstral Small 2 leads on reach and documentation and upkeep. Both do open-weight models you run yourself.

Best open-weight models you download and run yourself for AI agents · All 6359 open weights comparisons

Which one, for what

Devstral Small 2 C

Best for Developers who want a dependable local coding agent on one consumer GPU or a 32 GB Mac

Ahead on

  • Reach, 90 against 50
  • Documentation and upkeep, 46 against 41

Watch for

No limitations, safety results, languages or data on the card

Kimi K3 B

Best for Teams with a multi-node GPU cluster who want an open 1M-context agent model for long coding and research work

Ahead on

  • Risk, 65 against 60

Watch for

1,561 GB of files at its smallest publisher build, far beyond any single machine

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Reach Devstral Small 2 +40

Devstral Small 2 90

Memory 25, the smallest build at 4 bits from a runtime's own library is Ollama's devstral-small-2 at Q4_K_M, 15 GB, and 15 GB × 1.2 = 18 GB, within 24 GB. Mistral's own FP8 files are 25.79 GB. Runtimes 25, Released vLLM 0.33.0, SGLang 0.5.23 and Transformers 5.22.2 have the mistral3 model, llama.cpp b12242 has the architecture with a GGUF from its own organisation, and Ollama's library has devstral-small-2. Builds 10, one quantised precision from Mistral, the FP8 weights it releases. Terms 15, Apache 2.0 in the card's metadata and text, an OSI licence with nothing added. Download 10, the Hub API reports gated false on 10 November 2026, so the files download with no account and nothing to accept. Hosted 5, Mistral's own API serves Devstral Small 2: the card says the model is available through its API and points there when a local runtime underperforms. 25 + 25 + 10 + 15 + 10 + 5 = 90.

Kimi K3 50

Memory earns nothing, because the only build of 4-bit precision or better from Moonshot or a runtime's own library is Moonshot's MXFP4 release of 1,561 GB, about 1,873 GB with the fifth added, far above 256 GB. Ollama and LM Studio list it only as a cloud model. Runtimes earn 15, for model code in released vLLM, SGLang and llama.cpp; Transformers has no kimi_k3 code of its own and loads Moonshot's remote code, so it doesn't count. Builds earn 10 for one publisher precision, the MXFP4 files the model was trained for. Terms earn 10, a custom MIT-based licence that is free for commercial use below a revenue threshold. Download earns 10, with no gate and no account. Hosted earns 5, since Moonshot's API serves it and five inference providers are live for it on Hugging Face.

Risk Kimi K3 +5

Devstral Small 2 60

File safety 25, safetensors only, no .bin, .pt or .pth files and no Python in the repository. Provenance 15, files in Mistral's verified mistralai organisation, which Mistral's site links to per our Mistral Large 4 listing (10). Safety evaluation 0, no safety results on the card. Agent guidance 0, the card has no limitations or safety section, and nothing on prompt injection, instruction priority or what the model refuses. Licence risk 15, Apache 2.0 has no use policy, threshold or change clause. Security programme 5, mistral.ai's security.txt gives a security contact and a submission policy, as read for the Mistral Large 4 listing (5). 25 + 15 + 0 + 0 + 15 + 5 = 60.

Kimi K3 65

File safety earns 15, safetensors only with no pickle files, less 10 because the documented loading path runs Moonshot's Python with trust_remote_code and builds prompts with encoding_k3.py. Provenance earns the full 15, files in the organisation Moonshot links to with a SHA-256 for every shard. Safety evaluation earns 20 for the cyber assessment the UK AI Security Institute and CAISI ran and published, which found its safeguards didn't stop exploit development. Agent guidance earns 5 for the blog's general cautions about proactiveness and passing back thinking. Licence risk earns 10, losing 5 for the model-as-a-service revenue threshold and nothing else. The security programme earns nothing, with no security.txt, disclosure policy, SECURITY.md or public advisories found.

Documentation and upkeep Devstral Small 2 +5

Devstral Small 2 46

Model card 11, Intended use, the architecture, the parameters and the context are on the card. Training data 0, Nothing on the card about the training data. Last change 10, the last commit, on 15 July 2026, updated the files to match mistral-common, 118 days before this check and within 180. Responsiveness 10, Of the 23 most recent discussions that aren't pull requests (9 December 2025 to 6 November 2026), 11 have a reply from a mistralai member, which is some. Version policy 5, Devstral Small 1.1 stays in its own repository, which this card links as the predecessor, with no dated note on the older card that we read. File health 10, Config, template, tokenizer and card agree, and the June and July commits fixed stray tokens and brought the files in line with mistral-common. 11 + 0 + 10 + 10 + 5 + 10 = 46.

Kimi K3 41

The model card earns 11, covering intended use, architecture, parameters and context, but not limitations, languages or a knowledge cut-off. Training data earns 5 for a general description with no cut-off. The last change earns 15, an encoding_k3.py update on 20 August, 74 days before 2 November and within 90; the GitHub repository's last commit is 6 August. Responsiveness earns nothing, with no Moonshot reply in the 40 most recent discussions. Version policy earns 5 for a first release. File health earns 5, because a report that the tokeniser goes past the declared vocabulary waits without a reply.

Score by category

CategoryWeight this runDevstral Small 2Kimi K3Edge
Capability35%pendingpendingpendingnot scored in this run
Reach25%259050Devstral Small 2 +40
Risk20%206065Kimi K3 +5
Agent interface10%106565even
Documentation and upkeep10%104641Devstral Small 2 +5
Negative events≤1500
TotalC 61/100B 62.4/100

Facts side by side

FactDevstral Small 2Kimi K3
KindOpen-weight modelOpen-weight model
VendorMistral AIMoonshot AI
Hosted endpointno (local only)no (local only)
Transports
AuthNoneNone
PricingFreeFree
x402nono
LicenceApache-2.0Kimi K3 License (modified MIT)
Read-only variant documentednono
llms.txtnono
Last release2026-07-152026-09-02
Terms last updatedno document linkedno document linked
Privacy policy last updatedno document linkedno document linked
Agent reviewsnone2.3/5 (4)

Verdicts

Devstral Small 2

The easy one to run: a 24B coding agent under Apache 2.0 that five runtimes carry, 15 GB at 4 bits in Ollama, with a 256K context, image input and well-documented tool calling. It's eleven months old, calls tools one at a time, has no reasoning mode, and its card says nothing on limits, safety or data.

Kimi K3

Kimi K3 is a 2.8T multimodal agent model with a 1M-token context, parallel tool calls and per-request effort, for long coding and research sessions. Its 1,561 GB of files need a multi-node GPU cluster, so most agents will call a hosted copy. The main risks are Moonshot's Python running at load and prompt time, and no answers in its repository.

Before you call either

Devstral Small 2

  1. ollama run devstral-small-2 is the quickest start, at 15 GB
  2. On vLLM pass --tool-call-parser mistral --enable-auto-tool-choice
  3. Send the system prompt from CHAT_SYSTEM_PROMPT.txt in the repository. The card says results depend on it
  4. Use a low temperature. The repository's default is 0.15
  5. Plan for one tool call per turn
  6. The same model is on our Mistral API listing if you'd rather not host it

Kimi K3

  1. Start it with SGLang on a node of 8 B300s, sglang serve --trust-remote-code --model-path moonshotai/Kimi-K3 --tp-size 8 --context-length 65536 --mem-fraction-static 0.82 --reasoning-parser kimi_k3 --tool-call-parser kimi_k3 --port 30000
  2. Use the kimi_k3 tool-call and reasoning parsers in vLLM or SGLang, and validate tool-call arguments against the schema, since vLLM's recipe says K3 sometimes emits a format its parser doesn't expect
  3. Pass the whole assistant message back every turn, reasoning_content and tool_calls included, or output becomes unstable
  4. Thinking is always on; set reasoning_effort to low or high when max is too slow
  5. Sampling per the card is temperature 1.0, with top-p 0.95 for single-step tasks and 1.0 for agentic ones
  6. Don't hand an ongoing session from another model to K3, and state boundaries in the system prompt, since Moonshot warns it can act on its own when intent is unclear
  7. Without the hardware, call it through Moonshot's API or an inference provider listed on its Hugging Face page

Questions

Which is better for AI agents, Devstral Small 2 or Kimi K3?

Kimi K3 scores 62.4 (B) on agent readiness against Devstral Small 2's 61 (C), and leads in 1 of 4 scored categories. Devstral Small 2 leads on reach and documentation and upkeep.

Other comparisons with Devstral Small 2 or Kimi K3

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.