Best of · Models & inference
Best open-weight models you download and run yourself for AI agents
The 10 highest-scoring of 76 open-weight models you download and run yourself on the Anchor benchmark, with a pick for each need and where each one falls short. Scores come from public evidence, re-checked as vendors change.
- 76 ranked
- 5 agent-ready
- Updated 21 October 2026
Top three
Picks by need
Worked out from the scores, prices and facts, so they change when the research does.
The shortlist
| # | Tool | Grade | Best for | Price | Where |
|---|---|---|---|---|---|
| 1 | Gemma 4 31B IT Google DeepMind |
BB 74.1 | An agent with a 24 GB GPU or more that wants a dense multimodal cousin with tool calling under a plain licence | Free | Local |
| 2 | Gemma 4 26B A4B IT Google DeepMind |
BB 72.2 | Agents on a 24 GB or larger GPU or Mac that want a fast, permissively licensed tool-calling model with images and long context | Free | Local |
| 3 | Gemma 4 12B IT Google DeepMind |
BB 71.3 | Agents on a 12 GB or larger GPU or Mac that want a permissively licensed tool-calling model with images, audio and long context | Free | Local |
| 4 | gpt-oss-120b OpenAI |
BB 70.2 | Agents on one 80 GB GPU or a 128 GB unified machine that want an Apache 2.0 reasoning model with wide runtime support | Free | Local |
| 5 | Muse Glimmer 30B Meta |
BB 70 | Local agents and coding with image input on one 24 to 32 GB GPU or an Apple silicon laptop, where one tool call per turn is enough | Free | Local |
| 6 | gpt-oss-20b OpenAI |
B 69.9 | Local agents on a 24 GB GPU or Mac that want a permissive licence, wide runtime support and documented structured output, one tool call at a time | Free | Local |
| 7 | Nemotron 3 Ultra 550B A55B NVIDIA |
B 68.8 | Agents with a multi-GPU Blackwell or Hopper node, or using a hosted endpoint, that want NVIDIA's strongest open model for long agent runs | Free | Local |
| 8 | Qwen3.8-27B Alibaba (Qwen team) |
B 68.6 | Agents that want a capable open model with vision on one 24 to 32 GB GPU or Mac, with the widest runtime support and a plain licence. | Free | Local |
| 9 | Qwen3.6-27B Alibaba (Qwen team) |
B 67.2 | Agents that want an open coding and tool-calling model with vision on one 24 to 32 GB GPU or Mac, with plain licence terms. | Free | Local |
| 10 | Nemotron 3 Nano 30B A3B NVIDIA |
B 67.1 | Agents with a 24 to 32 GB GPU or Mac that want a quick tool-calling and reasoning cousin with very long context. | Free | Local |
66 more are ranked in the full table.
How to choose
- Licence and acceptable useRead the licence and any acceptable use policy it brings in, since a field-of-use limit or a user threshold can rule out the job before any test does.
- Memory at your precisionCheck the memory the weights need at the precision you will serve, plus room for the context, because a model that does not fit runs from disk or not at all.
- Tool-calling format and parserCheck the tool-calling format and the parser your runtime needs for it, because a mismatched parser returns tool calls as plain text the agent cannot act on.
- File format and code run on loadPrefer safetensors or GGUF files and check whether loading runs code from the repository, since a pickle file or remote code can execute on your machine.
How the benchmark tests this category. In this run, public evidence against the open-weight reading of the checklist, from the model card, the files, the licence and the runtimes' own code. When our local runs start, each model that fits in 128 GB is served on one machine through llama.cpp and vLLM at the largest precision that fits. We measure tokens per second and time to first token, then run the agent task suite with the model driving.
Each one in detail
Gemma 4 31B IT
BB 74.1/100Gemma 4 31B IT is Google DeepMind's open-weight dense multimodal model of 30.7B parameters, taking text and images with a 256K-token context, published on Hugging Face in BF16 with 4-bit QAT builds under Apache 2.0.
Verdict A dense multimodal cousin with documented tool calling, optional thinking and a 256K context, for general agent work and image reading on one 24 GB GPU through Google's 4-bit build. It runs in every major runtime under plain Apache 2.0. The main risk is that Google publishes no safety results and has gone quiet on the repository since July.
Best for An agent with a 24 GB GPU or more that wants a dense multimodal cousin with tool calling under a plain licence
Strengths
- Google's own 17.7 GB QAT Q4_0 GGUF fits a 24 GB GPU, and Ollama's gemma4:31b is 20 GB
- Released support in llama.cpp, Ollama, vLLM, SGLang and Transformers
- Apache 2.0 with nothing added, no gate, and hosted on Google's Gemini API too
Weaknesses
- No safety evaluation results published
- No prompt-injection guidance and no security route tied to Gemma
- Structured output isn't documented by Google
Price FreeAuth Nonex402 NoLocal
Gemma 4 26B A4B IT
BB 72.2/100Gemma 4 26B A4B IT is Google DeepMind's open-weight instruction-tuned mixture-of-experts model, 25.2B parameters with 3.8B active, taking text and images with a 262,144-token context, published on Hugging Face under Apache 2.0.
Verdict A fast mixture-of-experts cousin for general agent work with tools, optional thinking and image input, under Apache 2.0. Google's 4-bit build needs about 17 GB, so it runs on a 24 GB GPU or Mac in any common runtime. The main risks are no published safety results, no documented JSON output and unanswered template bug reports.
Best for Agents on a 24 GB or larger GPU or Mac that want a fast, permissively licensed tool-calling model with images and long context
Strengths
- Plain Apache 2.0, no account or gate, safetensors and GGUF only with a checksum on every file
- Google's own 4-bit QAT GGUF needs about 17 GB with working space
- Released support in vLLM, SGLang, llama.cpp, Ollama, LM Studio and Transformers
Weaknesses
- No safety evaluation figures published
- No documented way to ask for JSON or schema output
- Only one quantised precision from Google, no FP8
Price FreeAuth Nonex402 NoLocal
Gemma 4 12B IT
BB 71.3/100Gemma 4 12B IT is Google DeepMind's open-weight instruction-tuned dense model, 11.95B parameters in an encoder-free design, taking text, images, video frames and audio with a 262,144-token context, published on Hugging Face under Apache 2.0.
Verdict A mid-size cousin for agent work with tools, optional thinking and image and audio input, under Apache 2.0. Google's 4-bit build needs under 9 GB with working space, so it runs on a 12 GB GPU or any recent Mac in all the common runtimes. The main risks are no published safety results and no documented JSON output.
Best for Agents on a 12 GB or larger GPU or Mac that want a permissively licensed tool-calling model with images, audio and long context
Strengths
- Plain Apache 2.0, no account or gate, safetensors and GGUF only with a checksum on every file
- Google's own 4-bit QAT GGUF needs about 8.6 GB with working space
- Released support in vLLM, SGLang, llama.cpp, Ollama, LM Studio and Transformers
Weaknesses
- No safety evaluation figures published
- No documented way to ask for JSON or schema output
- No hosted version, since the Gemini API serves only the two largest sizes
Price FreeAuth Nonex402 NoLocal
gpt-oss-120b
BB 70.2/100gpt-oss-120b is OpenAI's open-weight mixture-of-experts reasoning model, 116.8B parameters with 5.1B active and a 131,072-token context, published on Hugging Face in MXFP4 under Apache 2.0.
Verdict A reasoning and agent cousin that runs in most runtimes under Apache 2.0 with published safety work. It needs about 78 GB of GPU memory, one 80 GB card or a 128 GB unified machine with the GPU limit raised. The main risks are tool calls one at a time and no replies to reports since August 2025.
Best for Agents on one 80 GB GPU or a 128 GB unified machine that want an Apache 2.0 reasoning model with wide runtime support
Strengths
- Runs in Transformers, vLLM, SGLang, llama.cpp and Ollama, with LM Studio linked from the card
- Apache 2.0, no gate, and 11 inference providers serve it for agents without the hardware
- Safetensors only, a trained instruction hierarchy and an outside biosecurity evaluation
Weaknesses
- Needs about 78 GB, too big for 64 GB machines, and OpenAI ships only one precision
- One tool call at a time
- No OpenAI reply in the last 40 discussions, including open template and tokenizer reports
Price FreeAuth Nonex402 NoLocal
Muse Glimmer 30B
BB 70/100Muse Glimmer 30B is Meta's open-weight dense multimodal model for local agent work, 29.6B parameters with text and image input and a 128K-token context, published on Hugging Face under Apache 2.0 with GGUF and ExecuTorch builds.
Verdict A dense 30B multimodal cousin for local agent and coding work with images, documented tool calls and adjustable reasoning. Meta's 17 GB GGUF fits a 24 GB card and runs in llama.cpp, Ollama, vLLM, SGLang and Transformers. The main risks are one tool call per turn, no documented JSON output and a usage policy that adds limits to Apache 2.0.
Best for Local agents and coding with image input on one 24 to 32 GB GPU or an Apple silicon laptop, where one tool call per turn is enough
Strengths
- Meta's own 16.8 GB Q4_K_M GGUF fits a 24 GB card, with a DFlash drafter for speculative decoding
- Released model code in vLLM, SGLang, Transformers, llama.cpp and Ollama, no trust_remote_code
- Tool format and parser names per runtime documented by Meta, with a Jinja template in the repository
Weaknesses
- One tool call per turn and no parallel calls
- No structured output documented, and reasoning can't be turned off
- A usage policy adds prohibited uses beyond the law to Apache 2.0
Price FreeAuth Nonex402 NoLocal
gpt-oss-20b
B 69.9/100gpt-oss-20b is OpenAI's open-weight mixture-of-experts reasoning model, 21B parameters with 3.6B active and a 131,072-token context, published on Hugging Face in MXFP4 under Apache 2.0 and trained on the harmony prompt format.
Verdict gpt-oss-20b is a cousin for reasoning, coding and single tool calls with structured output, under plain Apache 2.0. It needs about 17 GB with a working context, and five inference providers serve it. The main risks are that it trails o4-mini at keeping system instructions above user ones, and OpenAI hasn't answered a report since August 2025.
Best for Local agents on a 24 GB GPU or Mac that want a permissive licence, wide runtime support and documented structured output, one tool call at a time
Strengths
- Model code in released Transformers, vLLM, SGLang and llama.cpp, plus a 14 GB build in Ollama's library
- Fits a 24 GB machine at OpenAI's own MXFP4 precision, the one every evaluation used
- Apache 2.0 with no gate, no account and a usage policy that only asks users to follow the law
Weaknesses
- No reply from OpenAI to any discussion since August 2025, and template reports wait unanswered
- One tool call per message
- Reasoning can be lowered but not switched off
Price FreeAuth Nonex402 NoLocal
Nemotron 3 Ultra 550B A55B
B 68.8/100Nemotron 3 Ultra is NVIDIA's largest open-weight hybrid Mamba-2 and latent mixture-of-experts reasoning model for agents, 550B parameters with 55B active and up to 1M tokens of context, published on Hugging Face in BF16 and NVFP4 under OpenMDW-1.1.
Verdict NVIDIA's frontier-scale agent cousin with trained parallel tool calls, up to a 1M-token context and training against prompt injection through tool output. It needs at least four Blackwell GPUs for the 352 GB NVFP4 build, so most agents will use NVIDIA's API or a provider. The main risks are serving commands that run repository code and no published safety results.
Best for Agents with a multi-GPU Blackwell or Hopper node, or using a hosted endpoint, that want NVIDIA's strongest open model for long agent runs
Strengths
- Trained against prompt injection in tool output, as NVIDIA's technical report documents
- Tool calls documented with a named parser, several per turn, and thinking off or medium effort per request
- Up to 1M tokens of context, with RULER at 1M in the report
Weaknesses
- The smallest build is 352 GB and needs at least four B200-class GPUs
- The card's serving commands pass --trust-remote-code
- No safety evaluation results published
Price FreeAuth Nonex402 NoLocal
Qwen3.8-27B
B 68.6/100Qwen3.8-27B is Alibaba's open-weight dense vision-language model of 27B parameters, with thinking on by default and a 262,144-token context, published on Hugging Face in BF16 and FP8 under Apache 2.0.
Verdict A dense cousin for coding and agent work with image and video input that runs almost anywhere. Ollama's 4-bit build needs about 22 GB, and five providers host it for agents without the hardware. The main risk is that nobody at Alibaba answers, there are no safety results or agent guidance, and the tool format is only in the template.
Best for Agents that want a capable open model with vision on one 24 to 32 GB GPU or Mac, with the widest runtime support and a plain licence.
Strengths
- Released support in llama.cpp, Ollama, vLLM, SGLang and Transformers
- Ollama's 4-bit build fits in about 22 GB
- Plain Apache 2.0 with no gate, no account and no use policy
Weaknesses
- No safety results, prompt-injection guidance or security contact
- Tool calling and structured output aren't documented by Alibaba
- No training data, limitations, languages or knowledge cut-off on the card
Price FreeAuth Nonex402 NoLocal
Qwen3.6-27B
B 67.2/100Qwen3.6-27B is Alibaba's open-weight dense vision-language model of 27B parameters, tuned for agentic coding, with thinking on by default, preserved thinking across turns and a 262,144-token context, published on Hugging Face in BF16 and FP8 under Apache 2.0.
Verdict A capable dense cousin for agentic coding and tool calls with images and video in, running in every major runtime. Ollama's 4-bit build needs about 22 GB, and three providers host it. The main risk is silence, with no safety results, no agent guidance, no answers in its repository and Qwen3.8-27B now ahead of it.
Best for Agents that want an open coding and tool-calling model with vision on one 24 to 32 GB GPU or Mac, with plain licence terms.
Strengths
- Released support in llama.cpp, Ollama, vLLM, SGLang, LM Studio and Transformers
- Ollama's 4-bit build fits in about 22 GB
- Plain Apache 2.0 with no gate, no account and no use policy
Weaknesses
- No safety results, prompt-injection guidance or security contact
- Tool format is only in the template and structured output isn't documented
- No limitations, languages, knowledge cut-off or training data on the card
Price FreeAuth Nonex402 NoLocal
Nemotron 3 Nano 30B A3B
B 67.1/100Nemotron 3 Nano 30B A3B is NVIDIA's open-weight hybrid Mamba-Transformer mixture of experts for reasoning and agent work, 30B parameters with 3.5B active, published on Hugging Face in BF16, FP8 and NVFP4.
Verdict A fast hybrid Mamba-Transformer cousin with 3.5B active parameters for reasoning and tool calling over very long inputs, in every major local runtime. A 4-bit build needs about 22 to 23 GB. The main risks are that NVIDIA's documented vLLM path runs repository code, no safety results are published and NVIDIA hasn't answered reports since March.
Best for Agents with a 24 to 32 GB GPU or Mac that want a quick tool-calling and reasoning cousin with very long context.
Strengths
- Runs in released llama.cpp, Ollama, vLLM, SGLang, LM Studio and Transformers
- NVIDIA's own FP8 and NVFP4 builds beside BF16, and ungated files in NVIDIA's own organisation with a SHA-256 for every shard
- Tool calling documented end to end with a named parser, several calls to a turn, and thinking that can be turned off or budgeted
Weaknesses
- The documented vLLM, SGLang and TensorRT-LLM commands pass trust-remote-code and load a parser file from the repository
- No safety or bias results published, though the subcards say both were run
- No NVIDIA reply on any of the 10 discussions opened since 1 March 2026, including a chat-template report
Price FreeAuth Nonex402 NoLocal
Head to head
- Gemma 4 26B A4B IT vs Gemma 4 31B IT BB 72.2 vs BB 74.1
- Gemma 4 12B IT vs Gemma 4 31B IT BB 71.3 vs BB 74.1
- Gemma 4 31B IT vs gpt-oss-120b BB 74.1 vs BB 70.2
- Gemma 4 31B IT vs Muse Glimmer 30B BB 74.1 vs BB 70
- Gemma 4 12B IT vs Gemma 4 26B A4B IT BB 71.3 vs BB 72.2
- Gemma 4 26B A4B IT vs gpt-oss-120b BB 72.2 vs BB 70.2
- Gemma 4 26B A4B IT vs Muse Glimmer 30B BB 72.2 vs BB 70
- Gemma 4 12B IT vs gpt-oss-120b BB 71.3 vs BB 70.2
- Gemma 4 12B IT vs Muse Glimmer 30B BB 71.3 vs BB 70
- gpt-oss-120b vs Muse Glimmer 30B BB 70.2 vs BB 70
Questions
What are the highest-rated open-weight models you download and run yourself for AI agents?
Gemma 4 31B IT has the highest benchmark score of the 76 ranked open-weight models you download and run yourself, 74.1 (BB). Gemma 4 26B A4B IT is second with 72.2 (BB).
How many open-weight models you download and run yourself are agent-ready?
5 of the 76 ranked here grade BB or better, the bar for agent-ready on the Anchor benchmark.
Which open-weight models you download and run yourself accept x402 payments?
None of the ranked listings here accepts x402 for its main call yet.
How is this list ranked?
By the Anchor benchmark score out of 100, a weighted mean of the scored categories minus deductions for negative events, from public evidence re-checked as vendors change. Listings cannot pay for a place. The latest assessment behind this page is from 21 October 2026.
How this list is made
The order is the Anchor benchmark score, the same number as on each listing and in the top list. Each listing is graded from public evidence against the benchmark checklist, and the picks above are worked out from those grades, prices and facts. No listing pays for its place, and paid audits or listing help never change a score.
Full ranked table · 2870 head-to-head comparisons · Best tools in every category