# Best open-weight models you download and run yourself for AI agents (slim)

> Gemma 4 31B IT (BB), Gemma 4 26B A4B IT (BB) and Gemma 4 12B IT (BB) lead the 76 ranked open-weight models you download and run yourself. Picks by need, strengths, weaknesses and prices from the Anchor benchmark.

- Full: https://www.anchorterminal.com/best/open-weights/index.md (~5,600 tokens) · this version ~1,280 tokens · JSON https://www.anchorterminal.com/best/open-weights/index.json · canonical https://www.anchorterminal.com/best/open-weights/
- Index: https://www.anchorterminal.com/llms.txt · API: https://www.anchorterminal.com/api/v1/index.json · MCP: https://www.anchorterminal.com/mcp · Updated: 2026-10-21
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)

The 10 highest-scoring of 76 open-weight models you download and run yourself on the Anchor benchmark, with a pick for each need and where each one falls short. Scores come from public evidence, re-checked as vendors change.

- Ranked: 76 · agent-ready (BB or better): 5 · accept x402: 0 · hosted endpoints: 0
- Full ranked table: https://www.anchorterminal.com/categories/open-weights.md
- Head-to-head comparisons: https://www.anchorterminal.com/compare/open-weights/index.md (2870)
- Methodology: https://www.anchorterminal.com/benchmark/index.md

## The shortlist

| # | Tool | Grade | Score | Best for | Price | Where |
| --- | --- | --- | --- | --- | --- | --- |
| 1 | [Gemma 4 31B IT](https://www.anchorterminal.com/tools/gemma-4-31b-it.md) | BB | 74.1 | An agent with a 24 GB GPU or more that wants a dense multimodal cousin with tool calling under a plain licence | Free | Local |
| 2 | [Gemma 4 26B A4B IT](https://www.anchorterminal.com/tools/gemma-4-26b-a4b-it.md) | BB | 72.2 | Agents on a 24 GB or larger GPU or Mac that want a fast, permissively licensed tool-calling model with images and long context | Free | Local |
| 3 | [Gemma 4 12B IT](https://www.anchorterminal.com/tools/gemma-4-12b-it.md) | BB | 71.3 | Agents on a 12 GB or larger GPU or Mac that want a permissively licensed tool-calling model with images, audio and long context | Free | Local |
| 4 | [gpt-oss-120b](https://www.anchorterminal.com/tools/gpt-oss-120b.md) | BB | 70.2 | Agents on one 80 GB GPU or a 128 GB unified machine that want an Apache 2.0 reasoning model with wide runtime support | Free | Local |
| 5 | [Muse Glimmer 30B](https://www.anchorterminal.com/tools/muse-glimmer-30b.md) | BB | 70 | Local agents and coding with image input on one 24 to 32 GB GPU or an Apple silicon laptop, where one tool call per turn is enough | Free | Local |
| 6 | [gpt-oss-20b](https://www.anchorterminal.com/tools/gpt-oss-20b.md) | B | 69.9 | Local agents on a 24 GB GPU or Mac that want a permissive licence, wide runtime support and documented structured output, one tool call at a time | Free | Local |
| 7 | [Nemotron 3 Ultra 550B A55B](https://www.anchorterminal.com/tools/nemotron-3-ultra-550b-a55b.md) | B | 68.8 | Agents with a multi-GPU Blackwell or Hopper node, or using a hosted endpoint, that want NVIDIA's strongest open model for long agent runs | Free | Local |
| 8 | [Qwen3.8-27B](https://www.anchorterminal.com/tools/qwen3-8-27b.md) | B | 68.6 | Agents that want a capable open model with vision on one 24 to 32 GB GPU or Mac, with the widest runtime support and a plain licence. | Free | Local |
| 9 | [Qwen3.6-27B](https://www.anchorterminal.com/tools/qwen3-6-27b.md) | B | 67.2 | Agents that want an open coding and tool-calling model with vision on one 24 to 32 GB GPU or Mac, with plain licence terms. | Free | Local |
| 10 | [Nemotron 3 Nano 30B A3B](https://www.anchorterminal.com/tools/nemotron-3-nano-30b-a3b.md) | B | 67.1 | Agents with a 24 to 32 GB GPU or Mac that want a quick tool-calling and reasoning cousin with very long context. | Free | Local |

## Picks by need

- Highest score overall: [Gemma 4 31B IT](https://www.anchorterminal.com/tools/gemma-4-31b-it.md), BB, 74.1/100 on the benchmark. Also [Gemma 4 26B A4B IT](https://www.anchorterminal.com/tools/gemma-4-26b-a4b-it.md), BB, 72.2/100.

## How to choose

- Licence and acceptable use: Read the licence and any acceptable use policy it brings in, since a field-of-use limit or a user threshold can rule out the job before any test does.
- Memory at your precision: Check the memory the weights need at the precision you will serve, plus room for the context, because a model that does not fit runs from disk or not at all.
- Tool-calling format and parser: Check the tool-calling format and the parser your runtime needs for it, because a mismatched parser returns tool calls as plain text the agent cannot act on.
- File format and code run on load: Prefer safetensors or GGUF files and check whether loading runs code from the repository, since a pickle file or remote code can execute on your machine.

- How the benchmark tests this category: In this run, public evidence against the open-weight reading of the checklist, from the model card, the files, the licence and the runtimes' own code. When our local runs start, each model that fits in 128 GB is served on one machine through llama.cpp and vLLM at the largest precision that fits. We measure tokens per second and time to first token, then run the agent task suite with the model driving.

Each listing's verdict, strengths and weaknesses: https://www.anchorterminal.com/best/open-weights/index.md