Benchmarks · read 20 November 2026
Benchmarks
What public leaderboards publish about the models we list, from the ones whose terms let us republish it, with who ran every number. 18 benchmarks, 677 results on 94 models across 83 listings: 637 run by someone with nothing to sell in the model, 40 submitted by the model's maker.
Every benchmark
| Benchmark | Area | Unit | Run by | Models | Best | Newest result | Status |
|---|---|---|---|---|---|---|---|
| Berkeley Function Calling Leaderboard (BFCL) | Agent tasks | % overall accuracy | UC Berkeley (Gorilla) | 5 | Qwen3-30B-A3B-Instruct-2507 41.4% | 2025-12-16 | Stale |
| METR time horizons | Agent tasks | minutes at 50% success | METR | 1 | Gemini 3.1 Pro (preview) 384 min | 2026-09-08 | Discontinued |
| Terminal-Bench | Agent tasks | % of trials passed | Stanford and the Laude Institute (the Terminal-Bench maintainers) | 33 | Claude Opus 5.5 64.8% | 2026-11-20 | Current |
| τ²-bench | Agent tasks | % of tasks passed on the first try (pass^1) | Sierra | 6 | Kimi K3 37.1% | 2026-10-29 | Current, 4 submitted |
| Aider Polyglot | Coding | % of exercises solved in two tries | Aider (Paul Gauthier) | 1 | gpt-oss-120b 41.8% | 2025-10-03 | Stale |
| MirrorCode | Coding | % of tasks solved | Epoch AI | 19 | Claude Sonnet 5.5 97.8% | 2026-11-17 | Current, near its ceiling |
| SWE-bench Verified | Coding | % of issues resolved | Epoch AI | 1 | DeepSeek V4 Pro 77.6% | 2026-06-25 | Current |
| FrontierMath (Tiers 1-3) | Reasoning and knowledge | % of problems solved | Epoch AI | 36 | Claude Fable 5.2 95.1% | 2026-11-19 | Current, near its ceiling |
| FrontierMath Tier 4 | Reasoning and knowledge | % of problems solved | Epoch AI | 32 | Claude Fable 5.2 100.0% | 2026-11-19 | Current, near its ceiling |
| GPQA Diamond | Reasoning and knowledge | % of questions correct | Epoch AI | 45 | GPT-6.2 Astra 96.8% | 2026-11-19 | Current, near its ceiling |
| OTIS Mock AIME 2024-2025 | Reasoning and knowledge | % of problems solved | Epoch AI | 49 | Gemini 4 Neon 100.0% | 2026-11-19 | Current, near its ceiling |
| SimpleQA Verified | Reasoning and knowledge | % of questions correct | Epoch AI | 30 | Claude Fable 5.2 94.0% | 2026-11-19 | Current |
| MTEB English v2 | Embeddings | mean task score (0 to 100) | MTEB maintainers (an open community project started at Hugging Face), from results submitted by pull request | 11 | Qwen3-Embedding-8B 75.2 | 2026-11-20 | Current, 16 submitted |
| MTEB English v2 retrieval | Embeddings | mean nDCG@10 (0 to 100) | MTEB maintainers (an open community project started at Hugging Face), from results submitted by pull request | 12 | Qwen3-Embedding-8B 69.4 | 2026-11-20 | Current, 20 submitted |
| LMArena Agent Arena | Arenas: people's votes | IPS score | LMArena | 40 | Claude Fable 5.2 0.274 | 2026-11-18 | Current |
| LMArena Code Arena (WebDev) | Arenas: people's votes | arena rating (Bradley-Terry) | LMArena | 53 | Claude Fable 5.2 1823 | 2026-11-19 | Current |
| LMArena Text Arena | Arenas: people's votes | arena rating (Bradley-Terry) | LMArena | 64 | Gemini 4 Carbon 1527 | 2026-11-19 | Current |
| LMArena Vision Arena | Arenas: people's votes | arena rating (Bradley-Terry) | LMArena | 29 | Claude Fable 5.2 1305 | 2026-11-19 | Current |
Models counts the models we list with a result. Best is the top result among them, on the version of the test most of them took. Stale means nothing new in six months; near its ceiling, that the best result is within five points of a perfect score, so the test no longer tells the top models apart.
Every model against every benchmark
Each model's best result on the version of each test most models took, so a model that only took another version shows nothing here; its benchmark's page has every version. Shaded results were submitted by the model's maker; the rest were run independently.
| Model | Berkeley Function Calling Leaderboard (BFCL) | METR time horizons | Terminal-Bench | τ²-bench | Aider Polyglot | MirrorCode | SWE-bench Verified | FrontierMath (Tiers 1-3) | FrontierMath Tier 4 | GPQA Diamond | OTIS Mock AIME 2024-2025 | SimpleQA Verified | MTEB English v2 | MTEB English v2 retrieval | LMArena Agent Arena | LMArena Code Arena (WebDev) | LMArena Text Arena | LMArena Vision Arena |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.1 Pro (preview) | 384 min | 2.7% | 26.0% | 8.9% | 59.6% | 26.8% | 94.4% | 95.6% | 73.5% | -0.057 | 1446 | 1487 | 1280 | |||||
| Claude Opus 5.5 | 64.8% | 95.6% | 91.2% | 95.0% | 90.6% | 100.0% | 72.2% | 0.186 | 1818 | 1506 | 1293 | |||||||
| Claude Sonnet 5.5 | 61.8% | 97.8% | 88.8% | 80.5% | 95.6% | 100.0% | 46.5% | 0.156 | 1771 | 1473 | 1264 | |||||||
| DeepSeek V4.1 Flash | 40.0% | 64.4% | 71.2% | 46.3% | 93.2% | 99.4% | 42.9% | 0.008 | 1659 | 1475 | 1262 | |||||||
| GPT-6 Luna | 16.4% | 26.7% | 79.0% | 56.1% | 90.5% | 98.9% | 41.4% | 0.006 | 1579 | 1443 | 1234 | |||||||
| GPT-6 Sol | 49.4% | 48.9% | 89.8% | 90.0% | 94.3% | 100.0% | 60.7% | 0.065 | 1674 | 1458 | 1258 | |||||||
| GPT-6.1 Sol | 58.2% | 61.1% | 93.7% | 100.0% | 95.4% | 100.0% | 73.9% | 0.113 | 1744 | 1483 | 1290 | |||||||
| GPT-6.1 Terra | 51.8% | 76.7% | 90.9% | 92.7% | 94.3% | 100.0% | 57.3% | 0.105 | 1684 | 1474 | 1263 | |||||||
| Gemini 3.8 Flash | 19.1% | 38.9% | 68.4% | 21.9% | 95.4% | 98.9% | 69.7% | 0.015 | 1584 | 1497 | 1293 | |||||||
| Kimi K3 | 22.4% | 37.1% | 54.4% | 72.2% | 39.0% | 93.1% | 97.2% | 50.6% | -0.001 | 1658 | 1488 | |||||||
| Claude Fable 5.1 | 57.9% | 93.3% | 90.2% | 87.8% | 100.0% | 70.8% | 0.137 | 1741 | 1503 | 1290 | ||||||||
| Claude Fable 5.2 | 95.1% | 100.0% | 93.9% | 99.7% | 94.0% | 0.274 | 1823 | 1503 | 1305 | |||||||||
| Claude Haiku 5.5 | 40.0% | 78.9% | 75.1% | 46.3% | 89.7% | 98.9% | 23.8% | -0.006 | 1596 | 1421 | ||||||||
| GPT-6 Astra | 60.0% | 93.7% | 97.6% | 95.8% | 100.0% | 75.6% | 0.146 | 1763 | 1475 | 1281 | ||||||||
| GPT-6.2 Astra | 67.8% | 92.6% | 100.0% | 96.8% | 100.0% | 78.5% | 0.152 | 1759 | 1468 | |||||||||
| Gemini 4 Argon | 54.9% | 58.9% | 86.0% | 78.0% | 96.4% | 100.0% | 76.0% | 0.103 | 1681 | 1520 | ||||||||
| Grok 4.7 | 54.4% | 53.0% | 17.1% | 92.7% | 98.1% | 56.0% | 0.015 | 1638 | 1443 | 1250 | ||||||||
| Inkling | 0.0% | 25.0% | 33.3% | 4.9% | 88.3% | 88.9% | 40.3% | -0.185 | 1414 | 1442 | ||||||||
| Inkling-Small | 0.9% | 46.3% | 17.1% | 88.5% | 90.0% | 19.1% | -0.185 | 1408 | 1406 | 1208 | ||||||||
| Kimi K3.1 Pro | 60.9% | 43.3% | 78.2% | 65.8% | 94.2% | 100.0% | 62.2% | 0.039 | 1683 | 1482 | ||||||||
| DeepSeek V4 Pro | 6.7% | 77.6% | 45.3% | 2.4% | 90.9% | 96.7% | 47.0% | 1581 | 1464 | |||||||||
| GLM-5.3 | 41.8% | 68.8% | 29.3% | 90.9% | 91.1% | 41.0% | -0.000 | 1622 | 1482 | |||||||||
| GLM-5.3-Flash | 36.7% | 55.8% | 17.1% | 90.2% | 93.9% | -0.039 | 1610 | 1475 | 1274 | |||||||||
| Gemini 4 Carbon | 90.5% | 95.1% | 96.5% | 100.0% | 77.7% | 0.161 | 1769 | 1527 | ||||||||||
| Gemini 4 Hydrogen | 88.1% | 87.8% | 96.8% | 100.0% | 70.9% | 0.135 | 1701 | 1490 | ||||||||||
| Gemini 4 Neon | 91.2% | 97.6% | 96.3% | 100.0% | 75.2% | 0.184 | 1763 | 1513 | ||||||||||
| Muse Spark 1.3 | 14.6% | 54.4% | 74.4% | 46.3% | 99.2% | 0.011 | 1654 | 1498 | 1288 | |||||||||
| Grok 4.3 | 42.8% | 14.6% | 88.8% | 93.3% | 33.2% | 1357 | 1442 | 1241 | ||||||||||
| Mistral Large 4 | 27.9% | 90.2% | 99.4% | 22.4% | -0.071 | 1570 | 1423 | 1233 | ||||||||||
| Muse Wonder | 47.0% | 45.6% | 82.1% | 70.7% | 99.4% | 0.006 | 1648 | 1511 | ||||||||||
| Gemini 3.5 Flash-Lite | 26.0% | 0.0% | 83.3% | 71.1% | 1432 | 1455 | 1266 | |||||||||||
| MiniMax M3 | 7.3% | 90.9% | 71.1% | -0.109 | 1484 | 1434 | 1231 | |||||||||||
| Qwen3.5-397B-A17B | 9.8% | 31.2% | 86.4% | 88.9% | 1400 | 1442 | 1245 | |||||||||||
| Muse Spark Open | 69.8% | 31.7% | 88.6% | 98.1% | 56.6% | 1527 | ||||||||||||
| DeepSeek-V4-Flash-0731 | 57.5% | 24.4% | 91.0% | 94.4% | 33.6% | |||||||||||||
| Gemma 4 26B A4B IT | 73.2% | 82.2% | 1358 | 1438 | 1241 | |||||||||||||
| Nemotron 3.5 Super VL | 0.3% | -0.173 | 1402 | 1416 | 1211 | |||||||||||||
| GPT-6.2 Sol | 0.113 | 1754 | 1465 | |||||||||||||||
| Mistral Medium 3.5 | -0.145 | 1263 | 1426 | 1199 | ||||||||||||||
| Qwen3-30B-A3B-Instruct-2507 | 41.4% | 55.6% | 62.2% | 1382 | ||||||||||||||
| Qwen3.8-27B | -0.055 | 1589 | 1435 | 1244 | ||||||||||||||
| Step 5 Preview | -0.051 | 1570 | 1449 | 1246 | ||||||||||||||
| gpt-oss-120b | 41.8% | 75.8% | 88.9% | 1352 | ||||||||||||||
| DeepSeek-V3.2 | 1362 | 1425 | ||||||||||||||||
| GLM-4.7-Flash | 60.5% | 58.3% | 1365 | |||||||||||||||
| Gemini 3.1 Flash-Lite | 27.7% | 81.8% | 80.0% | |||||||||||||||
| Gemma 4 31B | 1370 | 1452 | 1265 | |||||||||||||||
| Gemma 4 31B IT | 75.8% | 73.3% | 10.4% | |||||||||||||||
| Hy3 | -0.086 | 1506 | 1454 | |||||||||||||||
| Inkling 2 Small | 0.015 | 1644 | 1412 | |||||||||||||||
| Llama 3.3 70B | 31.9% | 47.4% | 5.1% | |||||||||||||||
| Mistral Large 3 | 1233 | 1415 | 1200 | |||||||||||||||
| Muse Ember | 0.010 | 1609 | ||||||||||||||||
| Nemotron 3 Ultra 550B A55B | 85.3% | 86.7% | 1427 | |||||||||||||||
| Qwen3-4B-Instruct-2507 | 35.7% | 45.8% | 52.2% | |||||||||||||||
| Qwen3.5-122B-A10B | 1360 | 1416 | 1226 | |||||||||||||||
| Qwen3.6-27B | 35.1% | 85.9% | 91.1% | |||||||||||||||
| Qwen3.6-35B-A3B | 20.4% | 84.8% | 86.7% | |||||||||||||||
| Tencent Hy4 | 39.1% | 0.061 | 1650 | |||||||||||||||
| gpt-oss-20b | 60.8% | 65.3% | 1318 | |||||||||||||||
| Amazon Nova 2 Lite | 27.1% | 1335 | ||||||||||||||||
| Cohere Embed and Rerank | 66.2 | 55.7 | ||||||||||||||||
| EmbeddingGemma 2 | 67.5 | 51.3 | ||||||||||||||||
| EmbeddingGemma 300M | 69.7 | 55.7 | ||||||||||||||||
| Gemini Embedding | 73.3 | 64.3 | ||||||||||||||||
| Granite Embedding Small English R2 | 61.1 | 53.9 | ||||||||||||||||
| Jina Embeddings and Reranker | 71.1 | 60.1 | ||||||||||||||||
| MiniMax-M2.5 | 1386 | 1391 | ||||||||||||||||
| MiniMax-M2.7 | 1398 | 1415 | ||||||||||||||||
| MiniMax-M3.1-Flash | -0.037 | 1582 | ||||||||||||||||
| Nomic Embed | 63.6 | 50.7 | ||||||||||||||||
| Qwen3-Embedding-0.6B | 70.5 | 61.8 | ||||||||||||||||
| Qwen3-Embedding-4B | 74.6 | 68.5 | ||||||||||||||||
| Qwen3-Embedding-8B | 75.2 | 69.4 | ||||||||||||||||
| Qwen3.5-9B | 79.0% | 61.7% | ||||||||||||||||
| Qwen3.8-Flash-Next | -0.049 | 1626 | ||||||||||||||||
| jina-embeddings-v5-omni-small | 71.8 | 60.1 | ||||||||||||||||
| Beam 1.0 | 6.4% | |||||||||||||||||
| Command R | 1251 | |||||||||||||||||
| Command R+ 08-2024 | 1276 | |||||||||||||||||
| Command R7B | 32.1% | |||||||||||||||||
| DeepSeek V3.1 | 1417 | |||||||||||||||||
| GLM-4.5-Air | 1373 | |||||||||||||||||
| GPT-Live | ||||||||||||||||||
| Granite 4.2 30B | 1340 | |||||||||||||||||
| Granite 4.2 3B | 1289 | |||||||||||||||||
| Granite 4.2 8B | 1290 | |||||||||||||||||
| Laguna M.1 | 1350 | |||||||||||||||||
| Nemotron 3 Nano 30B A3B | 1314 | |||||||||||||||||
| Nemotron 3 Super 120B A12B | 1360 | |||||||||||||||||
| Nemotron 3.5 Lightning 30B A3B | 1352 | |||||||||||||||||
| OpenAI embeddings | 58.0 | |||||||||||||||||
| Qwen3.5-4B | 55.8% | |||||||||||||||||
| Step 3.5 Flash | 1393 |
Who ran it
- Independent. A party with nothing to sell in the model ran it and published how: Epoch AI, LMArena's votes, the Terminal-Bench maintainers, METR.
- Submitted by the maker. The model's maker ran it and the leaderboard took the number, as MTEB and some of τ²-bench's rows are. Shown, and marked.
- The maker's claim. A number only on the maker's own pages, a launch post or a model card. None is collected yet.
- Measured by Anchor Terminal. Our own runs, such as the lab's evaluation. Not in these files yet.
A rank on these pages is among the models we list with a result, at each model's best setting, not a place on the whole leaderboard. A model offered by several hosts is one set of results, shown on each of its listings. These results sit beside our grades and don't move them: how we grade a listing is a separate method, at /benchmark/.
Not here yet
SWE-bench's own leaderboard, Humanity's Last Exam and ARC-AGI, until their makers allow republishing (their terms don't today); Epoch AI's own SWE-bench Verified runs stand in for the first. Artificial Analysis, the speech and text-to-speech arenas, the benchmarks for coding agents, browsers, search and memory, and the numbers vendors publish about themselves come next.
Credits
- Berkeley Function Calling Leaderboard (BFCL): Berkeley Function Calling Leaderboard, UC Berkeley (Gorilla), Apache 2.0, Apache-2.0
- METR time horizons: METR, task-completion time horizons, CC BY 4.0, CC-BY-4.0
- Terminal-Bench: Terminal-Bench, Stanford and the Laude Institute, Apache 2.0, Apache-2.0
- τ²-bench: τ²-bench, Sierra Research, MIT, MIT
- Aider Polyglot: Aider polyglot leaderboard, Paul Gauthier and contributors, Apache 2.0, Apache-2.0
- MirrorCode: Epoch AI, Capabilities and benchmarking (CC BY 4.0), CC BY 4.0
- SWE-bench Verified: Epoch AI, Capabilities and benchmarking (CC BY 4.0), CC BY 4.0
- FrontierMath (Tiers 1-3): Epoch AI, Capabilities and benchmarking (CC BY 4.0), CC BY 4.0
- FrontierMath Tier 4: Epoch AI, Capabilities and benchmarking (CC BY 4.0), CC BY 4.0
- GPQA Diamond: Epoch AI, Capabilities and benchmarking (CC BY 4.0), CC BY 4.0
- OTIS Mock AIME 2024-2025: Epoch AI, Capabilities and benchmarking (CC BY 4.0), CC BY 4.0
- SimpleQA Verified: Epoch AI, Capabilities and benchmarking (CC BY 4.0), CC BY 4.0
- MTEB English v2: MTEB, embeddings-benchmark/results (CC0), CC0-1.0
- MTEB English v2 retrieval: MTEB, embeddings-benchmark/results (CC0), CC0-1.0
- LMArena Agent Arena: LMArena leaderboard dataset (lmarena-ai/leaderboard-dataset), CC BY 4.0, CC-BY-4.0
- LMArena Code Arena (WebDev): LMArena leaderboard dataset (lmarena-ai/leaderboard-dataset), CC BY 4.0, CC-BY-4.0
- LMArena Text Arena: LMArena leaderboard dataset (lmarena-ai/leaderboard-dataset), CC BY 4.0, CC-BY-4.0
- LMArena Vision Arena: LMArena leaderboard dataset (lmarena-ai/leaderboard-dataset), CC BY 4.0, CC-BY-4.0
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.