Benchmark · Coding

SWE-bench Verified

A model given a Python repository and a GitHub issue must change the code until the repository's own tests pass. Epoch AI runs 484 of the 500 human-checked issues in its own simple tool-calling scaffold, with no network, after a major scaffold upgrade in February 2026.

Score
% of issues resolved, higher is better
Run by
Epoch AI
Status
Current, newest result 25 June 2026
Updated
as models are released
Data
the published file, CC BY 4.0, read 20 November 2026
Method
epoch.ai/benchmarks/swe-bench-verified

Results

1 model we list, each at its best setting, best first.

#ModelScoreRangeWho ran itSettingDate
1DeepSeek V4 Pro D77.6%Independenteffort max2026-06-18source

A rank is among the models we list with a result, not a place on the whole leaderboard, and a model offered by several hosts is shown once, on its own listing. These results don't move our grades. Data: Epoch AI, Capabilities and benchmarking (CC BY 4.0), CC BY 4.0.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.