{
  "data": {
    "author": "Vlad Cealicu",
    "faq": [
      {
        "a": "Cognition's coding benchmark, released on 27 October 2026. It has 132 tasks written by maintainers of open-source repositories and grades whether a model's pull request is good enough to merge, using rubrics where any failed blocking check scores the attempt zero.",
        "q": "What is FrontierCode 2.0?"
      },
      {
        "a": "Claude Opus 5.5, at 45.5%, with GPT-6 Astra second at 42.8%. Every other model on the leaderboard is below 29%.",
        "q": "Which model scored best?"
      },
      {
        "a": "The prompts leave out engineering decisions on purpose, so the model has to work out what the maintainer wants from the codebase, such as fixing the same bug in a sibling code path or keeping the public API intact.",
        "q": "What does underspecified mean here?"
      },
      {
        "a": "Not on FrontierCode 2.0, by my estimate. Cognition publishes no human score. I'd put a principal engineer new to the repository at around 55%, a staff engineer around 50% and a good senior engineer around 45%, level with the best model. A principal's edge is taste from experience more than coding skill, and it shows most in their own codebase, where they're the maintainer at close to 100%.",
        "q": "Are AI models better than a principal engineer at coding?"
      },
      {
        "a": "At higher effort they write more code and touch more files, so they fail more scope checks, such as deleting a file that's part of a public API, even though they get more thorough.",
        "q": "Why do some models score lower at higher reasoning effort?"
      }
    ],
    "kicker": "Blog · Opinion · 28 October 2026",
    "lede": "Cognition's FrontierCode 2.0 grades whether a model's pull request should be merged, on tasks the maintainers left deliberately vague. The best model, Claude Opus 5.5, passes 45.5%. I think that puts it level with a good senior engineer in a codebase they've never seen, and not far behind a principal, whose edge is taste more than coding.",
    "readingMinutes": 8,
    "words": 1729
  },
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/blog/frontiercode-principal-engineer",
    "json": "https://www.anchorterminal.com/blog/frontiercode-principal-engineer.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/blog/frontiercode-principal-engineer.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/blog/frontiercode-principal-engineer.min.md"
  },
  "markdown": "By Vlad Cealicu, 28 October 2026. 1,729 words, about 8 min to read.\n\nCognition, the company behind the Devin coding agent, released FrontierCode 2.0 on Tuesday 27 October 2026 [[1]](#ref-1). As Cognition puts it, most coding evals check that a model's code meets a spec, their first version asked whether you'd merge the pull request, and this one asks \"not only would you merge this PR, but should you?\" [[6]](#ref-6). It does that with bigger tasks and prompts that leave the engineering decisions to the model [[1]](#ref-1). The best model, Claude Opus 5.5, scores 45.5% [[1]](#ref-1).\n\nI think this is a good thing to test. Staff and principal engineers are the top of our field when it comes to writing code (I mean the engineering track here, not management), and what sets them apart is judgement, which is the thing FrontierCode is trying to grade. So the question I wanted answered is whether the models are better than them yet.\n\n- 45.5%: Claude Opus 5.5's score, the best on the leaderboard\n- 132: tasks, each written by a maintainer of the repository it comes from\n- 40+: hours of a developer's time to build each task\n- 501: blocking checks for thoroughness, the biggest category, up from 168\n\nFrontierCode 2.0 as published on 27 October 2026. Source [1].\n\n## What changed from the first version?\n\nFrontierCode 1.0 came out on 8 June 2026, built with the maintainers of 36 open-source repositories, with each task taking more than 40 hours of a maintainer's time [[3]](#ref-3). The prompts were short on purpose, about a third the length of SWE-Bench Pro's, and Cognition said the benchmark \"expects the agent to infer the maintainer's intent\" [[3]](#ref-3).\n\nVersion 2.0 keeps the method and makes the tasks harder. The tasks come from 55% more repositories and now include security patches and performance work, there are 43% more blocking checks, and the reference solutions change 67% more files and 89% more lines in codebases 25% larger [[1]](#ref-1). In 80% of tasks the reference solution differs from the change that was merged upstream, up from 60% in 1.0 [[1]](#ref-1), so a model that remembers the original pull request doesn't get much from it (and a run that looks up the original is scored zero anyway [[2]](#ref-2)).\n\n## How is it graded?\n\nEach task has a rubric, and any check marked as blocking that fails scores the whole attempt zero [[1]](#ref-1). The checks are a mix of unit tests, shell commands, tests the model writes that have to fail on the old code, limits on which files and how much of the diff can change, and a model that reviews the diff for code quality against a written prompt [[3]](#ref-3). People write the checks and a Cognition researcher reviews every task, and Cognition says \"Rubric grading is subjective, so we built an extensive QC pipeline\" [[3]](#ref-3).\n\nThe tasks aren't public, to keep them out of training data [[3]](#ref-3). That's fair, and it also means nobody outside can check the grading, which matters for a company that sells a coding agent and has its own model, SWE-2, on the leaderboard at 23.9% [[1]](#ref-1). A mid-table score for their own model makes me trust the grading a bit more.\n\nScore on FrontierCode 1.1 and 2.0, rounded, in per cent\n\n| Model | 1.1 | 2.0 |\n| --- | --- | --- |\n| Claude Opus 5.5 | 55 | 46 |\n| GPT-6 Astra | 53 | 43 |\n| Grok 4.7 | 48 | 29 |\n| Gemini 4 Argon | 48 | 26 |\n| Muse Wonder 1.0 | 47 | 24 |\n| SWE-2 | 50 | 24 |\n| DeepSeek V4.1 Flash | 44 | 19 |\n| Kimi K3 | 44 | 18 |\n\nEach series has a zero-based scale.\n\nEvery model dropped, and the drop is uneven. Opus 5.5 and GPT-6 Astra lost about 10 points each, everyone else between 19 and 27, so the new version separates the top two from the rest far more than 1.1 did [[1]](#ref-1).\n\n## What do we expect from a principal engineer?\n\nGitLab publishes its career framework, and for a principal engineer it asks for someone \"Able to deliver results despite starting from unclear requirements\", who \"Makes responsible decisions and evaluates tradeoffs\" and \"Performs Code Reviews across the GitLab Codebase on a regular cadence\" [[4]](#ref-4). Google's public code review guide says a reviewer should approve a change \"once it is in a state where it definitely improves the overall code health of the system being worked on, even if the CL isn't perfect\" [[5]](#ref-5).\n\nPut those next to the code itself and I'd look for three things in a staff or principal engineer's pull request. They read the codebase before they write anything, they fix the whole problem and not only the line in the ticket, and they leave alone what they weren't asked to change.\n\nFrontierCode grades all three. Thoroughness has 501 blocking checks, things like fixing the same bug in a sibling code path, or handling every input type the code next to it already handles [[1]](#ref-1). Scope has 123 and backward compatibility 76 [[1]](#ref-1). What I couldn't find on Cognition's page is how it would score a model that asks a question when the prompt is vague, or tells the maintainer the request is the wrong fix [[1]](#ref-1). A principal engineer does both, and on a vague ticket asking is often the right move.\n\n## Two tasks that show the gap\n\nIn the Mock Service Worker task, `resetHandlers()` didn't restart generator resolvers. Opus 5.5 at maximum effort fixed the reported bug by dropping its reference to the old generator, which means the generator's `finally` block never runs and its cleanup never happens [[1]](#ref-1). It passed 0 of 5 runs at every effort level [[1]](#ref-1). The bug in the ticket was fixed, and a reviewer who knows how generators work would have sent it back.\n\nIn the PostHog Android SDK task, a race between the live queue and a legacy class, `PostHogSendCachedEventsIntegration.kt`, sent events out of order. GPT-6 Astra at low effort made a small change and passed, and at higher effort it often deleted the legacy file, which is part of the SDK's public API, and failed on scope [[1]](#ref-1). It passed 2 of 5 runs at medium, 1 at high, 2 at extra high and 1 at max [[1]](#ref-1).\n\nThat second one shows up across the whole benchmark. At higher effort the models write more code and touch more files, and scope pass rates fall, while thoroughness failures go from 26.0% at low effort to 15.3% at max [[1]](#ref-1). Thinking longer makes them more thorough and less disciplined about what they touch, and I think knowing what not to change is the part of the job that takes engineers the longest to learn.\n\n## Would a principal engineer do better?\n\nCognition doesn't publish a human score, so this part is my estimate, not a measurement. The maintainers wrote each task and a reference solution that passes it, so the repo's own maintainer is close to 100% by construction [[1]](#ref-1) [[3]](#ref-3). The fair comparison is someone strong who has never seen the repository, given the same prompt and the same code.\n\nIt would be hard for a principal engineer to jump into something they've never seen. Principal engineers are principal because of their experience, the decisions they've made and the failures behind them, and there isn't a huge difference in coding skill between them and a senior engineer. It's more taste. That taste is what FrontierCode's scope and thoroughness checks reward (keep the public API, find the sibling path, don't touch what you weren't asked to), so I'd still put a principal ahead, at around 55%, a staff engineer around 50%, a good senior engineer around 45%, level with Opus 5.5, and a mid-level engineer a quarter. The big jump is the maintainer, because on a deliberately vague prompt part of the intent lives only in their head, and a principal in their own codebase is the maintainer.\n\nPass rate on FrontierCode 2.0, in per cent\n\n| Who | Pass rate |\n| --- | --- |\n| The repository's own maintainer | 95 |\n| A principal engineer, new to the repo | 55 |\n| A staff engineer, new to the repo | 50 |\n| Claude Opus 5.5 | 46 |\n| A senior engineer, new to the repo | 45 |\n| GPT-6 Astra | 43 |\n| A mid-level engineer, new to the repo | 25 |\n\nEach series has a zero-based scale.\n\nSo are the models better than a staff or principal engineer? Not yet on this test, and the gap is smaller than the titles suggest, because a cold repository takes away most of what a principal's experience buys them. The best two are level with a good senior engineer who's just opened the repository and about ten points behind a principal, and from 1.1 to 2.0 their lead over every other model got wider [[1]](#ref-1). The parts of the job FrontierCode doesn't grade, asking the right question and saying the fix is wrong, are the ones I'd watch next.\n\n## What's next?\n\nCognition says it'll run the evaluation for every model maker, since the tasks stay private [[3]](#ref-3). I'd like to see a task where the right answer is a question, or one where the request itself is the bug, so the benchmark can grade the \"should you\" part all the way.\n\nWe list [Devin](/tools/devin) on Anchor Terminal, and we grade the tools agents call, not how well a model writes code, so FrontierCode is a number I'll keep reading rather than one we compute. If you're a maintainer and you'd write a task where the model ought to push back, that's the one I'd want to see the scores for.\n\n## The terms, explained\n\n### What is FrontierCode 2.0?\n\nCognition's coding benchmark, released on 27 October 2026. It has 132 tasks written by maintainers of open-source repositories and grades whether a model's pull request is good enough to merge, using rubrics where any failed blocking check scores the attempt zero.\n\n### Which model scored best?\n\nClaude Opus 5.5, at 45.5%, with GPT-6 Astra second at 42.8%. Every other model on the leaderboard is below 29%.\n\n### What does underspecified mean here?\n\nThe prompts leave out engineering decisions on purpose, so the model has to work out what the maintainer wants from the codebase, such as fixing the same bug in a sibling code path or keeping the public API intact.\n\n### Are AI models better than a principal engineer at coding?\n\nNot on FrontierCode 2.0, by my estimate. Cognition publishes no human score. I'd put a principal engineer new to the repository at around 55%, a staff engineer around 50% and a good senior engineer around 45%, level with the best model. A principal's edge is taste from experience more than coding skill, and it shows most in their own codebase, where they're the maintainer at close to 100%.\n\n### Why do some models score lower at higher reasoning effort?\n\nAt higher effort they write more code and touch more files, so they fail more scope checks, such as deleting a file that's part of a public API, even though they get more thorough.\n\n---\n\nMore in [Opinion](https://www.anchorterminal.com/blog/opinion/index.md) · Older opinion [Genie 4 runs a Gemini agent alongside the world model](https://www.anchorterminal.com/blog/genie-4-world-model.md)\n\n- Older: [Can you run Qwen3.8-27B locally? Yes, on a graphics card you might already own](https://www.anchorterminal.com/blog/run-qwen3-8-27b-locally.md)\n- All posts: https://www.anchorterminal.com/blog/index.md\n\n## References\n\nRead on 28 October 2026. Quotes are as published.\n\n[1] Cognition, \"FrontierCode 2.0\", 27 October 2026. Source for the release date, 132 tasks, 55% more repositories, security and performance tasks, 43% more blocking checks, 67% more files and 89% more lines, codebases 25% larger, 80% of reference solutions differing from upstream, the blocking checks by category, the thoroughness examples, scores at each effort level, the Mock Service Worker and PostHog tasks, and the 1.1 and 2.0 scores. https://cognition.com/blog/frontier-code-2.0\n\n[2] Cognition, \"FrontierCode leaderboard\". Source for runs that consult the original pull request being scored zero. https://cognition.com/frontiercode\n\n[3] Eric Lu, Ben Pan and others at Cognition, \"Introducing FrontierCode\", 8 June 2026. Source for 36 repositories, 40 hours per task, prompts a third the length of SWE-Bench Pro's, \"expects the agent to infer the maintainer's intent\", the grading methods, the QC quote, private tasks and the offer to model makers. https://cognition.com/blog/frontier-code\n\n[4] GitLab, \"Dev Career Framework, Principal Engineer\", GitLab Handbook. Source for the three quoted expectations of a principal engineer. https://handbook.gitlab.com/handbook/engineering/careers/matrix/development/dev/principal/\n\n[5] Google, \"The Standard of Code Review\", Google Engineering Practices. Source for when a reviewer should approve a change. https://google.github.io/eng-practices/review/reviewer/standard.html\n\n[6] Cognition, announcement of FrontierCode 2.0, 27 October 2026. Source for \"most coding evals\" checking a model meets a spec, 1.0's \"would you actually merge the PR?\" and \"not only would you merge this PR, but should you?\". https://x.com/cognition/status/2115132488775524793\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-10-28",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.7",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "Blog",
        "url": "https://www.anchorterminal.com/blog/"
      },
      {
        "name": "Opinion",
        "url": "https://www.anchorterminal.com/blog/opinion/"
      },
      {
        "name": "FrontierCode 2.0",
        "url": ""
      }
    ],
    "description": "Cognition released FrontierCode 2.0 on 27 October 2026, 132 tasks written by open-source maintainers and graded on whether the pull request should be merged. Claude Opus 5.5 leads at 45.5%. Vlad Cealicu compares what it grades with what we expect of a staff or principal engineer, the top of the field for writing code, and estimates how one would score.",
    "facts": [
      "Vlad Cealicu",
      "28 October 2026"
    ],
    "h1": "Is the best coding model better than a principal engineer? Not yet, but closer than the titles suggest",
    "image": "https://www.anchorterminal.com/assets/og/blog-frontiercode-principal-engineer.png",
    "path": "/blog/frontiercode-principal-engineer",
    "published": "2026-10-28",
    "section": "blog",
    "title": "FrontierCode 2.0, 45.5% at the top, and a principal engineer baseline | Anchor Terminal",
    "toc": null,
    "updated": "2026-10-28",
    "url": "https://www.anchorterminal.com/blog/frontiercode-principal-engineer"
  },
  "tokens": {
    "markdown": 3400,
    "slim": 1630
  },
  "version": 1
}