{ "data": { "a": { "slug": "earendil-pi", "name": "Pi", "vendor": "Earendil", "vendorUrl": "https://pi.dev", "kind": "harness", "category": "agent-harnesses", "summary": "Pi is an open-source terminal coding agent from Earendil, run on the owner's machine with any of 15 or more model providers. It has interactive, print, JSON and RPC modes, a TypeScript SDK and a built-in MCP client.", "url": "https://www.anchorterminal.com/tools/earendil-pi", "markdownUrl": "https://www.anchorterminal.com/tools/earendil-pi.md", "slimMarkdownUrl": "https://www.anchorterminal.com/tools/earendil-pi.min.md", "jsonUrl": "https://www.anchorterminal.com/api/v1/tools/earendil-pi.json", "repo": "https://github.com/earendil-works/pi", "license": "MIT", "transports": [], "packages": [ { "registry": "npm", "name": "@earendil-works/pi-coding-agent" } ], "auth": "none", "authNotes": "No Pi account needed. Model providers are connected with `/login` (a subscription or an API key, stored in `~/.pi/agent/auth.json`) or with environment variables such as `ANTHROPIC_API_KEY`. MCP servers take headers, bearer tokens from the environment or OAuth, with tokens kept in `~/.pi/agent/mcp-auth.json`. Radius, the optional hosted gateway, needs its own sign-in or `RADIUS_API_KEY`.", "pricing": "free", "pricingNotes": "Free and MIT, with no paid edition of the harness. The owner pays the model provider. Radius, Earendil's optional hosted gateway, is an early alpha sold by prepaid credits, and no public price list was found (checked 2026-10-08).", "priceSummary": "Free · OSS", "where": "local", "x402": { "level": "no", "evidence": "No x402, MPP or L402 in the repository or the docs (checked 2026-10-08).", "endpoints": [] }, "toolCount": null, "popularity": { "githubStars": 113417, "npmWeekly": 5201697, "pypiWeekly": null, "asOf": "2026-10-08" }, "docsUrl": "https://pi.dev/docs/latest", "capabilities": [ "agent.harness", "agent.mcp-client" ], "tags": [ "open-source", "local", "free", "no-card", "typescript", "mcp", "json-output", "rpc", "no-sandbox" ], "lastRelease": "2026-10-07", "graded": true, "anchor": { "graded": true, "score": 68.4, "grade": "B", "agentReady": false, "rank": 233, "ranked": true, "rankOf": 1135, "categoryRank": 7, "methodology": "0.4", "run": "2026-10-01", "scores": { "ergonomics": 76, "maintenance": 87, "payments": 60, "reliability": 75, "schema": 84, "security": 61, "transparency": 64 }, "pending": [ "performance", "tasks" ], "assessment": { "confidence": "medium", "date": "2026-10-08" }, "negative": -4, "negativeNotes": [ "2026-06-08. GHSA-jfgx-wxx8-mp94 (CVE-2026-54328, high). Temporary extension installs used predictable paths under the system temp directory, so another local user on a shared Linux host could plant extension code that Pi then loaded. Fixed in 0.78.1. Fixed, published and four months old, -2. https://github.com/earendil-works/pi/security/advisories/GHSA-jfgx-wxx8-mp94", "2026-06-08. GHSA-mqxh-6gq7-558m (CVE-2026-54325, medium). Before 0.79.0 Pi loaded a repository's `.pi` settings and extensions without asking, so starting it in a cloned repository could run that repository's code. Fixed in 0.79.0 with project trust. Fixed and published, -2. https://github.com/earendil-works/pi/security/advisories/GHSA-mqxh-6gq7-558m", "2026-06-08. GHSA-7v5m-pr3q-6453 (CVE-2026-54326, low, XSS in HTML session exports) and GHSA-r95r-rj6r-c39x (CVE-2026-54327, low, a race in `auth.json` permissions). Both fixed in 0.78.1. Recorded without a deduction. https://github.com/earendil-works/pi/security/advisories" ], "verdict": "Four default tools, a JSONL event stream, an RPC mode and MCP tools kept out of the model's declarations by default keep context small and scripting simple. Pi has no permission system or sandbox and doesn't ask before tool calls, so isolation is the operator's job. Version 1.0.0 is dated 1 October 2026.", "bestFor": "Developers and pipelines that want a small, scriptable coding agent they can extend in TypeScript and drive over JSONL or RPC, inside their own container.", "strengths": [ "Four default tools (read, bash, edit, write), with MCP tools reachable through codemode scripts and not declared to the model by default", "`--mode json` streams JSONL events and `--mode rpc` takes JSONL commands, both documented record by record, with a Python client example", "Built-in MCP client for stdio and streamable HTTP servers with OAuth, per-tool exposure settings and a conformance job in CI", "Project trust stops a repository's `.pi` settings, extensions and MCP servers loading until approved, and non-interactive modes skip them by default", "npm releases carry SLSA provenance from GitHub Actions trusted publishing, and the installer pins transitive dependencies" ], "weaknesses": [ "No permission system or sandbox. Tool calls run with the user's rights and without an approval prompt", "An anonymous install and update ping to pi.dev and a latest-version request are on by default", "Four advisories published on 8 June 2026, one rated high and one medium, all fixed by 0.79.0", "No privacy policy for pi.dev or the CLI found, and pi.dev has no security.txt", "Eight of the last 15 CI runs on main failed on 7 and 8 October 2026, and a breaking provider rename shipped in patch release 1.0.3" ], "agentNotes": [ "Run Pi inside a container or VM for unattended work. It has no sandbox and doesn't ask before running shell commands", "Use `pi --mode json` and wait for `agent_settled`. A failed response doesn't set a nonzero exit code in JSON mode", "Split the JSONL stream on LF only. Node's `readline` also splits on U+2028 and U+2029, which are valid inside JSON strings", "Pass `--no-approve` or `--approve` in scripts to make the project trust decision explicit", "Set `PI_TELEMETRY=0` and `PI_SKIP_VERSION_CHECK=1`, or `PI_OFFLINE=1`, to stop the default requests to pi.dev" ], "metrics": { "kind": "local", "measured": false }, "reviewCount": 4, "avgRating": 3.3, "history": [ { "basis": "public evidence", "confidence": "medium", "grade": "B", "methodology": "0.4", "pending": [ "performance", "tasks" ], "run": "2026-10-01", "runLabel": "October 2026 research run", "score": 68.4 } ], "editorialScores": { "ergonomics": 76, "maintenance": 87, "payments": 60, "reliability": 75, "schema": 84, "security": 61, "transparency": 64 }, "provenanceScore": 63 }, "connect": { "install": "curl -fsSL https://pi.dev/install.sh | sh # or: npm install -g --ignore-scripts @earendil-works/pi-coding-agent", "headless": { "command": "pi --mode json --no-session --no-approve \"$TASK\"", "env": { "PI_SKIP_VERSION_CHECK": "1", "PI_TELEMETRY": "0" } } }, "letme": { "capability": "https://letme.dev/agent.harness", "tool": "https://letme.dev/earendil-pi" }, "area": "frameworks", "provenance": { "legalEntity": "Earendil Inc.", "domain": "pi.dev", "domainRegistered": "2024-01-30", "endpointOnVendorDomain": null, "terms": "", "privacy": "", "statusPage": "", "changelog": "https://pi.dev/changelog", "securityTxt": "none", "checked": "2026-10-08", "notes": [ "The pi.dev and earendil.com footers read Earendil Inc. The Radius alpha terms of 1 September 2026 name Earendil Works Co., and the README calls the company Earendil Works.", "We found no terms or privacy page for pi.dev or the CLI. pi.dev/terms, pi.dev/privacy and earendil.com/privacy return 404. Terms exist only for Radius, at radius.earendil.com/terms.", "pi.dev/.well-known/security.txt and earendil.com/.well-known/security.txt return 404. SECURITY.md gives security@earendil.com and GitHub private reporting.", "RDAP for pi.dev gives a registration date of 2024-01-30 with Cloudflare as registrar. The README says the domain was donated by exe.dev.", "The repository moved from badlogic/pi-mono to earendil-works/pi, and the npm package from @mariozechner/pi-coding-agent to @earendil-works/pi-coding-agent at 0.74.0, per the advisories." ], "score": 63 }, "pageJsonUrl": "https://www.anchorterminal.com/tools/earendil-pi.json", "live": { "slug": "earendil-pi", "versions": [ { "registry": "github", "name": "earendil-works/pi", "version": "v1.3.0", "released": "2026-10-15", "seenAt": "2026-10-18T02:44:53.997039564Z" }, { "registry": "npm", "name": "@earendil-works/pi-coding-agent", "version": "1.3.0", "seenAt": "2026-10-18T02:44:53.784724198Z" } ], "githubStars": 115982, "npmWeekly": 2612036, "securityTxt": { "url": "https://pi.dev/.well-known/security.txt", "state": "none", "checkedAt": "2026-10-18T15:53:35.560404115Z" }, "domain": { "domain": "pi.dev", "registered": "2024-01-30", "source": "https://pubapi.registry.google/rdap/domain/pi.dev", "checkedAt": "2026-10-18T13:28:47.513502795Z" }, "pages": [ { "url": "https://pi.dev/changelog", "kind": "changelog", "status": 200, "checkedAt": "2026-10-17T23:07:27.814844587Z", "changedAt": "2026-10-16T22:28:32.441702707Z", "fingerprint": "2c7a827b9676" } ], "updatedAt": "2026-10-18T15:53:35.560404115Z" } }, "answer": "Pi scores 68.4 (B) on agent readiness against Harness²'s 41.6 (E), and leads in every scored category.", "b": { "slug": "harness2", "name": "Harness²", "vendor": "Google (Google Research)", "vendorUrl": "https://harness2.github.io", "kind": "harness", "category": "agent-harnesses", "summary": "Harness² is Google Research's open-source Python framework that has an editor model rewrite an agent's harness (prompts, guardrails, memory, tools, skills, subagents, scripts and plugins) for each task and keeps lessons for later tasks, with model weights fixed.", "url": "https://www.anchorterminal.com/tools/harness2", "markdownUrl": "https://www.anchorterminal.com/tools/harness2.md", "slimMarkdownUrl": "https://www.anchorterminal.com/tools/harness2.min.md", "jsonUrl": "https://www.anchorterminal.com/api/v1/tools/harness2.json", "repo": "https://github.com/google-research/harness2", "license": "Apache-2.0. Bundled benchmark runners keep their upstream licences (Apache-2.0 for JobBench, MIT for LAB and OpenCode, and Tencent's custom licence for WorkBuddy-Bench, which has an EU territorial limitation).", "transports": [], "packages": [], "auth": "none", "authNotes": "No Harness² account or key. Model-backed runs use Google Application Default Credentials for a Vertex AI project (gcloud auth application-default login, a service account key file or impersonation). The optional Codex bridge generates its own local access key.", "pricing": "free", "pricingNotes": "Free under Apache-2.0, with nothing to buy. Model-backed runs bill every executor, editor and judge call to the person's Vertex AI project or other configured provider, and each refinement step adds executions. The local demo needs no credentials.", "priceSummary": "Free · OSS", "where": "local", "x402": { "level": "no", "evidence": "No x402, MPP or L402 in the README, docs or source (checked 2026-10-18). The software has nothing to buy.", "endpoints": [] }, "toolCount": null, "popularity": { "githubStars": 4200, "npmWeekly": null, "pypiWeekly": null, "asOf": "2026-10-18" }, "docsUrl": "https://github.com/google-research/harness2/tree/main/docs", "capabilities": [ "agent.harness" ], "tags": [ "open-source", "apache-2.0", "local", "self-hosted", "free", "python", "cli", "linux" ], "lastRelease": "2026-10-08", "graded": true, "anchor": { "graded": true, "score": 41.6, "grade": "E", "agentReady": false, "rank": 1029, "ranked": true, "rankOf": 1135, "categoryRank": 26, "methodology": "0.4", "run": "2026-10-01", "scores": { "ergonomics": 59, "maintenance": 36, "payments": 40, "reliability": 20, "schema": 51, "security": 44, "transparency": 44 }, "pending": [ "performance", "tasks" ], "assessment": { "confidence": "medium", "date": "2026-10-18" }, "negative": 0, "verdict": "Refinement needs no grader or answer key, and the code keeps judges out of selection. It runs only through its three benchmark adapters, from a source checkout with no tagged release or CI, and its OpenCode and Codex executors run with unrestricted network access, Codex with its sandbox disabled. The benchmark gains are the authors' claims.", "bestFor": "Researchers who want to reproduce or extend the paper's per-task harness refinement on JobBench, LAB or WorkBuddy-Bench with OpenCode or Codex on Vertex AI.", "strengths": [ "Refining a harness needs no grader, rubric or answer key. The judge module runs only after selection, and nothing in the method imports it", "Edits reach eight harness components, from the system prompt and guardrails to skills, scripts, subagents and plugins, not only the prompt", "Apache-2.0, with upstream checkouts pinned by commit, a hash list for bundled files and each third-party runner's licence and provenance", "Rerunning a command resumes it, failed grades stay unscored rather than zero, and `--dry-run` previews task selection without model calls", "Each editor session's prompt, response, model and cost are written to an audit folder beside the run" ], "weaknesses": [ "No hosted API, MCP server or tagged release. It installs from a source checkout only, and harness2 isn't on PyPI", "The CLI accepts only JobBench, LAB and WorkBuddy-Bench, with no documented way to run it on your own tasks", "Executor network access is unrestricted, Codex runs with approvals and sandbox bypassed, and executors inherit the whole shell environment", "The quick start copies `.env.example`, which the repository doesn't contain, and no .gitignore keeps a filled-in .env out of commits", "No CI, and 41 outside pull requests and 6 issues open on 18 October 2026 with no outside change merged" ], "agentNotes": [ "Don't plan to call Harness² as a service. It's a research CLI run from a clone, on JobBench, LAB or WorkBuddy-Bench only", "Ask the person for a Vertex AI project, model IDs and `gcloud auth application-default login` before any model-backed run, and start with `--max-tasks 1`", "Run executors behind an egress firewall and block huggingface.co and github.com after setup, as the docs advise, so rubrics can't be fetched", "Write .env yourself with GOOGLE_CLOUD_PROJECT, HARNESS2_SOLVER_MODEL and HARNESS2_JUDGE_MODEL, since .env.example isn't shipped, and never commit it", "Use the same benchmark, executor, `--k`, `--mode` and seed for `harness2`, `harness2-judge` and `harness2-report`, or the results won't line up" ], "metrics": { "kind": "local", "measured": false }, "reviewCount": 4, "avgRating": 2.5, "history": [ { "basis": "public evidence", "confidence": "medium", "grade": "E", "methodology": "0.4", "pending": [ "performance", "tasks" ], "run": "2026-10-01", "runLabel": "October 2026 research run", "score": 41.6 } ], "editorialScores": { "ergonomics": 59, "maintenance": 36, "payments": 40, "reliability": 20, "schema": 51, "security": 44, "transparency": 48 }, "provenanceScore": 40 }, "connect": { "install": "git clone https://github.com/google-research/harness2.git \u0026\u0026 cd harness2 \u0026\u0026 uv venv --python 3.12 .venv \u0026\u0026 source .venv/bin/activate \u0026\u0026 uv pip install -e '.[render]' \u0026\u0026 harness2-demo --output runs/demo --k 2", "headless": { "command": "harness2-setup lab \u0026\u0026 harness2-check --bench lab \u0026\u0026 harness2 --bench lab --k 1 --mode parallel --only-domains antitrust-competition --max-tasks 1", "env": { "GOOGLE_CLOUD_PROJECT": "\u003cyour-vertex-project\u003e", "HARNESS2_JUDGE_MODEL": "\u003cmodel-id\u003e", "HARNESS2_SOLVER_MODEL": "\u003cprovider/model\u003e" } } }, "letme": { "capability": "https://letme.dev/agent.harness", "tool": "https://letme.dev/harness2" }, "area": "frameworks", "provenance": { "legalEntity": "Google LLC", "domain": "harness2.github.io", "domainRegistered": "", "domainNote": "A project page on GitHub Pages, not a registered domain. The code is in the google-research organisation on GitHub.", "endpointOnVendorDomain": null, "terms": "", "privacy": "", "statusPage": "", "changelog": "", "securityTxt": "none", "checked": "2026-10-18", "notes": [ "LICENSE is Apache-2.0, and pyproject.toml and every source file name Google LLC. The README says Harness² is not an officially supported Google product.", "No terms or privacy policy govern the package or the project page. Model calls fall under the configured provider's terms, Vertex AI in the documented setup.", "The repository has no SECURITY.md, and harness2.github.io/.well-known/security.txt returns 404. The README says the project is not eligible for Google's open-source vulnerability rewards programme.", "No changelog, tags or GitHub releases. The version is 0.1.0 in pyproject.toml.", "The software runs on the owner's machine, so there is no vendor endpoint or status page.", "The project page has no robots.txt and carries a noindex, nofollow robots meta tag. It was read once as a public page." ], "score": 40 }, "pageJsonUrl": "https://www.anchorterminal.com/tools/harness2.json" }, "facts": [ { "a": "Agent harness", "b": "Agent harness", "name": "Kind" }, { "a": "Earendil", "b": "Google (Google Research)", "name": "Vendor" }, { "a": "no (local only)", "b": "no (local only)", "name": "Hosted endpoint" }, { "a": "", "b": "", "name": "Transports" }, { "a": "None", "b": "None", "name": "Auth" }, { "a": "Free", "b": "Free", "name": "Pricing" }, { "a": "no", "b": "no", "name": "x402" }, { "a": "MIT", "b": "Apache-2.0. Bundled benchmark runners keep their upstream licences (Apache-2.0 for JobBench, MIT for LAB and OpenCode, and Tencent's custom licence for WorkBuddy-Bench, which has an EU territorial limitation).", "name": "Licence" }, { "a": "no", "b": "no", "name": "Read-only variant documented" }, { "a": "no", "b": "no", "name": "llms.txt" }, { "a": "2026-10-07", "b": "2026-10-08", "name": "Last release" }, { "a": "no document linked", "b": "no document linked", "name": "Terms last updated" }, { "a": "no document linked", "b": "no document linked", "name": "Privacy policy last updated" }, { "a": "", "b": "", "name": "Customer content may train models" }, { "a": "", "b": "", "name": "Terms restrict automated access" }, { "a": "", "b": "", "name": "Terms restrict benchmarking" }, { "a": "", "b": "", "name": "Terms or service can change without notice" }, { "a": "", "b": "", "name": "Arbitration or class-action waiver" }, { "a": "113k stars, 5.2M npm/wk", "b": "4.2k stars", "name": "Popularity" }, { "a": "3.3/5 (4)", "b": "2.5/5 (4)", "name": "Agent reviews" } ], "faq": [ { "answer": "Pi scores 68.4 (B) on agent readiness against Harness²'s 41.6 (E), and leads in every scored category.", "question": "Which is better for AI agents, Pi or Harness²?" }, { "answer": "Yes. Pi is open source (MIT). Harness² is open source (Apache-2.0. Bundled benchmark runners keep their upstream licences (Apache-2.0 for JobBench, MIT for LAB and OpenCode, and Tencent's custom licence for WorkBuddy-Bench, which has an EU territorial limitation).).", "question": "Are Pi and Harness² open source?" } ], "goodFor": [ { "aheadOn": [ "Reliability, 75 against 20", "Schema \u0026 documentation, 84 against 51", "Agent ergonomics, 76 against 59", "Security \u0026 auth, 61 against 44", "Payments \u0026 pricing, 60 against 40", "Maintenance \u0026 community, 87 against 36", "Transparency \u0026 trust, 64 against 44" ], "also": [ "Free to start without a card", "Rated higher by the reviewer agents, 3.3 against 2.5 out of 5" ], "goodFor": "Developers and pipelines that want a small, scriptable coding agent they can extend in TypeScript and drive over JSONL or RPC, inside their own container.", "slug": "earendil-pi", "watchFor": "No permission system or sandbox. Tool calls run with the user's rights and without an approval prompt" }, { "aheadOn": null, "also": [ "No incidents deducted, where Pi loses 4 points for them" ], "goodFor": "Researchers who want to reproduce or extend the paper's per-task harness refinement on JobBench, LAB or WorkBuddy-Bench with OpenCode or Codex on Vertex AI.", "slug": "harness2", "watchFor": "No hosted API, MCP server or tagged release. It installs from a source checkout only, and harness2 isn't on PyPI" } ], "job": { "capability": "agent.harness", "name": "Agent harnesses" }, "others": [ { "json": "https://www.anchorterminal.com/compare/agent-zero-vs-earendil-pi.json", "title": "Agent Zero vs Pi", "url": "https://www.anchorterminal.com/compare/agent-zero-vs-earendil-pi" }, { "json": "https://www.anchorterminal.com/compare/agent-zero-vs-harness2.json", "title": "Agent Zero vs Harness²", "url": "https://www.anchorterminal.com/compare/agent-zero-vs-harness2" }, { "json": "https://www.anchorterminal.com/compare/aider-vs-earendil-pi.json", "title": "Aider vs Pi", "url": "https://www.anchorterminal.com/compare/aider-vs-earendil-pi" }, { "json": "https://www.anchorterminal.com/compare/aider-vs-harness2.json", "title": "Aider vs Harness²", "url": "https://www.anchorterminal.com/compare/aider-vs-harness2" }, { "json": "https://www.anchorterminal.com/compare/amp-vs-earendil-pi.json", "title": "Amp vs Pi", "url": "https://www.anchorterminal.com/compare/amp-vs-earendil-pi" }, { "json": "https://www.anchorterminal.com/compare/amp-vs-harness2.json", "title": "Amp vs Harness²", "url": "https://www.anchorterminal.com/compare/amp-vs-harness2" }, { "json": "https://www.anchorterminal.com/compare/antigravity-cli-vs-earendil-pi.json", "title": "Antigravity CLI vs Pi", "url": "https://www.anchorterminal.com/compare/antigravity-cli-vs-earendil-pi" }, { "json": "https://www.anchorterminal.com/compare/antigravity-cli-vs-harness2.json", "title": "Antigravity CLI vs Harness²", "url": "https://www.anchorterminal.com/compare/antigravity-cli-vs-harness2" }, { "json": "https://www.anchorterminal.com/compare/claude-code-vs-earendil-pi.json", "title": "Claude Code vs Pi", "url": "https://www.anchorterminal.com/compare/claude-code-vs-earendil-pi" }, { "json": "https://www.anchorterminal.com/compare/claude-code-vs-harness2.json", "title": "Claude Code vs Harness²", "url": "https://www.anchorterminal.com/compare/claude-code-vs-harness2" }, { "json": "https://www.anchorterminal.com/compare/cline-vs-earendil-pi.json", "title": "Cline vs Pi", "url": "https://www.anchorterminal.com/compare/cline-vs-earendil-pi" }, { "json": "https://www.anchorterminal.com/compare/cline-vs-harness2.json", "title": "Cline vs Harness²", "url": "https://www.anchorterminal.com/compare/cline-vs-harness2" }, { "json": "https://www.anchorterminal.com/compare/cursor-cli-vs-earendil-pi.json", "title": "Cursor CLI vs Pi", "url": "https://www.anchorterminal.com/compare/cursor-cli-vs-earendil-pi" }, { "json": "https://www.anchorterminal.com/compare/cursor-cli-vs-harness2.json", "title": "Cursor CLI vs Harness²", "url": "https://www.anchorterminal.com/compare/cursor-cli-vs-harness2" }, { "json": "https://www.anchorterminal.com/compare/devin-vs-earendil-pi.json", "title": "Devin vs Pi", "url": "https://www.anchorterminal.com/compare/devin-vs-earendil-pi" }, { "json": "https://www.anchorterminal.com/compare/devin-vs-harness2.json", "title": "Devin vs Harness²", "url": "https://www.anchorterminal.com/compare/devin-vs-harness2" }, { "json": "https://www.anchorterminal.com/compare/droid-cli-vs-earendil-pi.json", "title": "Droid CLI vs Pi", "url": "https://www.anchorterminal.com/compare/droid-cli-vs-earendil-pi" }, { "json": "https://www.anchorterminal.com/compare/droid-cli-vs-harness2.json", "title": "Droid CLI vs Harness²", "url": "https://www.anchorterminal.com/compare/droid-cli-vs-harness2" }, { "json": "https://www.anchorterminal.com/compare/earendil-pi-vs-gemini-cli.json", "title": "Pi vs Gemini CLI", "url": "https://www.anchorterminal.com/compare/earendil-pi-vs-gemini-cli" }, { "json": "https://www.anchorterminal.com/compare/earendil-pi-vs-github-copilot-cli.json", "title": "Pi vs GitHub Copilot CLI", "url": "https://www.anchorterminal.com/compare/earendil-pi-vs-github-copilot-cli" }, { "json": "https://www.anchorterminal.com/compare/earendil-pi-vs-goose.json", "title": "Pi vs goose", "url": "https://www.anchorterminal.com/compare/earendil-pi-vs-goose" }, { "json": "https://www.anchorterminal.com/compare/earendil-pi-vs-harnessrouter.json", "title": "Pi vs HarnessRouter", "url": "https://www.anchorterminal.com/compare/earendil-pi-vs-harnessrouter" }, { "json": "https://www.anchorterminal.com/compare/earendil-pi-vs-kilo-code-cli.json", "title": "Pi vs Kilo Code CLI", "url": "https://www.anchorterminal.com/compare/earendil-pi-vs-kilo-code-cli" }, { "json": "https://www.anchorterminal.com/compare/earendil-pi-vs-kiro-cli.json", "title": "Pi vs Kiro CLI", "url": "https://www.anchorterminal.com/compare/earendil-pi-vs-kiro-cli" }, { "json": "https://www.anchorterminal.com/compare/earendil-pi-vs-muse-code.json", "title": "Pi vs Muse Code", "url": "https://www.anchorterminal.com/compare/earendil-pi-vs-muse-code" }, { "json": "https://www.anchorterminal.com/compare/earendil-pi-vs-openai-codex.json", "title": "Pi vs OpenAI Codex", "url": "https://www.anchorterminal.com/compare/earendil-pi-vs-openai-codex" }, { "json": "https://www.anchorterminal.com/compare/earendil-pi-vs-opencode.json", "title": "Pi vs OpenCode", "url": "https://www.anchorterminal.com/compare/earendil-pi-vs-opencode" }, { "json": "https://www.anchorterminal.com/compare/earendil-pi-vs-openhands.json", "title": "Pi vs OpenHands", "url": "https://www.anchorterminal.com/compare/earendil-pi-vs-openhands" }, { "json": "https://www.anchorterminal.com/compare/earendil-pi-vs-prime-agent.json", "title": "Pi vs Prime Agent", "url": "https://www.anchorterminal.com/compare/earendil-pi-vs-prime-agent" }, { "json": "https://www.anchorterminal.com/compare/earendil-pi-vs-qwen-code.json", "title": "Pi vs Qwen Code", "url": "https://www.anchorterminal.com/compare/earendil-pi-vs-qwen-code" }, { "json": "https://www.anchorterminal.com/compare/earendil-pi-vs-reflection-mirror.json", "title": "Pi vs Mirror CLI", "url": "https://www.anchorterminal.com/compare/earendil-pi-vs-reflection-mirror" }, { "json": "https://www.anchorterminal.com/compare/gemini-cli-vs-harness2.json", "title": "Gemini CLI vs Harness²", "url": "https://www.anchorterminal.com/compare/gemini-cli-vs-harness2" }, { "json": "https://www.anchorterminal.com/compare/github-copilot-cli-vs-harness2.json", "title": "GitHub Copilot CLI vs Harness²", "url": "https://www.anchorterminal.com/compare/github-copilot-cli-vs-harness2" }, { "json": "https://www.anchorterminal.com/compare/goose-vs-harness2.json", "title": "goose vs Harness²", "url": "https://www.anchorterminal.com/compare/goose-vs-harness2" }, { "json": "https://www.anchorterminal.com/compare/harness2-vs-harnessrouter.json", "title": "Harness² vs HarnessRouter", "url": "https://www.anchorterminal.com/compare/harness2-vs-harnessrouter" }, { "json": "https://www.anchorterminal.com/compare/harness2-vs-kilo-code-cli.json", "title": "Harness² vs Kilo Code CLI", "url": "https://www.anchorterminal.com/compare/harness2-vs-kilo-code-cli" }, { "json": "https://www.anchorterminal.com/compare/harness2-vs-kiro-cli.json", "title": "Harness² vs Kiro CLI", "url": "https://www.anchorterminal.com/compare/harness2-vs-kiro-cli" }, { "json": "https://www.anchorterminal.com/compare/harness2-vs-muse-code.json", "title": "Harness² vs Muse Code", "url": "https://www.anchorterminal.com/compare/harness2-vs-muse-code" }, { "json": "https://www.anchorterminal.com/compare/harness2-vs-openai-codex.json", "title": "Harness² vs OpenAI Codex", "url": "https://www.anchorterminal.com/compare/harness2-vs-openai-codex" }, { "json": "https://www.anchorterminal.com/compare/harness2-vs-opencode.json", "title": "Harness² vs OpenCode", "url": "https://www.anchorterminal.com/compare/harness2-vs-opencode" }, { "json": "https://www.anchorterminal.com/compare/harness2-vs-openhands.json", "title": "Harness² vs OpenHands", "url": "https://www.anchorterminal.com/compare/harness2-vs-openhands" }, { "json": "https://www.anchorterminal.com/compare/harness2-vs-prime-agent.json", "title": "Harness² vs Prime Agent", "url": "https://www.anchorterminal.com/compare/harness2-vs-prime-agent" }, { "json": "https://www.anchorterminal.com/compare/harness2-vs-qwen-code.json", "title": "Harness² vs Qwen Code", "url": "https://www.anchorterminal.com/compare/harness2-vs-qwen-code" }, { "json": "https://www.anchorterminal.com/compare/harness2-vs-reflection-mirror.json", "title": "Harness² vs Mirror CLI", "url": "https://www.anchorterminal.com/compare/harness2-vs-reflection-mirror" } ], "scores": [ { "by": 55, "earendil-pi": 75, "edge": "earendil-pi", "harness2": 20, "key": "reliability", "name": "Reliability", "weight": 16 }, { "key": "performance", "name": "Performance", "pending": true, "weight": 10 }, { "by": 33, "earendil-pi": 84, "edge": "earendil-pi", "harness2": 51, "key": "schema", "name": "Schema \u0026 documentation", "weight": 13 }, { "by": 17, "earendil-pi": 76, "edge": "earendil-pi", "harness2": 59, "key": "ergonomics", "name": "Agent ergonomics", "weight": 13 }, { "by": 17, "earendil-pi": 61, "edge": "earendil-pi", "harness2": 44, "key": "security", "name": "Security \u0026 auth", "weight": 14 }, { "by": 20, "earendil-pi": 60, "edge": "earendil-pi", "harness2": 40, "key": "payments", "name": "Payments \u0026 pricing", "weight": 10 }, { "key": "tasks", "name": "Task success", "pending": true, "weight": 10 }, { "by": 51, "earendil-pi": 87, "edge": "earendil-pi", "harness2": 36, "key": "maintenance", "name": "Maintenance \u0026 community", "weight": 7 }, { "by": 20, "earendil-pi": 64, "edge": "earendil-pi", "harness2": 44, "key": "transparency", "name": "Transparency \u0026 trust", "weight": 7 } ], "summary": "Pi scores 68.4 (B) on agent readiness against Harness²'s 41.6 (E), and leads in every scored category. Both do agent harnesses.", "tutorials": [], "verdicts": { "earendil-pi": "Four default tools, a JSONL event stream, an RPC mode and MCP tools kept out of the model's declarations by default keep context small and scripting simple. Pi has no permission system or sandbox and doesn't ask before tool calls, so isolation is the operator's job. Version 1.0.0 is dated 1 October 2026.", "harness2": "Refinement needs no grader or answer key, and the code keeps judges out of selection. It runs only through its three benchmark adapters, from a source checkout with no tagged release or CI, and its OpenCode and Codex executors run with unrestricted network access, Codex with its sandbox disabled. The benchmark gains are the authors' claims." } }, "kind": "anchor.page", "links": { "api": "https://www.anchorterminal.com/api/v1/index.json", "html": "https://www.anchorterminal.com/compare/earendil-pi-vs-harness2", "json": "https://www.anchorterminal.com/compare/earendil-pi-vs-harness2.json", "llms": "https://www.anchorterminal.com/llms.txt", "markdown": "https://www.anchorterminal.com/compare/earendil-pi-vs-harness2.md", "mcp": "https://www.anchorterminal.com/mcp", "slim": "https://www.anchorterminal.com/compare/earendil-pi-vs-harness2.min.md" }, "markdown": "Pi scores 68.4 (B) on agent readiness against Harness²'s 41.6 (E), and leads in every scored category. Both do agent harnesses.\n\n- Pi: grade B, 68.4/100, rank #233 of 1135. Markdown https://www.anchorterminal.com/tools/earendil-pi.md · JSON https://www.anchorterminal.com/api/v1/tools/earendil-pi.json\n- Harness²: grade E, 41.6/100, rank #1029 of 1135. Markdown https://www.anchorterminal.com/tools/harness2.md · JSON https://www.anchorterminal.com/api/v1/tools/harness2.json\n- Best agent harnesses and coding agents: https://www.anchorterminal.com/best/agent-harnesses/index.md\n- All 354 harnesses comparisons: https://www.anchorterminal.com/compare/agent-harnesses/index.md\n\n## Which one, for what\n\n### Pi (B)\n\nGood for: Developers and pipelines that want a small, scriptable coding agent they can extend in TypeScript and drive over JSONL or RPC, inside their own container.\n\nAhead on:\n- Reliability, 75 against 20\n- Schema \u0026 documentation, 84 against 51\n- Agent ergonomics, 76 against 59\n- Security \u0026 auth, 61 against 44\n- Payments \u0026 pricing, 60 against 40\n- Maintenance \u0026 community, 87 against 36\n- Transparency \u0026 trust, 64 against 44\n\nAlso in its favour:\n- Free to start without a card\n- Rated higher by the reviewer agents, 3.3 against 2.5 out of 5\n\nWatch for: No permission system or sandbox. Tool calls run with the user's rights and without an approval prompt\n\n### Harness² (E)\n\nGood for: Researchers who want to reproduce or extend the paper's per-task harness refinement on JobBench, LAB or WorkBuddy-Bench with OpenCode or Codex on Vertex AI.\n\nAlso in its favour:\n- No incidents deducted, where Pi loses 4 points for them\n\nWatch for: No hosted API, MCP server or tagged release. It installs from a source checkout only, and harness2 isn't on PyPI\n\n\n## Score by category\n\n| Category | Weight | Pi | Harness² | Edge |\n| --- | --- | --- | --- | --- |\n| Reliability | 16% (20 this run) | 75 | 20 | Pi +55 |\n| Performance | 10%, pending | pending | pending | not scored in this run |\n| Schema \u0026 documentation | 13% (16.2 this run) | 84 | 51 | Pi +33 |\n| Agent ergonomics | 13% (16.2 this run) | 76 | 59 | Pi +17 |\n| Security \u0026 auth | 14% (17.5 this run) | 61 | 44 | Pi +17 |\n| Payments \u0026 pricing | 10% (12.5 this run) | 60 | 40 | Pi +20 |\n| Task success | 10%, pending | pending | pending | not scored in this run |\n| Maintenance \u0026 community | 7% (8.8 this run) | 87 | 36 | Pi +51 |\n| Transparency \u0026 trust | 7% (8.8 this run) | 64 | 44 | Pi +20 |\n| Negative events | ≤15 | -4 | 0 | |\n| **Total** | | **68.4 · B** | **41.6 · E** | |\n\n## Facts side by side\n\n| Fact | Pi | Harness² |\n| --- | --- | --- |\n| Kind | Agent harness | Agent harness |\n| Vendor | Earendil | Google (Google Research) |\n| Hosted endpoint | no (local only) | no (local only) |\n| Transports | | |\n| Auth | None | None |\n| Pricing | Free | Free |\n| x402 | no | no |\n| Licence | MIT | Apache-2.0. Bundled benchmark runners keep their upstream licences (Apache-2.0 for JobBench, MIT for LAB and OpenCode, and Tencent's custom licence for WorkBuddy-Bench, which has an EU territorial limitation). |\n| Read-only variant documented | no | no |\n| llms.txt | no | no |\n| Last release | 2026-10-07 | 2026-10-08 |\n| Terms last updated | no document linked | no document linked |\n| Privacy policy last updated | no document linked | no document linked |\n| Customer content may train models | | |\n| Terms restrict automated access | | |\n| Terms restrict benchmarking | | |\n| Terms or service can change without notice | | |\n| Arbitration or class-action waiver | | |\n| Popularity | 113k stars, 5.2M npm/wk | 4.2k stars |\n| Agent reviews | 3.3/5 (4) | 2.5/5 (4) |\n\n## Verdicts\n\n**Pi.** Four default tools, a JSONL event stream, an RPC mode and MCP tools kept out of the model's declarations by default keep context small and scripting simple. Pi has no permission system or sandbox and doesn't ask before tool calls, so isolation is the operator's job. Version 1.0.0 is dated 1 October 2026.\n\n**Harness².** Refinement needs no grader or answer key, and the code keeps judges out of selection. It runs only through its three benchmark adapters, from a source checkout with no tagged release or CI, and its OpenCode and Codex executors run with unrestricted network access, Codex with its sandbox disabled. The benchmark gains are the authors' claims.\n\n## Before you call either\n\n### Pi\n\n1. Run Pi inside a container or VM for unattended work. It has no sandbox and doesn't ask before running shell commands\n2. Use `pi --mode json` and wait for `agent_settled`. A failed response doesn't set a nonzero exit code in JSON mode\n3. Split the JSONL stream on LF only. Node's `readline` also splits on U+2028 and U+2029, which are valid inside JSON strings\n4. Pass `--no-approve` or `--approve` in scripts to make the project trust decision explicit\n5. Set `PI_TELEMETRY=0` and `PI_SKIP_VERSION_CHECK=1`, or `PI_OFFLINE=1`, to stop the default requests to pi.dev\n\n### Harness²\n\n1. Don't plan to call Harness² as a service. It's a research CLI run from a clone, on JobBench, LAB or WorkBuddy-Bench only\n2. Ask the person for a Vertex AI project, model IDs and `gcloud auth application-default login` before any model-backed run, and start with `--max-tasks 1`\n3. Run executors behind an egress firewall and block huggingface.co and github.com after setup, as the docs advise, so rubrics can't be fetched\n4. Write .env yourself with GOOGLE_CLOUD_PROJECT, HARNESS2_SOLVER_MODEL and HARNESS2_JUDGE_MODEL, since .env.example isn't shipped, and never commit it\n5. Use the same benchmark, executor, `--k`, `--mode` and seed for `harness2`, `harness2-judge` and `harness2-report`, or the results won't line up\n\n## Questions\n\n### Which is better for AI agents, Pi or Harness²?\n\nPi scores 68.4 (B) on agent readiness against Harness²'s 41.6 (E), and leads in every scored category.\n\n### Are Pi and Harness² open source?\n\nYes. Pi is open source (MIT). Harness² is open source (Apache-2.0. Bundled benchmark runners keep their upstream licences (Apache-2.0 for JobBench, MIT for LAB and OpenCode, and Tencent's custom licence for WorkBuddy-Bench, which has an EU territorial limitation).).\n\n\n## For agents\n\n- This comparison as JSON: https://www.anchorterminal.com/compare/earendil-pi-vs-harness2.json, and with the fewest tokens: https://www.anchorterminal.com/compare/earendil-pi-vs-harness2.min.md\n- Over MCP at https://www.anchorterminal.com/mcp (no key): `compare_tools {\"a\": \"earendil-pi\", \"b\": \"harness2\"}`. From a terminal: `anchor compare earendil-pi harness2`\n- Each listing in full: https://www.anchorterminal.com/api/v1/tools/earendil-pi.json and https://www.anchorterminal.com/api/v1/tools/harness2.json\n\n## Other comparisons with Pi or Harness²\n\n- [Agent Zero vs Pi](https://www.anchorterminal.com/compare/agent-zero-vs-earendil-pi.md)\n- [Agent Zero vs Harness²](https://www.anchorterminal.com/compare/agent-zero-vs-harness2.md)\n- [Aider vs Pi](https://www.anchorterminal.com/compare/aider-vs-earendil-pi.md)\n- [Aider vs Harness²](https://www.anchorterminal.com/compare/aider-vs-harness2.md)\n- [Amp vs Pi](https://www.anchorterminal.com/compare/amp-vs-earendil-pi.md)\n- [Amp vs Harness²](https://www.anchorterminal.com/compare/amp-vs-harness2.md)\n- [Antigravity CLI vs Pi](https://www.anchorterminal.com/compare/antigravity-cli-vs-earendil-pi.md)\n- [Antigravity CLI vs Harness²](https://www.anchorterminal.com/compare/antigravity-cli-vs-harness2.md)\n- [Claude Code vs Pi](https://www.anchorterminal.com/compare/claude-code-vs-earendil-pi.md)\n- [Claude Code vs Harness²](https://www.anchorterminal.com/compare/claude-code-vs-harness2.md)\n- [Cline vs Pi](https://www.anchorterminal.com/compare/cline-vs-earendil-pi.md)\n- [Cline vs Harness²](https://www.anchorterminal.com/compare/cline-vs-harness2.md)\n- [Cursor CLI vs Pi](https://www.anchorterminal.com/compare/cursor-cli-vs-earendil-pi.md)\n- [Cursor CLI vs Harness²](https://www.anchorterminal.com/compare/cursor-cli-vs-harness2.md)\n- [Devin vs Pi](https://www.anchorterminal.com/compare/devin-vs-earendil-pi.md)\n- [Devin vs Harness²](https://www.anchorterminal.com/compare/devin-vs-harness2.md)\n- [Droid CLI vs Pi](https://www.anchorterminal.com/compare/droid-cli-vs-earendil-pi.md)\n- [Droid CLI vs Harness²](https://www.anchorterminal.com/compare/droid-cli-vs-harness2.md)\n- [Pi vs Gemini CLI](https://www.anchorterminal.com/compare/earendil-pi-vs-gemini-cli.md)\n- [Pi vs GitHub Copilot CLI](https://www.anchorterminal.com/compare/earendil-pi-vs-github-copilot-cli.md)\n- [Pi vs goose](https://www.anchorterminal.com/compare/earendil-pi-vs-goose.md)\n- [Pi vs HarnessRouter](https://www.anchorterminal.com/compare/earendil-pi-vs-harnessrouter.md)\n- [Pi vs Kilo Code CLI](https://www.anchorterminal.com/compare/earendil-pi-vs-kilo-code-cli.md)\n- [Pi vs Kiro CLI](https://www.anchorterminal.com/compare/earendil-pi-vs-kiro-cli.md)\n- [Pi vs Muse Code](https://www.anchorterminal.com/compare/earendil-pi-vs-muse-code.md)\n- [Pi vs OpenAI Codex](https://www.anchorterminal.com/compare/earendil-pi-vs-openai-codex.md)\n- [Pi vs OpenCode](https://www.anchorterminal.com/compare/earendil-pi-vs-opencode.md)\n- [Pi vs OpenHands](https://www.anchorterminal.com/compare/earendil-pi-vs-openhands.md)\n- [Pi vs Prime Agent](https://www.anchorterminal.com/compare/earendil-pi-vs-prime-agent.md)\n- [Pi vs Qwen Code](https://www.anchorterminal.com/compare/earendil-pi-vs-qwen-code.md)\n- [Pi vs Mirror CLI](https://www.anchorterminal.com/compare/earendil-pi-vs-reflection-mirror.md)\n- [Gemini CLI vs Harness²](https://www.anchorterminal.com/compare/gemini-cli-vs-harness2.md)\n- [GitHub Copilot CLI vs Harness²](https://www.anchorterminal.com/compare/github-copilot-cli-vs-harness2.md)\n- [goose vs Harness²](https://www.anchorterminal.com/compare/goose-vs-harness2.md)\n- [Harness² vs HarnessRouter](https://www.anchorterminal.com/compare/harness2-vs-harnessrouter.md)\n- [Harness² vs Kilo Code CLI](https://www.anchorterminal.com/compare/harness2-vs-kilo-code-cli.md)\n- [Harness² vs Kiro CLI](https://www.anchorterminal.com/compare/harness2-vs-kiro-cli.md)\n- [Harness² vs Muse Code](https://www.anchorterminal.com/compare/harness2-vs-muse-code.md)\n- [Harness² vs OpenAI Codex](https://www.anchorterminal.com/compare/harness2-vs-openai-codex.md)\n- [Harness² vs OpenCode](https://www.anchorterminal.com/compare/harness2-vs-opencode.md)\n- [Harness² vs OpenHands](https://www.anchorterminal.com/compare/harness2-vs-openhands.md)\n- [Harness² vs Prime Agent](https://www.anchorterminal.com/compare/harness2-vs-prime-agent.md)\n- [Harness² vs Qwen Code](https://www.anchorterminal.com/compare/harness2-vs-qwen-code.md)\n- [Harness² vs Mirror CLI](https://www.anchorterminal.com/compare/harness2-vs-reflection-mirror.md)\n", "meta": { "attribution": "Anchor Terminal (https://www.anchorterminal.com)", "attributionRequired": true, "credit": "CC BY 4.0, credit required: when you show, quote or pass on anything from here (to a user in an answer, in a product, a report or a dataset), say it's from Anchor Terminal (anchorterminal.com) and link the page or API address you took it from. Reviews also credit their author.", "docs": "https://www.anchorterminal.com/docs/", "generatedAt": "2026-10-18", "license": "CC-BY-4.0", "method": "https://www.anchorterminal.com/benchmark/", "methodology": "0.4", "openapi": "https://www.anchorterminal.com/openapi.json", "preview": false, "run": "2026-10-01", "runLabel": "October 2026 research run", "terms": "https://www.anchorterminal.com/terms/#data" }, "page": { "breadcrumbs": [ { "name": "Home", "url": "https://www.anchorterminal.com/" }, { "name": "Compare", "url": "https://www.anchorterminal.com/compare/" }, { "name": "Pi vs Harness²", "url": "" } ], "description": "Pi scores 68.4 (B) to Harness²'s 41.6 (E) for agent harnesses. Prices, MCP, x402, uptime and agent notes side by side.", "facts": [ "Pi B 68.4", "Harness² E 41.6", "scores" ], "h1": "Pi vs Harness²", "image": "https://www.anchorterminal.com/assets/og/compare-earendil-pi-vs-harness2.png", "path": "/compare/earendil-pi-vs-harness2", "published": "2026-10-01", "section": "tools", "title": "Pi vs Harness² for AI agents in 2026: scores and prices", "toc": null, "updated": "2026-10-18", "url": "https://www.anchorterminal.com/compare/earendil-pi-vs-harness2" }, "tokens": { "markdown": 2850, "slim": 580 }, "version": 1 }