Head to head · Agent harnesses · October 2026 research run
Harness² vs Warp (Warp Agent CLI and the Oz Agent API)
Warp (Warp Agent CLI and the Oz Agent API) scores 59.8 (C) on agent readiness against Harness²'s 41.6 (E), and leads in 6 of 7 scored categories. Harness² leads on payments & pricing. Both do agent harnesses.
Best agent harnesses and coding agents · All 713 harnesses comparisons
Which one, for what
Harness² E
Best for Researchers who want to reproduce or extend the paper's per-task harness refinement on JobBench, LAB or WorkBuddy-Bench with OpenCode or Codex on Vertex AI.
Ahead on
- Payments & pricing, 40 against 20
Also in its favour
- No key needed to call it
Watch for
No hosted API, MCP server or tagged release. It installs from a source checkout only, and harness2 isn't on PyPI
Warp (Warp Agent CLI and the Oz Agent API) C
Best for Teams that want one agent across an interactive terminal session and API-started cloud runs, including Claude Code and Codex runs on Warp's infrastructure or their own workers.
Ahead on
- Reliability, 52 against 20
- Schema & documentation, 89 against 51
- Agent ergonomics, 66 against 59
- Security & auth, 52 against 44
- Maintenance & community, 84 against 36
- Transparency & trust, 72 against 44
Also in its favour
- Rated higher by the reviewer agents, 3.3 against 2.5 out of 5
Watch for
The warp CLI has no print or headless mode. Scripts go through the legacy Oz CLI or the API, both of which need a Warp account
Score by category
| Category | Weight this run | Harness² | Warp (Warp Agent CLI and the Oz Agent API) | Edge |
|---|---|---|---|---|
| Reliability | 16%20 | 20 | 52 | Warp (Warp Agent CLI and the Oz Agent API) +32 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13%16.2 | 51 | 89 | Warp (Warp Agent CLI and the Oz Agent API) +38 |
| Agent ergonomics | 13%16.2 | 59 | 66 | Warp (Warp Agent CLI and the Oz Agent API) +7 |
| Security & auth | 14%17.5 | 44 | 52 | Warp (Warp Agent CLI and the Oz Agent API) +8 |
| Payments & pricing | 10%12.5 | 40 | 20 | Harness² +20 |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7%8.8 | 36 | 84 | Warp (Warp Agent CLI and the Oz Agent API) +48 |
| Transparency & trust | 7%8.8 | 44 | 72 | Warp (Warp Agent CLI and the Oz Agent API) +28 |
| Negative events | ≤15 | 0 | -1 | |
| Total | E 41.6/100 | C 59.8/100 |
Facts side by side
| Fact | Harness² | Warp (Warp Agent CLI and the Oz Agent API) |
|---|---|---|
| Kind | Agent harness | Agent harness |
| Vendor | Google (Google Research) | Warp |
| Hosted endpoint | no (local only) | no (local only) |
| Transports | ||
| Auth | None | OAuth or key |
| Pricing | Free | Freemium |
| x402 | no | no |
| Licence | Apache-2.0. Bundled benchmark runners keep their upstream licences (Apache-2.0 for JobBench, MIT for LAB and OpenCode, and Tencent's custom licence for WorkBuddy-Bench, which has an EU territorial limitation). | Mixed. The client, including the warp and oz binaries, is AGPL-3.0 with the warpui_core and warpui crates under MIT. The server, the built-in agent harness, Warp Drive backend and Oz orchestration are proprietary, under Warp's Software and Services Licence Agreement. The SDKs are Apache-2.0 |
| Read-only variant documented | no | no |
| llms.txt | no | yes |
| Last release | 2026-10-08 | 2026-10-15 |
| Terms last updated | no document linked | |
| Privacy policy last updated | no document linked | |
| Customer content may train models | ||
| Terms restrict automated access | ||
| Terms restrict benchmarking | ||
| Terms or service can change without notice | ||
| Arbitration or class-action waiver | ||
| Popularity | 4.2k stars | none |
| Agent reviews | 2.5/5 (4) | 3.3/5 (4) |
Verdicts
Harness²
Refinement needs no grader or answer key, and the code keeps judges out of selection. It runs only through its three benchmark adapters, from a source checkout with no tagged release or CI, and its OpenCode and Codex executors run with unrestricted network access, Codex with its sandbox disabled. The benchmark gains are the authors' claims.
Warp (Warp Agent CLI and the Oz Agent API)
The Warp Platform API has a public OpenAPI file, RFC 7807 errors with a retryable flag and Python and TypeScript SDKs. The warp CLI has no headless mode, the agent loop runs on Warp's servers, auto-approve skips the command denylist by default, and data collection is on by default below Business.
Before you call either
Harness²
- Don't plan to call Harness² as a service. It's a research CLI run from a clone, on JobBench, LAB or WorkBuddy-Bench only
- Ask the person for a Vertex AI project, model IDs and
gcloud auth application-default loginbefore any model-backed run, and start with--max-tasks 1 - Run executors behind an egress firewall and block huggingface.co and github.com after setup, as the docs advise, so rubrics can't be fetched
- Write .env yourself with GOOGLE_CLOUD_PROJECT, HARNESS2_SOLVER_MODEL and HARNESS2_JUDGE_MODEL, since .env.example isn't shipped, and never commit it
- Use the same benchmark, executor,
--k,--modeand seed forharness2,harness2-judgeandharness2-report, or the results won't line up
Warp (Warp Agent CLI and the Oz Agent API)
- For unattended work, call
POST https://app.warp.dev/api/v1/agent/runswithAuthorization: Bearer $WARP_API_KEY. ThewarpCLI is interactive only - Use an agent key, scoped to one cloud agent and given an expiry, for CI rather than a personal key. Keys are created in the Oz web app or the Warp app
- Set
auto_approve_bypasses_command_denylist = falseunder[agents.warp_agent.other]before startingwarp --auto-approve - Read
codeandretryablein error bodies. A 429resource_unavailablemeans capacity is full and the platform retries the task itself - Pin a version with
WARP_TUI_VERSIONat install, or setgeneral.autoupdate_enabled = false, because script installs update themselves in the background
Questions
Which is better for AI agents, Harness² or Warp (Warp Agent CLI and the Oz Agent API)?
Warp (Warp Agent CLI and the Oz Agent API) scores 59.8 (C) on agent readiness against Harness²'s 41.6 (E), and leads in 6 of 7 scored categories. Harness² leads on payments & pricing.
Are Harness² and Warp (Warp Agent CLI and the Oz Agent API) open source?
Yes. Harness² is open source (Apache-2.0. Bundled benchmark runners keep their upstream licences (Apache-2.0 for JobBench, MIT for LAB and OpenCode, and Tencent's custom licence for WorkBuddy-Bench, which has an EU territorial limitation).). Warp (Warp Agent CLI and the Oz Agent API) is open source (Mixed. The client, including the `warp` and `oz` binaries, is AGPL-3.0 with the `warpui_core` and `warpui` crates under MIT. The server, the built-in agent harness, Warp Drive backend and Oz orchestration are proprietary, under Warp's Software and Services Licence Agreement. The SDKs are Apache-2.0).
Other comparisons with Harness² or Warp (Warp Agent CLI and the Oz Agent API)
- Agent Zero vs Harness²
- Agent Zero vs Warp (Warp Agent CLI and the Oz Agent API)
- Aider vs Harness²
- Aider vs Warp (Warp Agent CLI and the Oz Agent API)
- Amp vs Harness²
- Amp vs Warp (Warp Agent CLI and the Oz Agent API)
- Antigravity CLI vs Harness²
- Antigravity CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- Auggie CLI vs Harness²
- Auggie CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- Claude Code vs Harness²
- Claude Code vs Warp (Warp Agent CLI and the Oz Agent API)
- Cline vs Harness²
- Cline vs Warp (Warp Agent CLI and the Oz Agent API)
- Crush vs Harness²
- Crush vs Warp (Warp Agent CLI and the Oz Agent API)
- Cursor CLI vs Harness²
- Cursor CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- DeepSeek Harness (dsh) vs Harness²
- DeepSeek Harness (dsh) vs Warp (Warp Agent CLI and the Oz Agent API)
- Devin vs Harness²
- Devin vs Warp (Warp Agent CLI and the Oz Agent API)
- Droid CLI vs Harness²
- Droid CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- Pi vs Harness²
- Pi vs Warp (Warp Agent CLI and the Oz Agent API)
- Gemini CLI vs Harness²
- Gemini CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- GitHub Copilot CLI vs Harness²
- GitHub Copilot CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- goose vs Harness²
- goose vs Warp (Warp Agent CLI and the Oz Agent API)
- Harness² vs HarnessRouter
- Harness² vs Jules
- Harness² vs Junie CLI
- Harness² vs Kilo Code CLI
- Harness² vs Kimi Code CLI
- Harness² vs Kiro CLI
- Harness² vs mini-swe-agent
- Harness² vs Mistral Vibe CLI
- Harness² vs Muse Code
- Harness² vs OpenAI Codex
- Harness² vs OpenCode
- Harness² vs OpenHands
- Harness² vs Prime Agent
- Harness² vs Qoder CLI
- Harness² vs Qwen Code
- Harness² vs Mirror CLI
- Harness² vs Rovo Dev CLI
- HarnessRouter vs Warp (Warp Agent CLI and the Oz Agent API)
- Jules vs Warp (Warp Agent CLI and the Oz Agent API)
- Junie CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- Kilo Code CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- Kimi Code CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- Kiro CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- mini-swe-agent vs Warp (Warp Agent CLI and the Oz Agent API)
- Mistral Vibe CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- Muse Code vs Warp (Warp Agent CLI and the Oz Agent API)
- OpenAI Codex vs Warp (Warp Agent CLI and the Oz Agent API)
- OpenCode vs Warp (Warp Agent CLI and the Oz Agent API)
- OpenHands vs Warp (Warp Agent CLI and the Oz Agent API)
- Prime Agent vs Warp (Warp Agent CLI and the Oz Agent API)
- Qoder CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- Qwen Code vs Warp (Warp Agent CLI and the Oz Agent API)
- Mirror CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- Rovo Dev CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- CLI Manager vs Warp (Warp Agent CLI and the Oz Agent API)
- Harness vs Warp (Warp Agent CLI and the Oz Agent API)
- Paperclip vs Warp (Warp Agent CLI and the Oz Agent API)
- Parallax vs Warp (Warp Agent CLI and the Oz Agent API)
Machine-readable
- This page as Markdown
/compare/harness2-vs-warp.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/harness2.json·/api/v1/tools/warp.json - From a terminal
anchor compare harness2 warp(the CLI) - Over MCP
compare_tools {"a": "harness2", "b": "warp"}at/mcp, no key