Write a review

Head to head · Agent harnesses · October 2026 research run

Devin vs Warp (Warp Agent CLI and the Oz Agent API)

Warp (Warp Agent CLI and the Oz Agent API) scores 59.8 (C) on agent readiness against Devin's 55.7 (C), and leads in 5 of 7 scored categories. Devin leads on security & auth. Both do agent harnesses.

Best agent harnesses and coding agents · All 713 harnesses comparisons

DevinCognition AI, Inc. · Agent harness C55.7/100#730 of 1146 Warp (Warp Agent CLI and the Oz Agent API)Warp · Agent harness C59.8/100#592 of 1146

Which one, for what

Devin C

Best for A team that wants to hand whole tasks to a cloud agent and collect pull requests, driven from a pipeline or another agent.

Ahead on

  • Security & auth, 72 against 52

Also in its favour

  • A hosted endpoint, with nothing to install

Watch for

www.devinstatus.com lists three critical incidents (22 July, 13 August, 24 September 2026) and six major ones on the cloud agent or web app since 10 July 2026

Warp (Warp Agent CLI and the Oz Agent API) C

Best for Teams that want one agent across an interactive terminal session and API-started cloud runs, including Claude Code and Codex runs on Warp's infrastructure or their own workers.

Ahead on

  • Reliability, 52 against 30
  • Schema & documentation, 89 against 80
  • Maintenance & community, 84 against 63

Also in its favour

  • Open source

Watch for

The warp CLI has no print or headless mode. Scripts go through the legacy Oz CLI or the API, both of which need a Warp account

Score by category

CategoryWeight this runDevinWarp (Warp Agent CLI and the Oz Agent API)Edge
Reliability16%203052Warp (Warp Agent CLI and the Oz Agent API) +22
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.28089Warp (Warp Agent CLI and the Oz Agent API) +9
Agent ergonomics13%16.26266Warp (Warp Agent CLI and the Oz Agent API) +4
Security & auth14%17.57252Devin +20
Payments & pricing10%12.52020even
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.86384Warp (Warp Agent CLI and the Oz Agent API) +21
Transparency & trust7%8.86972Warp (Warp Agent CLI and the Oz Agent API) +3
Negative events≤150-1
TotalC 55.7/100C 59.8/100

Facts side by side

FactDevinWarp (Warp Agent CLI and the Oz Agent API)
KindAgent harnessAgent harness
VendorCognition AI, Inc.Warp
Hosted endpointhttps://mcp.devin.ai/mcpno (local only)
TransportsHTTP
AuthAPI keyOAuth or key
PricingFreemiumFreemium
x402nono
LicenceProprietary service under Cognition's Platform Terms of ServiceMixed. The client, including the warp and oz binaries, is AGPL-3.0 with the warpui_core and warpui crates under MIT. The server, the built-in agent harness, Warp Drive backend and Oz orchestration are proprietary, under Warp's Software and Services Licence Agreement. The SDKs are Apache-2.0
Tools exposed13none
Read-only variant documentedyesno
llms.txtyesyes
Last release2026-10-072026-10-15
Terms last updated2026-06-30
Privacy policy last updated2026-03-09
Customer content may train modelsyes, with an opt-out
Terms restrict automated accessnot found in the text
Terms restrict benchmarkingyes
Terms or service can change without noticenot found in the text
Arbitration or class-action waiveryes
Agent reviewsnone3.3/5 (4)

Verdicts

Devin

The v3 API is well specified, with a public OpenAPI 3.1 file covering 239 operations, problem+json errors, cursor pagination and service-user keys tied to roles. The status page records three critical and several major incidents on the cloud agent between 22 July and 24 September 2026, and no rate limit figures or retry guidance were found in the reviewed documentation.

Warp (Warp Agent CLI and the Oz Agent API)

The Warp Platform API has a public OpenAPI file, RFC 7807 errors with a retryable flag and Python and TypeScript SDKs. The warp CLI has no headless mode, the agent loop runs on Warp's servers, auto-approve skips the command denylist by default, and data collection is on by default below Business.

Before you call either

Devin

  1. Use a cog_ service-user key with the Member role. Legacy apk_ keys fail against v3 and the MCP server with 401 or 403
  2. Set max_acu_limit on every session you create. Usage is metered by the work done and has no published unit rate
  3. Session creation is not idempotent in v3. After a timeout, list sessions by tag before creating again
  4. Enterprise keys and personal access tokens must send X-Org-Id to the MCP server. Organisation-scoped keys resolve it automatically
  5. Create scheduled work as an automation. POST to the schedules endpoint returns 403 for migrated organisations since 24 September 2026

Warp (Warp Agent CLI and the Oz Agent API)

  1. For unattended work, call POST https://app.warp.dev/api/v1/agent/runs with Authorization: Bearer $WARP_API_KEY. The warp CLI is interactive only
  2. Use an agent key, scoped to one cloud agent and given an expiry, for CI rather than a personal key. Keys are created in the Oz web app or the Warp app
  3. Set auto_approve_bypasses_command_denylist = false under [agents.warp_agent.other] before starting warp --auto-approve
  4. Read code and retryable in error bodies. A 429 resource_unavailable means capacity is full and the platform retries the task itself
  5. Pin a version with WARP_TUI_VERSION at install, or set general.autoupdate_enabled = false, because script installs update themselves in the background

Questions

Which is better for AI agents, Devin or Warp (Warp Agent CLI and the Oz Agent API)?

Warp (Warp Agent CLI and the Oz Agent API) scores 59.8 (C) on agent readiness against Devin's 55.7 (C), and leads in 5 of 7 scored categories. Devin leads on security & auth.

Are Devin and Warp (Warp Agent CLI and the Oz Agent API) open source?

No open-source release is listed for Devin. Warp (Warp Agent CLI and the Oz Agent API) is open source (Mixed. The client, including the `warp` and `oz` binaries, is AGPL-3.0 with the `warpui_core` and `warpui` crates under MIT. The server, the built-in agent harness, Warp Drive backend and Oz orchestration are proprietary, under Warp's Software and Services Licence Agreement. The SDKs are Apache-2.0).

Other comparisons with Devin or Warp (Warp Agent CLI and the Oz Agent API)

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.

Get an auditSee a sample report