Head to head · Agent harnesses · October 2026 research run
Devin vs OpenAI Agents API
OpenAI Agents API scores 73.3 (BB) on agent readiness against Devin's 57.3 (C), and leads in every scored category. Both do agent harnesses.
Best agent harnesses and coding agents · All 1041 harnesses comparisons
Which one, for what
Devin C
Best for A team that wants to hand whole tasks to a cloud agent and collect pull requests, driven from a pipeline or another agent.
No category where it leads by five points or more, and no fact that sets it apart.
Watch for
www.devinstatus.com lists two outages on 13 August and the web app down for about two hours on 24 September 2026
Best for Teams that want OpenAI to run the agent loop, with sandboxes, MCP tools and subagents, and are fine with a beta API, US-only data residency and no Zero Data Retention.
Ahead on
- Reliability, 60 against 35
- Schema & documentation, 88 against 80
- Agent ergonomics, 80 against 62
- Security & auth, 87 against 72
- Payments & pricing, 30 against 20
- Maintenance & community, 87 against 63
- Transparency & trust, 85 against 75
Also in its favour
- Agent-ready, a grade of BB or better
Watch for
Public beta since 10 September 2026 behind the agents=v1 beta header, with no GA date
Why the edge
Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.
Reliability OpenAI Agents API +25
Devin 35
Hosted reading. Status page at www.devinstatus.com (Atlassian Statuspage) with 14 components, among them Cloud Agent and Cloud Web Client, read from the page itself on 4 November 2026 (20). Its history feed for the 90 days from 6 August 2026 lists Devin unavailable on 13 August (20 minutes), Devin Cloud unavailable on 13 August (2 hours 40 minutes) and the web app down on 24 September (about 2 hours), with minor session and Ask Devin incidents on 10 and 11 August, 3 and 17 September and 26 October. Both 13 August entries report the service unavailable. The 24 September entry names only the web app, not the API or the MCP server, so it is read as partial, as dashboard outages are on other listings. Two majors (5). Two scheduled maintenance entries (7 and 17 August) are read as maintenance. 429 is a documented response, but no rate limit figures were found (0). No Retry-After or backoff guidance found, and v3 session creation has no idempotency key (0). The platform terms give no guarantee of availability, and enterprise terms weren't read (0). The v3 API is the current, recommended surface, with 27 of 274 operations under /v3beta1 (10).
OpenAI Agents API 60
Read as a hosted API. status.openai.com has an Agents component under APIs (20). Since release on 10 September (57 days, not 90), the feed names the Agents API in a degraded-performance incident on 14 September, the elevated errors across ChatGPT, Codex and the API on 29 September (the openai-api listing reads that one as a major, 17:52 to 23:14 UTC), computer use issues and degraded workspace agents on 6 and 7 October, degraded performance across the APIs for 22 minutes on 21 October, and overbilling for hosted containers on 19 September. One major, the rest minor (10). Rate limits come from the shared guide, published per tier and model with headers, and the errors page adds a Files API allowance on session creation (15). 429 and 503 guidance with Retry-After, capped backoff and a check-then-retry procedure for turns (15). The Scale Tier SLA in the openai-api listing isn't shown to cover a beta API (0). Public beta behind the agents=v1 header (0). 20 + 10 + 15 + 15 + 0 + 0 = 60.
Maintenance & community OpenAI Agents API +24
Devin 63
Closed service. Newest release note dated 2 November 2026 (30). Thirty dated release notes between 7 August and 2 November 2026, and API notes for August, September, October and November (20). Public release notes and a support address, with no public tracker or forum found (10). No official SDK for the Devin API found (a Python SDK in the federal docs covers analytics and ACU limits for air-gapped Desktop), and the Devin MCP server didn't appear in the official MCP registry per the 8 October check (0). No public CI or packages to assess, with the docs and three OpenAPI specs kept current (3).
OpenAI Agents API 87
Closed API. Last dated change 16 October 2026, Ultrafast mode and document skills (30). Four dated changelog entries in the 57 days since release, 10 September, 29 September, 9 October and 16 October (20). Changelog and help centre, as the openai-api listing reads them (12 of 15). Official SDKs carry beta.agents (15). SDK CI, per the openai-api listing (10). 30 + 20 + 12 + 15 + 10 = 87.
Agent ergonomics OpenAI Agents API +18
Devin 62
API and MCP reading. The MCP server documents 13 tools, several of them consolidated manage tools, and list endpoints take first to size a page (17). Cursor pagination on every v3 list endpoint and filters on session search by tag, playbook, origin, user and time (18). problem+json with error_code, field-level errors on 422 and a 400 that lists valid platform names, but no published code list (15). No idempotency key on v3 session creation. max_acu_limit caps what a session can spend, and the MCP tool annotations couldn't be read without a key (4). Only prompt is required to start a session. No official SDK found, only a Terraform provider (8).
OpenAI Agents API 80
Read as an API an agent drives. Context is compacted automatically, and session budgets and streamed events keep output sized (20 of 25). Sessions can be listed, continued and steered, events streamed or sent by webhook, artifacts fetched (15 of 20). Actionable error codes with recovery steps (20). No idempotency key; a documented check-then-retry procedure (10 of 20). One session call with defaults (medium sandbox, network enabled), SDKs in five languages (15). 20 + 15 + 20 + 10 + 15 = 80.
Security & auth OpenAI Agents API +15
Devin 72
Hosted reading. Service-user keys with roles and named permissions per endpoint, revocable, with rotation endpoints in beta for Enterprise and expiring personal tokens. Custom roles are an Enterprise option, and Teams has Admin and Member only (25). Security profiles restrict network, MCP servers, git and the Devin MCP to read-only, and read-only API permissions exist. They are opt-in, and session creation accepts bypass_approval (16). AI Guardrails screen user messages and pull request comments for prompt injection, on Enterprise only, and they don't cover web pages or repository content Devin reads (8). Audit logs through the API need an enterprise-level permission. Session events and insights are available on all plans (10). SOC 2 Type II and ISO/IEC 27001:2022, a disclosure address at security@cognition.ai, no security.txt, and the trust centre answers No to having a bug bounty (13).
OpenAI Agents API 87
Project keys with restricted per-endpoint permissions, service accounts and workload identity, per the openai-api listing; we didn't confirm a key can be limited to /v1/agents (27 of 30). Automatic review for approvals, a sandbox network restricted to a host allowlist, vaults for secrets and a sandbox security guide (17 of 20). The agent reads web pages, MCP results and browser content; the sandbox security guide and approval flow are the mitigations, with no prompt-injection guide of its own found (8 of 15). Tracing, observability and usage per session and admin audit logs (15). SOC 2 Type 2, ISO 27001 family, Bugcrowd bounty and a disclosure policy, per the openai-api listing (20). Not ZDR-eligible and US residency only, recorded under transparency. 27 + 17 + 8 + 15 + 20 = 87.
Payments & pricing OpenAI Agents API +10
Devin 20
Published rubric. No payment protocol (0). Plan prices are public, $20, $200 and $40 a seat with an $80 team minimum, but usage past the quota is billed at an unpublished rate (10). A Free plan exists. The pricing page lists cloud agents from Pro upward, and we couldn't confirm API use on Free or whether a card is needed (10). A person signs up in a browser and provisions the service user in settings (0).
OpenAI Agents API 30
No machine payment protocol (0). Model, tool and container prices published without a login, and no fee for the API itself (20). The Free tier reading of the openai-api listing, half (10). Sign-up in a browser (0). 0 + 20 + 10 + 0 = 30.
Transparency & trust OpenAI Agents API +10
Devin 75
Editorial half. Closed source with clear terms from Cognition AI, Inc. (15). Terms (30 June 2026), privacy policy, security page and data processing agreement agree with each other. Customer data may be used for model training by default on self-serve plans, with an opt-out on paid plans that also turns on zero data retention at model providers, and Enterprise data isn't trained on without written consent. Retention is stated as the length of the customer relationship, with no periods (18). The API versioning policy on the release notes page says breaking changes are announced at least 7 days in advance and a deprecated endpoint stays available through a deprecation period. Dated notices followed it for the schedules endpoints (announced 10 September, creation refused from 24 September 2026), and others cover the v2 clone endpoint and Playbooks moving to Skills (23 October). v1 and v2 are marked legacy with no end date (20). Sub-processors listed with purpose and country, and 15 days' notice of additions, per the 8 October check (18).
OpenAI Agents API 85
Closed service with clear terms, Apache-2.0 SDKs (15). Session state retained until deleted, deletable sessions and artifacts, not ZDR-eligible, US residency only, and the data-controls guide agrees; no retention period for artifacts after a sandbox expires (25 of 30). Dated deprecation notices with minimum periods for GA models; a beta API can change faster (15 of 20). Subprocessors and processing countries listed, per the openai-api listing (20). 15 + 25 + 15 + 20 = 75.
Score by category
| Category | Weight this run | Devin | OpenAI Agents API | Edge |
|---|---|---|---|---|
| Reliability | 16%20 | 35 | 60 | OpenAI Agents API +25 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13%16.2 | 80 | 88 | OpenAI Agents API +8 |
| Agent ergonomics | 13%16.2 | 62 | 80 | OpenAI Agents API +18 |
| Security & auth | 14%17.5 | 72 | 87 | OpenAI Agents API +15 |
| Payments & pricing | 10%12.5 | 20 | 30 | OpenAI Agents API +10 |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7%8.8 | 63 | 87 | OpenAI Agents API +24 |
| Transparency & trust | 7%8.8 | 75 | 85 | OpenAI Agents API +10 |
| Negative events | ≤15 | 0 | 0 | |
| Total | C 57.3/100 | BB 73.3/100 |
Facts side by side
| Fact | Devin | OpenAI Agents API |
|---|---|---|
| Kind | Agent harness | HTTP API |
| Vendor | Cognition AI, Inc. | OpenAI |
| Hosted endpoint | https://mcp.devin.ai/mcp | https://api.openai.com/v1/agents/sessions |
| Transports | HTTP | https |
| Auth | API key | API key |
| Pricing | Freemium | Pay per use |
| x402 | no | no |
| Licence | Proprietary service under Cognition's Platform Terms of Service | Proprietary hosted API under OpenAI's Services Agreement. The SDKs are Apache 2.0 |
| Tools exposed | 13 | none |
| Read-only variant documented | yes | no |
| llms.txt | yes | yes |
| Last release | 2026-11-02 | 2026-10-16 |
| Terms last updated | 2026-06-30 | couldn't be read |
| Privacy policy last updated | 2026-03-09 | couldn't be read |
| Customer content may train models | yes, with an opt-out | couldn't be read |
| Terms restrict automated access | not found in the text | couldn't be read |
| Terms restrict benchmarking | yes | couldn't be read |
| Terms or service can change without notice | not found in the text | couldn't be read |
| Arbitration or class-action waiver | yes | couldn't be read |
How the scores have moved
OpenAI Agents API
No score change since it was listed.
Verdicts
Devin
The v3 API is well specified, with a public OpenAPI 3.1 file covering 274 operations, problem+json errors, cursor pagination and service-user keys tied to roles. The status page records two outages on 13 August and a two-hour web app outage on 24 September 2026, and no rate limit figures or retry guidance were found in the reviewed documentation.
OpenAI Agents API
OpenAI's hosted agent runtime, the Codex harness behind one session call, with MCP, functions, web search, computer use, subagents and sandboxes on OpenAI or your own infrastructure. Documentation, errors and SDKs are thorough. It's still public beta, can't use Zero Data Retention and keeps data in the United States only.
Before you call either
Devin
- Use a
cog_service-user key with the Member role. Legacyapk_keys fail against v3 and the MCP server with 401 or 403 - Set
max_acu_limiton every session you create. Usage is metered by the work done and has no published unit rate - Session creation is not idempotent in v3. After a timeout, list sessions by tag before creating again
- Enterprise keys and personal access tokens must send
X-Org-Idto the MCP server. Organisation-scoped keys resolve it automatically - Create scheduled work as an automation. POST to the schedules endpoint returns 403 for migrated organisations since 24 September 2026
OpenAI Agents API
- POST https://api.openai.com/v1/agents/sessions with a project key and the header OpenAI-Beta: agents=v1, or use client.beta.agents in the SDKs
- Before retrying, retrieve the session and turn and check which actions completed, since there's no idempotency key
- Set the sandbox network to restricted with an allowlist of up to 100 hosts when the agent doesn't need the open web
- Save files from /workspace/outputs before deleting a session; no retention period is stated for artifacts
- Don't send data that needs Zero Data Retention or non-US residency; the Agents API supports neither
Questions
Which is better for AI agents, Devin or OpenAI Agents API?
OpenAI Agents API scores 73.3 (BB) on agent readiness against Devin's 57.3 (C), and leads in every scored category.
Can an agent call Devin and OpenAI Agents API without installing anything?
Yes. Devin has a hosted endpoint at https://mcp.devin.ai/mcp and OpenAI Agents API at https://api.openai.com/v1/agents/sessions.
Other comparisons with Devin or OpenAI Agents API
- Agent Zero vs Devin
- Agent Zero vs OpenAI Agents API
- Aider vs Devin
- Aider vs OpenAI Agents API
- Amp vs Devin
- Amp vs OpenAI Agents API
- Antigravity CLI vs Devin
- Antigravity CLI vs OpenAI Agents API
- Auggie CLI vs Devin
- Auggie CLI vs OpenAI Agents API
- AXLR vs Devin
- AXLR vs OpenAI Agents API
- Claude Code vs Devin
- Claude Code vs OpenAI Agents API
- Cline vs Devin
- Cline vs OpenAI Agents API
- Crush vs Devin
- Crush vs OpenAI Agents API
- Cursor CLI vs Devin
- Cursor CLI vs OpenAI Agents API
- DeepSeek Harness (dsh) vs Devin
- DeepSeek Harness (dsh) vs OpenAI Agents API
- dsh-superagent (DeepSeek) vs Devin
- dsh-superagent (DeepSeek) vs OpenAI Agents API
- Devin vs Droid CLI
- Devin vs Pi
- Devin vs Gemini CLI
- Devin vs GitHub Copilot CLI
- Devin vs goose
- Devin vs Harness²
- Devin vs HarnessRouter
- Devin vs Jules
- Devin vs Junie CLI
- Devin vs Kilo Code CLI
- Devin vs Kimi Code CLI
- Devin vs Kiro CLI
- Devin vs mini-swe-agent
- Devin vs MiniMax Code
- Devin vs Mistral Vibe CLI
- Devin vs Muse Code
- Devin vs OpenAI Codex
- Devin vs OpenCode
- Devin vs OpenHands
- Devin vs OpenMausBot
- Devin vs Prime Agent
- Devin vs Qoder CLI
- Devin vs Qwen Code
- Devin vs Mirror CLI
- Devin vs Rovo Dev CLI
- Devin vs Unstoppable
- Devin vs Vibe-Trading
- Devin vs Warp (Warp Agent CLI and the Oz Agent API)
- Droid CLI vs OpenAI Agents API
- Pi vs OpenAI Agents API
- Gemini CLI vs OpenAI Agents API
- GitHub Copilot CLI vs OpenAI Agents API
- goose vs OpenAI Agents API
- Harness² vs OpenAI Agents API
- HarnessRouter vs OpenAI Agents API
- Jules vs OpenAI Agents API
- Junie CLI vs OpenAI Agents API
- Kilo Code CLI vs OpenAI Agents API
- Kimi Code CLI vs OpenAI Agents API
- Kiro CLI vs OpenAI Agents API
- mini-swe-agent vs OpenAI Agents API
- MiniMax Code vs OpenAI Agents API
- Mistral Vibe CLI vs OpenAI Agents API
- Muse Code vs OpenAI Agents API
- OpenAI Agents API vs OpenAI Codex
- OpenAI Agents API vs OpenCode
- OpenAI Agents API vs OpenHands
- OpenAI Agents API vs OpenMausBot
- OpenAI Agents API vs Prime Agent
- OpenAI Agents API vs Qoder CLI
- OpenAI Agents API vs Qwen Code
- OpenAI Agents API vs Mirror CLI
- OpenAI Agents API vs Rovo Dev CLI
- OpenAI Agents API vs Unstoppable
- OpenAI Agents API vs Vibe-Trading
- OpenAI Agents API vs Warp (Warp Agent CLI and the Oz Agent API)
- Clawboo vs Devin
- CLI Manager vs Devin
- Devin vs Harness
- Devin vs Paperclip
- Devin vs Parallax
Machine-readable
- This page as Markdown
/compare/devin-vs-openai-agents-api.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/devin.json·/api/v1/tools/openai-agents-api.json - From a terminal
anchor compare devin openai-agents-api(the CLI) - Over MCP
compare_tools {"a": "devin", "b": "openai-agents-api"}at/mcp, no key