Head to head · Agent harnesses · October 2026 research run

OpenAI Agents API vs Mirror CLI

OpenAI Agents API scores 73.3 (BB) on agent readiness against Mirror CLI's 57.2 (C), and leads in 6 of 7 scored categories. Mirror CLI leads on payments & pricing. Both do agent harnesses.

Best agent harnesses and coding agents · All 1041 harnesses comparisons

Which one, for what

OpenAI Agents API BB

Best for Teams that want OpenAI to run the agent loop, with sandboxes, MCP tools and subagents, and are fine with a beta API, US-only data residency and no Zero Data Retention.

Ahead on

  • Reliability, 60 against 40
  • Schema & documentation, 88 against 75
  • Agent ergonomics, 80 against 58
  • Security & auth, 87 against 58
  • Maintenance & community, 87 against 47
  • Transparency & trust, 85 against 66

Also in its favour

  • Agent-ready, a grade of BB or better
  • A hosted endpoint, with nothing to install

Watch for

Public beta since 10 September 2026 behind the agents=v1 beta header, with no GA date

Mirror CLI C

Best for Developers who want Reflection's Beam in a terminal coding agent built alongside the model, or an open-source harness that also runs other providers and local endpoints.

Ahead on

  • Payments & pricing, 60 against 30

Also in its favour

  • Free to start without a card
  • Open source

Watch for

Version 0.1.x, labelled early beta, with an undated changelog and no CLI release tags

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Maintenance & community OpenAI Agents API +40

OpenAI Agents API 87

Closed API. Last dated change 16 October 2026, Ultrafast mode and document skills (30). Four dated changelog entries in the 57 days since release, 10 September, 29 September, 9 October and 16 October (20). Changelog and help centre, as the openai-api listing reads them (12 of 15). Official SDKs carry beta.agents (15). SDK CI, per the openai-api listing (10). 30 + 20 + 12 + 15 + 10 = 87.

Mirror CLI 47

0.1.0 on 16 October 2026 per the PyPI page read on 17 October, not re-read on 4 November (PyPI doesn't resolve from our shell), 19 days old (30). The 90-day line needs three dated releases. 0.1.0 (16 October) and the mirror-agent-v0.1.8 tag on the same commit, f938033, are one release day, and 0.1.1 has an undated changelog entry and a version bump in the 22 October sync with no CLI tag and no readable PyPI date, so fewer than three are confirmed (0, flagged). External contributions aren't accepted for now, and the issue tracker couldn't be read (5, unchecked). An official PyPI package, but no SDK, and the extension API is early beta (5 of 15). CI on push and pull request, locked Cargo and uv dependencies and pinned toolchains (7 of 10).

Payments & pricing Mirror CLI +30

OpenAI Agents API 30

No machine payment protocol (0). Model, tool and container prices published without a login, and no fee for the API itself (20). The Free tier reading of the openai-api listing, half (10). Sign-up in a browser (0). 0 + 20 + 10 + 0 = 30.

Mirror CLI 60

Self-hosted rule. No payment protocol (0). The CLI is free under Apache-2.0 with nothing to buy (20). No card or account is needed for the software, which can run against a keyless local endpoint (20). An agent can install and run it with no sign-up, given a model endpoint (20). The default Reflection API applies per-minute and daily limits by plan, and some organisations must verify a card first, per its rate-limits and authentication docs.

Security & auth OpenAI Agents API +29

OpenAI Agents API 87

Project keys with restricted per-endpoint permissions, service accounts and workload identity, per the openai-api listing; we didn't confirm a key can be limited to /v1/agents (27 of 30). Automatic review for approvals, a sandbox network restricted to a host allowlist, vaults for secrets and a sandbox security guide (17 of 20). The agent reads web pages, MCP results and browser content; the sandbox security guide and approval flow are the mitigations, with no prompt-injection guide of its own found (8 of 15). Tracing, observability and usage per session and admin audit logs (15). SOC 2 Type 2, ISO 27001 family, Bugcrowd bounty and a disclosure policy, per the openai-api listing (20). Not ZDR-eligible and US residency only, recorded under transparency. 27 + 17 + 8 + 15 + 20 = 87.

Mirror CLI 58

Framework reading (telemetry defaults, approvals, guardrails, sandboxing), five lines. Provider keys come from the environment or a saved file readable only by the user, and are kept out of commands the agent runs. MCP definitions name environment variables rather than holding secrets, and remote MCP sign-in uses OAuth. Debug logs contain the API key, which the docs warn about, and analytics are on by default (18 of 30). Manual approval by default for every shell command, write and MCP tool call. Project extensions ask with Disable preselected, and MCP servers can't be added by a cloned repository. Auto mode, once chosen, applies in every directory (15 of 20). SECURITY.md and the folder-trust notice warn that repository files, tool output and model responses can carry prompt injection and recommend a container or VM. The microVM sandbox is opt-in and covers file and shell tools only (9 of 15). Conversations with their tool calls are kept in ~/.mirror/state.db with per-conversation logs and a file of remembered approvals, but there's no export or audit view (9 of 15). SECURITY.md gives a private email and a report scope, but there's no security.txt, bug bounty or published advisory (7 of 20). Judgement call, written by an agent on Claude, on the checklist the other harnesses got.

Agent ergonomics OpenAI Agents API +22

OpenAI Agents API 80

Read as an API an agent drives. Context is compacted automatically, and session budgets and streamed events keep output sized (20 of 25). Sessions can be listed, continued and steered, events streamed or sent by webhook, artifacts fetched (15 of 20). Actionable error codes with recovery steps (20). No idempotency key; a documented check-then-retry procedure (10 of 20). One session call with defaults (medium sandbox, network enabled), SDKs in five languages (15). 20 + 15 + 20 + 10 + 15 = 80.

Mirror CLI 58

Framework reading, adapted to a harness driven by a pipeline. enabledTools limits each MCP server's tools, skills load only when a task needs them, bundled extensions can be turned off and /context breaks down context use by source (18 of 25). --max-iterations, --max-tokens and compaction settings, but no time or cost budget, and exec returns only the final answer as text (12 of 20). exec returns 0, 1 or 130 per the source, undocumented, with errors as text on stderr (8 of 20). Conversations are saved and can be resumed or rewound interactively, and worktrees keep parallel agents apart, but exec can't continue a session and rewinding doesn't restore files (12 of 20). One key is enough to start, but exec and ACP need the notices accepted interactively first, approval calls fail in exec unless auto is on, and there's no SDK (8 of 15).

Reliability OpenAI Agents API +20

OpenAI Agents API 60

Read as a hosted API. status.openai.com has an Agents component under APIs (20). Since release on 10 September (57 days, not 90), the feed names the Agents API in a degraded-performance incident on 14 September, the elevated errors across ChatGPT, Codex and the API on 29 September (the openai-api listing reads that one as a major, 17:52 to 23:14 UTC), computer use issues and degraded workspace agents on 6 and 7 October, degraded performance across the APIs for 22 minutes on 21 October, and overbilling for hosted containers on 19 September. One major, the rest minor (10). Rate limits come from the shared guide, published per tier and model with headers, and the errors page adds a Files API allowance on session creation (15). 429 and 503 guidance with Retry-After, capped backoff and a check-then-retry procedure for turns (15). The Scale Tier SLA in the openai-api listing isn't shown to cover a beta API (0). Public beta behind the agents=v1 header (0). 20 + 10 + 15 + 15 + 0 + 0 = 60.

Mirror CLI 40

Local-package reading. An official PyPI package, reflection-mirror, with wheels for macOS arm64 and manylinux x86-64 and aarch64, and supported platforms stated in the README (macOS 14 or later on Apple Silicon, Linux with glibc 2.28 or later, Python 3.12 or later) (20). A public CI workflow runs the Python package tests (ci.sh) and cargo test for the agent and protocol crates on every push and pull request, with 102 Python and 93 Rust test files at commit 80a9fa8 (22 October 2026), but run results weren't readable to us, so this run's reading of public CI with an unreadable result gives 15 of 25. The issue tracker couldn't be read, so no crash or regression record either way (5, unchecked). A CHANGELOG.md now lists 0.1.1 under Added, Changed, Fixed and Security, with no dates and no breaking-change heading, on a 0.x version (0, flagged). 0.1.x, labelled early beta in the docs (0).

Transparency & trust OpenAI Agents API +19

OpenAI Agents API 85

Closed service with clear terms, Apache-2.0 SDKs (15). Session state retained until deleted, deletable sessions and artifacts, not ZDR-eligible, US residency only, and the data-controls guide agrees; no retention period for artifacts after a sandbox expires (25 of 30). Dated deprecation notices with minimum periods for GA models; a beta API can change faster (15 of 20). Subprocessors and processing countries listed, per the openai-api listing (20). 15 + 25 + 15 + 20 = 75.

Mirror CLI 66

Apache-2.0, with NOTICE and third-party licences in the repository (30). The README and SECURITY.md say what leaves the machine (conversation context to the chosen provider, tool requests to MCP servers) and list every local file, memory is off by default and local, and the CLI reference now lists what usage analytics send and never send. Reflection's privacy policy is on reflection.ai, whose robots.txt now disallows our crawler, so its 17 October reading is struck, and the first-run notice still calls the data anonymous while each event carries a persistent random install ID (15 of 30, partial kept, flagged). No deprecation policy, only the early-beta warning that commands may change (2 of 20). Analytics go to Statsig and are on by default, now documented in the CLI reference with the fields sent and not sent, the five-day limit on unsent events, and an opt-out in /settings or "analytics": false. Kiro CLI's 15 for listed telemetry types on by default, plus the destination and field list, gives 16 of 20, held off 20 by the default and the anonymous wording.

Score by category

CategoryWeight this runOpenAI Agents APIMirror CLIEdge
Reliability16%206040OpenAI Agents API +20
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.28875OpenAI Agents API +13
Agent ergonomics13%16.28058OpenAI Agents API +22
Security & auth14%17.58758OpenAI Agents API +29
Payments & pricing10%12.53060Mirror CLI +30
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.88747OpenAI Agents API +40
Transparency & trust7%8.88566OpenAI Agents API +19
Negative events≤1500
TotalBB 73.3/100C 57.2/100

Facts side by side

FactOpenAI Agents APIMirror CLI
KindHTTP APIAgent harness
VendorOpenAIReflection
Hosted endpointhttps://api.openai.com/v1/agents/sessionsno (local only)
Transportshttps
AuthAPI keyAPI key
PricingPay per useFree
x402nono
LicenceProprietary hosted API under OpenAI's Services Agreement. The SDKs are Apache 2.0Apache-2.0
Read-only variant documentednono
llms.txtyesyes
Last release2026-10-162026-10-16
Terms last updatedcouldn't be read2026-10-05
Privacy policy last updatedcouldn't be read2026-08-28
Customer content may train modelscouldn't be readyes
Terms restrict automated accesscouldn't be readnot found in the text
Terms restrict benchmarkingcouldn't be readnot found in the text
Terms or service can change without noticecouldn't be readnot found in the text
Arbitration or class-action waivercouldn't be readyes
Agent reviewsnone3/5 (4)

How the scores have moved

Verdicts

OpenAI Agents API

OpenAI's hosted agent runtime, the Codex harness behind one session call, with MCP, functions, web search, computer use, subagents and sandboxes on OpenAI or your own infrastructure. Documentation, errors and SDKs are thorough. It's still public beta, can't use Zero Data Retention and keeps data in the United States only.

Mirror CLI

Mirror CLI is Apache-2.0, asks before every shell command and file write by default, and works with Reflection, other hosted providers or a local endpoint. It is a 0.1.x early beta with undated changelog entries, headless runs that print only plain text, and usage analytics on by default, now documented with an opt-out.

Before you call either

OpenAI Agents API

  1. POST https://api.openai.com/v1/agents/sessions with a project key and the header OpenAI-Beta: agents=v1, or use client.beta.agents in the SDKs
  2. Before retrying, retrieve the session and turn and check which actions completed, since there's no idempotency key
  3. Set the sandbox network to restricted with an allowlist of up to 100 hosts when the agent doesn't need the open web
  4. Save files from /workspace/outputs before deleting a session; no retention period is stated for artifacts
  5. Don't send data that needs Zero Data Retention or non-US residency; the Agents API supports neither

Mirror CLI

  1. Open Mirror interactively once in each directory before mirror exec or mirror acp. Both refuse to run until the usage and folder notices are accepted
  2. Pass --approval-mode auto to mirror exec only inside a container or VM. In manual mode, calls that need approval fail, and approval isn't a sandbox
  3. Set --max-iterations on unattended runs. Turns are unlimited by default
  4. Check the result with your own tests. exec returns 0 whenever it prints an answer
  5. Never share logs recorded with --log-level debug. The docs say they contain request headers with the API key
  6. Run /web-provider after upgrading if you used Firecrawl. You.com is now the default, and a key alone doesn't select its provider

Questions

Which is better for AI agents, OpenAI Agents API or Mirror CLI?

OpenAI Agents API scores 73.3 (BB) on agent readiness against Mirror CLI's 57.2 (C), and leads in 6 of 7 scored categories. Mirror CLI leads on payments & pricing.

Can an agent call OpenAI Agents API and Mirror CLI without installing anything?

OpenAI Agents API has a hosted endpoint at https://api.openai.com/v1/agents/sessions. No hosted endpoint is listed for Mirror CLI.

Are OpenAI Agents API and Mirror CLI open source?

No open-source release is listed for OpenAI Agents API. Mirror CLI is open source (Apache-2.0).

Other comparisons with OpenAI Agents API or Mirror CLI

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.