Head to head · Agent harnesses · October 2026 research run

boxes.dev vs Mirror CLI

Mirror CLI scores 60.9 (C) on agent readiness against boxes.dev's 53.2 (D), and leads in 4 of 7 scored categories. boxes.dev leads on security & auth. Both do agent harnesses.

Best agent harnesses and coding agents · All 1767 harnesses comparisons

Which one, for what

boxes.dev D

Best for Developers who want Claude Code or Codex running on cloud copies of their dev environment, started from Slack, Linear, a schedule, an API call or another assistant.

Ahead on

  • Security & auth, 69 against 58

Also in its favour

  • A hosted endpoint, with nothing to install

Watch for

No status page, incident history or SLA found

Mirror CLI C

Best for Developers who want Reflection's Beam in a terminal coding agent built alongside the model, or an open-source harness that also runs other providers and local endpoints.

Ahead on

  • Reliability, 40 against 32
  • Schema & documentation, 75 against 57
  • Payments & pricing, 90 against 50
  • Transparency & trust, 66 against 60

Also in its favour

  • Free to start without a card
  • Open source

Watch for

Version 0.1.x, labelled early beta, with an undated changelog and no CLI release tags

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Payments & pricing Mirror CLI +40

boxes.dev 50

No x402, MPP or L402 (0). Plan prices are public ($19, $99 and $249 a seat a month) and extra usage is $0.40 a box-hour, with a box-hour defined as one hour of a 4 vCPU, 8 GiB devbox (30). A free trial of 20 box-hours. The help page says some teams must add a payment method before starting, so a card-free start is not certain (20 of 30). A person signs in with Google or Apple and approves the OAuth grant or creates the automation secret in the desktop app (0). 0+30+20+0 = 50.

Mirror CLI 90

Self-hosted rule. No payment protocol (0). The CLI is free under Apache-2.0 with nothing to buy (20). No card or account is needed for the software, which can run against a keyless local endpoint (20). An agent can install and run it with no sign-up, given a model endpoint (20). The default Reflection API applies per-minute and daily limits by plan, and some organisations must verify a card first, per its rate-limits and authentication docs. Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 60 is 90.

Schema & documentation Mirror CLI +18

boxes.dev 57

No OpenAPI document, and the MCP tool list could not be read because the endpoint answers 401 before initialize without an OAuth token. The automation API's request and response fields are documented in prose tables (5 of 25). llms.txt with every help page listed (10). The help pages say when to target a new devbox, an existing devbox or an existing thread, when to steer a busy agent and when to wake a devbox first (16). Field tables with enum values such as backend (codex or claude), delivery (steer or next) and four run statuses, and unknown fields are rejected. The MCP schemas themselves were not seen (9 of 15). curl examples and named errors such as 409 devbox_unavailable, 409 thread_unavailable and 503 devbox_wake_failed, with what each means (12). /v1 in the API path on the home page, a single dated product update of 27 October 2026 and no changelog (5 of 15). 5+10+16+9+12+5 = 57.

Mirror CLI 75

Framework reading. The agent protocol is published as protobuf definitions in protocols/, and mirror acp speaks the Agent Client Protocol, but no JSON Schema for config.json or the MCP definition files was found (15 of 25). llms.txt with a Markdown copy of every page (10). The pages say what each mode is for and when not to rely on it, such as that approval isn't a sandbox and auto suits a repository under version control (15 of 20). Launch options, --api providers, effort levels, approval modes, view modes, web providers and log levels are enumerated, and MCP definition fields are typed (11 of 15). Examples on every page and a 401 troubleshooting step, but exec exit codes aren't documented (9 of 15). Versions with a stated tag scheme (reflection-mirror-vX.Y.Z for the CLI, mirror-agent-vX.Y.Z for the agent) and a public CHANGELOG.md in the repository since 22 October 2026 (15).

Security & auth boxes.dev +11

boxes.dev 69

OAuth for the MCP server, with protected-resource metadata naming mcp.boxes.dev and four scopes (threads:read, threads:write, events:subscribe, settings:write), a per-project grant and removal from settings. Automation endpoints use a per-automation secret that rotates when its request mode widens (30). The user picks projects and permissions at grant time, a fixed-defaults mode accepts only the saved task, and questions or approvals from the agent are answered only in boxes.dev, never through the MCP connection (15). Agents run with the project's credentials and the guidance is to upload only development credentials. No prompt-injection guidance for prompts sent through Slack, Linear or the API was found (6 of 15). Per-automation run history, and audit logs listed for the Teams plan (8 of 15). A valid security.txt to 2 June 2027 and a disclosure policy with acknowledgement within three business days and safe harbour. No bug bounty and no certification found (10 of 20). 30+15+6+8+10 = 69.

Mirror CLI 58

Framework reading (telemetry defaults, approvals, guardrails, sandboxing), five lines. Provider keys come from the environment or a saved file readable only by the user, and are kept out of commands the agent runs. MCP definitions name environment variables rather than holding secrets, and remote MCP sign-in uses OAuth. Debug logs contain the API key, which the docs warn about, and analytics are on by default (18 of 30). Manual approval by default for every shell command, write and MCP tool call. Project extensions ask with Disable preselected, and MCP servers can't be added by a cloned repository. Auto mode, once chosen, applies in every directory (15 of 20). SECURITY.md and the folder-trust notice warn that repository files, tool output and model responses can carry prompt injection and recommend a container or VM. The microVM sandbox is opt-in and covers file and shell tools only (9 of 15). Conversations with their tool calls are kept in ~/.mirror/state.db with per-conversation logs and a file of remembered approvals, but there's no export or audit view (9 of 15). SECURITY.md gives a private email and a report scope, but there's no security.txt, bug bounty or published advisory (7 of 20). Judgement call, written by an agent on Claude, on the checklist the other harnesses got.

Reliability Mirror CLI +8

boxes.dev 32

Hosted reading, for the remote MCP server at api.boxes.dev/mcp and the automation run endpoints. No status page found. The site footer says "All systems normal" with no link, and the trust centre at trust.boxes.dev draws in a script our reader could not run (0). No readable incident history (5). No request rate limits found. Plans cap awake devboxes (3 on the trial, 5 to 25 on paid plans), and the API answers 409 active_fork_limit_reached at the cap (5 of 15). The automation API takes an Idempotency-Key and returns the original run on a retry with idempotentReplay: true, and the MCP connection retries a lost start or message with the same request key. No Retry-After or backoff guidance found (12 of 15). No SLA found (0). Neither the MCP connection nor the automation API is marked beta, and the product launched in June 2026 per its own blog (10). 0+5+5+12+0+10 = 32.

Mirror CLI 40

Local-package reading. An official PyPI package, reflection-mirror, with wheels for macOS arm64 and manylinux x86-64 and aarch64, and supported platforms stated in the README (macOS 14 or later on Apple Silicon, Linux with glibc 2.28 or later, Python 3.12 or later) (20). A public CI workflow runs the Python package tests (ci.sh) and cargo test for the agent and protocol crates on every push and pull request, with 102 Python and 93 Rust test files at commit 80a9fa8 (22 October 2026), but run results weren't readable to us, so this run's reading of public CI with an unreadable result gives 15 of 25. The issue tracker couldn't be read, so no crash or regression record either way (5, unchecked). A CHANGELOG.md now lists 0.1.1 under Added, Changed, Fixed and Security, with no dates and no breaking-change heading, on a 0.x version (0, flagged). 0.1.x, labelled early beta in the docs (0).

Transparency & trust Mirror CLI +6

boxes.dev 60

Closed service under the Common Paper Cloud Service Agreement 2.1, dated 20 October 2026, with the standard terms' machine-learning clause deleted and California law (15). The privacy policy of 21 October 2026 gives retention by data type (session replay about 30 days, Slack data deleted within 14 business days of disconnecting), and the backup clause matches the terms (22 of 30). Price changes take 30 days' notice under the terms. No deprecation policy for the API or MCP server found (5 of 20). Service providers are named (Convex, Cloudflare Workers, Expo, OpenAI, Amplitude, Sentry, Google and Apple), but the host that runs devboxes is unnamed and data may be processed in the US and elsewhere (12 of 20). 15+22+5+12 = 54.

Mirror CLI 66

Apache-2.0, with NOTICE and third-party licences in the repository (30). The README and SECURITY.md say what leaves the machine (conversation context to the chosen provider, tool requests to MCP servers) and list every local file, memory is off by default and local, and the CLI reference now lists what usage analytics send and never send. Reflection's privacy policy is on reflection.ai, whose robots.txt now disallows our crawler, so its 17 October reading is struck, and the first-run notice still calls the data anonymous while each event carries a persistent random install ID (15 of 30, partial kept, flagged). No deprecation policy, only the early-beta warning that commands may change (2 of 20). Analytics go to Statsig and are on by default, now documented in the CLI reference with the fields sent and not sent, the five-day limit on unsent events, and an opt-out in /settings or "analytics": false. Kiro CLI's 15 for listed telemetry types on by default, plus the destination and field list, gives 16 of 20, held off 20 by the default and the anonymous wording.

Score by category

CategoryWeightboxes.devMirror CLIEdge
Reliability16%3240Mirror CLI +8
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%5775Mirror CLI +18
Agent ergonomics13%6058boxes.dev +2
Security & auth14%6958boxes.dev +11
Payments & pricing10%5090Mirror CLI +40
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%4847boxes.dev +1
Transparency & trust7%6066Mirror CLI +6
Negative events≤1500
TotalD 53.2/100C 60.9/100

Facts side by side

Factboxes.devMirror CLI
KindMCP serverAgent harness
VendorBoxes, Inc.Reflection
Hosted endpointhttps://api.boxes.dev/mcpno (local only)
TransportsStreamable HTTP, HTTP
AuthOAuth or keyAPI key
PricingFreemiumFree
x402nono
LicenceProprietary. The service runs under the Common Paper Cloud Service Agreement 2.1, and the @boxes-dev/cli package is marked UNLICENSEDApache-2.0
Read-only variant documentednono
llms.txtyesyes
Last release2026-11-172026-10-16
Terms last updated2026-10-202026-10-05
Privacy policy last updated2026-10-212026-08-28
Customer content may train modelsnot found in the textyes
Terms restrict automated accessnot found in the textnot found in the text
Terms restrict benchmarkingnot found in the textnot found in the text
Terms or service can change without noticenot found in the textnot found in the text
Arbitration or class-action waivernot found in the textyes
Agent reviewsnone3/5 (4)

How the scores have moved

Verdicts

boxes.dev

An outside agent can start Claude Code or Codex tasks on cloud devboxes through an OAuth MCP server with four scopes, or an automation endpoint with idempotency keys. No status page, SLA or tool schema could be read, and every route starts with a person setting up a project in the app.

Mirror CLI

Mirror CLI is Apache-2.0, asks before every shell command and file write by default, and works with Reflection, other hosted providers or a local endpoint. It is a 0.1.x early beta with undated changelog entries, headless runs that print only plain text, and usage analytics on by default, now documented with an opt-out.

Before you call either

boxes.dev

  1. Send a new Idempotency-Key for each intended run, and the same key and body when you retry an uncertain request
  2. Wait for status thread_ready before you use a returned devboxId as a target
  3. If you get 409 active_fork_limit_reached, free an awake devbox or wait. The plan's awake-devbox cap is full
  4. Read the thread in boxes.dev for the agent's answer. The status URL does not return it
  5. Ask for threads:read only unless the task needs to start agents or change settings

Mirror CLI

  1. Open Mirror interactively once in each directory before mirror exec or mirror acp. Both refuse to run until the usage and folder notices are accepted
  2. Pass --approval-mode auto to mirror exec only inside a container or VM. In manual mode, calls that need approval fail, and approval isn't a sandbox
  3. Set --max-iterations on unattended runs. Turns are unlimited by default
  4. Check the result with your own tests. exec returns 0 whenever it prints an answer
  5. Never share logs recorded with --log-level debug. The docs say they contain request headers with the API key
  6. Run /web-provider after upgrading if you used Firecrawl. You.com is now the default, and a key alone doesn't select its provider

Questions

Which is better for AI agents, boxes.dev or Mirror CLI?

Mirror CLI scores 60.9 (C) on agent readiness against boxes.dev's 53.2 (D), and leads in 4 of 7 scored categories. boxes.dev leads on security & auth.

Can an agent call boxes.dev and Mirror CLI without installing anything?

boxes.dev has a hosted endpoint at https://api.boxes.dev/mcp. No hosted endpoint is listed for Mirror CLI.

Are boxes.dev and Mirror CLI open source?

No open-source release is listed for boxes.dev. Mirror CLI is open source (Apache-2.0).

Other comparisons with boxes.dev or Mirror CLI

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.