Head to head · Agent harnesses · October 2026 research run
boxes.dev vs Droid CLI
Droid CLI scores 57.8 (C) on agent readiness against boxes.dev's 53.2 (D), and leads in 5 of 7 scored categories. boxes.dev leads on security & auth and payments & pricing. Both do agent harnesses.
Best agent harnesses and coding agents · All 1767 harnesses comparisons
Which one, for what
Best for Developers who want Claude Code or Codex running on cloud copies of their dev environment, started from Slack, Linear, a schedule, an API call or another assistant.
Ahead on
- Security & auth, 69 against 63
- Payments & pricing, 50 against 15
Also in its favour
- A hosted endpoint, with nothing to install
- No incidents deducted, where Droid CLI loses 3 points for them
Watch for
No status page, incident history or SLA found
Best for Teams that want a terminal coding agent with tiered autonomy, command rules and an optional sandbox, and that will pay for a Factory plan.
Ahead on
- Reliability, 54 against 32
- Schema & documentation, 79 against 57
- Agent ergonomics, 72 against 60
- Maintenance & community, 79 against 48
- Transparency & trust, 65 against 60
Watch for
No free tier or trial was found. Plans start at $20 a month and included usage is stated only as rolling rate limits without numbers
Why the edge
Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.
Payments & pricing boxes.dev +35
boxes.dev 50
No x402, MPP or L402 (0). Plan prices are public ($19, $99 and $249 a seat a month) and extra usage is $0.40 a box-hour, with a box-hour defined as one hour of a 4 vCPU, 8 GiB devbox (30). A free trial of 20 box-hours. The help page says some teams must add a payment method before starting, so a card-free start is not certain (20 of 30). A person signs in with Google or Apple and approves the OAuth grant or creates the automation secret in the desktop app (0). 0+30+20+0 = 50.
Droid CLI 15
Harness reading of the published rubric. This is paid, closed software tied to a subscription, so the self-hosted rule does not apply. No payment protocol (0). Plan prices are public, $20, $100 and $200 a month for individuals and $60 a team plus $40 a seat for Teams, but included usage is given only as rolling rate limits and multiples of Pro (10). No free tier or trial was found on the pricing pages. Using your own model key is free up to an allowance, but only on a paid plan (0). A person signs in through a browser to subscribe and to create the API key. The Service Accounts API can issue keys only inside an existing organisation (0). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 10 is 15.
Maintenance & community Droid CLI +31
boxes.dev 48
Closed service. CLI @boxes-dev/cli 1.0.1134 published on 17 November 2026 (30). Only one dated product update was found, on 27 October 2026, and earlier CLI versions' dates were not read, so three releases in 90 days are not established (0). A support address, an in-app chat and a blog, with no public tracker or changelog (8 of 25). A CLI but no SDK, and no entry in the official MCP registry was looked for (5 of 15). The CLI is published from GitHub Actions with npm trusted publishing and signatures (5 of 10). 30+0+8+5+5 = 48.
Droid CLI 79
The changelog's newest entry is CLI v1.3.0 of 4 November 2026 (30). 67 dated changelog entries between 6 August and 4 November 2026, 20 of them after 9 October (20). The public tracker had 33 open issues that are not pull requests on 9 October 2026, most with no reply, and three closed in the previous week after two or three comments, not re-read on 4 November. Support is also by email and Discord, where reply times are not visible (9). The TypeScript SDK repository moved to 0.10.0 with examples tested in CI on 20 October 2026, and the Python SDK is at 0.5.0 of 24 September 2026 (15). The npm package is published from GitHub Actions through a trusted publisher and the installer checks a checksum, per the 9 October check. There is no public CI for the CLI (5).
Reliability Droid CLI +22
boxes.dev 32
Hosted reading, for the remote MCP server at api.boxes.dev/mcp and the automation run endpoints. No status page found. The site footer says "All systems normal" with no link, and the trust centre at trust.boxes.dev draws in a script our reader could not run (0). No readable incident history (5). No request rate limits found. Plans cap awake devboxes (3 on the trial, 5 to 25 on paid plans), and the API answers 409 active_fork_limit_reached at the cap (5 of 15). The automation API takes an Idempotency-Key and returns the original run on a retry with idempotentReplay: true, and the MCP connection retries a lost start or message with the same request key. No Retry-After or backoff guidance found (12 of 15). No SLA found (0). Neither the MCP connection nor the automation API is marked beta, and the product launched in June 2026 per its own blog (10). 0+5+5+12+0+10 = 32.
Droid CLI 54
Local-package reading, as for the other closed-source harnesses. An install script that verifies a SHA-256 checksum, a PowerShell installer, a Homebrew cask and the droid npm package (Node 20 or later) with prebuilt binaries for Linux, macOS and Windows on x64 and arm64, per the 9 October check (20). The source is not published. The public repository holds documentation only, so there is no public CI or test suite for the CLI (0). Bugs are tracked in the open at Factory-AI/factory. On 9 October 2026 it had 33 open issues that are not pull requests, most with no reply, and three closed in the previous week with replies. Issues were not re-read on 4 November because GitHub's API refused our requests (10). Every release has a dated, versioned changelog entry and deprecations get their own section. The line moved from 0.251.0 to 1.0.0 on 29 October, with three minors since, so the version number has signalled breaking changes for only six days (9). CLI v1.0.0 was released on 29 October 2026 and v1.3.0 on 4 November (15). The 99 per cent availability target on factory.com/sla is not scored on this reading.
Schema & documentation Droid CLI +22
boxes.dev 57
No OpenAPI document, and the MCP tool list could not be read because the endpoint answers 401 before initialize without an OAuth token. The automation API's request and response fields are documented in prose tables (5 of 25). llms.txt with every help page listed (10). The help pages say when to target a new devbox, an existing devbox or an existing thread, when to steer a busy agent and when to wake a devbox first (16). Field tables with enum values such as backend (codex or claude), delivery (steer or next) and four run statuses, and unknown fields are rejected. The MCP schemas themselves were not seen (9 of 15). curl examples and named errors such as 409 devbox_unavailable, 409 thread_unavailable and 503 devbox_wake_failed, with what each means (12). /v1 in the API path on the home page, a single dated product update of 27 October 2026 and no changelog (5 of 15). 5+10+16+9+12+5 = 57.
Droid CLI 79
Framework reading. The docs index links an OpenAPI file for the Factory public API, which we did not open, and the headless mode has a documented JSON-RPC surface with named methods. droid exec --list-tools --output-format json and droid rules check --json give machine-readable output. We found no published schema for the settings file (15). docs.factory.com/llms.txt lists a Markdown copy of every page (10). The overview says when to use the CLI, and the autonomy table says what each level allows, blocks and suits (16). Autonomy levels, output formats, sandbox modes and MCP server fields are enumerated with types and defaults (12). Examples on every page read, exit codes 0, 1 and 2, and a JSON result with is_error and subtype, with no error catalogue (11). A changelog by CLI version with dates and an RSS feed, 236 entries back to 30 September 2025 (15).
Agent ergonomics Droid CLI +12
boxes.dev 60
The MCP tool count is unknown. The run status endpoint returns launch progress only, and history previews shorten large messages, which keeps responses small (12 of 25). Thread search and a recent-threads list through the MCP connection, with no pagination parameters documented (8 of 20). Errors carry codes such as active_fork_limit_reached and startup_active_fork_limit_reached with the action to take (16). Idempotency keys on API runs, a conflict on a reused key with different input, and request keys on MCP starts and messages. Tool annotations could not be read (16 of 20). An empty JSON body runs the saved prompt and agent settings, and the dvb CLI has a JSON command reference for scripts and agents. No SDK (8 of 15). 12+8+16+16+8 = 60.
Droid CLI 72
Harness reading of the framework line. --restrict-tools, --disabled-tools, --additional-tools and --list-tools control built-in tools per run, and disabledTools in mcp.json keeps named MCP tools out of context. We found nothing on deferred tool loading (18). Output is text, json, stream-json or stream-jsonrpc, and the JSON result carries turns and duration. The droid exec help lists no flag for a turn or spend limit (10). Exit codes 0, 1 and 2 are documented, and a run that exceeds its autonomy level stops with a non-zero code and no partial changes, per the docs (15). --session-id continues a session, --fork branches one, and --worktree isolates parallel runs (16). Headless runs are read-only without flags, and TypeScript and Python SDKs wrap the CLI. A run needs a paid Factory account and an API key (13).
Security & auth boxes.dev +6
boxes.dev 69
OAuth for the MCP server, with protected-resource metadata naming mcp.boxes.dev and four scopes (threads:read, threads:write, events:subscribe, settings:write), a per-project grant and removal from settings. Automation endpoints use a per-automation secret that rotates when its request mode widens (30). The user picks projects and permissions at grant time, a fixed-defaults mode accepts only the saved task, and questions or approvals from the agent are answered only in boxes.dev, never through the MCP connection (15). Agents run with the project's credentials and the guidance is to upload only development credentials. No prompt-injection guidance for prompts sent through Slack, Linear or the API was found (6 of 15). Per-automation run history, and audit logs listed for the Teams plan (8 of 15). A valid security.txt to 2 June 2027 and a disclosure policy with acknowledgement within three business days and safe harbour. No bug bounty and no certification found (10 of 20). 30+15+6+8+10 = 69.
Droid CLI 63
Harness reading of the framework checklist, as for the other harnesses. 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. The agent loop runs locally and the docs say no copy of the repository is uploaded or indexed. Prompts and context go to the model endpoints. Usage metrics go to Factory's collector by default, with no opt-out in the pages read outside an airgapped deployment, and message content is never sent to it. The security page says code is not used as training data. Zero data retention is listed as a Business plan item. Headless runs use a long-lived fk- key (17). droid exec is read-only by default, autonomy has four levels, permission rules allow, ask or block commands, and an OS sandbox confines commands with Seatbelt or bubblewrap and a network proxy. The sandbox is off by default and --skip-permissions-unsafe skips every prompt (16). The sandbox page names prompt injection and states what the sandbox does not stop, MCP approvals are bound to the server's command or URL, and Droid Shield scans commits for secrets. The Agent Safety page was not read (9). An organisation audit log, OpenTelemetry export and session tags exist, with audit logging listed from the Business plan up (10). A disclosure address (security@factory.com), SOC 2 and ISO 42001 named on the security page, per the 9 October check, and Security sections in the changelog. No security.txt (404 on 4 November), no bug bounty and no published advisories were found, OSV holds nothing for droid, and the trust centre was not readable (11).
Score by category
| Category | Weight | boxes.dev | Droid CLI | Edge |
|---|---|---|---|---|
| Reliability | 16% | 32 | 54 | Droid CLI +22 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13% | 57 | 79 | Droid CLI +22 |
| Agent ergonomics | 13% | 60 | 72 | Droid CLI +12 |
| Security & auth | 14% | 69 | 63 | boxes.dev +6 |
| Payments & pricing | 10% | 50 | 15 | boxes.dev +35 |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7% | 48 | 79 | Droid CLI +31 |
| Transparency & trust | 7% | 60 | 65 | Droid CLI +5 |
| Negative events | ≤15 | 0 | -3 | |
| Total | D 53.2/100 | C 57.8/100 |
Facts side by side
| Fact | boxes.dev | Droid CLI |
|---|---|---|
| Kind | MCP server | Agent harness |
| Vendor | Boxes, Inc. | Factory |
| Hosted endpoint | https://api.boxes.dev/mcp | no (local only) |
| Transports | Streamable HTTP, HTTP | |
| Auth | OAuth or key | OAuth or key |
| Pricing | Freemium | Paid |
| x402 | no | no |
| Licence | Proprietary. The service runs under the Common Paper Cloud Service Agreement 2.1, and the @boxes-dev/cli package is marked UNLICENSED | Proprietary, under Factory's Terms and Conditions. The droid npm package is marked UNLICENSED and the public repository holds documentation only. The Python SDK is Apache-2.0 |
| Read-only variant documented | no | yes |
| llms.txt | yes | yes |
| Last release | 2026-11-17 | 2026-11-04 |
| Terms last updated | 2026-10-20 | 2026-07-14 |
| Privacy policy last updated | 2026-10-21 | 2026-08-20 |
| Customer content may train models | not found in the text | not found in the text |
| Terms restrict automated access | not found in the text | not found in the text |
| Terms restrict benchmarking | not found in the text | yes |
| Terms or service can change without notice | not found in the text | not found in the text |
| Arbitration or class-action waiver | not found in the text | yes |
| Popularity | none | 49 stars, 7.8k npm/wk |
How the scores have moved
boxes.dev
No score change since it was listed.
Verdicts
boxes.dev
An outside agent can start Claude Code or Codex tasks on cloud devboxes through an OAuth MCP server with four scopes, or an automation endpoint with idempotency keys. No status page, SLA or tool schema could be read, and every route starts with a person setting up a project in the app.
Droid CLI
droid exec is read-only unless --auto raises the autonomy level, with command rules, an optional kernel-enforced sandbox, JSON output and documented exit codes. No free tier, status page or security.txt was found, the source is closed, usage metrics go to Factory by default, and security fixes ship in release notes without advisories.
Before you call either
boxes.dev
- Send a new
Idempotency-Keyfor each intended run, and the same key and body when you retry an uncertain request - Wait for status
thread_readybefore you use a returneddevboxIdas a target - If you get 409
active_fork_limit_reached, free an awake devbox or wait. The plan's awake-devbox cap is full - Read the thread in boxes.dev for the agent's answer. The status URL does not return it
- Ask for
threads:readonly unless the task needs to start agents or change settings
Droid CLI
- Set
FACTORY_API_KEY(startsfk-) from the API keys page in Factory settings for headless runs. Interactive use signs in through a browser - Start with
droid execand no flags for analysis. Add--auto lowfor edits and--auto mediumfor installs, tests and local commits - Use
--output-format jsonand readis_error,num_turnsandsession_id. Treat a non-zero exit code as failure - Set
sandbox.enabledto true before running on untrusted code. Indroid execa sandbox violation is denied without a prompt - Pin the version in CI with
npm install -g droid@<version>, or setFACTORY_DROID_AUTO_UPDATE_ENABLED=falseon standalone installs, which update themselves. Read each release's Security section before upgrading
Questions
Which is better for AI agents, boxes.dev or Droid CLI?
Droid CLI scores 57.8 (C) on agent readiness against boxes.dev's 53.2 (D), and leads in 5 of 7 scored categories. boxes.dev leads on security & auth and payments & pricing.
Can an agent call boxes.dev and Droid CLI without installing anything?
boxes.dev has a hosted endpoint at https://api.boxes.dev/mcp. No hosted endpoint is listed for Droid CLI.
Other comparisons with boxes.dev or Droid CLI
- Agent Zero vs boxes.dev
- Agent Zero vs Droid CLI
- Aider vs boxes.dev
- Aider vs Droid CLI
- Amp vs boxes.dev
- Amp vs Droid CLI
- Antigravity CLI vs boxes.dev
- Antigravity CLI vs Droid CLI
- Auggie CLI vs boxes.dev
- Auggie CLI vs Droid CLI
- AXLR vs boxes.dev
- AXLR vs Droid CLI
- boxes.dev vs Claude Code
- boxes.dev vs Click
- boxes.dev vs Cline
- boxes.dev vs Conductor
- boxes.dev vs Crush
- boxes.dev vs Cursor CLI
- boxes.dev vs DeepSeek Harness (dsh)
- boxes.dev vs dsh-superagent (DeepSeek)
- boxes.dev vs Devin
- boxes.dev vs Pi
- boxes.dev vs Emdash
- boxes.dev vs Friday
- boxes.dev vs Gemini CLI
- boxes.dev vs GitHub Copilot CLI
- boxes.dev vs goose
- boxes.dev vs Harness²
- boxes.dev vs HarnessRouter
- boxes.dev vs Jules
- boxes.dev vs Junie CLI
- boxes.dev vs Kilo Code CLI
- boxes.dev vs Kimi Code CLI
- boxes.dev vs Kiro CLI
- boxes.dev vs Letta
- boxes.dev vs mini-swe-agent
- boxes.dev vs MiniMax Code
- boxes.dev vs Mistral Vibe CLI
- boxes.dev vs Muse Code
- boxes.dev vs OpenAI Agents API
- boxes.dev vs OpenAI Codex
- boxes.dev vs OpenCode
- boxes.dev vs OpenHands
- boxes.dev vs OpenMausBot
- boxes.dev vs Orca
- boxes.dev vs Paseo
- boxes.dev vs Prime Agent
- boxes.dev vs Qoder CLI
- boxes.dev vs Qwen Code
- boxes.dev vs Mirror CLI
- boxes.dev vs Rovo Dev CLI
- boxes.dev vs Sculptor
- boxes.dev vs T3 Code
- boxes.dev vs Traycer
- boxes.dev vs Unstoppable
- boxes.dev vs Verdent
- boxes.dev vs Vibe-Trading
- boxes.dev vs Warp (Warp Agent CLI and the Oz Agent API)
- Claude Code vs Droid CLI
- Click vs Droid CLI
- Cline vs Droid CLI
- Conductor vs Droid CLI
- Crush vs Droid CLI
- Cursor CLI vs Droid CLI
- DeepSeek Harness (dsh) vs Droid CLI
- dsh-superagent (DeepSeek) vs Droid CLI
- Devin vs Droid CLI
- Droid CLI vs Pi
- Droid CLI vs Emdash
- Droid CLI vs Friday
- Droid CLI vs Gemini CLI
- Droid CLI vs GitHub Copilot CLI
- Droid CLI vs goose
- Droid CLI vs Harness²
- Droid CLI vs HarnessRouter
- Droid CLI vs Jules
- Droid CLI vs Junie CLI
- Droid CLI vs Kilo Code CLI
- Droid CLI vs Kimi Code CLI
- Droid CLI vs Kiro CLI
- Droid CLI vs Letta
- Droid CLI vs mini-swe-agent
- Droid CLI vs MiniMax Code
- Droid CLI vs Mistral Vibe CLI
- Droid CLI vs Muse Code
- Droid CLI vs OpenAI Agents API
- Droid CLI vs OpenAI Codex
- Droid CLI vs OpenCode
- Droid CLI vs OpenHands
- Droid CLI vs OpenMausBot
- Droid CLI vs Orca
- Droid CLI vs Paseo
- Droid CLI vs Prime Agent
- Droid CLI vs Qoder CLI
- Droid CLI vs Qwen Code
- Droid CLI vs Mirror CLI
- Droid CLI vs Rovo Dev CLI
- Droid CLI vs Sculptor
- Droid CLI vs T3 Code
- Droid CLI vs Traycer
- Droid CLI vs Unstoppable
- Droid CLI vs Verdent
- Droid CLI vs Vibe-Trading
- Droid CLI vs Warp (Warp Agent CLI and the Oz Agent API)
- Agent Relay vs boxes.dev
- Agent Relay vs Droid CLI
- boxes.dev vs Clawboo
- boxes.dev vs CLI Manager
- boxes.dev vs Harness
- boxes.dev vs herdr
- boxes.dev vs Paperclip
- boxes.dev vs Parallax
- Clawboo vs Droid CLI
- CLI Manager vs Droid CLI
- Droid CLI vs Harness
- Droid CLI vs herdr
- Droid CLI vs Paperclip
- Droid CLI vs Parallax
Machine-readable
- This page as Markdown
/compare/boxes-dev-vs-droid-cli.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/boxes-dev.json·/api/v1/tools/droid-cli.json - From a terminal
anchor compare boxes-dev droid-cli(the CLI) - Over MCP
compare_tools {"a": "boxes-dev", "b": "droid-cli"}at/mcp, no key