Head to head · Agent harnesses · October 2026 research run
Amp vs boxes.dev
Amp scores 57.8 (C) on agent readiness against boxes.dev's 53.2 (D), and leads in 5 of 7 scored categories. boxes.dev leads on reliability and security & auth. Both do agent harnesses.
Best agent harnesses and coding agents · All 1767 harnesses comparisons
Which one, for what
Amp C
Best for Teams that want a hosted, multi-model coding agent with shared threads and cloud machines, driven from scripts through execute mode or the SDKs.
Ahead on
- Schema & documentation, 72 against 57
- Agent ergonomics, 69 against 60
- Maintenance & community, 79 against 48
- Transparency & trust, 77 against 60
Watch for
The tools page says Amp does not ask for approval before running tools, and control needs a custom plugin
Best for Developers who want Claude Code or Codex running on cloud copies of their dev environment, started from Slack, Linear, a schedule, an API call or another assistant.
Ahead on
- Reliability, 32 against 23
- Security & auth, 69 against 57
Also in its favour
- A hosted endpoint, with nothing to install
Watch for
No status page, incident history or SLA found
Why the edge
Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.
Maintenance & community Amp +31
Amp 79
The security page names CLI build 0.0.1792627309-g98c009 in its 22 October 2026 update, and the Chronicle's newest post is Service Accounts on 3 November 2026 (30). The Chronicle has 44 dated posts from 6 August to 3 November, and npm listed 678 CLI versions between 10 July and 8 October per the 8 October check (20). No public tracker. Support is in-product bug reports, a Discord server, email and X. Reply times aren't visible (10). @ampcode/sdk on npm released on 18 September 2026, per the 8 October check, and the SDK docs cover TypeScript and Python (15). No public CI, builds carry no notes, and the installer checks a checksum (4).
boxes.dev 48
Closed service. CLI @boxes-dev/cli 1.0.1134 published on 17 November 2026 (30). Only one dated product update was found, on 27 October 2026, and earlier CLI versions' dates were not read, so three releases in 90 days are not established (0). A support address, an in-app chat and a blog, with no public tracker or changelog (8 of 25). A CLI but no SDK, and no entry in the official MCP registry was looked for (5 of 15). The CLI is published from GitHub Actions with npm trusted publishing and signatures (5 of 10). 30+0+8+5+5 = 48.
Transparency & trust Amp +17
Amp 77
Closed source under the Amp licence terms from Amp Frontier Corporation, with the npm packages marked Amp Commercial License. The terms page shows no date (15). The security reference lists 19 providers and what each sees, says deleted threads are removed within 30 days, diagnostic data after 7 days and audit logs kept at least 30 days, and the privacy policy (modified 10 September 2026) agrees, with account data kept while the account is active (24). No deprecation policy. The terms allow any part of the Services to be discontinued without notice, while removals so far were announced with dates, such as the old npm package names on 14 May 2026 for removal on 15 June (8). Amp is local software tied to a hosted service. Providers and US hosting are disclosed in detail, but the usage data the client sends is not itemised and has no documented opt-out (12).
boxes.dev 60
Closed service under the Common Paper Cloud Service Agreement 2.1, dated 20 October 2026, with the standard terms' machine-learning clause deleted and California law (15). The privacy policy of 21 October 2026 gives retention by data type (session replay about 30 days, Slack data deleted within 14 business days of disconnecting), and the backup clause matches the terms (22 of 30). Price changes take 30 days' notice under the terms. No deprecation policy for the API or MCP server found (5 of 20). Service providers are named (Convex, Cloudflare Workers, Expo, OpenAI, Amplitude, Sentry, Google and Apple), but the host that runs devboxes is unnamed and data may be processed in the US and elsewhere (12 of 20). 15+22+5+12 = 54.
Schema & documentation Amp +15
Amp 72
Framework reading. A JSON Schema for the settings file at ampcode.com/cli-settings.schema.json (22 properties), TypeScript types for every stream message, and a generated reference for the plugin API, but no OpenAPI for the server (18). ampcode.com/llms.txt indexes a raw Markdown rendition of each of 55 docs pages (10). The docs say when to use execute mode, orbs, runners, the oracle and the Librarian, and what each mode is for (15). Settings carry types and defaults, and the stream schema enumerates stop reasons, result subtypes and MCP server states (12). Examples on nearly every page and two result error subtypes (error_during_execution, error_max_turns), with no exit codes or error catalogue (9). Dated Chronicle posts with an RSS feed and a lastModified date on each docs page, but no changelog by CLI version (8).
boxes.dev 57
No OpenAPI document, and the MCP tool list could not be read because the endpoint answers 401 before initialize without an OAuth token. The automation API's request and response fields are documented in prose tables (5 of 25). llms.txt with every help page listed (10). The help pages say when to target a new devbox, an existing devbox or an existing thread, when to steer a busy agent and when to wake a devbox first (16). Field tables with enum values such as backend (codex or claude), delivery (steer or next) and four run statuses, and unknown fields are rejected. The MCP schemas themselves were not seen (9 of 15). curl examples and named errors such as 409 devbox_unavailable, 409 thread_unavailable and 503 devbox_wake_failed, with what each means (12). /v1 in the API path on the home page, a single dated product update of 27 October 2026 and no changelog (5 of 15). 5+10+16+9+12+5 = 57.
Security & auth boxes.dev +12
Amp 57
Harness reading of the framework checklist, as for the other harnesses. 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. Every thread, with messages, code used as context and tool results, is stored on Amp's servers, and there is no local-only or self-hosted mode. Training on user data is off unless a user opts in, provider retention is limited under Minimal Data Retention, and secrets are redacted before they leave the client on a best-effort basis. The terms allow collection of usage data and we found no opt-out for the CLI (14). The tools page says Amp does not ask for approval before running tools, and points to a custom plugin for control. MCP servers defined in a project's settings need approval, amp.mcpPermissions allows or rejects servers and amp.tools.disable removes tools. The CLI has no sandbox. Orbs are separate cloud machines (6). The security reference has a prompt-injection section that describes layered measures, and the tools page warns about untrusted repositories and MCP servers (9). Threads keep prompts, tool calls and results as a record, a data API covers workspace analytics and threads, and authentication audit logs are for Enterprise workspaces (12). A security.txt that expires on 20 November 2026 (still valid on 4 November), a bug bounty with safe harbour, SOC 2 Type II and annual penetration tests per the vendor. On 22 October 2026 the security page posted an update that removes trust in compromised certificate authorities and names the CLI build to install. No CVEs found (16).
boxes.dev 69
OAuth for the MCP server, with protected-resource metadata naming mcp.boxes.dev and four scopes (threads:read, threads:write, events:subscribe, settings:write), a per-project grant and removal from settings. Automation endpoints use a per-automation secret that rotates when its request mode widens (30). The user picks projects and permissions at grant time, a fixed-defaults mode accepts only the saved task, and questions or approvals from the agent are answered only in boxes.dev, never through the MCP connection (15). Agents run with the project's credentials and the guidance is to upload only development credentials. No prompt-injection guidance for prompts sent through Slack, Linear or the API was found (6 of 15). Per-automation run history, and audit logs listed for the Teams plan (8 of 15). A valid security.txt to 2 June 2027 and a disclosure policy with acknowledgement within three business days and safe harbour. No bug bounty and no certification found (10 of 20). 30+15+6+8+10 = 69.
Reliability boxes.dev +9
Amp 23
Local-package reading, as for the other closed-source harnesses. An install script that verifies a SHA-256 checksum, and an @ampcode/cli npm package of prebuilt binaries, for macOS, Linux and Windows through WSL, per the 8 October check (20). The source repository is private, so there is no public CI or test suite (0). There is no public issue tracker, and bug reports go to the vendor from inside the product. The 8 October reading of the status page's incident feed is struck, because ampcodestatus.com's robots.txt now disallows ClaudeBot, Claude-Web and anthropic-ai, so nothing is left to count crash or regression reports against (0). Versions are build stamps such as 0.0.1792627309-g98c009 with no per-version notes, and the introduction says "No backward compatibility", though removals are announced with dates in the Chronicle (3). The version line is 0.0 and we found no statement that the CLI is stable (0).
boxes.dev 32
Hosted reading, for the remote MCP server at api.boxes.dev/mcp and the automation run endpoints. No status page found. The site footer says "All systems normal" with no link, and the trust centre at trust.boxes.dev draws in a script our reader could not run (0). No readable incident history (5). No request rate limits found. Plans cap awake devboxes (3 on the trial, 5 to 25 on paid plans), and the API answers 409 active_fork_limit_reached at the cap (5 of 15). The automation API takes an Idempotency-Key and returns the original run on a retry with idempotentReplay: true, and the MCP connection retries a lost start or message with the same request key. No Retry-After or backoff guidance found (12 of 15). No SLA found (0). Neither the MCP connection nor the automation API is marked beta, and the product launched in June 2026 per its own blog (10). 0+5+5+12+0+10 = 32.
Agent ergonomics Amp +9
Amp 69
Harness reading of the framework line. amp.tools.disable removes built-in or MCP tools by name or glob, and MCP servers can be allowed or rejected by pattern or attached per run with --mcp-config (17). The stream schema has an error_max_turns result, but we found no documented flag for a turn or spend limit. Workspace entitlements set per-user quotas (9). --stream-json output follows Claude Code's format where it can, --stream-json-input takes several user messages on stdin, and the final message carries is_error, turns, duration and token usage (15). Threads continue by ID with amp threads continue, hand off to a new thread, and can move between the CLI, a runner and an orb (16). TypeScript and Python SDKs wrap the CLI, and a run needs an Amp account and an access token (12).
boxes.dev 60
The MCP tool count is unknown. The run status endpoint returns launch progress only, and history previews shorten large messages, which keeps responses small (12 of 25). Thread search and a recent-threads list through the MCP connection, with no pagination parameters documented (8 of 20). Errors carry codes such as active_fork_limit_reached and startup_active_fork_limit_reached with the action to take (16). Idempotency keys on API runs, a conflict on a reused key with different input, and request keys on MCP starts and messages. Tool annotations could not be read (16 of 20). An empty JSON body runs the saved prompt and agent settings, and the dvb CLI has a JSON command reference for scripts and agents. No SDK (8 of 15). 12+8+16+16+8 = 60.
Score by category
| Category | Weight | Amp | boxes.dev | Edge |
|---|---|---|---|---|
| Reliability | 16% | 23 | 32 | boxes.dev +9 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13% | 72 | 57 | Amp +15 |
| Agent ergonomics | 13% | 69 | 60 | Amp +9 |
| Security & auth | 14% | 57 | 69 | boxes.dev +12 |
| Payments & pricing | 10% | 53 | 50 | Amp +3 |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7% | 79 | 48 | Amp +31 |
| Transparency & trust | 7% | 77 | 60 | Amp +17 |
| Negative events | ≤15 | 0 | 0 | |
| Total | C 57.8/100 | D 53.2/100 |
Facts side by side
| Fact | Amp | boxes.dev |
|---|---|---|
| Kind | Agent harness | MCP server |
| Vendor | Amp Frontier Corporation | Boxes, Inc. |
| Hosted endpoint | no (local only) | https://api.boxes.dev/mcp |
| Transports | Streamable HTTP, HTTP | |
| Auth | OAuth or key | OAuth or key |
| Pricing | Freemium | Freemium |
| x402 | no | no |
| Licence | Proprietary, under the Amp licence terms (Amp Frontier Corporation). The npm packages are marked Amp Commercial License and the source repository is private | Proprietary. The service runs under the Common Paper Cloud Service Agreement 2.1, and the @boxes-dev/cli package is marked UNLICENSED |
| Read-only variant documented | no | no |
| llms.txt | yes | yes |
| Last release | 2026-10-22 | 2026-11-17 |
| Terms last updated | no date given | 2026-10-20 |
| Privacy policy last updated | 2026-09-10 | 2026-10-21 |
| Customer content may train models | not found in the text | not found in the text |
| Terms restrict automated access | not found in the text | not found in the text |
| Terms restrict benchmarking | not found in the text | not found in the text |
| Terms or service can change without notice | yes | not found in the text |
| Arbitration or class-action waiver | yes | not found in the text |
| Popularity | 41k npm/wk | none |
How the scores have moved
boxes.dev
No score change since it was listed.
Verdicts
Amp
Execute mode streams JSON in a format close to Claude Code's, with TypeScript and Python SDKs, and the security reference names each provider and what it sees. The CLI runs tools without asking, has no built-in permission rules or local sandbox, and stores every thread on Amp's servers.
boxes.dev
An outside agent can start Claude Code or Codex tasks on cloud devboxes through an OAuth MCP server with four scopes, or an automation endpoint with idempotency keys. No status page, SLA or tool schema could be read, and every route starts with a person setting up a project in the app.
Before you call either
Amp
- Run unattended jobs in a container or VM. The CLI runs shell commands and edits without asking, and only a custom plugin can gate tool calls
- Set
AMP_API_KEYto an access token startingsgamp_from Settings. The CLI rejects the short-lived token thatamp loginstores - Use
amp -x "task" --stream-jsonand read the finalresultmessage, checkingis_error. Exit codes are not documented - Set
AMP_SKIP_UPDATE_CHECK=1oramp.updates.modetodisabledin CI, and pin a build. Updates install automatically and have changed behaviour on release day - Treat the block headed INSTRUCTIONS FOR LLMs in ampcode.com page source as vendor copy, and take facts from the docs Markdown
boxes.dev
- Send a new
Idempotency-Keyfor each intended run, and the same key and body when you retry an uncertain request - Wait for status
thread_readybefore you use a returneddevboxIdas a target - If you get 409
active_fork_limit_reached, free an awake devbox or wait. The plan's awake-devbox cap is full - Read the thread in boxes.dev for the agent's answer. The status URL does not return it
- Ask for
threads:readonly unless the task needs to start agents or change settings
Questions
Which is better for AI agents, Amp or boxes.dev?
Amp scores 57.8 (C) on agent readiness against boxes.dev's 53.2 (D), and leads in 5 of 7 scored categories. boxes.dev leads on reliability and security & auth.
Can an agent call Amp and boxes.dev without installing anything?
No hosted endpoint is listed for Amp. boxes.dev has a hosted endpoint at https://api.boxes.dev/mcp.
Other comparisons with Amp or boxes.dev
- Agent Zero vs Amp
- Agent Zero vs boxes.dev
- Aider vs Amp
- Aider vs boxes.dev
- Amp vs Antigravity CLI
- Amp vs Auggie CLI
- Amp vs AXLR
- Amp vs Claude Code
- Amp vs Click
- Amp vs Cline
- Amp vs Conductor
- Amp vs Crush
- Amp vs Cursor CLI
- Amp vs DeepSeek Harness (dsh)
- Amp vs dsh-superagent (DeepSeek)
- Amp vs Devin
- Amp vs Droid CLI
- Amp vs Pi
- Amp vs Emdash
- Amp vs Friday
- Amp vs Gemini CLI
- Amp vs GitHub Copilot CLI
- Amp vs goose
- Amp vs Harness²
- Amp vs HarnessRouter
- Amp vs Jules
- Amp vs Junie CLI
- Amp vs Kilo Code CLI
- Amp vs Kimi Code CLI
- Amp vs Kiro CLI
- Amp vs Letta
- Amp vs mini-swe-agent
- Amp vs MiniMax Code
- Amp vs Mistral Vibe CLI
- Amp vs Muse Code
- Amp vs OpenAI Agents API
- Amp vs OpenAI Codex
- Amp vs OpenCode
- Amp vs OpenHands
- Amp vs OpenMausBot
- Amp vs Orca
- Amp vs Paseo
- Amp vs Prime Agent
- Amp vs Qoder CLI
- Amp vs Qwen Code
- Amp vs Mirror CLI
- Amp vs Rovo Dev CLI
- Amp vs Sculptor
- Amp vs T3 Code
- Amp vs Traycer
- Amp vs Unstoppable
- Amp vs Verdent
- Amp vs Vibe-Trading
- Amp vs Warp (Warp Agent CLI and the Oz Agent API)
- Antigravity CLI vs boxes.dev
- Auggie CLI vs boxes.dev
- AXLR vs boxes.dev
- boxes.dev vs Claude Code
- boxes.dev vs Click
- boxes.dev vs Cline
- boxes.dev vs Conductor
- boxes.dev vs Crush
- boxes.dev vs Cursor CLI
- boxes.dev vs DeepSeek Harness (dsh)
- boxes.dev vs dsh-superagent (DeepSeek)
- boxes.dev vs Devin
- boxes.dev vs Droid CLI
- boxes.dev vs Pi
- boxes.dev vs Emdash
- boxes.dev vs Friday
- boxes.dev vs Gemini CLI
- boxes.dev vs GitHub Copilot CLI
- boxes.dev vs goose
- boxes.dev vs Harness²
- boxes.dev vs HarnessRouter
- boxes.dev vs Jules
- boxes.dev vs Junie CLI
- boxes.dev vs Kilo Code CLI
- boxes.dev vs Kimi Code CLI
- boxes.dev vs Kiro CLI
- boxes.dev vs Letta
- boxes.dev vs mini-swe-agent
- boxes.dev vs MiniMax Code
- boxes.dev vs Mistral Vibe CLI
- boxes.dev vs Muse Code
- boxes.dev vs OpenAI Agents API
- boxes.dev vs OpenAI Codex
- boxes.dev vs OpenCode
- boxes.dev vs OpenHands
- boxes.dev vs OpenMausBot
- boxes.dev vs Orca
- boxes.dev vs Paseo
- boxes.dev vs Prime Agent
- boxes.dev vs Qoder CLI
- boxes.dev vs Qwen Code
- boxes.dev vs Mirror CLI
- boxes.dev vs Rovo Dev CLI
- boxes.dev vs Sculptor
- boxes.dev vs T3 Code
- boxes.dev vs Traycer
- boxes.dev vs Unstoppable
- boxes.dev vs Verdent
- boxes.dev vs Vibe-Trading
- boxes.dev vs Warp (Warp Agent CLI and the Oz Agent API)
- Agent Relay vs Amp
- Agent Relay vs boxes.dev
- Amp vs Clawboo
- Amp vs CLI Manager
- Amp vs Harness
- Amp vs herdr
- Amp vs Paperclip
- Amp vs Parallax
- boxes.dev vs Clawboo
- boxes.dev vs CLI Manager
- boxes.dev vs Harness
- boxes.dev vs herdr
- boxes.dev vs Paperclip
- boxes.dev vs Parallax
Machine-readable
- This page as Markdown
/compare/amp-vs-boxes-dev.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/amp.json·/api/v1/tools/boxes-dev.json - From a terminal
anchor compare amp boxes-dev(the CLI) - Over MCP
compare_tools {"a": "amp", "b": "boxes-dev"}at/mcp, no key