Head to head · Agent harnesses · October 2026 research run

Amp vs boxes.dev

Amp scores 57.8 (C) on agent readiness against boxes.dev's 53.2 (D), and leads in 5 of 7 scored categories. boxes.dev leads on reliability and security & auth. Both do agent harnesses.

Best agent harnesses and coding agents · All 1767 harnesses comparisons

Which one, for what

Amp C

Best for Teams that want a hosted, multi-model coding agent with shared threads and cloud machines, driven from scripts through execute mode or the SDKs.

Ahead on

  • Schema & documentation, 72 against 57
  • Agent ergonomics, 69 against 60
  • Maintenance & community, 79 against 48
  • Transparency & trust, 77 against 60

Watch for

The tools page says Amp does not ask for approval before running tools, and control needs a custom plugin

boxes.dev D

Best for Developers who want Claude Code or Codex running on cloud copies of their dev environment, started from Slack, Linear, a schedule, an API call or another assistant.

Ahead on

  • Reliability, 32 against 23
  • Security & auth, 69 against 57

Also in its favour

  • A hosted endpoint, with nothing to install

Watch for

No status page, incident history or SLA found

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Maintenance & community Amp +31

Amp 79

The security page names CLI build 0.0.1792627309-g98c009 in its 22 October 2026 update, and the Chronicle's newest post is Service Accounts on 3 November 2026 (30). The Chronicle has 44 dated posts from 6 August to 3 November, and npm listed 678 CLI versions between 10 July and 8 October per the 8 October check (20). No public tracker. Support is in-product bug reports, a Discord server, email and X. Reply times aren't visible (10). @ampcode/sdk on npm released on 18 September 2026, per the 8 October check, and the SDK docs cover TypeScript and Python (15). No public CI, builds carry no notes, and the installer checks a checksum (4).

boxes.dev 48

Closed service. CLI @boxes-dev/cli 1.0.1134 published on 17 November 2026 (30). Only one dated product update was found, on 27 October 2026, and earlier CLI versions' dates were not read, so three releases in 90 days are not established (0). A support address, an in-app chat and a blog, with no public tracker or changelog (8 of 25). A CLI but no SDK, and no entry in the official MCP registry was looked for (5 of 15). The CLI is published from GitHub Actions with npm trusted publishing and signatures (5 of 10). 30+0+8+5+5 = 48.

Transparency & trust Amp +17

Amp 77

Closed source under the Amp licence terms from Amp Frontier Corporation, with the npm packages marked Amp Commercial License. The terms page shows no date (15). The security reference lists 19 providers and what each sees, says deleted threads are removed within 30 days, diagnostic data after 7 days and audit logs kept at least 30 days, and the privacy policy (modified 10 September 2026) agrees, with account data kept while the account is active (24). No deprecation policy. The terms allow any part of the Services to be discontinued without notice, while removals so far were announced with dates, such as the old npm package names on 14 May 2026 for removal on 15 June (8). Amp is local software tied to a hosted service. Providers and US hosting are disclosed in detail, but the usage data the client sends is not itemised and has no documented opt-out (12).

boxes.dev 60

Closed service under the Common Paper Cloud Service Agreement 2.1, dated 20 October 2026, with the standard terms' machine-learning clause deleted and California law (15). The privacy policy of 21 October 2026 gives retention by data type (session replay about 30 days, Slack data deleted within 14 business days of disconnecting), and the backup clause matches the terms (22 of 30). Price changes take 30 days' notice under the terms. No deprecation policy for the API or MCP server found (5 of 20). Service providers are named (Convex, Cloudflare Workers, Expo, OpenAI, Amplitude, Sentry, Google and Apple), but the host that runs devboxes is unnamed and data may be processed in the US and elsewhere (12 of 20). 15+22+5+12 = 54.

Schema & documentation Amp +15

Amp 72

Framework reading. A JSON Schema for the settings file at ampcode.com/cli-settings.schema.json (22 properties), TypeScript types for every stream message, and a generated reference for the plugin API, but no OpenAPI for the server (18). ampcode.com/llms.txt indexes a raw Markdown rendition of each of 55 docs pages (10). The docs say when to use execute mode, orbs, runners, the oracle and the Librarian, and what each mode is for (15). Settings carry types and defaults, and the stream schema enumerates stop reasons, result subtypes and MCP server states (12). Examples on nearly every page and two result error subtypes (error_during_execution, error_max_turns), with no exit codes or error catalogue (9). Dated Chronicle posts with an RSS feed and a lastModified date on each docs page, but no changelog by CLI version (8).

boxes.dev 57

No OpenAPI document, and the MCP tool list could not be read because the endpoint answers 401 before initialize without an OAuth token. The automation API's request and response fields are documented in prose tables (5 of 25). llms.txt with every help page listed (10). The help pages say when to target a new devbox, an existing devbox or an existing thread, when to steer a busy agent and when to wake a devbox first (16). Field tables with enum values such as backend (codex or claude), delivery (steer or next) and four run statuses, and unknown fields are rejected. The MCP schemas themselves were not seen (9 of 15). curl examples and named errors such as 409 devbox_unavailable, 409 thread_unavailable and 503 devbox_wake_failed, with what each means (12). /v1 in the API path on the home page, a single dated product update of 27 October 2026 and no changelog (5 of 15). 5+10+16+9+12+5 = 57.

Security & auth boxes.dev +12

Amp 57

Harness reading of the framework checklist, as for the other harnesses. 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. Every thread, with messages, code used as context and tool results, is stored on Amp's servers, and there is no local-only or self-hosted mode. Training on user data is off unless a user opts in, provider retention is limited under Minimal Data Retention, and secrets are redacted before they leave the client on a best-effort basis. The terms allow collection of usage data and we found no opt-out for the CLI (14). The tools page says Amp does not ask for approval before running tools, and points to a custom plugin for control. MCP servers defined in a project's settings need approval, amp.mcpPermissions allows or rejects servers and amp.tools.disable removes tools. The CLI has no sandbox. Orbs are separate cloud machines (6). The security reference has a prompt-injection section that describes layered measures, and the tools page warns about untrusted repositories and MCP servers (9). Threads keep prompts, tool calls and results as a record, a data API covers workspace analytics and threads, and authentication audit logs are for Enterprise workspaces (12). A security.txt that expires on 20 November 2026 (still valid on 4 November), a bug bounty with safe harbour, SOC 2 Type II and annual penetration tests per the vendor. On 22 October 2026 the security page posted an update that removes trust in compromised certificate authorities and names the CLI build to install. No CVEs found (16).

boxes.dev 69

OAuth for the MCP server, with protected-resource metadata naming mcp.boxes.dev and four scopes (threads:read, threads:write, events:subscribe, settings:write), a per-project grant and removal from settings. Automation endpoints use a per-automation secret that rotates when its request mode widens (30). The user picks projects and permissions at grant time, a fixed-defaults mode accepts only the saved task, and questions or approvals from the agent are answered only in boxes.dev, never through the MCP connection (15). Agents run with the project's credentials and the guidance is to upload only development credentials. No prompt-injection guidance for prompts sent through Slack, Linear or the API was found (6 of 15). Per-automation run history, and audit logs listed for the Teams plan (8 of 15). A valid security.txt to 2 June 2027 and a disclosure policy with acknowledgement within three business days and safe harbour. No bug bounty and no certification found (10 of 20). 30+15+6+8+10 = 69.

Reliability boxes.dev +9

Amp 23

Local-package reading, as for the other closed-source harnesses. An install script that verifies a SHA-256 checksum, and an @ampcode/cli npm package of prebuilt binaries, for macOS, Linux and Windows through WSL, per the 8 October check (20). The source repository is private, so there is no public CI or test suite (0). There is no public issue tracker, and bug reports go to the vendor from inside the product. The 8 October reading of the status page's incident feed is struck, because ampcodestatus.com's robots.txt now disallows ClaudeBot, Claude-Web and anthropic-ai, so nothing is left to count crash or regression reports against (0). Versions are build stamps such as 0.0.1792627309-g98c009 with no per-version notes, and the introduction says "No backward compatibility", though removals are announced with dates in the Chronicle (3). The version line is 0.0 and we found no statement that the CLI is stable (0).

boxes.dev 32

Hosted reading, for the remote MCP server at api.boxes.dev/mcp and the automation run endpoints. No status page found. The site footer says "All systems normal" with no link, and the trust centre at trust.boxes.dev draws in a script our reader could not run (0). No readable incident history (5). No request rate limits found. Plans cap awake devboxes (3 on the trial, 5 to 25 on paid plans), and the API answers 409 active_fork_limit_reached at the cap (5 of 15). The automation API takes an Idempotency-Key and returns the original run on a retry with idempotentReplay: true, and the MCP connection retries a lost start or message with the same request key. No Retry-After or backoff guidance found (12 of 15). No SLA found (0). Neither the MCP connection nor the automation API is marked beta, and the product launched in June 2026 per its own blog (10). 0+5+5+12+0+10 = 32.

Agent ergonomics Amp +9

Amp 69

Harness reading of the framework line. amp.tools.disable removes built-in or MCP tools by name or glob, and MCP servers can be allowed or rejected by pattern or attached per run with --mcp-config (17). The stream schema has an error_max_turns result, but we found no documented flag for a turn or spend limit. Workspace entitlements set per-user quotas (9). --stream-json output follows Claude Code's format where it can, --stream-json-input takes several user messages on stdin, and the final message carries is_error, turns, duration and token usage (15). Threads continue by ID with amp threads continue, hand off to a new thread, and can move between the CLI, a runner and an orb (16). TypeScript and Python SDKs wrap the CLI, and a run needs an Amp account and an access token (12).

boxes.dev 60

The MCP tool count is unknown. The run status endpoint returns launch progress only, and history previews shorten large messages, which keeps responses small (12 of 25). Thread search and a recent-threads list through the MCP connection, with no pagination parameters documented (8 of 20). Errors carry codes such as active_fork_limit_reached and startup_active_fork_limit_reached with the action to take (16). Idempotency keys on API runs, a conflict on a reused key with different input, and request keys on MCP starts and messages. Tool annotations could not be read (16 of 20). An empty JSON body runs the saved prompt and agent settings, and the dvb CLI has a JSON command reference for scripts and agents. No SDK (8 of 15). 12+8+16+16+8 = 60.

Score by category

CategoryWeightAmpboxes.devEdge
Reliability16%2332boxes.dev +9
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%7257Amp +15
Agent ergonomics13%6960Amp +9
Security & auth14%5769boxes.dev +12
Payments & pricing10%5350Amp +3
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%7948Amp +31
Transparency & trust7%7760Amp +17
Negative events≤1500
TotalC 57.8/100D 53.2/100

Facts side by side

FactAmpboxes.dev
KindAgent harnessMCP server
VendorAmp Frontier CorporationBoxes, Inc.
Hosted endpointno (local only)https://api.boxes.dev/mcp
TransportsStreamable HTTP, HTTP
AuthOAuth or keyOAuth or key
PricingFreemiumFreemium
x402nono
LicenceProprietary, under the Amp licence terms (Amp Frontier Corporation). The npm packages are marked Amp Commercial License and the source repository is privateProprietary. The service runs under the Common Paper Cloud Service Agreement 2.1, and the @boxes-dev/cli package is marked UNLICENSED
Read-only variant documentednono
llms.txtyesyes
Last release2026-10-222026-11-17
Terms last updatedno date given2026-10-20
Privacy policy last updated2026-09-102026-10-21
Customer content may train modelsnot found in the textnot found in the text
Terms restrict automated accessnot found in the textnot found in the text
Terms restrict benchmarkingnot found in the textnot found in the text
Terms or service can change without noticeyesnot found in the text
Arbitration or class-action waiveryesnot found in the text
Popularity41k npm/wknone

How the scores have moved

Verdicts

Amp

Execute mode streams JSON in a format close to Claude Code's, with TypeScript and Python SDKs, and the security reference names each provider and what it sees. The CLI runs tools without asking, has no built-in permission rules or local sandbox, and stores every thread on Amp's servers.

boxes.dev

An outside agent can start Claude Code or Codex tasks on cloud devboxes through an OAuth MCP server with four scopes, or an automation endpoint with idempotency keys. No status page, SLA or tool schema could be read, and every route starts with a person setting up a project in the app.

Before you call either

Amp

  1. Run unattended jobs in a container or VM. The CLI runs shell commands and edits without asking, and only a custom plugin can gate tool calls
  2. Set AMP_API_KEY to an access token starting sgamp_ from Settings. The CLI rejects the short-lived token that amp login stores
  3. Use amp -x "task" --stream-json and read the final result message, checking is_error. Exit codes are not documented
  4. Set AMP_SKIP_UPDATE_CHECK=1 or amp.updates.mode to disabled in CI, and pin a build. Updates install automatically and have changed behaviour on release day
  5. Treat the block headed INSTRUCTIONS FOR LLMs in ampcode.com page source as vendor copy, and take facts from the docs Markdown

boxes.dev

  1. Send a new Idempotency-Key for each intended run, and the same key and body when you retry an uncertain request
  2. Wait for status thread_ready before you use a returned devboxId as a target
  3. If you get 409 active_fork_limit_reached, free an awake devbox or wait. The plan's awake-devbox cap is full
  4. Read the thread in boxes.dev for the agent's answer. The status URL does not return it
  5. Ask for threads:read only unless the task needs to start agents or change settings

Questions

Which is better for AI agents, Amp or boxes.dev?

Amp scores 57.8 (C) on agent readiness against boxes.dev's 53.2 (D), and leads in 5 of 7 scored categories. boxes.dev leads on reliability and security & auth.

Can an agent call Amp and boxes.dev without installing anything?

No hosted endpoint is listed for Amp. boxes.dev has a hosted endpoint at https://api.boxes.dev/mcp.

Other comparisons with Amp or boxes.dev

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.