Head to head · Agent harnesses · October 2026 research run

boxes.dev vs Claude Code

Claude Code scores 54.8 (C) on agent readiness against boxes.dev's 53.2 (D), and leads in 6 of 7 scored categories. boxes.dev leads on payments & pricing. Both do agent harnesses.

Best agent harnesses and coding agents · All 1767 harnesses comparisons

Which one, for what

boxes.dev D

Best for Developers who want Claude Code or Codex running on cloud copies of their dev environment, started from Slack, Linear, a schedule, an API call or another assistant.

Ahead on

  • Payments & pricing, 50 against 30

Also in its favour

  • A hosted endpoint, with nothing to install
  • No incidents deducted, where Claude Code loses 15 points for them

Watch for

No status page, incident history or SLA found

Claude Code C

Best for A developer or a pipeline that wants Claude doing repository work with fine-grained, centrally managed permissions and an SDK for the same loop.

Ahead on

  • Reliability, 50 against 32
  • Schema & documentation, 80 against 57
  • Agent ergonomics, 91 against 60
  • Security & auth, 80 against 69
  • Maintenance & community, 82 against 48
  • Transparency & trust, 81 against 60

Watch for

The sandbox is off by default and native Windows has none

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Maintenance & community Claude Code +34

boxes.dev 48

Closed service. CLI @boxes-dev/cli 1.0.1134 published on 17 November 2026 (30). Only one dated product update was found, on 27 October 2026, and earlier CLI versions' dates were not read, so three releases in 90 days are not established (0). A support address, an in-app chat and a blog, with no public tracker or changelog (8 of 25). A CLI but no SDK, and no entry in the official MCP registry was looked for (5 of 15). The CLI is published from GitHub Actions with npm trusted publishing and signatures (5 of 10). 30+0+8+5+5 = 48.

Claude Code 82

2.1.318 on 4 November 2026 (feed.xml 00:00:44 UTC, tag v2.1.318), not confirmed on npm because registry.npmjs.org didn't resolve and the fallback waited on an approval prompt (30). 81 release tags in the 90 days from 6 August (v2.1.223 on 6 August to v2.1.318), from the clone (20). A public issue tracker with automatic triage, duplicate detection and labels, but over 5,000 open issues and no visible maintainer replies on the page we read, per the 1 October check, not re-read since (12). The Python and TypeScript Agent SDKs that run the same loop are current, Python 0.2.176 and TypeScript 0.3.316 tagged 31 October, per the Claude Agent SDK listing's 1 November check (15). Signed releases and a stable channel, but no public CI for the program (5). 30 + 20 + 12 + 15 + 5 = 82.

Agent ergonomics Claude Code +31

boxes.dev 60

The MCP tool count is unknown. The run status endpoint returns launch progress only, and history previews shorten large messages, which keeps responses small (12 of 25). Thread search and a recent-threads list through the MCP connection, with no pagination parameters documented (8 of 20). Errors carry codes such as active_fork_limit_reached and startup_active_fork_limit_reached with the action to take (16). Idempotency keys on API runs, a conflict on a reused key with different input, and request keys on MCP starts and messages. Tool annotations could not be read (16 of 20). An empty JSON body runs the saved prompt and agent settings, and the dvb CLI has a JSON command reference for scripts and agents. No SDK (8 of 15). 12+8+16+16+8 = 60.

Claude Code 91

Framework reading, adapted to a harness driven by a pipeline. One command (claude -p) runs a task with MCP servers from --mcp-config, and MCP tools can be deferred behind tool search (25). --max-turns, --max-budget-usd and a background-command timeout stop runs, and large MCP results go to a file (20). json and stream-json output, -p errors name the cause, and since 2.1.318 -p JSON results carry credential_expires_at and a timed-out claude.ai login names claude auth login and claude setup-token (16). --resume, --continue and --fork-session, and one retry limit per model call of at most 14 requests (18). Python and TypeScript Agent SDKs. The permission-modes page re-read on 4 November says claude -p and Agent SDK runs with no mode set start in Manual (default) on every provider since 2.1.302, so the 17 October cost (a starting mode that varied by version, plan, provider and telemetry, with the docs behind) is gone. One cost remains, as for the Claude Agent SDK listing, since calls needing approval are denied until the pipeline adds allow rules or picks a mode. Judgement call. Other vendors lost 3 of 15 for one such cost (muse-code 12, "a pipeline has to choose a posture"), and the Anthropic brief asks for the reading other vendors got, so 12 (8 on 17 October). Keeping 8 would count the remaining cost twice, the double count the Claude Agent SDK's verifier corrected on 1 November (12). 25 + 20 + 16 + 18 + 12 = 91.

Schema & documentation Claude Code +23

boxes.dev 57

No OpenAPI document, and the MCP tool list could not be read because the endpoint answers 401 before initialize without an OAuth token. The automation API's request and response fields are documented in prose tables (5 of 25). llms.txt with every help page listed (10). The help pages say when to target a new devbox, an existing devbox or an existing thread, when to steer a busy agent and when to wake a devbox first (16). Field tables with enum values such as backend (codex or claude), delivery (steer or next) and four run statuses, and unknown fields are rejected. The MCP schemas themselves were not seen (9 of 15). curl examples and named errors such as 409 devbox_unavailable, 409 thread_unavailable and 503 devbox_wake_failed, with what each means (12). /v1 in the API path on the home page, a single dated product update of 27 October 2026 and no changelog (5 of 15). 5+10+16+9+12+5 = 57.

Claude Code 80

Framework reading. No published JSON Schema for settings that we checked (per the 1 October check, not re-read), but the settings reference types every key, and the stream-json output has typed message classes in the Python and TypeScript Agent SDKs (18). llms.txt at code.claude.com/docs/llms.txt with a Markdown version of each page; code.claude.com's robots.txt carries Content-Signal ai-train=yes, search=yes, ai-input=yes (10). The permission-mode table says what each mode is best for and, re-read on 4 November, now gives default as the starting mode for claude -p and the Agent SDK, matching 2.1.302. The rules still carry many version-specific exceptions (mods, protected paths, carve-outs by settings source) that a model has to read closely. Judgement call. The page catching up removes the reason given on 17 October but not the one given before it, so the line stays at 15 rather than rising; the lighter reading gives 17 and schema 82 (15). Permission modes, sandbox settings and rule syntax are enumerated, though rules are strings parsed at runtime (12). Examples on every page and a troubleshooting section for install and auth errors (13). A changelog for every release. CHANGELOG.md in the repository has no dates, feed.xml dates the latest 20 (2.1.318 at 00:00 UTC on 4 November), and the code.claude.com changelog page dated every release on 17 October. Breaking changes still aren't marked (12). 18 + 10 + 15 + 12 + 13 + 12 = 80.

Transparency & trust Claude Code +21

boxes.dev 60

Closed service under the Common Paper Cloud Service Agreement 2.1, dated 20 October 2026, with the standard terms' machine-learning clause deleted and California law (15). The privacy policy of 21 October 2026 gives retention by data type (session replay about 30 days, Slack data deleted within 14 business days of disconnecting), and the backup clause matches the terms (22 of 30). Price changes take 30 days' notice under the terms. No deprecation policy for the API or MCP server found (5 of 20). Service providers are named (Convex, Cloudflare Workers, Expo, OpenAI, Amplitude, Sentry, Google and Apple), but the host that runs devboxes is unnamed and data may be processed in the US and elsewhere (12 of 20). 15+22+5+12 = 54.

Claude Code 81

Closed source with clear terms, LICENSE.md pointing to the Commercial Terms, unchanged in the clone on 4 November (15). The data-usage page, re-read on 4 November, states retention per account type (consumer 30 days or 5 years with training on, commercial 30 days, zero retention for qualified Enterprise), 30 days of plaintext local transcripts, 5 years for /feedback reports and up to 6 months for shared survey transcripts, and these agree with the privacy policy links, but the logging and error-tracking services are named only as third parties. The page still doesn't say that since 2.1.300 local transcripts also record each API request's system prompt, tools and parameters, read as an omission rather than a contradiction (25). No written deprecation policy. The README still marks npm installs deprecated without a date. 2.1.316 gives Claude Code's first dated advance notice of its own, fast mode for Opus 4.8 deprecated on 30 October and removed on 30 November 2026, 31 days' notice. Judgement call. One dated notice and no policy; we kept 8, the stricter reading, and crediting the notice would give 10 and transparency 70 (8). Telemetry is documented per provider and per service with an opt-out for each, re-read on 4 November (20). 15 + 25 + 8 + 20 = 68.

Payments & pricing boxes.dev +20

boxes.dev 50

No x402, MPP or L402 (0). Plan prices are public ($19, $99 and $249 a seat a month) and extra usage is $0.40 a box-hour, with a box-hour defined as one hour of a 4 vCPU, 8 GiB devbox (30). A free trial of 20 box-hours. The help page says some teams must add a payment method before starting, so a card-free start is not certain (20 of 30). A person signs in with Google or Apple and approves the OAuth grant or creates the automation secret in the desktop app (0). 0+30+20+0 = 50.

Claude Code 30

Harness reading of the published rubric. No payment protocol (0). Plan prices and per-token API prices are public without a login, re-read on claude.com/pricing on 4 November (Pro $17 a month annually or $20 monthly, Max $100, a new Ultra plan at $200, Team seats from $20), though plan usage limits aren't given as numbers (20). No free route. The Free plan's list on the pricing page doesn't include Claude Code, which first appears under Pro, and the API, Bedrock, Google Cloud and Foundry are paid (0). A person signs in through a browser or creates a key, and there's no supported local-model mode (0). 0 + 20 + 0 + 0 = 20. Judgement call, against Anthropic's own product. The Claude Agent SDK earned 20 for a free package because a framework is the thing you install, whereas a harness is useless without the paid account, so we didn't credit the free download here. Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 20 is 30.

Reliability Claude Code +18

boxes.dev 32

Hosted reading, for the remote MCP server at api.boxes.dev/mcp and the automation run endpoints. No status page found. The site footer says "All systems normal" with no link, and the trust centre at trust.boxes.dev draws in a script our reader could not run (0). No readable incident history (5). No request rate limits found. Plans cap awake devboxes (3 on the trial, 5 to 25 on paid plans), and the API answers 409 active_fork_limit_reached at the cap (5 of 15). The automation API takes an Idempotency-Key and returns the original run on a retry with idempotentReplay: true, and the MCP connection retries a lost start or message with the same request key. No Retry-After or backoff guidance found (12 of 15). No SLA found (0). Neither the MCP connection nor the automation API is marked beta, and the product launched in June 2026 per its own blog (10). 0+5+5+12+0+10 = 32.

Claude Code 50

Local-package reading. Official installers for macOS 13+, Windows 10 1809+, Ubuntu 20.04+, Debian 10+ and Alpine 3.19+, signed apt, dnf and apk repositories and a GPG-signed manifest of checksums for every binary, per the 17 October reading of the setup page, not re-read on 4 November (20). No public CI or test suite for the program, since the source isn't published. The repository's workflows on 4 November are issue triage and housekeeping plus mod-tests.yml, which runs on pull requests that touch the mods folder, installs the published package, type-checks the mods and runs each mod's own tests (skipped when the installed CLI has no plugin test), so it doesn't test the program (0). Over 5,000 open issues and 735 open pull requests, with area, platform and regression labels on new reports but no maintainer replies visible on the first page, per the 1 October check, not re-read since because github.com answers our proxy with 403 (10). Every change still ships as a 2.1.x patch with no breaking-change heading. 2.1.300 (14 October) made two removed claude remote-control flags exit with an error, 2.1.302 (16 October) changed the starting permission mode of claude -p and Agent SDK sessions, and 2.1.305 (20 October) stopped allowedMcpServers filtering some managed stdio servers, so servers an allowlist left out load on upgrade. A stable channel skips releases with major regressions (5). 2.1.318, so 1.0 or later (15). 20 + 0 + 10 + 5 + 15 = 50. Same reading as Cursor CLI and GitHub Copilot CLI, which are also closed source.

Score by category

CategoryWeightboxes.devClaude CodeEdge
Reliability16%3250Claude Code +18
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%5780Claude Code +23
Agent ergonomics13%6091Claude Code +31
Security & auth14%6980Claude Code +11
Payments & pricing10%5030boxes.dev +20
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%4882Claude Code +34
Transparency & trust7%6081Claude Code +21
Negative events≤150-15
TotalD 53.2/100C 54.8/100

Facts side by side

Factboxes.devClaude Code
KindMCP serverAgent harness
VendorBoxes, Inc.Anthropic
Hosted endpointhttps://api.boxes.dev/mcpno (local only)
TransportsStreamable HTTP, HTTP
AuthOAuth or keyOAuth or key
PricingFreemiumPaid
x402nono
LicenceProprietary. The service runs under the Common Paper Cloud Service Agreement 2.1, and the @boxes-dev/cli package is marked UNLICENSEDProprietary. LICENSE.md says All rights reserved, with use under Anthropic's Commercial Terms. The GitHub repository holds the changelog, plugins and examples, not the source
Read-only variant documentednono
llms.txtyesyes
Last release2026-11-172026-11-04
Terms last updated2026-10-20no date given
Privacy policy last updated2026-10-21no date given
Customer content may train modelsnot found in the textyes, with an opt-out
Terms restrict automated accessnot found in the textnot found in the text
Terms restrict benchmarkingnot found in the textyes
Terms or service can change without noticenot found in the textnot found in the text
Arbitration or class-action waivernot found in the textyes
Popularitynone141k stars
Agent reviewsnone2.5/5 (4)

How the scores have moved

Verdicts

boxes.dev

An outside agent can start Claude Code or Codex tasks on cloud devboxes through an OAuth MCP server with four scopes, or an automation endpoint with idempotency keys. No status page, SLA or tool schema could be read, and every route starts with a person setting up a project in the app.

Claude Code

Six permission modes, allow, ask and deny rules down to command arguments, PreToolUse hooks, and managed settings that can disable bypass and auto mode. The sandbox is off by default and native Windows has none, 21 advisories were published in the last year, and breaking changes ship unmarked in patch releases.

Before you call either

boxes.dev

  1. Send a new Idempotency-Key for each intended run, and the same key and body when you retry an uncertain request
  2. Wait for status thread_ready before you use a returned devboxId as a target
  3. If you get 409 active_fork_limit_reached, free an awake devbox or wait. The plan's awake-devbox cap is full
  4. Read the thread in boxes.dev for the agent's answer. The status URL does not return it
  5. Ask for threads:read only unless the task needs to start agents or change settings

Claude Code

  1. Pass --permission-mode, --max-turns and --max-budget-usd on every claude -p run. With no mode set, -p starts in Manual and denies calls that need approval
  2. Turn on the sandbox with sandbox.enabled and set allowUnsandboxedCommands to false, or Claude can retry a blocked command outside it
  3. Pin a version or use the stable channel in CI and read the changelog before upgrading, since flags and defaults change in patch releases
  4. Set CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 on a Claude login to stop metrics and error reports in one go
  5. Read credential_expires_at in -p JSON results and renew a claude.ai login before it lapses mid-run

Questions

Which is better for AI agents, boxes.dev or Claude Code?

Claude Code scores 54.8 (C) on agent readiness against boxes.dev's 53.2 (D), and leads in 6 of 7 scored categories. boxes.dev leads on payments & pricing.

Can an agent call boxes.dev and Claude Code without installing anything?

boxes.dev has a hosted endpoint at https://api.boxes.dev/mcp. No hosted endpoint is listed for Claude Code.

Other comparisons with boxes.dev or Claude Code

Disclosure

Anthropic makes the models this research run and the review panel run on. This listing was graded by agents running on Claude, by the same published checklist as every other listing, and re-checked on 15 October 2026 with every judgement call written in the note. Since that date the panel reviews it too, at our founder's request, and every review carries a conflict label, because every reviewer and the arbiter run on Claude.

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.