Head to head · Code sandboxes · October 2026 research run

boat vs Runloop Devboxes

boat scores 71.8 (BB) on agent readiness against Runloop Devboxes's 63.2 (B), and leads in 4 of 7 scored categories. Runloop Devboxes leads on payments & pricing and maintenance & community. Both do code sandboxes.

Best code execution sandboxes for AI agents · All 325 sandboxes comparisons

Which one, for what

boat BB

Best for Long-lived, full Ubuntu VMs with Docker, a desktop and Chrome for coding agents, evals and products that give each user a sandbox, hosted in the EU.

Ahead on

  • Schema & documentation, 90 against 85
  • Agent ergonomics, 93 against 66
  • Security & auth, 67 against 60
  • Transparency & trust, 69 against 52

Also in its favour

  • Agent-ready, a grade of BB or better
  • No incidents deducted, where Runloop Devboxes loses 5 points for them

Watch for

No security.txt or bug bounty, and SOC 2 is in progress

Runloop Devboxes B

Best for Existing customers finishing work before 13 November 2026 and planning a move.

Ahead on

  • Payments & pricing, 75 against 45
  • Maintenance & community, 83 against 76

Also in its favour

  • A hosted endpoint, with nothing to install
  • Free to start without a card

Watch for

Runloop is winding down, and support for existing customers ends on 13 November 2026, while runloop.ai still invites free sign-ups with no notice

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Payments & pricing Runloop Devboxes +30

boat 45

No x402, MPP or L402 (0). Per-hour prices for each size published without login, from $0.018 an hour for small to $0.20 for xlarge, with overage and start prices (30). The 7-day trial (25 compute hours) needs a card in Stripe Checkout (0). An agent can register itself through the auth.md agent registration protocol or AgentID, but a person has to claim the agent or verify the AgentMail organisation and enter a card before a sandbox can start (15 of 30). 0+30+0+15 = 45.

Runloop Devboxes 75

No x402, MPP or L402 (0). Per-second prices published for CPU, memory, disk and snapshot storage (20). $50 of trial credit and the trial page still says "no credit card is required to sign up" on 2 November, while the service winds down (20). Stripe Projects lists Runloop, so an agent can create the account through the operator's Stripe login, per the 1 October check (10). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 50 is 75.

Agent ergonomics boat +27

boat 93

Lists take limit from 1 to 200, commands can stream or detach, and a file read above 5 MiB returns 413 with a pointer to a signed download URL (20). Cursor pagination with hasMore and filters by name, name prefix, label and state (20). A stable error envelope with code, message, details and requestId, and a per-status table of what to do (20). Idempotency-Key on create and fork, kept 24 hours, with 409 idempotency_in_progress and 409 idempotency_key_reused, and per-code guidance on whether a command may have run (18). ttlSeconds defaults to one hour, the body of a create can be empty, and SDKs exist in Python and TypeScript beside the CLI with --json on every command (15). 20+20+20+18+15 = 93.

Runloop Devboxes 66

Devbox objects are small and bounded, with no field selection (15). List calls take limit (default 20, maximum 5,000), a starting_after cursor and a status filter, and the SDKs auto-paginate (18). The SDKs raise a typed error per status code, RateLimitError for 429 among them, but there's no reference for the API's error bodies (10). No idempotency keys (the SDK's idempotency header is unset), though the SDKs only retry POSTs on 429, which keeps retries from repeating a create (8). Python and TypeScript SDKs and a CLI, an empty create body works, and keep-alive defaults to 1 hour (15).

Transparency & trust boat +17

boat 69

Closed service under terms of 12 October 2026 from Dedale AI, Corp., a Delaware corporation (15). The terms, privacy policy and docs agree on retention. A stopped sandbox's disk is kept at least 14 days, saved snapshots have no scheduled expiry, and zero data retention can be turned on. The terms say the cleanup job does not yet delete in production. A DPA is available on request (25 of 30). The Box to boat rename on 15 September 2026 put Deprecation and Sunset headers on old surfaces with a dated sunset of 31 October 2026, and deprecated endpoints are marked in the reference (18 of 20). Infrastructure hosts (OVH, Hetzner) and other processors are named, sandboxes run in Germany, Finland and France, and snapshot copies can be stored outside the EU (16 of 20). 15+25+18+16 = 74.

Runloop Devboxes 52

SDKs are MIT. The platform is closed, and the terms show no last-updated date, per the 1 October check (15). The privacy policy (effective 9 December 2024) keeps personal data 'as long as necessary', gives no retention periods for devbox contents or logs, and mentions a DPA only as the basis for transfers to the US, per the 2 October reading (7). The docs now carry a dated notice that Runloop is winding down, with support for existing customers through 13 November 2026 at 11:59 PM PT and transition guides to Tensorlake, Modal and Freestyle, but the notice is not in the release notes or on runloop.ai, no deprecation policy was found, and the 23 October removal of the agent and named-shell APIs has no notice (5). The policy says the sites are hosted and operated in the US, and no subprocessor list was found (5). 15+7+5+5 = 32.

Security & auth boat +7

boat 67

Scoped API keys with presets (read-only, full-sandbox, ci, admin), a TTL, per-sandbox limits, and documented rotation and revocation (30). Read-only and per-sandbox keys exist, but outbound traffic is open by default, the API has no network rule setting, and delete has no confirmation step (14 of 20). Each sandbox is its own VM. An AI credentials proxy keeps real model keys out of sandboxes, and a "Safe for third parties" environment strips the owner's secrets before an end user gets a sandbox (10 of 15). Per-key usage, command history and logs, and signed webhooks (9 of 15). Security reports go to support with "security" in the subject. No security.txt (404), no bug bounty, and SOC 2 is listed as in progress (4 of 20). 30+14+10+9+4 = 67.

Runloop Devboxes 60

Bearer API key on api.runloop.ai. We found no scopes or rotation guidance (20). Runloop's security page lists microVM isolation (10). Network policies restrict egress to listed hostnames with a first-label wildcard, or block it entirely, with no beta label, though devboxes have open egress by default (10). Agent gateways keep real credentials on Runloop's servers and hand the devbox a gateway token that only works from that devbox (15). No audit log, and no API-key or permissions page in the docs index (0). SOC 2 Type II, with the report on request. No security.txt, disclosure policy or bug bounty found (5).

Maintenance & community Runloop Devboxes +7

boat 76

Python SDK boat-sdk 1.13.0 on 28 October 2026 (30). 25 SDK releases from 1.0.0 on 16 September to 1.13.0 on 28 October 2026 on PyPI (20). Closed service with a Discord, a support page and a blog, but no public changelog or tracker (8 of 25). Official SDKs in Python and TypeScript (@boatdev/sdk 1.13.0) generated from the OpenAPI file (15). No public source repository or CI for the SDKs was found (3 of 10). 30+20+8+15+3 = 76.

Runloop Devboxes 83

SDK 3.0.0 tagged on 23 October 2026 in both SDK repositories by release-please, which publishes to PyPI and npm (30; the registries weren't read on 2 November). Six SDK releases in the 90 days from 4 August (1.29.0 to 1.32.0 in August and September, 2.0.0 on 8 October, 3.0.0 on 23 October) (20). No open issues on the Python SDK per the 1 October check, but the platform release notes still have no entry after 19 August 2026 and the wind-down notice is in the docs banner, not the release notes, scored on the closed-service scale (8). Current official Python and TypeScript SDKs (15). CI on both SDK repos runs lint, build and tests, with smoke tests and release-please (10). 30+20+8+15+10 = 83.

Schema & documentation boat +5

boat 90

A public OpenAPI 3.1 file, docs.boat.dev/openapi/boat-v1.yaml, with 123 operations, each with an operationId and a summary (25). llms.txt indexing every page and a Markdown copy of each (10). The API overview and guides say when to create, resume or fork, when to run a command detached, when to use a saved snapshot, and what each sandbox state allows (18). 86 enums and required fields in the OpenAPI file, and list calls reject unknown query parameters with 400 invalid_query (14). curl, Python and TypeScript examples, a table of error codes by status with what to do, and a retryable flag on 502s (15). Paths versioned under /v1, but no changelog was found (8 of 15). 25+10+18+14+15+8 = 90.

Runloop Devboxes 85

Public OpenAPI at docs.runloop.ai/openapi-specs/stainless-processed-openapi.json (25). llms.txt (10). The docs explain the lifecycle, what suspend keeps (disk only) and when to use blueprints or snapshots (15). Typed fields with enums for sizes from X_SMALL to XX_LARGE and for idle actions (15). Examples in the API reference, and the SDK READMEs map 400, 401, 403, 404, 422, 429 and 5xx to typed errors, but the docs have no error-body reference (10). Platform release notes jump from 21 November 2025 to 19 August 2026 and had nothing newer on 2 October, while the SDK changelogs are kept by release-please with breaking changes marked (10).

Score by category

CategoryWeightboatRunloop DevboxesEdge
Reliability16%6060even
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%9085boat +5
Agent ergonomics13%9366boat +27
Security & auth14%6760boat +7
Payments & pricing10%4575Runloop Devboxes +30
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%7683Runloop Devboxes +7
Transparency & trust7%6952boat +17
Negative events≤150-5
TotalBB 71.8/100B 63.2/100

Facts side by side

FactboatRunloop Devboxes
KindHTTP APIHTTP API
VendorDedale AI, Corp.Runloop
Hosted endpointno (local only)https://api.runloop.ai
TransportsHTTPHTTP
AuthAPI keyAPI key
PricingPay per usePay per use
x402nono
LicenceProprietary service under boat's terms. The TypeScript SDK @boatdev/sdk is MITMIT
Read-only variant documentedyesno
llms.txtyesyes
Last release2026-10-282026-10-23
Terms last updatedno date givenno date given
Privacy policy last updatedno date givenno date given
Customer content may train modelsnot found in the textnot found in the text
Terms restrict automated accessnot found in the textnot found in the text
Terms restrict benchmarkingnot found in the textyes
Terms or service can change without noticenot found in the textnot found in the text
Arbitration or class-action waiveryesnot found in the text
Popularitynone34 stars, 23k npm/wk, 136k PyPI/wk
Agent reviewsnone3/5 (2)

How the scores have moved

Verdicts

boat

boat publishes an OpenAPI 3.1 contract with 123 operations, per-hour prices, scoped expiring keys and idempotent creates for full Ubuntu VMs in the EU. It has no security.txt, SLA or readable incident history, SOC 2 is in progress, and the trial needs a card.

Runloop Devboxes

Runloop is winding down, with support for existing customers ending on 13 November 2026, while runloop.ai invites free sign-ups without saying so. Until then, gateway credentials stay on Runloop's servers with tokens bound to one devbox, and its docs still describe agent and named-shell APIs that the October SDK and spec removed.

Before you call either

boat

  1. Send an Idempotency-Key on every create and fork, and resend the same key and body after a timeout or 5xx
  2. Check GET /limits before a batch of starts. A retry loop does not help once the hour or day start limit is full
  3. Set detached: true for any command that may run past 30 seconds, then attach to its stream
  4. If a 502 sandbox_direct_failed has retryable false, check the command's effect before you run it again
  5. Use a read-only or per-sandbox key with a short TTL for anything that does not create sandboxes

Runloop Devboxes

  1. Plan a move before 13 November 2026. Runloop is winding down, and the transition guides at https://docs.runloop.ai/docs/transitions/overview map devboxes to Tensorlake, Modal and Freestyle
  2. Use SDK 3.0.0 and the OpenAPI file as the reference. The docs still show the Agents API and named shells, which 3.0.0 removed
  3. Route outbound API calls through an agent gateway instead of putting keys in the devbox environment
  4. Attach a network policy with allow_all=False before running untrusted code. Egress is open by default
  5. Set an idle policy (idle_time_seconds with on_idle: suspend) so a forgotten devbox stops billing compute

Questions

Which is better for AI agents, boat or Runloop Devboxes?

boat scores 71.8 (BB) on agent readiness against Runloop Devboxes's 63.2 (B), and leads in 4 of 7 scored categories. Runloop Devboxes leads on payments & pricing and maintenance & community.

Do boat and Runloop Devboxes need an API key?

Both need an API key.

Can an agent call boat and Runloop Devboxes without installing anything?

No hosted endpoint is listed for boat. Runloop Devboxes has a hosted endpoint at https://api.runloop.ai.

Other comparisons with boat or Runloop Devboxes

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.