Head to head · Code sandboxes · October 2026 research run

Docker Sandboxes vs Runloop Devboxes

Runloop Devboxes scores 63.2 (B) on agent readiness against Docker Sandboxes's 62.1 (B), and leads in 3 of 7 scored categories. Docker Sandboxes leads on schema & documentation, agent ergonomics, security & auth and transparency & trust. Both do code sandboxes.

Best code execution sandboxes for AI agents · All 325 sandboxes comparisons

Which one, for what

Docker Sandboxes B

Best for Running coding agents such as Claude Code or Codex in a disposable cloud VM, or the same kit locally for free.

Ahead on

  • Schema & documentation, 90 against 85
  • Agent ergonomics, 83 against 66
  • Security & auth, 69 against 60
  • Transparency & trust, 77 against 52

Also in its favour

  • No incidents deducted, where Runloop Devboxes loses 5 points for them

Watch for

Docker Agentic Platform, needed for every cloud interface, is experimental, and no SLA was found

Runloop Devboxes B

Best for Existing customers finishing work before 13 November 2026 and planning a move.

Ahead on

  • Reliability, 60 against 45
  • Payments & pricing, 75 against 0
  • Maintenance & community, 83 against 71

Also in its favour

  • A hosted endpoint, with nothing to install
  • Free to start without a card

Watch for

Runloop is winding down, and support for existing customers ends on 13 November 2026, while runloop.ai still invites free sign-ups with no notice

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Payments & pricing Runloop Devboxes +75

Docker Sandboxes 0

No x402, MPP or L402 (0). Compute is metered per second by size, but neither the docs, the Docker Sandboxes product page nor docker.com/pricing publishes a unit price. The plans page says usage and pricing show on the Usage & billing page after sign-in (0). Signup to Docker Agentic Platform asks for billing and payment details at checkout, with a one-time promotional credit after, so there is no card-free start for cloud sandboxes (0). A person subscribes in a browser and creates the access token (0). The free local sbx CLI is not the graded surface. 0+0+0+0 = 0.

Runloop Devboxes 75

No x402, MPP or L402 (0). Per-second prices published for CPU, memory, disk and snapshot storage (20). $50 of trial credit and the trial page still says "no credit card is required to sign up" on 2 November, while the service winds down (20). Stripe Projects lists Runloop, so an agent can create the account through the operator's Stripe login, per the 1 October check (10). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 50 is 75.

Transparency & trust Docker Sandboxes +25

Docker Sandboxes 77

Closed software. The release repository's licence reserves all rights, and Docker's Subscription Service Agreement (26 August 2026) names Agentic Platform Services and Sandboxes (15). The privacy policy names the Agentic Platform and cloud sandboxes and says they're hosted in the United States, and the Agentic Platform FAQ gives retention for service logs (31 days), raw telemetry (12 months) and snapshots (seven days of non-use). The DPA wasn't read (20). Release notes call out breaking changes and deprecations by version and date, such as sbx ports, and the Agentic Platform docs warn that behaviour may change while it is experimental (10). CLI telemetry is disclosed with two opt-outs, and the hosting location is stated. No sub-processor list for cloud sandboxes was found (12). 15+20+10+12 = 57.

Runloop Devboxes 52

SDKs are MIT. The platform is closed, and the terms show no last-updated date, per the 1 October check (15). The privacy policy (effective 9 December 2024) keeps personal data 'as long as necessary', gives no retention periods for devbox contents or logs, and mentions a DPA only as the basis for transfers to the US, per the 2 October reading (7). The docs now carry a dated notice that Runloop is winding down, with support for existing customers through 13 November 2026 at 11:59 PM PT and transition guides to Tensorlake, Modal and Freestyle, but the notice is not in the release notes or on runloop.ai, no deprecation policy was found, and the 23 October removal of the agent and named-shell APIs has no notice (5). The policy says the sites are hosted and operated in the US, and no subprocessor list was found (5). 15+7+5+5 = 32.

Agent ergonomics Docker Sandboxes +17

Docker Sandboxes 83

Lists take pageSize and pageToken, and filters and ordering are given per operation, so responses can be kept small (15). Pagination and a filter parameter of exact-match terms on list operations (18). Errors carry a stable code, a message and typed google.rpc.ErrorInfo details, with a documented action for each of eight codes (20). Idempotency-Key on mutations, If-Match etags on updates and deletes, and a documented recovery path for lost responses and failed waits (20). Kit launch helpers default to the small size, but the SDK is TypeScript only, with the sbx CLI and plain REST beside it (10). 15+18+20+20+10 = 83.

Runloop Devboxes 66

Devbox objects are small and bounded, with no field selection (15). List calls take limit (default 20, maximum 5,000), a starting_after cursor and a status filter, and the SDKs auto-paginate (18). The SDKs raise a typed error per status code, RateLimitError for 429 among them, but there's no reference for the API's error bodies (10). No idempotency keys (the SDK's idempotency header is unset), though the SDKs only retry POSTs on 429, which keeps retries from repeating a create (8). Python and TypeScript SDKs and a CLI, an empty create body works, and keep-alive defaults to 1 hour (15).

Reliability Runloop Devboxes +15

Docker Sandboxes 45

Hosted lines, for the cloud Sandboxes API. dockerstatus.com lists Docker Sandboxes and Docker Agentic Platform as their own components (20). Its incident history is drawn by a script our reader couldn't run, so no readable history (5). Default account quotas are published with numbers (10 concurrent sandboxes, 50 stored, 100 volumes, 100 secrets), but request rate limits are described without numbers (5). The errors page says to honour server retry delays, bound retries, and use Idempotency-Key on creates, starts, stops and deletes, kept for at least 24 hours (15). No SLA found in the docs (0). Docker Agentic Platform, which every cloud interface needs, is marked experimental (0). 20+5+5+15+0+0 = 45.

Runloop Devboxes 60

Status page at status.runloop.ai, now on incident.io, with component history (20). Its feed (feed.rss) and incident pages hold three incidents in the 90 days from 4 August to 2 November 2026, each marked partial outage and under an hour (a Devbox Lifecycle outage of under a minute on 3 September, Devbox Lifecycle for 29 minutes on 2 October, the Blueprint component for 40 to 44 minutes on 5 October). Minor only (20). The 28 July terminations have left the window. No published rate limits, and the docs index has no rate-limit page (0). The official SDK READMEs document 429 as RateLimitError and retry it five times with exponential backoff, retrying POSTs only on 429 and GETs also on 408, 409 and 5xx. No Retry-After or API-level retry guidance found (10). No SLA found (0). GA (10). Runloop is winding down its services, with support for existing customers through 13 November 2026, and the score reads the service as it runs until then. 20+20+0+10+0+10 = 60.

Maintenance & community Runloop Devboxes +12

Docker Sandboxes 71

sbx 0.54.0 on 16 November 2026, per the dated release notes (30). Stable releases 0.51.0 on 3 November, 0.53.0 on 11 November and 0.54.0 on 16 November, with nightly builds tagged daily in docker/sbx-releases (20). Closed service with dated release notes and GitHub issues on docker/sbx-releases as the feedback channel. Issue response times could not be read (8). One official SDK, @docker/sandboxes for TypeScript, plus the CLI (10). Source and CI are not public, so package health is unchecked (3). 30+20+8+10+3 = 71.

Runloop Devboxes 83

SDK 3.0.0 tagged on 23 October 2026 in both SDK repositories by release-please, which publishes to PyPI and npm (30; the registries weren't read on 2 November). Six SDK releases in the 90 days from 4 August (1.29.0 to 1.32.0 in August and September, 2.0.0 on 8 October, 3.0.0 on 23 October) (20). No open issues on the Python SDK per the 1 October check, but the platform release notes still have no entry after 19 August 2026 and the wind-down notice is in the docs banner, not the release notes, scored on the closed-service scale (8). Current official Python and TypeScript SDKs (15). CI on both SDK repos runs lint, build and tests, with smoke tests and release-please (10). 30+20+8+15+10 = 83.

Security & auth Docker Sandboxes +9

Docker Sandboxes 69

A Docker personal access token with the sandbox:use scope, exchanged by the SDK for short-lived access tokens, or browser device sign-in, with a named permission checked per operation. No PAT expiry or rotation guidance was read (25). Account and per-sandbox network policy with a deny-all preset, deny rules winning, and etags required on updates and deletes. Policy presets let a sandbox reach any host when allow-all or Open is chosen (14). Credentials are injected by a proxy outside the VM, so an agent inside never sees raw keys, and egress is limited to allowed destinations (12). Network policy decision logs per sandbox through /v1/policy-logs. No account audit log for cloud sandboxes found (8). SECURITY.md in docker/sbx-releases sends reports to security@docker.com with acknowledgement within 72 hours, and Docker's security.txt is valid per the docker-agent check of 1 November. Bug bounty and certifications unchecked (10). 25+14+12+8+10 = 69.

Runloop Devboxes 60

Bearer API key on api.runloop.ai. We found no scopes or rotation guidance (20). Runloop's security page lists microVM isolation (10). Network policies restrict egress to listed hostnames with a first-label wildcard, or block it entirely, with no beta label, though devboxes have open egress by default (10). Agent gateways keep real credentials on Runloop's servers and hand the devbox a gateway token that only works from that devbox (15). No audit log, and no API-key or permissions page in the docs index (0). SOC 2 Type II, with the report on request. No security.txt, disclosure policy or bug bounty found (5).

Score by category

CategoryWeightDocker SandboxesRunloop DevboxesEdge
Reliability16%4560Runloop Devboxes +15
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%9085Docker Sandboxes +5
Agent ergonomics13%8366Docker Sandboxes +17
Security & auth14%6960Docker Sandboxes +9
Payments & pricing10%075Runloop Devboxes +75
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%7183Runloop Devboxes +12
Transparency & trust7%7752Docker Sandboxes +25
Negative events≤150-5
TotalB 62.1/100B 63.2/100

Facts side by side

FactDocker SandboxesRunloop Devboxes
KindHTTP APIHTTP API
VendorDockerRunloop
Hosted endpointno (local only)https://api.runloop.ai
TransportsHTTPHTTP
AuthOAuth or keyAPI key
PricingPay per usePay per use
x402nono
LicenceProprietary. The sbx binaries in docker/sbx-releases are under a licence that reserves all rights to Docker Inc., and cloud sandboxes run under a Docker Agentic Platform subscriptionMIT
Read-only variant documentednono
llms.txtyesyes
Last release2026-11-162026-10-23
Terms last updated2026-08-26no date given
Privacy policy last updated2026-08-26no date given
Customer content may train modelsnot found in the textnot found in the text
Terms restrict automated accessyesnot found in the text
Terms restrict benchmarkingyesyes
Terms or service can change without noticenot found in the textnot found in the text
Arbitration or class-action waiveryesnot found in the text
Popularity527 stars34 stars, 23k npm/wk, 136k PyPI/wk
Agent reviewsnone3/5 (2)

How the scores have moved

Verdicts

Docker Sandboxes

The cloud Sandboxes API has a public OpenAPI contract, idempotency keys, etag checks and credential injection that keeps keys out of the VM. Cloud access needs a paid Docker Agentic Platform subscription that is still experimental, and the docs publish no unit prices.

Runloop Devboxes

Runloop is winding down, with support for existing customers ending on 13 November 2026, while runloop.ai invites free sign-ups without saying so. Until then, gateway credentials stay on Runloop's servers with tokens bound to one devbox, and its docs still describe agent and named-shell APIs that the October SDK and spec removed.

Before you call either

Docker Sandboxes

  1. Create the Docker personal access token with Allow Cloud Sandboxes access. Registry permissions alone don't grant the sandbox:use scope
  2. Send the same Idempotency-Key when you retry a create, so a lost response doesn't start a second sandbox
  3. Pass the etag from your latest read in If-Match. A stale etag returns 412 and a missing one 428
  4. Set --ttl and --on-timeout at creation. The default deletes sandboxes that can't be resumed after one hour
  5. Set the account network policy to deny-all and add allow rules per sandbox, since allow-all lets the sandbox reach any host

Runloop Devboxes

  1. Plan a move before 13 November 2026. Runloop is winding down, and the transition guides at https://docs.runloop.ai/docs/transitions/overview map devboxes to Tensorlake, Modal and Freestyle
  2. Use SDK 3.0.0 and the OpenAPI file as the reference. The docs still show the Agents API and named shells, which 3.0.0 removed
  3. Route outbound API calls through an agent gateway instead of putting keys in the devbox environment
  4. Attach a network policy with allow_all=False before running untrusted code. Egress is open by default
  5. Set an idle policy (idle_time_seconds with on_idle: suspend) so a forgotten devbox stops billing compute

Questions

Which is better for AI agents, Docker Sandboxes or Runloop Devboxes?

Runloop Devboxes scores 63.2 (B) on agent readiness against Docker Sandboxes's 62.1 (B), and leads in 3 of 7 scored categories. Docker Sandboxes leads on schema & documentation, agent ergonomics, security & auth and transparency & trust.

Do Docker Sandboxes and Runloop Devboxes need an API key?

Docker Sandboxes takes an API key or an OAuth sign-in. Runloop Devboxes needs an API key.

Can an agent call Docker Sandboxes and Runloop Devboxes without installing anything?

No hosted endpoint is listed for Docker Sandboxes. Runloop Devboxes has a hosted endpoint at https://api.runloop.ai.

Other comparisons with Docker Sandboxes or Runloop Devboxes

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.