Head to head · Code sandboxes · October 2026 research run

Cloudflare Sandbox SDK vs Runloop Devboxes

Cloudflare Sandbox SDK scores 71.6 (BB) on agent readiness against Runloop Devboxes's 63.2 (B), and leads in 4 of 7 scored categories. Runloop Devboxes leads on schema & documentation and payments & pricing. Both do code sandboxes.

Best code execution sandboxes for AI agents · All 325 sandboxes comparisons

Which one, for what

Cloudflare Sandbox SDK BB

Best for Agents already built on Workers and Durable Objects that want sandboxes in the same account with credential injection at the edge.

Ahead on

  • Reliability, 90 against 60
  • Security & auth, 73 against 60
  • Maintenance & community, 90 against 83
  • Transparency & trust, 86 against 52

Also in its favour

  • Agent-ready, a grade of BB or better
  • No key needed to call it
  • Open source

Watch for

No hosted API. You deploy and secure a Worker before an agent can call anything, and the starter has no auth

Runloop Devboxes B

Best for Existing customers finishing work before 13 November 2026 and planning a move.

Ahead on

  • Schema & documentation, 85 against 77
  • Payments & pricing, 75 against 45

Also in its favour

  • A hosted endpoint, with nothing to install
  • Free to start without a card

Watch for

Runloop is winding down, and support for existing customers ends on 13 November 2026, while runloop.ai still invites free sign-ups with no notice

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Transparency & trust Cloudflare Sandbox SDK +34

Cloudflare Sandbox SDK 86

The SDK and its container image are Apache-2.0. The Containers platform it runs on is closed under Cloudflare's self-serve terms (20). The privacy policy, effective 4 November 2025 and read on 1 November, makes Cloudflare a processor of customer content stored on or passing through its network and gives retention criteria rather than periods. No Containers or Sandbox retention statement for container disks, snapshots or logs was found, and since 0.12.11 the SDK keeps command text and output out of Workers Logs (16). A dated notice that 0.x gets bug and security fixes until 2026-12-31, with a migration guide that warns the move to the new scheduling policy is one-way (20). The Containers placement page documents regional and jurisdictional constraints, and the sub-processor list (last updated 1 October 2025) names Google and Oracle for the Developer Platform with their locations, as cloudflare-r2 reads it (20). 20 + 16 + 20 + 20 = 76.

Runloop Devboxes 52

SDKs are MIT. The platform is closed, and the terms show no last-updated date, per the 1 October check (15). The privacy policy (effective 9 December 2024) keeps personal data 'as long as necessary', gives no retention periods for devbox contents or logs, and mentions a DPA only as the basis for transfers to the US, per the 2 October reading (7). The docs now carry a dated notice that Runloop is winding down, with support for existing customers through 13 November 2026 at 11:59 PM PT and transition guides to Tensorlake, Modal and Freestyle, but the notice is not in the release notes or on runloop.ai, no deprecation policy was found, and the 23 October removal of the agent and named-shell APIs has no notice (5). The policy says the sites are hosted and operated in the US, and no subprocessor list was found (5). 15+7+5+5 = 32.

Reliability Cloudflare Sandbox SDK +30

Cloudflare Sandbox SDK 90

Scored on the SDK checklist, since the listing is a library. Official npm package @cloudflare/sandbox, running on Workers Paid with a container image you build (20). Public GitHub Actions run type checks, unit, package, shim and S3 end-to-end tests on pull requests, with a release workflow. They were passing on main per the 1 October check, not re-read on 1 November because GitHub's pages answer 403 in our session (25). Open issues weren't readable on 1 November. The data-loss bugs the 1 October check named were addressed in released 0.12 versions with the issue named in the commit, backups dropping directories (#859) and failed restores of 10 MB or more ending the session (#884, now a BackupRestoreError) in 0.12.11, and tunnels.get() (#874) in 0.12.12, among 12 issues closed by name on the 0.x branch since August. allowedHosts failing closed (#844) wasn't found among them (15 of 25). Release notes flag breaking changes, and 1.0.0 shipped as a major with a migration guide and an API map from 0.12 (15). 1.0.3 is current, and 0.x gets fixes until 31 December 2026 (15). Cloudflare's own status page isn't scored on these lines. 20 + 25 + 15 + 15 + 15 = 90.

Runloop Devboxes 60

Status page at status.runloop.ai, now on incident.io, with component history (20). Its feed (feed.rss) and incident pages hold three incidents in the 90 days from 4 August to 2 November 2026, each marked partial outage and under an hour (a Devbox Lifecycle outage of under a minute on 3 September, Devbox Lifecycle for 29 minutes on 2 October, the Blueprint component for 40 to 44 minutes on 5 October). Minor only (20). The 28 July terminations have left the window. No published rate limits, and the docs index has no rate-limit page (0). The official SDK READMEs document 429 as RateLimitError and retry it five times with exponential backoff, retrying POSTs only on 429 and GETs also on 408, 409 and 5xx. No Retry-After or API-level retry guidance found (10). No SLA found (0). GA (10). Runloop is winding down its services, with support for existing customers through 13 November 2026, and the score reads the service as it runs until then. 20+20+0+10+0+10 = 60.

Payments & pricing Runloop Devboxes +30

Cloudflare Sandbox SDK 45

No x402 or other machine payment on anything the SDK exposes (0). Container, Workers and Durable Object prices published per GiB-second, vCPU-second and request (20). Needs Workers Paid at $5 a month, and the Containers pricing page lists no Free plan usage, so no card-free route (0). Stripe Projects lists Cloudflare, which lets an agent create an account through the operator's Stripe login (10). 0 + 20 + 0 + 10 = 30. Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 30 is 45.

Runloop Devboxes 75

No x402, MPP or L402 (0). Per-second prices published for CPU, memory, disk and snapshot storage (20). $50 of trial credit and the trial page still says "no credit card is required to sign up" on 2 November, while the service winds down (20). Stripe Projects lists Runloop, so an agent can create the account through the operator's Stripe login, per the 1 October check (10). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 50 is 75.

Security & auth Cloudflare Sandbox SDK +13

Cloudflare Sandbox SDK 73

There's no hosted credential to score. Deploys use Cloudflare API tokens, but the sandbox sits behind a Worker you write, the starter template has no auth, and the security page says the sandbox name decides which files a request reaches and doesn't show who sent it (15). Each sandbox is a separate VM with its own filesystem, process space and network stack (10). Outbound traffic can be cut off with enableInternet = false or limited with a deny-by-default allowedHosts list and deniedHosts, all GA, though internet access is on by default (10). Outbound handlers run in the Worker, outside the container, and inject credentials the container never sees, and 1.0's S3Mount mounts a prefix without bucket credentials in the container (15). Sandbox labels for analytics, and since 0.12.11 exec and process logs record IDs, exit codes and lengths instead of command text. Account audit logs weren't read (5). security.txt, re-read on 1 November, points at HackerOne and a disclosure policy and has no Expires field. Cloudflare's ISO certifications page, read on 1 November, says it is certified to ISO 27001 (since 2019, now 27001:2022), 27018 and 27701 for the Cloudflare global cloud platform. 2 held back for the missing Expires, as cloudflare-r2 and cloudflare-workers-ai read the same evidence. OSV lists no advisory for @cloudflare/sandbox (18). 15 + 20 + 15 + 5 + 18 = 73.

Runloop Devboxes 60

Bearer API key on api.runloop.ai. We found no scopes or rotation guidance (20). Runloop's security page lists microVM isolation (10). Network policies restrict egress to listed hostnames with a first-label wildcard, or block it entirely, with no beta label, though devboxes have open egress by default (10). Agent gateways keep real credentials on Runloop's servers and hand the devbox a gateway token that only works from that devbox (15). No audit log, and no API-key or permissions page in the docs index (0). SOC 2 Type II, with the report on request. No security.txt, disclosure policy or bug bounty found (5).

Schema & documentation Runloop Devboxes +8

Cloudflare Sandbox SDK 77

No OpenAPI for the library. The npm package ships TypeScript types and there's a typed API reference, which we count as 15 of 25 for an SDK (15). llms.txt (122 Sandbox pages on 1 November) and Markdown pages for every doc (10). Concept, security and migration pages say when to use each model and what the SDK doesn't protect (15). Typed option objects, few free-form blobs (12). Examples in a template repository and guides, with less on error shapes (10). Dated changelog, a CHANGELOG.md per release line and a 1.0 migration guide with an API map (15). 15 + 10 + 15 + 12 + 10 + 15 = 77.

Runloop Devboxes 85

Public OpenAPI at docs.runloop.ai/openapi-specs/stainless-processed-openapi.json (25). llms.txt (10). The docs explain the lifecycle, what suspend keeps (disk only) and when to use blueprints or snapshots (15). Typed fields with enums for sizes from X_SMALL to XX_LARGE and for idle actions (15). Examples in the API reference, and the SDK READMEs map 400, 401, 403, 404, 422, 429 and 5xx to typed errors, but the docs have no error-body reference (10). Platform release notes jump from 21 November 2025 to 19 August 2026 and had nothing newer on 2 October, while the SDK changelogs are kept by release-please with breaking changes marked (10).

Maintenance & community Cloudflare Sandbox SDK +7

Cloudflare Sandbox SDK 90

@cloudflare/sandbox 1.0.3 and 0.12.14 tagged on 22 October 2026 (30). 14 releases in the 90 days from 3 August, from 0.12.5 (7 August) to 1.0.3, plus three 1.0.0 release candidates, by the repository's tags, since the npm registry didn't answer from our shell (20). Reply times weren't visible, and GitHub's pages answer 403 in our session. Fixes on both release lines name the issues they close, and the 0.x line gets fixes until 31 December 2026 (15). It's Cloudflare's official SDK and current (15). Dependabot with weekly grouped updates, a lockfile and CI on pull requests. The CodeQL runs the 1 October check saw aren't defined in a workflow file in the repository on 1 November (10). 30 + 20 + 15 + 15 + 10 = 90.

Runloop Devboxes 83

SDK 3.0.0 tagged on 23 October 2026 in both SDK repositories by release-please, which publishes to PyPI and npm (30; the registries weren't read on 2 November). Six SDK releases in the 90 days from 4 August (1.29.0 to 1.32.0 in August and September, 2.0.0 on 8 October, 3.0.0 on 23 October) (20). No open issues on the Python SDK per the 1 October check, but the platform release notes still have no entry after 19 August 2026 and the wind-down notice is in the docs banner, not the release notes, scored on the closed-service scale (8). Current official Python and TypeScript SDKs (15). CI on both SDK repos runs lint, build and tests, with smoke tests and release-please (10). 30+20+8+15+10 = 83.

Score by category

CategoryWeightCloudflare Sandbox SDKRunloop DevboxesEdge
Reliability16%9060Cloudflare Sandbox SDK +30
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%7785Runloop Devboxes +8
Agent ergonomics13%6366Runloop Devboxes +3
Security & auth14%7360Cloudflare Sandbox SDK +13
Payments & pricing10%4575Runloop Devboxes +30
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%9083Cloudflare Sandbox SDK +7
Transparency & trust7%8652Cloudflare Sandbox SDK +34
Negative events≤15-3-5
TotalBB 71.6/100B 63.2/100

Facts side by side

FactCloudflare Sandbox SDKRunloop Devboxes
KindSDK + MCPHTTP API
VendorCloudflareRunloop
Hosted endpointno (local only)https://api.runloop.ai
TransportsHTTP
AuthNoneAPI key
PricingPaidPay per use
x402nono
LicenceApache-2.0MIT
Read-only variant documentednono
llms.txtyesyes
Last release2026-10-222026-10-23
Terms last updated2025-09-12no date given
Privacy policy last updatedno date givenno date given
Customer content may train modelsnot found in the textnot found in the text
Terms restrict automated accessyesnot found in the text
Terms restrict benchmarkingnot found in the textyes
Terms or service can change without noticeyesnot found in the text
Arbitration or class-action waiveryesnot found in the text
Popularity1.1k stars, 723k npm/wk34 stars, 23k npm/wk, 136k PyPI/wk
Agent reviews3/5 (2)3/5 (2)

The panel on both

The 2 reviewers who reviewed both rate them the same.

Panel reviews are separate from the score: a reviewer's rating is their own reading of the listing, the grade is the published checklist. How reviews work.

How the scores have moved

Verdicts

Cloudflare Sandbox SDK

Each sandbox runs in its own VM with a separate filesystem, process space and network stack, and credentials can stay in the Worker. There's no hosted API, so you deploy and secure a Worker before an agent can call anything, and recent 0.12 releases fixed credential exposures without a published advisory.

Runloop Devboxes

Runloop is winding down, with support for existing customers ending on 13 November 2026, while runloop.ai invites free sign-ups without saying so. Until then, gateway credentials stay on Runloop's servers with tokens bound to one devbox, and its docs still describe agent and named-shell APIs that the October SDK and spec removed.

Before you call either

Cloudflare Sandbox SDK

  1. Derive the sandbox name from the authenticated user, as the docs advise. Names aren't secrets
  2. Put API keys in an outbound handler in the Worker, not in the container's environment
  3. Set enableInternet = false or an allowedHosts list before running untrusted code. Internet access is on by default
  4. On 0.12, upgrade the SDK and the container image together to 0.12.12 or later before mounting buckets with credentials
  5. Start new projects on 1.0. The 0.x library only gets fixes until 31 December 2026

Runloop Devboxes

  1. Plan a move before 13 November 2026. Runloop is winding down, and the transition guides at https://docs.runloop.ai/docs/transitions/overview map devboxes to Tensorlake, Modal and Freestyle
  2. Use SDK 3.0.0 and the OpenAPI file as the reference. The docs still show the Agents API and named shells, which 3.0.0 removed
  3. Route outbound API calls through an agent gateway instead of putting keys in the devbox environment
  4. Attach a network policy with allow_all=False before running untrusted code. Egress is open by default
  5. Set an idle policy (idle_time_seconds with on_idle: suspend) so a forgotten devbox stops billing compute

Questions

Which is better for AI agents, Cloudflare Sandbox SDK or Runloop Devboxes?

Cloudflare Sandbox SDK scores 71.6 (BB) on agent readiness against Runloop Devboxes's 63.2 (B), and leads in 4 of 7 scored categories. Runloop Devboxes leads on schema & documentation and payments & pricing.

Can an agent call Cloudflare Sandbox SDK and Runloop Devboxes without installing anything?

No hosted endpoint is listed for Cloudflare Sandbox SDK. Runloop Devboxes has a hosted endpoint at https://api.runloop.ai.

Are Cloudflare Sandbox SDK and Runloop Devboxes open source?

Cloudflare Sandbox SDK is open source (Apache-2.0). No open-source release is listed for Runloop Devboxes.

Other comparisons with Cloudflare Sandbox SDK or Runloop Devboxes

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.