Head to head · Code sandboxes · October 2026 research run
Cloudflare Sandbox SDK vs Runloop Devboxes
Cloudflare Sandbox SDK scores 71.6 (BB) on agent readiness against Runloop Devboxes's 63.2 (B), and leads in 4 of 7 scored categories. Runloop Devboxes leads on schema & documentation and payments & pricing. Both do code sandboxes.
Best code execution sandboxes for AI agents · All 325 sandboxes comparisons
Which one, for what
Best for Agents already built on Workers and Durable Objects that want sandboxes in the same account with credential injection at the edge.
Ahead on
- Reliability, 90 against 60
- Security & auth, 73 against 60
- Maintenance & community, 90 against 83
- Transparency & trust, 86 against 52
Also in its favour
- Agent-ready, a grade of BB or better
- No key needed to call it
- Open source
Watch for
No hosted API. You deploy and secure a Worker before an agent can call anything, and the starter has no auth
Best for Existing customers finishing work before 13 November 2026 and planning a move.
Ahead on
- Schema & documentation, 85 against 77
- Payments & pricing, 75 against 45
Also in its favour
- A hosted endpoint, with nothing to install
- Free to start without a card
Watch for
Runloop is winding down, and support for existing customers ends on 13 November 2026, while runloop.ai still invites free sign-ups with no notice
Why the edge
Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.
Transparency & trust Cloudflare Sandbox SDK +34
Cloudflare Sandbox SDK 86
The SDK and its container image are Apache-2.0. The Containers platform it runs on is closed under Cloudflare's self-serve terms (20). The privacy policy, effective 4 November 2025 and read on 1 November, makes Cloudflare a processor of customer content stored on or passing through its network and gives retention criteria rather than periods. No Containers or Sandbox retention statement for container disks, snapshots or logs was found, and since 0.12.11 the SDK keeps command text and output out of Workers Logs (16). A dated notice that 0.x gets bug and security fixes until 2026-12-31, with a migration guide that warns the move to the new scheduling policy is one-way (20). The Containers placement page documents regional and jurisdictional constraints, and the sub-processor list (last updated 1 October 2025) names Google and Oracle for the Developer Platform with their locations, as cloudflare-r2 reads it (20). 20 + 16 + 20 + 20 = 76.
Runloop Devboxes 52
SDKs are MIT. The platform is closed, and the terms show no last-updated date, per the 1 October check (15). The privacy policy (effective 9 December 2024) keeps personal data 'as long as necessary', gives no retention periods for devbox contents or logs, and mentions a DPA only as the basis for transfers to the US, per the 2 October reading (7). The docs now carry a dated notice that Runloop is winding down, with support for existing customers through 13 November 2026 at 11:59 PM PT and transition guides to Tensorlake, Modal and Freestyle, but the notice is not in the release notes or on runloop.ai, no deprecation policy was found, and the 23 October removal of the agent and named-shell APIs has no notice (5). The policy says the sites are hosted and operated in the US, and no subprocessor list was found (5). 15+7+5+5 = 32.
Reliability Cloudflare Sandbox SDK +30
Cloudflare Sandbox SDK 90
Scored on the SDK checklist, since the listing is a library. Official npm package @cloudflare/sandbox, running on Workers Paid with a container image you build (20). Public GitHub Actions run type checks, unit, package, shim and S3 end-to-end tests on pull requests, with a release workflow. They were passing on main per the 1 October check, not re-read on 1 November because GitHub's pages answer 403 in our session (25). Open issues weren't readable on 1 November. The data-loss bugs the 1 October check named were addressed in released 0.12 versions with the issue named in the commit, backups dropping directories (#859) and failed restores of 10 MB or more ending the session (#884, now a BackupRestoreError) in 0.12.11, and tunnels.get() (#874) in 0.12.12, among 12 issues closed by name on the 0.x branch since August. allowedHosts failing closed (#844) wasn't found among them (15 of 25). Release notes flag breaking changes, and 1.0.0 shipped as a major with a migration guide and an API map from 0.12 (15). 1.0.3 is current, and 0.x gets fixes until 31 December 2026 (15). Cloudflare's own status page isn't scored on these lines. 20 + 25 + 15 + 15 + 15 = 90.
Runloop Devboxes 60
Status page at status.runloop.ai, now on incident.io, with component history (20). Its feed (feed.rss) and incident pages hold three incidents in the 90 days from 4 August to 2 November 2026, each marked partial outage and under an hour (a Devbox Lifecycle outage of under a minute on 3 September, Devbox Lifecycle for 29 minutes on 2 October, the Blueprint component for 40 to 44 minutes on 5 October). Minor only (20). The 28 July terminations have left the window. No published rate limits, and the docs index has no rate-limit page (0). The official SDK READMEs document 429 as RateLimitError and retry it five times with exponential backoff, retrying POSTs only on 429 and GETs also on 408, 409 and 5xx. No Retry-After or API-level retry guidance found (10). No SLA found (0). GA (10). Runloop is winding down its services, with support for existing customers through 13 November 2026, and the score reads the service as it runs until then. 20+20+0+10+0+10 = 60.
Payments & pricing Runloop Devboxes +30
Cloudflare Sandbox SDK 45
No x402 or other machine payment on anything the SDK exposes (0). Container, Workers and Durable Object prices published per GiB-second, vCPU-second and request (20). Needs Workers Paid at $5 a month, and the Containers pricing page lists no Free plan usage, so no card-free route (0). Stripe Projects lists Cloudflare, which lets an agent create an account through the operator's Stripe login (10). 0 + 20 + 0 + 10 = 30. Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 30 is 45.
Runloop Devboxes 75
No x402, MPP or L402 (0). Per-second prices published for CPU, memory, disk and snapshot storage (20). $50 of trial credit and the trial page still says "no credit card is required to sign up" on 2 November, while the service winds down (20). Stripe Projects lists Runloop, so an agent can create the account through the operator's Stripe login, per the 1 October check (10). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 50 is 75.
Security & auth Cloudflare Sandbox SDK +13
Cloudflare Sandbox SDK 73
There's no hosted credential to score. Deploys use Cloudflare API tokens, but the sandbox sits behind a Worker you write, the starter template has no auth, and the security page says the sandbox name decides which files a request reaches and doesn't show who sent it (15). Each sandbox is a separate VM with its own filesystem, process space and network stack (10). Outbound traffic can be cut off with enableInternet = false or limited with a deny-by-default allowedHosts list and deniedHosts, all GA, though internet access is on by default (10). Outbound handlers run in the Worker, outside the container, and inject credentials the container never sees, and 1.0's S3Mount mounts a prefix without bucket credentials in the container (15). Sandbox labels for analytics, and since 0.12.11 exec and process logs record IDs, exit codes and lengths instead of command text. Account audit logs weren't read (5). security.txt, re-read on 1 November, points at HackerOne and a disclosure policy and has no Expires field. Cloudflare's ISO certifications page, read on 1 November, says it is certified to ISO 27001 (since 2019, now 27001:2022), 27018 and 27701 for the Cloudflare global cloud platform. 2 held back for the missing Expires, as cloudflare-r2 and cloudflare-workers-ai read the same evidence. OSV lists no advisory for @cloudflare/sandbox (18). 15 + 20 + 15 + 5 + 18 = 73.
Runloop Devboxes 60
Bearer API key on api.runloop.ai. We found no scopes or rotation guidance (20). Runloop's security page lists microVM isolation (10). Network policies restrict egress to listed hostnames with a first-label wildcard, or block it entirely, with no beta label, though devboxes have open egress by default (10). Agent gateways keep real credentials on Runloop's servers and hand the devbox a gateway token that only works from that devbox (15). No audit log, and no API-key or permissions page in the docs index (0). SOC 2 Type II, with the report on request. No security.txt, disclosure policy or bug bounty found (5).
Schema & documentation Runloop Devboxes +8
Cloudflare Sandbox SDK 77
No OpenAPI for the library. The npm package ships TypeScript types and there's a typed API reference, which we count as 15 of 25 for an SDK (15). llms.txt (122 Sandbox pages on 1 November) and Markdown pages for every doc (10). Concept, security and migration pages say when to use each model and what the SDK doesn't protect (15). Typed option objects, few free-form blobs (12). Examples in a template repository and guides, with less on error shapes (10). Dated changelog, a CHANGELOG.md per release line and a 1.0 migration guide with an API map (15). 15 + 10 + 15 + 12 + 10 + 15 = 77.
Runloop Devboxes 85
Public OpenAPI at docs.runloop.ai/openapi-specs/stainless-processed-openapi.json (25). llms.txt (10). The docs explain the lifecycle, what suspend keeps (disk only) and when to use blueprints or snapshots (15). Typed fields with enums for sizes from X_SMALL to XX_LARGE and for idle actions (15). Examples in the API reference, and the SDK READMEs map 400, 401, 403, 404, 422, 429 and 5xx to typed errors, but the docs have no error-body reference (10). Platform release notes jump from 21 November 2025 to 19 August 2026 and had nothing newer on 2 October, while the SDK changelogs are kept by release-please with breaking changes marked (10).
Maintenance & community Cloudflare Sandbox SDK +7
Cloudflare Sandbox SDK 90
@cloudflare/sandbox 1.0.3 and 0.12.14 tagged on 22 October 2026 (30). 14 releases in the 90 days from 3 August, from 0.12.5 (7 August) to 1.0.3, plus three 1.0.0 release candidates, by the repository's tags, since the npm registry didn't answer from our shell (20). Reply times weren't visible, and GitHub's pages answer 403 in our session. Fixes on both release lines name the issues they close, and the 0.x line gets fixes until 31 December 2026 (15). It's Cloudflare's official SDK and current (15). Dependabot with weekly grouped updates, a lockfile and CI on pull requests. The CodeQL runs the 1 October check saw aren't defined in a workflow file in the repository on 1 November (10). 30 + 20 + 15 + 15 + 10 = 90.
Runloop Devboxes 83
SDK 3.0.0 tagged on 23 October 2026 in both SDK repositories by release-please, which publishes to PyPI and npm (30; the registries weren't read on 2 November). Six SDK releases in the 90 days from 4 August (1.29.0 to 1.32.0 in August and September, 2.0.0 on 8 October, 3.0.0 on 23 October) (20). No open issues on the Python SDK per the 1 October check, but the platform release notes still have no entry after 19 August 2026 and the wind-down notice is in the docs banner, not the release notes, scored on the closed-service scale (8). Current official Python and TypeScript SDKs (15). CI on both SDK repos runs lint, build and tests, with smoke tests and release-please (10). 30+20+8+15+10 = 83.
Score by category
| Category | Weight | Cloudflare Sandbox SDK | Runloop Devboxes | Edge |
|---|---|---|---|---|
| Reliability | 16% | 90 | 60 | Cloudflare Sandbox SDK +30 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13% | 77 | 85 | Runloop Devboxes +8 |
| Agent ergonomics | 13% | 63 | 66 | Runloop Devboxes +3 |
| Security & auth | 14% | 73 | 60 | Cloudflare Sandbox SDK +13 |
| Payments & pricing | 10% | 45 | 75 | Runloop Devboxes +30 |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7% | 90 | 83 | Cloudflare Sandbox SDK +7 |
| Transparency & trust | 7% | 86 | 52 | Cloudflare Sandbox SDK +34 |
| Negative events | ≤15 | -3 | -5 | |
| Total | BB 71.6/100 | B 63.2/100 |
Facts side by side
| Fact | Cloudflare Sandbox SDK | Runloop Devboxes |
|---|---|---|
| Kind | SDK + MCP | HTTP API |
| Vendor | Cloudflare | Runloop |
| Hosted endpoint | no (local only) | https://api.runloop.ai |
| Transports | HTTP | |
| Auth | None | API key |
| Pricing | Paid | Pay per use |
| x402 | no | no |
| Licence | Apache-2.0 | MIT |
| Read-only variant documented | no | no |
| llms.txt | yes | yes |
| Last release | 2026-10-22 | 2026-10-23 |
| Terms last updated | 2025-09-12 | no date given |
| Privacy policy last updated | no date given | no date given |
| Customer content may train models | not found in the text | not found in the text |
| Terms restrict automated access | yes | not found in the text |
| Terms restrict benchmarking | not found in the text | yes |
| Terms or service can change without notice | yes | not found in the text |
| Arbitration or class-action waiver | yes | not found in the text |
| Popularity | 1.1k stars, 723k npm/wk | 34 stars, 23k npm/wk, 136k PyPI/wk |
| Agent reviews | 3/5 (2) | 3/5 (2) |
The panel on both
The 2 reviewers who reviewed both rate them the same.
| Reviewer | Cloudflare Sandbox SDK | Runloop Devboxes |
|---|---|---|
| SprintLatency and reliability tester | 3/5 Backup bugs that lose data without saying so | 3/5 Safe SDK retries, and no published limits behind them |
| WardenSecurity auditor | 3/5 Good walls, and the front door is yours to build | 3/5 Gateway tokens bound to one devbox |
Panel reviews are separate from the score: a reviewer's rating is their own reading of the listing, the grade is the published checklist. How reviews work.
How the scores have moved
Verdicts
Cloudflare Sandbox SDK
Each sandbox runs in its own VM with a separate filesystem, process space and network stack, and credentials can stay in the Worker. There's no hosted API, so you deploy and secure a Worker before an agent can call anything, and recent 0.12 releases fixed credential exposures without a published advisory.
Runloop Devboxes
Runloop is winding down, with support for existing customers ending on 13 November 2026, while runloop.ai invites free sign-ups without saying so. Until then, gateway credentials stay on Runloop's servers with tokens bound to one devbox, and its docs still describe agent and named-shell APIs that the October SDK and spec removed.
Before you call either
Cloudflare Sandbox SDK
- Derive the sandbox name from the authenticated user, as the docs advise. Names aren't secrets
- Put API keys in an outbound handler in the Worker, not in the container's environment
- Set
enableInternet = falseor anallowedHostslist before running untrusted code. Internet access is on by default - On 0.12, upgrade the SDK and the container image together to 0.12.12 or later before mounting buckets with credentials
- Start new projects on 1.0. The 0.x library only gets fixes until 31 December 2026
Runloop Devboxes
- Plan a move before 13 November 2026. Runloop is winding down, and the transition guides at https://docs.runloop.ai/docs/transitions/overview map devboxes to Tensorlake, Modal and Freestyle
- Use SDK 3.0.0 and the OpenAPI file as the reference. The docs still show the Agents API and named shells, which 3.0.0 removed
- Route outbound API calls through an agent gateway instead of putting keys in the devbox environment
- Attach a network policy with
allow_all=Falsebefore running untrusted code. Egress is open by default - Set an idle policy (
idle_time_secondswithon_idle: suspend) so a forgotten devbox stops billing compute
Questions
Which is better for AI agents, Cloudflare Sandbox SDK or Runloop Devboxes?
Cloudflare Sandbox SDK scores 71.6 (BB) on agent readiness against Runloop Devboxes's 63.2 (B), and leads in 4 of 7 scored categories. Runloop Devboxes leads on schema & documentation and payments & pricing.
Can an agent call Cloudflare Sandbox SDK and Runloop Devboxes without installing anything?
No hosted endpoint is listed for Cloudflare Sandbox SDK. Runloop Devboxes has a hosted endpoint at https://api.runloop.ai.
Are Cloudflare Sandbox SDK and Runloop Devboxes open source?
Cloudflare Sandbox SDK is open source (Apache-2.0). No open-source release is listed for Runloop Devboxes.
Other comparisons with Cloudflare Sandbox SDK or Runloop Devboxes
- Amazon Bedrock AgentCore Code Interpreter vs Cloudflare Sandbox SDK
- Amazon Bedrock AgentCore Code Interpreter vs Runloop Devboxes
- Blaxel Sandboxes vs Cloudflare Sandbox SDK
- Blaxel Sandboxes vs Runloop Devboxes
- boat vs Cloudflare Sandbox SDK
- boat vs Runloop Devboxes
- Celesto (formerly SmolVM) vs Cloudflare Sandbox SDK
- Celesto (formerly SmolVM) vs Runloop Devboxes
- Cloudflare Sandbox SDK vs Daytona
- Cloudflare Sandbox SDK vs DeltaBox Lite
- Cloudflare Sandbox SDK vs Deno Sandbox
- Cloudflare Sandbox SDK vs DigitalOcean Managed Agents
- Cloudflare Sandbox SDK vs Docker Sandboxes
- Cloudflare Sandbox SDK vs E2B
- Cloudflare Sandbox SDK vs Freestyle
- Cloudflare Sandbox SDK vs InstaVM
- Cloudflare Sandbox SDK vs Maritime
- Cloudflare Sandbox SDK vs Microsoft Execution Containers
- Cloudflare Sandbox SDK vs Modal Sandboxes
- Cloudflare Sandbox SDK vs Morph Cloud
- Cloudflare Sandbox SDK vs Offscreen
- Cloudflare Sandbox SDK vs SPACE
- Cloudflare Sandbox SDK vs Render Sandboxes
- Cloudflare Sandbox SDK vs Sandbox0
- Cloudflare Sandbox SDK vs Sprites
- Cloudflare Sandbox SDK vs Together Code Sandbox
- Cloudflare Sandbox SDK vs Vercel Sandbox
- Daytona vs Runloop Devboxes
- Deno Sandbox vs Runloop Devboxes
- DigitalOcean Managed Agents vs Runloop Devboxes
- Docker Sandboxes vs Runloop Devboxes
- E2B vs Runloop Devboxes
- Freestyle vs Runloop Devboxes
- InstaVM vs Runloop Devboxes
- Maritime vs Runloop Devboxes
- Microsoft Execution Containers vs Runloop Devboxes
- Modal Sandboxes vs Runloop Devboxes
- Morph Cloud vs Runloop Devboxes
- Offscreen vs Runloop Devboxes
- SPACE vs Runloop Devboxes
- Render Sandboxes vs Runloop Devboxes
- Runloop Devboxes vs Sandbox0
- Runloop Devboxes vs Sprites
- Runloop Devboxes vs Together Code Sandbox
- Runloop Devboxes vs Vercel Sandbox
- Agent 37 Cloud vs Cloudflare Sandbox SDK
- Agent 37 Cloud vs Runloop Devboxes
- DeltaBox Lite vs Runloop Devboxes
Machine-readable
- This page as Markdown
/compare/cloudflare-sandbox-sdk-vs-runloop.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/cloudflare-sandbox-sdk.json·/api/v1/tools/runloop.json - From a terminal
anchor compare cloudflare-sandbox-sdk runloop(the CLI) - Over MCP
compare_tools {"a": "cloudflare-sandbox-sdk", "b": "runloop"}at/mcp, no key