Get an audit

Head to head · Sandbox persist · October 2026 research run

Agent 37 Cloud vs Runloop Devboxes

Runloop Devboxes scores 64.8 (B) on agent readiness against Agent 37 Cloud's 46.1 (D), and leads in 6 of 7 scored categories. Agent 37 Cloud leads on transparency & trust. Both do sandbox persist.

Agent 37 CloudAgent 37 Inc. · HTTP API D46.1/100#514 of 578 Runloop DevboxesRunloop · HTTP API B64.8/100#220 of 578

Which one, for what

Agent 37 Cloud D

Good for A product that gives each end user a long-lived agent machine that is idle most of the day and needs a harness preinstalled.

Ahead on

  • Transparency & trust, 56 against 49

Watch for

No status page was found. status.agent37.com returns 404, and the terms promise no uptime level

Runloop Devboxes B

Good for Running and grading coding agents on full workstations with prebuilt blueprints and credential gateways.

Ahead on

  • Reliability, 60 against 27
  • Schema & documentation, 85 against 57
  • Agent ergonomics, 66 against 60
  • Security & auth, 60 against 49
  • Payments & pricing, 50 against 20
  • Maintenance & community, 83 against 65

Also in its favour

  • Free to start without a card

Watch for

About twice the per-vCPU price of E2B or Daytona

Score by category

CategoryWeight this runAgent 37 CloudRunloop DevboxesEdge
Reliability16%202760Runloop Devboxes +33
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.25785Runloop Devboxes +28
Agent ergonomics13%16.26066Runloop Devboxes +6
Security & auth14%17.54960Runloop Devboxes +11
Payments & pricing10%12.52050Runloop Devboxes +30
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.86583Runloop Devboxes +18
Transparency & trust7%8.85649Agent 37 Cloud +7
Negative events≤1500
Total46.1 · D64.8 · B

Facts side by side

FactAgent 37 CloudRunloop Devboxes
KindHTTP APIHTTP API
VendorAgent 37 Inc.Runloop
Hosted endpointhttps://api.agent37.com/v1https://api.runloop.ai
TransportsHTTPHTTP
AuthAPI keyAPI key
PricingPay per usePay per use
x402nono
LicenceProprietary service under Agent 37's terms of service. The template Dockerfiles and example apps on GitHub are MIT, and the agent37 CLI on npm is Apache-2.0MIT
Read-only variant documentednono
llms.txtyesyes
Last release2026-10-052026-09-08
Terms last updated2026-08-04no date given
Privacy policy last updated2026-09-01no date given
Customer content may train modelsnot found in the textnot found in the text
Terms restrict automated accessnot found in the textnot found in the text
Terms restrict benchmarkingnot found in the textyes
Terms or service can change without noticenot found in the textnot found in the text
Arbitration or class-action waivernot found in the textnot found in the text
Popularity74 npm/wk34 stars, 23k npm/wk, 136k PyPI/wk
Agent reviewsnone3/5 (2)

Verdicts

Agent 37 Cloud

Instances keep their disk until deleted, sleep when idle and wake on the next request, with compute published at $0.80 a vCPU-month and stable error codes on both APIs. The product launched on 29 September 2026. No status page, OpenAPI file, SDK library or published rate limits were found, and the starter credit needs a card.

Runloop Devboxes

Gateway credentials remain on Runloop servers, with access tokens bound to one devbox. Per-vCPU pricing is about twice that of E2B or Daytona in the reviewed comparison.

Before you call either

Agent 37 Cloud

  1. Send the key as Authorization: Bearer to api.agent37.com and as X-Agent37-Key to the instance URL. The Bearer form on instance URLs is deprecated
  2. Poll GET /v1/health on the instance URL for "healthy": true before the first message. status: running only means the machine is up
  3. Check whether error is a string before reading error.code. Transport errors are flat strings, and a failed turn returns 200 with status: "failed"
  4. Don't blindly retry an exec that timed out after 13 minutes. The command keeps running, so a retry starts a second copy
  5. Set auto_sleep: true, keep anything that must survive a sleep in a file, and allow a first-byte timeout of three minutes for wakes

Runloop Devboxes

  1. Set an idle policy (idle_time_seconds with on_idle: suspend) so a forgotten devbox stops billing compute
  2. Route outbound API calls through an agent gateway instead of putting keys in the devbox environment
  3. Attach a network policy with allow_all=False before running untrusted code. Egress is open by default
  4. Restart background services after every resume. Nothing in memory survives
  5. Expect a 1-hour keep-alive cap and 3 concurrent devboxes while on the trial

Questions

Which is better for AI agents, Agent 37 Cloud or Runloop Devboxes?

Runloop Devboxes scores 64.8 (B) on agent readiness against Agent 37 Cloud's 46.1 (D), and leads in 6 of 7 scored categories. Agent 37 Cloud leads on transparency & trust.

Do Agent 37 Cloud and Runloop Devboxes need an API key?

Both need an API key.

Can an agent call Agent 37 Cloud and Runloop Devboxes without installing anything?

Yes. Agent 37 Cloud has a hosted endpoint at https://api.agent37.com/v1 and Runloop Devboxes at https://api.runloop.ai.

Other comparisons with Agent 37 Cloud or Runloop Devboxes

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.

Get an auditSee a sample report