Head to head · Agent harnesses · October 2026 research run
boxes.dev vs dsh-superagent (DeepSeek)
boxes.dev scores 53.2 (D) on agent readiness against dsh-superagent (DeepSeek)'s 44.6 (E), and leads in 4 of 7 scored categories. dsh-superagent (DeepSeek) leads on agent ergonomics and payments & pricing. Both do agent harnesses.
Best agent harnesses and coding agents · All 1767 harnesses comparisons
Which one, for what
Best for Developers who want Claude Code or Codex running on cloud copies of their dev environment, started from Slack, Linear, a schedule, an API call or another assistant.
Ahead on
- Reliability, 32 against 15
- Security & auth, 69 against 33
- Maintenance & community, 48 against 34
- Transparency & trust, 60 against 52
Also in its favour
- A hosted endpoint, with nothing to install
Watch for
No status page, incident history or SLA found
Best for Teams on github.com that want a DeepSeek Harness agent to turn allowed people's issue requests into pull requests, run on an isolated machine or OS account.
Ahead on
- Agent ergonomics, 67 against 60
- Payments & pricing, 60 against 50
Also in its favour
- Free to start without a card
- Open source
Watch for
Executor and main Sessions run with danger-full-access as the operator's OS user, and the docs say a worker can read the operator's home directory
Why the edge
Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.
Security & auth boxes.dev +36
boxes.dev 69
OAuth for the MCP server, with protected-resource metadata naming mcp.boxes.dev and four scopes (threads:read, threads:write, events:subscribe, settings:write), a per-project grant and removal from settings. Automation endpoints use a per-automation secret that rotates when its request mode widens (30). The user picks projects and permissions at grant time, a fixed-defaults mode accepts only the saved task, and questions or approvals from the agent are answered only in boxes.dev, never through the MCP connection (15). Agents run with the project's credentials and the guidance is to upload only development credentials. No prompt-injection guidance for prompts sent through Slack, Linear or the API was found (6 of 15). Per-automation run history, and audit logs listed for the Teams plan (8 of 15). A valid security.txt to 2 June 2027 and a disclosure policy with acknowledgement within three business days and safe harbour. No bug bounty and no certification found (10 of 20). 30+15+6+8+10 = 69.
dsh-superagent (DeepSeek) 33
Harness reading of the framework checklist, 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. The superagent profile is created from DSH's Web template, and DSH documents that shipped profiles mount session-log-deepseek, enabled by default, which adds the session log to each official DeepSeek API request, up to 8 MiB. dsh-superagent's docs don't mention it, and its Sessions read issue, pull request and chat text and the repository. Setup and worker shells drop variables named KEY, PASSWORD, SECRET or TOKEN, and GH_TOKEN and GITHUB_TOKEN are removed from git (8). Both presets run danger-full-access without approval as the operator's OS user. docs/006 says the account wrappers don't isolate the process, a worker can run programs by path and read the operator's home directory, and recommends a dedicated OS account. The gate, observe default, blocked commands and no auto-merge as the agent account narrow what it does, not what it can do (6). docs/002 says messages, comments and quoted documents are inputs to a decision and can't change allowed lists, rules or Mandates, bots can never make explicit requests, and prompt edits don't change the gate. Workers with full access still read untrusted Thread text, and no further guidance on that was found (7). Decisions are recorded with reasons, Ticket details show the Trigger, Decision, scope and Policy version, the host records the prompt revisions each Session used, and every GitHub message starts with an automated-agent line by default (12). No SECURITY.md (404 at the root and in .github), no security.txt on www.deepseek.com (404), no bug bounty or advisory found. NVD has no record for the prototype. The CVEs against DSH itself are on the deepseek-harness listing (0).
Reliability boxes.dev +17
boxes.dev 32
Hosted reading, for the remote MCP server at api.boxes.dev/mcp and the automation run endpoints. No status page found. The site footer says "All systems normal" with no link, and the trust centre at trust.boxes.dev draws in a script our reader could not run (0). No readable incident history (5). No request rate limits found. Plans cap awake devboxes (3 on the trial, 5 to 25 on paid plans), and the API answers 409 active_fork_limit_reached at the cap (5 of 15). The automation API takes an Idempotency-Key and returns the original run on a retry with idempotentReplay: true, and the MCP connection retries a lost start or message with the same request key. No Retry-After or backoff guidance found (12 of 15). No SLA found (0). Neither the MCP connection nor the automation API is marked beta, and the product launched in June 2026 per its own blog (10). 0+5+5+12+0+10 = 32.
dsh-superagent (DeepSeek) 15
Local-package reading, since it runs as a DSH bundle on the operator's machine. Not published to npm (private: true in package.json, and the registry has no dsh-superagent). It installs from a git clone with pnpm install && pnpm build on top of @deepseek-ai/dsh@0.2.1-rc.3, with Node 22.19+ or 24+, git 2.32+, Python 3.8+, tmux and gh 2.48+ stated, so half credit (10). A vitest suite of about 25 spec files is listed in docs/007, and pnpm check runs the typecheck, tests and build. Two spec files were read on raw.githubusercontent.com. No CI workflow was found at five common paths under .github/workflows, and the directory couldn't be listed (5). Open issues couldn't be read, because github.com and api.github.com weren't reachable from our network. Unchecked, so scored absent (0). Version 0.1.0 with no tag, release or changelog found, so no semver record (0). A research prototype, not stable (0).
Maintenance & community boxes.dev +14
boxes.dev 48
Closed service. CLI @boxes-dev/cli 1.0.1134 published on 17 November 2026 (30). Only one dated product update was found, on 27 October 2026, and earlier CLI versions' dates were not read, so three releases in 90 days are not established (0). A support address, an in-app chat and a blog, with no public tracker or changelog (8 of 25). A CLI but no SDK, and no entry in the official MCP registry was looked for (5 of 15). The CLI is published from GitHub Actions with npm trusted publishing and signatures (5 of 10). 30+0+8+5+5 = 48.
dsh-superagent (DeepSeek) 34
Published on 31 October 2026, within 30 days, read as the last published change since no tag or release was found (30). One publication, so not three releases in 90 days (0). Issues, pull requests and Discussions couldn't be read, and the README frames the code as a reference prototype adapted from an internal version (0). No official SDK or registry entry (0). Exact dependency pins and a lockfile, but no CI workflow found, and the docs pin DSH 0.2.1-rc.3 (23 October 2026) while npm latest is 0.2.1-rc.5 (29 October 2026) (4).
Payments & pricing dsh-superagent (DeepSeek) +10
boxes.dev 50
No x402, MPP or L402 (0). Plan prices are public ($19, $99 and $249 a seat a month) and extra usage is $0.40 a box-hour, with a box-hour defined as one hour of a 4 vCPU, 8 GiB devbox (30). A free trial of 20 box-hours. The help page says some teams must add a payment method before starting, so a card-free start is not certain (20 of 30). A person signs in with Google or Apple and approves the OAuth grant or creates the automation secret in the desktop app (0). 0+30+20+0 = 50.
dsh-superagent (DeepSeek) 60
Self-hosted rule, as for the other open-source harnesses. No payment protocol (0). Free under MIT with nothing to buy (20). No card for the software (20). An agent can't get started alone, because the setup needs a person to create a dedicated GitHub user account, accept the repository invitation and run auth login for the wrappers, which SETUP.md lists as operator steps. 0, as the other harnesses that need an account were read (0). The model provider is a separate purchase. Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 40 is 60.
Transparency & trust boxes.dev +8
boxes.dev 60
Closed service under the Common Paper Cloud Service Agreement 2.1, dated 20 October 2026, with the standard terms' machine-learning clause deleted and California law (15). The privacy policy of 21 October 2026 gives retention by data type (session replay about 30 days, Slack data deleted within 14 business days of disconnecting), and the backup clause matches the terms (22 of 30). Price changes take 30 days' notice under the terms. No deprecation policy for the API or MCP server found (5 of 20). Service providers are named (Convex, Cloudflare Workers, Expo, OpenAI, Amplitude, Sentry, Google and Apple), but the host that runs devboxes is unnamed and data may be processed in the US and elsewhere (12 of 20). 15+22+5+12 = 54.
dsh-superagent (DeepSeek) 52
MIT, Copyright (c) 2026 DeepSeek Harness Team (30). The prototype has no data statement of its own. DeepSeek Harness's privacy policy (last updated 20 September 2026) names Hangzhou DeepSeek Artificial Intelligence Co., Ltd., says session logs are collected with official model services and may be used to train models with a right to opt out, and says custom model services aren't stored server-side. It agrees with DSH's session-log README, but neither is referred to from the prototype (12). No deprecation policy. The README says the published code differs from DeepSeek's internal version, and docs/006 warns that database migrations are one-way (3). Telemetry and the session-log upload are disclosed with opt-outs in DSH's own reference pages, not in this repository (6).
Agent ergonomics dsh-superagent (DeepSeek) +7
boxes.dev 60
The MCP tool count is unknown. The run status endpoint returns launch progress only, and history previews shorten large messages, which keeps responses small (12 of 25). Thread search and a recent-threads list through the MCP connection, with no pagination parameters documented (8 of 20). Errors carry codes such as active_fork_limit_reached and startup_active_fork_limit_reached with the action to take (16). Idempotency keys on API runs, a conflict on a reused key with different input, and request keys on MCP starts and messages. Tool annotations could not be read (16 of 20). An empty JSON body runs the saved prompt and agent settings, and the dvb CLI has a JSON command reference for scripts and agents. No SDK (8 of 15). 12+8+16+16+8 = 60.
dsh-superagent (DeepSeek) 67
Harness reading, scored on what a team or pipeline driving it has to supply. The worker preset is DSH's standard with web fetch, plan mode, questions, scheduling and workflows left out, the main Session can't write files, and tool results over 8,192 characters are pruned (18). A concurrency limit with one extra slot for a person's message, a 5-second foreground shell, a 10-minute prepare limit and 30 minutes for background work after a turn. No cost or turn limit per Ticket was found (12). Ticket stages, health entries per part, Reports of blocked or failed, API refusals with codes, and three retries with growing backoff on 408, 429, 5xx and dropped connections (17). Saves carry a revision and a stale one gets 409, state and Sessions survive a restart, and a restarted Assignment gets a recovery note to check git and GitHub before repeating any outside action (16). Sensible defaults (observe mode, port 3082, branch prefix agent/), but setup needs a dedicated GitHub user account, CLI wrappers, tmux and a profile, and there is no SDK. Work is requested by mentioning the agent account on GitHub or Feishu (4).
Score by category
| Category | Weight | boxes.dev | dsh-superagent (DeepSeek) | Edge |
|---|---|---|---|---|
| Reliability | 16% | 32 | 15 | boxes.dev +17 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13% | 57 | 61 | dsh-superagent (DeepSeek) +4 |
| Agent ergonomics | 13% | 60 | 67 | dsh-superagent (DeepSeek) +7 |
| Security & auth | 14% | 69 | 33 | boxes.dev +36 |
| Payments & pricing | 10% | 50 | 60 | dsh-superagent (DeepSeek) +10 |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7% | 48 | 34 | boxes.dev +14 |
| Transparency & trust | 7% | 60 | 52 | boxes.dev +8 |
| Negative events | ≤15 | 0 | 0 | |
| Total | D 53.2/100 | E 44.6/100 |
Facts side by side
| Fact | boxes.dev | dsh-superagent (DeepSeek) |
|---|---|---|
| Kind | MCP server | Agent harness |
| Vendor | Boxes, Inc. | DeepSeek |
| Hosted endpoint | https://api.boxes.dev/mcp | no (local only) |
| Transports | Streamable HTTP, HTTP | |
| Auth | OAuth or key | Token |
| Pricing | Freemium | Free |
| x402 | no | no |
| Licence | Proprietary. The service runs under the Common Paper Cloud Service Agreement 2.1, and the @boxes-dev/cli package is marked UNLICENSED | MIT |
| Read-only variant documented | no | no |
| llms.txt | yes | no |
| Last release | 2026-11-17 | 2026-10-31 |
| Terms last updated | 2026-10-20 | no document linked |
| Privacy policy last updated | 2026-10-21 | 2026-09-20 |
| Customer content may train models | not found in the text | not read yet |
| Terms restrict automated access | not found in the text | not read yet |
| Terms restrict benchmarking | not found in the text | not read yet |
| Terms or service can change without notice | not found in the text | not read yet |
| Arbitration or class-action waiver | not found in the text | not read yet |
How the scores have moved
boxes.dev
No score change since it was listed.
Verdicts
boxes.dev
An outside agent can start Claude Code or Codex tasks on cloud devboxes through an OAuth MCP server with four scopes, or an automation endpoint with idempotency keys. No status page, SLA or tool schema could be read, and every route starts with a person setting up a project in the app.
dsh-superagent (DeepSeek)
The authorisation design is careful, with allowed lists, Mandates and a Policy enforced in code, proactive work only recorded by default and auto-merge off. Executors run every command without approval under the operator's OS user, and the README calls it a research prototype with no release, changelog or security policy.
Before you call either
boxes.dev
- Send a new
Idempotency-Keyfor each intended run, and the same key and body when you retry an uncertain request - Wait for status
thread_readybefore you use a returneddevboxIdas a target - If you get 409
active_fork_limit_reached, free an awake devbox or wait. The plan's awake-devbox cap is full - Read the thread in boxes.dev for the agent's answer. The status URL does not return it
- Ask for
threads:readonly unless the task needs to start agents or change settings
dsh-superagent (DeepSeek)
- Run it under a dedicated OS account holding only the agent's credentials. Executors run commands without approval and can read the operator's home directory
- Set
enabled: falseon thesession-log-deepseekrow of thesuperagentprofile before using DeepSeek's official API on private code - Keep the accept mode at
observeand the allowed list short until the recorded Decisions look right - Wait for
scripts/dsh-superagent statusto stop exiting 1 after any save that answersrestarting: truebefore the next request - Send
x-orchestrator-request: 1and a loopback Host to the Management API, and sendrevisionwith each PUT, since a stale one gets 409
Questions
Which is better for AI agents, boxes.dev or dsh-superagent (DeepSeek)?
boxes.dev scores 53.2 (D) on agent readiness against dsh-superagent (DeepSeek)'s 44.6 (E), and leads in 4 of 7 scored categories. dsh-superagent (DeepSeek) leads on agent ergonomics and payments & pricing.
Can an agent call boxes.dev and dsh-superagent (DeepSeek) without installing anything?
boxes.dev has a hosted endpoint at https://api.boxes.dev/mcp. No hosted endpoint is listed for dsh-superagent (DeepSeek).
Are boxes.dev and dsh-superagent (DeepSeek) open source?
No open-source release is listed for boxes.dev. dsh-superagent (DeepSeek) is open source (MIT).
Other comparisons with boxes.dev or dsh-superagent (DeepSeek)
- Agent Zero vs boxes.dev
- Agent Zero vs dsh-superagent (DeepSeek)
- Aider vs boxes.dev
- Aider vs dsh-superagent (DeepSeek)
- Amp vs boxes.dev
- Amp vs dsh-superagent (DeepSeek)
- Antigravity CLI vs boxes.dev
- Antigravity CLI vs dsh-superagent (DeepSeek)
- Auggie CLI vs boxes.dev
- Auggie CLI vs dsh-superagent (DeepSeek)
- AXLR vs boxes.dev
- AXLR vs dsh-superagent (DeepSeek)
- boxes.dev vs Claude Code
- boxes.dev vs Click
- boxes.dev vs Cline
- boxes.dev vs Conductor
- boxes.dev vs Crush
- boxes.dev vs Cursor CLI
- boxes.dev vs DeepSeek Harness (dsh)
- boxes.dev vs Devin
- boxes.dev vs Droid CLI
- boxes.dev vs Pi
- boxes.dev vs Emdash
- boxes.dev vs Friday
- boxes.dev vs Gemini CLI
- boxes.dev vs GitHub Copilot CLI
- boxes.dev vs goose
- boxes.dev vs Harness²
- boxes.dev vs HarnessRouter
- boxes.dev vs Jules
- boxes.dev vs Junie CLI
- boxes.dev vs Kilo Code CLI
- boxes.dev vs Kimi Code CLI
- boxes.dev vs Kiro CLI
- boxes.dev vs Letta
- boxes.dev vs mini-swe-agent
- boxes.dev vs MiniMax Code
- boxes.dev vs Mistral Vibe CLI
- boxes.dev vs Muse Code
- boxes.dev vs OpenAI Agents API
- boxes.dev vs OpenAI Codex
- boxes.dev vs OpenCode
- boxes.dev vs OpenHands
- boxes.dev vs OpenMausBot
- boxes.dev vs Orca
- boxes.dev vs Paseo
- boxes.dev vs Prime Agent
- boxes.dev vs Qoder CLI
- boxes.dev vs Qwen Code
- boxes.dev vs Mirror CLI
- boxes.dev vs Rovo Dev CLI
- boxes.dev vs Sculptor
- boxes.dev vs T3 Code
- boxes.dev vs Traycer
- boxes.dev vs Unstoppable
- boxes.dev vs Verdent
- boxes.dev vs Vibe-Trading
- boxes.dev vs Warp (Warp Agent CLI and the Oz Agent API)
- Claude Code vs dsh-superagent (DeepSeek)
- Click vs dsh-superagent (DeepSeek)
- Cline vs dsh-superagent (DeepSeek)
- Conductor vs dsh-superagent (DeepSeek)
- Crush vs dsh-superagent (DeepSeek)
- Cursor CLI vs dsh-superagent (DeepSeek)
- DeepSeek Harness (dsh) vs dsh-superagent (DeepSeek)
- dsh-superagent (DeepSeek) vs Devin
- dsh-superagent (DeepSeek) vs Droid CLI
- dsh-superagent (DeepSeek) vs Pi
- dsh-superagent (DeepSeek) vs Emdash
- dsh-superagent (DeepSeek) vs Friday
- dsh-superagent (DeepSeek) vs Gemini CLI
- dsh-superagent (DeepSeek) vs GitHub Copilot CLI
- dsh-superagent (DeepSeek) vs goose
- dsh-superagent (DeepSeek) vs Harness²
- dsh-superagent (DeepSeek) vs HarnessRouter
- dsh-superagent (DeepSeek) vs Jules
- dsh-superagent (DeepSeek) vs Junie CLI
- dsh-superagent (DeepSeek) vs Kilo Code CLI
- dsh-superagent (DeepSeek) vs Kimi Code CLI
- dsh-superagent (DeepSeek) vs Kiro CLI
- dsh-superagent (DeepSeek) vs Letta
- dsh-superagent (DeepSeek) vs mini-swe-agent
- dsh-superagent (DeepSeek) vs MiniMax Code
- dsh-superagent (DeepSeek) vs Mistral Vibe CLI
- dsh-superagent (DeepSeek) vs Muse Code
- dsh-superagent (DeepSeek) vs OpenAI Agents API
- dsh-superagent (DeepSeek) vs OpenAI Codex
- dsh-superagent (DeepSeek) vs OpenCode
- dsh-superagent (DeepSeek) vs OpenHands
- dsh-superagent (DeepSeek) vs OpenMausBot
- dsh-superagent (DeepSeek) vs Orca
- dsh-superagent (DeepSeek) vs Paseo
- dsh-superagent (DeepSeek) vs Prime Agent
- dsh-superagent (DeepSeek) vs Qoder CLI
- dsh-superagent (DeepSeek) vs Qwen Code
- dsh-superagent (DeepSeek) vs Mirror CLI
- dsh-superagent (DeepSeek) vs Rovo Dev CLI
- dsh-superagent (DeepSeek) vs Sculptor
- dsh-superagent (DeepSeek) vs T3 Code
- dsh-superagent (DeepSeek) vs Traycer
- dsh-superagent (DeepSeek) vs Unstoppable
- dsh-superagent (DeepSeek) vs Verdent
- dsh-superagent (DeepSeek) vs Vibe-Trading
- dsh-superagent (DeepSeek) vs Warp (Warp Agent CLI and the Oz Agent API)
- Agent Relay vs boxes.dev
- Agent Relay vs dsh-superagent (DeepSeek)
- boxes.dev vs Clawboo
- boxes.dev vs CLI Manager
- boxes.dev vs Harness
- boxes.dev vs herdr
- boxes.dev vs Paperclip
- boxes.dev vs Parallax
- Clawboo vs dsh-superagent (DeepSeek)
- CLI Manager vs dsh-superagent (DeepSeek)
- dsh-superagent (DeepSeek) vs Harness
- dsh-superagent (DeepSeek) vs herdr
- dsh-superagent (DeepSeek) vs Paperclip
- dsh-superagent (DeepSeek) vs Parallax
Machine-readable
- This page as Markdown
/compare/boxes-dev-vs-deepseek-superagent.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/boxes-dev.json·/api/v1/tools/deepseek-superagent.json - From a terminal
anchor compare boxes-dev deepseek-superagent(the CLI) - Over MCP
compare_tools {"a": "boxes-dev", "b": "deepseek-superagent"}at/mcp, no key