Head to head · Agent harnesses · October 2026 research run

dsh-superagent (DeepSeek) vs Devin

Devin scores 55.8 (C) on agent readiness against dsh-superagent (DeepSeek)'s 42.1 (E), and leads in 5 of 7 scored categories. dsh-superagent (DeepSeek) leads on agent ergonomics and payments & pricing. Both do agent harnesses.

Best agent harnesses and coding agents · All 955 harnesses comparisons

Which one, for what

dsh-superagent (DeepSeek) E

Best for Teams on github.com that want a DeepSeek Harness agent to turn allowed people's issue requests into pull requests, run on an isolated machine or OS account.

Ahead on

  • Agent ergonomics, 67 against 62
  • Payments & pricing, 40 against 20

Also in its favour

  • Free to start without a card
  • Open source

Watch for

Executor and main Sessions run with danger-full-access as the operator's OS user, and the docs say a worker can read the operator's home directory

Devin C

Best for A team that wants to hand whole tasks to a cloud agent and collect pull requests, driven from a pipeline or another agent.

Ahead on

  • Reliability, 30 against 15
  • Schema & documentation, 80 against 61
  • Security & auth, 72 against 33
  • Maintenance & community, 63 against 34
  • Transparency & trust, 70 against 52

Also in its favour

  • A hosted endpoint, with nothing to install

Watch for

www.devinstatus.com lists three critical incidents (22 July, 13 August, 24 September 2026) and six major ones on the cloud agent or web app since 10 July 2026

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Security & auth Devin +39

dsh-superagent (DeepSeek) 33

Harness reading of the framework checklist, 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. The superagent profile is created from DSH's Web template, and DSH documents that shipped profiles mount session-log-deepseek, enabled by default, which adds the session log to each official DeepSeek API request, up to 8 MiB. dsh-superagent's docs don't mention it, and its Sessions read issue, pull request and chat text and the repository. Setup and worker shells drop variables named KEY, PASSWORD, SECRET or TOKEN, and GH_TOKEN and GITHUB_TOKEN are removed from git (8). Both presets run danger-full-access without approval as the operator's OS user. docs/006 says the account wrappers don't isolate the process, a worker can run programs by path and read the operator's home directory, and recommends a dedicated OS account. The gate, observe default, blocked commands and no auto-merge as the agent account narrow what it does, not what it can do (6). docs/002 says messages, comments and quoted documents are inputs to a decision and can't change allowed lists, rules or Mandates, bots can never make explicit requests, and prompt edits don't change the gate. Workers with full access still read untrusted Thread text, and no further guidance on that was found (7). Decisions are recorded with reasons, Ticket details show the Trigger, Decision, scope and Policy version, the host records the prompt revisions each Session used, and every GitHub message starts with an automated-agent line by default (12). No SECURITY.md (404 at the root and in .github), no security.txt on www.deepseek.com (404), no bug bounty or advisory found. NVD has no record for the prototype. The CVEs against DSH itself are on the deepseek-harness listing (0).

Devin 72

Hosted reading. Service-user keys with roles and named permissions per endpoint, revocable, with rotation endpoints in beta for Enterprise and expiring personal tokens. Custom roles are an Enterprise option, and Teams has Admin and Member only (25). Security profiles restrict network, MCP servers, git and the Devin MCP to read-only, and read-only API permissions exist. They are opt-in, and session creation accepts bypass_approval (16). AI Guardrails screen user messages and pull request comments for prompt injection, on Enterprise only, and they don't cover web pages or repository content Devin reads (8). Audit logs through the API need an enterprise-level permission. Session events and insights are available on all plans (10). SOC 2 Type II and ISO/IEC 27001:2022, a disclosure address at security@cognition.ai, no security.txt, and the trust centre answers No to having a bug bounty (13).

Maintenance & community Devin +29

dsh-superagent (DeepSeek) 34

Published on 31 October 2026, within 30 days, read as the last published change since no tag or release was found (30). One publication, so not three releases in 90 days (0). Issues, pull requests and Discussions couldn't be read, and the README frames the code as a reference prototype adapted from an internal version (0). No official SDK or registry entry (0). Exact dependency pins and a lockfile, but no CI workflow found, and the docs pin DSH 0.2.1-rc.3 (23 October 2026) while npm latest is 0.2.1-rc.5 (29 October 2026) (4).

Devin 63

Closed service. Newest release note dated 7 October 2026 (30). Nineteen dated release notes between 7 August and 7 October 2026, and API notes for August and September (20). Public release notes and a support address, with no public tracker or forum found (10). No official SDK found and the Devin MCP server didn't appear in a search of the official MCP registry (0). No public CI or packages to assess, with the docs and three OpenAPI specs kept current (3).

Payments & pricing dsh-superagent (DeepSeek) +20

dsh-superagent (DeepSeek) 40

Self-hosted rule, as for the other open-source harnesses. No payment protocol (0). Free under MIT with nothing to buy (20). No card for the software (20). An agent can't get started alone, because the setup needs a person to create a dedicated GitHub user account, accept the repository invitation and run auth login for the wrappers, which SETUP.md lists as operator steps. 0, as the other harnesses that need an account were read (0). The model provider is a separate purchase.

Devin 20

Published rubric. No payment protocol (0). Plan prices are public, $20, $200 and $40 a seat with an $80 team minimum, but usage past the quota is billed at an unpublished rate (10). A Free plan exists. The pricing page lists cloud agents from Pro upward, and we couldn't confirm API use on Free or whether a card is needed (10). A person signs up in a browser and provisions the service user in settings (0).

Schema & documentation Devin +19

dsh-superagent (DeepSeek) 61

Framework reading, as for deepseek-harness. No OpenAPI or JSON Schema is published. docs/006 documents the Management API's routes, fields, block shapes and refusals in tables, and the code validates settings with zod (12). Markdown docs in the repository, readable raw, and a SETUP.md written for a coding agent. No llms.txt (5). Nine numbered documents read in order, each stating purpose, limits and what not to do, such as never rerunning the profile-creation command for an upgrade (17). Closed sets for accept mode (off, observe, auto), admission cells, Source states, health states and listen (12). A worked curl example, refusals as 400, 404, 409 and 500 with issues, setup or code, a troubleshooting table and status exit codes 0 to 3 (13). package.json says 0.1.0 and pins an engines.dsh range, but no changelog or tags were found (2).

Devin 80

API reading. Public OpenAPI 3.1 spec for v3 with 239 operations, and separate v1 and v2 specs (25). llms.txt with 376 lines and a Markdown twin of every page (10). Each endpoint page names its required permission, but 61 of 239 operations have no description and the MCP tool descriptions are one line each (10). Enums on fields such as devin_mode, origin and category, with most strings unconstrained and nullable (9). curl examples in the quick starts, and a problem+json error schema declared on every operation, without a catalogue of error_code values (11). Versioned paths (v1, v2, v3, v3beta1) and a dated API release notes page (15).

Transparency & trust Devin +18

dsh-superagent (DeepSeek) 52

MIT, Copyright (c) 2026 DeepSeek Harness Team (30). The prototype has no data statement of its own. DeepSeek Harness's privacy policy (last updated 20 September 2026) names Hangzhou DeepSeek Artificial Intelligence Co., Ltd., says session logs are collected with official model services and may be used to train models with a right to opt out, and says custom model services aren't stored server-side. It agrees with DSH's session-log README, but neither is referred to from the prototype (12). No deprecation policy. The README says the published code differs from DeepSeek's internal version, and docs/006 warns that database migrations are one-way (3). Telemetry and the session-log upload are disclosed with opt-outs in DSH's own reference pages, not in this repository (6).

Devin 70

Editorial half. Closed source with clear terms from Cognition AI, Inc. (15). Terms, privacy policy, security page and data processing agreement agree with each other. Customer data may be used for model training by default on self-serve plans, with an opt-out on paid plans that also turns on zero data retention at model providers. Retention is stated as the length of the customer relationship, with no periods (18). Dated notices for the schedules endpoints (announced 10 September, creation refused from 24 September 2026) and the v2 clone endpoint, but no deprecation policy and no end date for v1 and v2 (10). Sub-processors listed with purpose and country, and 15 days' notice of additions (18).

Reliability Devin +15

dsh-superagent (DeepSeek) 15

Local-package reading, since it runs as a DSH bundle on the operator's machine. Not published to npm (private: true in package.json, and the registry has no dsh-superagent). It installs from a git clone with pnpm install && pnpm build on top of @deepseek-ai/dsh@0.2.1-rc.3, with Node 22.19+ or 24+, git 2.32+, Python 3.8+, tmux and gh 2.48+ stated, so half credit (10). A vitest suite of about 25 spec files is listed in docs/007, and pnpm check runs the typecheck, tests and build. Two spec files were read on raw.githubusercontent.com. No CI workflow was found at five common paths under .github/workflows, and the directory couldn't be listed (5). Open issues couldn't be read, because github.com and api.github.com weren't reachable from our network. Unchecked, so scored absent (0). Version 0.1.0 with no tag, release or changelog found, so no semver record (0). A research prototype, not stable (0).

Devin 30

Hosted reading. Status page at www.devinstatus.com with 14 components and incident history (20). In the 90 days to 8 October 2026 it lists three critical incidents (22 July, 43 minutes of platform-wide queuing, 13 August, Devin Cloud unavailable for 2 hours 40 minutes, 24 September, the web app down for 2 hours) and six major ones on session start-up, so several majors (0). 429 is a documented response, but no rate limit figures were found (0). No Retry-After or backoff guidance found, and v3 session creation has no idempotency key, although the legacy v1 API had one (0). The platform terms supply the service as is with no service level, and enterprise terms weren't read (0). The v3 API is the current, recommended surface, with 44 of 239 operations marked beta under /v3beta1 (10).

Score by category

CategoryWeight this rundsh-superagent (DeepSeek)DevinEdge
Reliability16%201530Devin +15
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.26180Devin +19
Agent ergonomics13%16.26762dsh-superagent (DeepSeek) +5
Security & auth14%17.53372Devin +39
Payments & pricing10%12.54020dsh-superagent (DeepSeek) +20
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.83463Devin +29
Transparency & trust7%8.85270Devin +18
Negative events≤1500
TotalE 42.1/100C 55.8/100

Facts side by side

Factdsh-superagent (DeepSeek)Devin
KindAgent harnessAgent harness
VendorDeepSeekCognition AI, Inc.
Hosted endpointno (local only)https://mcp.devin.ai/mcp
TransportsHTTP
AuthTokenAPI key
PricingFreeFreemium
x402nono
LicenceMITProprietary service under Cognition's Platform Terms of Service
Tools exposednone13
Read-only variant documentednoyes
llms.txtnoyes
Last release2026-10-312026-10-07
Terms last updatedno document linked2026-06-30
Privacy policy last updated2026-09-202026-03-09
Customer content may train modelsnot read yetyes, with an opt-out
Terms restrict automated accessnot read yetnot found in the text
Terms restrict benchmarkingnot read yetyes
Terms or service can change without noticenot read yetnot found in the text
Arbitration or class-action waivernot read yetyes

How the scores have moved

Verdicts

dsh-superagent (DeepSeek)

The authorization design is careful, with allowed lists, Mandates and a Policy enforced in code, proactive work only recorded by default and auto-merge off. Executors run every command without approval under the operator's OS user, and the README calls it a research prototype with no release, changelog or security policy.

Devin

The v3 API is well specified, with a public OpenAPI 3.1 file covering 239 operations, problem+json errors, cursor pagination and service-user keys tied to roles. The status page records three critical and several major incidents on the cloud agent between 22 July and 24 September 2026, and no rate limit figures or retry guidance were found in the reviewed documentation.

Before you call either

dsh-superagent (DeepSeek)

  1. Run it under a dedicated OS account holding only the agent's credentials. Executors run commands without approval and can read the operator's home directory
  2. Set enabled: false on the session-log-deepseek row of the superagent profile before using DeepSeek's official API on private code
  3. Keep the accept mode at observe and the allowed list short until the recorded Decisions look right
  4. Wait for scripts/dsh-superagent status to stop exiting 1 after any save that answers restarting: true before the next request
  5. Send x-orchestrator-request: 1 and a loopback Host to the Management API, and send revision with each PUT, since a stale one gets 409

Devin

  1. Use a cog_ service-user key with the Member role. Legacy apk_ keys fail against v3 and the MCP server with 401 or 403
  2. Set max_acu_limit on every session you create. Usage is metered by the work done and has no published unit rate
  3. Session creation is not idempotent in v3. After a timeout, list sessions by tag before creating again
  4. Enterprise keys and personal access tokens must send X-Org-Id to the MCP server. Organisation-scoped keys resolve it automatically
  5. Create scheduled work as an automation. POST to the schedules endpoint returns 403 for migrated organisations since 24 September 2026

Questions

Which is better for AI agents, dsh-superagent (DeepSeek) or Devin?

Devin scores 55.8 (C) on agent readiness against dsh-superagent (DeepSeek)'s 42.1 (E), and leads in 5 of 7 scored categories. dsh-superagent (DeepSeek) leads on agent ergonomics and payments & pricing.

Are dsh-superagent (DeepSeek) and Devin open source?

dsh-superagent (DeepSeek) is open source (MIT). No open-source release is listed for Devin.

Other comparisons with dsh-superagent (DeepSeek) or Devin

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.