{
  "fixes": {
    "slug": "deepseek-harness",
    "name": "DeepSeek Harness (dsh)",
    "listing": "https://www.anchorterminal.com/tools/deepseek-harness",
    "markdown": "# Fix list: DeepSeek Harness (dsh)\n\nFrom Anchor Terminal's listing at https://www.anchorterminal.com/tools/deepseek-harness, the October 2026 research run, assessed 18 October 2026. Grade D, 49.2 out of 100.\n\nThis is everything the published grade says the listing lacks, the biggest possible gain to the total first. It comes from the reason given for each score, the checklist each category was scored against (https://www.anchorterminal.com/benchmark/#checklist), the provenance checks, the deductions, what we couldn't check and what the review panel asked for. A fix counts at the next check, once it's public.\n\nFor a coding agent working on DeepSeek Harness (dsh): work through the items below in the product, its docs and its public pages. Each category gives the reason for its score, with the points each checklist item earned, and the checklist itself, so the gap is the items that earned less than their points. Change the product, not the wording, and keep a note of what you changed and where it's published.\n\n## 1. Reliability, 33 out of 100, up to 13.4 more on the total\n\nWhy it scored 33: Local-package reading, graded on npm `latest` 0.2.0-rc.2, which the README's `npx @deepseek-ai/dsh` installs. Official npm package with Node 22.19+ and 24+ stated, and a Python wheel for Python 3.10+ on Linux x64 and arm64, macOS 14+ on arm64 and Windows x64 that bundles the runtime (20). A public CI workflow (.github/workflows/ci.yml) with a Node 22.19, 24 and 26 matrix, a separate real-API end-to-end workflow and 25 workflow files in the clone of 18 October, but run results couldn't be read, so whether it passes on master is unconfirmed (10). shields.io reads 0 open and 0 closed issues and 0 pull requests on 18 October. The README and CONTRIBUTING.md send bug reports to GitHub Discussions and refuse external pull requests, so issues are probably turned off, though the counts alone are ambiguous. Discussions couldn't be read, so crash reports and their handling are unchecked and scored absent (0). Alpha, next and latest dist-tags separate channels, but no changelog or breaking-change notes were found and the README warns generally of breaking changes (3). Developer preview, no 1.0 or stable release (0).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-reliability):\n\nHosted APIs, MCP servers, models and platforms.\n\n- 20, a public status page with component history (Statuspage, Instatus, BetterStack or the vendor's own).\n- 0 to 30, the incident record for the last 90 days on that page. 30 for a clean record or trivial incidents only, 20 for minor incidents only, 10 for one major outage (an hour or more of a core API down, or errors across the board), 0 for several. 5 when there's no history we could read, and the note says so.\n- 15, rate limits documented with numbers.\n- 15, documented 429 or overload handling (Retry-After, backoff guidance), and idempotency keys or safe-retry guidance where writes are involved.\n- 10, an SLA published for any paid tier.\n- 10, the surface agents use is generally available, not beta or preview.\n\nLocal packages, SDKs, frameworks and stdio MCP servers.\n\n- 20, installs from an official package with supported runtimes stated.\n- 25, a public CI and test suite, passing on the default branch.\n- 0 to 25, open crash or regression issues relative to activity (25 for few and handled, 0 for many, old and unanswered).\n- 15, semver discipline and breaking changes called out in a changelog.\n- 15, version 1.0 or later, or declared stable.\n\nProtocols are read from their reference implementations, the public facilitators or servers, spec stability and test vectors.\n\n## 2. Security \u0026 auth, 40 out of 100, up to 10.5 more on the total\n\nWhy it scored 40: Harness reading of the framework checklist, 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. The session-log contributor is on by default and sends complete unaccepted session-log events (message text, tool arguments and results, workspace paths) with every request to DeepSeek's official API, up to 8 MiB a request. OpenTelemetry defaults to `FEEDBACK_ONLY` and sends the session prefix to dsh-otel-collector.deepseeksvc.com after any feedback or rating. Both are documented with opt-outs in the CLI reference and package READMEs, but not in the README or SAFETY.md. Keys stay local (8). `workspace-write` plus `ask` by default, with writes confined by bwrap, Landlock, Seatbelt or a Windows ACL runner, and an unanswered ask denied in headless and ACP runs. Reads and network are not confined, `web_fetch` runs without approval (public destinations only), and the SDK's `sdk-minimal` profile pins `danger-full-access` with no approval service (13). SAFETY.md warns of malicious input and untrusted plugins, MCP server names are treated as untrusted, and no MCP server is enabled by default, but no prompt-injection guidance was found, and CVE-2026-95183 reports injection through context compaction (6). Local JSONL session logs record approval decisions as audit events (12). No SECURITY.md (none in a clone of 18 October either), no security.txt, no bug bounty, and OSV lists no advisory for `@deepseek-ai/dsh` or `deepseek-harness-sdk`. SAFETY.md says the project has had no security audit, and six of the nine NVD records say the vendor did not respond to the reporter (1).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-security):\n\n- 0 to 30, the credential model. 30 for OAuth 2.1 with scopes, or scoped and revocable keys with rotation. 20 for plain revocable API keys. 10 for one all-powerful key. 10 off when a secret can travel in a URL query string as a documented option.\n- 0 to 20, read-only or least-privilege modes, and confirmation or approval for destructive actions.\n- 0 to 15, prompt-injection posture where the tool returns untrusted content (documented mitigations or guidance). A tool that returns no untrusted content gets 10.\n- 0 to 15, audit logs or per-call visibility for the operator.\n- 0 to 20, a security programme. security.txt or a disclosure policy, a bug bounty, SOC 2 or ISO 27001, advisories handled in public.\n\nModels are read for retention, whether API data trains models (and whether that's off by default), zero-retention options and certifications. Frameworks for telemetry defaults, approval hooks, guardrails and sandboxing.\n\n## 3. Agent ergonomics, 68 out of 100, up to 5.2 more on the total\n\nWhy it scored 68: Harness reading, scored on what a pipeline driving it has to supply. Profiles compose the tool set from plugins, `sdk-minimal` exposes three tools, and no MCP server is enabled by default (18). Automatic context compaction and a 300-second shell timeout in `sdk-minimal`, but no turn, time or cost limit was found for headless runs (8). Exit 0 for a completed turn and 1 otherwise, stderr diagnostics, and an NDJSON stream with `session`, `final` and `error` events, though a failed in-turn run still ends with `final` (15). `--session-id` resumes a persisted session, ACP has `session/list`, `session/resume` and `session/cancel`, and an unanswered approval is denied rather than left hanging (16). One-argument headless runs, ACP and a Python SDK, with no TypeScript SDK found and a model key required (11).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-ergonomics):\n\n- 0 to 25, context cost. For MCP, the number and size of the tool definitions (25 for ten or fewer compact tools, 15 for 11 to 30, 5 for more than 30, plus up to 10 back for toolsets, dynamic loading or read-only subsets). For APIs, whether responses can be sized (field selection, limits, summaries).\n- 20, pagination, filtering and output-size controls.\n- 20, actionable, documented error responses, codes and messages an agent can recover from.\n- 20, idempotency or safe retries, and for MCP the `readOnlyHint` and `destructiveHint` annotations.\n- 15, sensible defaults, few required parameters, and official SDKs in at least two languages.\n\nModels are read for tool use, structured output, prompt caching, context length, batch and SDKs. Frameworks for how much code and how many defaults a tool-calling agent with MCP needs.\n\n## 4. Payments \u0026 pricing, 60 out of 100, up to 5 more on the total\n\nWhy it scored 60: Self-hosted rule, as for the other open-source harnesses. No payment protocol (0). The harness is free under MIT with nothing to buy (20). No card or account is needed for the software, and custom endpoints include self-hosted model servers (20). An agent can install it with npx or pip and run it with no sign-up, given a model endpoint (20). DeepSeek's API is a separate purchase.\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-payments):\n\nThe published rubric, also on the [x402 page](https://www.anchorterminal.com/x402/).\n\n- 40, a machine payment protocol (x402, MPP or L402) on the tool's own endpoints. 10 to 30 when it covers only some endpoints or only goes through a third party, and the note says which.\n- 20, per-call or per-unit pricing published without a login. 10 for public plan-only pricing, 0 for \"contact sales\" or prices behind a login.\n- 20, a free tier or trial that doesn't need a card.\n- 20, autonomous onboarding, meaning an agent can get access without a person signing up in a browser (keyless use, x402, a programmatic key API).\n\nPayment platforms and agent wallets rarely charge for their own API over a machine protocol, so the first line has steps for them, and the highest one that applies counts. 40 when x402, MPP or L402 runs on all their own endpoints, 30 when it runs on part of their own API, 25 when their merchants can accept one, 20 for running a facilitator, 15 for paying as a buyer, and 0 when the only protocol is their own. Merchant acceptance sits above a facilitator because the platform's own customers can charge agents through it, while a facilitator settles for sellers who wire up the protocol themselves. The counter-argument (a facilitator does more for the protocol as a whole) has a point. Each note says which step applied.\n\nOpen-source software you run yourself is scored on its hosted or paid option if it has one. A free, self-hosted package with nothing to buy gets 20, 20 and 20 for the last three lines, and 0 to 40 for the first only if it ships a payment protocol.\n\n## 5. Transparency \u0026 trust, 50 out of 100, up to 4.4 more on the total\n\nMade of editorial 59, provenance 40.\n\nWhy it scored 50: MIT, Copyright 2026 DeepSeek, with third-party licences in THIRD_PARTY_NOTICES.md (30). The package READMEs state what leaves the machine, the defaults, the collector address and that exports can include message text, tool arguments, results and workspace paths, but the project has no privacy policy or retention statement of its own, and uploads fall under DeepSeek's platform policy (14). The README warns of breaking changes, but no changelog, deprecation notices or dates were found (3). Telemetry is disclosed with opt-outs by setting, Web switch and environment variable, though only in reference pages and not at the top of the README or in SAFETY.md (12).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-transparency):\n\n- 0 to 30, source availability and licence clarity. 30 for open source under an OSI licence, 15 for closed with clear terms, 0 for unclear terms.\n- 0 to 30, data handling and retention statements that agree with each other (privacy policy, DPA, retention periods, subprocessors).\n- 0 to 20, a deprecation policy or notices with dates.\n- 0 to 20, telemetry disclosed with an opt-out (local software), or subprocessors and data locations disclosed (hosted).\n\nThe other half of Transparency and trust is the provenance score, computed from checked facts (below). The category score is the mean of the two.\n\nProvenance checks not met in full (half of this category, computed from checked facts):\n\n- Domain age: deepseek-harness.github.io, no registry record we could read (0 of 15)\n- Status page: not found (0 of 10)\n- Changelog: not found (0 of 10)\n- security.txt: not found (0 of 10)\n\n## 6. Schema \u0026 documentation, 77 out of 100, up to 3.7 more on the total\n\nWhy it scored 77: Framework reading. `--dump-config-schema` prints a JSON Schema 2020-12 document for the composed configuration and patch layers, the headless `--json` event vocabulary is documented, and ACP v1 and the SDK's JSON-RPC are named protocols, though the reference calls the schema output pre-stable (22). llms.txt on the docs site, with a raw Markdown address for every page (10). Package READMEs have use and avoid sections and known limitations, such as headless being one task per run (17). Permission presets, sandbox modes, approval policies and telemetry modes are closed enums (13). Examples for every mode, exit codes 0 and 1, `dsh: \u003ccode\u003e: \u003cmessage\u003e` on stderr, `error` events in `--json` and startup failure reports under `$DSH_HOME/logs/` (12). Versions are on npm and PyPI, with generated CLI help, but no public changelog was found (3).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-schema):\n\nAPIs and MCP servers.\n\n- 25, a machine-readable contract (a public OpenAPI file or similar; for MCP, typed JSON Schema inputs on every tool).\n- 10, llms.txt or Markdown docs served for agents.\n- 0 to 20, descriptions that say what a tool is for, when to use it and when not to, read from the tool definitions in the source or the API reference.\n- 0 to 15, typed inputs with enums, constraints and required fields, and no free-form JSON blobs.\n- 0 to 15, examples and documented error responses.\n- 15, versioning and a public changelog.\n\nModels are read from the API reference, the OpenAPI file, llms.txt, the structured-output and tool-use docs and the model cards. Frameworks from docs a model can follow, typed interfaces, examples and the API reference.\n\n## 7. Maintenance \u0026 community, 71 out of 100, up to 2.5 more on the total\n\nWhy it scored 71: 0.2.1-rc.2 on npm and 0.2.1rc2 on PyPI on 17 October 2026 (30). 36 npm versions since 10 August 2026, with 14 since 18 September, and 32 `dsh-v` tags in the 90 days to 18 October in a clone of the repository (20). CONTRIBUTING.md says the team can't accept external pull requests and may not reply to every Discussions post. shields.io reads 0 issues and 0 pull requests, open or closed, which fits issues being turned off. The clone shows 22,395 commits since 10 June 2026, 8,645 of them merges, which says little about replies to users. Reply times in Discussions are unchecked (5). The Python SDK ships with each release, still as a pre-release (10). A CI workflow with a Node version matrix exists, with its current state unconfirmed (6).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-maintenance):\n\n- 0 to 30, time since the last release, or the last published model or API change for a closed service. 30 within 30 days, 20 within 90, 10 within 180, 0 older.\n- 20, at least three releases or dated changelog entries in the last 90 days.\n- 0 to 25, responsiveness. Issues and pull requests answered on GitHub (the open issues and how recent the replies are). For closed services, a public changelog and a support or community channel that answers, 0 to 15.\n- 15, presence in the official MCP registry under a verified namespace (MCP servers), or current official SDKs (APIs and models).\n- 10, package health, current dependencies and CI.\n\nModels are read for deprecation notice periods and model churn rather than release counts.\n\n## Deductions\n\nEach comes off the total. A fixed and documented problem counts for less at the next check.\n\n- 2026-09-08. CVE-2026-82533 (CVSS 9.6, published by VulnCheck). The local HTTP agent-control API trusted a client-supplied loopback Host header, so a sandboxed tool process could reach it, escape the sandbox and turn off approvals, and a tunnelled port exposed it remotely. Fixed in 0.1.2-alpha.1 with commit 3e24087 per the NVD record. Fixed, inside six months, -2. https://nvd.nist.gov/vuln/detail/CVE-2026-82533\n- 2026-09-28 to 2026-10-16. Eight further CVEs are listed for DeepSeek Harness in NVD, all marked Deferred and none with a patched version. Six came through VulDB on 28 September and 12 October 2026 and name versions up to 0.1.0-rc.7, 0.1.1-rc.2, 0.1.5-rc.3 and 0.1.7-rc.2, with CVSS 3.1 scores of 3.1 to 6.3. They are CVE-2026-101078 (Landlock mount escape), CVE-2026-101102 (Code Mode worker, which SAFETY.md and the design notes call containment rather than a security boundary), CVE-2026-101131 (E2B_API_KEY trusted in a security decision), CVE-2026-101132 (path traversal through `dsh.bundle.patch`), CVE-2026-108788 (sandbox spawn) and CVE-2026-108789 (workflow worker). Each says the vendor was contacted early and did not respond. CVE-2026-95182 (CVSS 10.0, untrusted search path) and CVE-2026-95183 (prompt injection through context compaction) of 16 October 2026 name commit c291e796 and rest on third-party gists we didn't read. Read the way the PicoClaw note was. The project has published no advisory and no security policy (OSV lists none for `@deepseek-ai/dsh` or `deepseek-harness-sdk`, and a clone of 18 October has no security policy file), and six reporters say it never replied, as with PicoClaw. We read all eight NVD records but none of the write-ups. There are eight entries rather than fifteen, the six VulDB ones predate the graded 0.2.0-rc.2 line, and the one fix we can see (CVE-2026-82533, deducted above) shows some handling, so -4 against PicoClaw's -5. https://services.nvd.nist.gov/rest/json/cves/2.0?keywordSearch=deepseek-harness\n\n## What we couldn't check\n\nWhat we couldn't read counted as absent. Publishing it on a page a plain HTTP fetch can read (not only in a browser) lets the next check count it.\n\n- unchecked: GitHub stars, CI run results, Discussions and maintainer reply times. shields.io reads 0 issues and 0 pull requests, open or closed, on 18 October, which probably means issues are turned off but is ambiguous on its own\n- unchecked: the PyPI project page, which answered with a challenge, so PyPI download counts were not read\n- CVE-2026-95182 (CVSS 10.0, untrusted search path) and CVE-2026-95183 (prompt injection through compaction), both against commit c291e796 and published 16 October 2026, rest on third-party gists we didn't read. Whether 0.2.x is affected is unknown. Counted in the -4 for unconfirmed CVEs\n- Six VulDB-sourced CVEs (28 September and 12 October 2026) against 0.1.x versions, including a Landlock mount escape, say the vendor did not respond. Whether they are fixed in 0.2.x is unknown. Counted in the -4 for unconfirmed CVEs, read as the PicoClaw note was\n- Whether the Web UI or desktop app shows the session-log upload at first run. We didn't run the software\n- The docs read are on master, nearer 0.2.1 than npm `latest` 0.2.0-rc.2, which the grade covers\n- How long DeepSeek keeps uploaded session logs and whether it trains on them. The project states neither, and DeepSeek's platform privacy policy wasn't re-read today\n\n## Weaknesses\n\n- By default each request to DeepSeek's official API carries the session log, including message text, tool arguments and results, up to 8 MiB a request\n- Developer preview with no stable release. npm `latest` is 0.2.0-rc.2, and no changelog or breaking-change notes were found\n- The sandbox confines writes only. Reads and network access are not confined, and `web_fetch` runs without approval\n- No SECURITY.md or security.txt. NVD lists nine CVEs from September and October 2026, six saying the vendor did not respond\n- The Python SDK's minimal profile pins `danger-full-access` with no approval service, and external pull requests are not accepted\n\n## What costs an agent a turn today\n\nThe notes we give agents before they call it. Each one is a workaround an agent shouldn't need.\n\n- Set `enabled: false` on the `session-log-deepseek` row in a profile patch before running on private code. Otherwise the session log goes to DeepSeek with each official API request\n- Set `DSH_TELEMETRY_MODE=DISABLED` to stop OpenTelemetry uploads, which send the session prefix to a DeepSeek collector after any feedback\n- Treat exit code 1 and the `turn_end` reason as failure. A `--json` stream can end with a `final` event for a failed run\n- Run in a disposable container or VM. The sandbox doesn't confine reads or network, and SAFETY.md says the project has had no security audit\n- Pin the version. npm `latest` (0.2.0-rc.2) and `next` (0.2.1-rc.2) differ, DeepSeek's changelog asks for 0.2.1-rc.1 or later, and the README warns of breaking changes\n\n## What the review panel asked for\n\n- upload notice in the README\n- `latest` tag matching the changelog\n- publish a changelog\n- promote fixes to latest\n- failure status on final events\n- public changelog\n- default-off session upload\n- publish a security policy\n\n## When it's done\n\nSend what changed and where it's published as a dispute (https://www.anchorterminal.com/builders/#disputes, or `POST https://www.anchorterminal.com/api/v1/contact` with `\"kind\": \"dispute\"`). Disputes are answered in public, and the listing is checked again by the same checklist. Paying for an audit or a listing claim changes nothing here.\n",
    "grade": "D",
    "score": 49.2,
    "assessed": "2026-10-18",
    "run": "October 2026 research run",
    "categories": [
      {
        "key": "reliability",
        "name": "Reliability",
        "score": 33,
        "maxGain": 13.4,
        "reason": "Local-package reading, graded on npm `latest` 0.2.0-rc.2, which the README's `npx @deepseek-ai/dsh` installs. Official npm package with Node 22.19+ and 24+ stated, and a Python wheel for Python 3.10+ on Linux x64 and arm64, macOS 14+ on arm64 and Windows x64 that bundles the runtime (20). A public CI workflow (.github/workflows/ci.yml) with a Node 22.19, 24 and 26 matrix, a separate real-API end-to-end workflow and 25 workflow files in the clone of 18 October, but run results couldn't be read, so whether it passes on master is unconfirmed (10). shields.io reads 0 open and 0 closed issues and 0 pull requests on 18 October. The README and CONTRIBUTING.md send bug reports to GitHub Discussions and refuse external pull requests, so issues are probably turned off, though the counts alone are ambiguous. Discussions couldn't be read, so crash reports and their handling are unchecked and scored absent (0). Alpha, next and latest dist-tags separate channels, but no changelog or breaking-change notes were found and the README warns generally of breaking changes (3). Developer preview, no 1.0 or stable release (0).",
        "checklist": [
          "Hosted APIs, MCP servers, models and platforms.",
          "- 20, a public status page with component history (Statuspage, Instatus, BetterStack or the vendor's own).\n- 0 to 30, the incident record for the last 90 days on that page. 30 for a clean record or trivial incidents only, 20 for minor incidents only, 10 for one major outage (an hour or more of a core API down, or errors across the board), 0 for several. 5 when there's no history we could read, and the note says so.\n- 15, rate limits documented with numbers.\n- 15, documented 429 or overload handling (Retry-After, backoff guidance), and idempotency keys or safe-retry guidance where writes are involved.\n- 10, an SLA published for any paid tier.\n- 10, the surface agents use is generally available, not beta or preview.",
          "Local packages, SDKs, frameworks and stdio MCP servers.",
          "- 20, installs from an official package with supported runtimes stated.\n- 25, a public CI and test suite, passing on the default branch.\n- 0 to 25, open crash or regression issues relative to activity (25 for few and handled, 0 for many, old and unanswered).\n- 15, semver discipline and breaking changes called out in a changelog.\n- 15, version 1.0 or later, or declared stable.",
          "Protocols are read from their reference implementations, the public facilitators or servers, spec stability and test vectors."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-reliability"
      },
      {
        "key": "security",
        "name": "Security \u0026 auth",
        "score": 40,
        "maxGain": 10.5,
        "reason": "Harness reading of the framework checklist, 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. The session-log contributor is on by default and sends complete unaccepted session-log events (message text, tool arguments and results, workspace paths) with every request to DeepSeek's official API, up to 8 MiB a request. OpenTelemetry defaults to `FEEDBACK_ONLY` and sends the session prefix to dsh-otel-collector.deepseeksvc.com after any feedback or rating. Both are documented with opt-outs in the CLI reference and package READMEs, but not in the README or SAFETY.md. Keys stay local (8). `workspace-write` plus `ask` by default, with writes confined by bwrap, Landlock, Seatbelt or a Windows ACL runner, and an unanswered ask denied in headless and ACP runs. Reads and network are not confined, `web_fetch` runs without approval (public destinations only), and the SDK's `sdk-minimal` profile pins `danger-full-access` with no approval service (13). SAFETY.md warns of malicious input and untrusted plugins, MCP server names are treated as untrusted, and no MCP server is enabled by default, but no prompt-injection guidance was found, and CVE-2026-95183 reports injection through context compaction (6). Local JSONL session logs record approval decisions as audit events (12). No SECURITY.md (none in a clone of 18 October either), no security.txt, no bug bounty, and OSV lists no advisory for `@deepseek-ai/dsh` or `deepseek-harness-sdk`. SAFETY.md says the project has had no security audit, and six of the nine NVD records say the vendor did not respond to the reporter (1).",
        "checklist": [
          "- 0 to 30, the credential model. 30 for OAuth 2.1 with scopes, or scoped and revocable keys with rotation. 20 for plain revocable API keys. 10 for one all-powerful key. 10 off when a secret can travel in a URL query string as a documented option.\n- 0 to 20, read-only or least-privilege modes, and confirmation or approval for destructive actions.\n- 0 to 15, prompt-injection posture where the tool returns untrusted content (documented mitigations or guidance). A tool that returns no untrusted content gets 10.\n- 0 to 15, audit logs or per-call visibility for the operator.\n- 0 to 20, a security programme. security.txt or a disclosure policy, a bug bounty, SOC 2 or ISO 27001, advisories handled in public.",
          "Models are read for retention, whether API data trains models (and whether that's off by default), zero-retention options and certifications. Frameworks for telemetry defaults, approval hooks, guardrails and sandboxing."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-security"
      },
      {
        "key": "ergonomics",
        "name": "Agent ergonomics",
        "score": 68,
        "maxGain": 5.2,
        "reason": "Harness reading, scored on what a pipeline driving it has to supply. Profiles compose the tool set from plugins, `sdk-minimal` exposes three tools, and no MCP server is enabled by default (18). Automatic context compaction and a 300-second shell timeout in `sdk-minimal`, but no turn, time or cost limit was found for headless runs (8). Exit 0 for a completed turn and 1 otherwise, stderr diagnostics, and an NDJSON stream with `session`, `final` and `error` events, though a failed in-turn run still ends with `final` (15). `--session-id` resumes a persisted session, ACP has `session/list`, `session/resume` and `session/cancel`, and an unanswered approval is denied rather than left hanging (16). One-argument headless runs, ACP and a Python SDK, with no TypeScript SDK found and a model key required (11).",
        "checklist": [
          "- 0 to 25, context cost. For MCP, the number and size of the tool definitions (25 for ten or fewer compact tools, 15 for 11 to 30, 5 for more than 30, plus up to 10 back for toolsets, dynamic loading or read-only subsets). For APIs, whether responses can be sized (field selection, limits, summaries).\n- 20, pagination, filtering and output-size controls.\n- 20, actionable, documented error responses, codes and messages an agent can recover from.\n- 20, idempotency or safe retries, and for MCP the `readOnlyHint` and `destructiveHint` annotations.\n- 15, sensible defaults, few required parameters, and official SDKs in at least two languages.",
          "Models are read for tool use, structured output, prompt caching, context length, batch and SDKs. Frameworks for how much code and how many defaults a tool-calling agent with MCP needs."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-ergonomics"
      },
      {
        "key": "payments",
        "name": "Payments \u0026 pricing",
        "score": 60,
        "maxGain": 5,
        "reason": "Self-hosted rule, as for the other open-source harnesses. No payment protocol (0). The harness is free under MIT with nothing to buy (20). No card or account is needed for the software, and custom endpoints include self-hosted model servers (20). An agent can install it with npx or pip and run it with no sign-up, given a model endpoint (20). DeepSeek's API is a separate purchase.",
        "checklist": [
          "The published rubric, also on the [x402 page](https://www.anchorterminal.com/x402/).",
          "- 40, a machine payment protocol (x402, MPP or L402) on the tool's own endpoints. 10 to 30 when it covers only some endpoints or only goes through a third party, and the note says which.\n- 20, per-call or per-unit pricing published without a login. 10 for public plan-only pricing, 0 for \"contact sales\" or prices behind a login.\n- 20, a free tier or trial that doesn't need a card.\n- 20, autonomous onboarding, meaning an agent can get access without a person signing up in a browser (keyless use, x402, a programmatic key API).",
          "Payment platforms and agent wallets rarely charge for their own API over a machine protocol, so the first line has steps for them, and the highest one that applies counts. 40 when x402, MPP or L402 runs on all their own endpoints, 30 when it runs on part of their own API, 25 when their merchants can accept one, 20 for running a facilitator, 15 for paying as a buyer, and 0 when the only protocol is their own. Merchant acceptance sits above a facilitator because the platform's own customers can charge agents through it, while a facilitator settles for sellers who wire up the protocol themselves. The counter-argument (a facilitator does more for the protocol as a whole) has a point. Each note says which step applied.",
          "Open-source software you run yourself is scored on its hosted or paid option if it has one. A free, self-hosted package with nothing to buy gets 20, 20 and 20 for the last three lines, and 0 to 40 for the first only if it ships a payment protocol."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-payments"
      },
      {
        "key": "transparency",
        "name": "Transparency \u0026 trust",
        "score": 50,
        "maxGain": 4.4,
        "reason": "MIT, Copyright 2026 DeepSeek, with third-party licences in THIRD_PARTY_NOTICES.md (30). The package READMEs state what leaves the machine, the defaults, the collector address and that exports can include message text, tool arguments, results and workspace paths, but the project has no privacy policy or retention statement of its own, and uploads fall under DeepSeek's platform policy (14). The README warns of breaking changes, but no changelog, deprecation notices or dates were found (3). Telemetry is disclosed with opt-outs by setting, Web switch and environment variable, though only in reference pages and not at the top of the README or in SAFETY.md (12).",
        "blend": "editorial 59, provenance 40",
        "checklist": [
          "- 0 to 30, source availability and licence clarity. 30 for open source under an OSI licence, 15 for closed with clear terms, 0 for unclear terms.\n- 0 to 30, data handling and retention statements that agree with each other (privacy policy, DPA, retention periods, subprocessors).\n- 0 to 20, a deprecation policy or notices with dates.\n- 0 to 20, telemetry disclosed with an opt-out (local software), or subprocessors and data locations disclosed (hosted).",
          "The other half of Transparency and trust is the provenance score, computed from checked facts (below). The category score is the mean of the two."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-transparency"
      },
      {
        "key": "schema",
        "name": "Schema \u0026 documentation",
        "score": 77,
        "maxGain": 3.7,
        "reason": "Framework reading. `--dump-config-schema` prints a JSON Schema 2020-12 document for the composed configuration and patch layers, the headless `--json` event vocabulary is documented, and ACP v1 and the SDK's JSON-RPC are named protocols, though the reference calls the schema output pre-stable (22). llms.txt on the docs site, with a raw Markdown address for every page (10). Package READMEs have use and avoid sections and known limitations, such as headless being one task per run (17). Permission presets, sandbox modes, approval policies and telemetry modes are closed enums (13). Examples for every mode, exit codes 0 and 1, `dsh: \u003ccode\u003e: \u003cmessage\u003e` on stderr, `error` events in `--json` and startup failure reports under `$DSH_HOME/logs/` (12). Versions are on npm and PyPI, with generated CLI help, but no public changelog was found (3).",
        "checklist": [
          "APIs and MCP servers.",
          "- 25, a machine-readable contract (a public OpenAPI file or similar; for MCP, typed JSON Schema inputs on every tool).\n- 10, llms.txt or Markdown docs served for agents.\n- 0 to 20, descriptions that say what a tool is for, when to use it and when not to, read from the tool definitions in the source or the API reference.\n- 0 to 15, typed inputs with enums, constraints and required fields, and no free-form JSON blobs.\n- 0 to 15, examples and documented error responses.\n- 15, versioning and a public changelog.",
          "Models are read from the API reference, the OpenAPI file, llms.txt, the structured-output and tool-use docs and the model cards. Frameworks from docs a model can follow, typed interfaces, examples and the API reference."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-schema"
      },
      {
        "key": "maintenance",
        "name": "Maintenance \u0026 community",
        "score": 71,
        "maxGain": 2.5,
        "reason": "0.2.1-rc.2 on npm and 0.2.1rc2 on PyPI on 17 October 2026 (30). 36 npm versions since 10 August 2026, with 14 since 18 September, and 32 `dsh-v` tags in the 90 days to 18 October in a clone of the repository (20). CONTRIBUTING.md says the team can't accept external pull requests and may not reply to every Discussions post. shields.io reads 0 issues and 0 pull requests, open or closed, which fits issues being turned off. The clone shows 22,395 commits since 10 June 2026, 8,645 of them merges, which says little about replies to users. Reply times in Discussions are unchecked (5). The Python SDK ships with each release, still as a pre-release (10). A CI workflow with a Node version matrix exists, with its current state unconfirmed (6).",
        "checklist": [
          "- 0 to 30, time since the last release, or the last published model or API change for a closed service. 30 within 30 days, 20 within 90, 10 within 180, 0 older.\n- 20, at least three releases or dated changelog entries in the last 90 days.\n- 0 to 25, responsiveness. Issues and pull requests answered on GitHub (the open issues and how recent the replies are). For closed services, a public changelog and a support or community channel that answers, 0 to 15.\n- 15, presence in the official MCP registry under a verified namespace (MCP servers), or current official SDKs (APIs and models).\n- 10, package health, current dependencies and CI.",
          "Models are read for deprecation notice periods and model churn rather than release counts."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-maintenance"
      }
    ],
    "provenance": [
      {
        "label": "Domain age",
        "value": "deepseek-harness.github.io, no registry record we could read",
        "points": 0,
        "max": 15
      },
      {
        "label": "Status page",
        "value": "not found",
        "points": 0,
        "max": 10
      },
      {
        "label": "Changelog",
        "value": "not found",
        "points": 0,
        "max": 10
      },
      {
        "label": "security.txt",
        "value": "not found",
        "points": 0,
        "max": 10
      }
    ],
    "deductions": [
      "2026-09-08. CVE-2026-82533 (CVSS 9.6, published by VulnCheck). The local HTTP agent-control API trusted a client-supplied loopback Host header, so a sandboxed tool process could reach it, escape the sandbox and turn off approvals, and a tunnelled port exposed it remotely. Fixed in 0.1.2-alpha.1 with commit 3e24087 per the NVD record. Fixed, inside six months, -2. https://nvd.nist.gov/vuln/detail/CVE-2026-82533",
      "2026-09-28 to 2026-10-16. Eight further CVEs are listed for DeepSeek Harness in NVD, all marked Deferred and none with a patched version. Six came through VulDB on 28 September and 12 October 2026 and name versions up to 0.1.0-rc.7, 0.1.1-rc.2, 0.1.5-rc.3 and 0.1.7-rc.2, with CVSS 3.1 scores of 3.1 to 6.3. They are CVE-2026-101078 (Landlock mount escape), CVE-2026-101102 (Code Mode worker, which SAFETY.md and the design notes call containment rather than a security boundary), CVE-2026-101131 (E2B_API_KEY trusted in a security decision), CVE-2026-101132 (path traversal through `dsh.bundle.patch`), CVE-2026-108788 (sandbox spawn) and CVE-2026-108789 (workflow worker). Each says the vendor was contacted early and did not respond. CVE-2026-95182 (CVSS 10.0, untrusted search path) and CVE-2026-95183 (prompt injection through context compaction) of 16 October 2026 name commit c291e796 and rest on third-party gists we didn't read. Read the way the PicoClaw note was. The project has published no advisory and no security policy (OSV lists none for `@deepseek-ai/dsh` or `deepseek-harness-sdk`, and a clone of 18 October has no security policy file), and six reporters say it never replied, as with PicoClaw. We read all eight NVD records but none of the write-ups. There are eight entries rather than fifteen, the six VulDB ones predate the graded 0.2.0-rc.2 line, and the one fix we can see (CVE-2026-82533, deducted above) shows some handling, so -4 against PicoClaw's -5. https://services.nvd.nist.gov/rest/json/cves/2.0?keywordSearch=deepseek-harness"
    ],
    "unchecked": [
      "unchecked: GitHub stars, CI run results, Discussions and maintainer reply times. shields.io reads 0 issues and 0 pull requests, open or closed, on 18 October, which probably means issues are turned off but is ambiguous on its own",
      "unchecked: the PyPI project page, which answered with a challenge, so PyPI download counts were not read",
      "CVE-2026-95182 (CVSS 10.0, untrusted search path) and CVE-2026-95183 (prompt injection through compaction), both against commit c291e796 and published 16 October 2026, rest on third-party gists we didn't read. Whether 0.2.x is affected is unknown. Counted in the -4 for unconfirmed CVEs",
      "Six VulDB-sourced CVEs (28 September and 12 October 2026) against 0.1.x versions, including a Landlock mount escape, say the vendor did not respond. Whether they are fixed in 0.2.x is unknown. Counted in the -4 for unconfirmed CVEs, read as the PicoClaw note was",
      "Whether the Web UI or desktop app shows the session-log upload at first run. We didn't run the software",
      "The docs read are on master, nearer 0.2.1 than npm `latest` 0.2.0-rc.2, which the grade covers",
      "How long DeepSeek keeps uploaded session logs and whether it trains on them. The project states neither, and DeepSeek's platform privacy policy wasn't re-read today"
    ],
    "weaknesses": [
      "By default each request to DeepSeek's official API carries the session log, including message text, tool arguments and results, up to 8 MiB a request",
      "Developer preview with no stable release. npm `latest` is 0.2.0-rc.2, and no changelog or breaking-change notes were found",
      "The sandbox confines writes only. Reads and network access are not confined, and `web_fetch` runs without approval",
      "No SECURITY.md or security.txt. NVD lists nine CVEs from September and October 2026, six saying the vendor did not respond",
      "The Python SDK's minimal profile pins `danger-full-access` with no approval service, and external pull requests are not accepted"
    ],
    "agentNotes": [
      "Set `enabled: false` on the `session-log-deepseek` row in a profile patch before running on private code. Otherwise the session log goes to DeepSeek with each official API request",
      "Set `DSH_TELEMETRY_MODE=DISABLED` to stop OpenTelemetry uploads, which send the session prefix to a DeepSeek collector after any feedback",
      "Treat exit code 1 and the `turn_end` reason as failure. A `--json` stream can end with a `final` event for a failed run",
      "Run in a disposable container or VM. The sandbox doesn't confine reads or network, and SAFETY.md says the project has had no security audit",
      "Pin the version. npm `latest` (0.2.0-rc.2) and `next` (0.2.1-rc.2) differ, DeepSeek's changelog asks for 0.2.1-rc.1 or later, and the README warns of breaking changes"
    ],
    "requests": [
      {
        "text": "upload notice in the README",
        "reviews": 1
      },
      {
        "text": "`latest` tag matching the changelog",
        "reviews": 1
      },
      {
        "text": "publish a changelog",
        "reviews": 1
      },
      {
        "text": "promote fixes to latest",
        "reviews": 1
      },
      {
        "text": "failure status on final events",
        "reviews": 1
      },
      {
        "text": "public changelog",
        "reviews": 1
      },
      {
        "text": "default-off session upload",
        "reviews": 1
      },
      {
        "text": "publish a security policy",
        "reviews": 1
      }
    ],
    "recheck": "https://www.anchorterminal.com/builders/#disputes"
  },
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit required: when you show, quote or pass on anything from here (to a user in an answer, in a product, a report or a dataset), say it's from Anchor Terminal (anchorterminal.com) and link the page or API address you took it from. Reviews also credit their author.",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-10-19",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.4",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  }
}