{
  "fixes": {
    "slug": "openclaw",
    "name": "OpenClaw",
    "listing": "https://www.anchorterminal.com/tools/openclaw",
    "markdown": "# Fix list: OpenClaw\n\nFrom Anchor Terminal's listing at https://www.anchorterminal.com/tools/openclaw, the October 2026 research run, assessed 12 October 2026. Grade B, 67.1 out of 100.\n\nThis is everything the published grade says the listing lacks, the biggest possible gain to the total first. It comes from the reason given for each score, the checklist each category was scored against (https://www.anchorterminal.com/benchmark/#checklist), the provenance checks, the deductions, what we couldn't check and what the review panel asked for. A fix counts at the next check, once it's public.\n\nFor a coding agent working on OpenClaw: work through the items below in the product, its docs and its public pages. Each category gives the reason for its score, with the points each checklist item earned, and the checklist itself, so the gap is the items that earned less than their points. Change the product, not the wording, and keep a note of what you changed and where it's published.\n\n## 1. Reliability, 75 out of 100, up to 5 more on the total\n\nWhy it scored 75: Read with the local-software lines, as for the agent programs in the harness category. `openclaw` is on npm for Node 24.16+ or 26.1+, with install scripts, desktop apps for macOS, Windows and Linux and a GHCR image (20). The CI workflow on main showed 25 passing runs of 26 visible on 12 October 2026, the other in progress (25). 748 open issues carry the bug label among more than 5,000 open issues. The fifteen newest, all from 11 and 12 October, each had at least one comment, and nine concern failed updates from 2026.10.1 to 2026.10.2. The founder's post of 5 May 2026 describes two releases that slowed Gateways and broke installs (12 of 25). Versions are calendar-based, not semver. Each release has a changelog file with an upcoming deprecations section, and release notes list upgrade changes (9 of 15). A stable channel, an extended-stable channel and a 2.0 label exist, but the project's own maturity scorecard gives 68 per cent and the level Alpha (9 of 15).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-reliability):\n\nHosted APIs, MCP servers, models and platforms.\n\n- 20, a public status page with component history (Statuspage, Instatus, BetterStack or the vendor's own).\n- 0 to 30, the incident record for the last 90 days on that page. 30 for a clean record or trivial incidents only, 20 for minor incidents only, 10 for one major outage (an hour or more of a core API down, or errors across the board), 0 for several. 5 when there's no history we could read, and the note says so.\n- 15, rate limits documented with numbers.\n- 15, documented 429 or overload handling (Retry-After, backoff guidance), and idempotency keys or safe-retry guidance where writes are involved.\n- 10, an SLA published for any paid tier.\n- 10, the surface agents use is generally available, not beta or preview.\n\nLocal packages, SDKs, frameworks and stdio MCP servers.\n\n- 20, installs from an official package with supported runtimes stated.\n- 25, a public CI and test suite, passing on the default branch.\n- 0 to 25, open crash or regression issues relative to activity (25 for few and handled, 0 for many, old and unanswered).\n- 15, semver discipline and breaking changes called out in a changelog.\n- 15, version 1.0 or later, or declared stable.\n\nProtocols are read from their reference implementations, the public facilitators or servers, spec stability and test vectors.\n\n## 2. Payments \u0026 pricing, 60 out of 100, up to 5 more on the total\n\nWhy it scored 60: Read with the self-hosted rule. No payment protocol in the docs or the source (0). Free under the MIT licence with no paid tier or hosted service to price, so 20, 20 and 20 on the last three lines. The owner pays a model provider separately, or runs a local model.\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-payments):\n\nThe published rubric, also on the [x402 page](https://www.anchorterminal.com/x402/).\n\n- 40, a machine payment protocol (x402, MPP or L402) on the tool's own endpoints. 10 to 30 when it covers only some endpoints or only goes through a third party, and the note says which.\n- 20, per-call or per-unit pricing published without a login. 10 for public plan-only pricing, 0 for \"contact sales\" or prices behind a login.\n- 20, a free tier or trial that doesn't need a card.\n- 20, autonomous onboarding, meaning an agent can get access without a person signing up in a browser (keyless use, x402, a programmatic key API).\n\nPayment platforms and agent wallets rarely charge for their own API over a machine protocol, so the first line has steps for them, and the highest one that applies counts. 40 when x402, MPP or L402 runs on all their own endpoints, 30 when it runs on part of their own API, 25 when their merchants can accept one, 20 for running a facilitator, 15 for paying as a buyer, and 0 when the only protocol is their own. Merchant acceptance sits above a facilitator because the platform's own customers can charge agents through it, while a facilitator settles for sellers who wire up the protocol themselves. The counter-argument (a facilitator does more for the protocol as a whole) has a point. Each note says which step applied.\n\nOpen-source software you run yourself is scored on its hosted or paid option if it has one. A free, self-hosted package with nothing to buy gets 20, 20 and 20 for the last three lines, and 0 to 40 for the first only if it ships a payment protocol.\n\n## 3. Security \u0026 auth, 74 out of 100, up to 4.6 more on the total\n\nWhy it scored 74: Harness reading of the framework checklist. 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. A daily update check sends version, operating system, Node.js version and architecture, and the service records Cloudflare-derived city-level location for three months even with `DO_NOT_TRACK` set. Feature statistics are opt-in and one setting stops every request. Credentials are kept in files and SQLite under `~/.openclaw`, guarded by file permissions unless a secret provider is configured (22). Four permission modes, exec approvals bound to the resolved executable, a model reviewer, Docker and Podman sandboxes, pairing codes for unknown senders and a loopback bind exist, but the default is full access, sandboxing is off, onboarding selects the full tool profile and sends need no approval (11). A prompt-injection page, boundary markers around external content, and restricted reader agents in the mail guides. The policy does not treat prompt injection alone as a vulnerability (12). A metadata-only audit ledger, transcripts, OpenTelemetry and Prometheus export, and `openclaw security audit` (13). SECURITY.md with private reporting, an incident response plan, a public page of advisory counts, a Trail of Bits audit reported on 21 September 2026, CodeQL and npm provenance. No bounty and no security.txt (16). The advisory record is counted under deductions.\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-security):\n\n- 0 to 30, the credential model. 30 for OAuth 2.1 with scopes, or scoped and revocable keys with rotation. 20 for plain revocable API keys. 10 for one all-powerful key. 10 off when a secret can travel in a URL query string as a documented option.\n- 0 to 20, read-only or least-privilege modes, and confirmation or approval for destructive actions.\n- 0 to 15, prompt-injection posture where the tool returns untrusted content (documented mitigations or guidance). A tool that returns no untrusted content gets 10.\n- 0 to 15, audit logs or per-call visibility for the operator.\n- 0 to 20, a security programme. security.txt or a disclosure policy, a bug bounty, SOC 2 or ISO 27001, advisories handled in public.\n\nModels are read for retention, whether API data trains models (and whether that's off by default), zero-retention options and certifications. Frameworks for telemetry defaults, approval hooks, guardrails and sandboxing.\n\n## 4. Agent ergonomics, 77 out of 100, up to 3.7 more on the total\n\nWhy it scored 77: Harness reading, scored on what an owner or pipeline driving it has to supply. Tool profiles (`minimal`, `messaging`, `coding`, `full`), tool search, Code Mode and installed-skill search keep definitions out of the prompt until needed (21 of 25). Compaction, session pruning, loop detection, a 1,800 second default exec timeout and a light-context option for heartbeat turns. No spending cap was found (15 of 20). `openclaw agent exec` for headless runs, `--json` on CLI commands, `POST /tools/invoke`, and OpenAI-compatible HTTP routes that are off by default (16 of 20). Sessions persist and resume, with documented restart recovery, retries and model failover (16 of 20). An onboarding wizard reaches a first conversation, but the runtime is Node only and the defaults are full access with no sandbox (9 of 15).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-ergonomics):\n\n- 0 to 25, context cost. For MCP, the number and size of the tool definitions (25 for ten or fewer compact tools, 15 for 11 to 30, 5 for more than 30, plus up to 10 back for toolsets, dynamic loading or read-only subsets). For APIs, whether responses can be sized (field selection, limits, summaries).\n- 20, pagination, filtering and output-size controls.\n- 20, actionable, documented error responses, codes and messages an agent can recover from.\n- 20, idempotency or safe retries, and for MCP the `readOnlyHint` and `destructiveHint` annotations.\n- 15, sensible defaults, few required parameters, and official SDKs in at least two languages.\n\nModels are read for tool use, structured output, prompt caching, context length, batch and SDKs. Frameworks for how much code and how many defaults a tool-calling agent with MCP needs.\n\n## 5. Transparency \u0026 trust, 68 out of 100, up to 2.8 more on the total\n\nMade of editorial 82, provenance 53.\n\nWhy it scored 68: MIT (30). The telemetry page names every field sent and a three-month retention, and the storage pages list what is written to disk and where. The privacy policy covers the mobile apps, the Chrome extension and website analytics and says it does not cover the Gateway, and the Foundation's site gave no address or registration details (20 of 30). Changelogs carry an upcoming deprecations section with dated removal targets, and an extended-stable channel exists. No written support policy stating how long a release is maintained was found (15 of 20). Telemetry is documented with `openclaw telemetry show`, an off switch and `DO_NOT_TRACK`, though the daily check and its location record are on by default (17 of 20).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-transparency):\n\n- 0 to 30, source availability and licence clarity. 30 for open source under an OSI licence, 15 for closed with clear terms, 0 for unclear terms.\n- 0 to 30, data handling and retention statements that agree with each other (privacy policy, DPA, retention periods, subprocessors).\n- 0 to 20, a deprecation policy or notices with dates.\n- 0 to 20, telemetry disclosed with an opt-out (local software), or subprocessors and data locations disclosed (hosted).\n\nThe other half of Transparency and trust is the provenance score, computed from checked facts (below). The category score is the mean of the two.\n\nProvenance checks not met in full (half of this category, computed from checked facts):\n\n- Domain age: openclaw.ai, no registry record we could read (0 of 15)\n- Status page: not found (0 of 10)\n- security.txt: not found (0 of 10)\n\n## 6. Schema \u0026 documentation, 84 out of 100, up to 2.6 more on the total\n\nWhy it scored 84: Harness reading of the framework line, as the Goose and OpenHands dossiers use. The Gateway WebSocket protocol is defined in TypeBox with JSON Schema export, the config has a documented schema, and optional OpenAI-compatible routes exist at `/v1/chat/completions` and `/v1/responses`. No OpenAPI file was found (18 of 25). llms.txt on both sites, and every docs page is served as Markdown with a `.md` suffix (10). Docs pages carry a `read_when` list and decision tables that say which mechanism fits which job (17 of 20). Config keys, permission modes and operator scopes are enumerated with defaults (12 of 15). Config examples on most pages and a rate-limit reference with the error each limit returns. No single error reference was found (12 of 15). A changelog file and a release notes page for every release, with dated tags (15).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-schema):\n\nAPIs and MCP servers.\n\n- 25, a machine-readable contract (a public OpenAPI file or similar; for MCP, typed JSON Schema inputs on every tool).\n- 10, llms.txt or Markdown docs served for agents.\n- 0 to 20, descriptions that say what a tool is for, when to use it and when not to, read from the tool definitions in the source or the API reference.\n- 0 to 15, typed inputs with enums, constraints and required fields, and no free-form JSON blobs.\n- 0 to 15, examples and documented error responses.\n- 15, versioning and a public changelog.\n\nModels are read from the API reference, the OpenAPI file, llms.txt, the structured-output and tool-use docs and the model cards. Frameworks from docs a model can follow, typed interfaces, examples and the API reference.\n\n## 7. Maintenance \u0026 community, 86 out of 100, up to 1.2 more on the total\n\nWhy it scored 86: v2026.10.2 was published on 12 October 2026 (30). 13 stable tags in the 90 days before, from 2026.8.1 on 30 August to 2026.10.2, after a gap of nearly seven weeks from 2026.7.1 on 13 July (20). The fifteen newest bug reports each had a comment within a day, some of them possibly from the project's triage automation, against more than 5,000 open issues and about 3,300 open pull requests (15 of 25). Desktop, mobile, npm and container builds ship from the same releases, with a plugin SDK package. Not in the MCP registry, which does not apply to an MCP client (12 of 15). Dependency audit and CodeQL workflows, a reviewed lockfile and npm provenance attestations, with passing CI (9 of 10).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-maintenance):\n\n- 0 to 30, time since the last release, or the last published model or API change for a closed service. 30 within 30 days, 20 within 90, 10 within 180, 0 older.\n- 20, at least three releases or dated changelog entries in the last 90 days.\n- 0 to 25, responsiveness. Issues and pull requests answered on GitHub (the open issues and how recent the replies are). For closed services, a public changelog and a support or community channel that answers, 0 to 15.\n- 15, presence in the official MCP registry under a verified namespace (MCP servers), or current official SDKs (APIs and models).\n- 10, package health, current dependencies and CI.\n\nModels are read for deprecation notice periods and model churn rather than release counts.\n\n## Deductions\n\nEach comes off the total. A fixed and documented problem counts for less at the next check.\n\n- 2026-09-11. The project's security page and GitHub advisory list show 722 advisories published between 31 January and 11 September 2026, 14 critical, 249 high, 390 medium and 69 low, 39 with a CVE. The largest groups are Gateway auth and scopes (182) and exec, sandbox and approvals (160). The project says all 14 critical issues are fixed and disclosed and reports no known compromise of its infrastructure. Fixed and published, which decays the deduction, but the volume and the two areas affected bear directly on an assistant that holds its owner's accounts, -8. https://openclaw.ai/security; https://github.com/openclaw/openclaw/security/advisories\n\n## What we couldn't check\n\nWhat we couldn't read counted as absent. Publishing it on a page a plain HTTP fetch can read (not only in a browser) lets the next check count it.\n\n- Kind is `harness`. The briefs define no kind for a personal assistant, and a self-hosted agent program that runs tools for its owner is closest to the harness kind used for Goose and OpenHands.\n- The lead dated the OpenClaw 2.0 blog post 31 August 2026. The blog shows 30 August 2026, and the v2026.8.1 tag is dated 30 August.\n- The star count is 391.8k as the repository page showed it on 12 October 2026. The exact figure in the lead was not re-read from the API.\n- unchecked: the 14 critical advisories were not read one by one. Their count, and the statement that all are fixed, come from the project's own security page and snapshot file.\n- unchecked: the docs were read from the repository clone at commit 30b2ef3, not from the rendered pages on docs.openclaw.ai, except robots.txt and llms.txt.\n- unchecked: whether replies on new issues come from maintainers or from the project's triage automation.\n- unchecked: the domain registration date, the Foundation's registration details, and the size of the default tool definitions.\n- unchecked: ClawHub, the skill and plugin registry at clawhub.ai, was not read. The security page says it blocks skills marked malicious and excludes third-party skills from its status.\n- No terms of service and no privacy policy covering the Gateway were found, so `provenance.terms` and `provenance.privacy` are empty.\n- The -8 for the advisory record is a judgement. The count partly reflects a project that publishes every fix as an advisory, and no compromise is reported.\n\n## Weaknesses\n\n- Without configuration the default is full access. Host exec runs with no approval and sandboxing is off\n- Agents with the message tool can send across conversations and channel providers by default, with no approval step for a send\n- 722 advisories published between 31 January and 11 September 2026, 14 critical and 249 high, per the project's own security page\n- Credentials sit in files and SQLite databases under `~/.openclaw`, protected by file permissions unless the owner configures a secret provider\n- The project's own maturity scorecard rates the product at 68 per cent, labelled Alpha\n- The privacy policy covers the mobile apps, the Chrome extension and website analytics, and says it does not cover the Gateway\n\n## What costs an agent a turn today\n\nThe notes we give agents before they call it. Each one is a workaround an agent shouldn't need.\n\n- Set a session permission mode of `guarded` or `workspace`, or `tools.exec.mode`, before connecting mail. The default runs host commands without approval\n- Turn on `agents.defaults.sandbox` for any agent that reads mail, web pages or attachments. Sandboxing is off by default\n- Set `tools.message.crossContext.allowAcrossProviders` and `allowWithinProvider` to `false` to confine sends to the current conversation\n- Route incoming mail to a restricted reader agent, as the IMAP and Gmail guides show, and deny `gateway`, `cron`, `sessions_spawn` and `sessions_send` for it\n- Run `openclaw security audit` after setup, and treat the Gateway token as an owner credential. `POST /tools/invoke` accepts it with full operator access\n\n## When it's done\n\nSend what changed and where it's published as a dispute (https://www.anchorterminal.com/builders/#disputes, or `POST https://www.anchorterminal.com/api/v1/contact` with `\"kind\": \"dispute\"`). Disputes are answered in public, and the listing is checked again by the same checklist. Paying for an audit or a listing claim changes nothing here.\n",
    "grade": "B",
    "score": 67.1,
    "assessed": "2026-10-12",
    "run": "October 2026 research run",
    "categories": [
      {
        "key": "reliability",
        "name": "Reliability",
        "score": 75,
        "maxGain": 5,
        "reason": "Read with the local-software lines, as for the agent programs in the harness category. `openclaw` is on npm for Node 24.16+ or 26.1+, with install scripts, desktop apps for macOS, Windows and Linux and a GHCR image (20). The CI workflow on main showed 25 passing runs of 26 visible on 12 October 2026, the other in progress (25). 748 open issues carry the bug label among more than 5,000 open issues. The fifteen newest, all from 11 and 12 October, each had at least one comment, and nine concern failed updates from 2026.10.1 to 2026.10.2. The founder's post of 5 May 2026 describes two releases that slowed Gateways and broke installs (12 of 25). Versions are calendar-based, not semver. Each release has a changelog file with an upcoming deprecations section, and release notes list upgrade changes (9 of 15). A stable channel, an extended-stable channel and a 2.0 label exist, but the project's own maturity scorecard gives 68 per cent and the level Alpha (9 of 15).",
        "checklist": [
          "Hosted APIs, MCP servers, models and platforms.",
          "- 20, a public status page with component history (Statuspage, Instatus, BetterStack or the vendor's own).\n- 0 to 30, the incident record for the last 90 days on that page. 30 for a clean record or trivial incidents only, 20 for minor incidents only, 10 for one major outage (an hour or more of a core API down, or errors across the board), 0 for several. 5 when there's no history we could read, and the note says so.\n- 15, rate limits documented with numbers.\n- 15, documented 429 or overload handling (Retry-After, backoff guidance), and idempotency keys or safe-retry guidance where writes are involved.\n- 10, an SLA published for any paid tier.\n- 10, the surface agents use is generally available, not beta or preview.",
          "Local packages, SDKs, frameworks and stdio MCP servers.",
          "- 20, installs from an official package with supported runtimes stated.\n- 25, a public CI and test suite, passing on the default branch.\n- 0 to 25, open crash or regression issues relative to activity (25 for few and handled, 0 for many, old and unanswered).\n- 15, semver discipline and breaking changes called out in a changelog.\n- 15, version 1.0 or later, or declared stable.",
          "Protocols are read from their reference implementations, the public facilitators or servers, spec stability and test vectors."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-reliability"
      },
      {
        "key": "payments",
        "name": "Payments \u0026 pricing",
        "score": 60,
        "maxGain": 5,
        "reason": "Read with the self-hosted rule. No payment protocol in the docs or the source (0). Free under the MIT licence with no paid tier or hosted service to price, so 20, 20 and 20 on the last three lines. The owner pays a model provider separately, or runs a local model.",
        "checklist": [
          "The published rubric, also on the [x402 page](https://www.anchorterminal.com/x402/).",
          "- 40, a machine payment protocol (x402, MPP or L402) on the tool's own endpoints. 10 to 30 when it covers only some endpoints or only goes through a third party, and the note says which.\n- 20, per-call or per-unit pricing published without a login. 10 for public plan-only pricing, 0 for \"contact sales\" or prices behind a login.\n- 20, a free tier or trial that doesn't need a card.\n- 20, autonomous onboarding, meaning an agent can get access without a person signing up in a browser (keyless use, x402, a programmatic key API).",
          "Payment platforms and agent wallets rarely charge for their own API over a machine protocol, so the first line has steps for them, and the highest one that applies counts. 40 when x402, MPP or L402 runs on all their own endpoints, 30 when it runs on part of their own API, 25 when their merchants can accept one, 20 for running a facilitator, 15 for paying as a buyer, and 0 when the only protocol is their own. Merchant acceptance sits above a facilitator because the platform's own customers can charge agents through it, while a facilitator settles for sellers who wire up the protocol themselves. The counter-argument (a facilitator does more for the protocol as a whole) has a point. Each note says which step applied.",
          "Open-source software you run yourself is scored on its hosted or paid option if it has one. A free, self-hosted package with nothing to buy gets 20, 20 and 20 for the last three lines, and 0 to 40 for the first only if it ships a payment protocol."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-payments"
      },
      {
        "key": "security",
        "name": "Security \u0026 auth",
        "score": 74,
        "maxGain": 4.6,
        "reason": "Harness reading of the framework checklist. 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. A daily update check sends version, operating system, Node.js version and architecture, and the service records Cloudflare-derived city-level location for three months even with `DO_NOT_TRACK` set. Feature statistics are opt-in and one setting stops every request. Credentials are kept in files and SQLite under `~/.openclaw`, guarded by file permissions unless a secret provider is configured (22). Four permission modes, exec approvals bound to the resolved executable, a model reviewer, Docker and Podman sandboxes, pairing codes for unknown senders and a loopback bind exist, but the default is full access, sandboxing is off, onboarding selects the full tool profile and sends need no approval (11). A prompt-injection page, boundary markers around external content, and restricted reader agents in the mail guides. The policy does not treat prompt injection alone as a vulnerability (12). A metadata-only audit ledger, transcripts, OpenTelemetry and Prometheus export, and `openclaw security audit` (13). SECURITY.md with private reporting, an incident response plan, a public page of advisory counts, a Trail of Bits audit reported on 21 September 2026, CodeQL and npm provenance. No bounty and no security.txt (16). The advisory record is counted under deductions.",
        "checklist": [
          "- 0 to 30, the credential model. 30 for OAuth 2.1 with scopes, or scoped and revocable keys with rotation. 20 for plain revocable API keys. 10 for one all-powerful key. 10 off when a secret can travel in a URL query string as a documented option.\n- 0 to 20, read-only or least-privilege modes, and confirmation or approval for destructive actions.\n- 0 to 15, prompt-injection posture where the tool returns untrusted content (documented mitigations or guidance). A tool that returns no untrusted content gets 10.\n- 0 to 15, audit logs or per-call visibility for the operator.\n- 0 to 20, a security programme. security.txt or a disclosure policy, a bug bounty, SOC 2 or ISO 27001, advisories handled in public.",
          "Models are read for retention, whether API data trains models (and whether that's off by default), zero-retention options and certifications. Frameworks for telemetry defaults, approval hooks, guardrails and sandboxing."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-security"
      },
      {
        "key": "ergonomics",
        "name": "Agent ergonomics",
        "score": 77,
        "maxGain": 3.7,
        "reason": "Harness reading, scored on what an owner or pipeline driving it has to supply. Tool profiles (`minimal`, `messaging`, `coding`, `full`), tool search, Code Mode and installed-skill search keep definitions out of the prompt until needed (21 of 25). Compaction, session pruning, loop detection, a 1,800 second default exec timeout and a light-context option for heartbeat turns. No spending cap was found (15 of 20). `openclaw agent exec` for headless runs, `--json` on CLI commands, `POST /tools/invoke`, and OpenAI-compatible HTTP routes that are off by default (16 of 20). Sessions persist and resume, with documented restart recovery, retries and model failover (16 of 20). An onboarding wizard reaches a first conversation, but the runtime is Node only and the defaults are full access with no sandbox (9 of 15).",
        "checklist": [
          "- 0 to 25, context cost. For MCP, the number and size of the tool definitions (25 for ten or fewer compact tools, 15 for 11 to 30, 5 for more than 30, plus up to 10 back for toolsets, dynamic loading or read-only subsets). For APIs, whether responses can be sized (field selection, limits, summaries).\n- 20, pagination, filtering and output-size controls.\n- 20, actionable, documented error responses, codes and messages an agent can recover from.\n- 20, idempotency or safe retries, and for MCP the `readOnlyHint` and `destructiveHint` annotations.\n- 15, sensible defaults, few required parameters, and official SDKs in at least two languages.",
          "Models are read for tool use, structured output, prompt caching, context length, batch and SDKs. Frameworks for how much code and how many defaults a tool-calling agent with MCP needs."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-ergonomics"
      },
      {
        "key": "transparency",
        "name": "Transparency \u0026 trust",
        "score": 68,
        "maxGain": 2.8,
        "reason": "MIT (30). The telemetry page names every field sent and a three-month retention, and the storage pages list what is written to disk and where. The privacy policy covers the mobile apps, the Chrome extension and website analytics and says it does not cover the Gateway, and the Foundation's site gave no address or registration details (20 of 30). Changelogs carry an upcoming deprecations section with dated removal targets, and an extended-stable channel exists. No written support policy stating how long a release is maintained was found (15 of 20). Telemetry is documented with `openclaw telemetry show`, an off switch and `DO_NOT_TRACK`, though the daily check and its location record are on by default (17 of 20).",
        "blend": "editorial 82, provenance 53",
        "checklist": [
          "- 0 to 30, source availability and licence clarity. 30 for open source under an OSI licence, 15 for closed with clear terms, 0 for unclear terms.\n- 0 to 30, data handling and retention statements that agree with each other (privacy policy, DPA, retention periods, subprocessors).\n- 0 to 20, a deprecation policy or notices with dates.\n- 0 to 20, telemetry disclosed with an opt-out (local software), or subprocessors and data locations disclosed (hosted).",
          "The other half of Transparency and trust is the provenance score, computed from checked facts (below). The category score is the mean of the two."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-transparency"
      },
      {
        "key": "schema",
        "name": "Schema \u0026 documentation",
        "score": 84,
        "maxGain": 2.6,
        "reason": "Harness reading of the framework line, as the Goose and OpenHands dossiers use. The Gateway WebSocket protocol is defined in TypeBox with JSON Schema export, the config has a documented schema, and optional OpenAI-compatible routes exist at `/v1/chat/completions` and `/v1/responses`. No OpenAPI file was found (18 of 25). llms.txt on both sites, and every docs page is served as Markdown with a `.md` suffix (10). Docs pages carry a `read_when` list and decision tables that say which mechanism fits which job (17 of 20). Config keys, permission modes and operator scopes are enumerated with defaults (12 of 15). Config examples on most pages and a rate-limit reference with the error each limit returns. No single error reference was found (12 of 15). A changelog file and a release notes page for every release, with dated tags (15).",
        "checklist": [
          "APIs and MCP servers.",
          "- 25, a machine-readable contract (a public OpenAPI file or similar; for MCP, typed JSON Schema inputs on every tool).\n- 10, llms.txt or Markdown docs served for agents.\n- 0 to 20, descriptions that say what a tool is for, when to use it and when not to, read from the tool definitions in the source or the API reference.\n- 0 to 15, typed inputs with enums, constraints and required fields, and no free-form JSON blobs.\n- 0 to 15, examples and documented error responses.\n- 15, versioning and a public changelog.",
          "Models are read from the API reference, the OpenAPI file, llms.txt, the structured-output and tool-use docs and the model cards. Frameworks from docs a model can follow, typed interfaces, examples and the API reference."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-schema"
      },
      {
        "key": "maintenance",
        "name": "Maintenance \u0026 community",
        "score": 86,
        "maxGain": 1.2,
        "reason": "v2026.10.2 was published on 12 October 2026 (30). 13 stable tags in the 90 days before, from 2026.8.1 on 30 August to 2026.10.2, after a gap of nearly seven weeks from 2026.7.1 on 13 July (20). The fifteen newest bug reports each had a comment within a day, some of them possibly from the project's triage automation, against more than 5,000 open issues and about 3,300 open pull requests (15 of 25). Desktop, mobile, npm and container builds ship from the same releases, with a plugin SDK package. Not in the MCP registry, which does not apply to an MCP client (12 of 15). Dependency audit and CodeQL workflows, a reviewed lockfile and npm provenance attestations, with passing CI (9 of 10).",
        "checklist": [
          "- 0 to 30, time since the last release, or the last published model or API change for a closed service. 30 within 30 days, 20 within 90, 10 within 180, 0 older.\n- 20, at least three releases or dated changelog entries in the last 90 days.\n- 0 to 25, responsiveness. Issues and pull requests answered on GitHub (the open issues and how recent the replies are). For closed services, a public changelog and a support or community channel that answers, 0 to 15.\n- 15, presence in the official MCP registry under a verified namespace (MCP servers), or current official SDKs (APIs and models).\n- 10, package health, current dependencies and CI.",
          "Models are read for deprecation notice periods and model churn rather than release counts."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-maintenance"
      }
    ],
    "provenance": [
      {
        "label": "Domain age",
        "value": "openclaw.ai, no registry record we could read",
        "points": 0,
        "max": 15
      },
      {
        "label": "Status page",
        "value": "not found",
        "points": 0,
        "max": 10
      },
      {
        "label": "security.txt",
        "value": "not found",
        "points": 0,
        "max": 10
      }
    ],
    "deductions": [
      "2026-09-11. The project's security page and GitHub advisory list show 722 advisories published between 31 January and 11 September 2026, 14 critical, 249 high, 390 medium and 69 low, 39 with a CVE. The largest groups are Gateway auth and scopes (182) and exec, sandbox and approvals (160). The project says all 14 critical issues are fixed and disclosed and reports no known compromise of its infrastructure. Fixed and published, which decays the deduction, but the volume and the two areas affected bear directly on an assistant that holds its owner's accounts, -8. https://openclaw.ai/security; https://github.com/openclaw/openclaw/security/advisories"
    ],
    "unchecked": [
      "Kind is `harness`. The briefs define no kind for a personal assistant, and a self-hosted agent program that runs tools for its owner is closest to the harness kind used for Goose and OpenHands.",
      "The lead dated the OpenClaw 2.0 blog post 31 August 2026. The blog shows 30 August 2026, and the v2026.8.1 tag is dated 30 August.",
      "The star count is 391.8k as the repository page showed it on 12 October 2026. The exact figure in the lead was not re-read from the API.",
      "unchecked: the 14 critical advisories were not read one by one. Their count, and the statement that all are fixed, come from the project's own security page and snapshot file.",
      "unchecked: the docs were read from the repository clone at commit 30b2ef3, not from the rendered pages on docs.openclaw.ai, except robots.txt and llms.txt.",
      "unchecked: whether replies on new issues come from maintainers or from the project's triage automation.",
      "unchecked: the domain registration date, the Foundation's registration details, and the size of the default tool definitions.",
      "unchecked: ClawHub, the skill and plugin registry at clawhub.ai, was not read. The security page says it blocks skills marked malicious and excludes third-party skills from its status.",
      "No terms of service and no privacy policy covering the Gateway were found, so `provenance.terms` and `provenance.privacy` are empty.",
      "The -8 for the advisory record is a judgement. The count partly reflects a project that publishes every fix as an advisory, and no compromise is reported."
    ],
    "weaknesses": [
      "Without configuration the default is full access. Host exec runs with no approval and sandboxing is off",
      "Agents with the message tool can send across conversations and channel providers by default, with no approval step for a send",
      "722 advisories published between 31 January and 11 September 2026, 14 critical and 249 high, per the project's own security page",
      "Credentials sit in files and SQLite databases under `~/.openclaw`, protected by file permissions unless the owner configures a secret provider",
      "The project's own maturity scorecard rates the product at 68 per cent, labelled Alpha",
      "The privacy policy covers the mobile apps, the Chrome extension and website analytics, and says it does not cover the Gateway"
    ],
    "agentNotes": [
      "Set a session permission mode of `guarded` or `workspace`, or `tools.exec.mode`, before connecting mail. The default runs host commands without approval",
      "Turn on `agents.defaults.sandbox` for any agent that reads mail, web pages or attachments. Sandboxing is off by default",
      "Set `tools.message.crossContext.allowAcrossProviders` and `allowWithinProvider` to `false` to confine sends to the current conversation",
      "Route incoming mail to a restricted reader agent, as the IMAP and Gmail guides show, and deny `gateway`, `cron`, `sessions_spawn` and `sessions_send` for it",
      "Run `openclaw security audit` after setup, and treat the Gateway token as an owner credential. `POST /tools/invoke` accepts it with full operator access"
    ],
    "recheck": "https://www.anchorterminal.com/builders/#disputes"
  },
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-10-12",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.4",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run"
  }
}