Cosmic vs Payload

Cosmic 88

openapi.json at www.cosmicjs.com covers 21 operations on 12 paths, each with a summary, a description and documented 401, 402, 404, 429 and 500 responses, though the docs describe more endpoints than it covers (revisions, AI, search); MCP tools have typed inputs (20 of 25). llms.txt and every docs page as Markdown (10). Tool descriptions say when to use each tool and when not to, such as never calling delete speculatively (18 of 20). 11 enums in the spec, typed metafields and validation messages (11 of 15). Examples on every API page and an error model with a sample (14 of 15). API v3 paths and a dated changelog (15). 20+10+18+11+14+15 = 88.

Payload 70

No OpenAPI file is published or generated by the core packages, because endpoints depend on each project's config. Every MCP tool carries a typed input schema built from the collection's fields, and the GraphQL schema is generated from the same config (20 of 25). llms.txt per major version, llms-full.txt and a Markdown copy of every docs page (10). Default MCP tool descriptions are one line, such as Create a document in a collection., unless the owner writes one in the plugin config. The docs tell owners to do so (10 of 20). Field inputs are typed with bounds on limit and depth, but where is JSON passed as a string. select became an object in the MCP find tools in 3.91.0 (10 of 15). REST operations show a request and a response each. No reference page for error responses or status codes was found (8 of 15). Semver, release notes on GitHub and separate docs per major version. REST paths carry no version (12 of 15).

Cosmic 82

24 tools, in the 11 to 30 band (15), plus 5 because omitting the write key gives a read-only subset (20 of 25). limit, props, depth, cursor pagination with after, status and locale filters (20). Status codes with specific messages and fix-or-retry guidance per code (17 of 20). readOnlyHint and destructiveHint on the MCP tools; no idempotency keys for writes, though agent signup is idempotent (14 of 20). Few required parameters, a TypeScript SDK and a CLI, no SDK in a second language found (11 of 15). 20+20+17+14+11 = 82.

Payload 64

Four MCP tools per enabled collection and two per global, so five collections give 20 tools, the 11 to 30 band (15). We added 7 because the owner enables collections and operations one by one, each key narrows the list again, and find tools default to 10 documents at depth 0 with select (22 of 25). Page-based pagination with limit and page, 17 where operators, sort, select and populate (20). REST errors come back in an errors array and MCP tools return a text line beginning Error:. Neither format is documented as a list an agent can match on (10 of 20). No idempotency keys, and no readOnlyHint or destructiveHint in the plugin source. An update by id is repeatable (4 of 20). Few required parameters and sensible defaults. The one official client, @payloadcms/sdk for TypeScript, is labelled beta (8 of 15).

Cosmic 90

Graded on the hosted service with the hosted lines. Statuspage at cosmicstatus.com with API and Web Dashboard components (20). The history page shows no incidents in September, October or November 2026, and the newest incident in the feed is from 26 May 2025. The MCP server isn't a component, so 25 of 30. 100 requests a second with a burst of 200, timeouts and size limits are documented (15). 429 is documented and the error page says which codes to fix and not retry; no Retry-After guidance was found. Agent signup is idempotent (12 of 15). The security page says Enterprise has an SLA from 99.95 per cent uptime; the SLA page itself wasn't read (8 of 10). Hosted MCP is announced as available now, with no beta label (10). 20+25+15+12+8+10 = 90.

Payload 76

Read with the local-software lines, since the graded surface is software its owner hosts. Official npm packages, with Node.js ^22.23.4 || >=24.22.1 in engines and in the installation docs since 3.92.0 (20). A public ci workflow runs unit, type, integration and end-to-end tests on every push to 3.x. GitHub's pages and API couldn't be read on 4 November, so the result is unread (15 of 25). 499 issues were open on 8 October, 140 of them awaiting triage, with 225 opened and 116 closed since 10 July 2026, not re-read (12 of 25). A written semver policy marks breaking commits with ! and lists them in release notes, but lets security fixes and experimental APIs change in minor releases (14 of 15). 3.x is the stable line, with 4.0 in beta (15).

Cosmic 47

Separate read and write keys per bucket, rotatable independently (25), less 10 because the docs give read_key as a query parameter on GET (15 of 30). Read-only MCP access by leaving out the write key, four user roles on every plan, and a delete tool told not to run speculatively; no approval step (14 of 20). Objects hold text written by people and AI generation writes into the bucket; no prompt-injection guidance was found (3 of 15). An append-only audit log of security-relevant actions on Workspace or Pro plans (10 of 15). No SOC 2, ISO 27001 or HIPAA by the security page's own statement, no security.txt, reports to support email (5 of 20). 15+14+3+10+5 = 47.

Payload 57

REST keys are per user, revocable and regenerable, shown once, and limited by that user's access control down to field level. MCP keys carry per-collection, per-operation switches. Both travel only in the Authorization header. Neither expires, and there is no OAuth (24 of 30). A find-only MCP key is a config switch away and the collection's access control still runs under it. Nothing asks a person to approve a delete or a publish in the free edition (14 of 20). Documents hold text written by other people, and no prompt-injection guidance was found. overrideResponse can strip fields before they reach a model (3 of 15). Versions record each change, and onEvent and verboseLogs expose MCP calls to the owner's own logging. No built-in audit log (7 of 15). SECURITY.md gives a reporting address and, since 16 October 2026, says Payload pays no monetary rewards and runs no bug bounty. Advisories are published on GitHub with affected versions, patches and workarounds. No security.txt or SOC 2 statement was found (9 of 20).

Cosmic 86

MCP server v1.7.1 was tagged on 2 November 2026 (30). v1.7.0 on 9 September and v1.7.1, plus changelog entries on 14 and 31 August, 1, 8, 10, 22 and 23 September and 5 and 9 October 2026 (20). A dated changelog, a Discord server and support email; GitHub issue activity wasn't read because the GitHub API refused us for its rate limit (12 of 25). com.cosmicjs/mcp in the official MCP registry under a domain namespace, though the registry has 1.6.0 while the repository is at 1.7.1 (15). CI, deploy and publish workflows in the MCP repository (9 of 10). 30+20+12+15+9 = 86.

Payload 78

3.92.1 is tagged on 27 October 2026, 8 days before this check (30). Nine stable releases from 3.87.1 on 6 August to 3.92.1 on 27 October 2026, with 4.0.0 betas alongside, which don't count (20). 225 issues opened against 116 closed since 10 July, 140 open issues awaiting triage and 763 open pull requests, per the 8 October check (12 of 25). @payloadcms/sdk ships with each release but is beta and TypeScript only, and the official MCP registry has no Payload entry, which fits a plugin that runs inside each install (8 of 15). CI runs on every push. The latest 3.x run had failed on 8 October 2026, and runs couldn't be read on 4 November (8 of 10).