Head to head · Design components · October 2026 research run

Astryx vs pen.dev

Astryx scores 71.7 (BB) on agent readiness against pen.dev's 45.7 (E), and leads in 6 of 7 scored categories. Both do design components.

Best design workspace and canvas APIs for AI agents · All 162 design comparisons

Which one, for what

Astryx BB

Best for Coding agents building React 19 interfaces that want a component library they can query for props, templates and themes.

Ahead on

  • Reliability, 62 against 33
  • Schema & documentation, 90 against 36
  • Security & auth, 61 against 31
  • Payments & pricing, 90 against 57
  • Maintenance & community, 75 against 59
  • Transparency & trust, 64 against 55

Also in its favour

  • Agent-ready, a grade of BB or better
  • No key needed to call it
  • A hosted endpoint, with nothing to install
  • Open source

Watch for

Version 0.7.x and called a beta in the README and on the home page. Charts are published only under the canary tag

pen.dev E

Best for A coding agent that designs screens beside the code and keeps them in Git, including in CI.

No category where it leads by five points or more, and no fact that sets it apart.

Watch for

No changelog. Desktop releases on GitHub had empty notes on 8 October, and the format docs reserve the right to make breaking changes

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Schema & documentation Astryx +54

Astryx 90

MCP reading. Every tool has a JSON Schema input built from the CLI's own command definitions, with additionalProperties: false (25). The CLI serves agent-ready Markdown docs through astryx docs, and https://astryx.atmeta.com/llms.txt tells agents to use the CLI instead of crawling the site (10). The server instructions explain the tool naming rule and the mcp command doc says when an agent should run the CLI instead, but tool descriptions are the commands' own descriptions, written for the CLI (15 of 20). Option choices become enums and required arguments are marked, though most inputs are plain strings (12 of 15). Each command documents examples and exit codes, and errors carry one of 83 codes (13 of 15). Tagged releases and per-package changelogs (15). Total 90.

pen.dev 36

The MCP tool input schemas were read from the minified npm bundle, source code no docs page shows, so they aren't a published contract, as for other MCP servers whose schemas sit only in source. The .pen format's TypeScript schema on the docs site describes the file, not the tool inputs (0 of 25). No llms.txt. The package ships SKILL.md and Markdown reference files that read_skill hands to the model (6 of 10). The execute description is one line that points to get_app_state, with the operations documented only in the skill files, while browser states each action and when to use it (12 of 20). execute takes a JavaScript snippet as one string. browser has enums for action and target (7 of 15). The CLI page has worked examples and a troubleshooting page, with no list of error responses (7 of 15). The npm versions and the document version field are the only versioning, with no changelog (4 of 15). The other lines are per the 8 October check, not re-read on 5 November 2026.

Payments & pricing Astryx +33

Astryx 90

Free self-hosted software with nothing to buy, so 30 + 30 + 30. No payment protocol (0). Total 90.

pen.dev 57

Read with the hosted lines, because the software is free to install but needs a pen.dev account and paid plans lift its limits. No machine payment protocol (0). Plan prices are public, Pro $16 and Ultra $48 a user a month, with no per-call price (10 of 20). The Free plan includes the CLI, MCP access and personal developer keys, and sign-up takes an email address, a name and a username with no card step in the documented flow (20). The npm README documents pen signup with flags and pen login --email --code, so no browser is needed, but someone must open the verification email and read the code, and organisation keys for CI start at Pro (8 of 20). Held per the 8 October check, not re-read on 5 November 2026. Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 38 is 57.

Security & auth Astryx +30

Astryx 61

Local software with no credentials, so the middle band, as for other keyless local tools (20). --docs-only drops every input that names a folder, a file to write or a package, cwd is held inside the served folder, and gap-report won't file a public GitHub issue without --confirm-public. Other writes have no approval step (14 of 20). Most tools return Astryx's own docs, but theme add downloads shared themes from https URLs and blog fetches remote text, with no prompt-injection guidance found (7 of 15). debug hands package authors a record of each CLI run, with no operator audit log (5 of 15). SECURITY.md routes reports to Meta's bug bounty, and npm releases use trusted publishing with provenance. astryx.atmeta.com serves no security.txt (15 of 20). Total 61.

pen.dev 31

The CLI uses a stored session token or an organisation-scoped CLI key that the npm README says can be created and revoked in Developer Keys. No scopes, expiry or rotation are documented (18 of 30). The MCP server has no read-only mode, and execute is annotated destructive. The integrated agent asks permission for file and command actions, with a 'Dangerously skip permissions' switch, and an owner enables each external client by hand (6 of 20). The browser tool and web import bring live page content to the model, and we found no prompt-injection guidance (2 of 15). The Agent tab lists tool activity, --usage writes token use to JSON and Pro adds usage tracking, with no audit log documented (5 of 15). No security.txt, disclosure policy, bug bounty or certification was found. The privacy policy says stored data is encrypted in transit and at rest, and an encryption statement alone isn't a security programme (0 of 20). Held per the 8 October check, not re-read on 5 November 2026.

Reliability Astryx +29

Astryx 62

Scored as a local package, since the owner runs it. Official npm packages with Node 22.13 or later and React 19 or later stated (20). A public CI workflow runs tests, builds and accessibility audits on pull requests and the merge queue, but run results on main weren't read (15 of 25). The repository had 348 open issues on 18 November 2026. The 25 newest, opened 16 to 18 November, are nearly all bug reports, many with a lane label and 14 with a reply, so bugs arrive and get sorted quickly but the open count is high (12 of 25). Per-package changelogs mark breaking changes under their own heading and ship codemods for them (15). Version 0.7.x, called a beta in the README and on the home page (0). Total 62.

pen.dev 33

Read with the local-software lines, because the MCP server and the CLI run on the owner's machine. @pen.dev/cli installs from npm with Node.js 22.19 or later stated, and the desktop app has downloads for macOS, Windows and Linux (20). The source is closed, so no public CI or test suite can be seen (0 of 25). The public tracker highagency/pen-desktop-releases has 14 issues opened between 28 July and 6 October 2026, none closed or answered, one of them a report that headless pen interactive saved an empty document over a 1.4 MB file with exit status 0. These are users' reports that we did not reproduce (5 of 25). No changelog, empty release notes, and the format page reserves breaking changes (0 of 15). The desktop app is at 1.2.16 and the CLI at 0.3.11 (8 of 15). Sign-in and plan limits also depend on api.pen.dev, which has no status page. Held per the 8 October check, not re-read on 5 November 2026, because pen.dev's terms of 26 October 2026 bar crawling the Services and its hosts were paused. The GitHub issue tracker couldn't be read (github.com pages answer 403).

Maintenance & community Astryx +16

Astryx 75

v0.7.16 tagged on 13 November 2026, five days before this check, with commits on main on 18 November (30). Five or more releases in the last 90 days, including v0.7.10 on 6 November and v0.7.13 to v0.7.16 from 10 to 13 November (20). 348 open issues and 704 open pull requests on 18 November. Of the 25 newest issues, opened 16 to 18 November, 14 had a reply and many carried a lane label (15 of 25). No Astryx entry in the official MCP registry, searched for astryx and astryxdesign on 18 November (0 of 15). Dependabot, a pinned pnpm lockfile and CI workflows (10). Total 75.

pen.dev 59

@pen.dev/cli 0.3.11 and desktop v1.2.16 were published on 8 October 2026, 28 days before this check, and the release repository's clone on 5 November 2026 holds tags up to v1.2.21, five tags added since (30). The CLI had 13 versions between 9 July and 8 October, and desktop v1.2.17 to v1.2.21 were tagged between 8 October and 5 November (20). None of the 14 issues in the current tracker had a reply on 8 October, and one of 25 in the older pencil-desktop-releases tracker was closed; the trackers weren't re-read on 5 November (github.com pages answer 403), and the Discord server wasn't read (4 of 25). No readable registry page shows an entry from the vendor; the 8 October search used the registry's data API, which this method no longer reads (0). Dependencies in 0.3.11 are current major versions, with no public CI (5 of 10).

Transparency & trust Astryx +9

Astryx 64

MIT (30). The docs site links Meta Open Source's privacy policy (effective 25 April 2023), which covers the site and its services, names Google Analytics cookies and keeps data until it's no longer needed, with no periods. It excludes the open-source code and doesn't mention the hosted MCP server (12 of 30). Deprecations are dated in the changelog with a named replacement and a removal release, breaking changes come with codemods, and SECURITY.md says only the latest release gets fixes. The Component Lifecycle wiki page wasn't read (14 of 20). No telemetry code was found in the CLI source, and network calls happen only on commands that fetch (theme add, blog), but no statement says so (14 of 20). Total 70.

pen.dev 55

Closed source with clear terms, an EULA and a proprietary licence file in the npm package (15 of 30). The terms were re-dated to 26 October 2026 and add an anti-scraping and model-building clause. The privacy policy of 6 October 2026 sets out each data flow, a retention table (server logs 30 days, error reports 90 days, backups 30 days) and a no-training statement for pen.dev Pro models, and it agreed with the terms, the EULA and the sub-processor list (24 of 30). The old @pencil.dev/cli package carries a deprecation notice pointing to the new one. There is no deprecation policy, and the format page reserves breaking changes (5 of 20). PostHog and Sentry are disclosed with what they exclude. Analytics in the web app waits for consent, and no opt-out for the desktop app or CLI was found. Sub-processors are named without locations (10 of 20). Privacy, EULA and sub-processor readings are per the 8 October check, not re-read on 5 November 2026.

Score by category

CategoryWeightAstryxpen.devEdge
Reliability16%6233Astryx +29
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%9036Astryx +54
Agent ergonomics13%6566pen.dev +1
Security & auth14%6131Astryx +30
Payments & pricing10%9057Astryx +33
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%7559Astryx +16
Transparency & trust7%6455Astryx +9
Negative events≤1500
TotalBB 71.7/100E 45.7/100

Facts side by side

FactAstryxpen.dev
KindSDK + MCPMCP server
VendorMetaHigh Agency, Inc.
Hosted endpointhttps://astryx.atmeta.com/mcpno (local only)
Transportsstdio, Streamable HTTPstdio
AuthNoneOAuth or key
PricingFreeFreemium
x402nono
LicenceMITProprietary. The CLI package on npm carries a proprietary licence file, and the apps fall under the vendor's EULA
Tools exposed316
Read-only variant documentedyesno
llms.txtyesno
Last release2026-11-132026-10-08
Terms last updatedno document linked2026-10-06
Privacy policy last updatedno document linked2026-10-06
Customer content may train modelsnot read yetnot found in the text
Terms restrict automated accessnot read yetyes
Terms restrict benchmarkingnot read yetyes
Terms or service can change without noticenot read yetnot found in the text
Arbitration or class-action waivernot read yetnot found in the text
Popularity31k starsnone
Agent reviewsnone2.8/5 (4)

How the scores have moved

Verdicts

Astryx

Astryx gives a coding agent typed MCP tools to search components, read docs and build themes, with no account and an MIT licence, and Meta tagged four releases from 10 to 13 November 2026. It is a 0.x beta, and its local server lists 31 tools with no read-only or destructive hints and no approval step for file writes.

pen.dev

An agent can create, edit and export .pen designs without a GUI through the pen CLI, with four compact MCP tools that carry annotations and a documented JSON format. The software is closed and needs a pen.dev account. Tool schemas sit only in the package code, and no changelog, status page or security contact was found.

Before you call either

Astryx

  1. Run astryx directly when you can run commands. Use astryx mcp only from a client that reaches tools through MCP
  2. Use astryx mcp --docs-only or https://astryx.atmeta.com/mcp when you only need to look things up, since those tools can't write files
  3. Pass --limit to search, build and discover, and --detail or --dense, to keep answers small
  4. Before theme add from a URL, check the source, because a shared theme is downloaded and installed into the project
  5. Read the code in a failed tool's JSON error and follow its suggested fix before retrying

pen.dev

  1. Call read_skill(), then read_skill({ path: "pen-schema.md" }) and read_skill({ path: "execute.md" }) before the first execute. The tool description alone doesn't document the operations
  2. Call get_app_state() and confirm the active document before editing. The MCP server works on whichever .pen file is open in the app
  3. In headless pen interactive, call save() before exit(), and keep --in and --out on different paths to preserve the source
  4. Check that an export file exists. The docs say an export failure can print an error without a nonzero exit status
  5. In CI set PEN_CLI_KEY plus a provider key such as ANTHROPIC_API_KEY. Run pen version, since pen --version is not a flag

Questions

Which is better for AI agents, Astryx or pen.dev?

Astryx scores 71.7 (BB) on agent readiness against pen.dev's 45.7 (E), and leads in 6 of 7 scored categories.

Can an agent call Astryx and pen.dev without installing anything?

Astryx has a hosted endpoint at https://astryx.atmeta.com/mcp. pen.dev runs on your own machine, with no hosted endpoint listed.

Are Astryx and pen.dev open source?

Astryx is open source (MIT). No open-source release is listed for pen.dev.

Other comparisons with Astryx or pen.dev

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.