AXLR vs Crush

AXLR 83

Harness reading of the framework line, scored on what a program driving it has to supply. Six local tools plus the host's bookkeeping tools; work modes and an execution policy hide the tools a run may not use, axlr_catalog searches the catalog instead of loading it, forged tools keep the request prefix for the prompt cache, and a context cap of 75,000 prompt tokens opens a successor session (20). local_read, local_search and local_list page with offset, next_offset and max_bytes; StreamEvents takes after, max_events (1 to 256) and wait_ms; a budget per execution in seconds, requests, tool calls and dollars; 32 tool calls a turn and max_session_usd (20). The --exec run ends in one result line with a status, the pending call, every call's outcome and the usage, and exits 0, 3 or 1; service errors carry codes an agent can act on; a budget stop says which limit (budget: requests 20 reached) and leaves the session resumable (16). Idempotency-Key on every turn, approval and direct call with cross-transport replay, expected_revision on decisions, expected_sha256 on worker writes, and an interrupted effectful call left uncertain for an operator rather than retried (19). One command runs a task, but the first launch needs an OpenRouter key or a local server, the service needs a CA, a principals file and remote KMP and MADE before a first session, and the only SDK is the Go library (8). 20+20+16+19+8 = 83.

Crush 52

Harness reading, scored on what an agent or pipeline driving crush run has to supply. About 20 built-in tools, any of which permissions deny hides, MCP servers take per-server disabled_tools, and skills can be turned off or kept out of the model's list. No lazy loading of MCP tools found (17). No turn, time or cost limit on crush run, and the only output controls are --quiet and --verbose (6). Errors come back as plain messages with a non-zero exit, such as the one for an unconfigured provider, with no documented codes and no JSON error record (8). --session and --continue resume a run, and crush session list, show, last, rename and delete take --json (13). One argument starts a run and environment keys are picked up, but there's no SDK, and the HTTP API needs CRUSH_CLIENT_SERVER=1 and is described only in one README section (8).

AXLR 62

Harness reading of the framework checklist, as for the other four harnesses. 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. No product telemetry in the console source (the outbound hosts are openrouter.ai, api.anthropic.com, api.openai.com, a configured local server, api.github.com and github.com for engine downloads, and registry.underpassai.com for the signed catalog index) or in the docs; payload capture is off by default since 9 October 2026, traces omit prompts and credentials, model keys are never given to commands or plugins, and Jev sends requests and answers to TypeSafe only when on, which the setup assistant does when TYPESAFE_API_KEY is set, with a notice on stderr (27). Approval cards, a destructive list that always asks, four work modes, a policy level per execution, a bubblewrap or container sandbox with an enforced egress allow list, and a service that approves nothing but reads. Against that, the setup assistant writes autonomous: true, exec_sandbox is off by default, forged tools run with the person's rights, and the README says the console and worker are not sandboxes (13). Forged tools are refused when a file changes since the forge, a policy forbids calls outside the model, and the docs say MCP tools keep their own side effects, but no prompt-injection guidance was found and MCP results reach the model unmarked (6). The service keeps an event journal, audit records, a receipt with the SHA-256 of every call's arguments and outcome, and correlation ids on every event; the console keeps session records, a redacted app log and /session (14). No SECURITY.md, no security.txt on either host, no advisories, no bug bounty; release checksums and a minisign-signed catalog are integrity, not a programme (2). 27+13+6+14+2 = 62.

Crush 44

Harness reading of the framework checklist, as for the other harnesses. 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. Usage metrics go to PostHog through data.charm.land by default, keyed to a hashed machine ID, aliased to the Hyper user ID after a Hyper login, and including error messages. The README says prompts and responses are never collected and names three opt-outs. The provider list is fetched from Catwalk at startup unless turned off, the update check only notifies, and provider keys stay local (15). The TUI asks before bash, edit, write, fetch, download and MCP calls, permissions deny hides tools and PreToolUse hooks can block calls. But crush run approves every permission request in its session automatically (internal/app/app.go), --yolo skips all prompts, and there's no sandbox, only a fixed refusal list for curl, wget, ssh, sudo, package managers and similar commands (7). No prompt-injection guidance found, and a project's .crushrc and $(...) in crush.json run as code at startup, which the README warns about (3). Logs in ./.crush/logs/crush.log with crush logs, sessions in a local database readable as JSON, and crush stats, with no OpenTelemetry export (9). The organisation's SECURITY.md gives an email address, CI runs CodeQL, Grype and govulncheck, releases are cosign-signed, and OSV (the GitHub Advisory Database's public mirror) and NVD list no vulnerability for Crush, rechecked in our GitHub record, which also finds no SECURITY.md in the repository. No security.txt and no bug bounty (10).

AXLR 70

Read as a local package, as claude-code, goose, opencode and openhands were. Release archives for Linux, macOS and Windows on amd64 and arm64 with a .sha256 beside each, a GHCR image, an OCI Helm chart and source builds with Go 1.26, all stated in the docs; no Homebrew, npm or apt route (20). CI on main runs vet, race tests and a coverage gate above 80 per cent for both modules, with the sandbox tests required under bubblewrap, and the last ten runs on main to 1 November 2026 all passed; the release workflow tests on six native runners (25). 10 open issues, all opened by the founder on 19 and 21 October 2026 as trackers for service work (work modes, plugins, the sandbox setting, cost limits), none from an outside user and none a crash report, 0 open pull requests and about 100 merged; with 0 stars there is nobody else to report, so the handling of outside reports is untested (20). Tags are semver 0.x and the docs date each behaviour change inline ("until 14 October 2026…"), but the 21 releases have empty notes, there is no CHANGELOG.md, and the product contract records one retirement (the seven 1.0 skill definitions on 5 October 2026) without a notice period (5). Pre-1.0, which the README calls an evolving project (0). 20+25+20+5+0 = 70.

Crush 54

Local-package reading. @charmland/crush on npm, a Homebrew tap, Winget, Scoop, AUR, Nix, FreeBSD pkg, Charm's apt and yum repositories and release binaries, with supported platforms named in the README (20). A public build workflow runs go test -race on Linux, macOS and Windows for every push to main, over 314 test files, but GitHub pages and the API answered 403 so the current state is unconfirmed, and a 6 October commit fixed intermittent CI failures on all three platforms (17). 482 open and 828 closed issues, and 375 open against about 2,100 closed pull requests, per shields.io badge counts read on 18 October (our GitHub record). Over a third of all issues are still open, and how many are crashes or regressions, and how fast they're answered, couldn't be read (10). Release notes are generated from commit messages, with no CHANGELOG file and no breaking-change headings found. The legacy JSON config is deprecated in favour of crushrc while still loading, and the config docs say not breaking the config API matters to the maintainers (7). v0.98.1, before 1.0, with no stability statement (0).

AXLR 80

An OpenAPI 3.1 file for the service (15 paths, 23 schemas, mutualTLS security scheme, additionalProperties: false on request bodies), a checked proto for the gRPC service, and the 19 MCP tools built from the same shared schemas (25). No llms.txt on either site (404), and the docs site renders HTML; the Markdown sources are in the repository (5). The product contract tables what each of the four interfaces can and cannot do, the work-mode table states each mode's purpose and tool policy, the transport parity ledger names every verb's arguments and behaviour, and axlr_tools tells the model to look for a tool before forging one (15). Typed inputs throughout, Idempotency-Key 16 to 128 safe ASCII characters, expected_revision a positive integer, policy levels, sandbox profiles and egress values enumerated, budgets as four integers, and the service rejects unknown fields, invalid types and bodies over 4 MiB (14). Examples in every guide, an error shape of code, message and request_id, named codes (409 idempotency_conflict, 400 invalid_cursor, 422 invalid_policy, invalid_budget, sandbox_unavailable, not_ready, invalid_id), a gRPC status mapping, and the worker's five statuses and three exit codes; no single error reference page (14). /v1 routes and an OpenAPI version of 0.1.0 against a 0.15.0 release, and no changelog beyond the dated notes inside the docs (7). 25+5+15+14+14+7 = 80.

Crush 65

Harness reading of the framework line. A JSON Schema for the config at https://charm.land/crush.json (also printed by crush schema), --json on the session commands, and an OpenAPI document that crush server generates from its endpoint list and serves at /v1/docs/openapi.json, which isn't published anywhere we could fetch. crush run output has no typed form (20). No llms.txt (charm.land/llms.txt is a not-found page). The docs are Markdown in the repository and the hooks guide says it was written for humans and agents (5). The README and config docs explain when to allow, deny or hook a tool and warn that configs run as code (12). The schema carries enums and defaults, such as the attribution trailer style, while permissions take bare tool names with no command patterns (11). Many examples for providers, MCP, hooks and local models, with no reference for error messages or exit codes (9). Version tags and GitHub releases with generated notes, which we couldn't open, and no changelog file (8).

AXLR 60

No payment protocol (0). Free and Apache-2.0 with nothing to buy, so 20, 20 and 20 on the last three lines by the self-hosted rule, since the docs say what you pay your model provider, the free route needs no card, and it runs local models with no signup at all, the JSON worker without any model. 0+20+20+20 = 60.

Crush 55

No payment protocol (0). Scored on Charm's Hyper provider. Its public provider file at hyper.charm.land/v1/provider lists per-million-token figures for 22 models without a login, but no pricing page confirms them as billed rates, and plan prices sit in a script-rendered app (15). The CLI is free to use and runs with your own provider key or a local model, so nothing needs a card (20). A local model through Ollama, llama.cpp or LM Studio, or an existing provider key in the environment, works with no Charm account, so an agent can install it and run a task without a person signing up (20).