Crush vs OpenAI Agents API

Crush 41

Harness reading of the framework checklist, as for the other harnesses. 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. Usage metrics go to PostHog through data.charm.land by default, keyed to a hashed machine ID, aliased to the Hyper user ID after a Hyper login, and including error messages. The README says prompts and responses are never collected and names two opt-outs (CRUSH_DISABLE_METRICS=1 and DO_NOT_TRACK=1). The provider list is fetched from Catwalk at startup unless turned off, the update check only notifies, and provider keys stay local (15). The TUI asks before bash, edit, write, fetch, download and MCP calls, permissions deny hides tools and PreToolUse hooks can block calls. But crush run approves every permission request in its session automatically (internal/app/app.go in v0.99.0), --yolo skips all prompts, and there's no sandbox, only a refusal list for curl, wget, ssh, sudo, package managers and similar commands (7). No prompt-injection guidance found, and a project's .crushrc and $(...) in crush.json run as code at startup, which the README warns about (3). Logs in ./.crush/logs/crush.log with crush logs, sessions in a local database readable as JSON, and crush stats, with no OpenTelemetry export (9). The organisation's SECURITY.md is a disclosure policy with a reporting address (vt100@charm.sh) and no response time. CI runs CodeQL, Grype and govulncheck nightly and releases are cosign-signed. OSV lists no advisory for the npm package or the Go module. No security.txt and no bug bounty. A disclosure route with no stated time is this run's 7 (was 10). 15 + 7 + 3 + 9 + 7 = 41.

OpenAI Agents API 87

Project keys with restricted per-endpoint permissions, service accounts and workload identity, per the openai-api listing; we didn't confirm a key can be limited to /v1/agents (27 of 30). Automatic review for approvals, a sandbox network restricted to a host allowlist, vaults for secrets and a sandbox security guide (17 of 20). The agent reads web pages, MCP results and browser content; the sandbox security guide and approval flow are the mitigations, with no prompt-injection guide of its own found (8 of 15). Tracing, observability and usage per session and admin audit logs (15). SOC 2 Type 2, ISO 27001 family, Bugcrowd bounty and a disclosure policy, per the openai-api listing (20). Not ZDR-eligible and US residency only, recorded under transparency. 27 + 17 + 8 + 15 + 20 = 87.

Crush 52

Harness reading, scored on what an agent or pipeline driving crush run has to supply. About 20 built-in tools, any of which permissions deny hides, MCP servers take per-server disabled_tools, and skills can be turned off or kept out of the model's list. No lazy loading of MCP tools found (17). No turn, time or cost limit on crush run, and the only output controls are --quiet and --verbose (6). Errors come back as plain messages with a non-zero exit, such as the one for an unconfigured provider, with no documented codes and no JSON error record (8). --session and --continue resume a run, and crush session list, show, last, rename and delete take --json (13). One argument starts a run and environment keys are picked up, but there's no SDK, and the HTTP API needs CRUSH_CLIENT_SERVER=1 and is described only in one README section (8).

OpenAI Agents API 80

Read as an API an agent drives. Context is compacted automatically, and session budgets and streamed events keep output sized (20 of 25). Sessions can be listed, continued and steered, events streamed or sent by webhook, artifacts fetched (15 of 20). Actionable error codes with recovery steps (20). No idempotency key; a documented check-then-retry procedure (10 of 20). One session call with defaults (medium sandbox, network enabled), SDKs in five languages (15). 20 + 15 + 20 + 10 + 15 = 80.

Crush 59

FSL-1.1-MIT, source-available, barring competing commercial use and converting each release to MIT on its second anniversary. Code from before 30 May 2025 is MIT. Not an OSI licence today (20). No privacy policy or terms for Crush were found on charm.land, and Hyper's legal pages render by script and weren't read. The README says Hyper is zero-data-retention, while Hyper's own app says some plans aren't and that usage on them may train Hyper's models, so the statements don't agree (8). The JSON config is marked deprecated with no removal date, and no deprecation policy was found (6). The README discloses usage metrics, links the source, says prompts aren't collected and gives two opt-outs, but doesn't mention the alias to the Hyper account (15).

OpenAI Agents API 85

Closed service with clear terms, Apache-2.0 SDKs (15). Session state retained until deleted, deletable sessions and artifacts, not ZDR-eligible, US residency only, and the data-controls guide agrees; no retention period for artifacts after a sandbox expires (25 of 30). Dated deprecation notices with minimum periods for GA models; a beta API can change faster (15 of 20). Subprocessors and processing countries listed, per the openai-api listing (20). 15 + 25 + 15 + 20 = 75.

Crush 55

No payment protocol (0). Scored on Charm's Hyper provider. Its public provider file at hyper.charm.land/v1/provider lists per-million-token figures for 22 models without a login, but no pricing page confirms them as billed rates, and plan prices sit in a script-rendered app (15). The CLI is free to use and runs with your own provider key or a local model, so nothing needs a card (20). A local model through Ollama, llama.cpp or LM Studio, or an existing provider key in the environment, works with no Charm account, so an agent can install it and run a task without a person signing up (20).

OpenAI Agents API 30

No machine payment protocol (0). Model, tool and container prices published without a login, and no fee for the API itself (20). The Free tier reading of the openai-api listing, half (10). Sign-up in a browser (0). 0 + 20 + 10 + 0 = 30.

Crush 65

Harness reading of the framework line. A JSON Schema for the config at https://charm.land/crush.json (also printed by crush schema), --json on the session commands, and an OpenAPI document that crush server generates from its endpoint list and serves at /v1/docs/openapi.json, which isn't published anywhere we could fetch. crush run output has no typed form (20). No llms.txt (charm.land/llms.txt is a not-found page). The docs are Markdown in the repository and the hooks guide says it was written for humans and agents (5). The README and config docs explain when to allow, deny or hook a tool and warn that configs run as code (12). The schema carries enums and defaults, such as the attribution trailer style, while permissions take bare tool names with no command patterns (11). Many examples for providers, MCP, hooks and local models, with no reference for error messages or exit codes (9). Version tags and GitHub releases with generated notes, which we couldn't open, and no changelog file (8).

OpenAI Agents API 88

The reference is linked from the overview and the SDKs carry beta.agents types; we didn't confirm /agents/sessions in OpenAI's public OpenAPI file, so 15 of 25. llms.txt with every Agents API page and Markdown twins (10). The overview, architecture page and the agents guide that compares the Agents API, Agents SDK and Responses API say when to use it (20). Typed configuration with enumerated container sizes, network modes and environment types (13 of 15). Examples in six languages and an errors page with codes per HTTP status, turn and session (15). Dated changelog entries and a versioned beta header (15). 15 + 10 + 20 + 13 + 15 + 15 = 88.

Crush 52

Local-package reading. @charmland/crush on npm, a Homebrew tap, Winget, Scoop, AUR, Nix, FreeBSD pkg, Charm's apt and yum repositories and release binaries, with supported platforms named in the README (20). The public build workflow runs go test -race -failfast ./... on Linux, macOS and Windows for every push to main and every pull request, but GitHub pages and the API answer 403 to us, so results are unread (15). The 18 October issue and pull request counts came from shields.io badges, a third-party copy of GitHub data, and are struck. The repository's own refs show 2,574 pull requests to #4224, 272 of them open with a merge ref, and 147 merges into main in the 90 days to 4 November. Crash and regression issues couldn't be read (10). Release notes are generated from commit messages, with no CHANGELOG file and no breaking-change headings found. The legacy JSON config is deprecated in favour of crushrc while still loading (7, held partial). v0.99.0, before 1.0, with no stability statement (0). 20 + 15 + 10 + 7 + 0 = 52.

OpenAI Agents API 60

Read as a hosted API. status.openai.com has an Agents component under APIs (20). Since release on 10 September (57 days, not 90), the feed names the Agents API in a degraded-performance incident on 14 September, the elevated errors across ChatGPT, Codex and the API on 29 September (the openai-api listing reads that one as a major, 17:52 to 23:14 UTC), computer use issues and degraded workspace agents on 6 and 7 October, degraded performance across the APIs for 22 minutes on 21 October, and overbilling for hosted containers on 19 September. One major, the rest minor (10). Rate limits come from the shared guide, published per tier and model with headers, and the errors page adds a Files API allowance on session creation (15). 429 and 503 guidance with Retry-After, capped backoff and a check-then-retry procedure for turns (15). The Scale Tier SLA in the openai-api listing isn't shown to cover a beta API (0). Public beta behind the agents=v1 header (0). 20 + 10 + 15 + 15 + 0 + 0 = 60.