AXLR vs Devin

AXLR 70

Read as a local package, as claude-code, goose, opencode and openhands were. Release archives for Linux, macOS and Windows on amd64 and arm64 with a .sha256 beside each, a GHCR image, an OCI Helm chart and source builds with Go 1.26, all stated in the docs; no Homebrew, npm or apt route (20). CI on main runs vet, race tests and a coverage gate above 80 per cent for both modules, with the sandbox tests required under bubblewrap, and the last ten runs on main to 1 November 2026 all passed; the release workflow tests on six native runners (25). 10 open issues, all opened by the founder on 19 and 21 October 2026 as trackers for service work (work modes, plugins, the sandbox setting, cost limits), none from an outside user and none a crash report, 0 open pull requests and about 100 merged; with 0 stars there is nobody else to report, so the handling of outside reports is untested (20). Tags are semver 0.x and the docs date each behaviour change inline ("until 14 October 2026…"), but the 21 releases have empty notes, there is no CHANGELOG.md, and the product contract records one retirement (the seven 1.0 skill definitions on 5 October 2026) without a notice period (5). Pre-1.0, which the README calls an evolving project (0). 20+25+20+5+0 = 70.

Devin 30

Hosted reading. Status page at www.devinstatus.com with 14 components and incident history (20). In the 90 days to 8 October 2026 it lists three critical incidents (22 July, 43 minutes of platform-wide queuing, 13 August, Devin Cloud unavailable for 2 hours 40 minutes, 24 September, the web app down for 2 hours) and six major ones on session start-up, so several majors (0). 429 is a documented response, but no rate limit figures were found (0). No Retry-After or backoff guidance found, and v3 session creation has no idempotency key, although the legacy v1 API had one (0). The platform terms supply the service as is with no service level, and enterprise terms weren't read (0). The v3 API is the current, recommended surface, with 44 of 239 operations marked beta under /v3beta1 (10).

AXLR 60

No payment protocol (0). Free and Apache-2.0 with nothing to buy, so 20, 20 and 20 on the last three lines by the self-hosted rule, since the docs say what you pay your model provider, the free route needs no card, and it runs local models with no signup at all, the JSON worker without any model. 0+20+20+20 = 60.

Devin 20

Published rubric. No payment protocol (0). Plan prices are public, $20, $200 and $40 a seat with an $80 team minimum, but usage past the quota is billed at an unpublished rate (10). A Free plan exists. The pricing page lists cloud agents from Pro upward, and we couldn't confirm API use on Free or whether a card is needed (10). A person signs up in a browser and provisions the service user in settings (0).

AXLR 83

Harness reading of the framework line, scored on what a program driving it has to supply. Six local tools plus the host's bookkeeping tools; work modes and an execution policy hide the tools a run may not use, axlr_catalog searches the catalog instead of loading it, forged tools keep the request prefix for the prompt cache, and a context cap of 75,000 prompt tokens opens a successor session (20). local_read, local_search and local_list page with offset, next_offset and max_bytes; StreamEvents takes after, max_events (1 to 256) and wait_ms; a budget per execution in seconds, requests, tool calls and dollars; 32 tool calls a turn and max_session_usd (20). The --exec run ends in one result line with a status, the pending call, every call's outcome and the usage, and exits 0, 3 or 1; service errors carry codes an agent can act on; a budget stop says which limit (budget: requests 20 reached) and leaves the session resumable (16). Idempotency-Key on every turn, approval and direct call with cross-transport replay, expected_revision on decisions, expected_sha256 on worker writes, and an interrupted effectful call left uncertain for an operator rather than retried (19). One command runs a task, but the first launch needs an OpenRouter key or a local server, the service needs a CA, a principals file and remote KMP and MADE before a first session, and the only SDK is the Go library (8). 20+20+16+19+8 = 83.

Devin 62

API and MCP reading. The MCP server documents 13 tools, several of them consolidated manage tools, and list endpoints take first to size a page (17). Cursor pagination on every v3 list endpoint and filters on session search by tag, playbook, origin, user and time (18). problem+json with error_code, field-level errors on 422 and a 400 that lists valid platform names, but no published code list (15). No idempotency key on v3 session creation. max_acu_limit caps what a session can spend, and the MCP tool annotations couldn't be read without a key (4). Only prompt is required to start a session. No official SDK found, only a Terraform provider (8).

AXLR 79

v0.15.0 on 2026-10-25, with commits on main to 2026-11-01 (30). 21 releases between 4 and 25 October 2026 (20). One maintainer (368 of 399 commits, the rest by two agent accounts), 10 open issues that are his own trackers with no comments, about 100 merged pull requests and no outside report to measure a reply against (15). Not in the MCP registry; the Go module and a proto to generate clients from are the only client code, beside the GHCR image and Helm chart (5). Go 1.26 toolchain pinned, release actions pinned by commit SHA, a go-licenses check that the third-party bundle matches the dependency graph, and CI passing on main (9). 30+20+15+5+9 = 79.

Devin 63

Closed service. Newest release note dated 7 October 2026 (30). Nineteen dated release notes between 7 August and 7 October 2026, and API notes for August and September (20). Public release notes and a support address, with no public tracker or forum found (10). No official SDK found and the Devin MCP server didn't appear in a search of the official MCP registry (0). No public CI or packages to assess, with the docs and three OpenAPI specs kept current (3).

AXLR 55

Apache-2.0 with a NOTICE (30). No privacy policy or terms on underpassai.com or axlr.underpassai.com; the docs' 'Where data goes' and 'Diagnostics and privacy' sections say that model requests go to OpenRouter, the local server or the native API, that Jev sends text to TypeSafe only when on, that sessions live under $XDG_STATE_HOME/axlr, that payload capture is off by default and redacted when on, and that traces are deleted after 30 days; nothing covers the engine downloads' or the catalog registry's handling of a client's requests (15). No deprecation policy; pre-1.0 changes are dated inline in the docs and the product contract records the 5 October 2026 retirement of the first skill catalogue after the fact (6). No telemetry to disclose, and the docs say in two places what leaves the machine and how to stop payload capture, though no single page says there is no telemetry (17). 30+15+6+17 = 68.

Devin 70

Editorial half. Closed source with clear terms from Cognition AI, Inc. (15). Terms, privacy policy, security page and data processing agreement agree with each other. Customer data may be used for model training by default on self-serve plans, with an opt-out on paid plans that also turns on zero data retention at model providers. Retention is stated as the length of the customer relationship, with no periods (18). Dated notices for the schedules endpoints (announced 10 September, creation refused from 24 September 2026) and the v2 clone endpoint, but no deprecation policy and no end date for v1 and v2 (10). Sub-processors listed with purpose and country, and 15 days' notice of additions (18).

AXLR 62

Harness reading of the framework checklist, as for the other four harnesses. 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. No product telemetry in the console source (the outbound hosts are openrouter.ai, api.anthropic.com, api.openai.com, a configured local server, api.github.com and github.com for engine downloads, and registry.underpassai.com for the signed catalog index) or in the docs; payload capture is off by default since 9 October 2026, traces omit prompts and credentials, model keys are never given to commands or plugins, and Jev sends requests and answers to TypeSafe only when on, which the setup assistant does when TYPESAFE_API_KEY is set, with a notice on stderr (27). Approval cards, a destructive list that always asks, four work modes, a policy level per execution, a bubblewrap or container sandbox with an enforced egress allow list, and a service that approves nothing but reads. Against that, the setup assistant writes autonomous: true, exec_sandbox is off by default, forged tools run with the person's rights, and the README says the console and worker are not sandboxes (13). Forged tools are refused when a file changes since the forge, a policy forbids calls outside the model, and the docs say MCP tools keep their own side effects, but no prompt-injection guidance was found and MCP results reach the model unmarked (6). The service keeps an event journal, audit records, a receipt with the SHA-256 of every call's arguments and outcome, and correlation ids on every event; the console keeps session records, a redacted app log and /session (14). No SECURITY.md, no security.txt on either host, no advisories, no bug bounty; release checksums and a minisign-signed catalog are integrity, not a programme (2). 27+13+6+14+2 = 62.

Devin 72

Hosted reading. Service-user keys with roles and named permissions per endpoint, revocable, with rotation endpoints in beta for Enterprise and expiring personal tokens. Custom roles are an Enterprise option, and Teams has Admin and Member only (25). Security profiles restrict network, MCP servers, git and the Devin MCP to read-only, and read-only API permissions exist. They are opt-in, and session creation accepts bypass_approval (16). AI Guardrails screen user messages and pull request comments for prompt injection, on Enterprise only, and they don't cover web pages or repository content Devin reads (8). Audit logs through the API need an enterprise-level permission. Session events and insights are available on all plans (10). SOC 2 Type II and ISO/IEC 27001:2022, a disclosure address at security@cognition.ai, no security.txt, and the trust centre answers No to having a bug bounty (13).