Auggie CLI vs AXLR

Auggie CLI 20

Harness reading of the published rubric, scored on the Augment subscription the CLI needs. The self-hosted rule doesn't apply. No payment protocol (0). Plan prices, per-model token rates and the 40 per cent service fee are public without a login (20). No free tier was found on the pricing page. The terms mention trials by order (0). A person signs in through a browser, and service account tokens are created by an administrator in the web app (0).

AXLR 60

No payment protocol (0). Free and Apache-2.0 with nothing to buy, so 20, 20 and 20 on the last three lines by the self-hosted rule, since the docs say what you pay your model provider, the free route needs no card, and it runs local models with no signup at all, the JSON worker without any model. 0+20+20+20 = 60.

Auggie CLI 43

Local-package reading, as for the other closed-source harnesses. The official npm package with macOS, Linux and WSL stated, but the README asks for Node 22 or later while the docs and the package engines say Node 20, so 2 are held back (18). No public CI or test suite, since the source isn't published. The repository has six workflow files, but it holds docs, examples and issues, not the CLI, and their results couldn't be read (0). The repository's tracker had 93 open and 18 closed issues on 18 October 2026, per our GitHub record, so most reports stay open. Which are crashes or regressions, and how fast maintainers reply, couldn't be read. Every run needs Augment's servers, and the status page records major outages on 1 September 2026 (Cosmos and the CLI unavailable, about 40 minutes) and 9 October 2026 (about five and a half hours across services) (5). A changelog by version with fixes and improvements in sections, dated release notes on the website, and a 1.x guide with a breaking-changes section, though the repository changelog carries no dates (10). The latest 0.x channel is described as the current stable line for production, while 1.x is in preview. Judgement call (10). 18+0+5+10+10 = 43.

AXLR 70

Read as a local package, as claude-code, goose, opencode and openhands were. Release archives for Linux, macOS and Windows on amd64 and arm64 with a .sha256 beside each, a GHCR image, an OCI Helm chart and source builds with Go 1.26, all stated in the docs; no Homebrew, npm or apt route (20). CI on main runs vet, race tests and a coverage gate above 80 per cent for both modules, with the sandbox tests required under bubblewrap, and the last ten runs on main to 1 November 2026 all passed; the release workflow tests on six native runners (25). 10 open issues, all opened by the founder on 19 and 21 October 2026 as trackers for service work (work modes, plugins, the sandbox setting, cost limits), none from an outside user and none a crash report, 0 open pull requests and about 100 merged; with 0 stars there is nobody else to report, so the handling of outside reports is untested (20). Tags are semver 0.x and the docs date each behaviour change inline ("until 14 October 2026…"), but the 21 releases have empty notes, there is no CHANGELOG.md, and the product contract records one retirement (the seven 1.0 skill definitions on 5 October 2026) without a notice period (5). Pre-1.0, which the README calls an evolving project (0). 20+25+20+5+0 = 70.

Auggie CLI 58

Graded on the stable line, as the brief asks. 0.36.0 on 21 August 2026, 58 days before this check (20). The 1.x preview and the 0.37.0 prerelease line had builds on 16 October, with 283 prereleases since 20 July. Judgement call to score the stable line. 0.34.0, 0.35.0 and 0.36.0 between 30 July and 21 August 2026 (20). The public tracker had 93 open and 18 closed issues, and 66 closed and no open pull requests, on 18 October 2026, per our GitHub record. Reply times couldn't be read. A support portal, a subreddit and a dated website changelog (6). TypeScript SDK 0.2.0 of 8 July 2026, and its docs still name older models. The Python SDK's PyPI page answered with a bot check (8). npm builds come from GitHub Actions through trusted publishing with registry signatures, and the changelog records dependency scan fixes. No public CI for the CLI. The repository's six workflow files don't build it, and their results couldn't be read. Its newest GitHub release is v0.28.0 from May 2026, while tags follow npm builds up to 16 October (4). 20+20+6+8+4 = 58.

AXLR 79

v0.15.0 on 2026-10-25, with commits on main to 2026-11-01 (30). 21 releases between 4 and 25 October 2026 (20). One maintainer (368 of 399 commits, the rest by two agent accounts), 10 open issues that are his own trackers with no comments, about 100 merged pull requests and no outside report to measure a reply against (15). Not in the MCP registry; the Go module and a proto to generate clients from are the only client code, beside the GHCR image and Helm chart (5). Go 1.26 toolchain pinned, release actions pinned by commit SHA, a go-licenses check that the third-party bundle matches the dependency graph, and CI passing on main (9). 30+20+15+5+9 = 79.

Auggie CLI 65

Framework reading, adapted to a harness driven by a pipeline. --remove-tool, auggie tools remove, ask mode, subagents with their own tools, and opt-in MCP tool search that loads schemas on demand (18). --max-turns, --quiet, --compact, --output-format json, --queue and --show-credits. Budgets per user exist only on Enterprise (15). No documented exit codes or error format for print mode. Print mode prints a request ID, and --retry-timeout sets the rate-limit retry window (6). --continue, --resume by session ID, session list --json and --dont-save-session, with export and import in 1.x (15). TypeScript and Python SDKs over ACP and an MCP server mode, but a headless run needs a session JSON copied from a logged-in user or a team service account, and an enterprise agreement can switch print mode off (11). 18+15+6+15+11 = 65.

AXLR 83

Harness reading of the framework line, scored on what a program driving it has to supply. Six local tools plus the host's bookkeeping tools; work modes and an execution policy hide the tools a run may not use, axlr_catalog searches the catalog instead of loading it, forged tools keep the request prefix for the prompt cache, and a context cap of 75,000 prompt tokens opens a successor session (20). local_read, local_search and local_list page with offset, next_offset and max_bytes; StreamEvents takes after, max_events (1 to 256) and wait_ms; a budget per execution in seconds, requests, tool calls and dollars; 32 tool calls a turn and max_session_usd (20). The --exec run ends in one result line with a status, the pending call, every call's outcome and the usage, and exits 0, 3 or 1; service errors carry codes an agent can act on; a budget stop says which limit (budget: requests 20 reached) and leaves the session resumable (16). Idempotency-Key on every turn, approval and direct call with cross-transport replay, expected_revision on decisions, expected_sha256 on worker writes, and an interrupted effectful call left uncertain for an operator rather than retried (19). One command runs a task, but the first launch needs an OpenRouter key or a local server, the service needs a CA, a principals file and remote KMP and MADE before a first session, and the only SDK is the Go library (8). 20+20+16+19+8 = 83.

Auggie CLI 47

Framework reading (telemetry defaults, approvals, guardrails, sandboxing), five lines as for the other harnesses. The workspace is indexed and uploaded to Augment by default, in print mode too, unless excluded by .gitignore or .augmentignore. A telemetry endpoint is listed with no description or opt-out found. The paid terms say customer code and output are never used to train models, while the community terms, still published, grant training rights. Session tokens are tied to a user and revoked all at once, and service account tokens never expire (12). Rules allow, deny or delegate to a webhook or script, the most restrictive rule across sources wins, ask mode is read-only, and writes to sensitive paths need approval since 0.34.0. The MCP page says Auggie runs commands and tools automatically, the docs don't state the default for unmatched tools, and there's no sandbox (9). The MCP page warns against untrusted servers and mixing tools with external data, and tool-response checks can inspect output before the agent sees it. No network control was found (6). Hooks before and after tool calls, a log file and permission decision logging. Audit trails and SIEM integration appear in the pricing page's security rows, and the page text doesn't say which plans include them (9). A disclosure address and GitHub private reporting in SECURITY.md, SOC 2 Type II and ISO/IEC 42001 named. No security.txt and no bug bounty found. OSV lists no published advisory for the package and NVD no CVE for Auggie, per our GitHub record, so there's no public advisory record to credit or deduct (11). 12+9+6+9+11 = 47.

AXLR 62

Harness reading of the framework checklist, as for the other four harnesses. 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. No product telemetry in the console source (the outbound hosts are openrouter.ai, api.anthropic.com, api.openai.com, a configured local server, api.github.com and github.com for engine downloads, and registry.underpassai.com for the signed catalog index) or in the docs; payload capture is off by default since 9 October 2026, traces omit prompts and credentials, model keys are never given to commands or plugins, and Jev sends requests and answers to TypeSafe only when on, which the setup assistant does when TYPESAFE_API_KEY is set, with a notice on stderr (27). Approval cards, a destructive list that always asks, four work modes, a policy level per execution, a bubblewrap or container sandbox with an enforced egress allow list, and a service that approves nothing but reads. Against that, the setup assistant writes autonomous: true, exec_sandbox is off by default, forged tools run with the person's rights, and the README says the console and worker are not sandboxes (13). Forged tools are refused when a file changes since the forge, a policy forbids calls outside the model, and the docs say MCP tools keep their own side effects, but no prompt-injection guidance was found and MCP results reach the model unmarked (6). The service keeps an event journal, audit records, a receipt with the SHA-256 of every call's arguments and outcome, and correlation ids on every event; the console keeps session records, a redacted app log and /session (14). No SECURITY.md, no security.txt on either host, no advisories, no bug bounty; release checksums and a minisign-signed catalog are integrity, not a programme (2). 27+13+6+14+2 = 62.

Auggie CLI 67

Framework reading. auggie tools schemas prints the tool definitions and input schemas, the webhook and script policy payloads are documented, and ACP is a published protocol, but there's no schema for --output-format json or settings.json (13). docs.augmentcode.com/llms.txt indexes every page with a Markdown twin (10). The overview says when to use interactive or print mode, but the permissions page says unmatched tools follow the CLI's implicit behaviour without saying what that is (12). Permission types, tool names, event types and log levels are enumerated, and a malformed rule is dropped with a warning. The 1.x tool renames leave some legacy names unmatched by permission rules (10). Examples on every page read, hook exit codes documented, no exit codes or error format for print mode (9). Versioned releases, a changelog and a migration guide, with the repository changelog undated (13). 13+10+12+10+9+13 = 67.

AXLR 80

An OpenAPI 3.1 file for the service (15 paths, 23 schemas, mutualTLS security scheme, additionalProperties: false on request bodies), a checked proto for the gRPC service, and the 19 MCP tools built from the same shared schemas (25). No llms.txt on either site (404), and the docs site renders HTML; the Markdown sources are in the repository (5). The product contract tables what each of the four interfaces can and cannot do, the work-mode table states each mode's purpose and tool policy, the transport parity ledger names every verb's arguments and behaviour, and axlr_tools tells the model to look for a tool before forging one (15). Typed inputs throughout, Idempotency-Key 16 to 128 safe ASCII characters, expected_revision a positive integer, policy levels, sandbox profiles and egress values enumerated, budgets as four integers, and the service rejects unknown fields, invalid types and bodies over 4 MiB (14). Examples in every guide, an error shape of code, message and request_id, named codes (409 idempotency_conflict, 400 invalid_cursor, 422 invalid_policy, invalid_budget, sandbox_unavailable, not_ready, invalid_id), a gRPC status mapping, and the worker's five statuses and three exit codes; no single error reference page (14). /v1 routes and an OpenAPI version of 0.1.0 against a 0.15.0 release, and no changelog beyond the dated notes inside the docs (7). 25+5+15+14+14+7 = 80.