AXLR vs mini-swe-agent

AXLR 79

v0.15.0 on 2026-10-25, with commits on main to 2026-11-01 (30). 21 releases between 4 and 25 October 2026 (20). One maintainer (368 of 399 commits, the rest by two agent accounts), 10 open issues that are his own trackers with no comments, about 100 merged pull requests and no outside report to measure a reply against (15). Not in the MCP registry; the Go module and a proto to generate clients from are the only client code, beside the GHCR image and Helm chart (5). Go 1.26 toolchain pinned, release actions pinned by commit SHA, a go-licenses check that the third-party bundle matches the dependency graph, and CI passing on main (9). 30+20+15+5+9 = 79.

mini-swe-agent 49

2.4.6 on PyPI on 2026-07-23, 87 days before this check (20). One release in the 90 days to 18 October (0). Three commits on main since 23 July, one from pre-commit-ci, the last on 3 September. 38 open issues against 247 closed and 61 open pull requests against 649 closed, per shields.io counts on 18 October, and outside fixes were merged within days through July, but maintainer reply times couldn't be read (12). The Python library ships in the same package as the CLI and is current (10). Dependabot and pre-commit-ci run, and CI workflows exist with their current state unconfirmed (7).

AXLR 80

An OpenAPI 3.1 file for the service (15 paths, 23 schemas, mutualTLS security scheme, additionalProperties: false on request bodies), a checked proto for the gRPC service, and the 19 MCP tools built from the same shared schemas (25). No llms.txt on either site (404), and the docs site renders HTML; the Markdown sources are in the repository (5). The product contract tables what each of the four interfaces can and cannot do, the work-mode table states each mode's purpose and tool policy, the transport parity ledger names every verb's arguments and behaviour, and axlr_tools tells the model to look for a tool before forging one (15). Typed inputs throughout, Idempotency-Key 16 to 128 safe ASCII characters, expected_revision a positive integer, policy levels, sandbox profiles and egress values enumerated, budgets as four integers, and the service rejects unknown fields, invalid types and bodies over 4 MiB (14). Examples in every guide, an error shape of code, message and request_id, named codes (409 idempotency_conflict, 400 invalid_cursor, 422 invalid_policy, invalid_budget, sandbox_unavailable, not_ready, invalid_id), a gRPC status mapping, and the worker's five statuses and three exit codes; no single error reference page (14). /v1 routes and an OpenAPI version of 0.1.0 against a 0.15.0 release, and no changelog beyond the dated notes inside the docs (7). 25+5+15+14+14+7 = 80.

mini-swe-agent 58

Framework reading. Typed pydantic config classes, typed protocols for agents, models and environments, py.typed, a mkdocstrings API reference and a documented trajectory format, but no machine-readable spec for configs or output (15). No llms.txt (/latest/llms.txt and /llms.txt returned 404). The docs are Markdown in the repository, readable raw, though they rely on build-time includes (3). The FAQ and environments page say when to pick mini over SWE-agent and which environment isolates, but the FAQ still says it doesn't use tool calls while the v2 guide says tool calls are now the default (12). Modes are a typed Literal and limits are typed fields, while model_kwargs passes free-form to LiteLLM (10). Hello-world and cookbook examples, a model troubleshooting page and a partial list of exit statuses (11). A dated v2 migration guide and kept v1 docs, with release notes only on GitHub, which we couldn't read (7).

AXLR 83

Harness reading of the framework line, scored on what a program driving it has to supply. Six local tools plus the host's bookkeeping tools; work modes and an execution policy hide the tools a run may not use, axlr_catalog searches the catalog instead of loading it, forged tools keep the request prefix for the prompt cache, and a context cap of 75,000 prompt tokens opens a successor session (20). local_read, local_search and local_list page with offset, next_offset and max_bytes; StreamEvents takes after, max_events (1 to 256) and wait_ms; a budget per execution in seconds, requests, tool calls and dollars; 32 tool calls a turn and max_session_usd (20). The --exec run ends in one result line with a status, the pending call, every call's outcome and the usage, and exits 0, 3 or 1; service errors carry codes an agent can act on; a budget stop says which limit (budget: requests 20 reached) and leaves the session resumable (16). Idempotency-Key on every turn, approval and direct call with cross-transport replay, expected_revision on decisions, expected_sha256 on worker writes, and an interrupted effectful call left uncertain for an operator rather than retried (19). One command runs a task, but the first launch needs an OpenRouter key or a local server, the service needs a CA, a principals file and remote KMP and MADE before a first session, and the only SDK is the Go library (8). 20+20+16+19+8 = 83.

mini-swe-agent 69

Harness reading of the framework line, scored on what an agent or pipeline driving it has to supply. One bash tool and a short system prompt, with output over 10,000 characters cut to its first and last 5,000, but no compaction of a long history (22). Step, cost and wall-time limits, global call and cost limits by environment variable, and a 30-second timeout per command (18). The trajectory JSON records Submitted, LimitsExceeded, TimeExceeded or RepeatedFormatError, but stdout is human-readable only and these aren't process exit codes (12). Model calls retry ten times with exponential backoff, and the trajectory is saved after every step, but there's no resume and no undo (8). Python bindings only. An unattended mini run needs -y, --exit-immediately and MSWEA_CONFIGURED, the last found only in the source (9). No MCP client, ACP or server mode.

AXLR 62

Harness reading of the framework checklist, as for the other four harnesses. 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. No product telemetry in the console source (the outbound hosts are openrouter.ai, api.anthropic.com, api.openai.com, a configured local server, api.github.com and github.com for engine downloads, and registry.underpassai.com for the signed catalog index) or in the docs; payload capture is off by default since 9 October 2026, traces omit prompts and credentials, model keys are never given to commands or plugins, and Jev sends requests and answers to TypeSafe only when on, which the setup assistant does when TYPESAFE_API_KEY is set, with a notice on stderr (27). Approval cards, a destructive list that always asks, four work modes, a policy level per execution, a bubblewrap or container sandbox with an enforced egress allow list, and a service that approves nothing but reads. Against that, the setup assistant writes autonomous: true, exec_sandbox is off by default, forged tools run with the person's rights, and the README says the console and worker are not sandboxes (13). Forged tools are refused when a file changes since the forge, a policy forbids calls outside the model, and the docs say MCP tools keep their own side effects, but no prompt-injection guidance was found and MCP results reach the model unmarked (6). The service keeps an event journal, audit records, a receipt with the SHA-256 of every call's arguments and outcome, and correlation ids on every event; the console keeps session records, a redacted app log and /session (14). No SECURITY.md, no security.txt on either host, no advisories, no bug bounty; release checksums and a minisign-signed catalog are integrity, not a programme (2). 27+13+6+14+2 = 62.

mini-swe-agent 56

Harness reading of the framework checklist, used for every harness in this batch. 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. No telemetry in mini's own source, and prompts go only to the configured provider. LiteLLM's own network behaviour wasn't checked (28). The CLI asks before each command by default, with regex whitelists and -y to skip, but the Python DefaultAgent never asks, the default local environment has no isolation and passes the full host environment to every command, and the experimental bubblewrap sandbox leaks host variables and shares the network (CVE-2026-108592). Docker forwards no host variables unless listed (8). No prompt-injection guidance found (2). A full trajectory of every command and output saved as JSON after each step, and a trajectory inspector (13). A security page naming four maintainers' emails with no response time, no bounty or advisory page found, and CVE-2026-108592 published by VulnCheck on 10 October with no fix released. Compromised LiteLLM 1.82.7 and 1.82.8 are excluded in the dependency pins since 24 March 2026 (#794) (5).