AutoCua vs Kane CLI

AutoCua 60

Free self-hosted package with nothing to buy: 0 + 20 + 20 + 20 = 60.

Kane CLI 20

No x402, MPP or L402 (0). Plan prices are public per the listing check of 1 November 2026 (Starter $17 a month billed annually, Pro $89), and run_end reports credits per run, but the credit price per run isn't published, so plan-only pricing (10). The release says "free to start" and the check found a 10,000-credit, 30-day trial; whether it needs a card couldn't be read today. Judgement call (10 of 20). A person signs up and approves OAuth in a browser, or copies an access key from the dashboard, before any run (0).

AutoCua 42

Framework reading. No API reference or typed interface beyond run_agent's documented arguments (5 of 25). No llms.txt (0). The README says what each mode drives and where it runs, and agent_capabilities.md lists what it can do (16 of 20). Modes and device are documented strings (6 of 15). Many examples, no documented errors (10 of 15). Tags without a changelog (5 of 15). 5 + 0 + 16 + 6 + 10 + 5 = 42.

Kane CLI 77

No OpenAPI or JSON Schema file. The NDJSON events are documented in typed Markdown tables, run_end is declared stable across versions, and each command family has its event list (10 of 25). Markdown docs in the repository, an agent skill and a context7.json for agents. The agents.md guide on testmuai.com wasn't readable and no llms.txt was checked (10). The user guide and skill say when to use run, testmd, testrun, assurance or Exploration, and when not (Exploration when there are no written requirements, read_only for data you care about) (17 of 20). Flag tables give defaults and allowed values for --mode, --target, --policy and --bug-detection; --variables and --session-context take free-form JSON (12 of 15). Examples throughout, exit-code tables and troubleshooting tables with named failures such as failed:setup_failed and RUN_ACTIVE. The README's run_end example names credits while the skill names credits_consumed (13 of 15). Semantic versions with a dated Keep a Changelog file and git tags (15).

AutoCua 25

Local package. Installs from PyPI as AutoCua with Python 3.10 or later and the supported platforms stated (20). No CI runs the tests; the only workflow, use_case_label.yml, labels issues (0). GitHub issues weren't read, so the open-issue line is scored absent (0). Four tags in a week with no changelog file (5). Version 0.1.9 (0). 20 + 0 + 0 + 5 + 0 = 25.

Kane CLI 50

Graded with the hosted lines, because every run, exploration and design job goes through TestMu AI's service and spends credits, and Exploration runs on HyperExecute. status.lambdatest.io (Statusfy) lists Kane CLI and KaneAI as their own components with incident history and an Atom feed (20). From 3 August to 1 November 2026 no incident names Kane CLI or KaneAI. The platform under it had test creation timeouts from 09:00 to 13:30 UTC on 28 September with slow Automation and HyperExecute starts, six hours of partial macOS HyperExecute degradation on 25 September, and real-device partial outages on 21 August and 22 October (57 minutes). Judgement call, 10 for multi-hour degradation of the service runs depend on. No request rate limit is published; the only numbers are Exploration's built-in limits (an hour, 400 actions or 800 model calls), 20 cloud browsers and a 15-minute real-device queue (5 of 15). No 429 or Retry-After guidance. maintain reconcile is documented as idempotent, Exploration refuses a second run with RUN_ACTIVE, and the skill says never to rerun a paid command just to get a support id (5 of 15). No SLA found; the terms weren't readable (0). Nothing in the repository labels Kane CLI or Exploration beta or preview (10).

AutoCua 38

Framework reading (telemetry, approvals, guardrails, sandboxing). Provider keys stored in a local folder with Chrome profiles (10 of 30). The worker tools are an allow-list with no write tool, unseen ids are refused, downloads need a yes, but on Windows the model answers UAC elevation prompts itself through a kernel driver, with no person asked (6 of 20). The scraping ruleset has a prompt-injection instruction to log page text that gives instructions (10 of 15). Runs are logged in the app, with no audit log (7 of 15). No security policy, disclosure route or advisories found (5 of 20, for the vault and certificate handling). 10 + 6 + 10 + 7 + 5 = 38.

Kane CLI 55

Two credential routes. OAuth with browser consent and token revocation on logout, scopes not documented; or the account's username and access key, which the docs say lets anyone holding it run tests on the account, passed as a command-line flag in CI. Not in a URL. 20 of 30 for revocable credentials without documented scopes. Exploration's read_only policy blocks write requests to any host and never presses Delete, Sign out or Pay, with --deny, --allow-domain and --allow-post, and the first mutating run in a project asks once on a terminal. The default is mutating, with no confirmation in agent mode (12 of 20). Results include text from the page under test. The 0.10.0 content fence masks secrets before they leave the worker, but no prompt-injection guidance was found (2 of 15). Every run lands in Test Manager with screenshots, evidence packs record steps and logs, and Exploration findings keep a history naming who changed what (13 of 15). SECURITY.md with security@testmuai.com, signed and notarised binaries per the README, npm registry signatures. No bug bounty, certification or security.txt could be checked, since testmuai.com wasn't readable, and SECURITY.md's supported versions list only 0.1.x (8 of 20).

AutoCua 65

v0.1.9 tagged on 4 November 2026 (30). Four tags in 90 days (20). Issues and pull requests weren't read (0 of 25). The PyPI wheel is current (15). No CI; dependencies pinned per OS in requirements files (0 of 10). 30 + 20 + 0 + 15 + 0 = 65.

Kane CLI 77

0.10.0 on npm on 28 October 2026, 4 days ago (30). 28 releases from 3 August to 1 November 2026, 0.8.4 to 0.10.0, each in the changelog (20). The repository has 337 commits, pull requests up to #222 merged within days by LambdaTest staff, a Discord and an issue tracker. GitHub's API wasn't reachable from this session, so open issues and reply times weren't counted (12 of 25). Not an MCP server. The CLI and skill are current on npm and a Homebrew tap, with no SDK (10 of 15). CI runs the skill installer's tests on Linux, macOS and Windows with Node 18 and 22. The CLI's own source and tests aren't public, and the main package carries no npm provenance attestation while the skill package does (5 of 10). Kane CLI is six and a half months old by its first npm release.

AutoCua 55

Framework reading. One call with a mode, provider, model and task runs a whole agent (20 of 25). A separate compression agent splices handoff summaries in, and runs are capped and stoppable (15 of 20). Permission and blocked-input errors are reported specifically, with no error reference (8 of 20). No resume or retry semantics described (2 of 20). Sensible defaults (simulators unless device is set), Python only (10 of 15). 20 + 15 + 8 + 2 + 10 = 55.

Kane CLI 65

Graded on the CLI as an agent drives it. run_end carries summary, one_liner and only the values asked for with "store as" in final_state, and --max-steps and --timeout bound a run. The skill an agent loads is about 40 KB with 25 reference files (15 of 25). testrun selects by paths, --match and --tags, explore list and explore sessions filter by run, and --json on design and review commands; few size controls beyond that (14 of 20). Exit codes separate pass, fail, setup error and timeout, failure reasons are named, and refusals name the command to run next (18 of 20). Replays of saved tests come from cache, reconcile resumes rather than re-billing, and Exploration allows one run at a time. A one-shot run has no idempotency key and Exploration's default can change data (10 of 20). One required argument for run; Exploration's mutating default isn't a safe one; no SDKs, only the CLI from npm, Homebrew and a shell script (8 of 15).