Kane CLI vs Lightpanda

Kane CLI 38

The editorial half. The wrapper, skill and docs are Apache-2.0. The runner, assurance and pinned Node binaries are labelled Apache-2.0 on npm with no source in the linked repository, and the service terms couldn't be read (12 of 30). The repository says what leaves the machine. Every run uploads to Test Manager with screenshots and an evidence pack recording the user's name and email, secrets are masked, credentials are stored with mode 0600, and the evidence viewer reads packs locally. The privacy policy, DPA and retention periods are on testmuai.com and weren't read (10 of 30). The changelog calls out breaking changes, such as the removal of the bare-objective shortcut in 0.4.1, but there's no deprecation policy or dated notice period (5 of 20). The changelog describes telemetry (a remote log, support ids, errors spooled and sent at exit) and a daily npm update check. The opt-out KANE_TELEMETRY=0 appears only in a skill reference, not the user guide (8 of 20).

Lightpanda 65

Graded on the browser. AGPL-3.0, an OSI licence, with a CLA for contributions. Lightpanda Cloud has no terms of service that we could find, so we take 2 off (28 of 30). The privacy policy (3 August 2026) covers the website, the browser's telemetry and crash reports, and the cloud (email, PostHog with the IP address and token ID per session), with cloud data kept while the subscription is active and deleted on unsubscribing. It names both an internal database and PostHog as the telemetry store, doesn't list the tool-call and MCP client events 1.0.2 sends, and says nothing on cloud session content or logs. No DPA (15 of 30). No deprecation policy. The CORS change was announced with a flag to turn it off, experimental options are marked as removable without notice, and security fixes go to main only (5 of 20). Telemetry is on by default and disclosed in the README and the privacy policy with an environment-variable opt-out, but it carries the IP address and a persistent instance ID, and the policy's field list is incomplete for 1.0.2 (10 of 20). 28+15+5+10 = 58.

Kane CLI 20

No x402, MPP or L402 (0). Plan prices are public per the listing check of 1 November 2026 (Starter $17 a month billed annually, Pro $89), and run_end reports credits per run, but the credit price per run isn't published, so plan-only pricing (10). The release says "free to start" and the check found a 10,000-credit, 30-day trial; whether it needs a card couldn't be read today. Judgement call (10 of 20). A person signs up and approves OAuth in a browser, or copies an access key from the dashboard, before any run (0).

Lightpanda 40

Open-source software with a hosted option, so pricing is scored on Lightpanda Cloud, as browserless's was. No x402, MPP or L402 (0). Plan prices and the $0.08 per browser-hour overage are published without login (20). Explorer is $0 with 10 browser hours a month, and the sign-up form asks only for an email address (20). A person signs up in a browser and receives the token by email (0). 0+20+20+0 = 40.

Kane CLI 50

Graded with the hosted lines, because every run, exploration and design job goes through TestMu AI's service and spends credits, and Exploration runs on HyperExecute. status.lambdatest.io (Statusfy) lists Kane CLI and KaneAI as their own components with incident history and an Atom feed (20). From 3 August to 1 November 2026 no incident names Kane CLI or KaneAI. The platform under it had test creation timeouts from 09:00 to 13:30 UTC on 28 September with slow Automation and HyperExecute starts, six hours of partial macOS HyperExecute degradation on 25 September, and real-device partial outages on 21 August and 22 October (57 minutes). Judgement call, 10 for multi-hour degradation of the service runs depend on. No request rate limit is published; the only numbers are Exploration's built-in limits (an hour, 400 actions or 800 model calls), 20 cloud browsers and a 15-minute real-device queue (5 of 15). No 429 or Retry-After guidance. maintain reconcile is documented as idempotent, Exploration refuses a second run with RUN_ACTIVE, and the skill says never to rerun a paid command just to get a support id (5 of 15). No SLA found; the terms weren't readable (0). Nothing in the repository labels Kane CLI or Exploration beta or preview (10).

Lightpanda 65

Scored as local software, on the open-source browser. Official Docker image (amd64 and arm64), a Homebrew formula and AUR package at 1.0.2, .deb packages, an npm wrapper and a pip package, with supported platforms stated (Linux glibc x86_64 and aarch64, macOS, WSL2 for Windows). The README's Homebrew line, its download links and the install script all default to nightly builds (18 of 20). Public CI runs zig fmt, unit tests and end-to-end tests through a proxy on every push to main, and WPT results are published daily. We couldn't read whether recent runs passed, because the GitHub API and github.com were unreachable from our network (12 of 25). The issue tracker went unread for the same reason. The vendor's 1.0 post lists dozens of use-after-free and double-free fixes, a crash handler reports crashes, and the agent skill tells agents to reinstall the nightly when crashes occur, so we score this line on thin evidence (10 of 25). Semver tags from 0.3.2 to 1.0.2, and the 1.0 post calls out the CORS default change with the flag to undo it. Release notes on GitHub went unread and there is no CHANGELOG file (10 of 15). 1.0.0 on 2 October 2026 is declared out of beta and stable enough for production (15). 18+12+10+10+15 = 65.

Kane CLI 65

Graded on the CLI as an agent drives it. run_end carries summary, one_liner and only the values asked for with "store as" in final_state, and --max-steps and --timeout bound a run. The skill an agent loads is about 40 KB with 25 reference files (15 of 25). testrun selects by paths, --match and --tags, explore list and explore sessions filter by run, and --json on design and review commands; few size controls beyond that (14 of 20). Exit codes separate pass, fail, setup error and timeout, failure reasons are named, and refusals name the command to run next (18 of 20). Replays of saved tests come from cache, reconcile resumes rather than re-billing, and Exploration allows one run at a time. A one-shot run has no idempotency key and Exploration's default can change data (10 of 20). One required argument for run; Exploration's mutating default isn't a safe one; no SDKs, only the CLI from npm, Homebrew and a shell script (8 of 15).

Lightpanda 75

32 MCP tools at 1.0.2 with no profiles or dynamic loading (5). The CLI fetch and the CDP server reach the same browser with no tool definitions in context, so we add back 5 (10 of 25). markdown, html and screenshot take selector, backendNodeId, maxBytes and strip, links takes limit, tree takes maxDepth, and read tools accept url to navigate and read in one call (20). Errors come back with isError, navigation failures name the cause (navigation failed: CouldntResolveHost), and structuredContent carries httpStatus. The docs warn that a 404 page or a timeout is still reported as success. No list of error codes (14 of 20). Every tool carries readOnlyHint, destructiveHint, idempotentHint and openWorldHint chosen per tool in an exhaustive switch, and nothing needs an idempotency key on a local browser (18 of 20). Most tools need one argument or none, and official packages exist for Node (npm wrapper) and Python (pip). The PyPI page answered a bot check, so the Python package's state went unread (13 of 15). 10+20+14+18+13 = 75.

Kane CLI 55

Two credential routes. OAuth with browser consent and token revocation on logout, scopes not documented; or the account's username and access key, which the docs say lets anyone holding it run tests on the account, passed as a command-line flag in CI. Not in a URL. 20 of 30 for revocable credentials without documented scopes. Exploration's read_only policy blocks write requests to any host and never presses Delete, Sign out or Pay, with --deny, --allow-domain and --allow-post, and the first mutating run in a project asks once on a terminal. The default is mutating, with no confirmation in agent mode (12 of 20). Results include text from the page under test. The 0.10.0 content fence masks secrets before they leave the worker, but no prompt-injection guidance was found (2 of 15). Every run lands in Test Manager with screenshots, evidence packs record steps and logs, and Exploration findings keep a history naming who changed what (13 of 15). SECURITY.md with security@testmuai.com, signed and notarised binaries per the README, npm registry signatures. No bug bounty, certification or security.txt could be checked, since testmuai.com wasn't readable, and SECURITY.md's supported versions list only 0.1.x (8 of 20).

Lightpanda 46

Graded on the local browser, with the cloud's documented token handling counted. No credential is needed locally, the servers bind 127.0.0.1 by default and serve --unix-socket gives an owner-only socket, but the CDP and HTTP MCP servers have no authentication at all when bound wider, and the MCP docs show --host 0.0.0.0 (18 of 30). Lightpanda Cloud's docs pass the API token as ?token= in the CDP, MCP and mcp-remote URLs as the standard route, a secret in a query string (minus 10). CORS on by default since 1.0, sub-resources off unless asked for, and --block-private-networks, --block-cidrs, --block-urls and adblock lists, all opt-in apart from CORS. No domain allowlist or confirmation step for clicks or form submits (12 of 20). The MCP initialize instructions tell the model to treat page content as untrusted and not to follow URLs a page suggests, and $LP_* placeholders keep credentials out of the model's context. getEnv with a name returns a secret's value (10 of 15). Logs go to stderr with levels that show requests and navigations, and the cloud console lists sessions with logs. No persistent audit log locally (8 of 15). SECURITY.md with a private address, fixes on main only, release binaries with SHA-256 digests checked by the install script and Homebrew, and hardening listed in the 1.0 post. No security.txt, bug bounty, certification or readable advisory list (8 of 20). 18-10+12+10+8+8 = 46.

Kane CLI 77

No OpenAPI or JSON Schema file. The NDJSON events are documented in typed Markdown tables, run_end is declared stable across versions, and each command family has its event list (10 of 25). Markdown docs in the repository, an agent skill and a context7.json for agents. The agents.md guide on testmuai.com wasn't readable and no llms.txt was checked (10). The user guide and skill say when to use run, testmd, testrun, assurance or Exploration, and when not (Exploration when there are no written requirements, read_only for data you care about) (17 of 20). Flag tables give defaults and allowed values for --mode, --target, --policy and --bug-detection; --variables and --session-context take free-form JSON (12 of 15). Examples throughout, exit-code tables and troubleshooting tables with named failures such as failed:setup_failed and RUN_ACTIVE. The README's run_end example names credits while the skill names credits_consumed (13 of 15). Semantic versions with a dated Keep a Changelog file and git tags (15).

Lightpanda 82

Graded on the MCP server at 1.0.2, read from source. Every tool declares a JSON Schema input, and page-changing tools declare an outputSchema for url, httpStatus and title (25). llms.txt indexes 46 docs pages and 31 blog posts, but its docs llms-full.txt link returns 404 and the docs have no Markdown twins (7 of 10). Descriptions state purpose and when to prefer another tool (html "Verbose; use only when you need attributes that markdown discards", evaluate as an escape hatch), and initialize returns a driving guide (18 of 20). Enums for load states, required lists and maxBytes caps, but extract takes its selector spec as a JSON string and evaluate takes arbitrary JavaScript (11 of 15). JSON-RPC request and result examples, extract schema examples and a known-behaviours section, with no error catalogue (11 of 15). Semver tags and release notes linked from the blog, which we couldn't read, no CHANGELOG file, and a docs tool reference that matches 1.0.2 but not main (10 of 15). 25+7+18+11+11+10 = 82.