Kane CLI vs TinyFish

Kane CLI 20

No x402, MPP or L402 (0). Plan prices are public per the listing check of 1 November 2026 (Starter $17 a month billed annually, Pro $89), and run_end reports credits per run, but the credit price per run isn't published, so plan-only pricing (10). The release says "free to start" and the check found a 10,000-credit, 30-day trial; whether it needs a card couldn't be read today. Judgement call (10 of 20). A person signs up and approves OAuth in a browser, or copies an access key from the dashboard, before any run (0).

TinyFish 55

POST /v1/wallet/top-up on agent.tinyfish.ai answers 402 with a Machine Payments Protocol challenge and takes a Stripe Shared Payment Token for $10 to $500. It refills a prepaid wallet instead of paying per call, needs an existing API key, and the docs still say it is rolling out gradually, so partial credit (15). Per-unit prices are public on 1 November, $0.016 an Agent step, $0.002 a Browser minute and $0.005 a Monitor check, with Search and Fetch at $0 (20). Search and Fetch are free within daily allowances and new accounts start with $8 of wallet funds, no card required per the pricing page (20). A person has to sign up in a browser to get a key or complete OAuth (0). 15+20+20+0 = 55.

Kane CLI 38

The editorial half. The wrapper, skill and docs are Apache-2.0. The runner, assurance and pinned Node binaries are labelled Apache-2.0 on npm with no source in the linked repository, and the service terms couldn't be read (12 of 30). The repository says what leaves the machine. Every run uploads to Test Manager with screenshots and an evidence pack recording the user's name and email, secrets are masked, credentials are stored with mode 0600, and the evidence viewer reads packs locally. The privacy policy, DPA and retention periods are on testmuai.com and weren't read (10 of 30). The changelog calls out breaking changes, such as the removal of the bare-objective shortcut in 0.4.1, but there's no deprecation policy or dated notice period (5 of 20). The changelog describes telemetry (a remote log, support ids, errors spooled and sent at exit) and a daily npm update check. The opt-out KANE_TELEMETRY=0 appears only in a skill reference, not the user guide (8 of 20).

TinyFish 59

The platform is closed with published terms, effective 17 August 2026. The MCP proxy and cookbook are MIT, and the SDK packages state no licence in npm or PyPI metadata (14). The terms allow Tiny Fish to review runs and to train and improve its models on customer data, which is stated plainly. The privacy policy of 14 December 2025 gives no retention periods and names TinyFish Inc. in Cupertino, while the terms name Tiny Fish, Inc. in Los Altos. No public DPA was found (10). No deprecation policy was found. The docs flag two parameters as deprecated without dates, and the terms give 30 days' notice of material changes to the terms (5). No subprocessor list or hosting location was found on readable pages. The enterprise page says data residency is agreed by contract, and the trust centre was unread (3). 14+10+5+3 = 32.

Kane CLI 65

Graded on the CLI as an agent drives it. run_end carries summary, one_liner and only the values asked for with "store as" in final_state, and --max-steps and --timeout bound a run. The skill an agent loads is about 40 KB with 25 reference files (15 of 25). testrun selects by paths, --match and --tags, explore list and explore sessions filter by run, and --json on design and review commands; few size controls beyond that (14 of 20). Exit codes separate pass, fail, setup error and timeout, failure reasons are named, and refusals name the command to run next (18 of 20). Replays of saved tests come from cache, reconcile resumes rather than re-billing, and Exploration allows one run at a time. A one-shot run has no idempotency key and Exploration's default can change data (10 of 20). One required argument for run; Exploration's mutating default isn't a safe one; no SDKs, only the CLI from npm, Homebrew and a shell script (8 of 15).

TinyFish 79

Fetch takes an output format, include and exclude selectors, a cache age and a per-URL timeout, and run reads choose whether screenshots come back as URLs or inline. The MCP docs page now lists 35 tools, among them dataset, monitor and research tools, and the proxy repository's reference (28 September) lists 28 (18). Run lists take a cursor, a limit of 1 to 100 and filters for status, goal text, dates and an owner of me or team, and Search pages from 0 to 10 (18). Errors carry a code, a message, optional details and a request_id, and wallet denials add non_retryable and balance fields (19). The docs still say run creation doesn't support idempotency keys and a retry may create duplicate runs. Cancels, research cancels and session deletes are idempotent and a replayed wallet top-up is credited once. We couldn't read MCP annotations without signing in (9). A run needs only url and goal, a browser session needs nothing, and there are official Python and TypeScript SDKs and a CLI (15). 18+18+19+9+15 = 79.

Kane CLI 50

Graded with the hosted lines, because every run, exploration and design job goes through TestMu AI's service and spends credits, and Exploration runs on HyperExecute. status.lambdatest.io (Statusfy) lists Kane CLI and KaneAI as their own components with incident history and an Atom feed (20). From 3 August to 1 November 2026 no incident names Kane CLI or KaneAI. The platform under it had test creation timeouts from 09:00 to 13:30 UTC on 28 September with slow Automation and HyperExecute starts, six hours of partial macOS HyperExecute degradation on 25 September, and real-device partial outages on 21 August and 22 October (57 minutes). Judgement call, 10 for multi-hour degradation of the service runs depend on. No request rate limit is published; the only numbers are Exploration's built-in limits (an hour, 400 actions or 800 model calls), 20 cloud browsers and a 15-minute real-device queue (5 of 15). No 429 or Retry-After guidance. maintain reconcile is documented as idempotent, Exploration refuses a second run with RUN_ACTIVE, and the skill says never to rerun a paid command just to get a support id (5 of 15). No SLA found; the terms weren't readable (0). Nothing in the repository labels Kane CLI or Exploration beta or preview (10).

TinyFish 60

Graded as a hosted API. agent.tinyfish.ai/status is the vendor's own page with five components and 90 daily bars each, but no incident write-ups (15 of 20). On 1 November agent.tinyfish.ai answered its robots.txt with a redirect to a sign-in page, which we treat as a blocked host, so the page wasn't re-read. Per the 8 October check, not re-read on 1 November, the Agent API showed no downtime from 11 July to 8 October 2026, the Browser API 420 minutes across six days (120 on 28 September), the Search API 1,500 minutes across seven days (420 on 7 August) and the Fetch API 240 on 27 August. Several outages of an hour or more, so 5 of 30, which is also the value for an unread history, so the unread weeks from 9 October can't move it. Limits are published on the pricing page and in the docs (Search 30 requests a minute and 500 an hour, Fetch 150 URLs a minute, 2 concurrent Agent runs, 5 concurrent Browser sessions) (15). Search and Fetch 429s carry Retry-After and X-RateLimit-* headers, the docs give backoff guidance and the SDKs retry 429 and 5xx. The Agent pending-run 429 has no Retry-After and run creation has no idempotency keys (11). The pricing page lists a 99.99 per cent uptime SLA for Enterprise, with no SLA document found (5). Browser, Search and MCP were declared generally available in June 2026 and the API is /v1. The SDKs are 0.x and max_steps is still beta-gated (9). 15+5+15+11+5+9 = 60.

Kane CLI 77

No OpenAPI or JSON Schema file. The NDJSON events are documented in typed Markdown tables, run_end is declared stable across versions, and each command family has its event list (10 of 25). Markdown docs in the repository, an agent skill and a context7.json for agents. The agents.md guide on testmuai.com wasn't readable and no llms.txt was checked (10). The user guide and skill say when to use run, testmd, testrun, assurance or Exploration, and when not (Exploration when there are no written requirements, read_only for data you care about) (17 of 20). Flag tables give defaults and allowed values for --mode, --target, --policy and --bug-detection; --variables and --session-context take free-form JSON (12 of 15). Examples throughout, exit-code tables and troubleshooting tables with named failures such as failed:setup_failed and RUN_ACTIVE. The README's run_end example names credits while the skill names credits_consumed (13 of 15). Semantic versions with a dated Keep a Changelog file and git tags (15).

TinyFish 85

Graded on the REST API. llms.txt now links OpenAPI documents for seven APIs (browser, datasets, fetch, main, monitor, research, search). They sit on agent.tinyfish.ai, which we treat as blocked since 1 November, so the counts below are per the 8 October check, not re-read on 1 November, from the main document (28 operations) and the Browser document (3) (25). llms.txt, a 531 KB llms-full.txt, a Markdown twin of every page and a single page for coding agents (10). All 31 operations read carry a description, several say which endpoint to pick instead, and the MCP tool definitions state when to use and when not to use each tool (17). 189 enums in the main document, ranges such as max_steps 1 to 500, and unknown fields rejected. 730 of 1,068 properties carry a description, goal is free text by design and the MCP search tool takes a JSON-encoded fetch string (12). 698 examples in the main document, every operation documents an error response, and an error-code page gives a fix for each code (14). The path is versioned /v1. The Pulse changelog still stops at 22 June 2026 and the docs have no API changelog (7). 25+10+17+12+14+7 = 85.

Kane CLI 55

Two credential routes. OAuth with browser consent and token revocation on logout, scopes not documented; or the account's username and access key, which the docs say lets anyone holding it run tests on the account, passed as a command-line flag in CI. Not in a URL. 20 of 30 for revocable credentials without documented scopes. Exploration's read_only policy blocks write requests to any host and never presses Delete, Sign out or Pay, with --deny, --allow-domain and --allow-post, and the first mutating run in a project asks once on a terminal. The default is mutating, with no confirmation in agent mode (12 of 20). Results include text from the page under test. The 0.10.0 content fence masks secrets before they leave the worker, but no prompt-injection guidance was found (2 of 15). Every run lands in Test Manager with screenshots, evidence packs record steps and logs, and Exploration findings keep a history naming who changed what (13 of 15). SECURITY.md with security@testmuai.com, signed and notarised binaries per the README, npm registry signatures. No bug bounty, certification or security.txt could be checked, since testmuai.com wasn't readable, and SECURITY.md's supported versions list only 0.1.x (8 of 20).

TinyFish 60

REST calls take an X-API-Key header. Keys are created and deleted in the dashboard and shown once, with no scopes found. Contract organisations add personal and workflow keys that admins can revoke, which separate ownership but not permissions. MCP uses OAuth 2.1, and run IDs are separated between CLI or REST tokens and MCP tokens (21). Vault credentials are opt-in per run with use_vault and can be limited with credential_item_ids, and the docs say the agent never sees passwords. Organisation workflow keys can't use vault credentials or profiles, and admins can set a spend cap that stops usage. No read-only key or approval step for agent actions was found, and max_steps is beta-gated (9). Pages are untrusted content. The plugin page mentions safety rules for untrusted content, and the Fireworks integration guide now tells callers to treat Search results as untrusted because they can contain prompt injection, with a system prompt that says never to follow instructions in them. The Agent and Browser docs carry no such guidance (6). Each run keeps per-step screenshots and optional HTML snapshots, usage lists exist for Search, Fetch and Browser sessions, and audit logs are listed under Enterprise (11). security.txt is valid until 1 September 2027, and the disclosure policy v1.1 of 15 September 2026 has safe harbour and discretionary rewards with no formal bounty. The terms and the pricing page cite ISO 27001, and the trust centre that would show it is drawn by script and was unread (13). 21+9+6+11+13 = 60.