BxPlaywright vs TinyFish

BxPlaywright 53

The snapshot build of 23 October 2026 is the latest published package, so 30 on time, though it isn't a release (30). No tagged releases or dated changelog entries in the 90 days, only one version rebuilt (0 of 20). 39 pull requests merged by Luis Majano between 30 September and 23 October. Issue replies couldn't be read and bugs go to Jira (10 of 25). Not in the official MCP registry, and the ForgeBox package has no stable version (5 of 15). Playwright for Java 1.63.0, Dependabot for Gradle monthly and Actions quarterly, a daily test run and a format check (8 of 10). 30+0+10+5+8 = 53.

TinyFish 80

tinyfish 0.11.0 was published to PyPI on 24 October 2026 and 0.10.0 on 21 October, per PyPI's release feed (30). Eight Python SDK releases since 3 August (0.5.0 on 4 August to 0.11.0), counted half as SDK tags, still more than three (20). npm couldn't be read on 1 November, so the TypeScript SDK and CLI dates rest on the 8 October check (@tiny-fish/cli 0.48.1 on 7 October, @tiny-fish/sdk 0.8.0 on 29 September). The Pulse changelog still stops at 22 June 2026. Support is by e-mail and Discord, and the public MCP proxy repository has commits to 31 October. We didn't read issue reply times (8 of 15 on the closed-service line). The hosted MCP server is in the official MCP registry as ai.tinyfish.agent/web-agent, version 1.0.1, published 17 February 2026, and both SDKs are current (15). The MCP proxy repository runs CI, Renovate, and OSV and secrets scanners, cleared 11 OSV dependency findings on 6 October, and publishes through npm trusted publishing. The SDK packages name no source repository (7). 30+20+8+15+7 = 80.

BxPlaywright 41

Read with the local-software lines and graded as a pre-release. Official package on ForgeBox with BoxLang 1.17.0 or later, Java 21 and the supported platforms stated, but the only version is 1.0.0-snapshot, rebuilt from the development branch on 23 October 2026, with no binary hash on the entry (14 of 20). tests.yml runs 24 test files on Ubuntu and Windows with JDK 21, real-browser tests on Linux and a test of the full distribution, on pull requests, pushes to development and daily. Whether runs pass wasn't read, since WebFetch refused GitHub (15 of 25). Issues weren't readable for the same reason, bugs go to Ortus's Jira, and the repository is a month old, so there's no record to read (5 of 25). Keep a Changelog with semantic versioning stated and a written public contract, but nothing has been released under it and the changelog reads only [Unreleased] (7 of 15). Ortus announced 1.0 as generally available on 13 October, but no 1.0.0 build or tag exists, so it isn't 1.0 for this line (0 of 15). 14+15+5+7+0 = 41.

TinyFish 60

Graded as a hosted API. agent.tinyfish.ai/status is the vendor's own page with five components and 90 daily bars each, but no incident write-ups (15 of 20). On 1 November agent.tinyfish.ai answered its robots.txt with a redirect to a sign-in page, which we treat as a blocked host, so the page wasn't re-read. Per the 8 October check, not re-read on 1 November, the Agent API showed no downtime from 11 July to 8 October 2026, the Browser API 420 minutes across six days (120 on 28 September), the Search API 1,500 minutes across seven days (420 on 7 August) and the Fetch API 240 on 27 August. Several outages of an hour or more, so 5 of 30, which is also the value for an unread history, so the unread weeks from 9 October can't move it. Limits are published on the pricing page and in the docs (Search 30 requests a minute and 500 an hour, Fetch 150 URLs a minute, 2 concurrent Agent runs, 5 concurrent Browser sessions) (15). Search and Fetch 429s carry Retry-After and X-RateLimit-* headers, the docs give backoff guidance and the SDKs retry 429 and 5xx. The Agent pending-run 429 has no Retry-After and run creation has no idempotency keys (11). The pricing page lists a 99.99 per cent uptime SLA for Enterprise, with no SLA document found (5). Browser, Search and MCP were declared generally available in June 2026 and the API is /v1. The SDKs are 0.x and max_steps is still beta-gated (9). 15+5+15+11+5+9 = 60.

BxPlaywright 66

Library reading. BoxLang is dynamically typed. The public API is listed in the docs' integrations page, help() and bxplaywright help --json describe it at run time, and the ten agent tools describe each argument in a sentence, with no published API reference (14 of 25). llms.txt with a Markdown copy of each of 18 docs pages (10). The AI page says when to use aiTools(), the registry or MCP, and when to turn registration off. Tool descriptions are one line each (13 of 20). Every agent-tool argument is a string, with no enums or limits; library options are checked, and InvalidOption lists the valid values (9 of 15). An errors page lists each error type with when it happens and the fix, eight runnable examples sit in the repository, and the docs have code on every page (13 of 15). Keep a Changelog and docs versioning on release, but no released version and no tags yet (7 of 15). 14+10+13+9+13+7 = 66.

TinyFish 85

Graded on the REST API. llms.txt now links OpenAPI documents for seven APIs (browser, datasets, fetch, main, monitor, research, search). They sit on agent.tinyfish.ai, which we treat as blocked since 1 November, so the counts below are per the 8 October check, not re-read on 1 November, from the main document (28 operations) and the Browser document (3) (25). llms.txt, a 531 KB llms-full.txt, a Markdown twin of every page and a single page for coding agents (10). All 31 operations read carry a description, several say which endpoint to pick instead, and the MCP tool definitions state when to use and when not to use each tool (17). 189 enums in the main document, ranges such as max_steps 1 to 500, and unknown fields rejected. 730 of 1,068 properties carry a description, goal is free text by design and the MCP search tool takes a JSON-encoded fetch string (12). 698 examples in the main document, every operation documents an error response, and an error-code page gives a fix for each code (14). The path is versioned /v1. The Pulse changelog still stops at 22 June 2026 and the docs have no API changelog (7). 25+10+17+12+14+7 = 85.

BxPlaywright 43

Library reading, as puppeteer and playwright-mcp were. No credential to leak and nothing to scope, so the middle band (20 of 30). Nothing is restricted by default. browser_visit takes any URL with no allow-list, browser_screenshot writes to any path the model passes, and the tools are added to bx-ai's global registry unless registerAiTools is false, so an agent given every registered tool gets a browser. Over MCP it starts Microsoft's server, which limits file access to workspace roots, and Ortus's plugin starts it with --headless but without --isolated, with browser_run_code_unsafe and browser_evaluate in the default set. HTTPS errors are not ignored by default (6 of 20). Page snapshots and text go to the model unfenced, and the docs give no prompt-injection guidance (0 of 15). Screenshot, trace and video policies, show-trace, and interception points such as onPageCreate and onPlaywrightArtifact give the operator a record, all off by default (10 of 15). CONTRIBUTING.md asks for reports to security@ortussolutions.com and the Slack #security channel; SECURITY.md points there. No security.txt (410, per the boxlang-ai check of the same day), no bounty, no advisories. Node.js downloads are checked against the release's SHA-256 list (7 of 20). 20+6+0+10+7 = 43.

TinyFish 60

REST calls take an X-API-Key header. Keys are created and deleted in the dashboard and shown once, with no scopes found. Contract organisations add personal and workflow keys that admins can revoke, which separate ownership but not permissions. MCP uses OAuth 2.1, and run IDs are separated between CLI or REST tokens and MCP tokens (21). Vault credentials are opt-in per run with use_vault and can be limited with credential_item_ids, and the docs say the agent never sees passwords. Organisation workflow keys can't use vault credentials or profiles, and admins can set a spend cap that stops usage. No read-only key or approval step for agent actions was found, and max_steps is beta-gated (9). Pages are untrusted content. The plugin page mentions safety rules for untrusted content, and the Fireworks integration guide now tells callers to treat Search results as untrusted because they can contain prompt injection, with a system prompt that says never to follow instructions in them. The Agent and Browser docs carry no such guidance (6). Each run keeps per-step screenshots and optional HTML snapshots, usage lists exist for Search, Fetch and Browser sessions, and audit logs are listed under Enterprise (11). security.txt is valid until 1 September 2027, and the disclosure policy v1.1 of 15 September 2026 has safe harbour and discretionary rewards with no formal bounty. The terms and the pricing page cite ISO 27001, and the trust centre that would show it is drawn by script and was unread (13). 21+9+6+11+13 = 60.

BxPlaywright 63

Library reading, scored on what a tool-calling agent gets. Ten compact in-process tools for BoxLang AI, with nothing to load until a tool is used (25 for ten or fewer), less 5 because every action tool returns the whole page's AI snapshot. Through MCP an agent gets Playwright's 24 default tools (20 of 25). The agent tools have no depth, selector or size limit, and browser_text returns the whole page. snapshot( selector ) exists in the library but not in the tools (8 of 20). Errors come back to the agent as text in the form Error [type]: message Fix: detail, and the error types are documented (16 of 20). Assertions retry until they pass. The in-process tools carry no read-only or destructive hints, and bx-ai doesn't read them; over MCP the annotations are Microsoft's (8 of 20). Works with defaults (headless Chromium, 1280 by 720) and profiles, but in one language, BoxLang, with the MCP route for any other client (11 of 15). 20+8+16+8+11 = 63.

TinyFish 79

Fetch takes an output format, include and exclude selectors, a cache age and a per-URL timeout, and run reads choose whether screenshots come back as URLs or inline. The MCP docs page now lists 35 tools, among them dataset, monitor and research tools, and the proxy repository's reference (28 September) lists 28 (18). Run lists take a cursor, a limit of 1 to 100 and filters for status, goal text, dates and an owner of me or team, and Search pages from 0 to 10 (18). Errors carry a code, a message, optional details and a request_id, and wallet denials add non_retryable and balance fields (19). The docs still say run creation doesn't support idempotency keys and a retry may create duplicate runs. Cancels, research cancels and session deletes are idempotent and a replayed wallet top-up is credited once. We couldn't read MCP annotations without signing in (9). A run needs only url and goal, a browser session needs nothing, and there are official Python and TypeScript SDKs and a CLI (15). 18+18+19+9+15 = 79.

BxPlaywright 70

Apache-2.0, OSI approved, in box.json and src/main/resources/LICENSE (30). Local software with no data statement for the module. The source contacts only nodejs.org, Playwright's browser downloads and the pages it is told to open, and Ortus's privacy policy covers its website and customers (15 of 30). The integrations page says a minor release only adds and removals wait for a major release, listed in the changelog. Nothing has been released under it, and the module was renamed from playwright to bxplaywright on 12 October before any release (10 of 20). No telemetry found in the source and no statement either way in the docs, scored as boxlang-ai was (10 of 20). 30+15+10+10 = 65.

TinyFish 59

The platform is closed with published terms, effective 17 August 2026. The MCP proxy and cookbook are MIT, and the SDK packages state no licence in npm or PyPI metadata (14). The terms allow Tiny Fish to review runs and to train and improve its models on customer data, which is stated plainly. The privacy policy of 14 December 2025 gives no retention periods and names TinyFish Inc. in Cupertino, while the terms name Tiny Fish, Inc. in Los Altos. No public DPA was found (10). No deprecation policy was found. The docs flag two parameters as deprecated without dates, and the terms give 30 days' notice of material changes to the terms (5). No subprocessor list or hosting location was found on readable pages. The enterprise page says data residency is agreed by contract, and the trust centre was unread (3). 14+10+5+3 = 32.