BxPlaywright vs Chrome DevTools MCP

BxPlaywright 41

Read with the local-software lines and graded as a pre-release. Official package on ForgeBox with BoxLang 1.17.0 or later, Java 21 and the supported platforms stated, but the only version is 1.0.0-snapshot, rebuilt from the development branch on 23 October 2026, with no binary hash on the entry (14 of 20). tests.yml runs 24 test files on Ubuntu and Windows with JDK 21, real-browser tests on Linux and a test of the full distribution, on pull requests, pushes to development and daily. Whether runs pass wasn't read, since WebFetch refused GitHub (15 of 25). Issues weren't readable for the same reason, bugs go to Ortus's Jira, and the repository is a month old, so there's no record to read (5 of 25). Keep a Changelog with semantic versioning stated and a written public contract, but nothing has been released under it and the changelog reads only [Unreleased] (7 of 15). Ortus announced 1.0 as generally available on 13 October, but no 1.0.0 build or tag exists, so it isn't 1.0 for this line (0 of 15). 14+15+5+7+0 = 41.

Chrome DevTools MCP 83

Scored as a local stdio package. Official npm package, 1.12.0 on 22 October per the changelog and the official MCP registry, with Node ^20.19, ^22.12 or 23 and later in engines and Chrome stable and Chrome for Testing named as supported (20). Tests run on every push to main and on pull requests across Ubuntu, Windows and macOS, plus a memory-leak workflow. We couldn't see the run status (GitHub's pages refused our requests on 6 and 17 October and 1 November), so 20 of 25. 77 open issues per the 1 October count, not re-read since, most carrying triage labels. Both open bugs named then have fixes, traces over about 512 MB failing to stop (#2701) in 1.10.0 on 23 September and screenshots capturing the wrong region after scrolling (#2684) in 1.11.0 on 15 October (#2685), and 1.12.0 lists 51 fixes (18). release-please writes the changelog from conventional commits, but 1.8.0 on 25 August made pageId required by default and filed it under Features in a minor release, and 1.11.0 turned the screencast tools on by default, also as a feature (10). 1.0.0 shipped on 18 May 2026 (15). 20+20+18+10+15 = 83.

BxPlaywright 53

The snapshot build of 23 October 2026 is the latest published package, so 30 on time, though it isn't a release (30). No tagged releases or dated changelog entries in the 90 days, only one version rebuilt (0 of 20). 39 pull requests merged by Luis Majano between 30 September and 23 October. Issue replies couldn't be read and bugs go to Jira (10 of 25). Not in the official MCP registry, and the ForgeBox package has no stable version (5 of 15). Playwright for Java 1.63.0, Dependabot for Gradle monthly and Actions quarterly, a daily test run and a format check (8 of 10). 30+0+10+5+8 = 53.

Chrome DevTools MCP 93

1.12.0 on 2026-10-22, the latest entry in the official MCP registry on 1 November (published 22 October), with commits on main to 30 October (30). Seven releases in the 90 days from 3 August, 1.7.0 on 10 August to 1.12.0 (20). Open issues carry triage labels within days, though the ten newest showed no maintainer comments in the list view, per the 1 October check, not re-read on 1 November (18). Listed in the official MCP registry as io.github.ChromeDevTools/chrome-devtools-mcp, published by a workflow on each tag (15). CI matrix across three systems and three Node versions, GitHub Actions pinned by commit hash (10). 30+20+18+15+10 = 93.

BxPlaywright 66

Library reading. BoxLang is dynamically typed. The public API is listed in the docs' integrations page, help() and bxplaywright help --json describe it at run time, and the ten agent tools describe each argument in a sentence, with no published API reference (14 of 25). llms.txt with a Markdown copy of each of 18 docs pages (10). The AI page says when to use aiTools(), the registry or MCP, and when to turn registration off. Tool descriptions are one line each (13 of 20). Every agent-tool argument is a string, with no enums or limits; library options are checked, and InvalidOption lists the valid values (9 of 15). An errors page lists each error type with when it happens and the fix, eight runnable examples sit in the repository, and the docs have code on every page (13 of 15). Keep a Changelog and docs versioning on release, but no released version and no tags yet (7 of 15). 14+10+13+9+13+7 = 66.

Chrome DevTools MCP 87

Every tool has a Zod input schema with types, required fields and enums (25). Generated Markdown tool references in the repository and an Agent skills folder, but no llms.txt (7). Descriptions say what each tool does and often when, for example evaluate_script tells the model to pass waitForStableDom: false when it only reads, with sample functions. Few say when not to use a tool (16). Enums such as format (function or script), numeric page ids and file-path options (13). Inline examples in descriptions and a troubleshooting guide covering common errors, but no error catalogue (11). CHANGELOG.md for every release since 0.x (15).

BxPlaywright 43

Library reading, as puppeteer and playwright-mcp were. No credential to leak and nothing to scope, so the middle band (20 of 30). Nothing is restricted by default. browser_visit takes any URL with no allow-list, browser_screenshot writes to any path the model passes, and the tools are added to bx-ai's global registry unless registerAiTools is false, so an agent given every registered tool gets a browser. Over MCP it starts Microsoft's server, which limits file access to workspace roots, and Ortus's plugin starts it with --headless but without --isolated, with browser_run_code_unsafe and browser_evaluate in the default set. HTTPS errors are not ignored by default (6 of 20). Page snapshots and text go to the model unfenced, and the docs give no prompt-injection guidance (0 of 15). Screenshot, trace and video policies, show-trace, and interception points such as onPageCreate and onPlaywrightArtifact give the operator a record, all off by default (10 of 15). CONTRIBUTING.md asks for reports to security@ortussolutions.com and the Slack #security channel; SECURITY.md points there. No security.txt (410, per the boxlang-ai check of the same day), no bounty, no advisories. Node.js downloads are checked against the release's SHA-256 list (7 of 20). 20+6+0+10+7 = 43.

Chrome DevTools MCP 63

No credentials to leak and nothing to scope, so the middle band (20). --javascript-evaluation false disables script tools and javascript:, data: and vbscript: URLs, --allowed-url-pattern and --blocked-url-pattern restrict the browser (now enforced across redirects), MCP roots confine file access, and --isolated uses a throwaway profile. 1.11.0 on 15 October released the --file-navigations flag that rejects file: URLs. No confirmation step for writes, and --isolated, header redaction and the file-navigation block are all off by default (14). SECURITY.md says page content comes back as-is and tells users to prefer trusted content or have the client guard against prompt injection, and since 1.11.0 also says the server trusts its working directory, environment and npm configuration, so it shouldn't be started in an untrusted workspace (8). A --log-file debug log only, no per-call audit (3). Reports go through Google's open-source vulnerability reward programme, and the two moderate advisories of 2026 (CVE-2026-53765 and CVE-2026-53766, both fixed in 1.1.0) are still the only ones in OSV for the package on 1 November. 1.12.0 also fixes filesystem paths that parse as http or ws URLs escaping path validation, with no advisory (18).

BxPlaywright 70

Apache-2.0, OSI approved, in box.json and src/main/resources/LICENSE (30). Local software with no data statement for the module. The source contacts only nodejs.org, Playwright's browser downloads and the pages it is told to open, and Ortus's privacy policy covers its website and customers (15 of 30). The integrations page says a minor release only adds and removals wait for a major release, listed in the changelog. Nothing has been released under it, and the module was renamed from playwright to bxplaywright on 12 October before any release (10 of 20). No telemetry found in the source and no statement either way in the docs, scored as boxlang-ai was (10 of 20). 30+15+10+10 = 65.

Chrome DevTools MCP 82

Apache-2.0 (30). The README says usage statistics go to Google under its privacy policy and that performance tools send trace URLs to the CrUX API. No retention figures for either (20). No deprecation policy. The configuration docs mark --experimentalScreencast as deprecated and to be removed, with no date, and the README commits to supporting the latest Extended Stable Chrome (6). Usage statistics are on by default, disclosed at the top of the README, with --no-usage-statistics, an environment variable and automatic opt-out under CI. CrUX lookups switch off with --no-performance-crux (20).

BxPlaywright 63

Library reading, scored on what a tool-calling agent gets. Ten compact in-process tools for BoxLang AI, with nothing to load until a tool is used (25 for ten or fewer), less 5 because every action tool returns the whole page's AI snapshot. Through MCP an agent gets Playwright's 24 default tools (20 of 25). The agent tools have no depth, selector or size limit, and browser_text returns the whole page. snapshot( selector ) exists in the library but not in the tools (8 of 20). Errors come back to the agent as text in the form Error [type]: message Fix: detail, and the error types are documented (16 of 20). Assertions retry until they pass. The in-process tools carry no read-only or destructive hints, and bx-ai doesn't read them; over MCP the annotations are Microsoft's (8 of 20). Works with defaults (headless Chromium, 1280 by 720) and profiles, but in one language, BoxLang, with the MCP route for any other client (11 of 15). 20+8+16+8+11 = 63.

Chrome DevTools MCP 73

59 tools in the full reference across 11 categories. 1.11.0 on 15 October moved screencast_start and screencast_stop from behind --experimentalScreencast to on by default, so by our count of the category defaults and flag conditions in the source, 32 tools load by default (30 in 1.10.1), unchanged in 1.12.0 on 22 October. Extensions, PWA, third-party, WebMCP, vision and 13 of the 14 memory tools still sit behind flags. More than 30 tools (5, was 15 at about 30). Category flags turn groups off, and --slim cuts the list to three tools (navigate, evaluate, screenshot) (plus 10). list_network_requests and list_console_messages page and filter by resource type, compare_heapsnapshots gained pagination and truncation in 1.11.0, the heap snapshot tools paginate by default since 1.12.0, and large outputs can go to a file path instead of inline (20). Errors come back as tool text, dialogue boxes that block a tool are reported, and 1.11.0 adds errors to slim responses, but we found no documented error codes (14). Every tool sets readOnlyHint (28 true, 39 false in the 1.12.0 source), with no destructiveHint (14). Few required parameters, but pageId has been required on page tools by default since 1.8.0. npm only (10).