agent-browser vs BxPlaywright

agent-browser 67

The tool needs no credential. Site passwords sit in a local vault encrypted with AES-256-GCM and are filled without reaching the model, while saved state files are plaintext unless a key is set (22 of 30). Domain allowlist, action policy file and confirmation for chosen action categories, all opt-in, with no restriction by default (16 of 20). Content boundary markers with a per-process nonce, output caps and a written threat model with ten known limitations, also opt-in (12 of 15). A local dashboard shows a command and result feed, with no persistent audit log found (7 of 15). No SECURITY.md in the repository and no published advisories. vercel.com's security.txt names HackerOne, and the changelog records hardening of the dashboard (0.35.2) and allowlist (0.32.0) (10 of 20).

BxPlaywright 43

Library reading, as puppeteer and playwright-mcp were. No credential to leak and nothing to scope, so the middle band (20 of 30). Nothing is restricted by default. browser_visit takes any URL with no allow-list, browser_screenshot writes to any path the model passes, and the tools are added to bx-ai's global registry unless registerAiTools is false, so an agent given every registered tool gets a browser. Over MCP it starts Microsoft's server, which limits file access to workspace roots, and Ortus's plugin starts it with --headless but without --isolated, with browser_run_code_unsafe and browser_evaluate in the default set. HTTPS errors are not ignored by default (6 of 20). Page snapshots and text go to the model unfenced, and the docs give no prompt-injection guidance (0 of 15). Screenshot, trace and video policies, show-trace, and interception points such as onPageCreate and onPlaywrightArtifact give the operator a record, all off by default (10 of 15). CONTRIBUTING.md asks for reports to security@ortussolutions.com and the Slack #security channel; SECURITY.md points there. No security.txt (410, per the boxlang-ai check of the same day), no bounty, no advisories. Node.js downloads are checked against the release's SHA-256 list (7 of 20). 20+6+0+10+7 = 43.

agent-browser 84

Graded on the CLI and its MCP server. Every MCP tool has a JSON Schema input with additionalProperties false, and agent-browser.schema.json covers the config file (25). llms.txt and a Markdown copy of each docs page at agent-browser.dev (10). Tool descriptions are one line, such as Scroll the page or an element, with parameter descriptions that say when to omit a field. Few say when not to use a tool (12). Enums, minimums, defaults and required lists, with a free-form extraArgs string array on every tool (12). Many CLI examples and a --json result shape, with error codes such as tab_gone named in passing and no error catalogue found (10). Versioned releases and CHANGELOG.md for each (15).

BxPlaywright 66

Library reading. BoxLang is dynamically typed. The public API is listed in the docs' integrations page, help() and bxplaywright help --json describe it at run time, and the ten agent tools describe each argument in a sentence, with no published API reference (14 of 25). llms.txt with a Markdown copy of each of 18 docs pages (10). The AI page says when to use aiTools(), the registry or MCP, and when to turn registration off. Tool descriptions are one line each (13 of 20). Every agent-tool argument is a string, with no enums or limits; library options are checked, and InvalidOption lists the valid values (9 of 15). An errors page lists each error type with when it happens and the fix, eight runnable examples sit in the repository, and the docs have code on every page (13 of 15). Keep a Changelog and docs versioning on release, but no released version and no tags yet (7 of 15). 14+10+13+9+13+7 = 66.

agent-browser 70

0.38.2 on 1 October 2026 (30). 19 tagged releases since 13 July 2026 (20). 34 commits to main since 8 September, but of the 19 newest open issues 12 had no comment and none had a label, including two off-topic ones left open (10 of 25). Not in the official MCP registry, where a search for agent-browser returned only other publishers. npm and Homebrew are current and crates.io is 19 minor versions behind (5 of 15). No runtime npm dependencies and a lockfile, with CI failing on main (5 of 10).

BxPlaywright 53

The snapshot build of 23 October 2026 is the latest published package, so 30 on time, though it isn't a release (30). No tagged releases or dated changelog entries in the 90 days, only one version rebuilt (0 of 20). 39 pull requests merged by Luis Majano between 30 September and 23 October. Issue replies couldn't be read and bugs go to Jira (10 of 25). Not in the official MCP registry, and the ForgeBox package has no stable version (5 of 15). Playwright for Java 1.63.0, Dependabot for Gradle monthly and Actions quarterly, a daily test run and a format check (8 of 10). 30+0+10+5+8 = 53.

agent-browser 78

The CLI adds no tool definitions to context, and the MCP server defaults to 29 core tools with seven further profiles and paged tools/list up to 158 (22 of 25). snapshot -i, -c, -d, -s and --delta, --max-output, and network request filters by type, method and status (20). Clicks fail early and name the covering element, and JSON errors carry codes such as tab_gone with recovery data, with no documented list of codes (14). readOnlyHint and openWorldHint on every tool, no destructiveHint, and open issue #2032 reports the CLI re-sending a command after it reached the daemon (12). Refs and defaults keep commands short. One CLI, no SDK in a second language (10).

BxPlaywright 63

Library reading, scored on what a tool-calling agent gets. Ten compact in-process tools for BoxLang AI, with nothing to load until a tool is used (25 for ten or fewer), less 5 because every action tool returns the whole page's AI snapshot. Through MCP an agent gets Playwright's 24 default tools (20 of 25). The agent tools have no depth, selector or size limit, and browser_text returns the whole page. snapshot( selector ) exists in the library but not in the tools (8 of 20). Errors come back to the agent as text in the form Error [type]: message Fix: detail, and the error types are documented (16 of 20). Assertions retry until they pass. The in-process tools carry no read-only or destructive hints, and bx-ai doesn't read them; over MCP the annotations are Microsoft's (8 of 20). Works with defaults (headless Chromium, 1280 by 720) and profiles, but in one language, BoxLang, with the MCP route for any other client (11 of 15). 20+8+16+8+11 = 63.

agent-browser 48

Scored as local software. Official npm and Homebrew packages at 0.38.2 with native binaries for five platforms, but crates.io holds 0.19.0 from 14 March 2026 while the README lists cargo install (18 of 20). Public CI runs Rust tests on Linux, macOS and Windows plus native end-to-end tests, and the ten most recent runs on main, 1 to 8 October 2026, concluded in failure. In the newest, Windows Rust tests and the end-to-end job failed and the other jobs passed (10 of 25). 840 open issues and pull requests. Open issues from the last week include click reporting success on hidden elements (#2056), the CLI re-sending a command the daemon already ran (#2032) and --allowed-domains failing on fresh launches (#2031), most without a reply (10 of 25). Every release has a changelog entry and breaking changes get their own heading, on a 0.x line (10 of 15). 0.38.2 with a Labs badge and no stable declaration (0).

BxPlaywright 41

Read with the local-software lines and graded as a pre-release. Official package on ForgeBox with BoxLang 1.17.0 or later, Java 21 and the supported platforms stated, but the only version is 1.0.0-snapshot, rebuilt from the development branch on 23 October 2026, with no binary hash on the entry (14 of 20). tests.yml runs 24 test files on Ubuntu and Windows with JDK 21, real-browser tests on Linux and a test of the full distribution, on pull requests, pushes to development and daily. Whether runs pass wasn't read, since WebFetch refused GitHub (15 of 25). Issues weren't readable for the same reason, bugs go to Ortus's Jira, and the repository is a month old, so there's no record to read (5 of 25). Keep a Changelog with semantic versioning stated and a written public contract, but nothing has been released under it and the changelog reads only [Unreleased] (7 of 15). Ortus announced 1.0 as generally available on 13 October, but no 1.0.0 build or tag exists, so it isn't 1.0 for this line (0 of 15). 14+15+5+7+0 = 41.

agent-browser 63

Apache-2.0, copyright Vercel Inc. (30). No privacy statement for the software. The docs say where credentials, keys and state are stored, that saved state expires after 30 days by default, and that chat goes to the Vercel AI Gateway (18 of 30). No deprecation policy. The changelog flagged the -C snapshot flag as deprecated under a Breaking Changes heading (6 of 20). No telemetry code found in the source, and the README and docs do not state this either way (14 of 20).

BxPlaywright 70

Apache-2.0, OSI approved, in box.json and src/main/resources/LICENSE (30). Local software with no data statement for the module. The source contacts only nodejs.org, Playwright's browser downloads and the pages it is told to open, and Ortus's privacy policy covers its website and customers (15 of 30). The integrations page says a minor release only adds and removals wait for a major release, listed in the changelog. Nothing has been released under it, and the module was renamed from playwright to bxplaywright on 12 October before any release (10 of 20). No telemetry found in the source and no statement either way in the docs, scored as boxlang-ai was (10 of 20). 30+15+10+10 = 65.