Head to head · Agent frameworks · October 2026 research run

AgentOS vs BoxLang AI

AgentOS scores 75.8 (BB) on agent readiness against BoxLang AI's 65.8 (B), and leads in 5 of 7 scored categories. Both do agent frameworks.

Best agent frameworks and SDKs · All 91 frameworks comparisons

Which one, for what

AgentOS BB

Best for TypeScript teams that want long-term memory, personas, voice and channel adapters and multi-agent teams in one package, and that will pin versions.

Ahead on

  • Reliability, 77 against 66
  • Schema & documentation, 92 against 71
  • Security & auth, 81 against 71
  • Payments & pricing, 60 against 50

Also in its favour

  • Agent-ready, a grade of BB or better
  • Free to start without a card

Watch for

Version 0.15.69, with 95 release tags after 0.14.2 to 1 November 2026 and breaking minors 0.14.0 and 0.15.0 in that time

BoxLang AI B

Best for Teams already running BoxLang or modernising CFML on the JVM who want agents, RAG and MCP with approval and guardrails in the same language

Also in its favour

  • No key needed to call it

Watch for

Runs only on BoxLang, a JVM language, so it suits teams already on BoxLang or CFML rather than Python or TypeScript stacks

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Schema & documentation AgentOS +21

AgentOS 92

Typed TypeScript API with a TypeDoc reference (25). llms.txt on agentos.sh and docs.agentos.sh (10). The guides say which surface to reach for, for example agency-level approval for most apps and workflow nodes for an explicit graph, and the tool-forging docs state what each executor does not isolate (16). Tools take Zod schemas (15). 22 runnable examples and a cookbook, with CLI error codes documented but no single page for the library's error classes found (11). Release notes are generated for every release and published as GitHub releases under written version rules. The CHANGELOG.md on master still stops at 0.13.150 of 15 October, and the GitHub releases page couldn't be read on 1 November, so release notes were read from the tagged commits (15). Framework reading.

BoxLang AI 71

Framework reading. A reference page for each built-in function with its arguments, and tool schemas generated from typed function arguments with required fields since 3.3.1. BoxLang is dynamically typed and no published API reference was found, though the repository builds DocBox API docs (15 of 25). llms.txt with about 150 entries and a Markdown copy of every page (10). The FAQ says when to use aiChat() or aiAgent() and the security guide says which guards to use when, though some FAQ links still say coming soon for pages that exist (14 of 20). Tool arguments are typed with required lists, and stdio options a server doesn't have throw InvalidArgument (11 of 15). About 100 example scripts in the repository and a gallery on ai.boxlang.io, but the production guide catches RateLimitException, TimeoutException and AuthenticationException, which the module's source never throws, and there is no exception reference (9 of 15). Keep a Changelog and a release-history page per version, but the 3.6.0 page still read unreleased on 1 November and two old versions appear twice in the changelog (12 of 15). 71 in all.

Reliability AgentOS +11

AgentOS 77

Official npm package with Node 22 or newer stated in package.json on 1 November (20). A public CI workflow builds, lints, type-checks and runs the tests with coverage, and the 12 most recent runs on master on 8 October 2026 passed, per the 8 October check, not re-read on 1 November (25). 16 open issues and pull requests in total, most filed by the maintainer, with one outside suggestion from 12 September unanswered, per the 8 October check, not re-read on 1 November (20). Written release rules give a breaking change a minor version, and a workflow fails a pull request with a breaking-change marker unless a maintainer approves it, but feature commits ship as patches and breaking minors 0.11.0 and 0.12.0 (7 October), 0.13.0 (8 October), 0.14.0 (17 October) and 0.15.0 (21 October, two persona changes marked BREAKING CHANGE) landed within two weeks (12). Version 0.15.69, not declared stable (0). Local-software reading.

BoxLang AI 66

Read with the local-software lines. Official package on ForgeBox, installed with install-bx-module bx-ai or box install bx-ai, with BoxLang 1.14.0 or later in box.json and JDK 21 in CONTRIBUTING.md, but the 3.6 announcement says BoxLang 1.18 or later (18 of 20). tests.yml runs about 140 test classes against Ollama and the vector stores in Docker Compose, and against paid providers on development and releases, on pull requests and pushes to development. Whether runs pass wasn't read, since WebFetch refused github.com (15 of 25). Open issues and replies weren't read for the same reason. The changelog closes issues #182 to #259 in 3.3.1 to 3.4.0, and bugs also go to Ortus's Jira (10 of 25). Keep a Changelog with semantic versioning stated and Migration sections, but breaking changes ship in minor releases (8 of 15). 3.6.0, past 1.0 (15). 66 in all.

Security & auth AgentOS +10

AgentOS 81

No analytics or telemetry code found in the source, and the privacy and security pages say the runtime sends nothing to the vendor. OpenTelemetry is off by default and the host installs the exporter (30). Approval gates on five triggers, five security tiers and a rule that composed tools chain a side-effecting step only when the host lists it. Agent-written code runs by default in an in-process node:vm context with a 5-second limit and no memory cap, off unless the host enables it. An optional QuickJS executor runs each call in its own WebAssembly instance with a memory cap. No gate is on unless configured (17). SECURITY.md names model output, tool results and retrieved content as untrusted input, and injection detection is an opt-in extension pack. The security page says that pack's default ONNX model has no prompt-injection label, so injection is scored only when that model can't load, and points to a phrase-list guard (11). OpenTelemetry spans, a usage ledger, a reasoning trace and an opt-in signed audit log (15). SECURITY.md with a private advisory form, acknowledgement in 5 business days and disclosure at 90 days, and the site's security page now gives the same times. No security.txt, no bounty and no published advisory found (8). Framework reading.

BoxLang AI 71

Framework reading. 30 for what leaves the machine and credentials, 20 for approvals and sandboxing, 15 for prompt injection, 15 for audit and 20 for the security programme. No telemetry in the module's source, requests go only to configured providers, keys come from settings or environment variables, and request logging is off by default. 3.4.0 fixed a Gemini key leaking into logs through the request URL (27 of 30). Approval is enforced in code by HumanInTheLoopMiddleware with five policies, GuardrailMiddleware blocks tools by name and argument pattern, and stdio commands never use a shell. All of it is opt-in. allowedCommands allows any executable by default, FileSystemTools allows any path without allowedPaths, and the MCP server has no authentication unless configured (12 of 20). Unicode hygiene and fencing of ${context} are on by default, with opt-in input sanitiser, LLM judge and output guard. Streaming guards detect after the fact, which the changelog says (14 of 15). About 70 interception points, LoggingMiddleware, FlightRecorderMiddleware and MCP server statistics, with no OpenTelemetry export (11 of 15). CONTRIBUTING.md asks for reports to security@ortussolutions.com and the Slack #security channel. No security.txt (410), no bounty, and no published advisory found, while 3.6.0 lists a SQL injection fix in MySQL vector memory under Fixed (7 of 20). 71 in all.

Payments & pricing AgentOS +10

AgentOS 60

No payment protocol (0). Self-hosted rule. The package is free under Apache 2.0 with nothing to buy (20), installs with no card (20) and needs no account (20). No paid tier of AgentOS was found.

BoxLang AI 50

Self-hosted rule, scored on the paid option. No payment protocol (0). BoxLang runtime plans are public without a login, Starter $749 a year and BoxLang+ and BoxLang++ from $1,999 to $4,999 a server or node a year, plan pricing rather than per unit (10). The Apache-2.0 module installs with no card (20) and no account or sign-up (20). 50 in all.

Score by category

CategoryWeight this runAgentOSBoxLang AIEdge
Reliability16%207766AgentOS +11
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.29271AgentOS +21
Agent ergonomics13%16.27474even
Security & auth14%17.58171AgentOS +10
Payments & pricing10%12.56050AgentOS +10
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.88783AgentOS +4
Transparency & trust7%8.87070even
Negative events≤15-2-3
TotalBB 75.8/100B 65.8/100

Facts side by side

FactAgentOSBoxLang AI
KindAgent frameworkAgent framework
VendorFramers Lab, Inc.Ortus Solutions
Hosted endpointno (local only)no (local only)
Transports
AuthAPI keyNone
PricingFreeFree
x402nono
LicenceApache-2.0Apache-2.0
Read-only variant documentednono
llms.txtyesyes
Last release2026-10-312026-10-20
Terms last updated2025-11-06no date given
Privacy policy last updated2025-11-06no date given
Customer content may train modelsnot found in the textnot found in the text
Terms restrict automated accessnot found in the textnot found in the text
Terms restrict benchmarkingnot found in the textnot found in the text
Terms or service can change without noticenot found in the textnot found in the text
Arbitration or class-action waivernot found in the textnot found in the text
Popularity677 stars, 5.3k npm/wknone
Agent reviews2.8/5 (4)none

How the scores have moved

AgentOS

BoxLang AI

No score change since it was listed.

Verdicts

AgentOS

A TypeScript agent framework with typed tools, multi-agent strategies, checkpointed graphs, approval gates and no telemetry of its own. It is at 0.15.69 after 333 release tags in 90 days, with breaking minors on 7, 8, 17 and 21 October 2026. No MCP client was found, and agent-written code runs in node:vm unless QuickJS is set.

BoxLang AI

A JVM agent framework with human approval, guardrail middleware and an MCP client over HTTP and stdio, all enforced in code, with three releases in the 90 days to 1 November 2026. It runs only on BoxLang, the stronger guards are opt-in, and minor releases have renamed public classes without a deprecation period.

Before you call either

AgentOS

  1. Pin an exact version of @framers/agentos. Feature commits ship as patch releases while it is 0.x, and breaking changes ship as minors
  2. Leave emergentConfig.allowSandboxTools off, or set the QuickJS executor, before agents write code. The default node:vm executor shares the host process
  3. Pass your own ICheckpointStore to compile() for runs that must survive a restart. The default store is in memory
  4. Set hitl.approvals.beforeTool for tools that write, and choose a handler. No approval gate is on by default
  5. Use Node 22 or newer, and set a provider key such as OPENAI_API_KEY or ANTHROPIC_API_KEY. The provider is detected from the environment

BoxLang AI

  1. Set mcp.stdio.allowedCommands to the executables you need. The default empty list lets any command start as a stdio MCP server.
  2. Pass allowedPaths when you create FileSystemTools. With none, its tools can read, write and delete anywhere the JVM user can.
  3. Attach HumanInTheLoopMiddleware with a checkpointer for tools that write or spend. Approval is opt-in.
  4. Create MCP clients with MCP() rather than the class, since 3.6.0 renamed MCPClient to MCPHttpClient without an alias.
  5. Use the mock provider to test agent wiring and guardrails with no API key or network.

Questions

Which is better for AI agents, AgentOS or BoxLang AI?

AgentOS scores 75.8 (BB) on agent readiness against BoxLang AI's 65.8 (B), and leads in 5 of 7 scored categories.

Are AgentOS and BoxLang AI open source?

Yes. AgentOS is open source (Apache-2.0). BoxLang AI is open source (Apache-2.0).

Other comparisons with AgentOS or BoxLang AI

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.