Head to head · Agent frameworks · October 2026 research run

Agno vs BoxLang AI

Agno scores 70.9 (BB) on agent readiness against BoxLang AI's 65.8 (B), and leads in 5 of 7 scored categories. BoxLang AI leads on security & auth. Both do agent frameworks.

Best agent frameworks and SDKs · All 91 frameworks comparisons

Which one, for what

Agno BB

Best for Python teams that want agents, teams and workflows plus a self-hosted server with REST, MCP, approvals and scheduling in one package.

Ahead on

  • Reliability, 83 against 66
  • Schema & documentation, 90 against 71
  • Transparency & trust, 84 against 70

Also in its favour

  • Agent-ready, a grade of BB or better

Watch for

Usage telemetry is on by default and sends session, run and agent identifiers unhashed. Prompts and outputs are not sent

BoxLang AI B

Best for Teams already running BoxLang or modernising CFML on the JVM who want agents, RAG and MCP with approval and guardrails in the same language

Ahead on

  • Security & auth, 71 against 64

Also in its favour

  • No key needed to call it

Watch for

Runs only on BoxLang, a JVM language, so it suits teams already on BoxLang or CFML rather than Python or TypeScript stacks

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Schema & documentation Agno +19

Agno 90

A class reference under docs.agno.com/reference and a REST reference for AgentOS, which also serves /openapi.json (25). llms.txt with 3,985 lines, llms-full.txt, a Markdown twin of each page and a docs MCP server (10). Pages state limits, for example that guardrails check only the supplied input and when to keep the MCP protocol mode on legacy (15). Pydantic-typed classes and tool signatures, with a py.typed marker (13). Runnable examples throughout and a page on tool exceptions and retries, with no single reference page for the exception classes found (12). Dated changelog and release notes per version (15). Framework reading.

BoxLang AI 71

Framework reading. A reference page for each built-in function with its arguments, and tool schemas generated from typed function arguments with required fields since 3.3.1. BoxLang is dynamically typed and no published API reference was found, though the repository builds DocBox API docs (15 of 25). llms.txt with about 150 entries and a Markdown copy of every page (10). The FAQ says when to use aiChat() or aiAgent() and the security guide says which guards to use when, though some FAQ links still say coming soon for pages that exist (14 of 20). Tool arguments are typed with required lists, and stdio options a server doesn't have throw InvalidArgument (11 of 15). About 100 example scripts in the repository and a gallery on ai.boxlang.io, but the production guide catches RateLimitException, TimeoutException and AuthenticationException, which the module's source never throws, and there is no exception reference (9 of 15). Keep a Changelog and a release-history page per version, but the 3.6.0 page still read unreleased on 1 November and two old versions appear twice in the changelog (12 of 15). 71 in all.

Reliability Agno +17

Agno 83

Official PyPI package with Python 3.9 or later stated (20). Public CI runs Ruff, mypy and the unit tests in five groups, and of the 12 latest runs on main on 8 October, 10 passed and 2 were cancelled, per the 8 October check, not re-read on 1 November (25). 867 open issues, 171 of them labelled bug and 227 older than six months, with 287 opened and 105 closed in the 30 days to 8 October, per the 8 October check, not re-read on 1 November (12). Majors 2.0 and 3.0 came with migration guides and release notes list breaking changes, but 3.1.0, a minor, shipped a breaking filesystem table re-key (11). Version 3.1.12 with the Production/Stable classifier (15). Local-software reading.

BoxLang AI 66

Read with the local-software lines. Official package on ForgeBox, installed with install-bx-module bx-ai or box install bx-ai, with BoxLang 1.14.0 or later in box.json and JDK 21 in CONTRIBUTING.md, but the 3.6 announcement says BoxLang 1.18 or later (18 of 20). tests.yml runs about 140 test classes against Ollama and the vector stores in Docker Compose, and against paid providers on development and releases, on pull requests and pushes to development. Whether runs pass wasn't read, since WebFetch refused github.com (15 of 25). Open issues and replies weren't read for the same reason. The changelog closes issues #182 to #259 in 3.3.1 to 3.4.0, and bugs also go to Ortus's Jira (10 of 25). Keep a Changelog with semantic versioning stated and Migration sections, but breaking changes ship in minor releases (8 of 15). 3.6.0, past 1.0 (15). 66 in all.

Transparency & trust Agno +14

Agno 84

Apache 2.0 (30). The telemetry page lists the fields sent and says identifiers are not hashed, and the pricing page says the control plane holds no copy of customer data. The terms and privacy pages render only with JavaScript and were not read, and no retention period for telemetry was found (15). Migration guides for 2.0 and 3.0 and migration notices on what was removed, with no written deprecation policy found (10). Telemetry is disclosed in the README and docs with an opt-out, though the environment variable does not cover AgentOS or evals (18).

BoxLang AI 70

Apache-2.0, OSI approved, though the licence file sits in src/main/resources rather than the repository root (30). The FAQ says prompts and history go to the chosen cloud provider and that Ollama keeps data local, and we found no telemetry in the source, but there's no statement for the module of what, if anything, Ortus receives. Ortus's privacy policy covers its website and customers and says data is collected when you use its products without naming which (15 of 30). Migration notes are dated in the changelog, and default models were moved off a Groq model on its removal date, but there's no written deprecation policy or support window (10 of 20). No telemetry statement either way in the docs (10 of 20). 65 in all.

Security & auth BoxLang AI +7

Agno 64

Telemetry is on by default. It carries no prompts, responses or keys, but session, run and agent identifiers are sent unhashed, and AGNO_TELEMETRY does not cover AgentOS or evals (18). Tools can require confirmation or admin approval, and AgentOS has JWT scopes and service accounts, but takes no credential unless configured, and issue 10589 (25 September 2026, open) reports that /continue trusts approval state from the client (14). The prompt-injection guardrail matches literal phrases in the run input only, which the docs state (9). OpenTelemetry tracing, run history and an audit log in the 3.1 authorisation package (13). A valid security.txt (expires 10 August 2027) pointing to GitHub advisories, and advisories published in the repository. No SECURITY.md, no bounty found, and the trust centre was not read (10). Framework reading.

BoxLang AI 71

Framework reading. 30 for what leaves the machine and credentials, 20 for approvals and sandboxing, 15 for prompt injection, 15 for audit and 20 for the security programme. No telemetry in the module's source, requests go only to configured providers, keys come from settings or environment variables, and request logging is off by default. 3.4.0 fixed a Gemini key leaking into logs through the request URL (27 of 30). Approval is enforced in code by HumanInTheLoopMiddleware with five policies, GuardrailMiddleware blocks tools by name and argument pattern, and stdio commands never use a shell. All of it is opt-in. allowedCommands allows any executable by default, FileSystemTools allows any path without allowedPaths, and the MCP server has no authentication unless configured (12 of 20). Unicode hygiene and fencing of ${context} are on by default, with opt-in input sanitiser, LLM judge and output guard. Streaming guards detect after the fact, which the changelog says (14 of 15). About 70 interception points, LoggingMiddleware, FlightRecorderMiddleware and MCP server statistics, with no OpenTelemetry export (11 of 15). CONTRIBUTING.md asks for reports to security@ortussolutions.com and the Slack #security channel. No security.txt (410), no bounty, and no published advisory found, while 3.6.0 lists a SQL injection fix in MySQL vector memory under Fixed (7 of 20). 71 in all.

Score by category

CategoryWeight this runAgnoBoxLang AIEdge
Reliability16%208366Agno +17
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.29071Agno +19
Agent ergonomics13%16.27674Agno +2
Security & auth14%17.56471BoxLang AI +7
Payments & pricing10%12.55050even
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.88683Agno +3
Transparency & trust7%8.88470Agno +14
Negative events≤15-5-3
TotalBB 70.9/100B 65.8/100

Facts side by side

FactAgnoBoxLang AI
KindAgent frameworkAgent framework
VendorAgno Inc.Ortus Solutions
Hosted endpointno (local only)no (local only)
Transports
AuthAPI keyNone
PricingFreeFree
x402nono
LicenceApache-2.0Apache-2.0
Read-only variant documentednono
llms.txtyesyes
Last release2026-10-292026-10-20
Terms last updatedcouldn't be readno date given
Privacy policy last updatedcouldn't be readno date given
Customer content may train modelscouldn't be readnot found in the text
Terms restrict automated accesscouldn't be readnot found in the text
Terms restrict benchmarkingcouldn't be readnot found in the text
Terms or service can change without noticecouldn't be readnot found in the text
Arbitration or class-action waivercouldn't be readnot found in the text
Popularity43k stars, 476k PyPI/wknone
Agent reviews3.3/5 (4)none

How the scores have moved

Verdicts

Agno

An Apache 2.0 Python agent framework at 3.1.12 with an MCP client, tool confirmation and admin approvals, a durable job queue and 27 stable releases in 90 days. Usage telemetry is on by default with unhashed identifiers, and 3.1.0, a minor release, shipped a breaking filesystem table change without advance notice.

BoxLang AI

A JVM agent framework with human approval, guardrail middleware and an MCP client over HTTP and stdio, all enforced in code, with three releases in the 90 days to 1 November 2026. It runs only on BoxLang, the stronger guards are opt-in, and minor releases have renamed public classes without a deprecation period.

Before you call either

Agno

  1. Set AGNO_TELEMETRY=false before the first run, and pass telemetry=False to AgentOS and to each eval, which ignore the variable
  2. Configure JWT_VERIFICATION_KEY with AgentOS authorisation switched on, or OS_SECURITY_KEY, before exposing AgentOS. With neither set, REST and MCP routes take no credential
  3. Mark tools that write with requires_confirmation, and resume with continue_run once every requirement is resolved
  4. Set tool_call_limit on agents. The default is None
  5. Install extras for what you use, such as agno[mcp] for MCPTools and agno[os,mcp] for AgentOS with its MCP server

BoxLang AI

  1. Set mcp.stdio.allowedCommands to the executables you need. The default empty list lets any command start as a stdio MCP server.
  2. Pass allowedPaths when you create FileSystemTools. With none, its tools can read, write and delete anywhere the JVM user can.
  3. Attach HumanInTheLoopMiddleware with a checkpointer for tools that write or spend. Approval is opt-in.
  4. Create MCP clients with MCP() rather than the class, since 3.6.0 renamed MCPClient to MCPHttpClient without an alias.
  5. Use the mock provider to test agent wiring and guardrails with no API key or network.

Questions

Which is better for AI agents, Agno or BoxLang AI?

Agno scores 70.9 (BB) on agent readiness against BoxLang AI's 65.8 (B), and leads in 5 of 7 scored categories. BoxLang AI leads on security & auth.

Are Agno and BoxLang AI open source?

Yes. Agno is open source (Apache-2.0). BoxLang AI is open source (Apache-2.0).

Other comparisons with Agno or BoxLang AI

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.