Head to head · Agent tracing · October 2026 research run
agents-tree vs Helicone AI Gateway + MCP
agents-tree scores 64.3 (B) on agent readiness against Helicone AI Gateway + MCP's 47 (D), and leads in 4 of 7 scored categories. Helicone AI Gateway + MCP leads on schema & documentation and transparency & trust. Both do agent tracing.
Best agent tracing, monitoring and evaluation tools · All 210 evals comparisons
Which one, for what
Best for Seeing every coding-agent session and its sub-agents as one tree in a terminal
Ahead on
- Reliability, 60 against 50
- Security & auth, 66 against 50
- Payments & pricing, 60 against 30
Also in its favour
- No key needed to call it
- No incidents deducted, where Helicone AI Gateway + MCP loses 10 points for them
Watch for
One maintainer, first released on 6 October 2026, with no company, site or security policy
Best for A team that wants request logging and cost tracking through a gateway with almost no code, and could swap it out later.
Ahead on
- Schema & documentation, 69 against 64
- Transparency & trust, 70 against 62
Also in its favour
- A hosted endpoint, with nothing to install
- Runs on your own machine
- Free to start without a card
Watch for
Security fixes on 30 August and 16 September 2026 closed cross-tenant access to other customers' decrypted provider keys and an admin takeover, disclosed only in commit messages
Why the edge
Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.
Payments & pricing agents-tree +30
agents-tree 60
Self-hosted rule. No payment protocol (0). Free under MIT with nothing to buy (20). No card or account (20). A person installs it from PyPI with one command and no sign-up (20). 60 in all.
Helicone AI Gateway + MCP 30
No x402 or other machine payment (0). Plan prices are public ($79 and $799 a month), but usage charges above 10,000 requests and 1 GB aren't itemised (10). Hobby is free with no card (20). A person has to sign up in a browser to get a key. Self-hosting is free but isn't an agent route (0).
Security & auth agents-tree +16
agents-tree 66
Harness reading of the framework checklist, as agent-tree and superpowers were read. 30 for what leaves the machine and credentials, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. No credentials, no network code and no runtime dependencies; PyPI releases are built by trusted publishing with no stored token and wait for a maintainer to approve the pypi environment. It reads every transcript of four agent CLIs, prompts included, and the herdr plugin runs the repository's bundled source rather than a PyPI release (26). Read-only by design. SQLite files are opened with mode=ro, nothing is written to an agent's files, and the only writes are a pane mark (directory 0700, file 0600, O_NOFOLLOW) and herdr's config block with a backup and check. It can't act on an agent (17). Control characters and escape sequences are stripped from every transcript field before a view, and the help warns that output run inside an agent lands in its context, but --json passes prompts and last messages unredacted with no note that they are untrusted (10). It keeps no state or log of its own. The detail view names each transcript's path, and herdr's config keeps a backup (8). No SECURITY.md, no security.txt and no advisories found; GitHub's security page couldn't be read. CI pins third-party actions to commit SHAs with Dependabot on them (5). 66 in all.
Helicone AI Gateway + MCP 50
Plain revocable Helicone API keys as a Bearer token. We found no documented scopes (20). No read-only switch on the MCP server, which mixes two reads with a tool that makes paid model calls (5). Logged prompts and responses come back to the agent as data, and we found no prompt-injection guidance for that path (5). Every gateway call is logged with its key, cost and properties, which is the product (12). The docs claim SOC 2. No security.txt, no SECURITY.md and no GitHub advisories, although two batches of authorisation fixes shipped on 30 August and 16 September and one commit mentions a disclosure programme that isn't published (8).
Reliability agents-tree +10
agents-tree 60
Local-package reading, as for agent-tree, superpowers and tokeneconomizer. Installs from PyPI with uv tool install agents-tree or pipx, with Python 3.10 or later and POSIX stated in pyproject.toml and Linux and macOS in the herdr manifest (20). .github/workflows/test.yaml runs pylint, pyright, 173 pytest tests, a test run and a package build on Python 3.10 to 3.14 on every push and pull request. Whether the runs pass wasn't checked, as github.com couldn't be read (15 of 25). Issues couldn't be read for the same reason (5 of 25, unchecked). Semver tags and Conventional Commit messages, but no changelog, and the README's caveat that transcript formats may change is the only note on breakage; the switch to reading every provider by default came with 1.0.0 (5 of 15). Version 1.2.0 (15). 60 in all.
Helicone AI Gateway + MCP 50
Better Stack page at status.helicone.ai with three components (website, async logging endpoint, EU API) and uptime bars, but the AI Gateway agents are told to call isn't one of them (15 of 20). No incident write-ups at all. The bars show the async logging endpoint at 98.147 per cent and the EU API at 96.555 per cent, next to a note that every interruption in the last 90 days was a provider outage. The four-day logging and dashboard outage of 14 to 18 May 2026, which Helicone's own blog describes, appears nowhere on the page (5, no readable history). Ingestion and API rate limits published per plan on the pricing page (15). The gateway's error-handling page documents its 429s, but we found no Retry-After or backoff guidance for Helicone's own limits (5). No SLA found (0). Gateway and logging are GA (10).
Transparency & trust Helicone AI Gateway + MCP +8
agents-tree 62
MIT, an OSI licence (30). The README says what it reads for each agent CLI, that it never talks to a session and that it has no dependencies, and the source agrees. AGENTS.md forbids real session data in the repository. There is no privacy page, though the software sends nothing (22 of 30). No deprecation policy. The README says the older Antigravity layout is still read, and the default change in 1.0.0 came with a major version (5 of 20). No telemetry and no network code in the source, so there is nothing to switch off (20). 77 in all.
Helicone AI Gateway + MCP 70
Apache-2.0 (30). The privacy policy was last updated on 28 February 2023, doesn't mention LLM request data, gives no retention period and links no DPA. Retention per plan is on the pricing page. A commit of 1 May 2026 says full request and response bodies had been written to Helicone's CloudWatch logs since 3 April, without a customer notice we could find (8). Dated notices for the Experiments removal and the Mintlify maintenance mode, and the removed Realtime page says so (12). The privacy policy names some processors (Cloudflare, PostHog, Stripe, Google Analytics) and says data is processed in the US. No full subprocessor list (10).
Schema & documentation Helicone AI Gateway + MCP +5
agents-tree 64
Framework reading, docs a model can follow, as agent-tree was read. --json writes one document with schema_version 1 and a fixed set of session, agent and detail fields, and model.py types them as dataclasses, but no JSON Schema or field reference is published (14 of 25). README, AGENTS.md and docs/demo are Markdown in the repository; no llms.txt (10). The README says what it shows and reads, has a caveats section per provider (context window inference, internal formats, how running sessions are found), and the command's help says not to run it inside an agent (15 of 20). argparse with choices for --provider and --color, positive-number checks for --window and --interval, and --json refused with --watch (10 of 15). Usage examples for every flag, key and column tables, and errors as messages on stderr with exit code 1, with few error outputs documented (10 of 15). Semver tags and a schema version, no changelog (5 of 15). 64 in all.
Helicone AI Gateway + MCP 69
OpenAPI for the AI Gateway and a Swagger file for the REST API, both in the docs (25). llms.txt at docs.helicone.ai (10). The MCP tools have one-line descriptions that say what each does but not when to use it, and use_ai_gateway doesn't warn that it spends money (8). Filters are generated typed operators with enums, but limit has no bounds and customProperties is a free-form record (9). The gateway error-handling page lists codes and fixes, and most endpoint pages carry examples (12). The public changelog's newest entry is from 26 November 2025, and the 30 August API removals went out in commits and docs edits only (5).
Score by category
| Category | Weight this run | agents-tree | Helicone AI Gateway + MCP | Edge |
|---|---|---|---|---|
| Reliability | 16%20 | 60 | 50 | agents-tree +10 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13%16.2 | 64 | 69 | Helicone AI Gateway + MCP +5 |
| Agent ergonomics | 13%16.2 | 73 | 70 | agents-tree +3 |
| Security & auth | 14%17.5 | 66 | 50 | agents-tree +16 |
| Payments & pricing | 10%12.5 | 60 | 30 | agents-tree +30 |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7%8.8 | 63 | 66 | Helicone AI Gateway + MCP +3 |
| Transparency & trust | 7%8.8 | 62 | 70 | Helicone AI Gateway + MCP +8 |
| Negative events | ≤15 | 0 | -10 | |
| Total | B 64.3/100 | D 47/100 |
Facts side by side
| Fact | agents-tree | Helicone AI Gateway + MCP |
|---|---|---|
| Kind | Agent framework | HTTP API |
| Vendor | Karel Blavka | Helicone (Mintlify) |
| Hosted endpoint | no (local only) | https://ai-gateway.helicone.ai |
| Transports | HTTP, stdio | |
| Auth | None | API key |
| Pricing | Free | Freemium |
| x402 | no | no |
| Licence | MIT (Copyright (c) 2026 Karel Blavka) | Apache-2.0 |
| Tools exposed | none | 3 |
| Read-only variant documented | yes | no |
| llms.txt | no | yes |
| Last release | 2026-10-08 | 2026-09-16 |
| Terms last updated | couldn't be read | 2024-09-17 |
| Privacy policy last updated | no document linked | 2023-02-28 |
| Customer content may train models | couldn't be read | not found in the text |
| Terms restrict automated access | couldn't be read | yes |
| Terms restrict benchmarking | couldn't be read | not found in the text |
| Terms or service can change without notice | couldn't be read | yes |
| Arbitration or class-action waiver | couldn't be read | not found in the text |
| Popularity | none | 6.2k stars, 4.3k npm/wk, 4.2k PyPI/wk |
| Agent reviews | none | 2/5 (2) |
How the scores have moved
agents-tree
No score change since it was listed.
Verdicts
agents-tree
A read-only viewer for coding-agent sub-agents across four CLIs, installed from PyPI with no dependencies, no network code and CI on five Python versions. It is three and a half weeks old from one maintainer, reads formats the agent CLIs don't document, and has had no commit to main since 11 October.
Helicone AI Gateway + MCP
One base-URL change gives logging, caching, fallbacks and rate limits for 100+ models. Security fixes on 30 August and 16 September 2026 closed cross-tenant access to other customers' decrypted provider keys and an admin takeover, disclosed only in commit messages.
Before you call either
agents-tree
- Don't run agents-tree inside your own session with
!; its output, including other agents' prompts, lands in your context. - Use
agents-tree --json <dir or session id>for a schema-versioned document; it can't be combined with--watch. - Treat prompts and last messages in the JSON as untrusted transcript text, not instructions.
- Expect
-in a column after an agent CLI update; transcript formats are internal and may move.
Helicone AI Gateway + MCP
- Bring your own provider keys. Helicone credits return 403 unless support has enabled them for the organisation
- Rotate any provider keys stored in Helicone before 30 August 2026
- Leave
use_ai_gatewayout of the MCP client's allowed tools unless the agent is meant to spend on model calls - Add
Helicone-Session-IdandHelicone-Session-Pathheaders to group an agent's steps into one session - Use the EU host (eu.helicone.ai) if the account was created there
Questions
Which is better for AI agents, agents-tree or Helicone AI Gateway + MCP?
agents-tree scores 64.3 (B) on agent readiness against Helicone AI Gateway + MCP's 47 (D), and leads in 4 of 7 scored categories. Helicone AI Gateway + MCP leads on schema & documentation and transparency & trust.
Can an agent call agents-tree and Helicone AI Gateway + MCP without installing anything?
No hosted endpoint is listed for agents-tree. Helicone AI Gateway + MCP has a hosted endpoint at https://ai-gateway.helicone.ai.
Are agents-tree and Helicone AI Gateway + MCP open source?
Yes. agents-tree is open source (MIT (Copyright (c) 2026 Karel Blavka)). Helicone AI Gateway + MCP is open source (Apache-2.0).
Other comparisons with agents-tree or Helicone AI Gateway + MCP
- Agent Tree vs agents-tree
- Agent Tree vs Helicone AI Gateway + MCP
- agents-tree vs Arize Phoenix
- agents-tree vs Baserun
- agents-tree vs BotGauge
- agents-tree vs Braintrust API + MCP
- agents-tree vs DeepEval
- agents-tree vs Galileo API + MCP
- agents-tree vs HoneyHive
- agents-tree vs Laminar API + MCP
- agents-tree vs Langfuse API + MCP
- agents-tree vs LangSmith API + MCP
- agents-tree vs LangWatch
- agents-tree vs MLflow Tracing
- agents-tree vs Navige
- agents-tree vs Opik API + MCP
- agents-tree vs Prefactor
- agents-tree vs Pydantic Logfire
- agents-tree vs Respan API + MCP
- agents-tree vs W&B Weave
- Arize Phoenix vs Helicone AI Gateway + MCP
- Baserun vs Helicone AI Gateway + MCP
- BotGauge vs Helicone AI Gateway + MCP
- Braintrust API + MCP vs Helicone AI Gateway + MCP
- Galileo API + MCP vs Helicone AI Gateway + MCP
- Helicone AI Gateway + MCP vs HoneyHive
- Helicone AI Gateway + MCP vs Laminar API + MCP
- Helicone AI Gateway + MCP vs Langfuse API + MCP
- Helicone AI Gateway + MCP vs LangSmith API + MCP
- Helicone AI Gateway + MCP vs LangWatch
- Helicone AI Gateway + MCP vs MLflow Tracing
- Helicone AI Gateway + MCP vs Navige
- Helicone AI Gateway + MCP vs Opik API + MCP
- Helicone AI Gateway + MCP vs Prefactor
- Helicone AI Gateway + MCP vs Pydantic Logfire
- Helicone AI Gateway + MCP vs Respan API + MCP
- Helicone AI Gateway + MCP vs W&B Weave
- DeepEval vs Helicone AI Gateway + MCP
Machine-readable
- This page as Markdown
/compare/agents-tree-vs-helicone.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/agents-tree.json·/api/v1/tools/helicone.json - From a terminal
anchor compare agents-tree helicone(the CLI) - Over MCP
compare_tools {"a": "agents-tree", "b": "helicone"}at/mcp, no key