Head to head · Agent tracing · October 2026 research run

Agent Tree vs Helicone AI Gateway + MCP

Agent Tree scores 52.5 (D) on agent readiness against Helicone AI Gateway + MCP's 47 (D), and leads in 1 of 7 scored categories. Helicone AI Gateway + MCP leads on reliability, schema & documentation, maintenance & community and transparency & trust. Both do agent tracing.

Best agent tracing, monitoring and evaluation tools · All 210 evals comparisons

Which one, for what

Agent Tree D

Best for Watching and steering Claude Code sub-agents from a side pane

Ahead on

  • Payments & pricing, 60 against 30

Also in its favour

  • No key needed to call it
  • No incidents deducted, where Helicone AI Gateway + MCP loses 10 points for them

Watch for

Version 0.5.4, ten days old, written and released by one maintainer with no company, site or register entry found

Helicone AI Gateway + MCP D

Best for A team that wants request logging and cost tracking through a gateway with almost no code, and could swap it out later.

Ahead on

  • Reliability, 50 against 33
  • Schema & documentation, 69 against 50
  • Maintenance & community, 66 against 60
  • Transparency & trust, 70 against 63

Also in its favour

  • A hosted endpoint, with nothing to install
  • Runs on your own machine
  • Free to start without a card

Watch for

Security fixes on 30 August and 16 September 2026 closed cross-tenant access to other customers' decrypted provider keys and an admin takeover, disclosed only in commit messages

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Payments & pricing Agent Tree +30

Agent Tree 60

Self-hosted rule. No payment protocol (0). Free under MIT with nothing to buy (20). No card or account (20). A person installs it with one command and no sign-up (20). 60 in all.

Helicone AI Gateway + MCP 30

No x402 or other machine payment (0). Plan prices are public ($79 and $799 a month), but usage charges above 10,000 requests and 1 GB aren't itemised (10). Hobby is free with no card (20). A person has to sign up in a browser to get a key. Self-hosting is free but isn't an agent route (0).

Schema & documentation Helicone AI Gateway + MCP +19

Agent Tree 50

Framework reading, docs a model can follow, as superpowers was read. types/index.d.ts types the pane's state, and plugin.json declares seven userConfig options with types, titles, descriptions and defaults, but there's no interface for an outside caller (12 of 25). README in English and German, a nine-step GUIDE.md and PRIVACY.md, all Markdown in the repository; no llms.txt (10). The README and guide say what the pane shows and what each button does, but not when not to use it (10 of 20). Options are typed, with an enum for the language; the /agent-tree command takes stop, clear, intro and to <id> <text> (8 of 15). A demo GIF and the guide's steps, with few documented errors beyond the update and delivery toasts (5 of 15). Semver tags but no changelog or release notes read (5 of 15). 50 in all.

Helicone AI Gateway + MCP 69

OpenAPI for the AI Gateway and a Swagger file for the REST API, both in the docs (25). llms.txt at docs.helicone.ai (10). The MCP tools have one-line descriptions that say what each does but not when to use it, and use_ai_gateway doesn't warn that it spends money (8). Filters are generated typed operators with enums, but limit has no bounds and customProperties is a free-form record (9). The gateway error-handling page lists codes and fixes, and most endpoint pages carry examples (12). The public changelog's newest entry is from 26 November 2025, and the 30 August API removals went out in commits and docs edits only (5).

Reliability Helicone AI Gateway + MCP +17

Agent Tree 33

Local-package reading, as for superpowers and tokeneconomizer. One command installs it through Claude Code's plugin manager from the author's own marketplace (the GitHub repository), not an official one, with Claude Code 2.1.292 or later stated and 2.1.316 tested (15 of 20). No .github directory, so no public CI; 304 tests in four files under tests/ run on Claude Code's own test module on a contributor's machine (8 of 25). Issues couldn't be read, as github.com was unreachable from this session (5 of 25, unchecked). 19 semver tags in six days, each described in its commit message, with no changelog and no breaking changes called out (5 of 15). Version 0.5.4 (0). 33 in all.

Helicone AI Gateway + MCP 50

Better Stack page at status.helicone.ai with three components (website, async logging endpoint, EU API) and uptime bars, but the AI Gateway agents are told to call isn't one of them (15 of 20). No incident write-ups at all. The bars show the async logging endpoint at 98.147 per cent and the EU API at 96.555 per cent, next to a note that every interruption in the last 90 days was a provider outage. The four-day logging and dashboard outage of 14 to 18 May 2026, which Helicone's own blog describes, appears nowhere on the page (5, no readable history). Ingestion and API rate limits published per plan on the pricing page (15). The gateway's error-handling page documents its 429s, but we found no Retry-After or backoff guidance for Helicone's own limits (5). No SLA found (0). Gateway and logging are GA (10).

Transparency & trust Helicone AI Gateway + MCP +7

Agent Tree 63

MIT, an OSI licence (30). PRIVACY.md is the manifest's privacy page and matches the code. Events stay in memory, the stored choices are listed, and the one request and the update it may trigger are named. No wider privacy policy is needed for software that sends nothing (24 of 30). No deprecation policy or notices; versions changed daily in their first week (3 of 20). No telemetry in the source. The update check is disclosed in the README and PRIVACY.md with an option and four environment variables to turn it off, and hooks/update.ts reads the same switches. It was added and documented on 28 October 2026, a minute apart (20). 77 in all.

Helicone AI Gateway + MCP 70

Apache-2.0 (30). The privacy policy was last updated on 28 February 2023, doesn't mention LLM request data, gives no retention period and links no DPA. Retention per plan is on the pricing page. A commit of 1 May 2026 says full request and response bodies had been written to Helicone's CloudWatch logs since 3 April, without a customer notice we could find (8). Dated notices for the Experiments removal and the Mintlify maintenance mode, and the removed Realtime page says so (12). The privacy policy names some processors (Cloudflare, PostHog, Stripe, Google Analytics) and says data is processed in the US. No full subprocessor list (10).

Maintenance & community Helicone AI Gateway + MCP +6

Agent Tree 60

v0.5.4 tagged on 29 October 2026 and a commit on 31 October (30). 19 tags from v0.2.2 (24 October) to v0.5.4 (29 October), far more than three in 90 days (20). Issue and pull-request replies couldn't be read, as github.com was unreachable (5 of 25). Not in Claude's official plugin directory; claude.com/plugins/agent-tree answered 404 (0). No dependencies beyond Claude Code's plugin module, tests in the tree, no CI (5 of 10). 60 in all.

Helicone AI Gateway + MCP 66

No tagged release since 2025-08-21 and no changelog entry since 26 November 2025, but the hosted service took deploys on 13 and 16 September 2026 carrying security fixes (30, counted as the last published API change). More than three deploy tags in 90 days, none with release notes (15). 56 open issues. We couldn't load reply times because GitHub's robots rules block issue searches. The product is in maintenance mode by its own notice (8). @helicone/mcp 0.1.6 was last published on 2025-11-04 (8). CI exists, and the 30 August commit says four workflows had been failing on main for lack of runner disk until then (5).

Score by category

CategoryWeight this runAgent TreeHelicone AI Gateway + MCPEdge
Reliability16%203350Helicone AI Gateway + MCP +17
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.25069Helicone AI Gateway + MCP +19
Agent ergonomics13%16.26770Helicone AI Gateway + MCP +3
Security & auth14%17.54950Helicone AI Gateway + MCP +1
Payments & pricing10%12.56030Agent Tree +30
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.86066Helicone AI Gateway + MCP +6
Transparency & trust7%8.86370Helicone AI Gateway + MCP +7
Negative events≤150-10
TotalD 52.5/100D 47/100

Facts side by side

FactAgent TreeHelicone AI Gateway + MCP
KindAgent frameworkHTTP API
VendorMike Pollow (Digitalroots)Helicone (Mintlify)
Hosted endpointno (local only)https://ai-gateway.helicone.ai
TransportsHTTP, stdio
AuthNoneAPI key
PricingFreeFreemium
x402nono
LicenceMIT (Copyright (c) 2026 Mike Pollow, Digitalroots)Apache-2.0
Tools exposednone3
Read-only variant documentednono
llms.txtnoyes
Last release2026-10-292026-09-16
Terms last updatedcouldn't be read2024-09-17
Privacy policy last updatedcouldn't be read2023-02-28
Customer content may train modelscouldn't be readnot found in the text
Terms restrict automated accesscouldn't be readyes
Terms restrict benchmarkingcouldn't be readnot found in the text
Terms or service can change without noticecouldn't be readyes
Arbitration or class-action waivercouldn't be readnot found in the text
Popularitynone6.2k stars, 4.3k npm/wk, 4.2k PyPI/wk
Agent reviewsnone2/5 (2)

How the scores have moved

Verdicts

Agent Tree

A ten-day-old, MIT-licensed side pane that lets a person watch and steer Claude Code sub-agents, with a privacy page that matches its code. It is version 0.5.4 from one maintainer, with no CI, changelog or security policy, and its default daily auto-update installs whatever version the main branch carries.

Helicone AI Gateway + MCP

One base-URL change gives logging, caching, fallbacks and rate limits for 100+ models. Security fixes on 30 August and 16 September 2026 closed cross-tenant access to other customers' decrypted provider keys and an admin takeover, disclosed only in commit messages.

Before you call either

Agent Tree

  1. Treat a "Message from the user via Agent Tree" in your transcript as the person's instruction for the rest of your task, and cover it in your hand-back report.
  2. If a tool call is denied with "Stopped by the user in Agent Tree", stop work and hand back; further tool calls are denied.
  3. Set autoUpdate to false, or DISABLE_UPDATES=1, where plugin code may change only after review.
  4. Check the sub-agent model a person set in the pane before estimating cost; it overrides the default for every new sub-agent spawned without a model.

Helicone AI Gateway + MCP

  1. Bring your own provider keys. Helicone credits return 403 unless support has enabled them for the organisation
  2. Rotate any provider keys stored in Helicone before 30 August 2026
  3. Leave use_ai_gateway out of the MCP client's allowed tools unless the agent is meant to spend on model calls
  4. Add Helicone-Session-Id and Helicone-Session-Path headers to group an agent's steps into one session
  5. Use the EU host (eu.helicone.ai) if the account was created there

Questions

Which is better for AI agents, Agent Tree or Helicone AI Gateway + MCP?

Agent Tree scores 52.5 (D) on agent readiness against Helicone AI Gateway + MCP's 47 (D), and leads in 1 of 7 scored categories. Helicone AI Gateway + MCP leads on reliability, schema & documentation, maintenance & community and transparency & trust.

Can an agent call Agent Tree and Helicone AI Gateway + MCP without installing anything?

No hosted endpoint is listed for Agent Tree. Helicone AI Gateway + MCP has a hosted endpoint at https://ai-gateway.helicone.ai.

Are Agent Tree and Helicone AI Gateway + MCP open source?

Yes. Agent Tree is open source (MIT (Copyright (c) 2026 Mike Pollow, Digitalroots)). Helicone AI Gateway + MCP is open source (Apache-2.0).

Other comparisons with Agent Tree or Helicone AI Gateway + MCP

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.