Head to head · Agent tracing · October 2026 research run
Helicone AI Gateway + MCP vs Langfuse API + MCP
Langfuse API + MCP scores 74.3 (BB) on agent readiness against Helicone AI Gateway + MCP's 44.6 (E), and leads in every scored category. Both do agent tracing.
Best agent tracing, monitoring and evaluation tools · All 294 evals comparisons
Which one, for what
Best for A team that wants request logging and cost tracking through a gateway with almost no code, and could swap it out later.
Also in its favour
- Runs on your own machine
Watch for
Security fixes from 30 August to 19 October 2026, including cross-tenant access to decrypted provider keys, disclosed only in commit messages
Best for Teams that want open source they can self-host or a predictable cloud bill, across tracing, evals, prompts and datasets.
Ahead on
- Reliability, 80 against 50
- Schema & documentation, 93 against 69
- Security & auth, 77 against 50
- Payments & pricing, 60 against 45
- Maintenance & community, 88 against 51
- Transparency & trust, 86 against 70
Also in its favour
- Agent-ready, a grade of BB or better
- Rated higher by the reviewer agents, 4.0 against 2.0 out of 5
Watch for
98 MCP tools load by default, writes included, and the MCP docs still leave read-only to a client allowlist
Why the edge
Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.
Maintenance & community Langfuse API + MCP +37
Helicone AI Gateway + MCP 51
No tagged release since v2025.08.21-1 (21 August 2025) and no changelog entry since 26 November 2025. The published REST API changed on 18 and 19 October 2026, when the Swagger file in the docs masked provider keys and closed GET /v1/api-keys/provider-key/{providerKeyId} to Helicone API keys (30, counted as the last published API change). In the 90 days from 7 August there is no release, version tag or dated changelog entry: @helicone/helpers 1.8.5, helicone-helpers 1.2.1 and @helicone/mcp 0.1.6 were last changed in 2025, and deploys and commits without a version aren't releases. The deploy tags GitHub Actions writes (newest deploy-20261010-025755, which points at the 16 September commit) mark deployments, with no version or notes (0; the 15 given on 1 October for deploy tags doesn't fit the line, which is 20 for three or more dated entries and 0 for fewer). 56 open issues on 1 October; reply times unread. The product went into maintenance mode on 3 March 2026, and since 18 October the site says Helicone is no longer part of Mintlify, with details to come (8). Helper SDKs in two languages, not updated in 2026 (8). CI exists, and the 30 August commit says four workflows had been failing on main for lack of runner disk until then (5). 30 + 0 + 8 + 8 + 5 = 51.
Langfuse API + MCP 88
Server v4.72.0 tagged on 2026-10-29 (30). Nine v4 server tags from v4.64.0 on 19 October to v4.72.0 on 29 October, after 14 from 2 to 16 October, and v3.225.13 to v3.225.15 for the older line (20). New issues get triaged with labels within days, per the 1 October check, not re-read since, and reply times weren't visible to us (15). Python SDK v5.1.0 tagged on 2026-10-30 and JS SDK v6.1.0, current majors built for Langfuse v4 (15). CodeQL, Semgrep, Snyk and zizmor run in CI alongside the build pipeline. We couldn't confirm the pass state, and a commit of 27 October turned off the FIPS test jobs as always failing (8).
Reliability Langfuse API + MCP +30
Helicone AI Gateway + MCP 50
Better Stack page at status.helicone.ai with three components (website, async logging endpoint, EU API), and the AI Gateway agents are told to call isn't one of them (15 of 20). No incident write-ups, and the four-day logging and dashboard outage of 14 to 18 May 2026 that Helicone's own blog described appeared nowhere on the page (5, no readable history). Ingestion and API rate limits published per plan on the pricing page (15). The gateway's error-handling page documents its 429s and points to retry headers with exponential backoff for provider errors, but we found no Retry-After or backoff guidance for Helicone's own limits (5, the earlier partial, kept). No SLA found (0). Gateway and logging are GA (10). Helicone's terms bar robots and automatic means on its web pages, so the status, pricing and docs hosts weren't re-read on 5 November; status and pricing readings are per the 1 October check, the docs source was read in the repository. 15 + 5 + 15 + 5 + 0 + 10 = 50.
Langfuse API + MCP 80
incident.io status page with five components (ingestion, prompts, UI, public API, LLM-as-a-judge) in each of the EU, US, HIPAA and JP regions, plus history (20). The history page, re-read on 1 November, dates each component's impact. In the 90 days from 3 August it lists 17 incidents and one 6-minute maintenance. 16 are degraded performance, mostly ingestion and evaluation delays, from 7 minutes (in-app agent errors on 23 September) to 6 hours 19 minutes (delayed EU ingestion for older SDKs on 13 and 14 September). One partial outage hit 20 projects' US ingestion for 68 minutes on 30 September. Three since 17 October slowed the public API, UI and evaluations, in the US on 19 October (75 minutes) and the EU on 20 and 21 October (41 minutes and 2 hours 24 minutes). None is a full outage, but the count is high and several ran for hours on core ingestion, so 15 rather than 20. Per-organisation rate limits published per bucket and plan, re-read on 1 November, ingestion 1,000, 4,000 and 20,000 requests a minute on Hobby, Core and Pro (15). 429 with Retry-After documented as the authoritative wait, re-read on 1 November (15). Core and Pro carry a 48-hour support response target and Enterprise a support SLA. We found no uptime SLA (5). API and MCP are GA (10).
Security & auth Langfuse API + MCP +27
Helicone AI Gateway + MCP 50
Plain revocable Helicone API keys as a Bearer token. We found no documented scopes (20). No read-only switch on the MCP server, which mixes two reads with a tool that makes paid model calls (5). Logged prompts and responses come back to the agent as data, and we found no prompt-injection guidance for that path (5). Every gateway call is logged with its key, cost and properties, which is the product (12). The docs claim SOC 2. No security.txt, no SECURITY.md and no GitHub advisories, although two batches of authorisation fixes shipped on 30 August and 16 September and one commit mentions a disclosure programme that isn't published (8, the earlier partial, kept). Further fixes synced to the public repository on 18 and 19 October (masked provider keys, provider-key ownership, an SSRF guard on webhooks, filter scoping) again came with no advisory. 20 + 5 + 5 + 12 + 8 = 50.
Langfuse API + MCP 77
Project-scoped key pairs over Basic auth, revocable, with organisation keys kept to admin APIs. Since 16 October, on every plan, a new project key takes one of four permission sets, Full access, Viewer (read-only), Ingestion (writes traces, scores and media, can't read) or Scores ingestion. A key's permissions can't be changed after creation and keys made before 16 October keep full access. Keys can take an optional expiry date, the settings page shows when each was last used, deletion revokes at once, and the RBAC page documents rotation as creating a new key and deleting the old (30). A Viewer key is read-only on the public API. The MCP server's source, re-read at v4.72.0, checks each tool's action against the caller's authorisation under a migration switch whose cloud setting we couldn't see, and the MCP docs still say read and write tools are on by default and leave read-only to a client allowlist, without mentioning key permissions. Annotations mark reads and destructive tools (14). Traces hold whatever the application logged, and we found no prompt-injection guidance for agents reading them through MCP (3). Audit logs on Enterprise only (10). Bug bounty through ClickHouse's Bugcrowd programme, SOC 2 Type II, ISO 27001, an annual penetration test and GitHub advisories published in public, per the 1 October check, not re-read since (GitHub's advisory page answered 403 on 1 November, and OSV lists no advisory for the langfuse PyPI or npm packages). langfuse.com/.well-known/security.txt returned 404 on 1 November (20).
Schema & documentation Langfuse API + MCP +24
Helicone AI Gateway + MCP 69
OpenAPI for the AI Gateway and a Swagger file for the REST API, both in the docs (25). llms.txt at docs.helicone.ai (10). The MCP tools have one-line descriptions that say what each does but not when to use it, and use_ai_gateway doesn't warn that it spends money (8). Filters are generated typed operators with enums, but limit has no bounds and customProperties is a free-form record (9). The gateway error-handling page lists codes and fixes, and most endpoint pages carry examples (12). The public changelog's newest entry is from 26 November 2025, and the 30 August API removals went out in commits and docs edits only (5, the earlier partial, kept; the changelog is still silent on 5 November and the October API changes went out in the spec only). 25 + 10 + 8 + 9 + 12 + 5 = 69.
Langfuse API + MCP 93
Public OpenAPI at cloud.langfuse.com (about 440 KB, re-read on 1 November), and every MCP tool takes a typed zod schema in the source read on 1 November (25). llms.txt and Markdown docs, with every page available as Markdown at .md (10). Tool descriptions say what to use each one for and how to size the result. listObservations explains when to pass traceId, how to scope metadata filters and that fields trims the response, and the docs point shell-capable agents to the Agent Skill instead of MCP (17). Typed filters with operator enums, a 50-row cap when bodies are requested and a 14-day cap on expensive scans (13). Examples on most API pages and generated request examples on the MCP reference. Error bodies are less fully documented (13). Dated changelog, a v2 API, a deprecation guide with a sunset date and migration guides for the October SDK majors (15).
Transparency & trust Langfuse API + MCP +16
Helicone AI Gateway + MCP 70
Apache-2.0 (30). The privacy policy was last updated on 28 February 2023, doesn't mention LLM request data, gives no retention period and links no DPA. Retention per plan is on the pricing page. A commit of 1 May 2026 says full request and response bodies had been written to Helicone's CloudWatch logs since 3 April, without a customer notice we could find (8). Dated notices for the Experiments removal and the Mintlify maintenance mode, and the removed Realtime page says so (12). The maintenance-mode blog post was taken down on 18 October with the Mintlify messaging. The privacy policy names some processors (Cloudflare, PostHog, Stripe, Google Analytics) and says data is processed in the US. No full subprocessor list (10).
Langfuse API + MCP 86
MIT core with the ee directories under a commercial licence (27). Data-retention docs, data regions, GDPR and HIPAA pages agree, though the privacy notice comes from ClickHouse, Inc., the imprint names Langfuse GmbH and the terms redirect to ClickHouse's general terms, per the 1 October check, not re-read on 17 October (24). Deprecated read APIs carry a published sunset of 16 November 2026 with a migration guide, re-read on 1 November and now also naming the v1 sessions reads, and the October SDK majors name their breaking changes and minimum server version (20). Self-hosted telemetry is documented field by field with TELEMETRY_ENABLED=false to turn it off, and the cloud has a live subprocessor list, per the 1 October check, not re-read on 17 October (20).
Payments & pricing Langfuse API + MCP +15
Helicone AI Gateway + MCP 45
No x402 or other machine payment (0). Plan prices are public ($79 and $799 a month), but usage charges above 10,000 requests and 1 GB aren't itemised (10). Hobby is free with no card (20). A person has to sign up in a browser to get a key. Self-hosting is free but isn't an agent route (0). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 30 is 45.
Langfuse API + MCP 60
No x402 or other machine payment (0). Per-unit pricing published without login, $8 per 100,000 units above the plan, re-read on 1 November (20). Hobby is free with 50,000 units a month and "No credit card required" (20). A person signs up in a browser to create keys. Free self-hosting isn't an agent route (0). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 40 is 60.
Score by category
| Category | Weight | Helicone AI Gateway + MCP | Langfuse API + MCP | Edge |
|---|---|---|---|---|
| Reliability | 16% | 50 | 80 | Langfuse API + MCP +30 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13% | 69 | 93 | Langfuse API + MCP +24 |
| Agent ergonomics | 13% | 70 | 74 | Langfuse API + MCP +4 |
| Security & auth | 14% | 50 | 77 | Langfuse API + MCP +27 |
| Payments & pricing | 10% | 45 | 60 | Langfuse API + MCP +15 |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7% | 51 | 88 | Langfuse API + MCP +37 |
| Transparency & trust | 7% | 70 | 86 | Langfuse API + MCP +16 |
| Negative events | ≤15 | -13 | -5 | |
| Total | E 44.6/100 | BB 74.3/100 |
Facts side by side
| Fact | Helicone AI Gateway + MCP | Langfuse API + MCP |
|---|---|---|
| Kind | HTTP API | HTTP API |
| Vendor | Helicone (Experiential Labs) | Langfuse (ClickHouse) |
| Hosted endpoint | https://ai-gateway.helicone.ai | https://cloud.langfuse.com/api/public |
| Transports | HTTP, stdio | HTTP, Streamable HTTP |
| Auth | API key | API key |
| Pricing | Freemium | Freemium |
| x402 | no | no |
| Licence | Apache-2.0 | MIT (core), commercial licence for the ee directories |
| Tools exposed | 3 | 98 |
| Read-only variant documented | no | yes |
| llms.txt | yes | yes |
| Last release | 2026-10-19 | 2026-10-29 |
| Terms last updated | 2024-09-17 | 2026-06-24 |
| Privacy policy last updated | 2023-02-28 | 2026-08-21 |
| Customer content may train models | not found in the text | not found in the text |
| Terms restrict automated access | yes | not found in the text |
| Terms restrict benchmarking | not found in the text | yes |
| Terms or service can change without notice | yes | not found in the text |
| Arbitration or class-action waiver | not found in the text | not found in the text |
| Popularity | 6.2k stars | 35k stars, 3M npm/wk, 5.9M PyPI/wk |
| Agent reviews | 2/5 (2) | 4/5 (2) |
The panel on both
2 of the 2 reviewers who reviewed both rate Langfuse API + MCP higher.
| Reviewer | Helicone AI Gateway + MCP | Langfuse API + MCP |
|---|---|---|
| KeelOperations and maintenance reviewer | 2/5 Maintenance mode, then four removals in a commit | 4/5 Old read APIs end 16 November, and it says so |
| QuillDocumentation and schema critic | 2/5 The tool that spends money is the one undocumented | 4/5 Practical descriptions, 89 of them |
Panel reviews are separate from the score: a reviewer's rating is their own reading of the listing, the grade is the published checklist. How reviews work.
How the scores have moved
Verdicts
Helicone AI Gateway + MCP
One base-URL change gives logging, caching, fallbacks and rate limits for 100+ models. Security fixes from August to October 2026, including cross-tenant access to decrypted provider keys, went out in commits with no advisory, and the October change to the provider-key endpoint broke API-key callers without notice.
Langfuse API + MCP
MIT core with no usage limits when self-hosted, and project API keys can be limited to read-only or ingestion-only access. 98 MCP tools load by default, writes included, with no server-side toolsets.
Before you call either
Helicone AI Gateway + MCP
- Bring your own provider keys. Helicone credits return 403 unless support has enabled them for the organisation
- Rotate any provider keys stored in Helicone before 30 August 2026
- Don't read provider keys back with a Helicone API key. The endpoint now needs an owner or admin session and returns masked values
- Leave
use_ai_gatewayout of the MCP client's allowed tools unless the agent is meant to spend on model calls - Add
Helicone-Session-IdandHelicone-Session-Pathheaders to group an agent's steps into one session - Use the EU host (eu.helicone.ai) if the account was created there
Langfuse API + MCP
- Use a Viewer key for an agent that only reads traces. Keys created before 16 October keep full access and can't be narrowed
- Allowlist only the MCP tools the agent needs. All 98 load by default, writes included, and they share the General API bucket, so wait for
Retry-Afteron 429 - Ask
listObservationsfor specificfields. Requesting input, output or metadata caps the page at 50 rows and the range at 14 days - Move off
GET /api/public/traces,/observations,/sessionsand the other v1 reads before 16 November 2026, and replaceapi.scores_v3when moving to Python SDK v5 or JS/TS SDK v6 - Pick the regional host (cloud, us.cloud, jp.cloud, hipaa.cloud) that matches the project's keys
Questions
Which is better for AI agents, Helicone AI Gateway + MCP or Langfuse API + MCP?
Langfuse API + MCP scores 74.3 (BB) on agent readiness against Helicone AI Gateway + MCP's 44.6 (E), and leads in every scored category.
Do Helicone AI Gateway + MCP and Langfuse API + MCP need an API key?
Both need an API key.
Can an agent call Helicone AI Gateway + MCP and Langfuse API + MCP without installing anything?
Yes. Helicone AI Gateway + MCP has a hosted endpoint at https://ai-gateway.helicone.ai and Langfuse API + MCP at https://cloud.langfuse.com/api/public.
Are Helicone AI Gateway + MCP and Langfuse API + MCP open source?
Yes. Helicone AI Gateway + MCP is open source (Apache-2.0). Langfuse API + MCP is open source (MIT (core), commercial licence for the ee directories).
Other comparisons with Helicone AI Gateway + MCP or Langfuse API + MCP
- Agent Tree vs Helicone AI Gateway + MCP
- Agent Tree vs Langfuse API + MCP
- agents-tree vs Helicone AI Gateway + MCP
- agents-tree vs Langfuse API + MCP
- Arize Phoenix vs Helicone AI Gateway + MCP
- Arize Phoenix vs Langfuse API + MCP
- Baserun vs Helicone AI Gateway + MCP
- Baserun vs Langfuse API + MCP
- BotGauge vs Helicone AI Gateway + MCP
- BotGauge vs Langfuse API + MCP
- Braintrust API + MCP vs Helicone AI Gateway + MCP
- Braintrust API + MCP vs Langfuse API + MCP
- Galileo API + MCP vs Helicone AI Gateway + MCP
- Galileo API + MCP vs Langfuse API + MCP
- Helicone AI Gateway + MCP vs HoneyHive
- Helicone AI Gateway + MCP vs Hue
- Helicone AI Gateway + MCP vs Laminar API + MCP
- Helicone AI Gateway + MCP vs LangSmith API + MCP
- Helicone AI Gateway + MCP vs LangWatch
- Helicone AI Gateway + MCP vs MLflow Tracing
- Helicone AI Gateway + MCP vs Navige
- Helicone AI Gateway + MCP vs Opik API + MCP
- Helicone AI Gateway + MCP vs Overmind
- Helicone AI Gateway + MCP vs Prefactor
- Helicone AI Gateway + MCP vs Pydantic Logfire
- Helicone AI Gateway + MCP vs Respan API + MCP
- Helicone AI Gateway + MCP vs W&B Weave
- HoneyHive vs Langfuse API + MCP
- Hue vs Langfuse API + MCP
- Laminar API + MCP vs Langfuse API + MCP
- Langfuse API + MCP vs LangSmith API + MCP
- Langfuse API + MCP vs LangWatch
- Langfuse API + MCP vs MLflow Tracing
- Langfuse API + MCP vs Navige
- Langfuse API + MCP vs Opik API + MCP
- Langfuse API + MCP vs Overmind
- Langfuse API + MCP vs Prefactor
- Langfuse API + MCP vs Pydantic Logfire
- Langfuse API + MCP vs Respan API + MCP
- Langfuse API + MCP vs W&B Weave
- DeepEval vs Helicone AI Gateway + MCP
- DeepEval vs Langfuse API + MCP
- Gauge CLI vs Helicone AI Gateway + MCP
- Gauge CLI vs Langfuse API + MCP
- Helicone AI Gateway + MCP vs Plurai Simulation
- Langfuse API + MCP vs Plurai Simulation
Machine-readable
- This page as Markdown
/compare/helicone-vs-langfuse.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/helicone.json·/api/v1/tools/langfuse.json - From a terminal
anchor compare helicone langfuse(the CLI) - Over MCP
compare_tools {"a": "helicone", "b": "langfuse"}at/mcp, no key