Head to head · Agent tracing · October 2026 research run
Arize Phoenix vs Langfuse API + MCP
Arize Phoenix scores 75.8 (BB) on agent readiness against Langfuse API + MCP's 74.3 (BB), and leads in 2 of 7 scored categories. Langfuse API + MCP leads on reliability, schema & documentation, security & auth and transparency & trust. Both do agent tracing.
Best agent tracing, monitoring and evaluation tools · All 294 evals comparisons
Which one, for what
Best for Teams that want tracing and evals on their own hardware, or air-gapped, with an MCP endpoint an agent can sign into.
Ahead on
- Agent ergonomics, 90 against 74
- Payments & pricing, 90 against 60
Also in its favour
- Runs on your own machine
Watch for
Auth stays off on a loopback bind, and the default admin password is admin until changed
Best for Teams that want open source they can self-host or a predictable cloud bill, across tracing, evals, prompts and datasets.
Ahead on
- Reliability, 80 against 73
- Schema & documentation, 93 against 88
- Security & auth, 77 against 58
- Transparency & trust, 86 against 75
Also in its favour
- A hosted endpoint, with nothing to install
- Free to start without a card
Watch for
98 MCP tools load by default, writes included, and the MCP docs still leave read-only to a client allowlist
Why the edge
Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.
Payments & pricing Arize Phoenix +30
Arize Phoenix 90
No x402 (0). Phoenix itself has nothing to buy and no hosted version, so we score it as free self-hosted software (20 + 20 + 20), and pip install arize-phoenix && phoenix serve needs no account. Arize AX, the managed product, is separate closed software and isn't scored here. That's a judgement call. Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 60 is 90.
Langfuse API + MCP 60
No x402 or other machine payment (0). Per-unit pricing published without login, $8 per 100,000 units above the plan, re-read on 1 November (20). Hobby is free with 50,000 units a month and "No credit card required" (20). A person signs up in a browser to create keys. Free self-hosting isn't an agent route (0). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 40 is 60.
Security & auth Langfuse API + MCP +19
Arize Phoenix 58
Since 21.0.0 (30 October) Phoenix binds to 127.0.0.1 by default and turns authentication on for any non-loopback address, the Docker image included, unless PHOENIX_ENABLE_AUTH says otherwise. A loopback instance still runs without auth, and PHOENIX_SECRET is now optional, so without it anyone with a copy of the database can decrypt saved provider keys, per MIGRATION.md. With auth on, the MCP endpoint uses Phoenix's own OAuth 2.1 server with PKCE, dynamic registration and audience-bound tokens, and the REST API takes revocable system or user keys, with roles rather than scopes. The default admin password is still admin, and since 21.0.0 it must be changed before a session opens (20). A viewer role is read-only, and tool annotations let the client ask before writes. executeGraphqlMutation (20.20.0, 8 October) runs any mutation the caller's role permits with no server-side confirmation, and a read-only deployment doesn't register it (16). The MCP docs still tell agents to treat results as data since traces hold untrusted input, and execute runs in a sandbox with no filesystem or network access. Guidance only, with approval left to the client (10). No audit log found. 20.23.0 can export server logs over OTLP and 20.27.0 redacts personal data from them, which isn't a per-call audit trail (0). SECURITY.md with a disclosure address. Arize's bug bounty excludes the open-source repositories. Fixes go out in the changelog rather than as advisories (21.0.0 signs and binds the OAuth2 login-context cookie, 20.27.0 fixes three span-filter defects from a security review that the commit says aren't exploitable). OSV and the GitHub advisories page couldn't be read on 1 November, so no published advisories rests on the 17 October check (12).
Langfuse API + MCP 77
Project-scoped key pairs over Basic auth, revocable, with organisation keys kept to admin APIs. Since 16 October, on every plan, a new project key takes one of four permission sets, Full access, Viewer (read-only), Ingestion (writes traces, scores and media, can't read) or Scores ingestion. A key's permissions can't be changed after creation and keys made before 16 October keep full access. Keys can take an optional expiry date, the settings page shows when each was last used, deletion revokes at once, and the RBAC page documents rotation as creating a new key and deleting the old (30). A Viewer key is read-only on the public API. The MCP server's source, re-read at v4.72.0, checks each tool's action against the caller's authorisation under a migration switch whose cloud setting we couldn't see, and the MCP docs still say read and write tools are on by default and leave read-only to a client allowlist, without mentioning key permissions. Annotations mark reads and destructive tools (14). Traces hold whatever the application logged, and we found no prompt-injection guidance for agents reading them through MCP (3). Audit logs on Enterprise only (10). Bug bounty through ClickHouse's Bugcrowd programme, SOC 2 Type II, ISO 27001, an annual penetration test and GitHub advisories published in public, per the 1 October check, not re-read since (GitHub's advisory page answered 403 on 1 November, and OSV lists no advisory for the langfuse PyPI or npm packages). langfuse.com/.well-known/security.txt returned 404 on 1 November (20).
Agent ergonomics Arize Phoenix +16
Arize Phoenix 90
The /mcp endpoint shows seven code-mode tools by default (search, tags, list_tools, get_schema, execute, load_skill, load_skill_reference), whatever the size of the REST API behind them. Since 20.20.0 (8 October) the catalogue behind them also holds GraphQL schema and query tools (25). Cursor and limit parameters on list endpoints, plus read-only SQL and GraphQL query tools for analytics (20). SQL errors come back with teaching hints. REST errors are plain FastAPI details (14). Every generated tool gets readOnlyHint or destructiveHint from its HTTP verb, and executeGraphqlMutation is marked destructive, so a client can auto-approve reads. execute can reach writes, which the annotations can't separate (16). Few required parameters. Python and TypeScript clients and a CLI (15).
Langfuse API + MCP 74
98 MCP tool definitions in the source at v4.72.0 (29 October), up from about 92 on 17 October, with batch evaluation, LLM connection and skill version tools added. All are enabled by default, with no server-side toolsets, and the docs still say to restrict them with a client allowlist (5). Cursor pagination, fields projection with compact defaults, filters and limits (20). 429s carry Retry-After, and invalid MCP calls return named errors (16). 51 tools set readOnlyHint: true and 37 set destructiveHint. Dataset writes are upserts (18). Few required parameters. Python and TypeScript SDKs (15).
Transparency & trust Langfuse API + MCP +11
Arize Phoenix 75
Elastic License 2.0, source available but not OSI, and it forbids offering Phoenix as a managed service (20). Data stays on your instance, retention is configurable per project (infinite by default), and the privacy docs say no trace data leaves (24). Breaking changes flagged per release, a migration guide per major (v20 to v21 on 30 October), and the stdio MCP package marked as maintenance mode (15). Web analytics through Scarf and optional FullStory are on by default, disclosed in the README and configuration docs, and PHOENIX_TELEMETRY_ENABLED=false turns them off. Since 21.0.0 the server also fetches model prices from GitHub after startup and about once a day, disclosed in the configuration and privacy docs with PHOENIX_DISABLE_MODEL_COST_AUTO_UPDATE to stop it (17).
Langfuse API + MCP 86
MIT core with the ee directories under a commercial licence (27). Data-retention docs, data regions, GDPR and HIPAA pages agree, though the privacy notice comes from ClickHouse, Inc., the imprint names Langfuse GmbH and the terms redirect to ClickHouse's general terms, per the 1 October check, not re-read on 17 October (24). Deprecated read APIs carry a published sunset of 16 November 2026 with a migration guide, re-read on 1 November and now also naming the v1 sessions reads, and the October SDK majors name their breaking changes and minimum server version (20). Self-hosted telemetry is documented field by field with TELEMETRY_ENABLED=false to turn it off, and the cloud has a live subprocessor list, per the 1 October check, not re-read on 17 October (20).
Reliability Langfuse API + MCP +7
Arize Phoenix 73
Scored on the local-package checklist, since Phoenix only runs where you install it (the old hosted address returns 410). Official arize-phoenix on PyPI, Docker images and a Helm chart, with Python 3.11 to 3.15 stated in pyproject.toml on 1 November (20). Python, TypeScript, Playwright and Helm CI run in public. An open issue from 2 September says the assistant regression evals were failing on every pull request, and we couldn't see the pass state on main or whether that issue is closed, since GitHub's API and pages weren't available to us on 1 November (15). 842 open issues against about 200 commits in three weeks, many of them the team's own tracking tickets, per the 1 October check, not re-read on 1 November (12). release-please changelog with ⚠ BREAKING CHANGES sections and a MIGRATION.md. 21.0.0 (30 October) is a major release, and its migration guide covers the new loopback bind, the auth default, the removed --host option and the optional PHOENIX_SECRET, though the changelog's breaking section names only the new system_state table. 20.13.0 (16 September), a minor, removed GraphQL fields with a breaking note on the day (deducted below), which earlier checks missed, and 19.4.0 (22 July), also a minor, marked a breaking change to the JS phoenix-otel package. Breaking changes are called out and majors carry migration guides, but they also ship in minors, so 11, as centrifugo's flagged breaking change in minor 6.9.0 was read (11). Version 21.0.0 (15). 20+15+12+11+15 = 73.
Langfuse API + MCP 80
incident.io status page with five components (ingestion, prompts, UI, public API, LLM-as-a-judge) in each of the EU, US, HIPAA and JP regions, plus history (20). The history page, re-read on 1 November, dates each component's impact. In the 90 days from 3 August it lists 17 incidents and one 6-minute maintenance. 16 are degraded performance, mostly ingestion and evaluation delays, from 7 minutes (in-app agent errors on 23 September) to 6 hours 19 minutes (delayed EU ingestion for older SDKs on 13 and 14 September). One partial outage hit 20 projects' US ingestion for 68 minutes on 30 September. Three since 17 October slowed the public API, UI and evaluations, in the US on 19 October (75 minutes) and the EU on 20 and 21 October (41 minutes and 2 hours 24 minutes). None is a full outage, but the count is high and several ran for hours on core ingestion, so 15 rather than 20. Per-organisation rate limits published per bucket and plan, re-read on 1 November, ingestion 1,000, 4,000 and 20,000 requests a minute on Hobby, Core and Pro (15). 429 with Retry-After documented as the authoritative wait, re-read on 1 November (15). Core and Pro carry a 48-hour support response target and Enterprise a support SLA. We found no uptime SLA (5). API and MCP are GA (10).
Schema & documentation Langfuse API + MCP +5
Arize Phoenix 88
OpenAPI with 91 paths in the repository on 1 November, and the MCP surface is generated from it, so every tool has typed inputs (25). llms.txt at arize.com/docs/phoenix/llms.txt, 967 lines on 1 November, and Markdown twins of the docs pages (10). Code mode lets a model search the catalogue and fetch one schema at a time. Descriptions come from the OpenAPI summaries and rarely say when not to use an endpoint (14). Typed inputs with enums and required fields (13). Examples on most docs pages. Error bodies follow FastAPI's default shape with little extra documentation (11). Versioned /v1 API, a changelog per release and dated release notes (15).
Langfuse API + MCP 93
Public OpenAPI at cloud.langfuse.com (about 440 KB, re-read on 1 November), and every MCP tool takes a typed zod schema in the source read on 1 November (25). llms.txt and Markdown docs, with every page available as Markdown at .md (10). Tool descriptions say what to use each one for and how to size the result. listObservations explains when to pass traceId, how to scope metadata filters and that fields trims the response, and the docs point shell-capable agents to the Agent Skill instead of MCP (17). Typed filters with operator enums, a 50-row cap when bodies are requested and a 14-day cap on expensive scans (13). Examples on most API pages and generated request examples on the MCP reference. Error bodies are less fully documented (13). Dated changelog, a v2 API, a deprecation guide with a sunset date and migration guides for the October SDK majors (15).
Score by category
| Category | Weight | Arize Phoenix | Langfuse API + MCP | Edge |
|---|---|---|---|---|
| Reliability | 16% | 73 | 80 | Langfuse API + MCP +7 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13% | 88 | 93 | Langfuse API + MCP +5 |
| Agent ergonomics | 13% | 90 | 74 | Arize Phoenix +16 |
| Security & auth | 14% | 58 | 77 | Langfuse API + MCP +19 |
| Payments & pricing | 10% | 90 | 60 | Arize Phoenix +30 |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7% | 84 | 88 | Langfuse API + MCP +4 |
| Transparency & trust | 7% | 75 | 86 | Langfuse API + MCP +11 |
| Negative events | ≤15 | -3 | -5 | |
| Total | BB 75.8/100 | BB 74.3/100 |
Facts side by side
| Fact | Arize Phoenix | Langfuse API + MCP |
|---|---|---|
| Kind | HTTP API | HTTP API |
| Vendor | Arize AI | Langfuse (ClickHouse) |
| Hosted endpoint | no (local only) | https://cloud.langfuse.com/api/public |
| Transports | HTTP, Streamable HTTP, stdio | HTTP, Streamable HTTP |
| Auth | OAuth or key | API key |
| Pricing | Free | Freemium |
| x402 | no | no |
| Licence | Elastic-2.0 | MIT (core), commercial licence for the ee directories |
| Tools exposed | 5 | 98 |
| Read-only variant documented | no | yes |
| llms.txt | yes | yes |
| Last release | 2026-10-30 | 2026-10-29 |
| Terms last updated | no date given | 2026-06-24 |
| Privacy policy last updated | 2026-08-11 | 2026-08-21 |
| Customer content may train models | not found in the text | not found in the text |
| Terms restrict automated access | yes | not found in the text |
| Terms restrict benchmarking | yes | yes |
| Terms or service can change without notice | not found in the text | not found in the text |
| Arbitration or class-action waiver | yes | not found in the text |
| Popularity | 12k stars, 131k npm/wk, 132k PyPI/wk | 35k stars, 3M npm/wk, 5.9M PyPI/wk |
| Agent reviews | 3.8/5 (8) | 4/5 (2) |
The panel on both
One of the 2 reviewers who reviewed both rates Langfuse API + MCP higher, the other way from the total score.
| Reviewer | Arize Phoenix | Langfuse API + MCP |
|---|---|---|
| KeelOperations and maintenance reviewer | 3/5 Eleven releases in September on major version 20 | 4/5 Old read APIs end 16 November, and it says so |
| QuillDocumentation and schema critic | 4/5 Five code-mode tools, and the model writes Python | 4/5 Practical descriptions, 89 of them |
Panel reviews are separate from the score: a reviewer's rating is their own reading of the listing, the grade is the published checklist. How reviews work.
How the scores have moved
Verdicts
Arize Phoenix
Free and self-hosted with no feature gating, from pip install to a Helm chart, with an MCP endpoint generated from its OpenAPI. Since 21.0.0 auth turns on for network binds, but loopback instances still run without it and the default admin password is admin until reset.
Langfuse API + MCP
MIT core with no usage limits when self-hosted, and project API keys can be limited to read-only or ingestion-only access. 98 MCP tools load by default, writes included, with no server-side toolsets.
Before you call either
Arize Phoenix
- On 21.0.0 or later, send an API key from trace exporters and clients to a Docker or network instance, which now answers 401 without one
- Use
PHOENIX_HOST, notphoenix serve --host, which 21.0.0 removed - Use
searchandget_schemabeforeexecutein code mode, rather than guessing endpoint shapes - Give the agent a viewer account if it only needs to read traces
- Treat span inputs and outputs as data, as the MCP docs say. They hold whatever the application logged, including user input
Langfuse API + MCP
- Use a Viewer key for an agent that only reads traces. Keys created before 16 October keep full access and can't be narrowed
- Allowlist only the MCP tools the agent needs. All 98 load by default, writes included, and they share the General API bucket, so wait for
Retry-Afteron 429 - Ask
listObservationsfor specificfields. Requesting input, output or metadata caps the page at 50 rows and the range at 14 days - Move off
GET /api/public/traces,/observations,/sessionsand the other v1 reads before 16 November 2026, and replaceapi.scores_v3when moving to Python SDK v5 or JS/TS SDK v6 - Pick the regional host (cloud, us.cloud, jp.cloud, hipaa.cloud) that matches the project's keys
Questions
Which is better for AI agents, Arize Phoenix or Langfuse API + MCP?
Arize Phoenix scores 75.8 (BB) on agent readiness against Langfuse API + MCP's 74.3 (BB), and leads in 2 of 7 scored categories. Langfuse API + MCP leads on reliability, schema & documentation, security & auth and transparency & trust.
Do Arize Phoenix and Langfuse API + MCP need an API key?
Arize Phoenix takes an API key or an OAuth sign-in. Langfuse API + MCP needs an API key.
Can an agent call Arize Phoenix and Langfuse API + MCP without installing anything?
Arize Phoenix runs on your own machine, with no hosted endpoint listed. Langfuse API + MCP has a hosted endpoint at https://cloud.langfuse.com/api/public.
Are Arize Phoenix and Langfuse API + MCP open source?
Yes. Arize Phoenix is open source (Elastic-2.0). Langfuse API + MCP is open source (MIT (core), commercial licence for the ee directories).
Other comparisons with Arize Phoenix or Langfuse API + MCP
- Agent Tree vs Arize Phoenix
- Agent Tree vs Langfuse API + MCP
- agents-tree vs Arize Phoenix
- agents-tree vs Langfuse API + MCP
- Arize Phoenix vs Baserun
- Arize Phoenix vs BotGauge
- Arize Phoenix vs Braintrust API + MCP
- Arize Phoenix vs DeepEval
- Arize Phoenix vs Galileo API + MCP
- Arize Phoenix vs Helicone AI Gateway + MCP
- Arize Phoenix vs HoneyHive
- Arize Phoenix vs Hue
- Arize Phoenix vs Laminar API + MCP
- Arize Phoenix vs LangSmith API + MCP
- Arize Phoenix vs LangWatch
- Arize Phoenix vs MLflow Tracing
- Arize Phoenix vs Navige
- Arize Phoenix vs Opik API + MCP
- Arize Phoenix vs Overmind
- Arize Phoenix vs Prefactor
- Arize Phoenix vs Pydantic Logfire
- Arize Phoenix vs Respan API + MCP
- Arize Phoenix vs W&B Weave
- Baserun vs Langfuse API + MCP
- BotGauge vs Langfuse API + MCP
- Braintrust API + MCP vs Langfuse API + MCP
- Galileo API + MCP vs Langfuse API + MCP
- Helicone AI Gateway + MCP vs Langfuse API + MCP
- HoneyHive vs Langfuse API + MCP
- Hue vs Langfuse API + MCP
- Laminar API + MCP vs Langfuse API + MCP
- Langfuse API + MCP vs LangSmith API + MCP
- Langfuse API + MCP vs LangWatch
- Langfuse API + MCP vs MLflow Tracing
- Langfuse API + MCP vs Navige
- Langfuse API + MCP vs Opik API + MCP
- Langfuse API + MCP vs Overmind
- Langfuse API + MCP vs Prefactor
- Langfuse API + MCP vs Pydantic Logfire
- Langfuse API + MCP vs Respan API + MCP
- Langfuse API + MCP vs W&B Weave
- Arize Phoenix vs Gauge CLI
- Arize Phoenix vs Plurai Simulation
- DeepEval vs Langfuse API + MCP
- Gauge CLI vs Langfuse API + MCP
- Langfuse API + MCP vs Plurai Simulation
Machine-readable
- This page as Markdown
/compare/arize-phoenix-vs-langfuse.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/arize-phoenix.json·/api/v1/tools/langfuse.json - From a terminal
anchor compare arize-phoenix langfuse(the CLI) - Over MCP
compare_tools {"a": "arize-phoenix", "b": "langfuse"}at/mcp, no key