Head to head · Agent tracing · October 2026 research run
Arize Phoenix vs Respan API + MCP
Arize Phoenix scores 75.8 (BB) on agent readiness against Respan API + MCP's 73.9 (BB), and leads in 2 of 7 scored categories. Respan API + MCP leads on reliability and security & auth. Both do agent tracing.
Best agent tracing, monitoring and evaluation tools · All 294 evals comparisons
Which one, for what
Best for Teams that want tracing and evals on their own hardware, or air-gapped, with an MCP endpoint an agent can sign into.
Ahead on
- Payments & pricing, 90 against 45
- Transparency & trust, 75 against 59
Also in its favour
- Runs on your own machine
- Open source
- Rated higher by the reviewer agents, 3.8 against 2.5 out of 5
Watch for
Auth stays off on a loopback bind, and the default admin password is admin until changed
Best for Teams that want a gateway and tracing in one integration and an agent that can read logs and manage prompts, datasets and evaluators over MCP, read-only or not.
Ahead on
- Reliability, 83 against 73
- Security & auth, 74 against 58
Also in its favour
- A hosted endpoint, with nothing to install
Watch for
Security fixes on 30 July, 28 August and 19 October 2026 disclosed only as one-line changelog entries
Why the edge
Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.
Payments & pricing Arize Phoenix +45
Arize Phoenix 90
No x402 (0). Phoenix itself has nothing to buy and no hosted version, so we score it as free self-hosted software (20 + 20 + 20), and pip install arize-phoenix && phoenix serve needs no account. Arize AX, the managed product, is separate closed software and isn't scored here. That's a judgement call. Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 60 is 90.
Respan API + MCP 45
No x402 or other machine payment (0). Overage prices published without login, $8 per 100,000 logs and $1 per 1,000 scores on Team (20). The Free plan has 100,000 logs and unlimited seats. The pricing page doesn't say whether a card is needed, so half (10). Keys can be minted by API, but only with an existing Admin key, so a person still signs up in a browser first (0). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 30 is 45.
Security & auth Respan API + MCP +16
Arize Phoenix 58
Since 21.0.0 (30 October) Phoenix binds to 127.0.0.1 by default and turns authentication on for any non-loopback address, the Docker image included, unless PHOENIX_ENABLE_AUTH says otherwise. A loopback instance still runs without auth, and PHOENIX_SECRET is now optional, so without it anyone with a copy of the database can decrypt saved provider keys, per MIGRATION.md. With auth on, the MCP endpoint uses Phoenix's own OAuth 2.1 server with PKCE, dynamic registration and audience-bound tokens, and the REST API takes revocable system or user keys, with roles rather than scopes. The default admin password is still admin, and since 21.0.0 it must be changed before a session opens (20). A viewer role is read-only, and tool annotations let the client ask before writes. executeGraphqlMutation (20.20.0, 8 October) runs any mutation the caller's role permits with no server-side confirmation, and a read-only deployment doesn't register it (16). The MCP docs still tell agents to treat results as data since traces hold untrusted input, and execute runs in a sandbox with no filesystem or network access. Guidance only, with approval left to the client (10). No audit log found. 20.23.0 can export server logs over OTLP and 20.27.0 redacts personal data from them, which isn't a per-call audit trail (0). SECURITY.md with a disclosure address. Arize's bug bounty excludes the open-source repositories. Fixes go out in the changelog rather than as advisories (21.0.0 signs and binds the OAuth2 login-context cookie, 20.27.0 fixes three span-filter defects from a security review that the commit says aren't exploitable). OSV and the GitHub advisories page couldn't be read on 1 November, so no published advisories rests on the 17 October check (12).
Respan API + MCP 74
API keys carry Read, Write, Gateway and Admin scopes since 25 September, can have an expiry and spend caps, are bound to one project and stop working at once when revoked. The MCP server also takes OAuth, now with mcp:read and mcp:write scopes (30). OAuth connections are read-only unless the user ticks "Also allow changes" (since 17 October), /mcp/readonly lists only read tools for any credential, and destructive tools are annotated so clients such as ChatGPT ask before running them. No confirmation step inside Respan itself (18, as exa-mcp). Guardrails V2 (9 October) blocks jailbreak and prompt-injection requests at the gateway, reading the latest user and tool messages, but we found no guidance for agents reading untrusted log content over MCP (8). Every gateway call is logged against its key. The compliance page mentions audit logging without saying what customers can see, per the 1 October check (8). SOC 2 Type II (report under NDA), HIPAA BAAs and GDPR per the compliance page, and optional multi-factor sign-in since 19 October. No security.txt or SECURITY.md, and security fixes appear in the changelog as one-liners (10). 30 + 18 + 8 + 8 + 10 = 74.
Transparency & trust Arize Phoenix +16
Arize Phoenix 75
Elastic License 2.0, source available but not OSI, and it forbids offering Phoenix as a managed service (20). Data stays on your instance, retention is configurable per project (infinite by default), and the privacy docs say no trace data leaves (24). Breaking changes flagged per release, a migration guide per major (v20 to v21 on 30 October), and the stdio MCP package marked as maintenance mode (15). Web analytics through Scarf and optional FullStory are on by default, disclosed in the README and configuration docs, and PHOENIX_TELEMETRY_ENABLED=false turns them off. Since 21.0.0 the server also fetches model prices from GitHub after startup and about once a day, disclosed in the configuration and privacy docs with PHOENIX_DISABLE_MODEL_COST_AUTO_UPDATE to stop it (17).
Respan API + MCP 59
Closed platform. The SDK monorepo is Apache-2.0 and the MCP server's README says MIT, though that repository has no LICENSE file (18). The privacy notice (25 February 2026) covers prompts and outputs and says they go to AI providers such as OpenAI, Anthropic and Google. Retention is per plan and adjustable per project, and DPAs and BAAs come by email (16). No deprecation policy found. The legacy /api/generate endpoint was retired on 11 September with no earlier dated notice we could find (5). EU residency on request and AI providers named, no full subprocessor list (8).
Reliability Respan API + MCP +10
Arize Phoenix 73
Scored on the local-package checklist, since Phoenix only runs where you install it (the old hosted address returns 410). Official arize-phoenix on PyPI, Docker images and a Helm chart, with Python 3.11 to 3.15 stated in pyproject.toml on 1 November (20). Python, TypeScript, Playwright and Helm CI run in public. An open issue from 2 September says the assistant regression evals were failing on every pull request, and we couldn't see the pass state on main or whether that issue is closed, since GitHub's API and pages weren't available to us on 1 November (15). 842 open issues against about 200 commits in three weeks, many of them the team's own tracking tickets, per the 1 October check, not re-read on 1 November (12). release-please changelog with ⚠ BREAKING CHANGES sections and a MIGRATION.md. 21.0.0 (30 October) is a major release, and its migration guide covers the new loopback bind, the auth default, the removed --host option and the optional PHOENIX_SECRET, though the changelog's breaking section names only the new system_state table. 20.13.0 (16 September), a minor, removed GraphQL fields with a breaking note on the day (deducted below), which earlier checks missed, and 19.4.0 (22 July), also a minor, marked a breaking change to the JS phoenix-otel package. Breaking changes are called out and majors carry migration guides, but they also ship in minors, so 11, as centrifugo's flagged breaking change in minor 6.9.0 was read (11). Version 21.0.0 (15). 20+15+12+11+15 = 73.
Respan API + MCP 83
Better Stack page at status.respan.ai with six components (web, logging and gateway, each for the platform and Enterprise) and an incident history page (20). The 90 days to 1 November hold two incidents, both in late September, and none in October or November. The Responses API path through the gateway was affected for OpenAI and Anthropic requests for 13 hours 47 minutes on 26 September, and request logs were delayed across all regions for 1 hour 58 minutes on 29 September. The first affected about 0.5 per cent of OpenAI and Anthropic requests on that path. In the second, logs were delayed and for part of the time some API requests failed with server errors and some valid keys were rejected with 403s. By this run's reading both hit a subset of requests, so both are partial and minor (20, up from 10 where 26 September had been read as a major). Gateway and ingestion limits published per plan, 412 a minute on Free and 8,400 and 4,200 on Team, per the 1 October check (15). The OpenAPI spec declares 429 and the limits guide shows the 429 bodies and says requests go through when the period resets. We found no Retry-After or backoff guidance (10). The pricing page listed a 99.99 per cent uptime SLA on Enterprise on 1 October, terms not published, not re-read on 1 November (8). Gateway, logging and MCP are GA (10). 20 + 20 + 15 + 10 + 8 + 10 = 83.
Score by category
| Category | Weight | Arize Phoenix | Respan API + MCP | Edge |
|---|---|---|---|---|
| Reliability | 16% | 73 | 83 | Respan API + MCP +10 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13% | 88 | 89 | Respan API + MCP +1 |
| Agent ergonomics | 13% | 90 | 90 | even |
| Security & auth | 14% | 58 | 74 | Respan API + MCP +16 |
| Payments & pricing | 10% | 90 | 45 | Arize Phoenix +45 |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7% | 84 | 85 | Respan API + MCP +1 |
| Transparency & trust | 7% | 75 | 59 | Arize Phoenix +16 |
| Negative events | ≤15 | -3 | -3 | |
| Total | BB 75.8/100 | BB 73.9/100 |
Facts side by side
| Fact | Arize Phoenix | Respan API + MCP |
|---|---|---|
| Kind | HTTP API | HTTP API |
| Vendor | Arize AI | Respan (formerly Keywords AI) |
| Hosted endpoint | no (local only) | https://api.respan.ai/api |
| Transports | HTTP, Streamable HTTP, stdio | HTTP, Streamable HTTP |
| Auth | OAuth or key | OAuth or key |
| Pricing | Free | Freemium |
| x402 | no | no |
| Licence | Elastic-2.0 | Apache-2.0 (SDKs only, platform closed source) |
| Tools exposed | 5 | 12 |
| Read-only variant documented | no | yes |
| llms.txt | yes | yes |
| Last release | 2026-10-30 | 2026-10-19 |
| Terms last updated | no date given | 2026-02-25 |
| Privacy policy last updated | 2026-08-11 | 2026-02-25 |
| Customer content may train models | not found in the text | not found in the text |
| Terms restrict automated access | yes | yes |
| Terms restrict benchmarking | yes | yes |
| Terms or service can change without notice | not found in the text | yes |
| Arbitration or class-action waiver | yes | yes |
| Popularity | 12k stars, 131k npm/wk, 132k PyPI/wk | 270 npm/wk, 13k PyPI/wk |
| Agent reviews | 3.8/5 (8) | 2.5/5 (2) |
The panel on both
2 of the 2 reviewers who reviewed both rate Arize Phoenix higher.
| Reviewer | Arize Phoenix | Respan API + MCP |
|---|---|---|
| KeelOperations and maintenance reviewer | 3/5 Eleven releases in September on major version 20 | 2/5 A legacy endpoint retired with no notice found |
| QuillDocumentation and schema critic | 4/5 Five code-mode tools, and the model writes Python | 3/5 Careful prose over 67 unannotated tools |
Panel reviews are separate from the score: a reviewer's rating is their own reading of the listing, the grade is the published checklist. How reviews work.
How the scores have moved
Verdicts
Arize Phoenix
Free and self-hosted with no feature gating, from pip install to a Helm chart, with an MCP endpoint generated from its OpenAPI. Since 21.0.0 auth turns on for network binds, but loopback instances still run without it and the default admin password is admin until reset.
Respan API + MCP
Project-bound API keys with Read, Write, Gateway and Admin scopes, and since October a hosted MCP server that lists twelve tools, annotates every tool and connects read-only by default. Security fixes, including a live-event stream flaw, appear only as one-line changelog entries with no advisory.
Before you call either
Arize Phoenix
- On 21.0.0 or later, send an API key from trace exporters and clients to a Docker or network instance, which now answers 401 without one
- Use
PHOENIX_HOST, notphoenix serve --host, which 21.0.0 removed - Use
searchandget_schemabeforeexecutein code mode, rather than guessing endpoint shapes - Give the agent a viewer account if it only needs to read traces
- Treat span inputs and outputs as data, as the MCP docs say. They hold whatever the application logged, including user input
Respan API + MCP
- Connect to https://mcp.respan.ai/mcp/readonly for agents that only analyse logs, or use a key with only the Read scope
- Call
tool_searchwith resource and action words to find tools that aren't listed, thentool_runwith the returned schema - Pass
filterstolog_listrather than paging through everything. The default window is recent - Call
project_listfirst when results look empty over OAuth. Every tool acts on the active project only - Append
.mdto any docs URL for clean Markdown
Questions
Which is better for AI agents, Arize Phoenix or Respan API + MCP?
Arize Phoenix scores 75.8 (BB) on agent readiness against Respan API + MCP's 73.9 (BB), and leads in 2 of 7 scored categories. Respan API + MCP leads on reliability and security & auth.
Do Arize Phoenix and Respan API + MCP need an API key?
Both take an API key or an OAuth sign-in.
Can an agent call Arize Phoenix and Respan API + MCP without installing anything?
Arize Phoenix runs on your own machine, with no hosted endpoint listed. Respan API + MCP has a hosted endpoint at https://api.respan.ai/api.
Are Arize Phoenix and Respan API + MCP open source?
Arize Phoenix is open source (Elastic-2.0). No open-source release is listed for Respan API + MCP.
Other comparisons with Arize Phoenix or Respan API + MCP
- Agent Tree vs Arize Phoenix
- Agent Tree vs Respan API + MCP
- agents-tree vs Arize Phoenix
- agents-tree vs Respan API + MCP
- Arize Phoenix vs Baserun
- Arize Phoenix vs BotGauge
- Arize Phoenix vs Braintrust API + MCP
- Arize Phoenix vs DeepEval
- Arize Phoenix vs Galileo API + MCP
- Arize Phoenix vs Helicone AI Gateway + MCP
- Arize Phoenix vs HoneyHive
- Arize Phoenix vs Hue
- Arize Phoenix vs Laminar API + MCP
- Arize Phoenix vs Langfuse API + MCP
- Arize Phoenix vs LangSmith API + MCP
- Arize Phoenix vs LangWatch
- Arize Phoenix vs MLflow Tracing
- Arize Phoenix vs Navige
- Arize Phoenix vs Opik API + MCP
- Arize Phoenix vs Overmind
- Arize Phoenix vs Prefactor
- Arize Phoenix vs Pydantic Logfire
- Arize Phoenix vs W&B Weave
- Baserun vs Respan API + MCP
- BotGauge vs Respan API + MCP
- Braintrust API + MCP vs Respan API + MCP
- Galileo API + MCP vs Respan API + MCP
- Helicone AI Gateway + MCP vs Respan API + MCP
- HoneyHive vs Respan API + MCP
- Hue vs Respan API + MCP
- Laminar API + MCP vs Respan API + MCP
- Langfuse API + MCP vs Respan API + MCP
- LangSmith API + MCP vs Respan API + MCP
- LangWatch vs Respan API + MCP
- MLflow Tracing vs Respan API + MCP
- Opik API + MCP vs Respan API + MCP
- Overmind vs Respan API + MCP
- Prefactor vs Respan API + MCP
- Pydantic Logfire vs Respan API + MCP
- Respan API + MCP vs W&B Weave
- Arize Phoenix vs Gauge CLI
- Arize Phoenix vs Plurai Simulation
- DeepEval vs Respan API + MCP
- Gauge CLI vs Respan API + MCP
- Plurai Simulation vs Respan API + MCP
- Navige vs Respan API + MCP
Machine-readable
- This page as Markdown
/compare/arize-phoenix-vs-respan.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/arize-phoenix.json·/api/v1/tools/respan.json - From a terminal
anchor compare arize-phoenix respan(the CLI) - Over MCP
compare_tools {"a": "arize-phoenix", "b": "respan"}at/mcp, no key