Head to head · Agent tracing · October 2026 research run
Arize Phoenix vs Galileo API + MCP
Arize Phoenix scores 75.8 (BB) on agent readiness against Galileo API + MCP's 46.1 (D), and leads in every scored category. Both do agent tracing.
Best agent tracing, monitoring and evaluation tools · All 294 evals comparisons
Best guardrails and safety filters for AI agents · All 157 guardrails comparisons
Which one, for what
Best for Teams that want tracing and evals on their own hardware, or air-gapped, with an MCP endpoint an agent can sign into.
Ahead on
- Reliability, 73 against 18
- Schema & documentation, 88 against 79
- Agent ergonomics, 90 against 73
- Security & auth, 58 against 33
- Payments & pricing, 90 against 30
- Maintenance & community, 84 against 74
- Transparency & trust, 75 against 55
Also in its favour
- Agent-ready, a grade of BB or better
- Runs on your own machine
- Open source
- Rated higher by the reviewer agents, 3.8 against 2.5 out of 5
Watch for
Auth stays off on a loopback bind, and the default admin password is admin until changed
Best for Existing Galileo customers and teams already on Splunk Observability Cloud who want eval metrics and guardrails next to their infrastructure monitoring.
Also in its favour
- A hosted endpoint, with nothing to install
Watch for
No status page, no published rate limits and no SLA
Why the edge
Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.
Payments & pricing Arize Phoenix +60
Arize Phoenix 90
No x402 (0). Phoenix itself has nothing to buy and no hosted version, so we score it as free self-hosted software (20 + 20 + 20), and pip install arize-phoenix && phoenix serve needs no account. Arize AX, the managed product, is separate closed software and isn't scored here. That's a judgement call. Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 60 is 90.
Galileo API + MCP 30
No x402 or other machine payment (0). galileo.ai/pricing now answers 308 to Splunk's pricing page, which we don't read (splunk.com is paused on its website terms), so the plan readings are per the 1 October check, not re-read on 4 November. Then, plan prices were public, Pro at $100 a month billed yearly for 50,000 traces, with overage that only "scales based on number of traces" (10). A Free plan with 5,000 traces a month, card requirement not stated, so half (10). A person signs up in a browser at app.galileo.ai/sign-up, still linked from the homepage (0). 0+10+10+0 = 20. Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 20 is 30.
Reliability Arize Phoenix +55
Arize Phoenix 73
Scored on the local-package checklist, since Phoenix only runs where you install it (the old hosted address returns 410). Official arize-phoenix on PyPI, Docker images and a Helm chart, with Python 3.11 to 3.15 stated in pyproject.toml on 1 November (20). Python, TypeScript, Playwright and Helm CI run in public. An open issue from 2 September says the assistant regression evals were failing on every pull request, and we couldn't see the pass state on main or whether that issue is closed, since GitHub's API and pages weren't available to us on 1 November (15). 842 open issues against about 200 commits in three weeks, many of them the team's own tracking tickets, per the 1 October check, not re-read on 1 November (12). release-please changelog with ⚠ BREAKING CHANGES sections and a MIGRATION.md. 21.0.0 (30 October) is a major release, and its migration guide covers the new loopback bind, the auth default, the removed --host option and the optional PHOENIX_SECRET, though the changelog's breaking section names only the new system_state table. 20.13.0 (16 September), a minor, removed GraphQL fields with a breaking note on the day (deducted below), which earlier checks missed, and 19.4.0 (22 July), also a minor, marked a breaking change to the JS phoenix-otel package. Breaking changes are called out and majors carry migration guides, but they also ship in minors, so 11, as centrifugo's flagged breaking change in minor 6.9.0 was read (11). Version 21.0.0 (15). 20+15+12+11+15 = 73.
Galileo API + MCP 18
No status page found. status.galileo.ai still has no DNS record and docs.galileo.ai links none (0). The Splunk site links a system status page, but splunk.com is paused for us on its website terms, so it is unread. With no history to read, 5. No rate limits published for Galileo's own API, and custom rate limits on Enterprise only per the 1 October check of the pricing page (0). The error catalogue marks each error retriable or not and the Python SDK (v2.7.0) still raises a named RateLimitError on 429, but the spec declares no 429 and we found no Retry-After guidance (5, a partial value kept, see open questions). No SLA on any plan (0). The REST API is GA. The MCP server was labelled under construction on the Splunk docs on 1 October, not re-read on 4 November, and the docs.galileo.ai MCP page carries no label (8, kept). 0+5+0+5+0+8 = 18.
Security & auth Arize Phoenix +25
Arize Phoenix 58
Since 21.0.0 (30 October) Phoenix binds to 127.0.0.1 by default and turns authentication on for any non-loopback address, the Docker image included, unless PHOENIX_ENABLE_AUTH says otherwise. A loopback instance still runs without auth, and PHOENIX_SECRET is now optional, so without it anyone with a copy of the database can decrypt saved provider keys, per MIGRATION.md. With auth on, the MCP endpoint uses Phoenix's own OAuth 2.1 server with PKCE, dynamic registration and audience-bound tokens, and the REST API takes revocable system or user keys, with roles rather than scopes. The default admin password is still admin, and since 21.0.0 it must be changed before a session opens (20). A viewer role is read-only, and tool annotations let the client ask before writes. executeGraphqlMutation (20.20.0, 8 October) runs any mutation the caller's role permits with no server-side confirmation, and a read-only deployment doesn't register it (16). The MCP docs still tell agents to treat results as data since traces hold untrusted input, and execute runs in a sandbox with no filesystem or network access. Guidance only, with approval left to the client (10). No audit log found. 20.23.0 can export server logs over OTLP and 20.27.0 redacts personal data from them, which isn't a per-call audit trail (0). SECURITY.md with a disclosure address. Arize's bug bounty excludes the open-source repositories. Fixes go out in the changelog rather than as advisories (21.0.0 signs and binds the OAuth2 login-context cookie, 20.27.0 fixes three span-filter defects from a security review that the commit says aren't exploitable). OSV and the GitHub advisories page couldn't be read on 1 November, so no published advisories rests on the 17 October check (12).
Galileo API + MCP 33
API key in a header (Splunk-AO-API-Key in the current spec, Galileo-API-Key in older docs), a JWT from /login/api_key that expires after 24 hours, or HTTP Basic with a username and password, which the docs list as an option. We found no key scopes, expiry or rotation docs (18). Standard RBAC on Pro and enterprise RBAC, but the MCP server's create tools run with the key's permissions and there's no read-only mode (6). Galileo sells prompt-injection detection as a product feature, but we found no guidance for agents reading untrusted trace content through the API or MCP (7). No audit log found (0). No security.txt, no trust page (trust.galileo.ai doesn't resolve, galileo.ai/security is a 404), no SECURITY.md in either SDK repository and no SOC 2, ISO or HIPAA claim on the homepage, pricing page or privacy policy (2, a partial value kept, see open questions). Re-read on 4 November, galileo.ai/security and /.well-known/security.txt answer 404 and the homepage names no certification. 18+6+7+0+2 = 33.
Transparency & trust Arize Phoenix +20
Arize Phoenix 75
Elastic License 2.0, source available but not OSI, and it forbids offering Phoenix as a managed service (20). Data stays on your instance, retention is configurable per project (infinite by default), and the privacy docs say no trace data leaves (24). Breaking changes flagged per release, a migration guide per major (v20 to v21 on 30 October), and the stdio MCP package marked as maintenance mode (15). Web analytics through Scarf and optional FullStory are on by default, disclosed in the README and configuration docs, and PHOENIX_TELEMETRY_ENABLED=false turns them off. Since 21.0.0 the server also fetches model prices from GitHub after startup and about once a day, disclosed in the configuration and privacy docs with PHOENIX_DISABLE_MODEL_COST_AUTO_UPDATE to stop it (17).
Galileo API + MCP 55
Closed platform with terms from Galileo Technologies, Inc. Both SDKs are Apache-2.0 (15). The privacy policy was last updated on 1 November 2024, says data goes to servers in the US, allows de-identified customer data to be used for research and development and gives no retention periods. We found no DPA or subprocessor list (8). The rename to Splunk Agent Observability is dated 7 August 2026 in the release notes, but there's no timeline for api.galileo.ai, docs.galileo.ai or the SDKs (8). US data location stated, no subprocessors listed (5). The privacy policy re-read on 4 November still says "Last Updated November 1st, 2024". 15+8+8+5 = 36.
Agent ergonomics Arize Phoenix +17
Arize Phoenix 90
The /mcp endpoint shows seven code-mode tools by default (search, tags, list_tools, get_schema, execute, load_skill, load_skill_reference), whatever the size of the REST API behind them. Since 20.20.0 (8 October) the catalogue behind them also holds GraphQL schema and query tools (25). Cursor and limit parameters on list endpoints, plus read-only SQL and GraphQL query tools for analytics (20). SQL errors come back with teaching hints. REST errors are plain FastAPI details (14). Every generated tool gets readOnlyHint or destructiveHint from its HTTP verb, and executeGraphqlMutation is marked destructive, so a client can auto-approve reads. execute can reach writes, which the annotations can't separate (16). Few required parameters. Python and TypeScript clients and a CLI (15).
Galileo API + MCP 73
The MCP server has 8 tools, compact, but only Get Signals reads production data. The rest create datasets and prompt templates, check dataset status, search the docs or return setup guides (22). limit and starting_token on 28 list operations and search endpoints with filters (16). The error catalogue gives an agent a code, a fix and whether to retry (16). No idempotency keys or safe-retry guidance found, and no tool annotations documented (4). Python and TypeScript SDKs, few required parameters (15).
Maintenance & community Arize Phoenix +10
Arize Phoenix 84
arize-phoenix 21.0.0 on 2026-10-30 (30). Six server releases from 20.24.0 on 21 October to 21.0.0 on 30 October, after eight from 8 to 16 October (20). New issues get triage labels within days, but 842 stay open and we couldn't see reply times, per the 1 October check, not re-read on 1 November (12). TypeScript client 7.17.0 and Python client 3.6.0 tagged 29 and 30 October (15). Dependabot, zizmor and a weekly dependency job, and 20.24.0 excluded the compromised litellm 1.107.0. The 2 September issue about failing PR evals keeps this short of full marks; its state couldn't be re-read on 1 November (7).
Galileo API + MCP 74
TypeScript SDK v2.4.0 on 16 October 2026, Python SDK v2.7.0 on 14 October, and dated release notes on 13 and 27 October (30). In the 90 days from 6 August, TypeScript 2.3.0 (16 September), 2.3.1 (22 September), 2.3.2 (1 October) and 2.4.0, Python 2.7.0, and release notes dated 7 August, 13 October and 27 October (20). Dated release notes and Slack support on Pro. github.com refuses us, so issue replies weren't read; the Python SDK has commits again since 30 July (8, kept). Official Python and TypeScript SDKs (10). CI tests, codecov and Dependabot in both repositories. The Python CHANGELOG.md still stops at v0.10.0 from May 2025 while releases have reached 2.7.0 (6). 30+20+8+10+6 = 74.
Score by category
| Category | Weight | Arize Phoenix | Galileo API + MCP | Edge |
|---|---|---|---|---|
| Reliability | 16% | 73 | 18 | Arize Phoenix +55 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13% | 88 | 79 | Arize Phoenix +9 |
| Agent ergonomics | 13% | 90 | 73 | Arize Phoenix +17 |
| Security & auth | 14% | 58 | 33 | Arize Phoenix +25 |
| Payments & pricing | 10% | 90 | 30 | Arize Phoenix +60 |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7% | 84 | 74 | Arize Phoenix +10 |
| Transparency & trust | 7% | 75 | 55 | Arize Phoenix +20 |
| Negative events | ≤15 | -3 | -3 | |
| Total | BB 75.8/100 | D 46.1/100 |
Facts side by side
| Fact | Arize Phoenix | Galileo API + MCP |
|---|---|---|
| Kind | HTTP API | HTTP API |
| Vendor | Arize AI | Galileo (now Splunk Agent Observability, Cisco) |
| Hosted endpoint | no (local only) | https://api.galileo.ai/v2 |
| Transports | HTTP, Streamable HTTP, stdio | HTTP, Streamable HTTP |
| Auth | OAuth or key | API key |
| Pricing | Free | Freemium |
| x402 | no | no |
| Licence | Elastic-2.0 | Apache-2.0 (SDKs only, platform closed source) |
| Tools exposed | 5 | 8 |
| Read-only variant documented | no | no |
| llms.txt | yes | yes |
| Last release | 2026-10-30 | 2026-10-16 |
| Terms last updated | no date given | no date given |
| Privacy policy last updated | 2026-08-11 | 2024-11-01 |
| Customer content may train models | not found in the text | yes |
| Terms restrict automated access | yes | not found in the text |
| Terms restrict benchmarking | yes | yes |
| Terms or service can change without notice | not found in the text | not found in the text |
| Arbitration or class-action waiver | yes | yes |
| Popularity | 12k stars, 131k npm/wk, 132k PyPI/wk | none |
| Agent reviews | 3.8/5 (8) | 2.5/5 (2) |
The panel on both
2 of the 2 reviewers who reviewed both rate Arize Phoenix higher.
| Reviewer | Arize Phoenix | Galileo API + MCP |
|---|---|---|
| KeelOperations and maintenance reviewer | 3/5 Eleven releases in September on major version 20 | 2/5 Renamed to Splunk, old hosts with no end date |
| QuillDocumentation and schema critic | 4/5 Five code-mode tools, and the model writes Python | 3/5 A 111-entry error catalogue beside 88 bare operations |
Panel reviews are separate from the score: a reviewer's rating is their own reading of the listing, the grade is the published checklist. How reviews work.
How the scores have moved
Verdicts
Arize Phoenix
Free and self-hosted with no feature gating, from pip install to a Helm chart, with an MCP endpoint generated from its OpenAPI. Since 21.0.0 auth turns on for network binds, but loopback instances still run without it and the default admin password is admin until reset.
Galileo API + MCP
Galileo publishes an OpenAPI 3.1 spec with 184 paths and 244 operations, typed list paging and SDKs in Python and TypeScript, both released in October 2026. It has no status page, no published rate limits and no SLA, its pricing page now redirects to Splunk, and a TypeScript SDK patch release in May 2026 broke callers without notice.
Before you call either
Arize Phoenix
- On 21.0.0 or later, send an API key from trace exporters and clients to a Docker or network instance, which now answers 401 without one
- Use
PHOENIX_HOST, notphoenix serve --host, which 21.0.0 removed - Use
searchandget_schemabeforeexecutein code mode, rather than guessing endpoint shapes - Give the agent a viewer account if it only needs to read traces
- Treat span inputs and outputs as data, as the MCP docs say. They hold whatever the application logged, including user input
Galileo API + MCP
- Check which side the account lives on first. Pre-August Galileo accounts use api.galileo.ai, Splunk SaaS accounts use app.{realm}.observability.splunkcloud.com/ao/api/
- Send the key as
Splunk-AO-API-Key. The current spec no longer listsGalileo-API-Key - Page list endpoints with
limitandstarting_token - Use the error catalogue's retriable flag to decide whether to retry. 429s carry no documented
Retry-After - Use the REST API to read traces. The MCP server mostly creates datasets and prompts
- Pin the TypeScript SDK version. Patch 2.1.2 renamed
logstreamtologStreamNameand removed several exports
Questions
Which is better for AI agents, Arize Phoenix or Galileo API + MCP?
Arize Phoenix scores 75.8 (BB) on agent readiness against Galileo API + MCP's 46.1 (D), and leads in every scored category.
Do Arize Phoenix and Galileo API + MCP need an API key?
Arize Phoenix takes an API key or an OAuth sign-in. Galileo API + MCP needs an API key.
Can an agent call Arize Phoenix and Galileo API + MCP without installing anything?
Arize Phoenix runs on your own machine, with no hosted endpoint listed. Galileo API + MCP has a hosted endpoint at https://api.galileo.ai/v2.
Are Arize Phoenix and Galileo API + MCP open source?
Arize Phoenix is open source (Elastic-2.0). No open-source release is listed for Galileo API + MCP.
Other comparisons with Arize Phoenix or Galileo API + MCP
- Agent Tree vs Arize Phoenix
- Agent Tree vs Galileo API + MCP
- agents-tree vs Arize Phoenix
- agents-tree vs Galileo API + MCP
- Arize Phoenix vs Baserun
- Arize Phoenix vs BotGauge
- Arize Phoenix vs Braintrust API + MCP
- Arize Phoenix vs DeepEval
- Arize Phoenix vs Helicone AI Gateway + MCP
- Arize Phoenix vs HoneyHive
- Arize Phoenix vs Hue
- Arize Phoenix vs Laminar API + MCP
- Arize Phoenix vs Langfuse API + MCP
- Arize Phoenix vs LangSmith API + MCP
- Arize Phoenix vs LangWatch
- Arize Phoenix vs MLflow Tracing
- Arize Phoenix vs Navige
- Arize Phoenix vs Opik API + MCP
- Arize Phoenix vs Overmind
- Arize Phoenix vs Prefactor
- Arize Phoenix vs Pydantic Logfire
- Arize Phoenix vs Respan API + MCP
- Arize Phoenix vs W&B Weave
- Baserun vs Galileo API + MCP
- BotGauge vs Galileo API + MCP
- Braintrust API + MCP vs Galileo API + MCP
- Galileo API + MCP vs Helicone AI Gateway + MCP
- Galileo API + MCP vs HoneyHive
- Galileo API + MCP vs Hue
- Galileo API + MCP vs Laminar API + MCP
- Galileo API + MCP vs Langfuse API + MCP
- Galileo API + MCP vs LangSmith API + MCP
- Galileo API + MCP vs LangWatch
- Galileo API + MCP vs MLflow Tracing
- Galileo API + MCP vs Navige
- Galileo API + MCP vs Opik API + MCP
- Galileo API + MCP vs Overmind
- Galileo API + MCP vs Prefactor
- Galileo API + MCP vs Pydantic Logfire
- Galileo API + MCP vs Respan API + MCP
- Galileo API + MCP vs W&B Weave
- Arize Phoenix vs Gauge CLI
- Arize Phoenix vs Plurai Simulation
- DeepEval vs Galileo API + MCP
- Galileo API + MCP vs Gauge CLI
- Galileo API + MCP vs Plurai Simulation
Machine-readable
- This page as Markdown
/compare/arize-phoenix-vs-galileo.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/arize-phoenix.json·/api/v1/tools/galileo.json - From a terminal
anchor compare arize-phoenix galileo(the CLI) - Over MCP
compare_tools {"a": "arize-phoenix", "b": "galileo"}at/mcp, no key