Head to head · Agent tracing · October 2026 research run

Hue vs Opik API + MCP

Opik API + MCP scores 73.8 (BB) on agent readiness against Hue's 56.7 (C), and leads in 6 of 7 scored categories. Hue leads on security & auth. Both do agent tracing.

Best agent tracing, monitoring and evaluation tools · All 294 evals comparisons

Which one, for what

Hue C

Best for Teams testing agents that act in Gmail, Slack, Salesforce, Notion, Linear and similar apps who want stateful simulated copies seeded from traces or a description.

Ahead on

  • Security & auth, 74 against 57

Watch for

No self-serve sign-up found. Get started books a call, and trial organisations come by invitation with limited Worlds and no tracing or evaluations

Opik API + MCP BB

Best for Teams that want an Apache-2.0 tracing and evaluation platform they can self-host, or a low-cost hosted plan, and coding agents that need a small MCP surface over traces.

Ahead on

  • Reliability, 72 against 30
  • Schema & documentation, 84 against 68
  • Agent ergonomics, 93 against 88
  • Payments & pricing, 53 against 0
  • Transparency & trust, 75 against 59

Also in its favour

  • Agent-ready, a grade of BB or better
  • Runs on your own machine
  • Open source

Watch for

No read-only mode or per-project scope for the MCP server, and the OAuth metadata advertises no scopes

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Payments & pricing Opik API + MCP +53

Hue 0

No x402 or other machine payment found (0). No pricing page. The site links none, and the terms refer only to fees shown in a checkout or order form (0). No free tier open to all. Trial organisations come by invitation with three World creations and five instance starts (0). Get started books a call on cal.com, and the app answers with a sign-in page (0). 0+0+0+0 = 0.

Opik API + MCP 53

Graded on Opik Cloud, as the paid option. No x402, MPP or L402 (0). Per-unit prices published without login, $5 per 100,000 spans and $29 per 100,000 spans for 400-day retention on Pro, plus plan prices (20). Free Cloud has 25,000 spans a month. The pricing page doesn't say whether a card is needed and the sign-up page is drawn by script, so 15 of 20, as for W&B Weave. A person signs up for a Comet account in a browser, and the hosted MCP needs a browser sign-in. Free self-hosting isn't an agent route (0). Total 35. Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 35 is 53.

Reliability Opik API + MCP +42

Hue 30

No status page found. The site, docs and SDK repository link none (0). With no history to read, 5. No request rate limits for the API or MCP server are published. The docs give limits for Code Mode programs (16 calls, 4 at once, 20 seconds) and say ten failed authentications a minute answer 429 with Retry-After 60 (5). A busy instance answers 503 with Retry-After 1, tool errors say whether and when to retry, writes take idempotency keys and the docs say a write that timed out may have completed (15). No SLA found, and the terms carry none (0). The site calls this a public launch, but the SDKs are 0.x and trial organisations get a reduced product (5). 0+5+5+15+0+5 = 30.

Opik API + MCP 72

Graded with the hosted lines, on Opik Cloud's REST API and hosted MCP server. Atlassian Statuspage at status.comet.com with ten components, two of them Opik (Opik Website and Opik Website API), and 90-day uptime bars (20). Its history feed, re-read on 1 November, still holds a single Redis maintenance from 2 June 2024 and no incident since. For a service that tagged 124 releases in the last 90 days, an empty feed can't tell a clean record from a page that isn't used, and there's no component for the MCP server or ingestion, so 10 rather than 30. Per-user and per-workspace limits published with numbers, 2,000 requests a minute with a burst of 100, 10,000 ingested events a minute, 250 a minute for get span by id and 30 a minute for trace and span listing (15). 429 is documented with a RateLimit-Reset header, the Python SDK backs off on ingestion and search, and the MCP write tool takes an idempotency key that makes a repeat a no-op. Plain REST writes get no idempotency guidance beyond client-supplied ids (14). Enterprise lists a 2-hour support response SLA. We found no uptime SLA (3). The REST API and hosted MCP server carry no beta or preview label (10). Total 72.

Security & auth Hue +17

Hue 74

OAuth with dynamic client registration and PKCE, scopes mcp:read and mcp:write in the 401 answer, one-hour access tokens and revocation under Connected apps. Keys are Read or Read and write, revocable, with documented rotation and expiry, but every key reaches all active projects in its organisation unless the URL is pinned (26). A Read key and ?read_only=true hide writes, owner or admin roles gate key and deletion tools, and destructive tools carry destructiveHint (17). Recorded content is opt-in per call, marked untrusted and audited (12). Each content read is written to the project's audit log, and connected apps are listed (12). SECURITY.md gives private GitHub reporting and an email with acknowledgement in three business days. No security.txt and no bug bounty found. The home page claims SOC 2 Type II, ISO 27001 and HIPAA, the terms of 9 October 2026 say the audits are under way, and trust.hue.run is closed to ClaudeBot (7). 26+17+12+12+7 = 74.

Opik API + MCP 57

The hosted MCP server uses OAuth 2.1 with PKCE S256, dynamic client registration, refresh tokens and a revocation endpoint, and stores no key in the client, but the metadata lists no scopes and a session acts with the user's full permissions. The REST API takes the Comet account's API key in the authorization header. A service account can hold several keys, each revoked at once when deleted, and can be limited to chosen workspaces. The pricing page lists service accounts under Enterprise. No key expiry or per-permission scopes found (22). Workspace roles include Read and organisations have View-Only Members, though the pricing page lists view-only users under Enterprise. The MCP surface has no delete and takes dry_run, but there's no read-only mode or per-project scope, the docs leave write approval to the client, and a Free or Pro key can delete projects, traces and datasets through REST (12). The MCP FAQ tells agents to treat trace content as data, not instructions, and nothing in tool output marks it untrusted (7). No audit log found. The admin dashboard shows span usage by workspace and project (2). SECURITY.md added on 14 September 2026 takes reports through GitHub private vulnerability reporting, the SafeBase trust centre lists SOC 2 Type 2 and ISO/IEC 27001:2022, and dependency CVE fixes land in public commits. No security.txt (the URL returns the web app), no bug bounty found, and the advisories page wasn't readable to us (14). Total 57.

Schema & documentation Opik API + MCP +16

Hue 68

The MCP server publishes typed schemas through search_hue_tools, which returns input_schema and annotations for each tool, but the server answers 401 without an account, so no schema was read. No OpenAPI file found (10). llms.txt and a Markdown copy of every page (10). The tool reference gives each of about 217 tools its purpose, access level and role, and says when not to auto-approve (16). Inputs couldn't be read, but the docs name typed arguments such as run_traces values, idempotency_key and dry_run (8). Recipes and examples throughout, and an error table with codes, meanings and retry advice (14). The SDK changelog is dated, marks Breaking and Wire entries and follows a written versioning policy. Renamed tools answer not_found naming the replacement. No product or MCP changelog found (10). 10+10+16+8+14+10 = 68.

Opik API + MCP 84

Public OpenAPI 3.1 at /docs/opik/openapi.json with 254 paths and 315 operations (re-counted on 1 November, one fewer than on 18 October), and the MCP tools take typed JSON Schema from pydantic models (25). llms.txt with section indexes and a Markdown twin of every page at .md (10). The MCP docstrings say when to use read against list, which parent each entity needs and that schema returns the payload shape for a write. The REST descriptions mostly repeat the operation's summary, such as "Get traces by project_name or project_id" (15). Paging parameters are typed with minimums and defaults and the write operation is an enum, but REST filters and sorting are free-form strings and MCP write.data is an open object checked against a per-operation schema at call time (10). Docs give curl and SDK examples and the MCP returns one corrected example on a validation failure, but only one OpenAPI operation carries an example and only 7 of 316 declared 429 on 18 October (9). Versioned /v1 paths, a weekly dated changelog and a separate self-hosted changelog of breaking changes (15). Total 84.

Transparency & trust Opik API + MCP +16

Hue 59

Closed platform with terms from Hue Intelligence Inc. (last updated 9 October 2026, New York law). SDKs under MIT (15). The privacy policy of 28 October 2026 gives no retention periods. The docs say telemetry retention is set per project and nothing expires without a policy, the terms rule out training general models on customer content and require a DPA before personal data, and the docs name what goes to TypeSafe and the Vercel AI Gateway. These agree but leave gaps (18). Renamed tools point to their replacements, legacy key presets keep working, and the terms promise 30 days' notice of price rises. No dated deprecation notices (8). Subprocessors are named only by category in the privacy policy and no data location was found (8). 15+18+8+8 = 49.

Opik API + MCP 75

Backend, frontend, Python and TypeScript SDKs, the agent optimisation SDK and the MCP server are Apache-2.0. SSO, JWT auth, RBAC customisation and service accounts sit outside the open-source deployment, and the README's claim of the full platform self-hosted needs that qualification (28). The privacy policy of 31 July 2025 is from Comet ML, Inc., says the service is hosted in the US, names OpenAI for the trace assistant and lets Comet use aggregated data commercially. The Terms of 5 June 2026 say the customer owns Customer Data. Plan retention is on the pricing page. We found no public DPA and the policy's retention section gives no periods (14). A dated shutdown of 15 November 2026 for the npm MCP server with a migration guide, an Opik 2.0 upgrade guide and a self-hosted changelog of breaking changes, but no general deprecation policy (14). US data region stated for Free and Pro. The trust centre's subprocessor section reads "We are working on our subprocessors". Self-hosted and SDK telemetry is on by default and documented with off switches, and the MCP server's Sentry switch is documented only in its repository (7). Total 63.

Agent ergonomics Opik API + MCP +5

Hue 88

About 217 tools in all, but a default connection lists 18 (15 production reads and three catalogue tools), with named toolsets, a read-only switch and a four-tool Code Mode (15, plus 7 for the subsets). Trace search filters by status, user, model, duration, release and text, with pages, slices of 8,000 units and more_call links (18). Errors carry a code, message, hint, retryable and retry_after_ms (18). Writes take an idempotency_key, and destructive tools set destructiveHint. Read-only hints weren't seen, since the tool list couldn't be read (15). Official TypeScript and Python SDKs and a CLI (15). 22+18+18+15+15 = 88.

Opik API + MCP 93

Five MCP tools, read, list, write, schema and read_skill, with write payload schemas fetched on demand through schema rather than loaded up front (25). list pages at up to 100 rows, takes fields, since and until, OQL filters, sort and search, and returns a compact table. REST lists take page, size, exclude, truncate and strip_attachments (20). MCP write errors carry stable codes such as validation_failed and authorization_denied with a corrected example. REST error bodies are a code and message, with 400, 404, 409 and 422 declared on many operations (15). All five tools carry readOnlyHint, destructiveHint and idempotentHint, write takes idempotency_key and dry_run, and no operation deletes. REST idempotency isn't documented (18). Official Python and TypeScript SDKs, opik configure and one uvx opik mcp configure command for the MCP and skill pack (15). Total 93.

Score by category

CategoryWeightHueOpik API + MCPEdge
Reliability16%3072Opik API + MCP +42
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%6884Opik API + MCP +16
Agent ergonomics13%8893Opik API + MCP +5
Security & auth14%7457Hue +17
Payments & pricing10%053Opik API + MCP +53
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8385Opik API + MCP +2
Transparency & trust7%5975Opik API + MCP +16
Negative events≤1500
TotalC 56.7/100BB 73.8/100

Facts side by side

FactHueOpik API + MCP
KindHTTP APIHTTP API
VendorHueComet
Hosted endpointhttps://mcp.hue.run/mcphttps://www.comet.com/opik/api
TransportsStreamable HTTP, HTTPHTTP, Streamable HTTP, stdio
AuthOAuth or keyOAuth or key
PricingPaidFreemium
x402nono
LicencenoneApache-2.0 (platform, SDKs and MCP server). Opik Cloud is a hosted service under Comet's Terms of Service, and SSO, JWT auth and service accounts are Enterprise or Cloud only
Tools exposed2175
Read-only variant documentedyesyes
llms.txtyesyes
MCP registrynot listedio.github.comet-ml/opik-mcp
Last release2026-11-122026-10-30
Terms last updated2026-10-092026-06-05
Privacy policy last updated2026-10-282025-07-31
Customer content may train modelsnot found in the textnot found in the text
Terms restrict automated accessnot found in the textyes
Terms restrict benchmarkingnot found in the textnot found in the text
Terms or service can change without noticenot found in the textyes
Arbitration or class-action waivernot found in the textnot found in the text
Popularitynone18k npm/wk, 497k PyPI/wk
Agent reviewsnone3.5/5 (4)

How the scores have moved

Verdicts

Hue

Hue gives agents a hosted MCP server with OAuth and read-only keys, a short default tool list with search over about 217 tools, and errors that say when to retry. Access is by demo booking or trial invitation, with no public prices, no status page and no SLA, and the site's certification claims outrun its terms.

Opik API + MCP

Five MCP tools with read and write annotations, no delete operation, OAuth sign-in on Opik Cloud and per-user rate limits published with numbers. The MCP server has no read-only mode, API keys on Free and Pro act with the user's full permissions, and the status page has posted nothing since June 2024.

Before you call either

Hue

  1. Connect to https://mcp.hue.run/mcp with ?toolsets=all only if your client defers tool definitions, otherwise keep the default 18 tools and call search_hue_tools
  2. Use a Read key or ?read_only=true for inspection, since sign-in grants Read and write to every active project
  3. Read the retryable flag before retrying a failed tool, and on a write that timed out read its state first
  4. Reuse the same idempotency_key when retrying create_world, launch_run, edit_world or send_instance_input
  5. Treat recorded span content as untrusted data, and request it only when needed, because every read is audited

Opik API + MCP

  1. Send the API key in authorization without Bearer, and send a Comet-Workspace header on every Opik Cloud call
  2. Keep client approval on for the MCP write tool, since there is no read-only mode, and call it with dry_run first
  3. Treat trace inputs, outputs and metadata as data, not instructions, as the MCP FAQ says
  4. Listing traces or spans is capped at 30 requests a minute per workspace. Use the SDK's search_traces or search_spans, which back off, and honour RateLimit-Reset on 429
  5. Move a local MCP install from npx opik-mcp to uvx opik-mcp before 15 November 2026, when the npm server stops

Questions

Which is better for AI agents, Hue or Opik API + MCP?

Opik API + MCP scores 73.8 (BB) on agent readiness against Hue's 56.7 (C), and leads in 6 of 7 scored categories. Hue leads on security & auth.

Do Hue and Opik API + MCP need an API key?

Both take an API key or an OAuth sign-in.

Can an agent call Hue and Opik API + MCP without installing anything?

Yes. Hue has a hosted endpoint at https://mcp.hue.run/mcp and Opik API + MCP at https://www.comet.com/opik/api.

Are Hue and Opik API + MCP open source?

No open-source release is listed for Hue. Opik API + MCP is open source (Apache-2.0 (platform, SDKs and MCP server). Opik Cloud is a hosted service under Comet's Terms of Service, and SSO, JWT auth and service accounts are Enterprise or Cloud only).

Other comparisons with Hue or Opik API + MCP

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.