Gauge CLI vs Opik API + MCP

Gauge CLI 60

CLI reading. No machine-readable contract, -o json on read commands (10 of 25). No llms.txt; instructions bundled in the CLI for agents (5). The README says what it finds and when local or hosted fits (16 of 20). Commands with help text; typed flags (9 of 15). Worked examples, no error reference (8 of 15). Dated changelog and semver tags (12 of 15). 10 + 5 + 16 + 9 + 8 + 12 = 60.

Opik API + MCP 84

Public OpenAPI 3.1 at /docs/opik/openapi.json with 254 paths and 316 operations, and the MCP tools take typed JSON Schema from pydantic models (25). llms.txt with section indexes and a Markdown twin of every page at .md (10). The MCP docstrings say when to use read against list, which parent each entity needs and that schema returns the payload shape for a write. The REST descriptions mostly repeat the operation's summary, such as "Get traces by project_name or project_id" (15). Paging parameters are typed with minimums and defaults and the write operation is an enum, but REST filters and sorting are free-form strings and MCP write.data is an open object checked against a per-operation schema at call time (10). Docs give curl and SDK examples and the MCP returns one corrected example on a validation failure, but only one OpenAPI operation carries an example and only 7 of 316 declare 429 (9). Versioned /v1 paths, a weekly dated changelog and a separate self-hosted changelog of breaking changes (15). Total 84.

Gauge CLI 53

MIT (30). Local data stays in .gauge/ and telemetry fields are listed; hosted data handling wasn't read (18 of 30). Dated changelog, no deprecation policy (10 of 20). Telemetry on by default, disclosed with an opt-out (20). 30 + 18 + 10 + 20 = 78.

Opik API + MCP 75

Backend, frontend, Python and TypeScript SDKs, the agent optimisation SDK and the MCP server are Apache-2.0. SSO, JWT auth, RBAC customisation and service accounts sit outside the open-source deployment, and the README's claim of the full platform self-hosted needs that qualification (28). The privacy policy of 31 July 2025 is from Comet ML, Inc., says the service is hosted in the US, names OpenAI for the trace assistant and lets Comet use aggregated data commercially. The Terms of 5 June 2026 say the customer owns Customer Data. Plan retention is on the pricing page. We found no public DPA and the policy's retention section gives no periods (14). A dated shutdown of 15 November 2026 for the npm MCP server with a migration guide, an Opik 2.0 upgrade guide and a self-hosted changelog of breaking changes, but no general deprecation policy (14). US data region stated for Free and Pro. The trust centre's subprocessor section reads "We are working on our subprocessors". Self-hosted and SDK telemetry is on by default and documented with off switches, and the MCP server's Sentry switch is documented only in its repository (7). Total 63.

Gauge CLI 75

CLI reading for an agent driving it. Bundled instructions match the installed version, and output is filterable with -o json (20 of 25). Run summaries and Findings with queryable fields (15 of 20). Errors not documented as a reference (10 of 20). Reruns from checkpoints and a recheck command with a confirm step (15 of 20). Four commands to start, Node only (15 of 15). 20 + 15 + 10 + 15 + 15 = 75.

Opik API + MCP 93

Five MCP tools, read, list, write, schema and read_skill, with write payload schemas fetched on demand through schema rather than loaded up front (25). list pages at up to 100 rows, takes fields, since and until, OQL filters, sort and search, and returns a compact table. REST lists take page, size, exclude, truncate and strip_attachments (20). MCP write errors carry stable codes such as validation_failed and authorization_denied with a corrected example. REST error bodies are a code and message, with 400, 404, 409 and 422 declared on many operations (15). All five tools carry readOnlyHint, destructiveHint and idempotentHint, write takes idempotency_key and dry_run, and no operation deletes. REST idempotency isn't documented (18). Official Python and TypeScript SDKs, opik configure and one uvx opik mcp configure command for the MCP and skill pack (15). Total 93.

Gauge CLI 60

Local package. Installs from npm as @withgauge/cli with Node 22.13 or later and Docker stated (20). checks.yml runs npm run check on push and pull request across a matrix, and the repository has 76 test files; whether main passes wasn't read (20 of 25). Issues weren't read (5, unchecked). A dated changelog with Added sections per release (15). Version 0.28.0 (0). 20 + 20 + 5 + 15 + 0 = 60.

Opik API + MCP 72

Graded with the hosted lines, on Opik Cloud's REST API and hosted MCP server. Atlassian Statuspage at status.comet.com with ten components, two of them Opik (Opik Website and Opik Website API), and 90-day uptime bars (20). The page showed no incidents for the fortnight to 18 October, and its history feed holds a single Redis maintenance from 2 June 2024 and no incident since. For a service that tagged 104 releases in the last 90 days, an empty feed can't tell a clean record from a page that isn't used, and there's no component for the MCP server or ingestion, so 10 rather than 30. Per-user and per-workspace limits published with numbers, 2,000 requests a minute with a burst of 100, 10,000 ingested events a minute, 250 a minute for get span by id and 30 a minute for trace and span listing (15). 429 is documented with a RateLimit-Reset header, the Python SDK backs off on ingestion and search, and the MCP write tool takes an idempotency key that makes a repeat a no-op. Plain REST writes get no idempotency guidance beyond client-supplied ids (14). Enterprise lists a 2-hour support response SLA. We found no uptime SLA (3). The REST API and hosted MCP server carry no beta or preview label (10). Total 72.

Gauge CLI 40

Scored on the hosted option. No payment protocol (0). Credit prices weren't found on the pages read (0). Local mode is free with no card (20). Local mode needs no sign-up (20). 0 + 0 + 20 + 20 = 40.

Opik API + MCP 35

Graded on Opik Cloud, as the paid option. No x402, MPP or L402 (0). Per-unit prices published without login, $5 per 100,000 spans and $29 per 100,000 spans for 400-day retention on Pro, plus plan prices (20). Free Cloud has 25,000 spans a month. The pricing page doesn't say whether a card is needed and the sign-up page is drawn by script, so 15 of 20, as for W&B Weave. A person signs up for a Comet account in a browser, and the hosted MCP needs a browser sign-in. Free self-hosting isn't an agent route (0). Total 35.