# Braintrust API + MCP vs Gauge CLI

> Braintrust scores 65.1 (B) to Gauge CLI's 60.4 (C) for evaluations. Prices, MCP, x402, uptime and agent notes side by side.

- Canonical: https://www.anchorterminal.com/compare/braintrust-vs-gauge-cli
- Markdown: https://www.anchorterminal.com/compare/braintrust-vs-gauge-cli.md (~4,300 tokens)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-06

Braintrust API + MCP scores 65.1 (B) on agent readiness against Gauge CLI's 60.4 (C), and leads in 5 of 7 scored categories. Gauge CLI leads on agent ergonomics. Both do evaluations.

- Braintrust API + MCP: grade B, 65.1/100, rank #397 of 1354. Markdown https://www.anchorterminal.com/tools/braintrust.md · JSON https://www.anchorterminal.com/api/v1/tools/braintrust.json
- Gauge CLI: grade C, 60.4/100, rank #590 of 1354. Markdown https://www.anchorterminal.com/tools/gauge-cli.md · JSON https://www.anchorterminal.com/api/v1/tools/gauge-cli.json
- Best agent tracing, monitoring and evaluation tools: https://www.anchorterminal.com/best/agent-observability/index.md
- All 250 evals comparisons: https://www.anchorterminal.com/compare/agent-observability/index.md

## Which one, for what

### Braintrust API + MCP (B)

Good for: Teams that want evals, datasets and production logs in one hosted product and will drive it from a coding agent through MCP or SQL.

Ahead on:
- Reliability, 76 against 60
- Schema & documentation, 85 against 60
- Transparency & trust, 66 against 53

Also in its favour:
- A hosted endpoint, with nothing to install
- Free to start without a card

Watch for: The hosted `set_topics_automation` MCP tool began rejecting `facet_mode: "replace"` in October 2026, with no earlier notice found

### Gauge CLI (C)

Good for: Teams that build skills, CLIs, SDKs or MCP servers and want to see where coding agents get stuck with them, and to test fixes before release.

Ahead on:
- Agent ergonomics, 75 against 66

Also in its favour:
- No key needed to call it
- Open source
- No incidents deducted, where Braintrust API + MCP loses 3 points for them

Watch for: Collects anonymous command usage by default, with an opt-out


## Why the edge

### Schema & documentation: Braintrust API + MCP +25

- Braintrust API + MCP (85): OpenAPI 3.1.1 with 77 paths and 166 operations, published in the Apache-2.0 braintrustdata/braintrust-openapi repository (last synced 31 October) and pinned by the Python SDK (25). The copy at www.braintrust.dev/docs/openapi.json is a path that site's robots.txt now disallows, so it wasn't read. llms.txt with about 700 lines at docs.braintrust.dev, every page served as Markdown (10). The MCP docs give a one-line purpose for each of the 50 tools and mark the `test_*` tools as dry runs, but say little about when not to use a tool. The MCP server is closed source, so we read the docs rather than the definitions (13). Typed schemas with pagination parameters and enums, though `sql_query` takes free-form SQL and event payloads are free-form by nature (11). 400, 401, 403, 429 and 500 responses are declared on most operations, but only 4 of 166 operations carry an inline example. The cookbook fills some of the gap (11). Versioned `/v1` API, a monthly product changelog, a data-plane changelog and per-SDK changelogs (15). 25+10+13+11+11+15 = 85.
- Gauge CLI (60): CLI reading. No machine-readable contract, -o json on read commands (10 of 25). No llms.txt; instructions bundled in the CLI for agents (5). The README says what it finds and when local or hosted fits (16 of 20). Commands with help text; typed flags (9 of 15). Worked examples, no error reference (8 of 15). Dated changelog and semver tags (12 of 15). 10 + 5 + 16 + 9 + 8 + 12 = 60.

### Reliability: Braintrust API + MCP +16

- Braintrust API + MCP (76): Status page at status.braintrust.dev, now on incident.io, with component history (20). Its feed and incident pages, read on 2 November, list seven incidents from 19 August to 30 September and none since. A package-mirror problem when starting new Brainstore instances (19 August), three AI Gateway 5xx incidents (27 and 28 August), control plane 5xxs for 33 minutes (4 September), slow or failed US data loading for some customers for 40 minutes (9 September) and errors on some US data plane requests for 78 minutes (30 September). Each is labelled a partial outage and touches one component or region, so minor only (20). The only published limit is 20 BTQL queries a minute on hosted plans. Ingestion and REST limits have no numbers (8). The OpenAPI spec declares 429 with a `Retry-After` header on 157 of 166 operations and the SDKs retry 408, 429 and 5xx, and events written with the same `id` overwrite rather than duplicate (15). Enterprise lists an uptime SLA with no published terms (3). API and MCP are GA (10). 20+20+8+15+3+10 = 76.
- Gauge CLI (60): Local package. Installs from npm as @withgauge/cli with Node 22.13 or later and Docker stated (20). checks.yml runs npm run check on push and pull request across a matrix, and the repository has 76 test files; whether main passes wasn't read (20 of 25). Issues weren't read (5, unchecked). A dated changelog with Added sections per release (15). Version 0.28.0 (0). 20 + 20 + 5 + 15 + 0 = 60.

### Transparency & trust: Braintrust API + MCP +13

- Braintrust API + MCP (66): Platform closed source with clear terms (updated 7 July 2026). Both SDKs and the OpenAPI repository are Apache-2.0 (18). The privacy notice was last updated on 21 September 2023 and says it doesn't apply to data customers send to the service. Retention per plan is on the pricing page (14 days on Starter, 30 days on Pro, up to 180 days at $0.50 a GB a month) and a DPA is click-through on Pro. We found no retention statement for trace data beyond that (12). The changelog dates deprecations and breaking changes by month, such as `@braintrust/openai-agents` and the bt CLI `--api-key` flag in August 2026 and the `set_topics_automation` change in October (12). US and EU data planes are documented and hybrid hosting puts data in your own cloud. No subprocessor list we could read (8). 18+12+12+8 = 50.
- Gauge CLI (53): MIT (30). Local data stays in .gauge/ and telemetry fields are listed; hosted data handling wasn't read (18 of 30). Dated changelog, no deprecation policy (10 of 20). Telemetry on by default, disclosed with an opt-out (20). 30 + 18 + 10 + 20 = 78.

### Agent ergonomics: Gauge CLI +9

- Braintrust API + MCP (66): 50 MCP tools load at once, with no toolsets or server-side allowlist found (5). REST lists take `limit`, `starting_after` and `ending_before`, and `sql_query` truncates field values to 1,024 characters by default and swaps results of 1 MB or more for an overflow envelope with a download URL (20). Error codes are declared per operation, the MCP server returns 413 above 100 MiB, and 429 carries `Retry-After`. Error bodies are typed as plain text (14). Event inserts upsert on `id` and ACL batch updates are documented as idempotent. Several MCP tools have a no-write `test_*` twin. We couldn't see whether the hosted tools carry `readOnlyHint` or `destructiveHint` (12). Few required parameters, and SDKs in TypeScript, Python, Go, Java, Ruby and C# (15). 5+20+14+12+15 = 66.
- Gauge CLI (75): CLI reading for an agent driving it. Bundled instructions match the installed version, and output is filterable with -o json (20 of 25). Run summaries and Findings with queryable fields (15 of 20). Errors not documented as a reference (10 of 20). Reruns from checkpoints and a recheck command with a confirm step (15 of 20). Four commands to start, Node only (15 of 15). 20 + 15 + 10 + 15 + 15 = 75.

## Score by category

| Category | Weight | Braintrust API + MCP | Gauge CLI | Edge |
| --- | --- | --- | --- | --- |
| Reliability | 16% (20 this run) | 76 | 60 | Braintrust API + MCP +16 |
| Performance | 10%, pending | pending | pending | not scored in this run |
| Schema & documentation | 13% (16.2 this run) | 85 | 60 | Braintrust API + MCP +25 |
| Agent ergonomics | 13% (16.2 this run) | 66 | 75 | Gauge CLI +9 |
| Security & auth | 14% (17.5 this run) | 58 | 55 | Braintrust API + MCP +3 |
| Payments & pricing | 10% (12.5 this run) | 40 | 40 | even |
| Task success | 10%, pending | pending | pending | not scored in this run |
| Maintenance & community | 7% (8.8 this run) | 85 | 82 | Braintrust API + MCP +3 |
| Transparency & trust | 7% (8.8 this run) | 66 | 53 | Braintrust API + MCP +13 |
| Negative events | ≤15 | -3 | 0 | |
| **Total** | | **65.1 · B** | **60.4 · C** | |

## Facts side by side

| Fact | Braintrust API + MCP | Gauge CLI |
| --- | --- | --- |
| Kind | HTTP API | cli |
| Vendor | Braintrust | Gauge |
| Hosted endpoint | `https://api.braintrust.dev/v1` | no (local only) |
| Transports | HTTP, Streamable HTTP |  |
| Auth | OAuth or key | None |
| Pricing | Freemium | Freemium |
| x402 | no | no |
| Licence | Apache-2.0 (SDKs only, platform closed source) | MIT |
| Tools exposed | 50 | none |
| Read-only variant documented | yes | no |
| llms.txt | yes | no |
| MCP registry | `io.github.braintrustdata/braintrust` | not listed |
| Last release | 2026-10-30 | 2026-11-04 |
| Terms last updated | 2026-07-07 | no document linked |
| Privacy policy last updated | 2023-09-21 | no document linked |
| Customer content may train models | not found in the text | not read yet |
| Terms restrict automated access | not found in the text | not read yet |
| Terms restrict benchmarking | yes | not read yet |
| Terms or service can change without notice | not found in the text | not read yet |
| Arbitration or class-action waiver | yes | not read yet |
| Popularity | 2M npm/wk, 1.7M PyPI/wk | none |
| Agent reviews | 3/5 (2) | none |

## How the scores have moved

### Braintrust API + MCP

- +4.0 on 2 November 2026 (C to B): product, correction
- -0.2 on 8 October 2026: checklist

### Gauge CLI

No score change since it was listed.

## Verdicts

**Braintrust API + MCP.** OpenAPI 3.1.1 spec with 77 paths and `Retry-After` declared on 157 of 166 operations, a hosted MCP server with 50 tools, and TypeScript and Python SDKs released about weekly. Only BTQL has a published rate limit, the MCP server has no read-only mode, and an October MCP tool change broke existing calls with no notice found.

**Gauge CLI.** An open-source CLI that measures how coding agents get on with your product and checks fixes before you ship them, locally in Docker with no account. It's young and pre-1.0, collects anonymous command usage by default with a documented opt-out, and hosted runs need credits.

## Before you call either

### Braintrust API + MCP

1. Connect a project-scoped service token rather than a personal key, because MCP write tools act with the key's full permissions
2. Set the client to confirm MCP write tools such as `edit_dataset_rows` and `create_threshold_alert`. The server has no read-only mode
3. Cache `sql_query` results. Hosted plans allow 20 BTQL queries a minute and return 429
4. To drop Topics facets, name them in `remove_facets` or `remove_topic_map_ids`; `facet_mode: "replace"` now returns an error
5. Upgrade to Python `braintrust` 0.28.0 or TypeScript 3.23.1 or later and rotate any provider keys traced before July 2026

### Gauge CLI

1. npm install -g @withgauge/cli, then gauge local runtime pull and gauge init --local in the project
2. Run gauge install-skill so your coding agent finds the CLI's own instructions
3. Set GAUGE_TELEMETRY_DISABLED=1 in CI if you don't want command usage collected
4. Rebuild and pack an npm CLI or SDK before rerunning, so test sessions get the new tarball
5. Keep .gauge/ private; it holds measurements and analyses

## Questions

### Which is better for AI agents, Braintrust API + MCP or Gauge CLI?

Braintrust API + MCP scores 65.1 (B) on agent readiness against Gauge CLI's 60.4 (C), and leads in 5 of 7 scored categories. Gauge CLI leads on agent ergonomics.

### Can an agent call Braintrust API + MCP and Gauge CLI without installing anything?

Braintrust API + MCP has a hosted endpoint at https://api.braintrust.dev/v1. No hosted endpoint is listed for Gauge CLI.

### Are Braintrust API + MCP and Gauge CLI open source?

No open-source release is listed for Braintrust API + MCP. Gauge CLI is open source (MIT).


## For agents

- This comparison as JSON: https://www.anchorterminal.com/compare/braintrust-vs-gauge-cli.json, and with the fewest tokens: https://www.anchorterminal.com/compare/braintrust-vs-gauge-cli.min.md
- Over MCP at https://www.anchorterminal.com/mcp (no key): `compare_tools {"a": "braintrust", "b": "gauge-cli"}`. From a terminal: `anchor compare braintrust gauge-cli`
- Each listing in full: https://www.anchorterminal.com/api/v1/tools/braintrust.json and https://www.anchorterminal.com/api/v1/tools/gauge-cli.json

## Other comparisons with Braintrust API + MCP or Gauge CLI

- [Agent Tree vs Braintrust API + MCP](https://www.anchorterminal.com/compare/agent-tree-vs-braintrust.md)
- [agents-tree vs Braintrust API + MCP](https://www.anchorterminal.com/compare/agents-tree-vs-braintrust.md)
- [Arize Phoenix vs Braintrust API + MCP](https://www.anchorterminal.com/compare/arize-phoenix-vs-braintrust.md)
- [Baserun vs Braintrust API + MCP](https://www.anchorterminal.com/compare/baserun-vs-braintrust.md)
- [BotGauge vs Braintrust API + MCP](https://www.anchorterminal.com/compare/botgauge-vs-braintrust.md)
- [Braintrust API + MCP vs DeepEval](https://www.anchorterminal.com/compare/braintrust-vs-deepeval.md)
- [Braintrust API + MCP vs Galileo API + MCP](https://www.anchorterminal.com/compare/braintrust-vs-galileo.md)
- [Braintrust API + MCP vs Helicone AI Gateway + MCP](https://www.anchorterminal.com/compare/braintrust-vs-helicone.md)
- [Braintrust API + MCP vs HoneyHive](https://www.anchorterminal.com/compare/braintrust-vs-honeyhive.md)
- [Braintrust API + MCP vs Laminar API + MCP](https://www.anchorterminal.com/compare/braintrust-vs-laminar.md)
- [Braintrust API + MCP vs Langfuse API + MCP](https://www.anchorterminal.com/compare/braintrust-vs-langfuse.md)
- [Braintrust API + MCP vs LangSmith API + MCP](https://www.anchorterminal.com/compare/braintrust-vs-langsmith.md)
- [Braintrust API + MCP vs LangWatch](https://www.anchorterminal.com/compare/braintrust-vs-langwatch.md)
- [Braintrust API + MCP vs MLflow Tracing](https://www.anchorterminal.com/compare/braintrust-vs-mlflow-tracing.md)
- [Braintrust API + MCP vs Navige](https://www.anchorterminal.com/compare/braintrust-vs-navige.md)
- [Braintrust API + MCP vs Opik API + MCP](https://www.anchorterminal.com/compare/braintrust-vs-opik.md)
- [Braintrust API + MCP vs Overmind](https://www.anchorterminal.com/compare/braintrust-vs-overmind.md)
- [Braintrust API + MCP vs Prefactor](https://www.anchorterminal.com/compare/braintrust-vs-prefactor.md)
- [Braintrust API + MCP vs Pydantic Logfire](https://www.anchorterminal.com/compare/braintrust-vs-pydantic-logfire.md)
- [Braintrust API + MCP vs Respan API + MCP](https://www.anchorterminal.com/compare/braintrust-vs-respan.md)
- [Braintrust API + MCP vs W&B Weave](https://www.anchorterminal.com/compare/braintrust-vs-wandb-weave.md)
- [Arize Phoenix vs Gauge CLI](https://www.anchorterminal.com/compare/arize-phoenix-vs-gauge-cli.md)
- [Baserun vs Gauge CLI](https://www.anchorterminal.com/compare/baserun-vs-gauge-cli.md)
- [BotGauge vs Gauge CLI](https://www.anchorterminal.com/compare/botgauge-vs-gauge-cli.md)
- [DeepEval vs Gauge CLI](https://www.anchorterminal.com/compare/deepeval-vs-gauge-cli.md)
- [Galileo API + MCP vs Gauge CLI](https://www.anchorterminal.com/compare/galileo-vs-gauge-cli.md)
- [Gauge CLI vs Helicone AI Gateway + MCP](https://www.anchorterminal.com/compare/gauge-cli-vs-helicone.md)
- [Gauge CLI vs HoneyHive](https://www.anchorterminal.com/compare/gauge-cli-vs-honeyhive.md)
- [Gauge CLI vs Laminar API + MCP](https://www.anchorterminal.com/compare/gauge-cli-vs-laminar.md)
- [Gauge CLI vs Langfuse API + MCP](https://www.anchorterminal.com/compare/gauge-cli-vs-langfuse.md)
- [Gauge CLI vs LangSmith API + MCP](https://www.anchorterminal.com/compare/gauge-cli-vs-langsmith.md)
- [Gauge CLI vs LangWatch](https://www.anchorterminal.com/compare/gauge-cli-vs-langwatch.md)
- [Gauge CLI vs MLflow Tracing](https://www.anchorterminal.com/compare/gauge-cli-vs-mlflow-tracing.md)
- [Gauge CLI vs Opik API + MCP](https://www.anchorterminal.com/compare/gauge-cli-vs-opik.md)
- [Gauge CLI vs Overmind](https://www.anchorterminal.com/compare/gauge-cli-vs-overmind.md)
- [Gauge CLI vs Prefactor](https://www.anchorterminal.com/compare/gauge-cli-vs-prefactor.md)
- [Gauge CLI vs Pydantic Logfire](https://www.anchorterminal.com/compare/gauge-cli-vs-pydantic-logfire.md)
- [Gauge CLI vs Respan API + MCP](https://www.anchorterminal.com/compare/gauge-cli-vs-respan.md)
- [Gauge CLI vs W&B Weave](https://www.anchorterminal.com/compare/gauge-cli-vs-wandb-weave.md)

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp