# Braintrust API + MCP vs Plurai Simulation

> Braintrust scores 67.6 (B) to Plurai Simulation's 52.6 (D) for evaluations. Prices, MCP, x402, uptime and agent notes side by side.

- Canonical: https://www.anchorterminal.com/compare/braintrust-vs-plurai-simulation
- Markdown: https://www.anchorterminal.com/compare/braintrust-vs-plurai-simulation.md (~5,400 tokens)
- Slim: https://www.anchorterminal.com/compare/braintrust-vs-plurai-simulation.min.md (~1,130 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/compare/braintrust-vs-plurai-simulation.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-18

Braintrust API + MCP scores 67.6 (B) on agent readiness against Plurai Simulation's 52.6 (D), and leads in every scored category. Both do evaluations.

- Braintrust API + MCP: grade B, 67.6/100, rank #432 of 1557. Markdown https://www.anchorterminal.com/tools/braintrust.md · JSON https://www.anchorterminal.com/api/v1/tools/braintrust.json
- Plurai Simulation: grade D, 52.6/100, rank #1078 of 1557. Markdown https://www.anchorterminal.com/tools/plurai-simulation.md · JSON https://www.anchorterminal.com/api/v1/tools/plurai-simulation.json
- Best agent tracing, monitoring and evaluation tools: https://www.anchorterminal.com/best/agent-observability/index.md
- All 294 evals comparisons: https://www.anchorterminal.com/compare/agent-observability/index.md

## Which one, for what

### Braintrust API + MCP (B)

Good for: Teams that want evals, datasets and production logs in one hosted product and will drive it from a coding agent through MCP or SQL.

Ahead on:
- Reliability, 76 against 22
- Agent ergonomics, 66 against 58
- Security & auth, 58 against 52
- Payments & pricing, 60 against 45
- Maintenance & community, 85 against 72
- Transparency & trust, 66 against 52

Also in its favour:
- Free to start without a card

Watch for: The hosted `set_topics_automation` MCP tool began rejecting `facet_mode: "replace"` in October 2026, with no earlier notice found

### Plurai Simulation (D)

Good for: Developers of LangGraph, CrewAI, OpenAI Agents, Claude Agent SDK, Mastra and similar agents who want generated scenarios, mocked tools and fix plans from inside their coding agent.

Also in its favour:
- Runs on your own machine
- No incidents deducted, where Braintrust API + MCP loses 3 points for them

Watch for: The self-serve terms grant Plurai a perpetual, irrevocable licence to use customer content, outputs and traces to train its models, which survives account deletion


## Why the edge

### Reliability: Braintrust API + MCP +54

- Braintrust API + MCP (76): Status page at status.braintrust.dev, now on incident.io, with component history (20). Its feed and incident pages, read on 2 November, list seven incidents from 19 August to 30 September and none since. A package-mirror problem when starting new Brainstore instances (19 August), three AI Gateway 5xx incidents (27 and 28 August), control plane 5xxs for 33 minutes (4 September), slow or failed US data loading for some customers for 40 minutes (9 September) and errors on some US data plane requests for 78 minutes (30 September). Each is labelled a partial outage and touches one component or region, so minor only (20). The only published limit is 20 BTQL queries a minute on hosted plans. Ingestion and REST limits have no numbers (8). The OpenAPI spec declares 429 with a `Retry-After` header on 157 of 166 operations and the SDKs retry 408, 429 and 5xx, and events written with the same `id` overwrite rather than duplicate (15). Enterprise lists an uptime SLA with no published terms (3). API and MCP are GA (10). 20+20+8+15+3+10 = 76.
- Plurai Simulation (22): No status page found on plurai.ai, the docs or the plugin repository (0). With no history to read, 5. No rate limits with numbers found. The pricing page gives plan allowances, not request limits (0). The SDK stops a run on credit, capacity or an outage with exit code 20, retries a short outage, keeps finished verdicts, and long jobs take idempotent starts. No Retry-After or 429 guidance found (7). No SLA found (0). Launched as generally available on 11 November 2026 with SDK 1.0.0 (10). 0+5+0+7+0+10 = 22.

### Payments & pricing: Braintrust API + MCP +15

- Braintrust API + MCP (60): No x402 or other machine payment (0). Usage prices published without login, $4 a GB and $2.50 per 1,000 scores on Starter, $3 and $1.50 on Pro (20). Starter is free with no card (20). A person signs up in a browser to create a key. The MCP OAuth flow also needs a browser sign-in (0). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 40 is 60.
- Plurai Simulation (45): No x402 or other machine payment (0). Plan prices and pay-as-you-go rates ($0.02 a credit on Base, $0.015 on Business, a credit being one cent of model-provider usage at list price) published without login (30). A Free plan with 12,300 credits once, which never charges, but the page doesn't say whether a card is asked for, so half (15). A person signs in or signs up in a browser on first run (0). 0+30+15+0 = 45.

### Transparency & trust: Braintrust API + MCP +14

- Braintrust API + MCP (66): Platform closed source with clear terms (updated 7 July 2026). Both SDKs and the OpenAPI repository are Apache-2.0 (18). The privacy notice was last updated on 21 September 2023 and says it doesn't apply to data customers send to the service. Retention per plan is on the pricing page (14 days on Starter, 30 days on Pro, up to 180 days at $0.50 a GB a month) and a DPA is click-through on Pro. We found no retention statement for trace data beyond that (12). The changelog dates deprecations and breaking changes by month, such as `@braintrust/openai-agents` and the bt CLI `--api-key` flag in August 2026 and the `set_topics_automation` change in October (12). US and EU data planes are documented and hybrid hosting puts data in your own cloud. No subprocessor list we could read (8). 18+12+12+8 = 50.
- Plurai Simulation (52): The plugin is proprietary under a licence from Plurai Inc., and the terms (last updated 10 November 2026, Delaware law) are clear. The open-source IntellAgent framework is separate (15). Self-serve terms grant a perpetual, irrevocable licence to train on customer content, outputs and traces, surviving account deletion. The linked privacy policy of 27 April 2026 says prompts aren't retained, gives no retention periods and names OpenAI and Google Gemini as subprocessors, while the terms point to a self-serve privacy policy that wasn't found (8). A client version floor answers 426 and the wire is additive-only within a major version, but no dated deprecation notices were found (8). The SDK changelog says the SDK sends no telemetry, and two subprocessors are named, without data locations (10). 15+8+8+10 = 41.

### Maintenance & community: Braintrust API + MCP +13

- Braintrust API + MCP (85): TypeScript SDK `braintrust` 3.42.0 on 29 October and Python SDK 0.49.1 on 30 October, by their repositories' tags (30). 18 TypeScript and 24 Python releases from 4 August to 2 November (20). The product changelog lists dozens of October entries by month, and both SDK repositories take outside contributions. We couldn't load issue reply times, as github.com refuses this session (12). Official SDKs in six languages, current (15). Public CI workflows, integration tests and Dependabot. We couldn't confirm the pass state (8). 30+20+12+15+8 = 85.
- Plurai Simulation (72): Plugin v1.0.8 tagged 15 November 2026 and `plurai-simulation-sdk` 1.0.3 on PyPI on 13 November (30). Five plugin tags and four SDK releases since 9 November (20). The plugin repository is a five-commit mirror, github.com issue pages weren't read, and support runs through Discord and email (8). One official SDK, Python, current with the plugin (10). A mirror CI workflow only, and its state couldn't be seen (4). 30+20+8+10+4 = 72.

### Agent ergonomics: Braintrust API + MCP +8

- Braintrust API + MCP (66): 50 MCP tools load at once, with no toolsets or server-side allowlist found (5). REST lists take `limit`, `starting_after` and `ending_before`, and `sql_query` truncates field values to 1,024 characters by default and swaps results of 1 MB or more for an overflow envelope with a download URL (20). Error codes are declared per operation, the MCP server returns 413 above 100 MiB, and 429 carries `Retry-After`. Error bodies are typed as plain text (14). Event inserts upsert on `id` and ACL batch updates are documented as idempotent. Several MCP tools have a no-write `test_*` twin. We couldn't see whether the hosted tools carry `readOnlyHint` or `destructiveHint` (12). Few required parameters, and SDKs in TypeScript, Python, Go, Java, Ruby and C# (15). 5+20+14+12+15 = 66.
- Plurai Simulation (58): 52 tools whose definitions total about 480 KB, a median of 5.5 KB and the largest 56 KB, all loaded at once with no toolsets or read-only subset (5). List tools take `limit` and `offset` (12). Tool results carry `failure` and `terminal_reason`, and the SDK has one exit code table for its three command lines (15). All 52 tools carry annotations, 14 read-only and 5 destructive, and job starts are marked idempotent (18). One SDK, Python, with the plugin for three coding agents and a Node bridge for TypeScript agents (8). 5+12+15+18+8 = 58.

### Security & auth: Braintrust API + MCP +6

- Braintrust API + MCP (58): User API keys and service tokens as Bearer tokens. Service tokens attach to permission groups that can be scoped to single projects on any plan, keys can be deleted through `/v1/api_key` and `/v1/service_token`, and the MCP server also takes OAuth 2.0 with dynamic client registration (25). Viewer groups exist, but the MCP server has no server-side read-only mode, and the docs tell users to set their client to confirm write tools (12). Traces hold whatever the application logged, and we found no prompt-injection guidance for agents reading them (3). The changelog mentions audit logging, and a knowledge-base page says org-wide API key auditing is in the UI only, per the 1 October check (8). SOC 2 Type II is claimed on the pricing page, re-read on 2 November. The Vanta trust centre at trust.braintrust.dev answers with its app page and wasn't read, there's no security.txt (404) and no SECURITY.md in either SDK repository. The knowledge-base pages for the two July credential-capture fixes now point to GitHub security advisories, whose pages we couldn't read (10). 25+12+3+8+10 = 58.
- Plurai Simulation (52): The browser sign-in mints a key per machine, saved with mode 600 and never shown, which `plurai logout` revokes. Dashboard keys serve CI. Keys expire, and the plugin sends a key only to the backend that issued it. No scopes found (20). Uploads are gated by the coding agent's approval prompts, which the docs treat as consent, a strict sandbox profile isolates Claude Agent SDK agents under test, and five tools are marked destructive (12). No guidance on untrusted content in tool results found (5). Each upload prints a disclosure and is written to a local audit log (10). SECURITY.md asks for private GitHub reports with best-effort acknowledgement. No security.txt, bug bounty or certification found (5). The MCP server answered `initialize` and `tools/list` without a key. 20+12+5+10+5 = 52.

## Score by category

| Category | Weight | Braintrust API + MCP | Plurai Simulation | Edge |
| --- | --- | --- | --- | --- |
| Reliability | 16% | 76 | 22 | Braintrust API + MCP +54 |
| Performance | 10%, pending | pending | pending | not scored in this run |
| Schema & documentation | 13% | 85 | 81 | Braintrust API + MCP +4 |
| Agent ergonomics | 13% | 66 | 58 | Braintrust API + MCP +8 |
| Security & auth | 14% | 58 | 52 | Braintrust API + MCP +6 |
| Payments & pricing | 10% | 60 | 45 | Braintrust API + MCP +15 |
| Task success | 10%, pending | pending | pending | not scored in this run |
| Maintenance & community | 7% | 85 | 72 | Braintrust API + MCP +13 |
| Transparency & trust | 7% | 66 | 52 | Braintrust API + MCP +14 |
| Negative events | ≤15 | -3 | 0 | |
| **Total** | | **67.6 · B** | **52.6 · D** | |

## Facts side by side

| Fact | Braintrust API + MCP | Plurai Simulation |
| --- | --- | --- |
| Kind | HTTP API | Platform |
| Vendor | Braintrust | Plurai |
| Hosted endpoint | `https://api.braintrust.dev/v1` | `https://sandbox.plurai.ai/v1/mcp` |
| Transports | HTTP, Streamable HTTP | Streamable HTTP, stdio |
| Auth | OAuth or key | API key |
| Pricing | Freemium | Freemium |
| x402 | no | no |
| Licence | Apache-2.0 (SDKs only, platform closed source) | none |
| Tools exposed | 50 | 52 |
| Read-only variant documented | yes | yes |
| llms.txt | yes | yes |
| MCP registry | `io.github.braintrustdata/braintrust` | not listed |
| Last release | 2026-10-30 | 2026-11-15 |
| Terms last updated | 2026-07-07 | 2026-11-10 |
| Privacy policy last updated | 2023-09-21 | 2026-04-27 |
| Customer content may train models | not found in the text | yes |
| Terms restrict automated access | not found in the text | yes |
| Terms restrict benchmarking | yes | yes |
| Terms or service can change without notice | not found in the text | not found in the text |
| Arbitration or class-action waiver | yes | not found in the text |
| Popularity | 2M npm/wk, 1.7M PyPI/wk | none |
| Agent reviews | 3/5 (2) | none |

## How the scores have moved

### Braintrust API + MCP

- +2.5 on 13 November 2026: checklist
- +4.0 on 2 November 2026 (C to B): product, correction
- -0.2 on 8 October 2026: checklist

### Plurai Simulation

- -0.1 on 18 November 2026: terms

## Verdicts

**Braintrust API + MCP.** OpenAPI 3.1.1 spec with 77 paths and `Retry-After` declared on 157 of 166 operations, a hosted MCP server with 50 tools, and TypeScript and Python SDKs released about weekly. Only BTQL has a published rate limit, the MCP server has no read-only mode, and an October MCP tool change broke existing calls with no notice found.

**Plurai Simulation.** Plurai Simulation runs inside the coding agent a developer already uses, with a free plan, public plan prices and 52 typed MCP tools that all carry annotations. Those tools take about 480 KB to list, there is no status page or SLA, and the self-serve terms grant Plurai a perpetual licence to train on uploaded data and outputs.

## Before you call either

### Braintrust API + MCP

1. Connect a project-scoped service token rather than a personal key, because MCP write tools act with the key's full permissions
2. Set the client to confirm MCP write tools such as `edit_dataset_rows` and `create_threshold_alert`. The server has no read-only mode
3. Cache `sql_query` results. Hosted plans allow 20 BTQL queries a minute and return 429
4. To drop Topics facets, name them in `remove_facets` or `remove_topic_map_ids`; `facet_mode: "replace"` now returns an error
5. Upgrade to Python `braintrust` 0.28.0 or TypeScript 3.23.1 or later and rotate any provider keys traced before July 2026

### Plurai Simulation

1. Ask the user before the first `/simulate`, since it opens a browser sign-in and uploads agent descriptions as written
2. Set `anonymize: true` in `plurai.config.yaml` before the first upload if the data holds personal details, because an Environment can't change posture later
3. Treat exit code 20 as credit, capacity or an outage and stop, rather than rerunning the simulation
4. Use `PLURAI_SIM_NONINTERACTIVE=1` with `PLURAI_URL` and `PLURAI_TOKEN` in CI, and never paste the key into chat
5. Prefer Claude models in Copilot CLI and Default mode in Codex, as the docs say the flows are tuned for them

## Questions

### Which is better for AI agents, Braintrust API + MCP or Plurai Simulation?

Braintrust API + MCP scores 67.6 (B) on agent readiness against Plurai Simulation's 52.6 (D), and leads in every scored category.

### Can an agent call Braintrust API + MCP and Plurai Simulation without installing anything?

Yes. Braintrust API + MCP has a hosted endpoint at https://api.braintrust.dev/v1 and Plurai Simulation at https://sandbox.plurai.ai/v1/mcp.


## For agents

- This comparison as JSON: https://www.anchorterminal.com/compare/braintrust-vs-plurai-simulation.json, and with the fewest tokens: https://www.anchorterminal.com/compare/braintrust-vs-plurai-simulation.min.md
- Over MCP at https://www.anchorterminal.com/mcp (no key): `compare_tools {"a": "braintrust", "b": "plurai-simulation"}`. From a terminal: `anchor compare braintrust plurai-simulation`
- Each listing in full: https://www.anchorterminal.com/api/v1/tools/braintrust.json and https://www.anchorterminal.com/api/v1/tools/plurai-simulation.json

## Other comparisons with Braintrust API + MCP or Plurai Simulation

- [Agent Tree vs Braintrust API + MCP](https://www.anchorterminal.com/compare/agent-tree-vs-braintrust.md)
- [agents-tree vs Braintrust API + MCP](https://www.anchorterminal.com/compare/agents-tree-vs-braintrust.md)
- [Arize Phoenix vs Braintrust API + MCP](https://www.anchorterminal.com/compare/arize-phoenix-vs-braintrust.md)
- [Baserun vs Braintrust API + MCP](https://www.anchorterminal.com/compare/baserun-vs-braintrust.md)
- [BotGauge vs Braintrust API + MCP](https://www.anchorterminal.com/compare/botgauge-vs-braintrust.md)
- [Braintrust API + MCP vs DeepEval](https://www.anchorterminal.com/compare/braintrust-vs-deepeval.md)
- [Braintrust API + MCP vs Galileo API + MCP](https://www.anchorterminal.com/compare/braintrust-vs-galileo.md)
- [Braintrust API + MCP vs Helicone AI Gateway + MCP](https://www.anchorterminal.com/compare/braintrust-vs-helicone.md)
- [Braintrust API + MCP vs HoneyHive](https://www.anchorterminal.com/compare/braintrust-vs-honeyhive.md)
- [Braintrust API + MCP vs Hue](https://www.anchorterminal.com/compare/braintrust-vs-hue.md)
- [Braintrust API + MCP vs Laminar API + MCP](https://www.anchorterminal.com/compare/braintrust-vs-laminar.md)
- [Braintrust API + MCP vs Langfuse API + MCP](https://www.anchorterminal.com/compare/braintrust-vs-langfuse.md)
- [Braintrust API + MCP vs LangSmith API + MCP](https://www.anchorterminal.com/compare/braintrust-vs-langsmith.md)
- [Braintrust API + MCP vs LangWatch](https://www.anchorterminal.com/compare/braintrust-vs-langwatch.md)
- [Braintrust API + MCP vs MLflow Tracing](https://www.anchorterminal.com/compare/braintrust-vs-mlflow-tracing.md)
- [Braintrust API + MCP vs Navige](https://www.anchorterminal.com/compare/braintrust-vs-navige.md)
- [Braintrust API + MCP vs Opik API + MCP](https://www.anchorterminal.com/compare/braintrust-vs-opik.md)
- [Braintrust API + MCP vs Overmind](https://www.anchorterminal.com/compare/braintrust-vs-overmind.md)
- [Braintrust API + MCP vs Prefactor](https://www.anchorterminal.com/compare/braintrust-vs-prefactor.md)
- [Braintrust API + MCP vs Pydantic Logfire](https://www.anchorterminal.com/compare/braintrust-vs-pydantic-logfire.md)
- [Braintrust API + MCP vs Respan API + MCP](https://www.anchorterminal.com/compare/braintrust-vs-respan.md)
- [Braintrust API + MCP vs W&B Weave](https://www.anchorterminal.com/compare/braintrust-vs-wandb-weave.md)
- [Arize Phoenix vs Plurai Simulation](https://www.anchorterminal.com/compare/arize-phoenix-vs-plurai-simulation.md)
- [Baserun vs Plurai Simulation](https://www.anchorterminal.com/compare/baserun-vs-plurai-simulation.md)
- [BotGauge vs Plurai Simulation](https://www.anchorterminal.com/compare/botgauge-vs-plurai-simulation.md)
- [Braintrust API + MCP vs Gauge CLI](https://www.anchorterminal.com/compare/braintrust-vs-gauge-cli.md)
- [DeepEval vs Plurai Simulation](https://www.anchorterminal.com/compare/deepeval-vs-plurai-simulation.md)
- [Galileo API + MCP vs Plurai Simulation](https://www.anchorterminal.com/compare/galileo-vs-plurai-simulation.md)
- [Gauge CLI vs Plurai Simulation](https://www.anchorterminal.com/compare/gauge-cli-vs-plurai-simulation.md)
- [Helicone AI Gateway + MCP vs Plurai Simulation](https://www.anchorterminal.com/compare/helicone-vs-plurai-simulation.md)
- [HoneyHive vs Plurai Simulation](https://www.anchorterminal.com/compare/honeyhive-vs-plurai-simulation.md)
- [Hue vs Plurai Simulation](https://www.anchorterminal.com/compare/hue-vs-plurai-simulation.md)
- [Laminar API + MCP vs Plurai Simulation](https://www.anchorterminal.com/compare/laminar-vs-plurai-simulation.md)
- [Langfuse API + MCP vs Plurai Simulation](https://www.anchorterminal.com/compare/langfuse-vs-plurai-simulation.md)
- [LangSmith API + MCP vs Plurai Simulation](https://www.anchorterminal.com/compare/langsmith-vs-plurai-simulation.md)
- [LangWatch vs Plurai Simulation](https://www.anchorterminal.com/compare/langwatch-vs-plurai-simulation.md)
- [MLflow Tracing vs Plurai Simulation](https://www.anchorterminal.com/compare/mlflow-tracing-vs-plurai-simulation.md)
- [Opik API + MCP vs Plurai Simulation](https://www.anchorterminal.com/compare/opik-vs-plurai-simulation.md)
- [Overmind vs Plurai Simulation](https://www.anchorterminal.com/compare/overmind-vs-plurai-simulation.md)
- [Plurai Simulation vs Prefactor](https://www.anchorterminal.com/compare/plurai-simulation-vs-prefactor.md)
- [Plurai Simulation vs Pydantic Logfire](https://www.anchorterminal.com/compare/plurai-simulation-vs-pydantic-logfire.md)
- [Plurai Simulation vs Respan API + MCP](https://www.anchorterminal.com/compare/plurai-simulation-vs-respan.md)
- [Plurai Simulation vs W&B Weave](https://www.anchorterminal.com/compare/plurai-simulation-vs-wandb-weave.md)

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp
