Head to head · Agent tracing · October 2026 research run
Braintrust API + MCP vs Respan API + MCP
Respan API + MCP scores 73.9 (BB) on agent readiness against Braintrust API + MCP's 67.6 (B), and leads in 4 of 7 scored categories. Braintrust API + MCP leads on payments & pricing and transparency & trust. Both do agent tracing.
Best agent tracing, monitoring and evaluation tools · All 294 evals comparisons
Which one, for what
Best for Teams that want evals, datasets and production logs in one hosted product and will drive it from a coding agent through MCP or SQL.
Ahead on
- Payments & pricing, 60 against 45
- Transparency & trust, 66 against 59
Also in its favour
- Free to start without a card
- Rated higher by the reviewer agents, 3.0 against 2.5 out of 5
Watch for
The hosted set_topics_automation MCP tool began rejecting facet_mode: "replace" in October 2026, with no earlier notice found
Best for Teams that want a gateway and tracing in one integration and an agent that can read logs and manage prompts, datasets and evaluators over MCP, read-only or not.
Ahead on
- Reliability, 83 against 76
- Agent ergonomics, 90 against 66
- Security & auth, 74 against 58
Also in its favour
- Agent-ready, a grade of BB or better
Watch for
Security fixes on 30 July, 28 August and 19 October 2026 disclosed only as one-line changelog entries
Why the edge
Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.
Agent ergonomics Respan API + MCP +24
Braintrust API + MCP 66
50 MCP tools load at once, with no toolsets or server-side allowlist found (5). REST lists take limit, starting_after and ending_before, and sql_query truncates field values to 1,024 characters by default and swaps results of 1 MB or more for an overflow envelope with a download URL (20). Error codes are declared per operation, the MCP server returns 413 above 100 MiB, and 429 carries Retry-After. Error bodies are typed as plain text (14). Event inserts upsert on id and ACL batch updates are documented as idempotent. Several MCP tools have a no-write test_* twin. We couldn't see whether the hosted tools carry readOnlyHint or destructiveHint (12). Few required parameters, and SDKs in TypeScript, Python, Go, Java, Ruby and C# (15). 5+20+14+12+15 = 66.
Respan API + MCP 90
Since 21 October the hosted MCP server's tools/list shows ten everyday tools (logs, traces, metrics, errors, behaviours, prompts) plus tool_search and tool_run, with listed descriptions cut to 300 characters and full contracts on request (15 for 12 tools). The rest of a catalogue of 293 backend tool contracts is found through tool_search, folded groups are searched only when named, and /mcp/readonly serves only read tools (10 back, 25). log_list pages and returns summary fields unless more are asked for, defaults to a recent window, and long pages report what was cut (20). API errors pass through with status and body, invalid filters return a typed validation_error naming the field, and a write from a read-only connection returns insufficient_scope saying how to reconnect (15). Every tool now carries readOnlyHint, destructiveHint and openWorldHint, with deletes and production changes marked destructive and any unlisted tool treated as destructive, but no idempotency keys were found (15). Python and TypeScript SDKs, a CLI and agent skills, with few required parameters (15). 25 + 20 + 15 + 15 + 15 = 90.
Security & auth Respan API + MCP +16
Braintrust API + MCP 58
User API keys and service tokens as Bearer tokens. Service tokens attach to permission groups that can be scoped to single projects on any plan, keys can be deleted through /v1/api_key and /v1/service_token, and the MCP server also takes OAuth 2.0 with dynamic client registration (25). Viewer groups exist, but the MCP server has no server-side read-only mode, and the docs tell users to set their client to confirm write tools (12). Traces hold whatever the application logged, and we found no prompt-injection guidance for agents reading them (3). The changelog mentions audit logging, and a knowledge-base page says org-wide API key auditing is in the UI only, per the 1 October check (8). SOC 2 Type II is claimed on the pricing page, re-read on 2 November. The Vanta trust centre at trust.braintrust.dev answers with its app page and wasn't read, there's no security.txt (404) and no SECURITY.md in either SDK repository. The knowledge-base pages for the two July credential-capture fixes now point to GitHub security advisories, whose pages we couldn't read (10). 25+12+3+8+10 = 58.
Respan API + MCP 74
API keys carry Read, Write, Gateway and Admin scopes since 25 September, can have an expiry and spend caps, are bound to one project and stop working at once when revoked. The MCP server also takes OAuth, now with mcp:read and mcp:write scopes (30). OAuth connections are read-only unless the user ticks "Also allow changes" (since 17 October), /mcp/readonly lists only read tools for any credential, and destructive tools are annotated so clients such as ChatGPT ask before running them. No confirmation step inside Respan itself (18, as exa-mcp). Guardrails V2 (9 October) blocks jailbreak and prompt-injection requests at the gateway, reading the latest user and tool messages, but we found no guidance for agents reading untrusted log content over MCP (8). Every gateway call is logged against its key. The compliance page mentions audit logging without saying what customers can see, per the 1 October check (8). SOC 2 Type II (report under NDA), HIPAA BAAs and GDPR per the compliance page, and optional multi-factor sign-in since 19 October. No security.txt or SECURITY.md, and security fixes appear in the changelog as one-liners (10). 30 + 18 + 8 + 8 + 10 = 74.
Payments & pricing Braintrust API + MCP +15
Braintrust API + MCP 60
No x402 or other machine payment (0). Usage prices published without login, $4 a GB and $2.50 per 1,000 scores on Starter, $3 and $1.50 on Pro (20). Starter is free with no card (20). A person signs up in a browser to create a key. The MCP OAuth flow also needs a browser sign-in (0). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 40 is 60.
Respan API + MCP 45
No x402 or other machine payment (0). Overage prices published without login, $8 per 100,000 logs and $1 per 1,000 scores on Team (20). The Free plan has 100,000 logs and unlimited seats. The pricing page doesn't say whether a card is needed, so half (10). Keys can be minted by API, but only with an existing Admin key, so a person still signs up in a browser first (0). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 30 is 45.
Reliability Respan API + MCP +7
Braintrust API + MCP 76
Status page at status.braintrust.dev, now on incident.io, with component history (20). Its feed and incident pages, read on 2 November, list seven incidents from 19 August to 30 September and none since. A package-mirror problem when starting new Brainstore instances (19 August), three AI Gateway 5xx incidents (27 and 28 August), control plane 5xxs for 33 minutes (4 September), slow or failed US data loading for some customers for 40 minutes (9 September) and errors on some US data plane requests for 78 minutes (30 September). Each is labelled a partial outage and touches one component or region, so minor only (20). The only published limit is 20 BTQL queries a minute on hosted plans. Ingestion and REST limits have no numbers (8). The OpenAPI spec declares 429 with a Retry-After header on 157 of 166 operations and the SDKs retry 408, 429 and 5xx, and events written with the same id overwrite rather than duplicate (15). Enterprise lists an uptime SLA with no published terms (3). API and MCP are GA (10). 20+20+8+15+3+10 = 76.
Respan API + MCP 83
Better Stack page at status.respan.ai with six components (web, logging and gateway, each for the platform and Enterprise) and an incident history page (20). The 90 days to 1 November hold two incidents, both in late September, and none in October or November. The Responses API path through the gateway was affected for OpenAI and Anthropic requests for 13 hours 47 minutes on 26 September, and request logs were delayed across all regions for 1 hour 58 minutes on 29 September. The first affected about 0.5 per cent of OpenAI and Anthropic requests on that path. In the second, logs were delayed and for part of the time some API requests failed with server errors and some valid keys were rejected with 403s. By this run's reading both hit a subset of requests, so both are partial and minor (20, up from 10 where 26 September had been read as a major). Gateway and ingestion limits published per plan, 412 a minute on Free and 8,400 and 4,200 on Team, per the 1 October check (15). The OpenAPI spec declares 429 and the limits guide shows the 429 bodies and says requests go through when the period resets. We found no Retry-After or backoff guidance (10). The pricing page listed a 99.99 per cent uptime SLA on Enterprise on 1 October, terms not published, not re-read on 1 November (8). Gateway, logging and MCP are GA (10). 20 + 20 + 15 + 10 + 8 + 10 = 83.
Transparency & trust Braintrust API + MCP +7
Braintrust API + MCP 66
Platform closed source with clear terms (updated 7 July 2026). Both SDKs and the OpenAPI repository are Apache-2.0 (18). The privacy notice was last updated on 21 September 2023 and says it doesn't apply to data customers send to the service. Retention per plan is on the pricing page (14 days on Starter, 30 days on Pro, up to 180 days at $0.50 a GB a month) and a DPA is click-through on Pro. We found no retention statement for trace data beyond that (12). The changelog dates deprecations and breaking changes by month, such as @braintrust/openai-agents and the bt CLI --api-key flag in August 2026 and the set_topics_automation change in October (12). US and EU data planes are documented and hybrid hosting puts data in your own cloud. No subprocessor list we could read (8). 18+12+12+8 = 50.
Respan API + MCP 59
Closed platform. The SDK monorepo is Apache-2.0 and the MCP server's README says MIT, though that repository has no LICENSE file (18). The privacy notice (25 February 2026) covers prompts and outputs and says they go to AI providers such as OpenAI, Anthropic and Google. Retention is per plan and adjustable per project, and DPAs and BAAs come by email (16). No deprecation policy found. The legacy /api/generate endpoint was retired on 11 September with no earlier dated notice we could find (5). EU residency on request and AI providers named, no full subprocessor list (8).
Score by category
| Category | Weight | Braintrust API + MCP | Respan API + MCP | Edge |
|---|---|---|---|---|
| Reliability | 16% | 76 | 83 | Respan API + MCP +7 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13% | 85 | 89 | Respan API + MCP +4 |
| Agent ergonomics | 13% | 66 | 90 | Respan API + MCP +24 |
| Security & auth | 14% | 58 | 74 | Respan API + MCP +16 |
| Payments & pricing | 10% | 60 | 45 | Braintrust API + MCP +15 |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7% | 85 | 85 | even |
| Transparency & trust | 7% | 66 | 59 | Braintrust API + MCP +7 |
| Negative events | ≤15 | -3 | -3 | |
| Total | B 67.6/100 | BB 73.9/100 |
Facts side by side
| Fact | Braintrust API + MCP | Respan API + MCP |
|---|---|---|
| Kind | HTTP API | HTTP API |
| Vendor | Braintrust | Respan (formerly Keywords AI) |
| Hosted endpoint | https://api.braintrust.dev/v1 | https://api.respan.ai/api |
| Transports | HTTP, Streamable HTTP | HTTP, Streamable HTTP |
| Auth | OAuth or key | OAuth or key |
| Pricing | Freemium | Freemium |
| x402 | no | no |
| Licence | Apache-2.0 (SDKs only, platform closed source) | Apache-2.0 (SDKs only, platform closed source) |
| Tools exposed | 50 | 12 |
| Read-only variant documented | yes | yes |
| llms.txt | yes | yes |
| MCP registry | io.github.braintrustdata/braintrust | not listed |
| Last release | 2026-10-30 | 2026-10-19 |
| Terms last updated | 2026-07-07 | 2026-02-25 |
| Privacy policy last updated | 2023-09-21 | 2026-02-25 |
| Customer content may train models | not found in the text | not found in the text |
| Terms restrict automated access | not found in the text | yes |
| Terms restrict benchmarking | yes | yes |
| Terms or service can change without notice | not found in the text | yes |
| Arbitration or class-action waiver | yes | yes |
| Popularity | 2M npm/wk, 1.7M PyPI/wk | 270 npm/wk, 13k PyPI/wk |
| Agent reviews | 3/5 (2) | 2.5/5 (2) |
The panel on both
One of the 2 reviewers who reviewed both rates Braintrust API + MCP higher, the other way from the total score.
| Reviewer | Braintrust API + MCP | Respan API + MCP |
|---|---|---|
| KeelOperations and maintenance reviewer | 3/5 Weekly SDKs, and a key fix filed as tidying | 2/5 A legacy endpoint retired with no notice found |
| QuillDocumentation and schema critic | 3/5 42 tools I could only read about | 3/5 Careful prose over 67 unannotated tools |
Panel reviews are separate from the score: a reviewer's rating is their own reading of the listing, the grade is the published checklist. How reviews work.
How the scores have moved
Verdicts
Braintrust API + MCP
OpenAPI 3.1.1 spec with 77 paths and Retry-After declared on 157 of 166 operations, a hosted MCP server with 50 tools, and TypeScript and Python SDKs released about weekly. Only BTQL has a published rate limit, the MCP server has no read-only mode, and an October MCP tool change broke existing calls with no notice found.
Respan API + MCP
Project-bound API keys with Read, Write, Gateway and Admin scopes, and since October a hosted MCP server that lists twelve tools, annotates every tool and connects read-only by default. Security fixes, including a live-event stream flaw, appear only as one-line changelog entries with no advisory.
Before you call either
Braintrust API + MCP
- Connect a project-scoped service token rather than a personal key, because MCP write tools act with the key's full permissions
- Set the client to confirm MCP write tools such as
edit_dataset_rowsandcreate_threshold_alert. The server has no read-only mode - Cache
sql_queryresults. Hosted plans allow 20 BTQL queries a minute and return 429 - To drop Topics facets, name them in
remove_facetsorremove_topic_map_ids;facet_mode: "replace"now returns an error - Upgrade to Python
braintrust0.28.0 or TypeScript 3.23.1 or later and rotate any provider keys traced before July 2026
Respan API + MCP
- Connect to https://mcp.respan.ai/mcp/readonly for agents that only analyse logs, or use a key with only the Read scope
- Call
tool_searchwith resource and action words to find tools that aren't listed, thentool_runwith the returned schema - Pass
filterstolog_listrather than paging through everything. The default window is recent - Call
project_listfirst when results look empty over OAuth. Every tool acts on the active project only - Append
.mdto any docs URL for clean Markdown
Questions
Which is better for AI agents, Braintrust API + MCP or Respan API + MCP?
Respan API + MCP scores 73.9 (BB) on agent readiness against Braintrust API + MCP's 67.6 (B), and leads in 4 of 7 scored categories. Braintrust API + MCP leads on payments & pricing and transparency & trust.
Do Braintrust API + MCP and Respan API + MCP need an API key?
Both take an API key or an OAuth sign-in.
Can an agent call Braintrust API + MCP and Respan API + MCP without installing anything?
Yes. Braintrust API + MCP has a hosted endpoint at https://api.braintrust.dev/v1 and Respan API + MCP at https://api.respan.ai/api.
Other comparisons with Braintrust API + MCP or Respan API + MCP
- Agent Tree vs Braintrust API + MCP
- Agent Tree vs Respan API + MCP
- agents-tree vs Braintrust API + MCP
- agents-tree vs Respan API + MCP
- Arize Phoenix vs Braintrust API + MCP
- Arize Phoenix vs Respan API + MCP
- Baserun vs Braintrust API + MCP
- Baserun vs Respan API + MCP
- BotGauge vs Braintrust API + MCP
- BotGauge vs Respan API + MCP
- Braintrust API + MCP vs DeepEval
- Braintrust API + MCP vs Galileo API + MCP
- Braintrust API + MCP vs Helicone AI Gateway + MCP
- Braintrust API + MCP vs HoneyHive
- Braintrust API + MCP vs Hue
- Braintrust API + MCP vs Laminar API + MCP
- Braintrust API + MCP vs Langfuse API + MCP
- Braintrust API + MCP vs LangSmith API + MCP
- Braintrust API + MCP vs LangWatch
- Braintrust API + MCP vs MLflow Tracing
- Braintrust API + MCP vs Navige
- Braintrust API + MCP vs Opik API + MCP
- Braintrust API + MCP vs Overmind
- Braintrust API + MCP vs Prefactor
- Braintrust API + MCP vs Pydantic Logfire
- Braintrust API + MCP vs W&B Weave
- Galileo API + MCP vs Respan API + MCP
- Helicone AI Gateway + MCP vs Respan API + MCP
- HoneyHive vs Respan API + MCP
- Hue vs Respan API + MCP
- Laminar API + MCP vs Respan API + MCP
- Langfuse API + MCP vs Respan API + MCP
- LangSmith API + MCP vs Respan API + MCP
- LangWatch vs Respan API + MCP
- MLflow Tracing vs Respan API + MCP
- Opik API + MCP vs Respan API + MCP
- Overmind vs Respan API + MCP
- Prefactor vs Respan API + MCP
- Pydantic Logfire vs Respan API + MCP
- Respan API + MCP vs W&B Weave
- Braintrust API + MCP vs Gauge CLI
- Braintrust API + MCP vs Plurai Simulation
- DeepEval vs Respan API + MCP
- Gauge CLI vs Respan API + MCP
- Plurai Simulation vs Respan API + MCP
- Navige vs Respan API + MCP
Machine-readable
- This page as Markdown
/compare/braintrust-vs-respan.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/braintrust.json·/api/v1/tools/respan.json - From a terminal
anchor compare braintrust respan(the CLI) - Over MCP
compare_tools {"a": "braintrust", "b": "respan"}at/mcp, no key