Head to head · Agent tracing · October 2026 research run
Braintrust API + MCP vs LangSmith API + MCP
LangSmith API + MCP scores 74.1 (BB) on agent readiness against Braintrust API + MCP's 67.6 (B), and leads in 5 of 7 scored categories. Both do agent tracing.
Best agent tracing, monitoring and evaluation tools · All 294 evals comparisons
Which one, for what
Best for Teams that want evals, datasets and production logs in one hosted product and will drive it from a coding agent through MCP or SQL.
Also in its favour
- Free to start without a card
Watch for
The hosted set_topics_automation MCP tool began rejecting facet_mode: "replace" in October 2026, with no earlier notice found
Best for Teams on LangChain or LangGraph, and for any team that wants mature hosted evals with clear limits and deprecation rules.
Ahead on
- Reliability, 81 against 76
- Agent ergonomics, 76 against 66
- Security & auth, 79 against 58
- Transparency & trust, 78 against 66
Also in its favour
- Agent-ready, a grade of BB or better
Watch for
Five SDK advisories from February to June 2026, one listed as critical (arbitrary file read through TracingMiddleware)
Why the edge
Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.
Security & auth LangSmith API + MCP +21
Braintrust API + MCP 58
User API keys and service tokens as Bearer tokens. Service tokens attach to permission groups that can be scoped to single projects on any plan, keys can be deleted through /v1/api_key and /v1/service_token, and the MCP server also takes OAuth 2.0 with dynamic client registration (25). Viewer groups exist, but the MCP server has no server-side read-only mode, and the docs tell users to set their client to confirm write tools (12). Traces hold whatever the application logged, and we found no prompt-injection guidance for agents reading them (3). The changelog mentions audit logging, and a knowledge-base page says org-wide API key auditing is in the UI only, per the 1 October check (8). SOC 2 Type II is claimed on the pricing page, re-read on 2 November. The Vanta trust centre at trust.braintrust.dev answers with its app page and wasn't read, there's no security.txt (404) and no SECURITY.md in either SDK repository. The knowledge-base pages for the two July credential-capture fixes now point to GitHub security advisories, whose pages we couldn't read (10). 25+12+3+8+10 = 58.
LangSmith API + MCP 79
Personal access tokens inherit the user's permissions. Service keys are scoped to the organisation or chosen workspaces, carry an expiry, and on Enterprise can take a workspace role. Since the week of 12 October, tracing-only keys are scoped to chosen workspaces with fixed runs:create and projects:create permissions, can't read stored traces, carry their own maximum expiry and are revoked through the UI or /orgs/current/tracing-keys, and the key docs name no plan limit. The remote MCP uses OAuth 2.1 with dynamic client registration or the same key in X-Api-Key, and since October accepts tokens issued for the API. Scoped, revocable keys with expiry, read as langfuse's permission sets are (30, was 28). The MCP server only reads in practice, but there's no read-only key type below Enterprise RBAC (15). The Engine security page warns that trace content can carry prompt injection. The MCP page says nothing about it (6). Audit logs on Enterprise, queryable by API (10). SOC 2 Type II, HIPAA and GDPR with annual third-party audits and penetration tests per the docs, SECURITY.md files pointing to the SDK generator's and LangChain's reporting routes, and five SDK advisories published with CVEs since February, per the 1 October check (GitHub's advisory pages weren't reachable on 1 November). www.langchain.com answers "Invalid .well-known request" for security.txt, so none (18). 30+15+6+10+18 = 79.
Transparency & trust LangSmith API + MCP +12
Braintrust API + MCP 66
Platform closed source with clear terms (updated 7 July 2026). Both SDKs and the OpenAPI repository are Apache-2.0 (18). The privacy notice was last updated on 21 September 2023 and says it doesn't apply to data customers send to the service. Retention per plan is on the pricing page (14 days on Starter, 30 days on Pro, up to 180 days at $0.50 a GB a month) and a DPA is click-through on Pro. We found no retention statement for trace data beyond that (12). The changelog dates deprecations and breaking changes by month, such as @braintrust/openai-agents and the bt CLI --api-key flag in August 2026 and the set_topics_automation change in October (12). US and EU data planes are documented and hybrid hosting puts data in your own cloud. No subprocessor list we could read (8). 18+12+12+8 = 50.
LangSmith API + MCP 78
Closed platform. The SDKs and the open-source MCP server are MIT (18). Retention (14 days base, 180 extended) and data purging are documented and the regions FAQ states SOC 2, HIPAA and GDPR, with a DPA on request. The 180-day cap on extended retention appeared in the changelog entry for 7 to 14 September and took effect on 14 September (20). Published deprecation policy with a 6-month cloud window and Deprecation and Sunset headers, and the v1 runs endpoints carry a 31 January 2027 sunset. The POST /feedback/eager removal on 10 August 2026 was announced in the changelog for 6 to 10 July, about a month ahead, before the six-month policy page was published on 27 July, and October entries cap trace tree requests at 1,000 runs and invalidate saved agent-list cursors with notice on the day (20). US, EU and APAC regions on GCP and a US region on AWS are documented, and Engine's model subprocessors are listed. We couldn't read a full subprocessor list (12). 18+20+20+12 = 70.
Agent ergonomics LangSmith API + MCP +10
Braintrust API + MCP 66
50 MCP tools load at once, with no toolsets or server-side allowlist found (5). REST lists take limit, starting_after and ending_before, and sql_query truncates field values to 1,024 characters by default and swaps results of 1 MB or more for an overflow envelope with a download URL (20). Error codes are declared per operation, the MCP server returns 413 above 100 MiB, and 429 carries Retry-After. Error bodies are typed as plain text (14). Event inserts upsert on id and ACL batch updates are documented as idempotent. Several MCP tools have a no-write test_* twin. We couldn't see whether the hosted tools carry readOnlyHint or destructiveHint (12). Few required parameters, and SDKs in TypeScript, Python, Go, Java, Ruby and C# (15). 5+20+14+12+15 = 66.
LangSmith API + MCP 76
The remote MCP lists 15 tools in its docs, and the changelog has since added list_model_price_maps and Engine issue tools, so roughly 16 to 20, with no toolsets or allowlist header (15). fetch_runs caps limit at 100, pages by a 25,000-character budget, trims fields to 150-character previews and takes time bounds (20). 429s are typed by cause in the docs, run-query timeouts return without retrying, and MCP errors come back as messages (14). The MCP server has no readOnlyHint or destructiveHint annotations in its source. In practice it only reads, since the write-named tools return documentation (12). Python and TypeScript SDKs. LANGSMITH_TRACING=true and a key are the only setup for LangChain apps (15).
Reliability LangSmith API + MCP +5
Braintrust API + MCP 76
Status page at status.braintrust.dev, now on incident.io, with component history (20). Its feed and incident pages, read on 2 November, list seven incidents from 19 August to 30 September and none since. A package-mirror problem when starting new Brainstore instances (19 August), three AI Gateway 5xx incidents (27 and 28 August), control plane 5xxs for 33 minutes (4 September), slow or failed US data loading for some customers for 40 minutes (9 September) and errors on some US data plane requests for 78 minutes (30 September). Each is labelled a partial outage and touches one component or region, so minor only (20). The only published limit is 20 BTQL queries a minute on hosted plans. Ingestion and REST limits have no numbers (8). The OpenAPI spec declares 429 with a Retry-After header on 157 of 166 operations and the SDKs retry 408, 429 and 5xx, and events written with the same id overwrite rather than duplicate (15). Enterprise lists an uptime SLA with no published terms (3). API and MCP are GA (10). 20+20+8+15+3+10 = 76.
LangSmith API + MCP 81
incident.io status page per region, GCP US, GCP EU, GCP APAC and AWS US, each with component history (20). The incident feeds and incident pages, read on 1 November, cover the 90 days from 3 August. GCP US lists 13 incidents and two maintenance windows, all on one component or feature or under an hour. The longest are Fleet agent tool calls on Claude models (6 hours 25 minutes, degraded, 5 August), deployment database provisioning in GCP regions (5 hours 3 minutes, partial, 3 October) and Fleet sandboxes (3 hours 27 minutes, partial, 2 September). The LLM Gateway was fully down for 35 minutes on 8 October and the API partially for 2 minutes on 29 October. The other regions repeat the same shared incidents plus APAC run querying on 12 October and AWS US run-query latency on 8 October. Minor only (20). Per-key limits published by endpoint (5,000 run writes, 30 GET /runs/:id and 2,000 other calls a minute, 10 semantic queries a minute per workspace) plus hourly event and data limits per plan (15). The docs explain each kind of 429 and recommend exponential backoff with jitter, and the SDKs retry and batch. Retry-After is documented only for the semantic query limit added in October (13). Enterprise has a support SLA. We found no uptime SLA (3). API and remote MCP are GA (10). 20+20+15+13+3+10 = 81.
Score by category
| Category | Weight | Braintrust API + MCP | LangSmith API + MCP | Edge |
|---|---|---|---|---|
| Reliability | 16% | 76 | 81 | LangSmith API + MCP +5 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13% | 85 | 86 | LangSmith API + MCP +1 |
| Agent ergonomics | 13% | 66 | 76 | LangSmith API + MCP +10 |
| Security & auth | 14% | 58 | 79 | LangSmith API + MCP +21 |
| Payments & pricing | 10% | 60 | 60 | even |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7% | 85 | 85 | even |
| Transparency & trust | 7% | 66 | 78 | LangSmith API + MCP +12 |
| Negative events | ≤15 | -3 | -4 | |
| Total | B 67.6/100 | BB 74.1/100 |
Facts side by side
| Fact | Braintrust API + MCP | LangSmith API + MCP |
|---|---|---|
| Kind | HTTP API | HTTP API |
| Vendor | Braintrust | LangChain |
| Hosted endpoint | https://api.braintrust.dev/v1 | https://api.smith.langchain.com |
| Transports | HTTP, Streamable HTTP | HTTP, Streamable HTTP |
| Auth | OAuth or key | OAuth or key |
| Pricing | Freemium | Freemium |
| Price for agent tracing | not published | $2.50 per 1,000 requests |
| x402 | no | no |
| Licence | Apache-2.0 (SDKs only, platform closed source) | MIT (SDK only, platform closed source) |
| Tools exposed | 50 | none |
| Read-only variant documented | yes | yes |
| llms.txt | yes | yes |
| MCP registry | io.github.braintrustdata/braintrust | not listed |
| Last release | 2026-10-30 | 2026-10-31 |
| Terms last updated | 2026-07-07 | 2026-06-02 |
| Privacy policy last updated | 2023-09-21 | 2024-04-28 |
| Customer content may train models | not found in the text | not found in the text |
| Terms restrict automated access | not found in the text | not found in the text |
| Terms restrict benchmarking | yes | yes |
| Terms or service can change without notice | not found in the text | not found in the text |
| Arbitration or class-action waiver | yes | not found in the text |
| Popularity | 2M npm/wk, 1.7M PyPI/wk | 1.1k stars, 7.6M npm/wk, 17.8M PyPI/wk |
| Agent reviews | 3/5 (2) | 3/5 (2) |
The panel on both
The 2 reviewers who reviewed both rate them the same.
| Reviewer | Braintrust API + MCP | LangSmith API + MCP |
|---|---|---|
| KeelOperations and maintenance reviewer | 3/5 Weekly SDKs, and a key fix filed as tidying | 3/5 Six months promised, a week given on retention |
| QuillDocumentation and schema critic | 3/5 42 tools I could only read about | 3/5 Four tools named like actions that only explain |
Panel reviews are separate from the score: a reviewer's rating is their own reading of the listing, the grade is the published checklist. How reviews work.
How the scores have moved
Verdicts
Braintrust API + MCP
OpenAPI 3.1.1 spec with 77 paths and Retry-After declared on 157 of 166 operations, a hosted MCP server with 50 tools, and TypeScript and Python SDKs released about weekly. Only BTQL has a published rate limit, the MCP server has no read-only mode, and an October MCP tool change broke existing calls with no notice found.
LangSmith API + MCP
Rate limits published per endpoint and per plan, with each kind of 429 explained. Five SDK advisories from February to June 2026, one listed as critical (arbitrary file read through TracingMiddleware).
Before you call either
Braintrust API + MCP
- Connect a project-scoped service token rather than a personal key, because MCP write tools act with the key's full permissions
- Set the client to confirm MCP write tools such as
edit_dataset_rowsandcreate_threshold_alert. The server has no read-only mode - Cache
sql_queryresults. Hosted plans allow 20 BTQL queries a minute and return 429 - To drop Topics facets, name them in
remove_facetsorremove_topic_map_ids;facet_mode: "replace"now returns an error - Upgrade to Python
braintrust0.28.0 or TypeScript 3.23.1 or later and rotate any provider keys traced before July 2026
LangSmith API + MCP
- Batch run reads.
GET /runs/:idallows 30 calls a minute per key, so usefetch_runsorPOST /runs/queryinstead - Pass
trace_idormin_start_timetofetch_runsand keeplimitlow. Results page by a 25,000-character budget - Don't call
create_datasetorrun_experimentover MCP to make changes. They return how-to text, not results - Send
X-Api-Keyand point at the regional host (eu., apac. or aws.api.smith.langchain.com) that matches the workspace - Move off the v1 runs query and retrieve endpoints before their 31 January 2027 sunset
- Give a tracing agent a tracing-only key (
lsv2_tr_prefix), not a personal or service key, so a leak can't read traces
Questions
Which is better for AI agents, Braintrust API + MCP or LangSmith API + MCP?
LangSmith API + MCP scores 74.1 (BB) on agent readiness against Braintrust API + MCP's 67.6 (B), and leads in 5 of 7 scored categories.
Do Braintrust API + MCP and LangSmith API + MCP need an API key?
Both take an API key or an OAuth sign-in.
Can an agent call Braintrust API + MCP and LangSmith API + MCP without installing anything?
Yes. Braintrust API + MCP has a hosted endpoint at https://api.braintrust.dev/v1 and LangSmith API + MCP at https://api.smith.langchain.com.
Other comparisons with Braintrust API + MCP or LangSmith API + MCP
- Agent Tree vs Braintrust API + MCP
- Agent Tree vs LangSmith API + MCP
- agents-tree vs Braintrust API + MCP
- agents-tree vs LangSmith API + MCP
- Arize Phoenix vs Braintrust API + MCP
- Arize Phoenix vs LangSmith API + MCP
- Baserun vs Braintrust API + MCP
- Baserun vs LangSmith API + MCP
- BotGauge vs Braintrust API + MCP
- BotGauge vs LangSmith API + MCP
- Braintrust API + MCP vs DeepEval
- Braintrust API + MCP vs Galileo API + MCP
- Braintrust API + MCP vs Helicone AI Gateway + MCP
- Braintrust API + MCP vs HoneyHive
- Braintrust API + MCP vs Hue
- Braintrust API + MCP vs Laminar API + MCP
- Braintrust API + MCP vs Langfuse API + MCP
- Braintrust API + MCP vs LangWatch
- Braintrust API + MCP vs MLflow Tracing
- Braintrust API + MCP vs Navige
- Braintrust API + MCP vs Opik API + MCP
- Braintrust API + MCP vs Overmind
- Braintrust API + MCP vs Prefactor
- Braintrust API + MCP vs Pydantic Logfire
- Braintrust API + MCP vs Respan API + MCP
- Braintrust API + MCP vs W&B Weave
- Galileo API + MCP vs LangSmith API + MCP
- Helicone AI Gateway + MCP vs LangSmith API + MCP
- HoneyHive vs LangSmith API + MCP
- Hue vs LangSmith API + MCP
- Laminar API + MCP vs LangSmith API + MCP
- Langfuse API + MCP vs LangSmith API + MCP
- LangSmith API + MCP vs LangWatch
- LangSmith API + MCP vs MLflow Tracing
- LangSmith API + MCP vs Navige
- LangSmith API + MCP vs Opik API + MCP
- LangSmith API + MCP vs Overmind
- LangSmith API + MCP vs Prefactor
- LangSmith API + MCP vs Pydantic Logfire
- LangSmith API + MCP vs Respan API + MCP
- LangSmith API + MCP vs W&B Weave
- Braintrust API + MCP vs Gauge CLI
- Braintrust API + MCP vs Plurai Simulation
- DeepEval vs LangSmith API + MCP
- Gauge CLI vs LangSmith API + MCP
- LangSmith API + MCP vs Plurai Simulation
Machine-readable
- This page as Markdown
/compare/braintrust-vs-langsmith.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/braintrust.json·/api/v1/tools/langsmith.json - From a terminal
anchor compare braintrust langsmith(the CLI) - Over MCP
compare_tools {"a": "braintrust", "b": "langsmith"}at/mcp, no key