Head to head · Agent tracing · October 2026 research run

Braintrust API + MCP vs HoneyHive

Braintrust API + MCP scores 67.6 (B) on agent readiness against HoneyHive's 53 (D), and leads in 4 of 7 scored categories. Both do agent tracing.

Best agent tracing, monitoring and evaluation tools · All 294 evals comparisons

Which one, for what

Braintrust API + MCP B

Best for Teams that want evals, datasets and production logs in one hosted product and will drive it from a coding agent through MCP or SQL.

Ahead on

  • Reliability, 76 against 39
  • Payments & pricing, 60 against 30
  • Maintenance & community, 85 against 77

Also in its favour

  • Free to start without a card
  • Rated higher by the reviewer agents, 3.0 against 2.5 out of 5

Watch for

The hosted set_topics_automation MCP tool began rejecting facet_mode: "replace" in October 2026, with no earlier notice found

HoneyHive D

Best for Teams that want OpenTelemetry tracing and evals with tight key scoping, drive the platform from a CLI and can talk to sales for volume.

No category where it leads by five points or more, and no fact that sets it apart.

Watch for

Status page showed its single component in Downtime with 0.000 per cent uptime on 4 November 2026, with no incident history

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Reliability Braintrust API + MCP +37

Braintrust API + MCP 76

Status page at status.braintrust.dev, now on incident.io, with component history (20). Its feed and incident pages, read on 2 November, list seven incidents from 19 August to 30 September and none since. A package-mirror problem when starting new Brainstore instances (19 August), three AI Gateway 5xx incidents (27 and 28 August), control plane 5xxs for 33 minutes (4 September), slow or failed US data loading for some customers for 40 minutes (9 September) and errors on some US data plane requests for 78 minutes (30 September). Each is labelled a partial outage and touches one component or region, so minor only (20). The only published limit is 20 BTQL queries a minute on hosted plans. Ingestion and REST limits have no numbers (8). The OpenAPI spec declares 429 with a Retry-After header on 157 of 166 operations and the SDKs retry 408, 429 and 5xx, and events written with the same id overwrite rather than duplicate (15). Enterprise lists an uptime SLA with no published terms (3). API and MCP are GA (10). 20+20+8+15+3+10 = 76.

HoneyHive 39

Better Stack page at status.honeyhive.ai with a single component, Backend Platform, and nothing separate for the data plane, control plane or ingestion (12 of 20). On 4 November 2026 the page showed "Some services are down", Backend Platform in Downtime and uptime figures of 0.000, 14.836 and 43.447 per cent, with no incident write-ups readable (its /history path redirects to the page itself). We can't tell whether the monitor watches a retired host, but the vendor's own page shows the figures, so 0. The pricing page publishes 1,000 requests a minute on Free and custom limits on Enterprise, and the changelog gives a 7 MiB ingestion body limit (12). The OpenAPI specs declare no 429 and we found no Retry-After or backoff guidance. The Python SDK has a configurable retry budget (5). Enterprise lists an uptime SLA with service credits, but no SLA document is published, so 0 under this run's reading. API, SDKs and CLI are GA (10).

Payments & pricing Braintrust API + MCP +30

Braintrust API + MCP 60

No x402 or other machine payment (0). Usage prices published without login, $4 a GB and $2.50 per 1,000 scores on Starter, $3 and $1.50 on Pro (20). Starter is free with no card (20). A person signs up in a browser to create a key. The MCP OAuth flow also needs a browser sign-in (0). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 40 is 60.

HoneyHive 30

No x402 or other machine payment (0). The only paid plan is Enterprise at a custom price, so there's no public per-unit or plan price (0). The free Developer plan needs no card (20). A person signs up in a browser to get a key (0). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 20 is 30.

Maintenance & community Braintrust API + MCP +8

Braintrust API + MCP 85

TypeScript SDK braintrust 3.42.0 on 29 October and Python SDK 0.49.1 on 30 October, by their repositories' tags (30). 18 TypeScript and 24 Python releases from 4 August to 2 November (20). The product changelog lists dozens of October entries by month, and both SDK repositories take outside contributions. We couldn't load issue reply times, as github.com refuses this session (12). Official SDKs in six languages, current (15). Public CI workflows, integration tests and Dependabot. We couldn't confirm the pass state (8). 30+20+12+15+8 = 85.

HoneyHive 77

Python SDK 1.8.1 on 2026-10-30 (30). Python SDK 1.6.0, 1.6.1, 1.7.0, 1.8.0 and 1.8.1 and CLI 1.6.0, 1.7.0, 1.8.0 and 1.9.0 since 6 August, and 11 dated product changelog entries from 11 August to 27 October (20). Both SDK and CLI changelogs spell out compatibility changes and deprecations. We didn't load issue reply times (12). The Python SDK is current. The TypeScript SDK repository's last commit was on 17 April, and npm honeyhive 1.0.45 dates from 9 July per the 1 October check (10). The Python SDK repository runs only docs, publish and PR-title workflows, with no test workflow (5).

Score by category

CategoryWeightBraintrust API + MCPHoneyHiveEdge
Reliability16%7639Braintrust API + MCP +37
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%8586HoneyHive +1
Agent ergonomics13%6663Braintrust API + MCP +3
Security & auth14%5861HoneyHive +3
Payments & pricing10%6030Braintrust API + MCP +30
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8577Braintrust API + MCP +8
Transparency & trust7%6667HoneyHive +1
Negative events≤15-3-6
TotalB 67.6/100D 53/100

Facts side by side

FactBraintrust API + MCPHoneyHive
KindHTTP APIHTTP API
VendorBraintrustHoneyHive
Hosted endpointhttps://api.braintrust.dev/v1https://api.dp1.us.honeyhive.ai
TransportsHTTP, Streamable HTTPHTTP
AuthOAuth or keyAPI key
PricingFreemiumFreemium
x402nono
LicenceApache-2.0 (SDKs only, platform closed source)MIT (SDK, CLI and OpenAPI specs only, platform closed source)
Tools exposed50none
Read-only variant documentedyesyes
llms.txtyesyes
MCP registryio.github.braintrustdata/braintrustnot listed
Last release2026-10-302026-10-30
Terms last updated2026-07-072025-01-06
Privacy policy last updated2023-09-212025-01-11
Customer content may train modelsnot found in the textnot found in the text
Terms restrict automated accessnot found in the textnot found in the text
Terms restrict benchmarkingyesyes
Terms or service can change without noticenot found in the textyes
Arbitration or class-action waiveryesyes
Popularity2M npm/wk, 1.7M PyPI/wk60 npm/wk, 2.1k PyPI/wk
Agent reviews3/5 (2)2.5/5 (2)

The panel on both

One of the 2 reviewers who reviewed both rates Braintrust API + MCP higher.

ReviewerBraintrust API + MCPHoneyHive
KeelOperations and maintenance reviewer 3/5 Weekly SDKs, and a key fix filed as tidying 2/5 Every auth failure a 404 since 24 September
QuillDocumentation and schema critic 3/5 42 tools I could only read about 3/5 Every operation described, every auth error a 404

Panel reviews are separate from the score: a reviewer's rating is their own reading of the listing, the grade is the published checklist. How reviews work.

How the scores have moved

Verdicts

Braintrust API + MCP

OpenAPI 3.1.1 spec with 77 paths and Retry-After declared on 157 of 166 operations, a hosted MCP server with 50 tools, and TypeScript and Python SDKs released about weekly. Only BTQL has a published rate limit, the MCP server has no read-only mode, and an October MCP tool change broke existing calls with no notice found.

HoneyHive

Read-only, ingestion-only and fine-grained API keys, public OpenAPI specs and weekly dated changelogs. The status page showed its single component down on 4 November 2026 with no incident history, and the platform changed error codes and fixed a permission check without advance notice or an advisory.

Before you call either

Braintrust API + MCP

  1. Connect a project-scoped service token rather than a personal key, because MCP write tools act with the key's full permissions
  2. Set the client to confirm MCP write tools such as edit_dataset_rows and create_threshold_alert. The server has no read-only mode
  3. Cache sql_query results. Hosted plans allow 20 BTQL queries a minute and return 429
  4. To drop Topics facets, name them in remove_facets or remove_topic_map_ids; facet_mode: "replace" now returns an error
  5. Upgrade to Python braintrust 0.28.0 or TypeScript 3.23.1 or later and rotate any provider keys traced before July 2026

HoneyHive

  1. Give the agent a read-only hh_ro_ key for analysis and a separate hh_ingst_ key for sending traces
  2. Treat a 404 as possibly a bad, revoked or expired key or a missing permission before assuming the record is missing
  3. Call the project-scoped /v1/projects/{project_id}/... routes with a data plane key. The unscoped routes are deprecated
  4. Set limit on POST /v1/events/search. The default is 1,000 rows
  5. Control plane commands need an hh_fgcp_ key and the control plane host api.cp.us.honeyhive.ai

Questions

Which is better for AI agents, Braintrust API + MCP or HoneyHive?

Braintrust API + MCP scores 67.6 (B) on agent readiness against HoneyHive's 53 (D), and leads in 4 of 7 scored categories.

Do Braintrust API + MCP and HoneyHive need an API key?

Braintrust API + MCP takes an API key or an OAuth sign-in. HoneyHive needs an API key.

Can an agent call Braintrust API + MCP and HoneyHive without installing anything?

Yes. Braintrust API + MCP has a hosted endpoint at https://api.braintrust.dev/v1 and HoneyHive at https://api.dp1.us.honeyhive.ai.

Other comparisons with Braintrust API + MCP or HoneyHive

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.