Honeycomb MCP

# Fix list: Honeycomb MCP

From Anchor Terminal's listing at https://www.anchorterminal.com/tools/honeycomb-mcp, the October 2026 research run, assessed 9 October 2026. Grade C, 58.3 out of 100.

This is everything the published grade says the listing lacks, the biggest possible gain to the total first. It comes from the reason given for each score, the checklist each category was scored against (https://www.anchorterminal.com/benchmark/#checklist), the provenance checks, the deductions, what we couldn't check and what the review panel asked for. A fix counts at the next check, once it's public.

For a coding agent working on Honeycomb MCP: work through the items below in the product, its docs and its public pages. Each category gives the reason for its score, with the points each checklist item earned, and the checklist itself, so the gap is the items that earned less than their points. Change the product, not the wording, and keep a note of what you changed and where it's published.

## 1. Payments & pricing, 30 out of 100, up to 8.8 more on the total

Why it scored 30: No x402, MPP or L402 (0). Plan prices are public, Free and Pro from $150 a month for 50 million events, and MCP is in every plan with no per-call price (10). The Free plan is described as free forever and the signup guide asks for an email address or Google account with no payment step (20). A person signs up in a browser, a Team Owner enables Honeycomb Intelligence, and then OAuth consent or a Team Owner's API key is needed (0).

The checklist (https://www.anchorterminal.com/benchmark/#checklist-payments):

The published rubric, also on the [x402 page](https://www.anchorterminal.com/x402/).

- 40, a machine payment protocol (x402, MPP or L402) on the tool's own endpoints. 10 to 30 when it covers only some endpoints or only goes through a third party, and the note says which.
- 20, per-call or per-unit pricing published without a login. 10 for public plan-only pricing, 0 for "contact sales" or prices behind a login.
- 20, a free tier or trial that doesn't need a card.
- 20, autonomous onboarding, meaning an agent can get access without a person signing up in a browser (keyless use, x402, a programmatic key API).

Payment platforms and agent wallets rarely charge for their own API over a machine protocol, so the first line has steps for them, and the highest one that applies counts. 40 when x402, MPP or L402 runs on all their own endpoints, 30 when it runs on part of their own API, 25 when their merchants can accept one, 20 for running a facilitator, 15 for paying as a buyer, and 0 when the only protocol is their own. Merchant acceptance sits above a facilitator because the platform's own customers can charge agents through it, while a facilitator settles for sellers who wire up the protocol themselves. The counter-argument (a facilitator does more for the protocol as a whole) has a point. Each note says which step applied.

Open-source software you run yourself is scored on its hosted or paid option if it has one. A free, self-hosted package with nothing to buy gets 20, 20 and 20 for the last three lines, and 0 to 40 for the first only if it ships a payment protocol.

## 2. Agent ergonomics, 53 out of 100, up to 7.6 more on the total

Why it scored 53: 34 tools is above the 30 line (5). The `X-MCP-Toolset` header loads `query` or `canvas` subsets, and the docs don't say how many tools each holds (add 8, so 13). Size controls are described per tool, such as up to 100 datasets or 100 columns a call, subtree zoom on `get_trace` and query limits. Pagination parameters aren't documented (10). Errors are documented with causes and fixes, and the rate-limit message names the retry time. No error code list was found (12). The reference labels each tool read or write and `update_slo` keeps omitted fields. No idempotency keys were found, and we couldn't confirm readOnlyHint or destructiveHint annotations (8). `get_workspace_context` takes no parameters, relative time is the default, and setup is one command or link in nine documented clients. SDKs don't apply (10).

The checklist (https://www.anchorterminal.com/benchmark/#checklist-ergonomics):

- 0 to 25, context cost. For MCP, the number and size of the tool definitions (25 for ten or fewer compact tools, 15 for 11 to 30, 5 for more than 30, plus up to 10 back for toolsets, dynamic loading or read-only subsets). For APIs, whether responses can be sized (field selection, limits, summaries).
- 20, pagination, filtering and output-size controls.
- 20, actionable, documented error responses, codes and messages an agent can recover from.
- 20, idempotency or safe retries, and for MCP the `readOnlyHint` and `destructiveHint` annotations.
- 15, sensible defaults, few required parameters, and official SDKs in at least two languages.

Models are read for tool use, structured output, prompt caching, context length, batch and SDKs. Frameworks for how much code and how many defaults a tool-calling agent with MCP needs.

## 3. Reliability, 65 out of 100, up to 7 more on the total

Why it scored 65: Scored as a hosted MCP server. status.honeycomb.io is an Atlassian Statuspage with 14 components for US1, EU1, the website and the sandbox. It has no MCP component, though MCP incidents are posted there (17). Four incidents in the 90 days to 9 October 2026, all marked minor. Querying failures for about 30 minutes on 31 July, with a postmortem. Triggers failing intermittently in the EU on 27 and 28 July. US1 queries over older data erroring for 67 minutes on 10 August. Canvas not loading for 38 minutes on 21 August. The 10 August outage hit the query path the server depends on for over an hour, so we score between minor-only and one major (15). Per-tool limits are published with numbers (15). Rate-limit errors carry the time to retry after. No backoff guidance or idempotency keys for the nine write tools were found (8). No SLA found in the terms, and honeycomb.io/sla returns 404 (0). The docs list MCP as graduated to general availability (10).

The checklist (https://www.anchorterminal.com/benchmark/#checklist-reliability):

Hosted APIs, MCP servers, models and platforms.

- 20, a public status page with component history (Statuspage, Instatus, BetterStack or the vendor's own).
- 0 to 30, the incident record for the last 90 days on that page. 30 for a clean record or trivial incidents only, 20 for minor incidents only, 10 for one major outage (an hour or more of a core API down, or errors across the board), 0 for several. 5 when there's no history we could read, and the note says so.
- 15, rate limits documented with numbers.
- 15, documented 429 or overload handling (Retry-After, backoff guidance), and idempotency keys or safe-retry guidance where writes are involved.
- 10, an SLA published for any paid tier.
- 10, the surface agents use is generally available, not beta or preview.

Local packages, SDKs, frameworks and stdio MCP servers.

- 20, installs from an official package with supported runtimes stated.
- 25, a public CI and test suite, passing on the default branch.
- 0 to 25, open crash or regression issues relative to activity (25 for few and handled, 0 for many, old and unanswered).
- 15, semver discipline and breaking changes called out in a changelog.
- 15, version 1.0 or later, or declared stable.

Protocols are read from their reference implementations, the public facilitators or servers, spec stability and test vectors.

## 4. Schema & documentation, 57 out of 100, up to 7 more on the total

Why it scored 57: The server is closed source and `tools/list` needs sign-in, so we could not read the input schemas. Partial credit for a contract that exists but is not public (12). docs.honeycomb.io publishes llms.txt and a Markdown twin of every page (10). The tools reference says what each of the 34 tools returns and when an agent reaches for it, for example calling `get_workspace_context` first and `list_boards` before `update_board`. These are docs, not the definitions the model sees (14). Parameters are mentioned only in passing, such as `metric_name` and `usage_mode`, with no types, enums or required fields published (4). A use-cases page and a troubleshooting page document the Unauthorized, rate-limit, permission and not-found errors with causes and fixes (9). The registry entry carries version 1.0.0 and Honeycomb links a UI changelog and an API changelog, but both hosts disallow unnamed crawlers, so we could not confirm MCP entries (8).

The checklist (https://www.anchorterminal.com/benchmark/#checklist-schema):

APIs and MCP servers.

- 25, a machine-readable contract (a public OpenAPI file or similar; for MCP, typed JSON Schema inputs on every tool).
- 10, llms.txt or Markdown docs served for agents.
- 0 to 20, descriptions that say what a tool is for, when to use it and when not to, read from the tool definitions in the source or the API reference.
- 0 to 15, typed inputs with enums, constraints and required fields, and no free-form JSON blobs.
- 0 to 15, examples and documented error responses.
- 15, versioning and a public changelog.

Models are read from the API reference, the OpenAPI file, llms.txt, the structured-output and tool-use docs and the model cards. Frameworks from docs a model can follow, typed interfaces, examples and the API reference.

## 5. Security & auth, 71 out of 100, up to 5.1 more on the total

Why it scored 71: OAuth authorisation code with PKCE S256, dynamic client registration, client ID metadata documents, refresh tokens and the scopes `mcp:read` and `mcp:write`, with the token accepted in the header only. Sessions are listed on the account's Integrations tab. The alternative is a Management API key whose scopes are fixed at creation and which can be disabled or deleted (28). Tools are read-only until `mcp:write` is granted at consent or on the key, API keys are limited to chosen environments, the Activity Log environment is shown through MCP only to Team Owners, and there is no delete tool. No confirmation step for writes was found (15). Trace, log and span content is untrusted text, and no injection guidance was found in the MCP docs (2). The Activity Log records OAuth sessions with requested and granted scopes and one event per query run with its source, MCP included. The compliance page describes activity logging as for enterprise customers (11). SOC 2 Type II each year, annual third-party penetration tests, HIPAA BAAs and a bug bounty by email with discretionary rewards. No security.txt, the trust centre is open only to Pro and Enterprise customers, and the bounty scope names ui.honeycomb.io and api.honeycomb.io, not the MCP hosts (15).

The checklist (https://www.anchorterminal.com/benchmark/#checklist-security):

- 0 to 30, the credential model. 30 for OAuth 2.1 with scopes, or scoped and revocable keys with rotation. 20 for plain revocable API keys. 10 for one all-powerful key. 10 off when a secret can travel in a URL query string as a documented option.
- 0 to 20, read-only or least-privilege modes, and confirmation or approval for destructive actions.
- 0 to 15, prompt-injection posture where the tool returns untrusted content (documented mitigations or guidance). A tool that returns no untrusted content gets 10.
- 0 to 15, audit logs or per-call visibility for the operator.
- 0 to 20, a security programme. security.txt or a disclosure policy, a bug bounty, SOC 2 or ISO 27001, advisories handled in public.

Models are read for retention, whether API data trains models (and whether that's off by default), zero-retention options and certifications. Frameworks for telemetry defaults, approval hooks, guardrails and sandboxing.

## 6. Maintenance & community, 61 out of 100, up to 3.4 more on the total

Why it scored 61: The product changelogs at changelog.honeycomb.io and api-changelog.honeycomb.io disallow unnamed crawlers, so we didn't read them and scored from what else is dated. The docs sitemap gives 29 September 2026 for the tools reference, the companion plugin reached 1.2.2 on 20 September, and the AI policy page was updated on 21 September. Docs dates are weaker evidence than a release note (20). The plugin repository has 14 commits since 11 July, with fixes released on 21 August, 26 August and 20 September, and one explicit version bump. Half credit without the changelog (10). A support site, a community Slack and a `feedback` tool reach the team, and the plugin repository merged an outside contributor's fix on 17 July and has five open issues (10). `io.honeycomb/mcp` 1.0.0 is in the official MCP registry under the vendor's domain namespace, published 29 April 2026 (15). The plugin repository runs structural and scenario test workflows. The server itself is closed (6).

The checklist (https://www.anchorterminal.com/benchmark/#checklist-maintenance):

- 0 to 30, time since the last release, or the last published model or API change for a closed service. 30 within 30 days, 20 within 90, 10 within 180, 0 older.
- 20, at least three releases or dated changelog entries in the last 90 days.
- 0 to 25, responsiveness. Issues and pull requests answered on GitHub (the open issues and how recent the replies are). For closed services, a public changelog and a support or community channel that answers, 0 to 15.
- 15, presence in the official MCP registry under a verified namespace (MCP servers), or current official SDKs (APIs and models).
- 10, package health, current dependencies and CI.

Models are read for deprecation notice periods and model churn rather than release counts.

## 7. Transparency & trust, 68 out of 100, up to 2.8 more on the total

Made of editorial 64, provenance 71.

Why it scored 68: The hosted server is closed, under published Terms of Service and Honeycomb Intelligence Supplemental Terms. The companion plugin is MIT (17). The privacy notice, the DPA and an AI policy page dated 21 September 2026 are public. The AI page says MCP's model provider is whichever the client uses, and that Honeycomb uses no provider that trains foundation models on input. Telemetry is deleted after the retention window, 60 days for most customers. The terms also let Honeycomb use customer data to improve its services, and the AI page says Honeycomb may fine-tune pre-trained models (22). The self-hosted server carries a deprecation notice dated 21 August 2025. No deprecation policy was found, the AI terms allow any Honeycomb Intelligence feature to be changed or ended at any time with commercially reasonable efforts at notice, and the docs say rate limits may change (6). The subprocessor list is public with purposes and countries, the DPA promises 30 days' notice of new subprocessors, and customers choose a US or EU data location (19).

The checklist (https://www.anchorterminal.com/benchmark/#checklist-transparency):

- 0 to 30, source availability and licence clarity. 30 for open source under an OSI licence, 15 for closed with clear terms, 0 for unclear terms.
- 0 to 30, data handling and retention statements that agree with each other (privacy policy, DPA, retention periods, subprocessors).
- 0 to 20, a deprecation policy or notices with dates.
- 0 to 20, telemetry disclosed with an opt-out (local software), or subprocessors and data locations disclosed (hosted).

The other half of Transparency and trust is the provenance score, computed from checked facts (below). The category score is the mean of the two.

Provenance checks not met in full (half of this category, computed from checked facts):

- Domain age: honeycomb.io, no registry record we could read (0 of 15)
- Terms of service: read, states 6 of the 7 things a reader expects, and has 1 clause that costs points (7.1 of 10)
- Privacy policy: read, states 7 of the 8 things a reader expects (9.3 of 10)
- security.txt: not found (0 of 10)

## What we couldn't check

What we couldn't read counted as absent. Publishing it on a page a plain HTTP fetch can read (not only in a browser) lets the next check count it.

- unchecked: changelog.honeycomb.io and api-changelog.honeycomb.io, whose robots.txt disallows every path to unnamed crawlers. Release cadence, breaking changes and the general availability date of the MCP server rest on docs dates, the registry and the plugin repository
- unchecked: tool input schemas, the size of the tool definitions and whether tools carry readOnlyHint and destructiveHint, since `tools/list` needs a Honeycomb account
- unchecked: how many tools the `query` and `canvas` toolsets load
- unchecked: the registration date of honeycomb.io. No RDAP service answered for the .io registry
- unchecked: whether signup asks for a card. The signup guide names no payment step and we did not open the signup form
- unchecked: documents inside the trust centre, which is open only to Pro and Enterprise customers
- The terms forbid using any technology or means not authorised by Honeycomb to access the service or extract data. The MCP server is an authorised route, and the clause matters before our probes run
- The lead was right on the endpoints, OAuth and the Honeycomb Intelligence requirement. It did not mention the Management API key route or that MCP is in the Free plan

## Weaknesses

- Tool input schemas and annotations are visible only through `tools/list` after sign-in. The public reference gives descriptions, not parameters
- No guidance on prompt injection was found, although trace and log content returned to the agent is untrusted text
- The status page has no MCP component. US1 queries over older data failed for 67 minutes on 10 August 2026
- No SLA was found, and the AI terms allow Honeycomb to change or end any Honeycomb Intelligence feature at any time
- A Team Owner must enable Honeycomb Intelligence first, and API key scopes can't be changed after creation

## What costs an agent a turn today

The notes we give agents before they call it. Each one is a workaround an agent shouldn't need.

- Use https://mcp.honeycomb.io/mcp for US teams and https://mcp.eu1.honeycomb.io/mcp for EU teams. A team's data lives in one region only
- If no tools appear after sign-in, ask a Team Owner to enable Honeycomb Intelligence in team settings
- Send `X-MCP-Toolset: query` or `canvas` to load a subset of the 34 tools. An empty or unknown value makes every request fail
- Call `get_workspace_context` first, then `find_columns`, before `run_query`. Pass absolute time ranges as epoch timestamps
- For unattended runs send `Authorization: Bearer <KEY_ID>:<SECRET_KEY>` from a Management key with only Model Context Protocol and Environments scopes

## When it's done

Send what changed and where it's published as a dispute (https://www.anchorterminal.com/builders/#disputes, or `POST https://www.anchorterminal.com/api/v1/contact` with `"kind": "dispute"`). Disputes are answered in public, and the listing is checked again by the same checklist. Paying for an audit or a listing claim changes nothing here.