Head to head · Event analytics · October 2026 research run

Flagsmith vs Heap

Flagsmith scores 61.2 (C) on agent readiness against Heap's 48.2 (D), and leads in every scored category. Both do event analytics.

Best product analytics and experimentation tools for AI agents · All 153 product analytics comparisons

Which one, for what

Flagsmith C

Best for Feature flags and experiments for teams that want open source or self-hosting, managed by an agent through MCP.

Ahead on

  • Reliability, 65 against 47
  • Schema & documentation, 67 against 57
  • Agent ergonomics, 55 against 49
  • Security & auth, 65 against 41
  • Payments & pricing, 40 against 25
  • Maintenance & community, 83 against 63
  • Transparency & trust, 77 against 66

Also in its favour

  • Runs on your own machine
  • Open source

Watch for

The MCP docs call the server an early release, and experiment tools are marked beta

Heap D

Best for An agent that records backend events or enriches user and account properties in a company's existing Heap project, or that files privacy deletions.

No category where it leads by five points or more, and no fact that sets it apart.

Watch for

No query, export or definitions endpoint in the developer reference. Reading results needs the app, Heap Connect or the MCP server we couldn't read

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Security & auth Flagsmith +24

Flagsmith 65

OAuth for interactive MCP clients with Flagsmith as the authorisation server, Organisation API keys in the Authorization header that act as admin or with chosen RBAC roles, and OIDC trust relationships that swap a CI token for a short-lived one. Granular key roles need Scale-Up or Enterprise, so a Free key is all-powerful (25 of 30). RBAC roles can limit a key to reads, and change requests put a flag change through approval on Scale-Up and Enterprise. There is no read-only switch on the MCP server itself (14 of 20). No prompt-injection guidance; tools return names, descriptions and trait values written by people (3 of 15). Audit logs record every admin action and stream by webhook, on Scale-Up and Enterprise (11 of 15). The docs say Flagsmith has a SOC 2 Type 2 report available on request, the terms promise a yearly third-party penetration test, and SECURITY.md routes reports to support email. No security.txt, bug bounty or public advisory process was found (12 of 20). 25+14+3+11+12 = 65.

Heap 41

Track, identify and property calls carry no secret. The OpenAPI fragment declares an empty security requirement and the only identifier is the environment ID that the web snippet publishes. User deletion uses one admin-generated API key per account, exchanged for a temporary token. No scopes or rotation guidance found (8 of 30). Ingest is write-only by design. Deletion of up to 10,000 users has no confirmation step (5). The API returns no untrusted content (10). Audit logs are unchecked because help.heap.io answered with a bot check, so none are counted (0). Contentsquare publishes a signed security.txt valid to 31 January 2028 on contentsquare.com (not on heap.io), a disclosure policy, an invitation-only YesWeHack bounty, SOC 2 Type II and ISO 27001 (18). Total 41. Per the 7 October check, not re-read on 4 November, because heap.io and contentsquare.com terms bar robots and crawling.

Maintenance & community Flagsmith +20

Flagsmith 83

v2.290.0 was released on 5 November 2026 (30). 34 tagged releases between 8 August and 5 November (20). Commits land daily (60 since 8 August in the 60-commit clone), with a Discord server and support email. We couldn't read issue response times because GitHub's API refused us for its rate limit (15 of 25). Not in the official MCP registry, which returned no Flagsmith entry; official SDKs are current (8 of 15). Renovate updates, CI workflows for the API, frontend, docs and MCP, and a Trivy scan (10). 30+20+15+8+10 = 83.

Heap 63

heap.js 5.3.17 shipped on 6 October 2026, but that is the browser SDK. The server-side track reference last changed on 3 June 2026 and has no changelog, so we scored this line between the two (20 of 30). heap.js 5.3.15, 5.3.16 and 5.3.17 and Android Core 0.9.5 fall inside 90 days (20). Dated public changelogs and a community at community.contentsquare.com, whose response times we didn't check (8). Client SDKs for web, Android, iOS, React Native and Flutter are current. The Node server client is from 2016 (10). CI not checked (5). Total 63. Per the 7 October check, not re-read on 4 November (developers.heap.io is paused on its terms). The three heap.js releases still fall inside the 90 days from 6 August.

Reliability Flagsmith +18

Flagsmith 65

Graded on Flagsmith Cloud with the hosted lines. Statuspage at status.flagsmith.com with components for the Edge API, Admin API, dashboard and website (20). In the 90 days from 8 August, an Admin and SDK API outage on 28 October marked critical, posted at 21:56 UTC and resolved at 22:12 with no cause given, and a minor delay in processing flag changes on 10 September. The posted span of the outage is 16 minutes and its start isn't stated, so we read it as one serious incident under an hour (15 of 30). The Management API limit of 500 requests a minute is documented (15). No 429 handling, Retry-After or idempotency guidance was found in the docs (0). A public SLA commits to 99.95 per cent monthly uptime with service credits (10). The Management API is generally available, but the MCP docs call the server an early release, so half (5 of 10). 20+15+15+0+10+5 = 65.

Heap 47

Graded on the server-side API with the hosted lines. Heap's status page is an Atlassian Statuspage at status.heap.io, the address heap.io links to, which sits on a heap.io host whose terms bar robots, so it wasn't read on 4 November. The same page also answers at heap.statuspage.io, but that copy names status.heap.io as its canonical address and no Heap page links to it, so it wasn't used. The 7 October component and incident readings came only from Statuspage's JSON API, which this run doesn't use, so they are struck. No component list could be read (0) and no incident history could be read (5). Rate limits published with numbers, 30 requests per 30 seconds per identity on track and 15,000 events a minute per environment on bulk track, per the 7 October check, not re-read on 4 November (15). No 429 or backoff guidance found, but track takes an idempotency_key, per the 7 October check, not re-read on 4 November (7, the earlier partial kept, since keys without any 429 or backoff guidance meet none of the line's values in full). Contentsquare's support package states 99.5 per cent availability for Product Analytics Pro and Enterprise, measured on the user interface, per the 7 October check, not re-read on 4 November (10). The API carries no beta label (10). 0+5+15+7+10+10 = 47.

Payments & pricing Flagsmith +15

Flagsmith 40

No x402, MPP or L402 (0). Plan prices and the per-million overage are public without login, though the page gives two overage rates for Start-Up ($50 per million and $7 per 100,000) (20). A $0 Free plan with 50,000 requests a month and a free BSD-3-Clause self-hosted edition. The pricing page doesn't say whether a card is needed, but the Free plan has no charge and the self-hosted edition needs no account (20). A person signs up in a browser, then approves OAuth or creates a key (0). 0+20+20+0 = 40.

Heap 25

No x402, MPP or L402 (0). The Free plan and its 10,000-session limit are public. Growth needs an estimate after signup and Pro and Premier say Contact Us, so no paid price is published (5). Free plan with APIs included. No card is mentioned on the pricing page, and the signup form needs JavaScript we couldn't run (20). A person signs up in a browser and copies the environment ID (0). Total 25. Per the 7 October check, not re-read on 4 November (www.heap.io is paused on its terms).

Transparency & trust Flagsmith +11

Flagsmith 77

BSD-3-Clause for the core and the MCP server; the about page says most of the product is under that licence and some CI runs with private packages, so not every Cloud feature is in the open repository (25 of 30). The terms set Flagsmith as processor with 48-hour breach notice and EEA transfer rules, and the sub-processor list is dated 13 October 2025. The privacy policy dates from October 2023 and states no retention periods (16 of 30). The Gram-hosted MCP server was retired with a dated notice (30 June 2026), and three spec operations are marked deprecated, without a stated deprecation policy (10 of 20). The sub-processor list names providers and their countries (16 of 20). 25+16+10+16 = 67.

Heap 66

Closed service with published terms, now Contentsquare's master services agreement (15). The DPA (June 2026) caps Product Analytics retention at 37 months from collection unless agreed otherwise and links the sub-processor list. heap.io still carries a 2019 website terms page from Heap Inc. beside the Contentsquare documents (22). No deprecation policy for the API was found. Contentsquare's docs carry migration guides from the Heap SDKs to the CSQ SDK without an end date that we found (4). Sub-processor list (June 2026) with locations, AWS in Ireland, Frankfurt and Virginia and OpenAI for Sense Chat (20). Total 61. Per the 7 October check, not re-read on 4 November, apart from the heap.io website terms (still dated 30 August 2019) and Contentsquare's terms of service (29 January 2026).

Schema & documentation Flagsmith +10

Flagsmith 67

openapi.yaml in the repository is OpenAPI 3.1 with 624 operations, and the MCP server builds its 54 tools from the operations tagged mcp, so every tool has a JSON Schema input (25). No llms.txt (404) and no Markdown served for agents on docs.flagsmith.com; the docs are Markdown only in the repository (0). Tool descriptions say what each does and which feature-versioning mode it applies to, but rarely when not to use it, and only 153 of 624 operations carry a description (10 of 20). 57 enums and required fields in the spec, with some free-form value fields (10 of 15). Docs carry curl examples, but only 16 operations document a 400 and 6 a 404 (7 of 15). A /api/v1 path, release-please changelog and dated release notes (15). 25+0+10+10+7+15 = 67.

Heap 57

OpenAPI 3.1 fragments are embedded in the Markdown of the track and identify pages. No single spec file was found, the property pages we read had empty fragments and user deletion is prose only (15). llms.txt and a .md copy of every page (10). Descriptions state purpose, limits and reserved keys, with little on when not to use a call (12). Required fields are marked and lengths stated in prose, with string types throughout, no enums and a free-form properties object (8). Curl examples on every page. Track and identify document 200 and 400 with an empty object, and only user deletion describes its error body (7). SDK changelogs are dated, but no changelog or version policy for the server-side API was found. Paths mix unversioned, v1 and v0 (5). Total 57. Per the 7 October check, not re-read on 4 November, because developers.heap.io sits on a host whose terms bar robots.

Score by category

CategoryWeight this runFlagsmithHeapEdge
Reliability16%206547Flagsmith +18
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.26757Flagsmith +10
Agent ergonomics13%16.25549Flagsmith +6
Security & auth14%17.56541Flagsmith +24
Payments & pricing10%12.54025Flagsmith +15
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.88363Flagsmith +20
Transparency & trust7%8.87766Flagsmith +11
Negative events≤15-20
TotalC 61.2/100D 48.2/100

Facts side by side

FactFlagsmithHeap
KindHTTP APIHTTP API
VendorBullet Train LtdContentsquare (Content Square, Inc.)
Hosted endpointhttps://mcp.flagsmith.comhttps://heapanalytics.com
TransportsHTTP, Streamable HTTP, stdioHTTP
AuthOAuth or keyOAuth or key
PricingFreemiumFreemium
x402nono
LicenceBSD-3-ClauseProprietary service under Contentsquare's master services agreement. The React Native bridge and the Node client on npm are MIT
Tools exposed54none
Read-only variant documentedyesno
llms.txtnoyes
Last release2026-11-052026-10-06
Terms last updated2026-09-01no date given
Privacy policy last updated2023-10-01couldn't be read
Customer content may train modelsnot found in the textyes
Terms restrict automated accessnot found in the textnot found in the text
Terms restrict benchmarkingnot found in the textyes
Terms or service can change without noticenot found in the textnot found in the text
Arbitration or class-action waivernot found in the textnot found in the text
Popularity6.6k stars213 npm/wk

How the scores have moved

Verdicts

Flagsmith

Flagsmith pairs a BSD-licensed core with a hosted MCP server whose 54 tools come from its OpenAPI spec and carry read and destructive hints. A public 99.95 per cent SLA and SOC 2 report back the Cloud service. The MCP server is still an early release, and fine-grained key roles need a paid Scale-Up or Enterprise plan.

Heap

Heap's server-side API suits an agent that records events. Limits are published, bulk calls take 1,000 items and track accepts an idempotency key. The API returns no analytics, and ingest calls carry no secret beyond the environment ID. Heap's help centre lists a Heap MCP article, which a bot check stopped us reading, so the MCP server is ungraded.

Before you call either

Flagsmith

  1. Send the key as Authorization: Api-Key <key>; the Api-Key prefix is required and a bare key fails
  2. Check whether the environment uses v2 feature versioning before choosing between segment-override and version-state tools
  3. Use a key with a read-only RBAC role for analysis tasks, since a Free plan key has organisation admin rights
  4. Pace Management API calls under 500 a minute; no Retry-After guidance is documented
  5. Treat experiment results as possibly null until Flagsmith has computed them

Heap

  1. Send EU projects to https://c.eu.heap-api.com instead of https://heapanalytics.com. The path stays the same
  2. Pass one of identity or user_id on track, never both
  3. Set idempotency_key on every track event so a retry doesn't duplicate it
  4. Stay under 30 requests per 30 seconds per identity, or batch up to 1,000 events in one bulk call
  5. For deletion, exchange the app_id and API key of the Main Production environment for a token first. Any other environment ID returns Unauthorized

Questions

Which is better for AI agents, Flagsmith or Heap?

Flagsmith scores 61.2 (C) on agent readiness against Heap's 48.2 (D), and leads in every scored category.

Do Flagsmith and Heap need an API key?

Both take an API key or an OAuth sign-in.

Can an agent call Flagsmith and Heap without installing anything?

Yes. Flagsmith has a hosted endpoint at https://mcp.flagsmith.com and Heap at https://heapanalytics.com.

Are Flagsmith and Heap open source?

Flagsmith is open source (BSD-3-Clause). No open-source release is listed for Heap.

Other comparisons with Flagsmith or Heap

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.