Flagsmith vs Matomo

Flagsmith 65

Graded on Flagsmith Cloud with the hosted lines. Statuspage at status.flagsmith.com with components for the Edge API, Admin API, dashboard and website (20). In the 90 days from 8 August, an Admin and SDK API outage on 28 October marked critical, posted at 21:56 UTC and resolved at 22:12 with no cause given, and a minor delay in processing flag changes on 10 September. The posted span of the outage is 16 minutes and its start isn't stated, so we read it as one serious incident under an hour (15 of 30). The Management API limit of 500 requests a minute is documented (15). No 429 handling, Retry-After or idempotency guidance was found in the docs (0). A public SLA commits to 99.95 per cent monthly uptime with service credits (10). The Management API is generally available, but the MCP docs call the server an early release, so half (5 of 10). 20+15+15+0+10+5 = 65.

Matomo 38

Graded on the hosted lines, for a Matomo Cloud instance reached through the Reporting HTTP API and the MCP server included with Cloud. status.matomo.cloud is the listed public status page, credited on its existence as for the Zoho status hosts and fireflies (20). On 4 November its robots.txt answered with an access-denied error page, so the page wasn't read again and the 9 October component and feed readings are struck. No 90-day incident record could be read then or now (5). No API rate limit numbers were found. The terms reserve suspension for excessively frequent requests, and API.getBulkRequest is capped at 50 URLs for anonymous callers (3 of 15). No 429 or retry guidance and no idempotency keys were found (0). No SLA for Cloud was found, and the terms supply the service as is (0). The Reporting API and MCP plugin 5.1.1 are stable releases (10).

Flagsmith 55

54 tools, more than 30, with no toolsets or read-only subset; enterprise tools only appear where installed (5 of 25). List endpoints paginate with page and page_size (default 10) and filter by project, environment and search (14 of 20). Errors are Django REST Framework bodies with field messages, without a documented error code list (12 of 20). Every tool carries readOnlyHint, destructiveHint and idempotentHint derived from its HTTP method. No idempotency keys for POST (14 of 20). Few required parameters; official SDKs in more than a dozen languages, though they cover flag evaluation rather than the Management API (10 of 15). 5+14+12+14+10 = 55.

Matomo 78

The MCP server has 19 tools, 12 of them visible by default (15), plus 6 because the seven raw API tools stay hidden until an administrator allows them by read, create, update or delete class (21). filter_limit, filter_offset, sorting, pattern filters, showColumns and segments on the API, and cursors with a 250-row cap on the MCP report tool (20). Errors are HTTP codes with a message and an optional integer code, with no code catalogue. The MCP server separates schema failures (-32602) from tool result errors that name the arguments involved (14). Every MCP tool carries readOnlyHint, destructiveHint and idempotentHint. API writes have no idempotency keys and the OpenAPI descriptions declare delete methods as GET (13). Report calls need only idSite, period and date. The official clients for JavaScript, PHP and Java cover tracking, not the Reporting API (10).

Flagsmith 40

No x402, MPP or L402 (0). Plan prices and the per-million overage are public without login, though the page gives two overage rates for Start-Up ($50 per million and $7 per 100,000) (20). A $0 Free plan with 50,000 requests a month and a free BSD-3-Clause self-hosted edition. The pricing page doesn't say whether a card is needed, but the Free plan has no charge and the self-hosted edition needs no account (20). A person signs up in a browser, then approves OAuth or creates a key (0). 0+20+20+0 = 40.

Matomo 30

Scored for Matomo Cloud, the hosted option. No x402, MPP or L402 (0). Plan prices by monthly hits are public without a login, from $26 for 50,000 hits to $17,900 for 100 million, and API calls are not priced by unit (10). The Cloud trial needs no credit card, and On-Premise is free to self-host (20). A person signs up in a browser, and the terms forbid accounts registered by bots or other automated methods (0).

Flagsmith 67

openapi.yaml in the repository is OpenAPI 3.1 with 624 operations, and the MCP server builds its 54 tools from the operations tagged mcp, so every tool has a JSON Schema input (25). No llms.txt (404) and no Markdown served for agents on docs.flagsmith.com; the docs are Markdown only in the repository (0). Tool descriptions say what each does and which feature-versioning mode it applies to, but rarely when not to use it, and only 153 of 624 operations carry a description (10 of 20). 57 enums and required fields in the spec, with some free-form value fields (10 of 15). Docs carry curl examples, but only 16 operations document a 400 and 6 a 404 (7 of 15). A /api/v1 path, release-please changelog and dated release notes (15). 25+0+10+10+7+15 = 67.

Matomo 73

Each of the 72 module pages in the API reference embeds an OpenAPI 3.1 description, and every MCP tool has a typed JSON Schema input. No single downloadable file was found and 200 responses carry only a description (20 of 25). developer.matomo.org/llms.txt still answers 404 on 4 November, and matomo.org/llms.txt did on 9 October. The docs source is Markdown in a public repository (2). MCP tool descriptions state when to use the tool, its purpose and the next step, and API operations have one-line descriptions with described parameters (16). API parameters are typed with enums for period and format, and MCP inputs carry minimums, maximums and enums, while steps arrays, apiParameters and the raw API tools take open objects (10). Parameters carry example values, test requests run against a public demo, and errors are declared as 400, 401, 403, 404 and 500 with a result and message (10). Docs are versioned 4.x, 5.x and 6.x, with a developer changelog per release (15).

Flagsmith 65

OAuth for interactive MCP clients with Flagsmith as the authorisation server, Organisation API keys in the Authorization header that act as admin or with chosen RBAC roles, and OIDC trust relationships that swap a CI token for a short-lived one. Granular key roles need Scale-Up or Enterprise, so a Free key is all-powerful (25 of 30). RBAC roles can limit a key to reads, and change requests put a flag change through approval on Scale-Up and Enterprise. There is no read-only switch on the MCP server itself (14 of 20). No prompt-injection guidance; tools return names, descriptions and trait values written by people (3 of 15). Audit logs record every admin action and stream by webhook, on Scale-Up and Enterprise (11 of 15). The docs say Flagsmith has a SOC 2 Type 2 report available on request, the terms promise a yearly third-party penetration test, and SECURITY.md routes reports to support email. No security.txt, bug bounty or public advisory process was found (12 of 20). 25+14+3+11+12 = 65.

Matomo 71

OAuth 2.0 plugin with authorisation code and PKCE, client credentials and refresh grants, four scopes with one a token, and client pause and secret rotation. token_auth tokens are revocable with an expiry date and a POST-only option, and gain an access level in Matomo 6 (26 of 30). Less 10 because the API docs present token_auth as a URL parameter (16). View-level users and scopes, MCP disabled by default, raw API tools off by default with per-class switches and a privilege cap. No confirmation step in the server (16 of 20). Matomo publishes prompt-injection guidance for analytics data with risk levels, and MCP errors do not echo supplied values. No filtering of tool output was found (11 of 15). Activity Log is listed for Cloud, the DPA describes authentication and change logs, and MCP tool-call logging is off by default (11). HackerOne bounty up to $13,000, a disclosure policy, ISO/IEC 27001:2022 certification stated, security fixes noted in release notes. No security.txt (17).