Countly vs Flagsmith

Countly 23

Graded on the hosted lines, for a Countly Flex server reached through its HTTP API and the official MCP server. No status page was found on countly.com, the Flex page, the docs or the legal pages, and status.countly.com did not resolve on 4 November (0). With no page there is no readable incident history (5). The MCP server's HTTP endpoint is limited to 120 requests a minute per IP, but the Countly server's own api_rate_limit_requests setting ships at 0 (off), and the Flex terms of 20 October 2026 reserve usage limits without numbers (8 of 15, the earlier partial kept). No 429 or retry guidance and no idempotency keys were found in the API reference source (0). The pricing page lists an SLA as optional on Flex and included in Enterprise, with no SLA document published, which this run scores 0 (0). The API and MCP server 1.7.0 are generally available (10).

Flagsmith 65

Graded on Flagsmith Cloud with the hosted lines. Statuspage at status.flagsmith.com with components for the Edge API, Admin API, dashboard and website (20). In the 90 days from 8 August, an Admin and SDK API outage on 28 October marked critical, posted at 21:56 UTC and resolved at 22:12 with no cause given, and a minor delay in processing flag changes on 10 September. The posted span of the outage is 16 minutes and its start isn't stated, so we read it as one serious incident under an hour (15 of 30). The Management API limit of 500 requests a minute is documented (15). No 429 handling, Retry-After or idempotency guidance was found in the docs (0). A public SLA commits to 99.95 per cent monthly uptime with service credits (10). The Management API is generally available, but the MCP docs call the server an early release, so half (5 of 10). 20+15+15+0+10+5 = 65.

Countly 17

No x402, MPP or L402 (0). The pricing page gives only a starting price for Flex, $175 a month, priced on monthly active users and data points, with tier prices shown during signup and Enterprise on custom pricing (7 of 20, the earlier partial kept). The 14-day Flex trial needs a credit card, per the Flex page. Countly Lite is free to self-host with no card, without funnels, retention, cohorts or Drill, so half credit (10). A person signs up in a browser, creates a server and makes a token in the dashboard (0).

Flagsmith 40

No x402, MPP or L402 (0). Plan prices and the per-million overage are public without login, though the page gives two overage rates for Start-Up ($50 per million and $7 per 100,000) (20). A $0 Free plan with 50,000 requests a month and a free BSD-3-Clause self-hosted edition. The pricing page doesn't say whether a card is needed, but the Free plan has no charge and the self-hosted edition needs no account (20). A person signs up in a browser, then approves OAuth or creates a key (0). 0+20+20+0 = 40.

Countly 66

The MCP README counts 209 tools in 43 categories, far past 30 (5). Add back 8 because the server hides tools the edition, enabled plugins or user's permissions cannot run, and COUNTLY_TOOLS_* settings switch categories off (13). List endpoints take iDisplayStart and iDisplayLength, and MCP list tools take skip, limit and search (15). API errors are {"result": "message"} strings with HTTP status codes and no error codes. The MCP server returns failures as isError results that name the missing plugin or permission (12). MCP tools carry readOnlyHint, destructiveHint and idempotentHint since 1.7.0. No idempotency keys on API writes (14). MCP tools accept app_name in place of app_id and set defaults. The SDKs cover data collection, not the read API (12).

Flagsmith 55

54 tools, more than 30, with no toolsets or read-only subset; enterprise tools only appear where installed (5 of 25). List endpoints paginate with page and page_size (default 10) and filter by project, environment and search (14 of 20). Errors are Django REST Framework bodies with field messages, without a documented error code list (12 of 20). Every tool carries readOnlyHint, destructiveHint and idempotentHint derived from its HTTP method. No idempotency keys for POST (14 of 20). Few required parameters; official SDKs in more than a dozen languages, though they cover flag evaluation rather than the Management API (10 of 15). 5+14+12+14+10 = 55.

Countly 75

No OpenAPI document was found in the API reference source (Countly/countly-docs-site) or in Countly/countly-server. Every MCP tool has a typed JSON Schema input (18 of 25). countly.com/llms.txt (77 KB) indexes the help centre for agents, while api.count.ly has no llms.txt and help article pages answered a bot check on 8 October (8). MCP tool descriptions name the endpoint, the plugin required and the tool to use instead, and API pages state permissions and behaviour (16). 82 enums across the tool sources, but the HTTP API takes JSON arrays and objects as query-string values (9). API pages carry request examples, response fields and error responses, as plain message strings (11). The API reference is versioned 26.01, 25.03 and 24.05 (611 pages for 26.01 in the source), and the server and MCP server keep public changelogs (13).

Flagsmith 67

openapi.yaml in the repository is OpenAPI 3.1 with 624 operations, and the MCP server builds its 54 tools from the operations tagged mcp, so every tool has a JSON Schema input (25). No llms.txt (404) and no Markdown served for agents on docs.flagsmith.com; the docs are Markdown only in the repository (0). Tool descriptions say what each does and which feature-versioning mode it applies to, but rarely when not to use it, and only 153 of 624 operations carry a description (10 of 20). 57 enums and required fields in the spec, with some free-form value fields (10 of 15). Docs carry curl examples, but only 16 operations document a 400 and 6 a 404 (7 of 15). A /api/v1 path, release-please changelog and dated release notes (15). 25+0+10+10+7+15 = 67.

Countly 72

Countly Lite is AGPL-3.0 with added branding terms under Section 7, the MCP server is MIT, and the Enterprise plugins that hold funnels, retention, cohorts and Drill are closed (20 of 30). Flex now has its own terms, privacy policy, DPA and sub-processor list, all dated 20 October 2026. The Flex privacy policy excludes data customers process and gives no retention periods, and the DPA covers that data (20). The MCP changelog deprecated auth tokens in URL parameters on 23 April 2026 (1.3.0), and the server warns on their use, with no notice period (15, the settled value for dated notices without a period). The Flex sub-processor list names locations, with 30 days' notice of changes in the DPA, and Flex customers choose a Google Cloud region. A Connected Services and Telemetry notice (20 October 2026) says server reports to stats.count.ly are on by default and which settings turn them off, and MCP usage reporting is disclosed with an opt-out (18).

Flagsmith 77

BSD-3-Clause for the core and the MCP server; the about page says most of the product is under that licence and some CI runs with private packages, so not every Cloud feature is in the open repository (25 of 30). The terms set Flagsmith as processor with 48-hour breach notice and EEA transfer rules, and the sub-processor list is dated 13 October 2025. The privacy policy dates from October 2023 and states no retention periods (16 of 30). The Gram-hosted MCP server was retired with a dated notice (30 June 2026), and three spec operations are marked deprecated, without a stated deprecation policy (10 of 20). The sub-processor list names providers and their countries (16 of 20). 25+16+10+16 = 67.