Countly vs OpenPanel

Countly 23

Graded on the hosted lines, for a Countly Flex server reached through its HTTP API and the official MCP server. No status page was found on countly.com, the Flex page, the docs or the legal pages, and status.countly.com did not resolve on 4 November (0). With no page there is no readable incident history (5). The MCP server's HTTP endpoint is limited to 120 requests a minute per IP, but the Countly server's own api_rate_limit_requests setting ships at 0 (off), and the Flex terms of 20 October 2026 reserve usage limits without numbers (8 of 15, the earlier partial kept). No 429 or retry guidance and no idempotency keys were found in the API reference source (0). The pricing page lists an SLA as optional on Flex and included in Enterprise, with no SLA document published, which this run scores 0 (0). The API and MCP server 1.7.0 are generally available (10).

OpenPanel 70

Graded on OpenPanel Cloud with the hosted lines. OpenStatus page at status.openpanel.dev with API and Tracker components (20). In the 90 days from 8 August, one incident, degraded API performance for about two hours on 1 October 2026, caused by Hetzner private-network problems reaching the ClickHouse servers (15 of 30). Limits are documented with numbers for MCP (60 a minute per client) and the Export, Insights and Manage APIs (15). The docs say a 429 is returned and to back off exponentially; no Retry-After or idempotency guidance (10 of 15). No SLA was found (0). Generally available (10). 20+15+15+10+0+10 = 70.

Countly 62

Auth tokens are scoped to apps and endpoint patterns, expire after a set lifetime, are revocable and carry create, read, update and delete permissions since 25.03.54. The dashboard api_key has its user's full access (26 of 30). Less 10 because api_key and auth_token are documented as query-string parameters, with the countly-token header as the recommended alternative (16). MCP read-only mode and per-category CRUD levels, bounded by the token's permissions. No confirmation step for destructive tools (16). Tools return event names, user properties and crash data written by end users, and no prompt-injection guidance was found (3). A system logs API records dashboard actions (12, kept; the 8 October count of log types came from the help centre's JSON API and is struck). No security.txt. SECURITY.md gives security@count.ly as the reporting route with no acknowledgement time and a bounty whose rewards are at Countly's discretion, which this run doesn't count as a bounty. Countly publishes advisories for countly-server on GitHub, and the site claims ISO 27001, 27017, 27018 and SOC 2 Type II without public reports. 15, as daytona has for a disclosure route and certifications with a discretionary reward.

OpenPanel 41

Client credentials with three types (write, read for one project, root for the organisation), created and deleted in the dashboard (25), less 10 because the docs give ?token= in the MCP URL as an option (15 of 30). Read clients are read-only and limited to one project, and onboarding creates a read-only MCP client. No approval step for root-client writes (14 of 20). Tools return event properties, page titles and profile data sent from tracked sites, and no prompt-injection guidance was found (3 of 15). No audit log was found (0). SECURITY.md with a private advisory route and response times, a valid security.txt, and no bug bounty or certification found (9 of 20). 15+14+3+0+9 = 41.

Countly 17

No x402, MPP or L402 (0). The pricing page gives only a starting price for Flex, $175 a month, priced on monthly active users and data points, with tier prices shown during signup and Enterprise on custom pricing (7 of 20, the earlier partial kept). The 14-day Flex trial needs a credit card, per the Flex page. Countly Lite is free to self-host with no card, without funnels, retention, cohorts or Drill, so half credit (10). A person signs up in a browser, creates a server and makes a token in the dashboard (0).

OpenPanel 35

No x402, MPP or L402 (0). Public prices by event tier from $2.50 a month, with no overage charges; tiers rather than a per-event rate (15 of 20). 30-day free trial with no card, and free self-hosting (20). Signup is in a browser and client keys are created in the dashboard (0). 0+15+20+0 = 35.

Countly 66

The MCP README counts 209 tools in 43 categories, far past 30 (5). Add back 8 because the server hides tools the edition, enabled plugins or user's permissions cannot run, and COUNTLY_TOOLS_* settings switch categories off (13). List endpoints take iDisplayStart and iDisplayLength, and MCP list tools take skip, limit and search (15). API errors are {"result": "message"} strings with HTTP status codes and no error codes. The MCP server returns failures as isError results that name the missing plugin or permission (12). MCP tools carry readOnlyHint, destructiveHint and idempotentHint since 1.7.0. No idempotency keys on API writes (14). MCP tools accept app_name in place of app_id and set defaults. The SDKs cover data collection, not the read API (12).

OpenPanel 54

51 MCP tools, more than 30 (5). We added 3 because a read client doesn't get the 9 dashboard management tools (8 of 25). Insights endpoints take startDate, endDate, range, filters, cursor and limit (18 of 20). Errors aren't documented beyond 401 and 429 (12 of 20). No readOnlyHint or destructiveHint in the MCP source, no idempotency keys; delete_dashboard refuses a non-empty dashboard without forceDelete (3 of 20). Few required parameters, and SDKs for JavaScript, React, Next.js, Vue, React Native, Swift, Kotlin and Python (13 of 15). 8+18+12+3+13 = 54.

Countly 72

Countly Lite is AGPL-3.0 with added branding terms under Section 7, the MCP server is MIT, and the Enterprise plugins that hold funnels, retention, cohorts and Drill are closed (20 of 30). Flex now has its own terms, privacy policy, DPA and sub-processor list, all dated 20 October 2026. The Flex privacy policy excludes data customers process and gives no retention periods, and the DPA covers that data (20). The MCP changelog deprecated auth tokens in URL parameters on 23 April 2026 (1.3.0), and the server warns on their use, with no notice period (15, the settled value for dated notices without a period). The Flex sub-processor list names locations, with 30 days' notice of changes in the DPA, and Flex customers choose a Google Cloud region. A Connected Services and Telemetry notice (20 October 2026) says server reports to stats.count.ly are on by default and which settings turn them off, and MCP usage reporting is disclosed with an opt-out (18).

OpenPanel 83

AGPL-3.0 for the whole product, and the about page says there is no enterprise edition (30). Terms and privacy policy of 21 May 2026 agree, with a DPA published, transient IP use, data ownership kept by the customer, and providers named with locations. Retention periods are general rather than stated in days (22 of 30). No deprecation policy or dated notices found; v3.0.0 was a major release on 16 October 2026 (4 of 20). Sub-processors and data locations disclosed in the privacy policy (Hetzner, Germany) (18 of 20). 30+22+4+18 = 74.