Countly vs Mostly Good Metrics

Countly 62

Auth tokens are scoped to apps and endpoint patterns, expire after a set lifetime, are revocable and carry create, read, update and delete permissions since 25.03.54. The dashboard api_key has its user's full access (26 of 30). Less 10 because api_key and auth_token are documented as query-string parameters, with the countly-token header as the recommended alternative (16). MCP read-only mode and per-category CRUD levels, bounded by the token's permissions. No confirmation step for destructive tools (16). Tools return event names, user properties and crash data written by end users, and no prompt-injection guidance was found (3). A system logs API records dashboard actions (12, kept; the 8 October count of log types came from the help centre's JSON API and is struck). No security.txt. SECURITY.md gives security@count.ly as the reporting route with no acknowledgement time and a bounty whose rewards are at Countly's discretion, which this run doesn't count as a bounty. Countly publishes advisories for countly-server on GitHub, and the site claims ISO 27001, 27017, 27018 and SOC 2 Type II without public reports. 15, as daytona has for a disclosure route and certifications with a discretionary reward.

Mostly Good Metrics 33

OAuth 2.0 with PKCE, dynamic client registration and a revocation endpoint for MCP and the CLI; project ingest keys in a Bearer header that can be pinned to origins or app IDs. The only tool scope advertised is mcp:tools, so a connection gets every tool the user can use (24 of 30). Restricted keys and a confirmation step for unrestricted keys in the CLI; no read-only MCP mode or approval for deletes was found (6 of 20). Event properties come from apps and end users and reach the agent, and no prompt-injection guidance was found (3 of 15). No audit log was found (0). No security.txt, disclosure policy or certification was found (0). 24+6+3+0+0 = 33.

Countly 72

Countly Lite is AGPL-3.0 with added branding terms under Section 7, the MCP server is MIT, and the Enterprise plugins that hold funnels, retention, cohorts and Drill are closed (20 of 30). Flex now has its own terms, privacy policy, DPA and sub-processor list, all dated 20 October 2026. The Flex privacy policy excludes data customers process and gives no retention periods, and the DPA covers that data (20). The MCP changelog deprecated auth tokens in URL parameters on 23 April 2026 (1.3.0), and the server warns on their use, with no notice period (15, the settled value for dated notices without a period). The Flex sub-processor list names locations, with 30 days' notice of changes in the DPA, and Flex customers choose a Google Cloud region. A Connected Services and Telemetry notice (20 October 2026) says server reports to stats.count.ly are on by default and which settings turn them off, and MCP usage reporting is disclosed with an opt-out (18).

Mostly Good Metrics 45

The service is closed; the SDKs and CLI are MIT. Terms of December 2025 with no legal entity named (18 of 30). The privacy policy and terms agree on raw-event retention for the subscription's life and deletion within 14 days of account deletion; the about page says data is not used to train models. No DPA was found (18 of 30). The legacy X-MGM-Key header is still supported, and no deprecation policy was found (3 of 20). The terms name eight sub-processors with their purpose, without data locations (12 of 20). 18+18+3+12 = 51.

Countly 75

No OpenAPI document was found in the API reference source (Countly/countly-docs-site) or in Countly/countly-server. Every MCP tool has a typed JSON Schema input (18 of 25). countly.com/llms.txt (77 KB) indexes the help centre for agents, while api.count.ly has no llms.txt and help article pages answered a bot check on 8 October (8). MCP tool descriptions name the endpoint, the plugin required and the tool to use instead, and API pages state permissions and behaviour (16). 82 enums across the tool sources, but the HTTP API takes JSON arrays and objects as query-string values (9). API pages carry request examples, response fields and error responses, as plain message strings (11). The API reference is versioned 26.01, 25.03 and 24.05 (611 pages for 26.01 in the source), and the server and MCP server keep public changelogs (13).

Mostly Good Metrics 49

No OpenAPI file was found. The docs say every MCP tool describes its own parameters, and the query definition object is documented in full; we didn't open an MCP session to read the schemas (12 of 25). No llms.txt (404) and no Markdown pages served for agents (0). Each tool area is documented with examples, and the docs say when to use query_metrics instead of repeated execute_query calls (10 of 20). The query definition lists metrics, dimensions, date ranges and filter operators (9 of 15). The ingestion API documents its error body and status codes, and query_metrics reports per-item errors (11 of 15). /v1 and /api/v2 paths and dated SDK tags, but no product changelog (7 of 15). 12+0+10+9+11+7 = 49.

Countly 66

The MCP README counts 209 tools in 43 categories, far past 30 (5). Add back 8 because the server hides tools the edition, enabled plugins or user's permissions cannot run, and COUNTLY_TOOLS_* settings switch categories off (13). List endpoints take iDisplayStart and iDisplayLength, and MCP list tools take skip, limit and search (15). API errors are {"result": "message"} strings with HTTP status codes and no error codes. The MCP server returns failures as isError results that name the missing plugin or permission (12). MCP tools carry readOnlyHint, destructiveHint and idempotentHint since 1.7.0. No idempotency keys on API writes (14). MCP tools accept app_name in place of app_id and set defaults. The SDKs cover data collection, not the read API (12).

Mostly Good Metrics 49

53 tools, more than 30 (5). We added 3 because Goals and Alerts tools load only where enabled and query_metrics batches up to 10 queries (8 of 25). Date ranges, grouping and filters on queries; no documented page or limit parameters for lists (12 of 20). JSON errors with a message, and per-item status in batch results (13 of 20). Annotations weren't readable without a session, the tool list includes deletes and key revocation, and no idempotency keys were found (4 of 20). Five client SDKs and a CLI with --json output (12 of 15). 8+12+13+4+12 = 49.

Countly 17

No x402, MPP or L402 (0). The pricing page gives only a starting price for Flex, $175 a month, priced on monthly active users and data points, with tier prices shown during signup and Enterprise on custom pricing (7 of 20, the earlier partial kept). The 14-day Flex trial needs a credit card, per the Flex page. Countly Lite is free to self-host with no card, without funnels, retention, cohorts or Drill, so half credit (10). A person signs up in a browser, creates a server and makes a token in the dashboard (0).

Mostly Good Metrics 30

No x402, MPP or L402 (0). Plan prices with event allowances are on the home page; no per-event price (10). Free plan with no card, stated on the home and about pages (20). Signup and OAuth approval happen in a browser (0). 0+10+20+0 = 30.

Countly 23

Graded on the hosted lines, for a Countly Flex server reached through its HTTP API and the official MCP server. No status page was found on countly.com, the Flex page, the docs or the legal pages, and status.countly.com did not resolve on 4 November (0). With no page there is no readable incident history (5). The MCP server's HTTP endpoint is limited to 120 requests a minute per IP, but the Countly server's own api_rate_limit_requests setting ships at 0 (off), and the Flex terms of 20 October 2026 reserve usage limits without numbers (8 of 15, the earlier partial kept). No 429 or retry guidance and no idempotency keys were found in the API reference source (0). The pricing page lists an SLA as optional on Flex and included in Enterprise, with no SLA document published, which this run scores 0 (0). The API and MCP server 1.7.0 are generally available (10).

Mostly Good Metrics 29

Graded on the hosted service with the hosted lines. No status page was found (0) and no incident history could be read (5). Ingestion limits are documented per plan (1,000 to 100,000 events a minute and 10,000 to 1,000,000 an hour), but no limit for MCP or query calls was found (12 of 15). 429 with Retry-After in the header and body, and the SDKs queue events and back off. No idempotency or safe-retry guidance for writes (12 of 15). No SLA (0). The press kit calls the product a public beta (0). 0+5+12+12+0+0 = 29.