Countly vs Harness Feature Management & Experimentation

Countly 23

Graded on the hosted lines, for a Countly Flex server reached through its HTTP API and the official MCP server. No status page was found on countly.com, the Flex page, the docs or the legal pages, and status.countly.com did not resolve on 4 November (0). With no page there is no readable incident history (5). The MCP server's HTTP endpoint is limited to 120 requests a minute per IP, but the Countly server's own api_rate_limit_requests setting ships at 0 (off), and the Flex terms of 20 October 2026 reserve usage limits without numbers (8 of 15, the earlier partial kept). No 429 or retry guidance and no idempotency keys were found in the API reference source (0). The pricing page lists an SLA as optional on Flex and included in Enterprise, with no SLA document published, which this run scores 0 (0). The API and MCP server 1.7.0 are generally available (10).

Harness Feature Management & Experimentation 80

Graded on the hosted FME Admin API with the hosted lines. Statuspage at status.harness.io with an FME component among 196 (20). In the 90 days from 8 August, two FME incidents: API write operations returned 499 errors for 30 minutes on 20 August (minor) and the FME user interface was unavailable for 15 minutes on 23 August (major, interface only; flag evaluation was unaffected per the post). No API outage of an hour or more (20 of 30). About 20 Admin API requests per 10 seconds per organisation and per IP, documented on the MCP page (15). 429 responses carry remaining and reset headers with guidance to wait for the larger reset, and the MCP server backs off and retries three times (15). No SLA was found on harness.io/pricing or the FME docs (0). FME is generally available and the MCP server is at 3.2.40 (10). 20+20+15+15+0+10 = 80.

Countly 66

The MCP README counts 209 tools in 43 categories, far past 30 (5). Add back 8 because the server hides tools the edition, enabled plugins or user's permissions cannot run, and COUNTLY_TOOLS_* settings switch categories off (13). List endpoints take iDisplayStart and iDisplayLength, and MCP list tools take skip, limit and search (15). API errors are {"result": "message"} strings with HTTP status codes and no error codes. The MCP server returns failures as isError results that name the missing plugin or permission (12). MCP tools carry readOnlyHint, destructiveHint and idempotentHint since 1.7.0. No idempotency keys on API writes (14). MCP tools accept app_name in place of app_id and set defaults. The SDKs cover data collection, not the read API (12).

Harness Feature Management & Experimentation 83

11 consolidated tools, and --toolsets=fme narrows the server to FME (25). Flag lists filter server-side by name, tag and rollout status, and the Admin API paginates with markerLimit (17 of 20). Documented status codes, 429 headers, and a harness_diagnose tool. Error bodies aren't documented per endpoint (14 of 20). harness_list and harness_get carry readOnlyHint, write tools ask for confirmation by elicitation, and harness_delete is marked destructive. No idempotency keys (15 of 20). Two required settings (API key, optional org and project, account ID read from the token); a Python Admin API wrapper and CLI, and SDKs in more than a dozen languages for evaluation (12 of 15). 25+17+14+15+12 = 83.

Countly 62

Auth tokens are scoped to apps and endpoint patterns, expire after a set lifetime, are revocable and carry create, read, update and delete permissions since 25.03.54. The dashboard api_key has its user's full access (26 of 30). Less 10 because api_key and auth_token are documented as query-string parameters, with the countly-token header as the recommended alternative (16). MCP read-only mode and per-category CRUD levels, bounded by the token's permissions. No confirmation step for destructive tools (16). Tools return event names, user properties and crash data written by end users, and no prompt-injection guidance was found (3). A system logs API records dashboard actions (12, kept; the 8 October count of log types came from the help centre's JSON API and is struck). No security.txt. SECURITY.md gives security@count.ly as the reporting route with no acknowledgement time and a bounty whose rewards are at Countly's discretion, which this run doesn't count as a bounty. Countly publishes advisories for countly-server on GitHub, and the site claims ISO 27001, 27017, 27018 and SOC 2 Type II without public reports. 15, as daytona has for a disclosure route and certifications with a discretionary reward.

Harness Feature Management & Experimentation 79

Admin API keys with combinable roles that can't be widened after creation, Harness PATs and SATs for central management, and Harness Platform OAuth on the hosted MCP endpoint. Credentials travel in headers (28 of 30). HARNESS_READ_ONLY removes every write, writes go through elicitation with a risk threshold for auto-approval, and FME change requests add approval for flag changes (19 of 20). No prompt-injection guidance; the tools return flag names, descriptions and rule data written by people (3 of 15). FME admin, environment and object audit logs, plus structured MCP audit events to file, webhook or OpenTelemetry sinks (13 of 15). SOC 1 and SOC 2 Type II, ISO 27001, 27017 and 27018, FedRAMP 20x, a security@harness.io disclosure route with a 24-hour response, a private bug bounty on request and annual external penetration tests. No security.txt (16 of 20). 28+19+3+13+16 = 79.

Countly 72

Countly Lite is AGPL-3.0 with added branding terms under Section 7, the MCP server is MIT, and the Enterprise plugins that hold funnels, retention, cohorts and Drill are closed (20 of 30). Flex now has its own terms, privacy policy, DPA and sub-processor list, all dated 20 October 2026. The Flex privacy policy excludes data customers process and gives no retention periods, and the DPA covers that data (20). The MCP changelog deprecated auth tokens in URL parameters on 23 April 2026 (1.3.0), and the server warns on their use, with no notice period (15, the settled value for dated notices without a period). The Flex sub-processor list names locations, with 30 days' notice of changes in the DPA, and Flex customers choose a Google Cloud region. A Connected Services and Telemetry notice (20 October 2026) says server reports to stats.count.ly are on by default and which settings turn them off, and MCP usage reporting is disclosed with an opt-out (18).

Harness Feature Management & Experimentation 61

A closed service under Harness's subscription terms; the MCP server is MIT (18 of 30). The privacy statement covers website and sales data and defers customer data to the customer agreement and a published DPA; retention periods for customer data weren't found (18 of 30). A versioning policy for SDKs with support windows, and Split-era API endpoints marked deprecated with migration guides (14 of 20). Sub-processors are on the trust centre, which we didn't read, and the security page doesn't name data locations (10 of 20). 18+18+14+10 = 60.