# Countly vs Harness Feature Management & Experimentation

> Harness Feature Management & Experimentation scores 69.2 (B) to Countly's 38.8 (E) for feature flags. Prices, MCP, x402, uptime and agent notes side by side.

- Canonical: https://www.anchorterminal.com/compare/countly-vs-harness-fme
- Markdown: https://www.anchorterminal.com/compare/countly-vs-harness-fme.md (~4,900 tokens)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-07

Harness Feature Management & Experimentation scores 69.2 (B) on agent readiness against Countly's 38.8 (E), and leads in 4 of 7 scored categories. Countly leads on transparency & trust. Both do feature flags.

- Countly: grade E, 38.8/100, rank #1352 of 1510. Markdown https://www.anchorterminal.com/tools/countly.md · JSON https://www.anchorterminal.com/api/v1/tools/countly.json
- Harness Feature Management & Experimentation: grade B, 69.2/100, rank #258 of 1510. Markdown https://www.anchorterminal.com/tools/harness-fme.md · JSON https://www.anchorterminal.com/api/v1/tools/harness-fme.json
- Best product analytics and experimentation tools for AI agents: https://www.anchorterminal.com/best/product-analytics/index.md
- All 170 product analytics comparisons: https://www.anchorterminal.com/compare/product-analytics/index.md

## Which one, for what

### Countly (E)

Good for: An agent whose owner runs Countly on its own infrastructure or a dedicated Flex server and wants analytics, crash and remote-config data through MCP with per-category permissions.

Ahead on:
- Transparency & trust, 72 against 61

Also in its favour:
- Open source

Watch for: No public status page or incident history was found for Flex servers or for mcp.count.ly

### Harness Feature Management & Experimentation (B)

Good for: Teams already on Harness, or Split customers, who want agents to inspect and change flags and segments with guard rails.

Ahead on:
- Reliability, 80 against 23
- Agent ergonomics, 83 against 66
- Security & auth, 79 against 62

Also in its favour:
- No incidents deducted, where Countly loses 15 points for them

Watch for: No FME prices are published; Enterprise is by quotation and free plan limits are unstated


## Why the edge

### Reliability: Harness Feature Management & Experimentation +57

- Countly (23): Graded on the hosted lines, for a Countly Flex server reached through its HTTP API and the official MCP server. No status page was found on countly.com, the Flex page, the docs or the legal pages, and status.countly.com did not resolve on 4 November (0). With no page there is no readable incident history (5). The MCP server's HTTP endpoint is limited to 120 requests a minute per IP, but the Countly server's own `api_rate_limit_requests` setting ships at 0 (off), and the Flex terms of 20 October 2026 reserve usage limits without numbers (8 of 15, the earlier partial kept). No 429 or retry guidance and no idempotency keys were found in the API reference source (0). The pricing page lists an SLA as optional on Flex and included in Enterprise, with no SLA document published, which this run scores 0 (0). The API and MCP server 1.7.0 are generally available (10).
- Harness Feature Management & Experimentation (80): Graded on the hosted FME Admin API with the hosted lines. Statuspage at status.harness.io with an FME component among 196 (20). In the 90 days from 8 August, two FME incidents: API write operations returned 499 errors for 30 minutes on 20 August (minor) and the FME user interface was unavailable for 15 minutes on 23 August (major, interface only; flag evaluation was unaffected per the post). No API outage of an hour or more (20 of 30). About 20 Admin API requests per 10 seconds per organisation and per IP, documented on the MCP page (15). 429 responses carry remaining and reset headers with guidance to wait for the larger reset, and the MCP server backs off and retries three times (15). No SLA was found on harness.io/pricing or the FME docs (0). FME is generally available and the MCP server is at 3.2.40 (10). 20+20+15+15+0+10 = 80.

### Agent ergonomics: Harness Feature Management & Experimentation +17

- Countly (66): The MCP README counts 209 tools in 43 categories, far past 30 (5). Add back 8 because the server hides tools the edition, enabled plugins or user's permissions cannot run, and `COUNTLY_TOOLS_*` settings switch categories off (13). List endpoints take `iDisplayStart` and `iDisplayLength`, and MCP list tools take `skip`, `limit` and `search` (15). API errors are `{"result": "message"}` strings with HTTP status codes and no error codes. The MCP server returns failures as `isError` results that name the missing plugin or permission (12). MCP tools carry readOnlyHint, destructiveHint and idempotentHint since 1.7.0. No idempotency keys on API writes (14). MCP tools accept `app_name` in place of `app_id` and set defaults. The SDKs cover data collection, not the read API (12).
- Harness Feature Management & Experimentation (83): 11 consolidated tools, and `--toolsets=fme` narrows the server to FME (25). Flag lists filter server-side by name, tag and rollout status, and the Admin API paginates with markerLimit (17 of 20). Documented status codes, 429 headers, and a harness_diagnose tool. Error bodies aren't documented per endpoint (14 of 20). harness_list and harness_get carry readOnlyHint, write tools ask for confirmation by elicitation, and harness_delete is marked destructive. No idempotency keys (15 of 20). Two required settings (API key, optional org and project, account ID read from the token); a Python Admin API wrapper and CLI, and SDKs in more than a dozen languages for evaluation (12 of 15). 25+17+14+15+12 = 83.

### Security & auth: Harness Feature Management & Experimentation +17

- Countly (62): Auth tokens are scoped to apps and endpoint patterns, expire after a set lifetime, are revocable and carry create, read, update and delete permissions since 25.03.54. The dashboard `api_key` has its user's full access (26 of 30). Less 10 because `api_key` and `auth_token` are documented as query-string parameters, with the `countly-token` header as the recommended alternative (16). MCP read-only mode and per-category CRUD levels, bounded by the token's permissions. No confirmation step for destructive tools (16). Tools return event names, user properties and crash data written by end users, and no prompt-injection guidance was found (3). A system logs API records dashboard actions (12, kept; the 8 October count of log types came from the help centre's JSON API and is struck). No security.txt. SECURITY.md gives security@count.ly as the reporting route with no acknowledgement time and a bounty whose rewards are at Countly's discretion, which this run doesn't count as a bounty. Countly publishes advisories for countly-server on GitHub, and the site claims ISO 27001, 27017, 27018 and SOC 2 Type II without public reports. 15, as daytona has for a disclosure route and certifications with a discretionary reward.
- Harness Feature Management & Experimentation (79): Admin API keys with combinable roles that can't be widened after creation, Harness PATs and SATs for central management, and Harness Platform OAuth on the hosted MCP endpoint. Credentials travel in headers (28 of 30). HARNESS_READ_ONLY removes every write, writes go through elicitation with a risk threshold for auto-approval, and FME change requests add approval for flag changes (19 of 20). No prompt-injection guidance; the tools return flag names, descriptions and rule data written by people (3 of 15). FME admin, environment and object audit logs, plus structured MCP audit events to file, webhook or OpenTelemetry sinks (13 of 15). SOC 1 and SOC 2 Type II, ISO 27001, 27017 and 27018, FedRAMP 20x, a security@harness.io disclosure route with a 24-hour response, a private bug bounty on request and annual external penetration tests. No security.txt (16 of 20). 28+19+3+13+16 = 79.

### Transparency & trust: Countly +11

- Countly (72): Countly Lite is AGPL-3.0 with added branding terms under Section 7, the MCP server is MIT, and the Enterprise plugins that hold funnels, retention, cohorts and Drill are closed (20 of 30). Flex now has its own terms, privacy policy, DPA and sub-processor list, all dated 20 October 2026. The Flex privacy policy excludes data customers process and gives no retention periods, and the DPA covers that data (20). The MCP changelog deprecated auth tokens in URL parameters on 23 April 2026 (1.3.0), and the server warns on their use, with no notice period (15, the settled value for dated notices without a period). The Flex sub-processor list names locations, with 30 days' notice of changes in the DPA, and Flex customers choose a Google Cloud region. A Connected Services and Telemetry notice (20 October 2026) says server reports to stats.count.ly are on by default and which settings turn them off, and MCP usage reporting is disclosed with an opt-out (18).
- Harness Feature Management & Experimentation (61): A closed service under Harness's subscription terms; the MCP server is MIT (18 of 30). The privacy statement covers website and sales data and defers customer data to the customer agreement and a published DPA; retention periods for customer data weren't found (18 of 30). A versioning policy for SDKs with support windows, and Split-era API endpoints marked deprecated with migration guides (14 of 20). Sub-processors are on the trust centre, which we didn't read, and the security page doesn't name data locations (10 of 20). 18+18+14+10 = 60.

## Score by category

| Category | Weight | Countly | Harness Feature Management & Experimentation | Edge |
| --- | --- | --- | --- | --- |
| Reliability | 16% (20 this run) | 23 | 80 | Harness Feature Management & Experimentation +57 |
| Performance | 10%, pending | pending | pending | not scored in this run |
| Schema & documentation | 13% (16.2 this run) | 75 | 71 | Countly +4 |
| Agent ergonomics | 13% (16.2 this run) | 66 | 83 | Harness Feature Management & Experimentation +17 |
| Security & auth | 14% (17.5 this run) | 62 | 79 | Harness Feature Management & Experimentation +17 |
| Payments & pricing | 10% (12.5 this run) | 17 | 15 | Countly +2 |
| Task success | 10%, pending | pending | pending | not scored in this run |
| Maintenance & community | 7% (8.8 this run) | 80 | 81 | Harness Feature Management & Experimentation +1 |
| Transparency & trust | 7% (8.8 this run) | 72 | 61 | Countly +11 |
| Negative events | ≤15 | -15 | 0 | |
| **Total** | | **38.8 · E** | **69.2 · B** | |

## Facts side by side

| Fact | Countly | Harness Feature Management & Experimentation |
| --- | --- | --- |
| Kind | HTTP API | HTTP API |
| Vendor | Countly Ltd | Harness Inc. |
| Hosted endpoint | `https://mcp.count.ly/mcp` | `https://mcp.harness.io/mcp` |
| Transports | HTTP, stdio, Streamable HTTP | HTTP, stdio, Streamable HTTP |
| Auth | API key | OAuth or key |
| Pricing | Paid | Freemium |
| x402 | no | no |
| Licence | Countly Lite is AGPL-3.0 with added branding terms under Section 7. Enterprise plugins are proprietary. Flex is a hosted service under Countly's terms. The MCP server is MIT | none |
| Tools exposed | 209 | 11 |
| Read-only variant documented | yes | no |
| llms.txt | yes | yes |
| Last release | 2026-10-27 | 2026-11-04 |
| Terms last updated | not read yet | no date given |
| Privacy policy last updated | not read yet | no date given |
| Customer content may train models | not read yet | not found in the text |
| Terms restrict automated access | not read yet | not found in the text |
| Terms restrict benchmarking | not read yet | yes |
| Terms or service can change without notice | not read yet | not found in the text |
| Arbitration or class-action waiver | not read yet | not found in the text |
| Popularity | 28k npm/wk | 8.8k npm/wk |

## How the scores have moved

### Countly

- -16.1 on 4 November 2026 (C to E): security, correction
- +0.1 on 27 October 2026: checklist

### Harness Feature Management & Experimentation

No score change since it was listed.

## Verdicts

**Countly.** Countly suits an agent working against a server its owner already runs. The official MCP server filters its 209 tools by edition, plugin and permission and has per-category read-only settings. Five countly-server CVEs were published since September, all fixed, and no status page, OpenAPI document or published API rate limit was found.

**Harness Feature Management & Experimentation.** Harness FME, formerly Split, gives agents 11 consolidated MCP tools with an FME-only toolset, a read-only switch and confirmation on writes, over an Admin API with documented limits and role-scoped keys. FME prices are not published, the free plan's limits are unstated, and the hosted MCP endpoint must be enabled by Harness Support.

## Before you call either

### Countly

1. Create a dedicated token in the Token Manager, limited to the apps and endpoints the task needs, and send it in the `countly-token` header, not the URL
2. Set `COUNTLY_TOOLS_ALL=R` unless the task needs writes. The token's own permissions still apply and the stricter of the two wins
3. Call `apps_list` first. Most tools need `app_id` or an exact `app_name`
4. Check `get_version` for the detected edition before planning funnel, retention or cohort queries, which need Enterprise plugins
5. Set `ENABLE_ANALYTICS=false` to stop MCP usage reports to stats.count.ly, and ask the server owner to check the tracking settings for server reports

### Harness Feature Management & Experimentation

1. Start the MCP server with `--toolsets=fme` so only FME resource types load
2. Call `harness_describe(resource_type='fme_feature_flag')` before a create or update to see the fields
3. Filter fme_feature_flag lists by name, tags or rollout status rather than listing every flag
4. Set HARNESS_READ_ONLY for analysis tasks; leave confirmation on for kill, restore and delete
5. On 429, wait at least the larger X-RateLimit-Reset-Seconds value before retrying

## Questions

### Which is better for AI agents, Countly or Harness Feature Management & Experimentation?

Harness Feature Management & Experimentation scores 69.2 (B) on agent readiness against Countly's 38.8 (E), and leads in 4 of 7 scored categories. Countly leads on transparency & trust.

### Do Countly and Harness Feature Management & Experimentation need an API key?

Countly needs an API key. Harness Feature Management & Experimentation takes an API key or an OAuth sign-in.

### Can an agent call Countly and Harness Feature Management & Experimentation without installing anything?

Yes. Countly has a hosted endpoint at https://mcp.count.ly/mcp and Harness Feature Management & Experimentation at https://mcp.harness.io/mcp.

### Are Countly and Harness Feature Management & Experimentation open source?

Countly is open source (Countly Lite is AGPL-3.0 with added branding terms under Section 7. Enterprise plugins are proprietary. Flex is a hosted service under Countly's terms. The MCP server is MIT). No open-source release is listed for Harness Feature Management & Experimentation.


## For agents

- This comparison as JSON: https://www.anchorterminal.com/compare/countly-vs-harness-fme.json, and with the fewest tokens: https://www.anchorterminal.com/compare/countly-vs-harness-fme.min.md
- Over MCP at https://www.anchorterminal.com/mcp (no key): `compare_tools {"a": "countly", "b": "harness-fme"}`. From a terminal: `anchor compare countly harness-fme`
- Each listing in full: https://www.anchorterminal.com/api/v1/tools/countly.json and https://www.anchorterminal.com/api/v1/tools/harness-fme.json

## Other comparisons with Countly or Harness Feature Management & Experimentation

- [Amplitude vs Countly](https://www.anchorterminal.com/compare/amplitude-vs-countly.md)
- [Basedash MCP vs Countly](https://www.anchorterminal.com/compare/basedash-mcp-vs-countly.md)
- [Countly vs Fullstory](https://www.anchorterminal.com/compare/countly-vs-fullstory.md)
- [Countly vs GrowthBook](https://www.anchorterminal.com/compare/countly-vs-growthbook.md)
- [Countly vs Matomo](https://www.anchorterminal.com/compare/countly-vs-matomo.md)
- [Countly vs Mixpanel](https://www.anchorterminal.com/compare/countly-vs-mixpanel.md)
- [Countly vs Mostly Good Metrics](https://www.anchorterminal.com/compare/countly-vs-mostly-good-metrics.md)
- [Countly vs OpenPanel](https://www.anchorterminal.com/compare/countly-vs-openpanel.md)
- [Countly vs Optimizely Experimentation](https://www.anchorterminal.com/compare/countly-vs-optimizely.md)
- [Countly vs Pendo](https://www.anchorterminal.com/compare/countly-vs-pendo.md)
- [Countly vs PostHog](https://www.anchorterminal.com/compare/countly-vs-posthog.md)
- [Countly vs RudderStack MCP](https://www.anchorterminal.com/compare/countly-vs-rudderstack-mcp.md)
- [Countly vs Statsig](https://www.anchorterminal.com/compare/countly-vs-statsig.md)
- [Countly vs Userpilot](https://www.anchorterminal.com/compare/countly-vs-userpilot.md)
- [Countly vs Woopra](https://www.anchorterminal.com/compare/countly-vs-woopra.md)
- [Countly vs Heap](https://www.anchorterminal.com/compare/countly-vs-heap.md)
- [Fullstory vs Harness Feature Management & Experimentation](https://www.anchorterminal.com/compare/fullstory-vs-harness-fme.md)
- [Harness Feature Management & Experimentation vs Heap](https://www.anchorterminal.com/compare/harness-fme-vs-heap.md)
- [Harness Feature Management & Experimentation vs OpenPanel](https://www.anchorterminal.com/compare/harness-fme-vs-openpanel.md)
- [Harness Feature Management & Experimentation vs RudderStack MCP](https://www.anchorterminal.com/compare/harness-fme-vs-rudderstack-mcp.md)
- [Harness Feature Management & Experimentation vs Woopra](https://www.anchorterminal.com/compare/harness-fme-vs-woopra.md)
- [Amplitude vs Harness Feature Management & Experimentation](https://www.anchorterminal.com/compare/amplitude-vs-harness-fme.md)
- [Countly vs Flagsmith](https://www.anchorterminal.com/compare/countly-vs-flagsmith.md)
- [Countly vs LaunchDarkly](https://www.anchorterminal.com/compare/countly-vs-launchdarkly.md)
- [Flagsmith vs Harness Feature Management & Experimentation](https://www.anchorterminal.com/compare/flagsmith-vs-harness-fme.md)
- [Harness Feature Management & Experimentation vs LaunchDarkly](https://www.anchorterminal.com/compare/harness-fme-vs-launchdarkly.md)
- [Harness Feature Management & Experimentation vs Mixpanel](https://www.anchorterminal.com/compare/harness-fme-vs-mixpanel.md)
- [Harness Feature Management & Experimentation vs Optimizely Experimentation](https://www.anchorterminal.com/compare/harness-fme-vs-optimizely.md)
- [Harness Feature Management & Experimentation vs PostHog](https://www.anchorterminal.com/compare/harness-fme-vs-posthog.md)
- [Harness Feature Management & Experimentation vs Statsig](https://www.anchorterminal.com/compare/harness-fme-vs-statsig.md)
- [GrowthBook vs Harness Feature Management & Experimentation](https://www.anchorterminal.com/compare/growthbook-vs-harness-fme.md)
- [Harness Feature Management & Experimentation vs Matomo](https://www.anchorterminal.com/compare/harness-fme-vs-matomo.md)
- [Harness Feature Management & Experimentation vs Mostly Good Metrics](https://www.anchorterminal.com/compare/harness-fme-vs-mostly-good-metrics.md)

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp