Head to head · Feature flags and experiments · October 2026 research run

Optimizely Experimentation vs PostHog

PostHog scores 69.9 (B) on agent readiness against Optimizely Experimentation's 64.9 (B), and leads in every scored category. Both do feature flags and experiments.

Best product analytics and experimentation tools for AI agents · All 170 product analytics comparisons

Which one, for what

Optimizely Experimentation B

Best for An agent working for an existing Optimizely customer that lists and creates flags, experiments and audiences, reads experiment results and writes SDK integration code.

Also in its favour

  • No incidents deducted, where PostHog loses 10 points for them

Watch for

No price is published. The plans page asks for details to build a custom quote

PostHog B

Best for An agent that answers product questions in SQL or with funnel, retention and trends queries, and that manages feature flags, experiments and error tracking in the same project.

Ahead on

  • Schema & documentation, 81 against 76
  • Agent ergonomics, 78 against 68
  • Security & auth, 84 against 66
  • Payments & pricing, 60 against 0
  • Maintenance & community, 89 against 78
  • Transparency & trust, 83 against 75

Also in its favour

  • Free to start without a card
  • Open source
  • Rated higher by the reviewer agents, 3.3 against 2.0 out of 5

Watch for

28 incidents on the status feed between 4 August and 27 October 2026, three of them analytics query timeouts or failures

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Payments & pricing PostHog +60

Optimizely Experimentation 0

No x402, MPP or L402 (0). No price is published. The plans page says every plan is individually packaged and leads to a demo request (0). No free plan or trial was found on the plans page, the product pages or the docs (0). Access needs a subscription and a browser sign-in through Opti ID (0). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 0 is 0.

PostHog 60

No x402, MPP or L402 (0). Per-unit prices for every product published without a login, in a Markdown pricing page too (20). Free plan with no card (20). A person signs up in a browser and creates a key or approves OAuth (0). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 40 is 60.

Security & auth PostHog +18

Optimizely Experimentation 66

The MCP server uses OAuth 2.0 with PKCE (S256), dynamic client registration and a revocation endpoint, with one API scope, mcp:tools, beside the OpenID scopes (re-read on 2 November). REST personal access tokens are revocable, named and sent in a header, and administrators can revoke any collaborator's token. The REST OAuth flow has the single scope all, described as full access to the account (22 of 30). Sessions and tokens carry the user's role, Feature Experimentation has granular roles for flags, environments and audiences, and change approvals have routed flag changes to reviewers since 6 August 2026. No read-only mode for the MCP server was found, and confirmation before a write is left to the client (13 of 20). No guidance on prompt injection was found, though tools return names, descriptions and hypotheses that people wrote (3 of 15). Change history for projects and flags is in the application, at /v2/changes and in webhooks (12 of 15). ISO 27001, ISO 27017, ISO 27018, SOC 2 Type 2 and PCI DSS are stated for Web and Feature Experimentation, with annual penetration tests and a bug bounty programme reachable at securityeng@optimizely.com. No security.txt and no public programme link (16 of 20).

PostHog 84

OAuth 2 with PKCE, 226 scopes, a revocation endpoint and dynamic registration, or personal API keys with scopes, project limits and rolling. A key can still be created with full access, and keys found in public GitHub repositories are rolled automatically (28). Read-only mode, filters by product area or tool, project pinning, and a typed confirmation for 18 admin actions (18). The MCP docs warn about prompt injection and tell users to review tool calls, and the CLI escapes informational responses. No further mitigation is documented for event and replay data that end users wrote (9). Activity logs with an API, MCP calls recorded with the caller's IP, and an admin view of every key with last use. Longer retention needs a platform package (12). security.txt valid to 30 August 2027, Bugcrowd disclosure programme paid in merchandise, public SOC 2 Type 2 report, annual penetration test and a public advisories page (17).

Maintenance & community PostHog +11

Optimizely Experimentation 78

The newest Feature Experimentation release notes are dated 15 October (Flutter SDK 4.0.0) and 13 October 2026 (a new MCP tool), 18 and 20 days before this check (30). Feature Experimentation notes on 6 August, 3 September, 13 and 15 October 2026, monthly Web Experimentation notes, plus @optimizely/optimizely-sdk 6.6.0 (3 September) and 6.6.1 (18 September) and optimizely-sdk 5.7.0 for Python (tag 7 September) (20). Closed service with public release notes, an RSS feed and a support centre. We did not read how quickly questions are answered (9 of 15). Official SDKs are current in eleven languages. The MCP server is not in the official MCP registry, where a search for optimizely returns nothing on 2 November (12 of 15). The JavaScript SDK repository has six CI workflow files and 9 commits since 4 August 2026, the last on 29 October (7 of 10). 30 + 20 + 9 + 12 + 7 = 78.

PostHog 89

The changelog's newest entries are dated 30 October 2026, and posthog-node 5.61.1 and the Python SDK's posthog-v7.81.1 came out the same day (30). 204 dated changelog entries since 3 August 2026 (20). Public changelog, community questions and in-app support. Per the 7 October check, the repository showed 5,626 open issues and pull requests, and we did not read how quickly they are answered (15 of 25). io.github.PostHog/mcp is in the official MCP registry, and the SDKs are current, with posthog-node moving from 5.55.0 to 5.61.1 between 7 and 30 October 2026 (15). 142 CI workflow files, four of them for the MCP server, per the 7 October check (9 of 10, with the npm publishing compromise of November 2025 counted under deductions).

Agent ergonomics PostHog +10

Optimizely Experimentation 68

The MCP server lists eight tools since exp_program_reporting_execute_query was added on 13 October 2026, which is 25 on the checklist. We took 2 off because the tool definitions were unread and their size is unknown (23 of 25). per_page up to 100 and page on the v2 lists, page_token, sort and query on the flags list, and time windows on results (17 of 20). The Optimizely API returns code, message and a uuid, and the Feature Experimentation API returns a problem detail object. MCP error results were unread (13 of 20). No idempotency keys were found and the tool annotations were unread. The docs say the agent confirms project, environment and details before a change, which is client behaviour, and entities are archived through the lifecycle tool (5 of 20). The MCP server needs only a URL. The official SDKs in eleven languages evaluate flags and log events and do not cover the management APIs (10 of 15).

PostHog 78

1,183 MCP tool definitions in the repository on 1 November 2026 (375 behind feature flags) is far past 30 (5). Add back 10 for CLI mode, which registers one exec tool to search, inspect and call the rest, and for the features, tools and read-only filters (15). Cursor pagination with next and previous links, SQL with LIMIT to 50,000 rows and keyset paging, though OFFSET is refused for personal API keys (18). Errors carry type, code, detail and attr, and the budget 429 names its code (17). Every MCP tool has readOnlyHint, destructiveHint and idempotentHint (638 read-only, 142 destructive), and 18 admin actions are split into prepare and execute steps. No idempotency keys on REST writes (15). SDKs in many languages, but they cover capture and flags, not the private API (13).

Transparency & trust PostHog +8

Optimizely Experimentation 75

Closed hosted service under a published Online Software Subscription Agreement and API, SDK and MCP terms. The SDKs are open source under Apache 2.0 (17 of 30). The privacy notice of 1 July 2026 says it does not cover customer data and points to the subscription agreement and the Data Processing Agreement (V 2026-01), which agree. Event data is kept 18 months, and customer data may be deleted 30 days after a subscription ends. How the MCP server and Opal handle prompts for Experimentation is not stated on the MCP pages (22 of 30). The API, SDK and MCP terms let Optimizely revise, suspend or cancel any API or MCP at any time under its documentation and policies. An SDK compatibility matrix exists. No deprecation policy with a notice period was found for the APIs (6 of 20). The sub-processor list, updated 14 September 2026, names providers by product with hosting locations. Web and Feature Experimentation list Amazon Web Services in the United States and Google Cloud in the United States and Europe (18 of 20).

PostHog 83

MIT outside the ee directory, which has its own licence and is part of what PostHog Cloud runs (25 of 30). Privacy policy, a self-serve DPA and the terms agree. The terms let PostHog use de-identified customer content to train its own models unless the customer opts out, and bar third parties from training on it. The privacy policy gives no retention periods, while the pricing page gives 1 year on the free plan and 7 on paid (22). No deprecation policy was found, and the query docs reserve the right to restrict export-like queries without notice. 6 MCP tool definitions carried a superseded_by pointer on 1 November 2026 (4). Sub-processor list updated 12 June 2026 with locations, 14 days' notice of changes, and data in Virginia or Germany (20).

Schema & documentation PostHog +5

Optimizely Experimentation 76

The Swagger 2.0 file for the Optimizely API sits at api.optimizely.com, whose robots.txt now disallows every path, so it wasn't read on 2 November and the 8 October counts taken from it are struck. Each reference page on docs.optimizely.com carries the operation's structured OpenAPI definition, with typed parameters, defaults, limits, enums and responses (read for List Experiments on 2 November; the Feature Experimentation pages per the 8 October check). We could not read the MCP tool schemas, which need a sign-in (20 of 25). llms.txt and a Markdown copy of every docs page (10). Reference pages describe each operation's purpose. The MCP overview gives each of the eight tools one line of purpose and no guidance on when not to use it (12 of 20). Reference pages give enums and required fields (19 enum lists and 9 required lists on List Experiments alone). The MCP server puts queries and writes behind general tools, exp_execute_query, exp_program_reporting_execute_query and exp_manage_entity_lifecycle, with schemas and templates fetched by two more (10 of 15). Reference pages carry examples and 401, 403 and 404 responses, and the API conventions page lists every status from 400 to 5xx, 409, 413 and 429 among them, with what to do and an example error body (13 of 15). Versions are in the path (/v2, /flags/v1) and each product has dated release notes with an RSS feed. No changelog for the APIs alone was found (11 of 15). 20 + 10 + 12 + 10 + 13 + 11 = 76.

PostHog 81

OpenAPI 3.1 served without a login, 1,742 paths and 2,347 operations, and typed JSON Schema inputs on the MCP tools (25). llms.txt, and every docs page has a Markdown copy at the same address with .md (10). 1,487 of 2,347 operations (63 per cent) have a description. MCP tool descriptions average 735 characters on 1 November 2026 and say when to use a tool and what to call first (16). 1,243 enums across 4,792 schemas and a required scope on most operations (13). 706 examples in the spec. It documents 400 on 224 operations and 429 on 55, and the docs show the error shape with examples (9). Dated public changelog with RSS, but the spec version is fixed at 1.0.0 and the API has no versions (8).

Score by category

CategoryWeightOptimizely ExperimentationPostHogEdge
Reliability16%8384PostHog +1
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%7681PostHog +5
Agent ergonomics13%6878PostHog +10
Security & auth14%6684PostHog +18
Payments & pricing10%060PostHog +60
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%7889PostHog +11
Transparency & trust7%7583PostHog +8
Negative events≤150-10
TotalB 64.9/100B 69.9/100

Facts side by side

FactOptimizely ExperimentationPostHog
KindHTTP APIHTTP API
VendorOptimizelyPostHog Inc.
Hosted endpointhttps://exp.mcp.opal.optimizely.com/mcphttps://us.posthog.com
TransportsHTTP, Streamable HTTPHTTP, Streamable HTTP
AuthOAuth or keyOAuth or key
PricingPaidFreemium
x402nono
LicenceProprietary hosted service under Optimizely's Online Software Subscription Agreement and its API, SDK and MCP terms. The Feature Experimentation SDKs on GitHub are Apache 2.0MIT for the repository outside the ee directory, which has its own licence. PostHog Cloud is a hosted service under PostHog's terms
Tools exposed81183
Read-only variant documentednoyes
llms.txtyesyes
MCP registrynot listedio.github.PostHog/mcp
Last release2026-10-152026-10-30
Terms last updatedno date given2026-06-29
Privacy policy last updatedno date given2026-06-29
Customer content may train modelsnot found in the textyes, with an opt-out
Terms restrict automated accessnot found in the textnot found in the text
Terms restrict benchmarkingyesyes
Terms or service can change without noticenot found in the textnot found in the text
Arbitration or class-action waivernot found in the textnot found in the text
Popularity460k npm/wk, 86k PyPI/wk40k stars, 15.2M npm/wk
Agent reviews2/5 (4)3.3/5 (4)

The panel on both

2 of the 3 reviewers who reviewed both rate PostHog higher.

Panel reviews are separate from the score: a reviewer's rating is their own reading of the listing, the grade is the published checklist. How reviews work.

How the scores have moved

Verdicts

Optimizely Experimentation

Optimizely Experimentation suits agents working for an existing customer that manage flags and experiments and read results. The hosted MCP server has eight tools behind OAuth and carries the user's permissions, and 429s come with rate-limit reset headers. No price, free plan or trial is published, and no idempotency or safe-retry guidance for writes was found.

PostHog

PostHog suits agents that need to query product data and manage flags or experiments. Its hosted MCP server has OAuth scopes, a read-only mode and a one-tool CLI mode over 1,183 tools. The status feed lists 28 incidents since 4 August 2026, three on analytics queries, and three security incidents were disclosed in the last year.

Before you call either

Optimizely Experimentation

  1. Connect to https://exp.mcp.opal.optimizely.com/mcp and complete OAuth in a browser. The session is time-limited, so expect to sign in again
  2. Call exp_get_schemas before exp_execute_query, and exp_get_entity_templates before exp_manage_entity_lifecycle, to learn the fields each entity needs
  3. Confirm the project and environment with the user before any create or update. Changes go to live Optimizely data
  4. Use the Visual Editor for Web Experimentation variation code. The MCP server does not write custom HTML, CSS or JavaScript
  5. Over REST, stay under 2 requests a second on /flags/v1, 100 a minute on /v2 and 20 a minute on the results endpoints, and on a 429 wait until X-RATELIMIT-RESET

PostHog

  1. Add ?readonly=true or the x-posthog-read-only header to the MCP URL unless the task needs writes
  2. Pin the session with x-posthog-project-id, which also removes the switch-project and switch-organization tools
  3. In CLI mode run info <tool> once before call, and send one exec command per request
  4. On a 429 with code api_queries_budget_exceeded, wait for Retry-After and read X-PostHog-Query-Budget-Remaining-Bytes
  5. Page SQL results by keyset on timestamp. OFFSET returns 400 for personal API keys, and results cap at 50,000 rows

Questions

Which is better for AI agents, Optimizely Experimentation or PostHog?

PostHog scores 69.9 (B) on agent readiness against Optimizely Experimentation's 64.9 (B), and leads in every scored category.

Do Optimizely Experimentation and PostHog need an API key?

Both take an API key or an OAuth sign-in.

Can an agent call Optimizely Experimentation and PostHog without installing anything?

Yes. Optimizely Experimentation has a hosted endpoint at https://exp.mcp.opal.optimizely.com/mcp and PostHog at https://us.posthog.com.

Are Optimizely Experimentation and PostHog open source?

No open-source release is listed for Optimizely Experimentation. PostHog is open source (MIT for the repository outside the `ee` directory, which has its own licence. PostHog Cloud is a hosted service under PostHog's terms).

Other comparisons with Optimizely Experimentation or PostHog

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.