{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/product-analytics.experiment-readout/",
    "json": "https://www.anchorterminal.com/niches/product-analytics.experiment-readout/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/product-analytics.experiment-readout/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/product-analytics.experiment-readout/index.min.md"
  },
  "markdown": "Two or three companies' docs show a full fit.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 2 | 9 | 11 | 9 | 5 |\n\n## What counts as done\n\nPer-variant metric values, uplift, interval or probability and sample sizes, read through the API or MCP, with the vendor's exposure and test-account filters applied.\n\nMeasured, for our own tests: the values are identical to the experiment page, read in at most 3 calls.\n\nHow agents ask for it: did the checkout test win; summarise our completed experiments; is this experiment significant; show results by device; which experiments were inconclusive\n\n## Does the job\n\nThe docs show it meets the test above, and nothing we read says otherwise.\n\n- **Optimizely Experimentation (Optimizely North America Inc.)**: Get Experiment results takes a baseline, an interval (start included, end excluded), a confidence threshold and one segmentation (device, browser, source or attribute); exp_program_reporting_execute_query returns lift, significance, intervals and visitor counts.\n- **Statsig (Amplitude, Inc.)**: experiment_read returns overall and per-metric dimension results with CUPED and confidence options from Statsig's own Pulse, and cumulative_exposures and dimensional_exposures give sample sizes.\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **Amplitude (Amplitude, Inc.)**: use_amp_experiments analyses variant performance and significance over MCP, but no REST results endpoint was found and the docs don't say the experiment page's exposure filters apply; this is the vendor's tool description.\n- **Countly (Countly Ltd)**: ab_experiments_details returns per-variant results and statistical significance, but sample sizes, intervals and exposure filters aren't described, and the ab-testing plugin is Enterprise only.\n- **Flagsmith (Bullet Train Ltd)**: get_experiment_results and get_experiment_exposures (beta) return statistical results and exposure counts, but return null until computed, and the docs compute results on a UI Refresh click at most every 5 minutes.\n- **GrowthBook (GrowthBook, Inc.)**: GET /experiments/{id}/results and POST snapshot with a dimension (now including combos) return per-variation results, but results still list deleted metrics (open issue 4799) and appear only after a snapshot runs.\n- **Harness Feature Management \u0026 Experimentation (Harness Inc.)**: fme_experiment_result lists evaluated per-metric results for each comparison treatment, filterable by metric, but only the latest calculation run, and sample sizes and intervals in the output aren't documented.\n- **LaunchDarkly (Catamorphic, Co. (dba LaunchDarkly))**: get-experiment-results summarises every metric on an iteration and get-experiment-metric-results gives detailed statistics, but the tool text doesn't promise sample sizes or the page's attribute filters, and the public REST spec has no results endpoint.\n- **Matomo (InnoCraft Limited)**: The AbTesting API reads experiment reports, but it is a premium plugin and its result fields (uplift, significance, sample size) couldn't be read from the JavaScript-rendered reference.\n- **Mixpanel (Mixpanel, Inc.)**: Get-Experiment returns configuration and results, with interpretation guidance, but the tools are an open beta and the docs don't list the result fields or confirm they match the experiment page.\n- **Mostly Good Metrics (Mostly Good Metrics)**: get_experiment and the experiments feature describe enrolment from first exposure and conversions joined on user ID, but the result fields (uplift, interval, sample size) aren't listed in the docs read.\n- **PostHog (PostHog Inc. (formerly Hiberly Inc.))**: experiment-results-get returns metric and exposure results with per-metric metadata, so the agent needn't write its own query (acc-17), but dimension breakdowns aren't documented and MCP-created metrics broke the experiment page (acc-07, open).\n\n- not this job: Fullstory (Fullstory, Inc.): No experiments product in the MCP tools or server API.\n- not this job: Heap (Content Square, Inc.): No experiments product in the developer reference.\n- not this job: OpenPanel (OpenPanel AB): No experiments product in the MCP tools or API.\n- not this job: Pendo (Pendo.io, Inc.): listGuideExperiments returns guide A/B experiments' status and configuration only; no tool returns per-variant results, uplift or intervals.\n- not this job: RudderStack MCP (RudderStack, Inc.): No experiments.\n- not this job: Userpilot (Userpilot, Inc.): The MCP permissions and use cases cover no experiment results.\n- not this job: Woopra (Woopra, Inc.): No experiments product.\n\n## What stops agents finishing\n\n- results null until the vendor computes them\n- dimension breakdowns only computable from the UI\n- agent recomputes results with its own query and gets different numbers\n- deleted metrics still listed in results\n\n## Around it\n\nBefore: product-analytics.flag-experiment-change. After: product-analytics.flag-experiment-change, product-analytics.stale-flag-cleanup.\n\n## Where this comes from\n\nWeight 9 of 100 in Product analytics \u0026 experimentation: 5 of 55 account links; the buyer check names it, but first-hand agent readouts are few and two of the five are vendor-written\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/product-analytics.experiment-readout.json, or the MCP tool get_use_case with id product-analytics.experiment-readout. Draft, read 2026-11-15 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Read an experiment's result and say whether to ship",
        "url": ""
      }
    ],
    "description": "Read an experiment's result and say whether to ship: 2 companies fully do it, 9 with a limit, 11 carry the tag. Every listing in Product analytics \u0026 experimentation with the reason and a source.",
    "facts": null,
    "h1": "Read an experiment's result and say whether to ship",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/product-analytics.experiment-readout/",
    "published": "",
    "section": "agents",
    "title": "Read an experiment's result and say whether to ship | Anchor Terminal",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/product-analytics.experiment-readout/"
  },
  "tokens": {
    "markdown": 1600,
    "slim": 330
  },
  "version": 1
}
