Niches · Product analytics & experimentation
Read an experiment's result and say whether to ship
Two or three companies' docs show a full fit.
What counts as done
Per-variant metric values, uplift, interval or probability and sample sizes, read through the API or MCP, with the vendor's exposure and test-account filters applied.
Measured, for our own tests: the values are identical to the experiment page, read in at most 3 calls. No docs we read state these figures, so they don't count against a listing here.
How agents ask for it: did the checkout test win
, summarise our completed experiments
, is this experiment significant
, show results by device
, which experiments were inconclusive
Does the job 2
The docs show it meets the test above, and nothing we read says otherwise.
| Listing | Why |
|---|---|
| Optimizely Experimentation B Optimizely North America Inc. | Get Experiment results takes a baseline, an interval (start included, end excluded), a confidence threshold and one segmentation (device, browser, source or attribute); exp_program_reporting_execute_query returns lift, significance, intervals and visitor counts. source |
| Statsig BB Amplitude, Inc. | experiment_read returns overall and per-metric dimension results with CUPED and confidence options from Statsig's own Pulse, and cumulative_exposures and dimensional_exposures give sample sizes. source |
Does the job, with a limit 10
It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.
| Listing | Why |
|---|---|
| Amplitude BB Amplitude, Inc. | use_amp_experiments analyses variant performance and significance over MCP, but no REST results endpoint was found and the docs don't say the experiment page's exposure filters apply; this is the vendor's tool description. source |
| Countly E Countly Ltd | ab_experiments_details returns per-variant results and statistical significance, but sample sizes, intervals and exposure filters aren't described, and the ab-testing plugin is Enterprise only. source |
| Flagsmith B Bullet Train Ltd | get_experiment_results and get_experiment_exposures (beta) return statistical results and exposure counts, but return null until computed, and the docs compute results on a UI Refresh click at most every 5 minutes. source |
| GrowthBook BB GrowthBook, Inc. | GET /experiments/{id}/results and POST snapshot with a dimension (now including combos) return per-variation results, but results still list deleted metrics (open issue 4799) and appear only after a snapshot runs. source |
| Harness Feature Management & Experimentation BB Harness Inc. | fme_experiment_result lists evaluated per-metric results for each comparison treatment, filterable by metric, but only the latest calculation run, and sample sizes and intervals in the output aren't documented. source |
| LaunchDarkly BB Catamorphic, Co. (dba LaunchDarkly) | get-experiment-results summarises every metric on an iteration and get-experiment-metric-results gives detailed statistics, but the tool text doesn't promise sample sizes or the page's attribute filters, and the public REST spec has no results endpoint. source |
| Matomo B InnoCraft Limited | The AbTesting API reads experiment reports, but it is a premium plugin and its result fields (uplift, significance, sample size) couldn't be read from the JavaScript-rendered reference. source |
| Mixpanel B Mixpanel, Inc. | Get-Experiment returns configuration and results, with interpretation guidance, but the tools are an open beta and the docs don't list the result fields or confirm they match the experiment page. source |
| Mostly Good Metrics E Mostly Good Metrics | get_experiment and the experiments feature describe enrolment from first exposure and conversions joined on user ID, but the result fields (uplift, interval, sample size) aren't listed in the docs read. source |
| PostHog B PostHog Inc. (formerly Hiberly Inc.) | experiment-results-get returns metric and exposure results with per-metric metadata, so the agent needn't write its own query (acc-17), but dimension breakdowns aren't documented and MCP-created metrics broke the experiment page (acc-07, open). source |
Doesn't do this job 7
Listings in the category that do something else, with the reason.
| Listing | Why |
|---|---|
| Fullstory B Fullstory, Inc. | No experiments product in the MCP tools or server API. source |
| Heap D Content Square, Inc. | No experiments product in the developer reference. source |
| OpenPanel B OpenPanel AB | No experiments product in the MCP tools or API. source |
| Pendo E Pendo.io, Inc. | listGuideExperiments returns guide A/B experiments' status and configuration only; no tool returns per-variant results, uplift or intervals. source |
| RudderStack MCP C RudderStack, Inc. | No experiments. source |
| Userpilot B Userpilot, Inc. | The MCP permissions and use cases cover no experiment results. source |
| Woopra D Woopra, Inc. | No experiments product. source |
What stops agents finishing
- results null until the vendor computes them
- dimension breakdowns only computable from the UI
- agent recomputes results with its own query and gets different numbers
- deleted metrics still listed in results
Around it
Before: product-analytics.flag-experiment-change. After: product-analytics.flag-experiment-change, product-analytics.stale-flag-cleanup.
Where this comes from
Weight 9 of 100 in Product analytics & experimentation: 5 of 55 account links; the buyer check names it, but first-hand agent readouts are few and two of the five are vendor-written
The same as data: /api/v1/use-cases/product-analytics.experiment-readout.json, or the MCP tool get_use_case with id product-analytics.experiment-readout. Other jobs in this category: /api/v1/niches?category=product-analytics.
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.