Mostly Good Metrics vs Statsig

Mostly Good Metrics 29

Graded on the hosted service with the hosted lines. No status page was found (0) and no incident history could be read (5). Ingestion limits are documented per plan (1,000 to 100,000 events a minute and 10,000 to 1,000,000 an hour), but no limit for MCP or query calls was found (12 of 15). 429 with Retry-After in the header and body, and the SDKs queue events and back off. No idempotency or safe-retry guidance for writes (12 of 15). No SLA (0). The press kit calls the product a public beta (0). 0+5+12+12+0+0 = 29.

Statsig 79

Graded on the hosted service with the hosted lines. Statuspage at status.statsig.com with 14 components, among them Console API, Config Delivery API and Log Event API (20). Three incidents between 3 August and 1 November 2026, all Warehouse Native delays (export delays from 6 August resolved 8 August, delayed exposure results on 26 August and its follow-up resolved 31 August), marked minor or none per the 8 October check, and nothing posted since 31 August (20 of 30). The 8 July 2026 major, when launched gates returned false through the V2 Config Delivery API for about 4 hours 44 minutes, is outside the window. The docs give about 1,800 mutation attempts per 15 minutes per project, while the OpenAPI description gave about 100 per 10 seconds and 900 per 15 minutes per the 8 October check, and no number was found for reads (12 of 15). The docs ask for exponential backoff with jitter on 429. No Retry-After header and no idempotency keys were found (10 of 15). The enterprise terms give 99.95 per cent monthly availability with credits, for customers with premium support, measured on the Console user interface (7 of 10). The Console API is versioned and the MCP V3 docs carry no beta label (10).

Mostly Good Metrics 33

OAuth 2.0 with PKCE, dynamic client registration and a revocation endpoint for MCP and the CLI; project ingest keys in a Bearer header that can be pinned to origins or app IDs. The only tool scope advertised is mcp:tools, so a connection gets every tool the user can use (24 of 30). Restricted keys and a confirmation step for unrestricted keys in the CLI; no read-only MCP mode or approval for deletes was found (6 of 20). Event properties come from apps and end users and reach the agent, and no prompt-injection guidance was found (3 of 15). No audit log was found (0). No security.txt, disclosure policy or certification was found (0). 24+6+3+0+0 = 33.

Statsig 74

OAuth for the MCP server with PKCE, dynamic client registration and a device code grant, issuing personal Console API keys that carry the user's role. Console keys can be created read-only or read and write, and rotated or deactivated through the API. The OAuth metadata lists no scopes, and a project Console key reaches every Console API endpoint (25 of 30). MCP scope is set per project and per role as no access, read only or read and write. Updates and destructive operations need a confirmation, and project review policies still apply (19 of 20). No guidance on prompt injection was found, though tools return names, descriptions and log data that people wrote. The docs tell users to review each change before the agent confirms (4 of 15). Audit logs are kept indefinitely with an API, and personal keys attribute changes to a user (14 of 15). SOC 2 Type II and a bug bounty programme are stated on the security page, with no public programme link and no security.txt (12 of 20).

Mostly Good Metrics 49

No OpenAPI file was found. The docs say every MCP tool describes its own parameters, and the query definition object is documented in full; we didn't open an MCP session to read the schemas (12 of 25). No llms.txt (404) and no Markdown pages served for agents (0). Each tool area is documented with examples, and the docs say when to use query_metrics instead of repeated execute_query calls (10 of 20). The query definition lists metrics, dimensions, date ranges and filter operators (9 of 15). The ingestion API documents its error body and status codes, and query_metrics reports per-item errors (11 of 15). /v1 and /api/v2 paths and dated SDK tags, but no product changelog (7 of 15). 12+0+10+9+11+7 = 49.

Statsig 82

Public OpenAPI 3.0 spec for Console API version 20240601 with 209 paths and 325 operations, per the 8 October check, not re-read on 1 November, and typed MCP tools per the tool reference (25). llms.txt, llms-full.txt, a Markdown copy of each page and a public docs MCP server (10). Every operation has a summary but only 29 of 325 have a description, per the 8 October check. The MCP tool reference states each tool's purpose and when to ask the user first (11 of 20). 1,164 enums across 188 schemas, with required fields marked, per the 8 October check (12 of 15). Examples on most responses. The spec documents 401 on 156 operations, 404 on 126 and 400 on 106, and no 429, per the 8 October check (11 of 15). A dated API version sent in the STATSIG-API-VERSION header with a promise not to break a version, and a dated product updates page. No changelog for the API alone was found (13 of 15).

Mostly Good Metrics 45

The service is closed; the SDKs and CLI are MIT. Terms of December 2025 with no legal entity named (18 of 30). The privacy policy and terms agree on raw-event retention for the subscription's life and deletion within 14 days of account deletion; the about page says data is not used to train models. No DPA was found (18 of 30). The legacy X-MGM-Key header is still supported, and no deprecation policy was found (3 of 20). The terms name eight sub-processors with their purpose, without data locations (12 of 20). 18+18+3+12 = 51.

Statsig 76

Closed hosted service under a published self-service agreement and enterprise terms. The SDKs are open source under ISC (17 of 30). The terms, DPA and AI governance page mostly agree. The DPA names Amplitude, Inc. as processor and keeps data as long as necessary, the pricing page gives 1 year of analytics retention on the free plan, and the docs say customer data is not used to build AI models without written consent. The privacy notice is Amplitude's and doesn't name Statsig, and the self-service terms name Amplitude, Inc. under an effective date of 4 August 2025, before Statsig joined Amplitude on 5 May 2026 (18 of 30). Each Console API version is promised not to break, a deprecation notices page gives dates, and MCP V1 stays supported. No notice period is stated (12 of 20). Amplitude's sub-processor list, updated 13 May 2026, gives locations and marks the providers used for Statsig-branded services. EU hosting is an Enterprise option (17 of 20).

Mostly Good Metrics 49

53 tools, more than 30 (5). We added 3 because Goals and Alerts tools load only where enabled and query_metrics batches up to 10 queries (8 of 25). Date ranges, grouping and filters on queries; no documented page or limit parameters for lists (12 of 20). JSON errors with a message, and per-item status in batch results (13 of 20). Annotations weren't readable without a session, the tool list includes deletes and key revocation, and no idempotency keys were found (4 of 20). Five client SDKs and a CLI with --json output (12 of 15). 8+12+13+4+12 = 49.

Statsig 71

The MCP V3 server shows 18 default tools, which is 15 on the checklist. Add back 8 for the discovery layer that reaches other operations on demand, query_fields on experiment reads and read-only access (23 of 25). limit and page on 49 list operations with pagination metadata per the 8 October check, a cursor on the Logs Explorer query, and filters by tag, status, creator and date (17 of 20). Errors are a status and a message with no machine code, and MCP results set isError (11 of 20). No idempotency keys were found. Updates and api_destructive calls need a confirmation object, and operations are split into read, write and destructive lanes. We could not read the live tool annotations (10 of 20). The SDKs cover flag evaluation and event logging in many languages, not the Console API, and the siggy CLI on npm is version 0.0.4 from September 2024 (10 of 15).

Mostly Good Metrics 30

No x402, MPP or L402 (0). Plan prices with event allowances are on the home page; no per-event price (10). Free plan with no card, stated on the home and about pages (20). Signup and OAuth approval happen in a browser (0). 0+10+20+0 = 30.

Statsig 40

No x402, MPP or L402 (0). Prices are public without a login. Pro is $150 a month with 5 million events, then $0.05 per 1,000 events (20). The Developer plan is free with no card and 2 million events a month (20). A person signs up in a browser, then creates a key or approves OAuth (0).