BotGauge vs Galileo API + MCP

BotGauge 53

No OpenAPI file for the eval or red-team endpoints was found in the docs or llms-full.txt. Ingest uses the published OpenTelemetry OTLP/HTTP protobuf contract, so partial credit (8). llms.txt, llms-full.txt and Markdown for every page on Mintlify (10). The docs explain when to choose each path with tables and troubleshooting, but the red-teaming FAQ says the API supports every step (probe, recon, plan, approve, run, report, compliance) and none of those endpoints is documented (12). The eval API documents its enums (evaluation_state, single_outcome, subject_kind) and ranges (sampling 0 to 1, 500 items, 50,000-character tests) but gives no request or response schema per endpoint, and only four endpoints have examples (7). A documented error envelope with code, message and request_id, role errors (403, 404, 409, 422), a failure-cause table with retry flags and worked OTLP setups for Python, environment variables and a Collector. No list of error codes (11). /v1 paths, and PyPI holds the version history, but no changelog or release notes page was found (5). 8+10+12+7+11+5 = 53.

Galileo API + MCP 79

OpenAPI 3.1 at api.galileo.ai/public/v2/openapi.json. The copy pinned in the Python SDK on 13 July has 184 paths and 244 operations (25). llms.txt on docs.galileo.ai and on the Splunk docs site, with Markdown pages (10). 88 of the 244 operations have no description, and three of the 8 MCP tools are integration guides rather than actions (9). Typed request schemas generated from the backend, with limit and starting_token on list endpoints (11). The spec declares only 200 and 422 responses, but the Splunk docs carry an error catalogue of 111 entries, each with a code, HTTP status, cause, fix and a retriable flag. Cookbooks give examples (11). Versioned /v2 API and dated release notes, split between docs.galileo.ai (to 7 August) and the Splunk docs (15 September) (13).

BotGauge 56

Project API keys reach one project, are shown once, stored in a form that can't be read back per the DPA, show their last-used time and revoke immediately, with documented rotation steps. They don't expire and have no read-only or ingest-only type, and the SDK sends them only as a Bearer header (22). Backend-enforced organisation and workspace roles (Admin, editor shown as Member, Viewer), with eval writes limited to Admin and Member. Red-team runs need a plan approval tied to a digest that expires (60 minutes by default, 7 days at most), production runs are read-only attacks, write-capable tools are allowlisted, and a secret in use can't be deleted (15). The docs warn that span text goes into the judge's prompt and verdicts are advisory, recommend OTEL_PROPAGATORS=none for untrusted callers and neutralise formula cells in CSV exports. No guidance for agents reading traces or transcripts back (8). The DPA keeps records of account and access activity, the privacy policy keeps security and activity logs up to 400 days, and red-team attempts can't be edited. No customer-facing audit log is documented (5). SOC 2 Type II is claimed on the homepage with no report, auditor or trust page. No security.txt, disclosure policy or bug bounty found. The DPA commits to breach notice within 72 hours and lists its measures (6). 22+15+8+5+6 = 56.

Galileo API + MCP 33

API key in a header (Splunk-AO-API-Key in the current spec, Galileo-API-Key in older docs), a JWT from /login/api_key that expires after 24 hours, or HTTP Basic with a username and password, which the docs list as an option. We found no key scopes, expiry or rotation docs (18). Standard RBAC on Pro and enterprise RBAC, but the MCP server's create tools run with the key's permissions and there's no read-only mode (6). Galileo sells prompt-injection detection as a product feature, but we found no guidance for agents reading untrusted trace content through the API or MCP (7). No audit log found (0). No security.txt, no trust page (trust.galileo.ai doesn't resolve, galileo.ai/security is a 404), no SECURITY.md in either SDK repository and no SOC 2, ISO or HIPAA claim on the homepage, pricing page or privacy policy (2).

BotGauge 55

API responses can be narrowed by subject_ids and subject_kind, a results query takes up to 500 items, and judges read at most 20,000 characters, but no field selection or summaries are documented (12). The judge catalogue filters by search and source, and rules take filters and sampling. No pagination is documented on any list endpoint (10). Errors carry a code, a message and a request id. Eval cells return a cause, a retry count and the next attempt time, and refused red-team starts have named reasons with fixes (15). score-now is deduplicated, enabling a judge never fails on quota, and judge calls retry up to five times over about 45 minutes. No idempotency keys on other writes (8). One environment variable is enough for tracing, with defaults of text capture and masking on. Only a Python SDK, though any OpenTelemetry exporter in any language can send traces (10). 12+10+15+8+10 = 55.

Galileo API + MCP 73

The MCP server has 8 tools, compact, but only Get Signals reads production data. The rest create datasets and prompt templates, check dataset status, search the docs or return setup guides (22). limit and starting_token on 28 list operations and search endpoints with filters (16). The error catalogue gives an agent a code, a fix and whether to retry (16). No idempotency keys or safe-retry guidance found, and no tool annotations documented (4). Python and TypeScript SDKs, few required parameters (15).

BotGauge 10

No x402 or other machine payment (0). Paid plans for the agent product have no published prices, and botgauge.com/pricing prices the separate QA service (0). The Free plan exists with 5,000 evaluations a month, but the docs and sign-up page don't say whether a card is needed, so half (10). A person signs up in a browser at cloud.botgauge.com and creates keys in the console (0). 0+0+10+0 = 10.

Galileo API + MCP 20

No x402 or other machine payment (0). Plan prices are public, Pro at $100 a month billed yearly for 50,000 traces, but overage only "scales based on number of traces" (10). The Free plan exists with 5,000 traces a month. The pricing page doesn't say whether a card is needed, so half (10). A person signs up in a browser at app.galileo.ai/sign-up (0).

BotGauge 64

Closed platform under terms dated 28 October 2026 that name BotGauge AI Inc. The SDK is source-available under FSL-1.1-ALv2, not an OSI licence, with each release becoming Apache-2.0 after two years, and its repository isn't public (16). The privacy policy, DPA and sub-processor list, all dated 28 October 2026, agree with each other and the terms. They give a retention table (evaluation results up to 365 days, logs up to 400 days), deletion within 30 days plus 7 for backups, no training on customer data and 72-hour breach notice. Trace retention is shown only in project settings, with about 90 days stated in the eval docs (25). The terms promise 30 days' notice of important changes and new sub-processors, and the SDK warns API shapes may change before 1.0. No deprecation policy for the API (5). Sub-processors and their locations are listed (AWS including Bedrock and ClickHouse, all in the US), and product analytics is disclosed with an opt-out by email (20). 16+25+5+20 = 66.

Galileo API + MCP 55

Closed platform with terms from Galileo Technologies, Inc. Both SDKs are Apache-2.0 (15). The privacy policy was last updated on 1 November 2024, says data goes to servers in the US, allows de-identified customer data to be used for research and development and gives no retention periods. We found no DPA or subprocessor list (8). The rename to Splunk Agent Observability is dated 7 August 2026 in the release notes, but there's no timeline for api.galileo.ai, docs.galileo.ai or the SDKs (8). US data location stated, no subprocessors listed (5).

BotGauge 67

botgauge, botgauge-tracing and botgauge-cli 0.1.3 on 2026-10-31 (30). Four releases of each since 20 October 2026, 0.1.0, 0.1.1, 0.1.2 and 0.1.3 (20). Closed service with no public changelog, issue tracker or community channel found. Support by email at support@botgauge.com (4). One official SDK, Python, current (10). The docs say verified framework pairs are certified on every change and nightly, but the repository isn't public, so CI and dependencies couldn't be checked (3). 30+20+4+10+3 = 67. The product is 12 days old by its first package release.

Galileo API + MCP 74

TypeScript SDK 2.3.2 on 2026-10-01 (30). TypeScript 2.3.0, 2.3.1 and 2.3.2 in September and Python 2.5.0, 2.5.1 and 2.6.0 in July (20). Dated release notes on two sites and Slack support on Pro, but the Python SDK has had no commit since 30 July and the product is mid-migration (8). Official Python and TypeScript SDKs, neither of which references the new Splunk SaaS hosts (10). CI tests, codecov and Dependabot. The Python CHANGELOG.md stops at v0.10.0 from May 2025 while releases have reached 2.6.0, and the TypeScript SDK shipped a breaking rename (logstream to logStreamName) in patch 2.1.2 (6).