# BotGauge vs DeepEval (slim)

> DeepEval scores 64.9 (B) to BotGauge's 40.7 (E) for agent tracing. Prices, MCP, x402, uptime and agent notes side by side.

- Full: https://www.anchorterminal.com/compare/botgauge-vs-deepeval.md (~5,550 tokens) · this version ~3,430 tokens · JSON https://www.anchorterminal.com/compare/botgauge-vs-deepeval.json · canonical https://www.anchorterminal.com/compare/botgauge-vs-deepeval
- Index: https://www.anchorterminal.com/llms.txt · API: https://www.anchorterminal.com/api/v1/index.json · MCP: https://www.anchorterminal.com/mcp · Updated: 2026-11-01
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)

DeepEval scores 64.9 (B) on agent readiness against BotGauge's 40.7 (E), and leads in 6 of 7 scored categories. BotGauge leads on security & auth. Both do agent tracing.

- BotGauge: E 40.7, rank #1132 of 1260 · https://www.anchorterminal.com/tools/botgauge.min.md
- DeepEval: B 64.9, rank #367 of 1260 · https://www.anchorterminal.com/tools/deepeval.min.md
- BotGauge, good for A team that wants adversarial testing of a tool-calling agent before release, with traces and AI-judge evals in the same place, and that is outside the countries the terms exclude. Ahead on Security & auth, 56 against 47. Also A hosted endpoint, with nothing to install.
- DeepEval, good for Teams that want evaluations in pytest or a CLI on their own machines, with agent, RAG, multi-turn and MCP metrics. Ahead on Reliability, 59 against 18; Schema & documentation, 78 against 53; Agent ergonomics, 72 against 55; Payments & pricing, 60 against 10; Maintenance & community, 83 against 67. Also Free to start without a card; Open source; No incidents deducted, where BotGauge loses 3 points for them.

| Category | BotGauge | DeepEval |
| --- | --- | --- |
| Reliability (16%) | 18 | 59 |
| Performance (10%) | pending | pending |
| Schema & documentation (13%) | 53 | 78 |
| Agent ergonomics (13%) | 55 | 72 |
| Security & auth (14%) | 56 | 47 |
| Payments & pricing (10%) | 10 | 60 |
| Task success (10%) | pending | pending |
| Maintenance & community (7%) | 67 | 83 |
| Transparency & trust (7%) | 64 | 66 |

| Fact (where they differ) | BotGauge | DeepEval |
| --- | --- | --- |
| Kind | HTTP API | SDK + MCP |
| Vendor | BotGauge AI Inc. | Confident AI, Inc. |
| Hosted endpoint | `https://api.botgauge.com` | no (local only) |
| Transports | HTTP |  |
| Auth | API key | OAuth or key |
| Price for agent tracing | not published | $1 per GB per month |
| Licence | Proprietary hosted service under BotGauge's terms (last updated 28 October 2026). The Python SDK on PyPI is source-available under FSL-1.1-ALv2, which allows any use except a competing commercial product and turns each release into Apache-2.0 two years after publication. Its repository isn't public | Apache 2.0 for the Python and TypeScript packages and the agent skills. Confident AI, the hosted platform, is a proprietary service under its own terms |
| Read-only variant documented | yes | no |
| Last release | 2026-10-31 | 2026-10-02 |
| Terms last updated | 2026-10-28 | no document linked |
| Privacy policy last updated | 2026-10-28 | no document linked |
| Customer content may train models | not found in the text | not read yet |
| Terms restrict automated access | not found in the text | not read yet |
| Terms restrict benchmarking | not found in the text | not read yet |
| Terms or service can change without notice | not found in the text | not read yet |
| Arbitration or class-action waiver | not found in the text | not read yet |
| Popularity | none | 19k stars, 35k npm/wk, 736k PyPI/wk |
| Agent reviews | none | 2.8/5 (4) |

Why the edge:
- Payments & pricing, DeepEval +50: BotGauge 10 (No x402 or other machine payment (0). Paid plans for the agent product have no published prices, and botgauge.com/pricing prices the separate QA service (0). The Free plan exists with 5,000 evaluations a month, but the docs and sign-up page don't say whether a card is needed, so half (10). A person signs up in a browser at cloud.botgauge.com and creates keys in the console (0). 0+0+10+0 = 10.); DeepEval 60 (No x402, MPP or L402 (0). DeepEval is free, and the paid option is the hosted Confident AI platform, whose prices are public without a login. Starter is $200 a month, Team $2,000 a month, and trace spans beyond the plan are $1 per GB-month (20). The framework is free and Confident AI's Free plan needs no card (20). `pip install -U deepeval` and a local run need no account, so an agent can start without a person signing up. Confident AI itself needs a browser sign-up, and its auth.md says there is no self-serve registration endpoint for API keys. Scoring this line on the local framework is a judgement call (20).)
- Reliability, DeepEval +41: BotGauge 18 (No status page. status.botgauge.com doesn't resolve and neither the site nor the docs link one (0). With no history to read, 5. No request rate limits for ingest or the eval API. The docs give a monthly quota of 5,000 evaluations on Free, size caps (500 items a query, 2,000-character filters) and 30 red-team connection tests an hour per project (5). No 429 or Retry-After guidance. `score-now` returns the same request when asked twice, and the SDK exports on a background thread that never crashes the app (3). The service is supplied "as is" and "as available" under the terms, and no SLA was found for this product. The 99.9 per cent SLA on the /ai-agents page belongs to the QA service (0). The service isn't labelled beta, but the SDK is 0.x, marked "4 - Beta" on PyPI, and the docs say API shapes may change before 1.0 (5). 0+5+5+3+0+5 = 18.); DeepEval 59 (Scored on the local-software lines, since DeepEval runs where the owner installs it. Official `deepeval` packages on PyPI and npm, with Python 3.9 or later stated (20). Public CI in GitHub Actions with 17 workflow files and 332 Python test files, but the 25 most recent Py Core Tests runs had failed when read on 9 October 2026, two of them pushes to main on 7 October, and open issue #3372 of 26 September reports the suite red on main (8). 322 open issues and 387 open pull requests on a repository with 18.7k stars. The newest page of issues is mostly scoring and provider defects filed since 23 September, each with four comments or fewer (12). The changelog has Breaking Change sections, but breaking changes ship in minor and patch releases. 4.2.0 reversed the score direction of four metrics and 3.9.8 and 3.9.9 removed options (7). Python is at 4.2.8. The TypeScript package is at 0.9.21 (12).)
- Schema & documentation, DeepEval +25: BotGauge 53 (No OpenAPI file for the eval or red-team endpoints was found in the docs or llms-full.txt. Ingest uses the published OpenTelemetry OTLP/HTTP protobuf contract, so partial credit (8). llms.txt, llms-full.txt and Markdown for every page on Mintlify (10). The docs explain when to choose each path with tables and troubleshooting, but the red-teaming FAQ says the API supports every step (probe, recon, plan, approve, run, report, compliance) and none of those endpoints is documented (12). The eval API documents its enums (`evaluation_state`, `single_outcome`, `subject_kind`) and ranges (sampling 0 to 1, 500 items, 50,000-character tests) but gives no request or response schema per endpoint, and only four endpoints have examples (7). A documented error envelope with `code`, `message` and `request_id`, role errors (403, 404, 409, 422), a failure-cause table with retry flags and worked OTLP setups for Python, environment variables and a Collector. No list of error codes (11). `/v1` paths, and PyPI holds the version history, but no changelog or release notes page was found (5). 8+10+12+7+11+5 = 53.); DeepEval 78 (Read on the framework lines. The Python package ships `py.typed` and pydantic models, the TypeScript package ships type declarations, and the CLI is built on typer. A library has no OpenAPI file to publish (18). llms.txt on deepeval.com indexes the docs, and pages carry a Copy Markdown control (10). Each metric has its own page with required parameters and usage, and three agent skills in the repository tell a coding agent how to add evaluations and tracing (16). Typed test case and metric classes, with enums for evaluation parameters and CLI choices (12). Many runnable examples, a troubleshooting page and an environment variable reference. Error types are not listed in one place (11). Tagged releases for both packages and a public changelog by year, but the 2026 changelog stops at 4.2.0 of 25 August and has no entry for 4.2.1 to 4.2.8 (11).)
- Agent ergonomics, DeepEval +17: BotGauge 55 (API responses can be narrowed by `subject_ids` and `subject_kind`, a results query takes up to 500 items, and judges read at most 20,000 characters, but no field selection or summaries are documented (12). The judge catalogue filters by `search` and `source`, and rules take filters and sampling. No pagination is documented on any list endpoint (10). Errors carry a code, a message and a request id. Eval cells return a cause, a retry count and the next attempt time, and refused red-team starts have named reasons with fixes (15). `score-now` is deduplicated, enabling a judge never fails on quota, and judge calls retry up to five times over about 45 minutes. No idempotency keys on other writes (8). One environment variable is enough for tracing, with defaults of text capture and masking on. Only a Python SDK, though any OpenTelemetry exporter in any language can send traces (10). 12+10+15+8+10 = 55.); DeepEval 72 (Read on the framework lines. One install and a pytest-style test file give a first evaluation, and results land in JSON files a coding agent can read. `deepeval inspect` is a terminal interface for a person, and no MCP server ships with the framework (18). `deepeval test run` takes flags for display, pytest markers, parallel processes and a results folder, and the SQLite store can be queried across runs (14). `--ignore-errors`, `--skip-on-missing-params`, a `diagnose` command, debug controls and a troubleshooting page (14). `--use-cache` reuses metric results and retry settings cover provider calls with backoff. Scores from LLM judges can differ between runs (12). Python and TypeScript packages with few required parameters. The default judge needs `OPENAI_API_KEY` (14).)
- Maintenance & community, DeepEval +16: BotGauge 67 (botgauge, botgauge-tracing and botgauge-cli 0.1.3 on 2026-10-31 (30). Four releases of each since 20 October 2026, 0.1.0, 0.1.1, 0.1.2 and 0.1.3 (20). Closed service with no public changelog, issue tracker or community channel found. Support by email at support@botgauge.com (4). One official SDK, Python, current (10). The docs say verified framework pairs are certified on every change and nightly, but the repository isn't public, so CI and dependencies couldn't be checked (3). 30+20+4+10+3 = 67. The product is 12 days old by its first package release.); DeepEval 83 (Python 4.2.8 was tagged on 2 October 2026, seven days before the check (30). 14 Python releases tagged since 9 August, from 4.1.6 to 4.2.8 (20). 124 commits in the 30 days to 9 October, most by three maintainers, and a Discord community. 322 issues and 387 pull requests are open and reply times were not read (14). Current official packages in Python and TypeScript, the latter at 0.9.21 on 30 September (15). A `poetry.lock` file and CI on pull requests. No Dependabot configuration was found and the core test workflow is failing on main (4).)
- Security & auth, BotGauge +9: BotGauge 56 (Project API keys reach one project, are shown once, stored in a form that can't be read back per the DPA, show their last-used time and revoke immediately, with documented rotation steps. They don't expire and have no read-only or ingest-only type, and the SDK sends them only as a Bearer header (22). Backend-enforced organisation and workspace roles (Admin, editor shown as Member, Viewer), with eval writes limited to Admin and Member. Red-team runs need a plan approval tied to a digest that expires (60 minutes by default, 7 days at most), production runs are read-only attacks, write-capable tools are allowlisted, and a secret in use can't be deleted (15). The docs warn that span text goes into the judge's prompt and verdicts are advisory, recommend `OTEL_PROPAGATORS=none` for untrusted callers and neutralise formula cells in CSV exports. No guidance for agents reading traces or transcripts back (8). The DPA keeps records of account and access activity, the privacy policy keeps security and activity logs up to 400 days, and red-team attempts can't be edited. No customer-facing audit log is documented (5). SOC 2 Type II is claimed on the homepage with no report, auditor or trust page. No security.txt, disclosure policy or bug bounty found. The DPA commits to breach notice within 72 hours and lists its measures (6). 22+15+8+5+6 = 56.); DeepEval 47 (Local runs need no credential. Provider keys and `CONFIDENT_API_KEY` are read from the environment or a dotenv file, and storing keys in the legacy `.deepeval/.deepeval` file is deprecated in the source. Confident AI project keys take a name and an expiry and can be deactivated, rotated with a grace period or deleted. No per-permission scopes were found (20). The framework has no read-only mode. Once logged in, test runs and traces upload to Confident AI (8). Metrics pass application output to an LLM judge, and no guidance on untrusted content reaching the judge was found in the pages read (5). Each run is recorded in local JSON or SQLite. Confident AI documents audit log exports, which we did not read (7). GitHub shows no security policy and no published advisories, and security.txt answered 404 on both domains. No bug bounty was found. The Confident AI site states SOC 2 Type II and HIPAA, and its trust centre is drawn by script and was not read (7).)