# Attack an agent before release and record what held

> Attack an agent before release and record what held: 0 companies fully do it, 2 with a limit, 22 carry the tag. Every listing in Agent observability & evals with the reason and a source.

- Canonical: https://www.anchorterminal.com/niches/agent-observability.redteam-before-release/
- Markdown: https://www.anchorterminal.com/niches/agent-observability.redteam-before-release/index.md (~1,350 tokens)
- Slim: https://www.anchorterminal.com/niches/agent-observability.redteam-before-release/index.min.md (~230 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/niches/agent-observability.redteam-before-release/index.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-15

No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.

| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |
| --- | --- | --- | --- | --- |
| 0 | 2 | 22 | 2 | 2 |

## What counts as done

A report of attacks tried per rule, each with the trace behind it, replayable to confirm a fix, and a clear statement of which rules were not tested.

How agents ask for it: red-team my agent; try prompt injection against the endpoint; what breaks if the tool returns garbage

## Does the job, with a limit

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

- **BotGauge (BotGauge AI Inc. (Delaware, United States))**: Each run reports Found, Held and Errors per rule with findings, Verify fix replays the exact attack, Rerun errors repeats failures, Not tested is a state, and reports export as JSON, CSV or PDF via the API. The red-team API has no published endpoints and production runs read-only attacks.
- **LangWatch (Reasoning Engine B.V. (doing business as LangWatch))**: RedTeamAgent (crescendo or GOAT) replaces the user simulator for multi-turn attacks in your own CI and scenario redteam-report lists findings, transcripts and severity. Reports are per criterion rather than per rule, and the agent under test is whatever your adapter calls.

- not this job: Agent Tree (No legal entity found. The LICENSE, plugin.json and marketplace.json name "Mike Pollow, Digitalroots"): No attack runner.
- not this job: agents-tree (No legal entity found. The LICENSE and pyproject.toml name Karel Blavka): No attack runner.
- not this job: Arize Phoenix (Arize AI, Inc.): No attack runner or red-team report in the docs index.
- not this job: Braintrust API + MCP (Braintrust Data, Inc.): No attack runner or red-team report in the docs index.
- not this job: DeepEval (Confident AI, Inc.): Red teaming lives in the separate DeepTeam package; it is not in the DeepEval docs index.
- not this job: Galileo API + MCP (Galileo Technologies, Inc.): Prompt injection is a metric, not an attack runner; no red-team report.
- not this job: Gauge CLI (Gauge): No attack rules or red-team report.
- not this job: Helicone AI Gateway + MCP (Helicone, Inc.): No attack runner or red-team report in the docs index.
- not this job: HoneyHive (HoneyHive Inc.): No attack runner or red-team report in the docs index.
- not this job: Laminar API + MCP (LMNR AI, Inc.): No attack runner or red-team report in the docs index.
- not this job: Langfuse API + MCP (Langfuse GmbH (a ClickHouse, Inc. company)): No attack runner, rule set or red-team report anywhere in the docs index.
- not this job: LangSmith API + MCP (LangChain, Inc.): No attack runner or red-team report in the observability docs index.
- not this job: MLflow Tracing (MLflow Project, a Series of LF Projects, LLC): Conversation simulation mentions stress-testing with personas, but there is no attack rule set, finding report or replay.
- not this job: Navige (Navige Ltd): No attack runner.
- not this job: Opik API + MCP (Comet ML, Inc.): Guardrails and a prompt injection model exist, but no attack runner, rules or red-team report.
- not this job: Overmind (Overmind Technology Ltd (England and Wales, company 16585996)): No attack runner or red-team report in the docs index.
- not this job: Prefactor (Prefactor Pty Ltd): No attack runner.
- not this job: Pydantic Logfire (Pydantic Services Inc.): No attack runner or red-team report in the docs index.
- not this job: Respan API + MCP (Keywords AI Inc. (d/b/a Respan)): No attack runner or red-team report in the docs index.
- not this job: W&B Weave (Weights and Biases, LLC): No attack runner or red-team report in the Weave docs.

## What stops agents finishing

- attacks can call real tools in production
- red-team API has no published endpoints
- docs, authenticated servers and APIs need the hosted tier and credits

## Around it

Before: agent-observability.instrument-app. After: agent-observability.agent-trajectory-eval.

## Where this comes from

Weight 2 of 100 in Agent observability & evals: No first-hand account of agent red-teaming found; the one BotGauge comment is about its earlier QA product and the one Gauge issue is a CLI flag. Weight set low and rests on vendor pages (BotGauge, DeepEval, Gauge).

As data: https://www.anchorterminal.com/api/v1/use-cases/agent-observability.redteam-before-release.json, or the MCP tool get_use_case with id agent-observability.redteam-before-release. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp
