{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/agent-observability.redteam-before-release/",
    "json": "https://www.anchorterminal.com/niches/agent-observability.redteam-before-release/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/agent-observability.redteam-before-release/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/agent-observability.redteam-before-release/index.min.md"
  },
  "markdown": "No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 0 | 2 | 22 | 2 | 2 |\n\n## What counts as done\n\nA report of attacks tried per rule, each with the trace behind it, replayable to confirm a fix, and a clear statement of which rules were not tested.\n\nHow agents ask for it: red-team my agent; try prompt injection against the endpoint; what breaks if the tool returns garbage\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **BotGauge (BotGauge AI Inc. (Delaware, United States))**: Each run reports Found, Held and Errors per rule with findings, Verify fix replays the exact attack, Rerun errors repeats failures, Not tested is a state, and reports export as JSON, CSV or PDF via the API. The red-team API has no published endpoints and production runs read-only attacks.\n- **LangWatch (Reasoning Engine B.V. (doing business as LangWatch))**: RedTeamAgent (crescendo or GOAT) replaces the user simulator for multi-turn attacks in your own CI and scenario redteam-report lists findings, transcripts and severity. Reports are per criterion rather than per rule, and the agent under test is whatever your adapter calls.\n\n- not this job: Agent Tree (No legal entity found. The LICENSE, plugin.json and marketplace.json name \"Mike Pollow, Digitalroots\"): No attack runner.\n- not this job: agents-tree (No legal entity found. The LICENSE and pyproject.toml name Karel Blavka): No attack runner.\n- not this job: Arize Phoenix (Arize AI, Inc.): No attack runner or red-team report in the docs index.\n- not this job: Braintrust API + MCP (Braintrust Data, Inc.): No attack runner or red-team report in the docs index.\n- not this job: DeepEval (Confident AI, Inc.): Red teaming lives in the separate DeepTeam package; it is not in the DeepEval docs index.\n- not this job: Galileo API + MCP (Galileo Technologies, Inc.): Prompt injection is a metric, not an attack runner; no red-team report.\n- not this job: Gauge CLI (Gauge): No attack rules or red-team report.\n- not this job: Helicone AI Gateway + MCP (Helicone, Inc.): No attack runner or red-team report in the docs index.\n- not this job: HoneyHive (HoneyHive Inc.): No attack runner or red-team report in the docs index.\n- not this job: Laminar API + MCP (LMNR AI, Inc.): No attack runner or red-team report in the docs index.\n- not this job: Langfuse API + MCP (Langfuse GmbH (a ClickHouse, Inc. company)): No attack runner, rule set or red-team report anywhere in the docs index.\n- not this job: LangSmith API + MCP (LangChain, Inc.): No attack runner or red-team report in the observability docs index.\n- not this job: MLflow Tracing (MLflow Project, a Series of LF Projects, LLC): Conversation simulation mentions stress-testing with personas, but there is no attack rule set, finding report or replay.\n- not this job: Navige (Navige Ltd): No attack runner.\n- not this job: Opik API + MCP (Comet ML, Inc.): Guardrails and a prompt injection model exist, but no attack runner, rules or red-team report.\n- not this job: Overmind (Overmind Technology Ltd (England and Wales, company 16585996)): No attack runner or red-team report in the docs index.\n- not this job: Prefactor (Prefactor Pty Ltd): No attack runner.\n- not this job: Pydantic Logfire (Pydantic Services Inc.): No attack runner or red-team report in the docs index.\n- not this job: Respan API + MCP (Keywords AI Inc. (d/b/a Respan)): No attack runner or red-team report in the docs index.\n- not this job: W\u0026B Weave (Weights and Biases, LLC): No attack runner or red-team report in the Weave docs.\n\n## What stops agents finishing\n\n- attacks can call real tools in production\n- red-team API has no published endpoints\n- docs, authenticated servers and APIs need the hosted tier and credits\n\n## Around it\n\nBefore: agent-observability.instrument-app. After: agent-observability.agent-trajectory-eval.\n\n## Where this comes from\n\nWeight 2 of 100 in Agent observability \u0026 evals: No first-hand account of agent red-teaming found; the one BotGauge comment is about its earlier QA product and the one Gauge issue is a CLI flag. Weight set low and rests on vendor pages (BotGauge, DeepEval, Gauge).\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/agent-observability.redteam-before-release.json, or the MCP tool get_use_case with id agent-observability.redteam-before-release. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Attack an agent before release and record what held",
        "url": ""
      }
    ],
    "description": "Attack an agent before release and record what held: 0 companies fully do it, 2 with a limit, 22 carry the tag. Every listing in Agent observability \u0026 evals with the reason and a source.",
    "facts": null,
    "h1": "Attack an agent before release and record what held",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/agent-observability.redteam-before-release/",
    "published": "",
    "section": "agents",
    "title": "Attack an agent before release and record what held | Anchor Terminal",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/agent-observability.redteam-before-release/"
  },
  "tokens": {
    "markdown": 1350,
    "slim": 230
  },
  "version": 1
}
