Niches · Agent observability & evals

Find why one agent run failed

One company's docs show a full fit.

1companies, full fit
16companies, fit with a limit
22companies carry the tag
19share of the category's work
21first-hand accounts

What counts as done

The run's tree with each model call, tool call and retry as its own span, the failed span found by a status or error filter in one query, with the tool's input and output and the model's input and output readable.

Measured, for our own tests: all of this is readable within 2 minutes of the run ending. No docs we read state these figures, so they don't count against a listing here.

How agents ask for it: why did this run fail, which tool call errored, show me the trace for this request, what did the agent send to the model before it picked that tool

Does the job 1

The docs show it meets the test above, and nothing we read says otherwise.

ListingWhy
Pydantic Logfire B
Pydantic Services Inc.
One trace per agent run with model, tool and database spans, every Pydantic AI tool call and retry as a child span, SQL over the same table and MCP queries; a production Slack agent team used SQL to find runs that called specific tools. source

Does the job, with a limit 16

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
Arize Phoenix BB
Arize AI, Inc.
Filter expressions such as span.status_code == 'ERROR' work in the UI, GraphQL and the traces REST filter; px trace get returns the tree as JSON. Spans with NUL characters were acknowledged then dropped (issue 17380) and 44 GB made browsing unusable (8420). source
BotGauge E
BotGauge AI Inc. (Delaware, United States)
Logs show sessions, traces and spans with cost, duration and scores, plus messages and timeline tabs; the SDK logs 401s and a probe reports rejected spans. No error-status filter is documented and rule filters are API-only. source
Braintrust API + MCP B
Braintrust Data, Inc.
SQL WHERE filters on logs in UI, CLI, API and MCP; span errors show with their input and output; tool spans are a span type. Queries silently drop data outside the plan's retention window (14 days on Starter) and AI SDK calls logged twice (issue 992). source
Galileo API + MCP D
Galileo Technologies, Inc.
Sessions, traces and spans with tool spans and a Tool Error metric; traces appear in a Log stream after the quickstart. No status filter or API read path was in the pages read, and accounts are split between Galileo and Splunk hosts. source
Helicone AI Gateway + MCP E
Helicone, Inc.
Each gateway request is logged with status, cost and body; Helicone-Session-Id and -Path headers build a tree and the MCP query_requests tool filters with bodies. Tool calls outside the model request appear only if you set session paths yourself. source
HoneyHive D
HoneyHive Inc.
Sessions and events form a tree with model and tool spans, the Traces page filters events, and POST events/search filters by field with a 7,500 row cap. No error-status filter is named and auth failures return 404, which can look like a missing trace. source
Laminar API + MCP C
LMNR AI, Inc.
Read-only ClickHouse SQL over spans with status and cost columns and get_trace_context for an LLM-sized summary with errors; the transcript view shows tool inputs and outputs. Browser session replay failed to record (issue 634) and rendered pages wrongly (1039). source
Langfuse API + MCP BB
Langfuse GmbH (a ClickHouse, Inc. company)
Observation tree with level:ERROR and type:TOOL filters in the search bar; tool and model I/O recorded. But interrupts show as ERROR (issue 10962) and one user saw spans take minutes to appear. source
LangSmith API + MCP BB
LangChain, Inc.
status:error in the search bar, has_error and tree_filter in runs.query, run types for tool calls. Native tool calling leaves no prompt to show (forum 2319), traces stopped arriving with no errors after a langgraph-api upgrade (forum 1293) and one user found it adding latency. source
LangWatch B
Reasoning Engine B.V. (doing business as LangWatch)
The MCP digest shows the prompt the model received, the tool call, its arguments and the error; the trace filter language finds erroring traces. Custom trace_ids are no longer accepted, and no first-hand account exists. source
MLflow Tracing C
MLflow Project, a Series of LF Projects, LLC
search_traces filters trace.status = 'ERROR', spans carry OK or ERROR status and log levels, and the MCP search tool takes the same filter. Streamed OpenAI completions lost their tool calls (issue 26812) and Pydantic AI spans vanished after an update (22208). source
Opik API + MCP BB
Comet ML, Inc.
Trace plus span tree via the MCP read tool, list filtered by status, Ollie for debugging. A Google ADK agent over SSE streaming left LLM spans stuck at started with no output (issue 5524) and traces landed under the wrong thread (4794). source
Overmind C
Overmind Technology Ltd (England and Wales, company 16585996)
query_traces filters errors, sessions, models and time, query_failures returns scored failures, and the trace resource exposes a span tree with status, tool, input and output. No UI latency is stated and the SDK is alpha. source
Prefactor B
Prefactor Pty Ltd
Spans carry Pending, Active, Complete, Failed or Cancelled status with payload and result JSON, a conversation view and redacted sensitive values; nested children are not shown inline. No status filter parameter was found in the full docs. source
Respan API + MCP BB
Keywords AI Inc. (d/b/a Respan)
Logs filter by Status, Provider and Model with error bars per row, the signals panel drills into failed spans and read-only ClickHouse SQL has an error-rate template. Message search needs the Team plan and Free keeps logs seven days. source
W&B Weave B
Weights and Biases, LLC
Calls can be filtered, sorted and exported from the UI, SDK or Service API and the Agents view renders GenAI sessions, turns and tool calls. No status filter is named on the pages read, and Bedrock Agent Runtime had no patch (issue 4658). source

Doesn't do this job 5

Listings in the category that do something else, with the reason.

ListingWhy
Agent Tree C
No legal entity found. The LICENSE, plugin.json and marketplace.json name "Mike Pollow, Digitalroots"
A live side pane for the current Claude Code session; nothing is stored after the session and there is no filter or query. source
agents-tree B
No legal entity found. The LICENSE and pyproject.toml name Karel Blavka
A live terminal tree read from transcripts on disk; it keeps nothing and has no error filter. source
DeepEval B
Confident AI, Inc.
DeepEval is a local evaluation framework; @observe traces exist only during a test run and the hosted store is Confident AI, a separate product. source
Gauge CLI B
Gauge
Gauge records sessions of agents it launches against your product; it is not a trace store for your own agent runs. source
Navige E
Navige Ltd
Navige records usage, cost and metadata, not prompts, completions or a span tree; it says so on the proxy page. source

What stops agents finishing

  • tool calls missing from streamed completions
  • framework interrupt shown as ERROR
  • ingestion delay of minutes while the agent waits
  • trace only readable in a browser UI

Around it

Before: inference.tool-loop, agent-harnesses.embed-in-program. After: agent-observability.offline-eval-regression, agent-observability.online-llm-judge.

Where this comes from

Weight 19 of 100 in Agent observability & evals: 21 of 86 accounts (counted in more than one use case where they fit): the Voker, agent-triage, Argus and skele threads, Langfuse issues on ERROR status and missing breakdowns, MLflow and Opik issues on tool calls missing from spans, the LangSmith forum thread on the tool-choice prompt, Laminar browser session replay issues

The same as data: /api/v1/use-cases/agent-observability.debug-failed-run.json, or the MCP tool get_use_case with id agent-observability.debug-failed-run. Other jobs in this category: /api/v1/niches?category=agent-observability.

Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.