Niches · Agent observability & evals
Trace Claude Code, Codex and other coding-agent sessions and their cost
Four or more companies' docs show a full fit.
What counts as done
Every session appears as a trace with its turns, tool calls and sub-agents, token usage per session, and a session id the team can cite, with no proxy in the agent's network path and no silent loss of sessions.
How agents ask for it: what did Claude Code do in that session
, which sub-agent spent the tokens
, trace my Codex runs for the team
, connect Claude Code on a subscription plan to a trace store
Does the job 5
The docs show it meets the test above, and nothing we read says otherwise.
| Listing | Why |
|---|---|
| Arize Phoenix BB Arize AI, Inc. | The coding-harness-tracing plugin hooks Claude Code lifecycle events and Codex notify events: one trace per turn with LLM, nested tool and subagent spans, token cost, grouped by session, no proxy. The Agent SDK needs ClaudeSDKClient, not query(). source |
| Braintrust API + MCP B Braintrust Data, Inc. | bt trace enable claude or codex installs plugins that record session, turn, model, tool and subagent spans with no proxy; Claude Code's native OTel export is the alternative. The coding agent insights dashboard breaks cost down by user, key, model and branch. source |
| Laminar API + MCP C LMNR AI, Inc. | lmnr-cli installs Claude Code and Codex plugins through the harnesses' own marketplaces; each turn is a trace with LLM calls, tool executions and subagents grouped as a session, with a project key in a config file and no proxy. The Agent SDK is wrapped in code. source |
| LangSmith API + MCP BB LangChain, Inc. | Plugins for Claude Code and Codex send each turn as a trace grouped into a thread, with managed settings for fleets; no proxy. Tracing stops and reminds the user in chat when a sign-in expires, so gaps are not silent. source |
| Respan API + MCP BB Keywords AI Inc. (d/b/a Respan) | The session-traces plugin runs inside Claude Code with no hook script or proxy and sends each turn as a trace with a span per model request and tool call, subagents nested, and token, cache and timing data; a Codex guide covers tool calls and tokens. source |
Does the job, with a limit 10
It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.
| Listing | Why |
|---|---|
| Agent Tree C No legal entity found. The LICENSE, plugin.json and marketplace.json name "Mike Pollow, Digitalroots" | Shows a Claude Code session's sub-agents as a live tree with reports, files touched, usage and a log, through hooks and no proxy. Claude Code only, and no trace or session id survives the session. source |
| agents-tree B No legal entity found. The LICENSE and pyproject.toml name Karel Blavka | Reads Claude Code, Codex, Grok Build and Antigravity CLI transcripts on disk and shows sessions, sub-agents, models, tool calls and tokens with no proxy or hook. Nothing is stored, no cost, and a harness update can blank columns. source |
| Helicone AI Gateway + MCP E Helicone, Inc. | Claude Code logs through ANTHROPIC_BASE_URL pointed at anthropic.helicone.ai with the key in the URL, a method marked maintained but not developed; Codex uses the gateway base URL. Both put a proxy in the agent's path and record requests, not hooks or subagents. source |
| Langfuse API + MCP BB Langfuse GmbH (a ClickHouse, Inc. company) | Claude Code is traced by a Stop-hook script that reaches into SDK 4.x internals and must be pinned; Codex has a plugin with a Stop hook that fails open unless LANGFUSE_CODEX_FAIL_ON_ERROR is set. Sessions, tools and cached tokens are grouped per session_id. source |
| MLflow Tracing C MLflow Project, a Series of LF Projects, LLC | mlflow autolog claude writes hooks that record tools, nested subagents, skills and per-session token cost; Codex and Gemini CLI use a notify hook, no proxy. The Stop hook once dropped traces silently (issue 25517) and only Claude Code was supported at first (20307). source |
| Navige E Navige Ltd | Claude Code calls go through ANTHROPIC_BASE_URL with x-navige-key, measured per session and subagent and separating plan usage from paid extra usage, with trimming and caching off. A proxy sits in the path and tool calls are checked, not recorded with content. source |
| Opik API + MCP BB Comet ML, Inc. | The Claude Code plugin captures every turn, tool call and nested subagent with cost, no proxy; the Codex path is Codex's own OTel exporter, which carries operational spans only, with no prompt text, tokens or cost. source |
| Overmind C Overmind Technology Ltd (England and Wales, company 16585996) | Claude Code spans arrive over OTel with OTEL_LOG_USER_PROMPTS=1, but tool input and responses need detailed tracing that only allowlisted organisations get in interactive sessions; no hook or plugin and no Codex path. source |
| Pydantic Logfire B Pydantic Services Inc. | Codex has two paths: native OTel logs and metrics to /v1/logs, and the Logfire Exporter plugin for turns with tool calls; both need a write token. No Claude Code session integration was found, only the MCP plugin for querying. source |
| W&B Weave B Weights and Biases, LLC | Agent Lens, named Weave's successor on the same backend, has Claude Code and Codex plugins that trace every turn, tool call and subagent with no proxy; a weave-claude-code plugin must be removed first and no PII scrubbing is implemented. source |
Couldn't check 1
The docs were closed to us, or silent. These carry no mark either way.
| Listing | Why |
|---|---|
| LangWatch B Reasoning Engine B.V. (doing business as LangWatch) | The listing claims Claude Code and Codex session tracking, but the docs index has no integration page for either harness, only Claude Platform OTel and a governance ingestion endpoint for external platforms. |
Doesn't do this job 6
Listings in the category that do something else, with the reason.
| Listing | Why |
|---|---|
| BotGauge E BotGauge AI Inc. (Delaware, United States) | No Claude Code or Codex integration in the docs index. source |
| DeepEval B Confident AI, Inc. | No Claude Code or Codex integration. source |
| Galileo API + MCP D Galileo Technologies, Inc. | No Claude Code or Codex integration in the docs index. source |
| Gauge CLI B Gauge | Gauge launches its own coding-agent sessions and keeps their transcripts under .gauge/runs; it does not ingest a team's Claude Code or Codex sessions. source |
| HoneyHive D HoneyHive Inc. | The coding agents page equips Cursor or Claude with skills, a CLI and a docs MCP to integrate HoneyHive; it does not trace Claude Code or Codex sessions. source |
| Prefactor B Prefactor Pty Ltd | No Claude Code or Codex integration. source |
What stops agents finishing
- harness exposes no hooks so a proxy is needed
- subscription plan without an API key
- hook drops traces with no error
- only one harness supported
Around it
Before: agent-harnesses.pair-in-terminal, agent-harnesses.parallel-worktrees. After: agent-observability.cost-per-run, agent-observability.debug-failed-run.
Where this comes from
Weight 7 of 100 in Agent observability & evals: 8 of 86 accounts: the LLM proxy thread (OTel from Codex and Claude Code, MLflow session tracing, Laminar's in-process proxy), MLflow plugin issues on silently dropped Stop-hook traces and other CLIs, the Opik Claude Code SDK request, the Phoenix Max-plan thread, the sub-agent cost CLI
The same as data: /api/v1/use-cases/agent-observability.coding-agent-sessions.json, or the MCP tool get_use_case with id agent-observability.coding-agent-sessions. Other jobs in this category: /api/v1/niches?category=agent-observability.
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.