Niches · Agent observability & evals
Get traces out of an agent framework into a backend
Four or more companies' docs show a full fit.
What counts as done
Spans arrive under one trace per request with the framework's model and tool steps as children, and no root span is lost under async.
Measured, for our own tests: setup takes under 30 lines, and a test request is visible within 5 minutes of starting. No docs we read state these figures, so they don't count against a listing here.
How agents ask for it: add tracing to my LangGraph agent
, send OpenTelemetry spans to it
, keep FastAPI and the agent in one trace
, why is my root span missing
Does the job 5
The docs show it meets the test above, and nothing we read says otherwise.
| Listing | Why |
|---|---|
| Arize Phoenix BB Arize AI, Inc. | OTLP over gRPC or HTTP, phoenix.otel register, OpenInference instrumentors for LangGraph, ADK, Pydantic AI and others, and a context propagation page covering asyncio and threads. An HN self-hoster had no tracing complaints. source |
| Laminar API + MCP C LMNR AI, Inc. | OTLP over gRPC, HTTP protobuf and HTTP JSON on documented ports for cloud and self-hosted, OpenLLMetry conventions, integrations for LangChain, Vercel AI and the Claude Agent SDK; ingest answers 400, 401 and 413 and flush_by_size avoids oversized batches. source |
| LangWatch B Reasoning Engine B.V. (doing business as LangWatch) | OTLP at /api/otel/v1/traces with a project key, Python, TypeScript and Go SDKs, a LangGraph node-by-node page and semantic conventions tutorial; gateway traffic emits traces automatically. Cost enrichment runs in the same pipeline. source |
| Pydantic Logfire B Pydantic Services Inc. | OTel-native: any OTLP exporter posts to logfire-us or logfire-eu with a write token, FastAPI and Pydantic AI instrument in one line and spans appear within seconds; two HN users found Pydantic AI plus Logfire the easiest agent debugging. source |
| Respan API + MCP BB Keywords AI Inc. (d/b/a Respan) | OTLP/HTTP in JSON or protobuf at /api/v2/traces with a Bearer key, gen_ai attributes shown, plus an SDK that auto-instruments OpenAI, Anthropic, Bedrock and Vertex and integration pages for LangGraph, ADK, Pydantic AI and Vercel AI. source |
Does the job, with a limit 11
It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.
| Listing | Why |
|---|---|
| BotGauge E BotGauge AI Inc. (Delaware, United States) | OTLP/HTTP protobuf at api.botgauge.com/v1/traces with a Bearer key from the SDK or any exporter, integrations for LangGraph, ADK, OpenAI Agents and Pydantic AI. Bare exporters get no session ids or masking, and the service excludes the EEA and UK. source |
| Braintrust API + MCP B Braintrust Data, Inc. | OTLP at api.braintrust.dev/otel with x-bt-parent and documented gen_ai and OpenInference attribute mapping, plus framework pages for LangGraph, ADK, Pydantic AI and Vercel AI. wrapAISDK produced nested duplicate spans (992) and a logged span exposed the API key (912). source |
| Galileo API + MCP D Galileo Technologies, Inc. | GalileoSpanProcessor exports OTLP to api.galileo.ai/otel/traces with OpenInference instrumentors for ADK, Pydantic AI and Vercel AI, and traceparent joins distributed spans. The ADK example warns about asyncio context cleanup, and self-hosted endpoints differ. source |
| Helicone AI Gateway + MCP E Helicone, Inc. | Traces arrive by changing the base URL to the gateway or through the OpenLLMetry async integration; framework pages cover LangGraph, OpenAI Agents and Vercel AI. No general OTLP endpoint is documented, so non-model steps need session headers. source |
| HoneyHive D HoneyHive Inc. | The SDK wraps an OTel TracerProvider, accepts OpenInference and OpenLLMetry instrumentors, and propagates W3C context across services. A raw OTLP exporter without the SDK is available only by contacting HoneyHive. source |
| Langfuse API + MCP BB Langfuse GmbH (a ClickHouse, Inc. company) | OTLP endpoint /api/public/otel plus OTel-native SDKs and framework pages for LangGraph, ADK, Pydantic AI and Vercel AI. Root traces went missing under FastAPI plus LangGraph (issue 13023) and most traces vanished at sampling 1.0 (issue 8837). source |
| LangSmith API + MCP BB LangChain, Inc. | OTLP at the API URL plus /otel with x-api-key and a Langsmith-Project or agent headers, and integration pages for LangGraph, ADK, Pydantic AI and Vercel AI. Trace context did not reach the app's own OTel spans (SDK issue 58) and Python below 3.11 splits async traces. source |
| MLflow Tracing C MLflow Project, a Series of LF Projects, LLC | OTLP at /v1/traces with an x-mlflow-experiment-id header, GenAI semantic conventions, a FastAPI example and W3C traceparent propagation. Threads need contextvars copied, expired credentials drop traces with only a log line, and autolog regressed for Pydantic AI (22208). source |
| Opik API + MCP BB Comet ML, Inc. | OTLP at /api/v1/private/otel with projectName and Comet-Workspace headers, mapping GenAI, OpenInference and Opik attributes; SDK batching and an offline fallback that replays failed messages. Streamed ADK spans never finalised (5524). source |
| Overmind C Overmind Technology Ltd (England and Wales, company 16585996) | OTLP protobuf or JSON at /api/v1/traces with X-Api-Key, normalising GenAI, OpenLLMetry and OpenInference attributes; the Python SDK instruments OpenAI, Anthropic, Google, Agno and LangChain or LangGraph and warns against competing TracerProviders. Alpha SDK. source |
| W&B Weave B Weights and Biases, LLC | OTLP protobuf to /agents/otel/v1/traces with wandb-api-key and GenAI conventions, no SDK needed. Spans without entity and project are dropped, a doubled path yields a 404 and silently dropped spans, and the docs now point new agent users to Agent Lens. source |
Doesn't do this job 6
Listings in the category that do something else, with the reason.
| Listing | Why |
|---|---|
| Agent Tree C No legal entity found. The LICENSE, plugin.json and marketplace.json name "Mike Pollow, Digitalroots" | A Claude Code plugin, not a backend. source |
| agents-tree B No legal entity found. The LICENSE and pyproject.toml name Karel Blavka | A CLI that reads transcripts, not a backend. source |
| DeepEval B Confident AI, Inc. | Tracing is for attaching metrics to spans during evaluation, not a backend that ingests production traces. source |
| Gauge CLI B Gauge | Nothing to instrument; Gauge runs Claude Code, Codex, OpenCode, Pi or Antigravity in Docker or hosted VMs. source |
| Navige E Navige Ltd | A proxy with no OTLP ingest; traceparent is read only to correlate calls into a transaction. source |
| Prefactor B Prefactor Pty Ltd | The full documentation never mentions OpenTelemetry or OTLP; spans come only from the TypeScript and Python SDKs against your own activity schema. source |
What stops agents finishing
- framework not in the integration list and no OTLP endpoint
- OTel context conflict with the framework's own instrumentation
- traces stop after an SDK or platform upgrade
- no way to point the SDK at another backend
- agent reachable only over HTTP with no instrumentation inside
Around it
Before: agent-harnesses.embed-in-program, inference.tool-loop. After: agent-observability.debug-failed-run, agent-observability.cost-per-run.
Where this comes from
Weight 17 of 100 in Agent observability & evals: 19 of 86 accounts: Langfuse issues on async root spans, trace_context, callback handlers and sampling; Logfire, Weave, Opik, Braintrust and LangSmith SDK issues on integrations, context propagation, exporters, dependencies and leaked keys; Phoenix issues on black-box agents and dropped spans; the LangChain forum outage thread
The same as data: /api/v1/use-cases/agent-observability.instrument-app.json, or the MCP tool get_use_case with id agent-observability.instrument-app. Other jobs in this category: /api/v1/niches?category=agent-observability.
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.