Niches · Agent frameworks & SDKs

See every step of a run, and control what leaves the machine

No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.

0companies, full fit
17companies, fit with a limit
25companies carry the tag
5share of the category's work
5first-hand accounts

What counts as done

Each model call, tool call with arguments, hand-off and retry appears as its own span exported over OpenTelemetry to a backend the team chooses, the docs list what is sent to the vendor by default, and one documented switch stops it for every entry point (library, server, evals).

Measured, for our own tests: a packet capture of a first run with telemetry off shows no calls to the vendor. No docs we read state these figures, so they don't count against a listing here.

How agents ask for it: why did the agent fail at step 4, show me what each agent saw, turn off the framework's telemetry, send traces to our own collector

Does the job, with a limit 17

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
AgentOS A
Framers Lab, Inc.
All OpenTelemetry signals are off by default and go to the host's own exporter, with a content-safe switch. Emitted spans cover turns, tool-result handling and saves; per model call, hand-off and retry spans are not listed. source
Agno BB
Agno Inc.
OpenTelemetry spans cover runs, model calls, tool calls and team members. Usage telemetry is on by default and AGNO_TELEMETRY=false doesn't cover AgentOS or evals; an account saw requests continue with it set (agno #2172). source
Claude Agent SDK BB
Anthropic, PBC
OTLP export of metrics, log events and spans for model requests, tool calls and hooks, but traces need CLAUDE_CODE_ENHANCED_TELEMETRY_BETA. Usage metrics go to Anthropic by default on the Claude API; DISABLE_TELEMETRY or CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC turns them off. source
CrewAI C
CrewAI, Inc.
Anonymous telemetry is on by default, off with CREWAI_DISABLE_TELEMETRY; AMP tracing has its own switch and needs an account. An account saw the tracing prompt persist with tracing=False and the variable set (crewAI #3789). source
Cursor Agent SDK B
Anysphere, Inc.
Traces, metrics and logs go over OTLP to your collector only when an endpoint is set, with OTEL_LOG_TOOL_CONTENT for payloads. Every model turn still runs through Cursor's backend, and no list of what Cursor receives by default was found. source
Docker Agent BB
Docker, Inc.
--otel emits gen_ai and mcp spans for turns, model calls, tool calls, hand-offs and fallbacks to any OTLP endpoint. Product telemetry is on by default and can carry prompts given as arguments; TELEMETRY_ENABLED=false turns it off. source
Agent Development Kit (ADK) BB
Google LLC
OTLP traces with invoke_agent, invoke_workflow, execute_tool and generate_content spans go to any OTel backend. The traces page doesn't state what, if anything, is sent to Google by default, so the telemetry half of the criterion isn't documented. source
guilds.run C
Cigale AI
Every run keeps the prompt the model saw, each tool call and memory change in the app, but there is no OpenTelemetry export; Sentry error reporting is configured by environment variables. source
LangGraph BB
LangChain, Inc.
Every step is traced to LangSmith once its environment variables are set, which needs a LangSmith account. An OpenTelemetry route to any collector and a list of what is sent by default weren't found in the pages read. source
LlamaIndex Framework BB
LlamaIndex, Inc.
llama-index-observability-otel traces LLM, agent and pipeline events to any OpenTelemetry backend. The docs don't list what is sent by default; the risk instead is OpenAI as the default LLM and embedding provider. source
Mastra A
Kepler Software, Inc.
Tracing records model calls, tool calls and workflow steps, with exporters including OpenTelemetry and a sensitive-data filter on by default. The CLI, server and Studio send usage analytics by default per the listing; I didn't reopen the telemetry page. source
Microsoft Agent Framework A
Microsoft Corporation
Traces, logs and metrics follow OpenTelemetry GenAI conventions to any exporter, with sensitive data opt-in. The observability page doesn't list what goes to Microsoft by default; the listing records a User-Agent and feature token, which I didn't re-check. source
OpenAI Agents SDK AA
OpenAI OpCo, LLC
Tracing is on by default and goes to OpenAI; OPENAI_AGENTS_DISABLE_TRACING=1 or set_tracing_disabled turns it off, and custom processors send spans elsewhere. Tracing is unavailable to Zero Data Retention organisations. source
Pydantic AI AA
Pydantic Services Inc.
Instrumentation is plain OpenTelemetry, so spans go to any collector or Logfire; nothing reaches Logfire without a token. The page doesn't list what is sent by default or give a single off switch, since none is on. source
smolagents B
Hugging Face, Inc.
Runs are instrumented with OpenTelemetry to Phoenix, Langfuse or any backend. The docs read don't state what, if anything, is sent to Hugging Face by default. source
Strands Agents A
Amazon Web Services, Inc.
OpenTelemetry traces with model and tool invocation spans, including tool inputs and outputs, to any backend. The page doesn't say what is sent to AWS by default or give a single off switch. source
AI SDK AA
Vercel Inc.
Telemetry is OpenTelemetry through @ai-sdk/otel, registered by the app, with recordInputs and recordOutputs controls. The telemetry page doesn't say what is sent to Vercel by default; string model ids route through Vercel's AI Gateway per the listing. source

Doesn't do this job 3

Listings in the category that do something else, with the reason.

ListingWhy
BoxLang AI B
Ortus Solutions, Corp.
Tracing is logging and FlightRecorder middleware and BoxLang interception points; no OpenTelemetry export appears in the docs index. source
NitroStack C
Nitrostack Inc.
Logs show server output and every MCP message in Studio and NitroCloud; no OpenTelemetry export appears in the docs index. The listing's record of CLI PostHog events with no opt-out was not re-checked. source
TradingAgents B
Tauric Research
Reports are saved as one HTML page; the README documents no OpenTelemetry export or telemetry setting. source

What stops agents finishing

  • telemetry off switch not honoured by every entry point
  • tracing on by default with prompt and tool content sent to the vendor
  • a tracing library that breaks requests in the hosted runtime
  • tracing unavailable to zero-data-retention organisations

Around it

Before: frameworks.tool-calling-agent, frameworks.multi-agent-handoff. After: agent-observability.debug-failed-run, agent-observability.instrument-app.

Where this comes from

Weight 5 of 100 in Agent frameworks & SDKs: 3 of 61 accounts have this as their main job (share 4.9, rounded by largest remainder to sum 100)

The same as data: /api/v1/use-cases/frameworks.trace-a-run.json, or the MCP tool get_use_case with id frameworks.trace-a-run. Other jobs in this category: /api/v1/niches?category=frameworks.

Draft, read 2026-11-15 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.