Niches · Agent frameworks & SDKs
See every step of a run, and control what leaves the machine
No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.
What counts as done
Each model call, tool call with arguments, hand-off and retry appears as its own span exported over OpenTelemetry to a backend the team chooses, the docs list what is sent to the vendor by default, and one documented switch stops it for every entry point (library, server, evals).
Measured, for our own tests: a packet capture of a first run with telemetry off shows no calls to the vendor. No docs we read state these figures, so they don't count against a listing here.
How agents ask for it: why did the agent fail at step 4
, show me what each agent saw
, turn off the framework's telemetry
, send traces to our own collector
Does the job, with a limit 17
It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.
| Listing | Why |
|---|---|
| AgentOS A Framers Lab, Inc. | All OpenTelemetry signals are off by default and go to the host's own exporter, with a content-safe switch. Emitted spans cover turns, tool-result handling and saves; per model call, hand-off and retry spans are not listed. source |
| Agno BB Agno Inc. | OpenTelemetry spans cover runs, model calls, tool calls and team members. Usage telemetry is on by default and AGNO_TELEMETRY=false doesn't cover AgentOS or evals; an account saw requests continue with it set (agno #2172). source |
| Claude Agent SDK BB Anthropic, PBC | OTLP export of metrics, log events and spans for model requests, tool calls and hooks, but traces need CLAUDE_CODE_ENHANCED_TELEMETRY_BETA. Usage metrics go to Anthropic by default on the Claude API; DISABLE_TELEMETRY or CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC turns them off. source |
| CrewAI C CrewAI, Inc. | Anonymous telemetry is on by default, off with CREWAI_DISABLE_TELEMETRY; AMP tracing has its own switch and needs an account. An account saw the tracing prompt persist with tracing=False and the variable set (crewAI #3789). source |
| Cursor Agent SDK B Anysphere, Inc. | Traces, metrics and logs go over OTLP to your collector only when an endpoint is set, with OTEL_LOG_TOOL_CONTENT for payloads. Every model turn still runs through Cursor's backend, and no list of what Cursor receives by default was found. source |
| Docker Agent BB Docker, Inc. | --otel emits gen_ai and mcp spans for turns, model calls, tool calls, hand-offs and fallbacks to any OTLP endpoint. Product telemetry is on by default and can carry prompts given as arguments; TELEMETRY_ENABLED=false turns it off. source |
| Agent Development Kit (ADK) BB Google LLC | OTLP traces with invoke_agent, invoke_workflow, execute_tool and generate_content spans go to any OTel backend. The traces page doesn't state what, if anything, is sent to Google by default, so the telemetry half of the criterion isn't documented. source |
| guilds.run C Cigale AI | Every run keeps the prompt the model saw, each tool call and memory change in the app, but there is no OpenTelemetry export; Sentry error reporting is configured by environment variables. source |
| LangGraph BB LangChain, Inc. | Every step is traced to LangSmith once its environment variables are set, which needs a LangSmith account. An OpenTelemetry route to any collector and a list of what is sent by default weren't found in the pages read. source |
| LlamaIndex Framework BB LlamaIndex, Inc. | llama-index-observability-otel traces LLM, agent and pipeline events to any OpenTelemetry backend. The docs don't list what is sent by default; the risk instead is OpenAI as the default LLM and embedding provider. source |
| Mastra A Kepler Software, Inc. | Tracing records model calls, tool calls and workflow steps, with exporters including OpenTelemetry and a sensitive-data filter on by default. The CLI, server and Studio send usage analytics by default per the listing; I didn't reopen the telemetry page. source |
| Microsoft Agent Framework A Microsoft Corporation | Traces, logs and metrics follow OpenTelemetry GenAI conventions to any exporter, with sensitive data opt-in. The observability page doesn't list what goes to Microsoft by default; the listing records a User-Agent and feature token, which I didn't re-check. source |
| OpenAI Agents SDK AA OpenAI OpCo, LLC | Tracing is on by default and goes to OpenAI; OPENAI_AGENTS_DISABLE_TRACING=1 or set_tracing_disabled turns it off, and custom processors send spans elsewhere. Tracing is unavailable to Zero Data Retention organisations. source |
| Pydantic AI AA Pydantic Services Inc. | Instrumentation is plain OpenTelemetry, so spans go to any collector or Logfire; nothing reaches Logfire without a token. The page doesn't list what is sent by default or give a single off switch, since none is on. source |
| smolagents B Hugging Face, Inc. | Runs are instrumented with OpenTelemetry to Phoenix, Langfuse or any backend. The docs read don't state what, if anything, is sent to Hugging Face by default. source |
| Strands Agents A Amazon Web Services, Inc. | OpenTelemetry traces with model and tool invocation spans, including tool inputs and outputs, to any backend. The page doesn't say what is sent to AWS by default or give a single off switch. source |
| AI SDK AA Vercel Inc. | Telemetry is OpenTelemetry through @ai-sdk/otel, registered by the app, with recordInputs and recordOutputs controls. The telemetry page doesn't say what is sent to Vercel by default; string model ids route through Vercel's AI Gateway per the listing. source |
Doesn't do this job 3
Listings in the category that do something else, with the reason.
| Listing | Why |
|---|---|
| BoxLang AI B Ortus Solutions, Corp. | Tracing is logging and FlightRecorder middleware and BoxLang interception points; no OpenTelemetry export appears in the docs index. source |
| NitroStack C Nitrostack Inc. | Logs show server output and every MCP message in Studio and NitroCloud; no OpenTelemetry export appears in the docs index. The listing's record of CLI PostHog events with no opt-out was not re-checked. source |
| TradingAgents B Tauric Research | Reports are saved as one HTML page; the README documents no OpenTelemetry export or telemetry setting. source |
What stops agents finishing
- telemetry off switch not honoured by every entry point
- tracing on by default with prompt and tool content sent to the vendor
- a tracing library that breaks requests in the hosted runtime
- tracing unavailable to zero-data-retention organisations
Around it
Before: frameworks.tool-calling-agent, frameworks.multi-agent-handoff. After: agent-observability.debug-failed-run, agent-observability.instrument-app.
Where this comes from
Weight 5 of 100 in Agent frameworks & SDKs: 3 of 61 accounts have this as their main job (share 4.9, rounded by largest remainder to sum 100)
The same as data: /api/v1/use-cases/frameworks.trace-a-run.json, or the MCP tool get_use_case with id frameworks.trace-a-run. Other jobs in this category: /api/v1/niches?category=frameworks.
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.