{
  "fixes": {
    "slug": "pipecat",
    "name": "Pipecat",
    "listing": "https://www.anchorterminal.com/tools/pipecat",
    "markdown": "# Fix list: Pipecat\n\nFrom Anchor Terminal's listing at https://www.anchorterminal.com/tools/pipecat, the October 2026 research run, assessed 17 October 2026. Grade B, 68.9 out of 100.\n\nThis is everything the published grade says the listing lacks, the biggest possible gain to the total first. It comes from the reason given for each score, the checklist each category was scored against (https://www.anchorterminal.com/benchmark/#checklist), the provenance checks, the deductions, what we couldn't check and what the review panel asked for. A fix counts at the next check, once it's public.\n\nFor a coding agent working on Pipecat: work through the items below in the product, its docs and its public pages. Each category gives the reason for its score, with the points each checklist item earned, and the checklist itself, so the gap is the items that earned less than their points. Change the product, not the wording, and keep a note of what you changed and where it's published.\n\n## 1. Security \u0026 auth, 57 out of 100, up to 7.5 more on the total\n\nWhy it scored 57: Framework reading, as for the frameworks category. We found no telemetry in the source, and tracing is off by default and needs an exporter the developer supplies. Up to 1.12.0, though, the OpenAI-compatible provider services (Groq, Cerebras, OpenRouter, Together and fifteen more) fall back to `OPENAI_API_KEY` when given no key, sending it to that provider, and error logs could carry API keys and signed URLs. Both are fixed on main and unreleased on 17 October (20 of 30). No tool-approval or confirmation primitive. `tools_filter` allowlists MCP tools and `tools_arguments` pins values the model cannot change (8 of 20). A caller's speech and MCP tool results go to the model, and we found no prompt-injection guidance in the docs (3 of 15). OpenTelemetry spans per service and tool call, with `FunctionCallObserver` and `LLMLogObserver` for per-call visibility (15). SECURITY.md routes reports to an email address at Daily, and GitHub advisories are published with CVEs and fixed versions, four since April 2026. No security.txt or bug bounty. Daily states SOC 2 Type 2, which isn't scored for a framework (11 of 20).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-security):\n\n- 0 to 30, the credential model. 30 for OAuth 2.1 with scopes, or scoped and revocable keys with rotation. 20 for plain revocable API keys. 10 for one all-powerful key. 10 off when a secret can travel in a URL query string as a documented option.\n- 0 to 20, read-only or least-privilege modes, and confirmation or approval for destructive actions.\n- 0 to 15, prompt-injection posture where the tool returns untrusted content (documented mitigations or guidance). A tool that returns no untrusted content gets 10.\n- 0 to 15, audit logs or per-call visibility for the operator.\n- 0 to 20, a security programme. security.txt or a disclosure policy, a bug bounty, SOC 2 or ISO 27001, advisories handled in public.\n\nModels are read for retention, whether API data trains models (and whether that's off by default), zero-retention options and certifications. Frameworks for telemetry defaults, approval hooks, guardrails and sandboxing.\n\n## 2. Reliability, 72 out of 100, up to 5.6 more on the total\n\nWhy it scored 72: Local-software reading of the framework. pipecat-ai installs from PyPI with Python 3.11 or later declared, and a compatibility workflow builds it on 3.11 to 3.14 (20). A public tests workflow runs pytest over 345 test files on pushes to main and on pull requests, with coverage and licence-check jobs, but GitHub's Actions pages weren't readable from our session, so a green main is not confirmed (15 of 25). GitHub counted 376 open issues and pull requests on 10 October per the repos.ecosyste.ms mirror. Recent open issues in the issues.ecosyste.ms mirror include regressions such as Anthropic streaming dropping all but the last tool call (6084) and default requests failing on a current Claude model (6107), several with no reply yet, against about 3,200 commits in 90 days. The mirror's open states weren't fully reliable, so this is a judgement from the issues read (12 of 25). Keep a Changelog with dates, semver stated and breaking changes flagged with a warning sign, but 1.8.0 and 1.10.0 carry 16 and 4 breaking or behaviour changes as minor releases (10 of 15). 1.12.0, classed Production/Stable (15).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-reliability):\n\nHosted APIs, MCP servers, models and platforms.\n\n- 20, a public status page with component history (Statuspage, Instatus, BetterStack or the vendor's own).\n- 0 to 30, the incident record for the last 90 days on that page. 30 for a clean record or trivial incidents only, 20 for minor incidents only, 10 for one major outage (an hour or more of a core API down, or errors across the board), 0 for several. 5 when there's no history we could read, and the note says so.\n- 15, rate limits documented with numbers.\n- 15, documented 429 or overload handling (Retry-After, backoff guidance), and idempotency keys or safe-retry guidance where writes are involved.\n- 10, an SLA published for any paid tier.\n- 10, the surface agents use is generally available, not beta or preview.\n\nLocal packages, SDKs, frameworks and stdio MCP servers.\n\n- 20, installs from an official package with supported runtimes stated.\n- 25, a public CI and test suite, passing on the default branch.\n- 0 to 25, open crash or regression issues relative to activity (25 for few and handled, 0 for many, old and unanswered).\n- 15, semver discipline and breaking changes called out in a changelog.\n- 15, version 1.0 or later, or declared stable.\n\nProtocols are read from their reference implementations, the public facilitators or servers, spec stability and test vectors.\n\n## 3. Payments \u0026 pricing, 60 out of 100, up to 5 more on the total\n\nWhy it scored 60: No x402, MPP or L402 (0). Read with the self-hosted rule, with the pricing line scored on Pipecat Cloud, the paid option. Its per-minute prices by agent size, transport, telephony and recording are published without a login (20). The BSD-2-Clause package installs with no card. Pipecat Cloud needs billing credentials or a trial code before sessions start (20, on the package). The framework runs with no account, on the developer's own provider keys (20).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-payments):\n\nThe published rubric, also on the [x402 page](https://www.anchorterminal.com/x402/).\n\n- 40, a machine payment protocol (x402, MPP or L402) on the tool's own endpoints. 10 to 30 when it covers only some endpoints or only goes through a third party, and the note says which.\n- 20, per-call or per-unit pricing published without a login. 10 for public plan-only pricing, 0 for \"contact sales\" or prices behind a login.\n- 20, a free tier or trial that doesn't need a card.\n- 20, autonomous onboarding, meaning an agent can get access without a person signing up in a browser (keyless use, x402, a programmatic key API).\n\nPayment platforms and agent wallets rarely charge for their own API over a machine protocol, so the first line has steps for them, and the highest one that applies counts. 40 when x402, MPP or L402 runs on all their own endpoints, 30 when it runs on part of their own API, 25 when their merchants can accept one, 20 for running a facilitator, 15 for paying as a buyer, and 0 when the only protocol is their own. Merchant acceptance sits above a facilitator because the platform's own customers can charge agents through it, while a facilitator settles for sellers who wire up the protocol themselves. The counter-argument (a facilitator does more for the protocol as a whole) has a point. Each note says which step applied.\n\nOpen-source software you run yourself is scored on its hosted or paid option if it has one. A free, self-hosted package with nothing to buy gets 20, 20 and 20 for the last three lines, and 0 to 40 for the first only if it ships a payment protocol.\n\n## 4. Agent ergonomics, 72 out of 100, up to 4.6 more on the total\n\nWhy it scored 72: Framework reading. `MCPClient` registers an MCP server's tools with the LLM, with `tools_filter` to allow only named tools and `tools_arguments` to fix values and hide them from the schema. The repository's streamable HTTP MCP example is about 130 lines of code, most of it the voice pipeline rather than the tool wiring (20 of 25). `tools_output_filters` trims tool results, and context summarisation and Flows context strategies limit what the model sees (15 of 20). Errors carry a category that marks permanent failures, services expose `is_usable`, `ProcessorUnusablePolicy` decides whether a bot ends, and MCP failures return to the model as text (17 of 20). Function calls cancel on interruption by default, with per-tool timeouts and opt-in asynchronous or model-cancellable calls, but there is no idempotency guidance for tools with side effects (10 of 20). A bot needs a transport, speech-to-text, LLM, text-to-speech, VAD and context aggregators wired by hand, which `pipecat init` scaffolds. Server code is Python only, with client SDKs in six languages (10 of 15).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-ergonomics):\n\n- 0 to 25, context cost. For MCP, the number and size of the tool definitions (25 for ten or fewer compact tools, 15 for 11 to 30, 5 for more than 30, plus up to 10 back for toolsets, dynamic loading or read-only subsets). For APIs, whether responses can be sized (field selection, limits, summaries).\n- 20, pagination, filtering and output-size controls.\n- 20, actionable, documented error responses, codes and messages an agent can recover from.\n- 20, idempotency or safe retries, and for MCP the `readOnlyHint` and `destructiveHint` annotations.\n- 15, sensible defaults, few required parameters, and official SDKs in at least two languages.\n\nModels are read for tool use, structured output, prompt caching, context length, batch and SDKs. Frameworks for how much code and how many defaults a tool-calling agent with MCP needs.\n\n## 5. Transparency \u0026 trust, 81 out of 100, up to 1.7 more on the total\n\nMade of editorial 83, provenance 79.\n\nWhy it scored 81: BSD-2-Clause (30). The framework keeps no data. Daily's privacy policy (6 January 2025) keeps personal data while an account is open, the Pipecat Cloud security guide says session data is removed when a session ends, logs are stored by default and administrative API calls are kept a year, and Daily publishes a DPA (1 May 2025) and a sub-processor list (1 September 2026). Control-plane data is in the US whatever the agent's region (24 of 30). The changelog has Deprecated sections in most releases, and recent entries name removal in 2.0.0, with a migration guide to 1.0, but no dated removal windows (13 of 20). No telemetry found, and tracing is opt-in, though the reference page states the opposite default. The optional ai-coustics filter registers a Pipecat identifier with that vendor's SDK, which the docs don't mention (16 of 20).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-transparency):\n\n- 0 to 30, source availability and licence clarity. 30 for open source under an OSI licence, 15 for closed with clear terms, 0 for unclear terms.\n- 0 to 30, data handling and retention statements that agree with each other (privacy policy, DPA, retention periods, subprocessors).\n- 0 to 20, a deprecation policy or notices with dates.\n- 0 to 20, telemetry disclosed with an opt-out (local software), or subprocessors and data locations disclosed (hosted).\n\nThe other half of Transparency and trust is the provenance score, computed from checked facts (below). The category score is the mean of the two.\n\nProvenance checks not met in full (half of this category, computed from checked facts):\n\n- Domain age: pipecat.ai, registered 2024-05-01 (2 years) (7 of 15)\n- security.txt: not found (0 of 10)\n\n## 6. Schema \u0026 documentation, 90 out of 100, up to 1.6 more on the total\n\nWhy it scored 90: Framework reading. Typed Python (py.typed) with an API reference page per service, transport, frame type and utility (25). llms.txt (113 KB), Markdown copies of each page and llms-full.txt (4.2 MB) (10). Guides say when to reach for Flows, multiple LLM agents or a single pipeline, and the error-handling page says which failures are worth retrying. The OpenTelemetry reference gives `PipelineWorker` defaults of `enable_tracing` True and `enable_turn_tracking` False, the reverse of the 1.12.0 source (14 of 20). Pydantic `Settings` classes per service, a provider-neutral `FunctionSchema`, and an `ErrorCategory` enum (12 of 15). 24 example folders, from function calling and MCP to telephony, and documented error categories and Pipecat Cloud error codes (14 of 15). Dated changelog with pull-request links, towncrier fragments and a migration guide to 1.0 (15).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-schema):\n\nAPIs and MCP servers.\n\n- 25, a machine-readable contract (a public OpenAPI file or similar; for MCP, typed JSON Schema inputs on every tool).\n- 10, llms.txt or Markdown docs served for agents.\n- 0 to 20, descriptions that say what a tool is for, when to use it and when not to, read from the tool definitions in the source or the API reference.\n- 0 to 15, typed inputs with enums, constraints and required fields, and no free-form JSON blobs.\n- 0 to 15, examples and documented error responses.\n- 15, versioning and a public changelog.\n\nModels are read from the API reference, the OpenAPI file, llms.txt, the structured-output and tool-use docs and the model cards. Frameworks from docs a model can follow, typed interfaces, examples and the API reference.\n\n## 7. Maintenance \u0026 community, 87 out of 100, up to 1.1 more on the total\n\nWhy it scored 87: pipecat-ai 1.12.0 on PyPI on 26 September 2026, 21 days before the check (30). Eight releases in the 90 days to 17 October, 1.6.0 to 1.12.0 (20). In the past year 80 of 158 issues were closed, taking about 26 days on average, and 565 of 779 pull requests, per issues.ecosyste.ms. A needs-repro workflow labels issues, and reply times on GitHub weren't readable from our session (15 of 25). The Python framework and the JavaScript client (1.14.0) are current, with client SDKs in six languages (15). CI, coverage, a licence check, a uv lockfile and trusted publishing to PyPI are configured. No Dependabot or Renovate configuration was found, and a passing main wasn't confirmed (7 of 10).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-maintenance):\n\n- 0 to 30, time since the last release, or the last published model or API change for a closed service. 30 within 30 days, 20 within 90, 10 within 180, 0 older.\n- 20, at least three releases or dated changelog entries in the last 90 days.\n- 0 to 25, responsiveness. Issues and pull requests answered on GitHub (the open issues and how recent the replies are). For closed services, a public changelog and a support or community channel that answers, 0 to 15.\n- 15, presence in the official MCP registry under a verified namespace (MCP servers), or current official SDKs (APIs and models).\n- 10, package health, current dependencies and CI.\n\nModels are read for deprecation notice periods and model churn rather than release counts.\n\n## Deductions\n\nEach comes off the total. A fixed and documented problem counts for less at the next check.\n\n- 2025-11-10 to 2026-06-17. Three published advisories, fixed and between four and eleven months old, so 1 point each. Remote code execution through `pickle.loads()` in the optional, deprecated LivekitFrameSerializer (GHSA-c2jg-5cp7-6wc7, CVE-2025-62373, critical, fixed in 0.0.94 on 10 November 2025, advisory published 23 April 2026), arbitrary file read through the development runner's `/files` endpoint (GHSA-3363-2ph6-35wh, high, fixed in 1.2.0 on 14 May 2026) and unauthenticated call control through the runner's telephony `/ws` endpoint (GHSA-j8cv-x86q-rj85, high, fixed in 1.4.0 on 17 June 2026). The moderate GHSA-hqm2-cfx2-2vgh of 15 October 2026 covers behaviour documented as by design and isn't deducted. https://osv.dev/vulnerability/GHSA-c2jg-5cp7-6wc7, https://osv.dev/vulnerability/GHSA-3363-2ph6-35wh, https://osv.dev/vulnerability/GHSA-j8cv-x86q-rj85\n- 2026-10-16. changelog/6193.security.md says the services for OpenAI-compatible providers other than OpenAI sent `OPENAI_API_KEY` to their provider when given no `api_key`, including when `os.getenv` read an unset variable. Fixed on main on 16 October 2026 (commit 89d0e031a) and not in a release at the check, with no advisory, so 1 point. https://github.com/pipecat-ai/pipecat/blob/main/changelog/6193.security.md\n\n## What we couldn't check\n\nWhat we couldn't read counted as absent. Publishing it on a page a plain HTTP fetch can read (not only in a browser) lets the next check count it.\n\n- Judgement call on category. Listed in voice agents rather than frameworks, because its job is running spoken conversations as a speech-to-text, LLM and text-to-speech pipeline, which is what voice agents compares, and it lands in head-to-heads with Vapi, Retell, Ultravox and ElevenLabs Agents. `voice.agent` goes first since the framework runs the conversation, and `agent.mcp-client` and `agent.multi-agent` are added from the frameworks vocabulary.\n- Graded on the open-source framework with the framework and local-software lines. Pipecat Cloud and Pipecat Enterprise are described in details with Pipecat Cloud's published prices and are not graded, except that the Payments pricing line uses Pipecat Cloud's public prices as the paid option, as Pydantic AI's uses Logfire's.\n- firstReleased is 2024-04-04, tag v0.0.2 (17:45 on 3 April 2024 in the tagger's time zone), when the project and package were named `dailyai`. The first `pipecat-ai` release on PyPI is 0.0.9 on 13 May 2024. unchecked: whether `dailyai` was published to PyPI, because pypi.org/project/dailyai/ answered with a bot check and wasn't read.\n- unchecked: whether CI passes on main, open issue and pull request counts on GitHub itself, and reply times. GitHub's pages answered 403 through the session's proxy and WebFetch refused github.com. Star and open counts come from repos.ecosyste.ms (synced 10 October 2026) and issue statistics from issues.ecosyste.ms (synced 16 October 2026). The issues mirror listed some items as open that GitHub's own count suggests are closed, so its open totals weren't used.\n- PhoneLLM Mini is recorded in details from Cloudflare's changelog and model page of 15 October 2026 and the Hugging Face card, which states BSD-2-Clause and keeps the NVIDIA Nemotron Open Model License for the base model. Not graded and not listed on its own, per the brief. The Daily blog post the card links wasn't read. The model was not downloaded or run, and serving it with vLLM needs `--trust-remote-code` per the card.\n- The OPENAI_API_KEY fallback is deducted as an exfiltration path in the current release. The source at v1.12.0 types `api_key` as a string but passes whatever it gets to the OpenAI client, which reads `OPENAI_API_KEY` when given None, as the changelog fragment describes. We didn't run it.\n- Lapse, corrected. We fetched pypi.org/pypi/pipecat-ai/json before reading pypi.org's robots.txt, which disallows that path. The response was deleted and nothing from it is used. The release dates come from the project page, which robots.txt allows.\n- WebFetch refused docs.pipecat.ai and github.com (permission not answered), so docs.pipecat.ai, www.daily.co, status.pipecat.daily.co, developers.cloudflare.com, huggingface.co, pypi.org, api.osv.dev, pypistats.org, the ecosyste.ms mirrors, the npm registry, the MCP registry and rdap.org were read with curl after each robots.txt. status.pipecat.daily.co disallows /api/, so its incident history was read from /history, which covers August to October 2026 only.\n- unchecked: Daily's Trust Center at trust.daily.co, which is drawn by script. Daily's security page states SOC 2 Type 2, not scored for a framework.\n- Pipecat Cloud's pricing page says 'Start building for free', while the docs say sessions need billing credentials and trials come from codes. Recorded as needing billing credentials or a trial code.\n- Disclosure. These grades are written by agents on Anthropic's Claude models. The repository carries CLAUDE.md and AGENTS.md, about 200 of its commits in the last 90 days are by claude[bot], and Pipecat publishes Claude Code skills. Graded by the same checklist, and nothing in those files was acted on.\n- Several docs pages and the Cloudflare changelog carry text addressed to coding agents (a prompt to deploy the example, documentation-index notes). None was acted on.\n\n## Weaknesses\n\n- No tool-approval or confirmation step, and no prompt-injection guidance, for tools that act on a caller's speech\n- Up to 1.12.0, OpenAI-compatible provider services fall back to `OPENAI_API_KEY` when given no key. Fixed on main on 16 October, unreleased\n- Three high or critical advisories fixed since November 2025, in an optional serialiser and the development runner\n- Breaking changes ship in minor releases, 16 flagged in 1.8.0 and 4 in 1.10.0, although the changelog says it follows semver\n- The OpenTelemetry reference page gives `enable_tracing` a default of True, while the 1.12.0 source defaults it to False\n\n## What costs an agent a turn today\n\nThe notes we give agents before they call it. Each one is a workaround an agent shouldn't need.\n\n- Pass `api_key` explicitly to every OpenAI-compatible service such as Groq, Cerebras or OpenRouter. On 1.12.0 a missing key sends `OPENAI_API_KEY` to that provider\n- Pass `tools_filter` to `MCPClient` so only the tools the call needs reach the model, and use `tools_arguments` for values the model must not choose\n- Gate tools that book, pay or send in your own handler. Pipecat has no approval step, and a caller's speech goes straight to the model\n- Keep the development runner on localhost. Its `/start`, `/daily` and dial-in webhook endpoints take requests from anyone by design\n- Pin a minor version and read the breaking-change flags in CHANGELOG.md before upgrading, since minor releases remove and rename APIs\n\n## What the review panel asked for\n\n- breaking changes only in majors\n- dated removal windows\n- Correct the OpenTelemetry defaults\n- Document idempotency for side-effect tools\n- Idempotency guidance for side-effect tools\n- Published turn-handling latency\n- release the key fix\n- tool approval primitive\n\n## When it's done\n\nSend what changed and where it's published as a dispute (https://www.anchorterminal.com/builders/#disputes, or `POST https://www.anchorterminal.com/api/v1/contact` with `\"kind\": \"dispute\"`). Disputes are answered in public, and the listing is checked again by the same checklist. Paying for an audit or a listing claim changes nothing here.\n",
    "grade": "B",
    "score": 68.9,
    "assessed": "2026-10-17",
    "run": "October 2026 research run",
    "categories": [
      {
        "key": "security",
        "name": "Security \u0026 auth",
        "score": 57,
        "maxGain": 7.5,
        "reason": "Framework reading, as for the frameworks category. We found no telemetry in the source, and tracing is off by default and needs an exporter the developer supplies. Up to 1.12.0, though, the OpenAI-compatible provider services (Groq, Cerebras, OpenRouter, Together and fifteen more) fall back to `OPENAI_API_KEY` when given no key, sending it to that provider, and error logs could carry API keys and signed URLs. Both are fixed on main and unreleased on 17 October (20 of 30). No tool-approval or confirmation primitive. `tools_filter` allowlists MCP tools and `tools_arguments` pins values the model cannot change (8 of 20). A caller's speech and MCP tool results go to the model, and we found no prompt-injection guidance in the docs (3 of 15). OpenTelemetry spans per service and tool call, with `FunctionCallObserver` and `LLMLogObserver` for per-call visibility (15). SECURITY.md routes reports to an email address at Daily, and GitHub advisories are published with CVEs and fixed versions, four since April 2026. No security.txt or bug bounty. Daily states SOC 2 Type 2, which isn't scored for a framework (11 of 20).",
        "checklist": [
          "- 0 to 30, the credential model. 30 for OAuth 2.1 with scopes, or scoped and revocable keys with rotation. 20 for plain revocable API keys. 10 for one all-powerful key. 10 off when a secret can travel in a URL query string as a documented option.\n- 0 to 20, read-only or least-privilege modes, and confirmation or approval for destructive actions.\n- 0 to 15, prompt-injection posture where the tool returns untrusted content (documented mitigations or guidance). A tool that returns no untrusted content gets 10.\n- 0 to 15, audit logs or per-call visibility for the operator.\n- 0 to 20, a security programme. security.txt or a disclosure policy, a bug bounty, SOC 2 or ISO 27001, advisories handled in public.",
          "Models are read for retention, whether API data trains models (and whether that's off by default), zero-retention options and certifications. Frameworks for telemetry defaults, approval hooks, guardrails and sandboxing."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-security"
      },
      {
        "key": "reliability",
        "name": "Reliability",
        "score": 72,
        "maxGain": 5.6,
        "reason": "Local-software reading of the framework. pipecat-ai installs from PyPI with Python 3.11 or later declared, and a compatibility workflow builds it on 3.11 to 3.14 (20). A public tests workflow runs pytest over 345 test files on pushes to main and on pull requests, with coverage and licence-check jobs, but GitHub's Actions pages weren't readable from our session, so a green main is not confirmed (15 of 25). GitHub counted 376 open issues and pull requests on 10 October per the repos.ecosyste.ms mirror. Recent open issues in the issues.ecosyste.ms mirror include regressions such as Anthropic streaming dropping all but the last tool call (6084) and default requests failing on a current Claude model (6107), several with no reply yet, against about 3,200 commits in 90 days. The mirror's open states weren't fully reliable, so this is a judgement from the issues read (12 of 25). Keep a Changelog with dates, semver stated and breaking changes flagged with a warning sign, but 1.8.0 and 1.10.0 carry 16 and 4 breaking or behaviour changes as minor releases (10 of 15). 1.12.0, classed Production/Stable (15).",
        "checklist": [
          "Hosted APIs, MCP servers, models and platforms.",
          "- 20, a public status page with component history (Statuspage, Instatus, BetterStack or the vendor's own).\n- 0 to 30, the incident record for the last 90 days on that page. 30 for a clean record or trivial incidents only, 20 for minor incidents only, 10 for one major outage (an hour or more of a core API down, or errors across the board), 0 for several. 5 when there's no history we could read, and the note says so.\n- 15, rate limits documented with numbers.\n- 15, documented 429 or overload handling (Retry-After, backoff guidance), and idempotency keys or safe-retry guidance where writes are involved.\n- 10, an SLA published for any paid tier.\n- 10, the surface agents use is generally available, not beta or preview.",
          "Local packages, SDKs, frameworks and stdio MCP servers.",
          "- 20, installs from an official package with supported runtimes stated.\n- 25, a public CI and test suite, passing on the default branch.\n- 0 to 25, open crash or regression issues relative to activity (25 for few and handled, 0 for many, old and unanswered).\n- 15, semver discipline and breaking changes called out in a changelog.\n- 15, version 1.0 or later, or declared stable.",
          "Protocols are read from their reference implementations, the public facilitators or servers, spec stability and test vectors."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-reliability"
      },
      {
        "key": "payments",
        "name": "Payments \u0026 pricing",
        "score": 60,
        "maxGain": 5,
        "reason": "No x402, MPP or L402 (0). Read with the self-hosted rule, with the pricing line scored on Pipecat Cloud, the paid option. Its per-minute prices by agent size, transport, telephony and recording are published without a login (20). The BSD-2-Clause package installs with no card. Pipecat Cloud needs billing credentials or a trial code before sessions start (20, on the package). The framework runs with no account, on the developer's own provider keys (20).",
        "checklist": [
          "The published rubric, also on the [x402 page](https://www.anchorterminal.com/x402/).",
          "- 40, a machine payment protocol (x402, MPP or L402) on the tool's own endpoints. 10 to 30 when it covers only some endpoints or only goes through a third party, and the note says which.\n- 20, per-call or per-unit pricing published without a login. 10 for public plan-only pricing, 0 for \"contact sales\" or prices behind a login.\n- 20, a free tier or trial that doesn't need a card.\n- 20, autonomous onboarding, meaning an agent can get access without a person signing up in a browser (keyless use, x402, a programmatic key API).",
          "Payment platforms and agent wallets rarely charge for their own API over a machine protocol, so the first line has steps for them, and the highest one that applies counts. 40 when x402, MPP or L402 runs on all their own endpoints, 30 when it runs on part of their own API, 25 when their merchants can accept one, 20 for running a facilitator, 15 for paying as a buyer, and 0 when the only protocol is their own. Merchant acceptance sits above a facilitator because the platform's own customers can charge agents through it, while a facilitator settles for sellers who wire up the protocol themselves. The counter-argument (a facilitator does more for the protocol as a whole) has a point. Each note says which step applied.",
          "Open-source software you run yourself is scored on its hosted or paid option if it has one. A free, self-hosted package with nothing to buy gets 20, 20 and 20 for the last three lines, and 0 to 40 for the first only if it ships a payment protocol."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-payments"
      },
      {
        "key": "ergonomics",
        "name": "Agent ergonomics",
        "score": 72,
        "maxGain": 4.6,
        "reason": "Framework reading. `MCPClient` registers an MCP server's tools with the LLM, with `tools_filter` to allow only named tools and `tools_arguments` to fix values and hide them from the schema. The repository's streamable HTTP MCP example is about 130 lines of code, most of it the voice pipeline rather than the tool wiring (20 of 25). `tools_output_filters` trims tool results, and context summarisation and Flows context strategies limit what the model sees (15 of 20). Errors carry a category that marks permanent failures, services expose `is_usable`, `ProcessorUnusablePolicy` decides whether a bot ends, and MCP failures return to the model as text (17 of 20). Function calls cancel on interruption by default, with per-tool timeouts and opt-in asynchronous or model-cancellable calls, but there is no idempotency guidance for tools with side effects (10 of 20). A bot needs a transport, speech-to-text, LLM, text-to-speech, VAD and context aggregators wired by hand, which `pipecat init` scaffolds. Server code is Python only, with client SDKs in six languages (10 of 15).",
        "checklist": [
          "- 0 to 25, context cost. For MCP, the number and size of the tool definitions (25 for ten or fewer compact tools, 15 for 11 to 30, 5 for more than 30, plus up to 10 back for toolsets, dynamic loading or read-only subsets). For APIs, whether responses can be sized (field selection, limits, summaries).\n- 20, pagination, filtering and output-size controls.\n- 20, actionable, documented error responses, codes and messages an agent can recover from.\n- 20, idempotency or safe retries, and for MCP the `readOnlyHint` and `destructiveHint` annotations.\n- 15, sensible defaults, few required parameters, and official SDKs in at least two languages.",
          "Models are read for tool use, structured output, prompt caching, context length, batch and SDKs. Frameworks for how much code and how many defaults a tool-calling agent with MCP needs."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-ergonomics"
      },
      {
        "key": "transparency",
        "name": "Transparency \u0026 trust",
        "score": 81,
        "maxGain": 1.7,
        "reason": "BSD-2-Clause (30). The framework keeps no data. Daily's privacy policy (6 January 2025) keeps personal data while an account is open, the Pipecat Cloud security guide says session data is removed when a session ends, logs are stored by default and administrative API calls are kept a year, and Daily publishes a DPA (1 May 2025) and a sub-processor list (1 September 2026). Control-plane data is in the US whatever the agent's region (24 of 30). The changelog has Deprecated sections in most releases, and recent entries name removal in 2.0.0, with a migration guide to 1.0, but no dated removal windows (13 of 20). No telemetry found, and tracing is opt-in, though the reference page states the opposite default. The optional ai-coustics filter registers a Pipecat identifier with that vendor's SDK, which the docs don't mention (16 of 20).",
        "blend": "editorial 83, provenance 79",
        "checklist": [
          "- 0 to 30, source availability and licence clarity. 30 for open source under an OSI licence, 15 for closed with clear terms, 0 for unclear terms.\n- 0 to 30, data handling and retention statements that agree with each other (privacy policy, DPA, retention periods, subprocessors).\n- 0 to 20, a deprecation policy or notices with dates.\n- 0 to 20, telemetry disclosed with an opt-out (local software), or subprocessors and data locations disclosed (hosted).",
          "The other half of Transparency and trust is the provenance score, computed from checked facts (below). The category score is the mean of the two."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-transparency"
      },
      {
        "key": "schema",
        "name": "Schema \u0026 documentation",
        "score": 90,
        "maxGain": 1.6,
        "reason": "Framework reading. Typed Python (py.typed) with an API reference page per service, transport, frame type and utility (25). llms.txt (113 KB), Markdown copies of each page and llms-full.txt (4.2 MB) (10). Guides say when to reach for Flows, multiple LLM agents or a single pipeline, and the error-handling page says which failures are worth retrying. The OpenTelemetry reference gives `PipelineWorker` defaults of `enable_tracing` True and `enable_turn_tracking` False, the reverse of the 1.12.0 source (14 of 20). Pydantic `Settings` classes per service, a provider-neutral `FunctionSchema`, and an `ErrorCategory` enum (12 of 15). 24 example folders, from function calling and MCP to telephony, and documented error categories and Pipecat Cloud error codes (14 of 15). Dated changelog with pull-request links, towncrier fragments and a migration guide to 1.0 (15).",
        "checklist": [
          "APIs and MCP servers.",
          "- 25, a machine-readable contract (a public OpenAPI file or similar; for MCP, typed JSON Schema inputs on every tool).\n- 10, llms.txt or Markdown docs served for agents.\n- 0 to 20, descriptions that say what a tool is for, when to use it and when not to, read from the tool definitions in the source or the API reference.\n- 0 to 15, typed inputs with enums, constraints and required fields, and no free-form JSON blobs.\n- 0 to 15, examples and documented error responses.\n- 15, versioning and a public changelog.",
          "Models are read from the API reference, the OpenAPI file, llms.txt, the structured-output and tool-use docs and the model cards. Frameworks from docs a model can follow, typed interfaces, examples and the API reference."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-schema"
      },
      {
        "key": "maintenance",
        "name": "Maintenance \u0026 community",
        "score": 87,
        "maxGain": 1.1,
        "reason": "pipecat-ai 1.12.0 on PyPI on 26 September 2026, 21 days before the check (30). Eight releases in the 90 days to 17 October, 1.6.0 to 1.12.0 (20). In the past year 80 of 158 issues were closed, taking about 26 days on average, and 565 of 779 pull requests, per issues.ecosyste.ms. A needs-repro workflow labels issues, and reply times on GitHub weren't readable from our session (15 of 25). The Python framework and the JavaScript client (1.14.0) are current, with client SDKs in six languages (15). CI, coverage, a licence check, a uv lockfile and trusted publishing to PyPI are configured. No Dependabot or Renovate configuration was found, and a passing main wasn't confirmed (7 of 10).",
        "checklist": [
          "- 0 to 30, time since the last release, or the last published model or API change for a closed service. 30 within 30 days, 20 within 90, 10 within 180, 0 older.\n- 20, at least three releases or dated changelog entries in the last 90 days.\n- 0 to 25, responsiveness. Issues and pull requests answered on GitHub (the open issues and how recent the replies are). For closed services, a public changelog and a support or community channel that answers, 0 to 15.\n- 15, presence in the official MCP registry under a verified namespace (MCP servers), or current official SDKs (APIs and models).\n- 10, package health, current dependencies and CI.",
          "Models are read for deprecation notice periods and model churn rather than release counts."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-maintenance"
      }
    ],
    "provenance": [
      {
        "label": "Domain age",
        "value": "pipecat.ai, registered 2024-05-01 (2 years)",
        "points": 7,
        "max": 15
      },
      {
        "label": "security.txt",
        "value": "not found",
        "points": 0,
        "max": 10
      }
    ],
    "deductions": [
      "2025-11-10 to 2026-06-17. Three published advisories, fixed and between four and eleven months old, so 1 point each. Remote code execution through `pickle.loads()` in the optional, deprecated LivekitFrameSerializer (GHSA-c2jg-5cp7-6wc7, CVE-2025-62373, critical, fixed in 0.0.94 on 10 November 2025, advisory published 23 April 2026), arbitrary file read through the development runner's `/files` endpoint (GHSA-3363-2ph6-35wh, high, fixed in 1.2.0 on 14 May 2026) and unauthenticated call control through the runner's telephony `/ws` endpoint (GHSA-j8cv-x86q-rj85, high, fixed in 1.4.0 on 17 June 2026). The moderate GHSA-hqm2-cfx2-2vgh of 15 October 2026 covers behaviour documented as by design and isn't deducted. https://osv.dev/vulnerability/GHSA-c2jg-5cp7-6wc7, https://osv.dev/vulnerability/GHSA-3363-2ph6-35wh, https://osv.dev/vulnerability/GHSA-j8cv-x86q-rj85",
      "2026-10-16. changelog/6193.security.md says the services for OpenAI-compatible providers other than OpenAI sent `OPENAI_API_KEY` to their provider when given no `api_key`, including when `os.getenv` read an unset variable. Fixed on main on 16 October 2026 (commit 89d0e031a) and not in a release at the check, with no advisory, so 1 point. https://github.com/pipecat-ai/pipecat/blob/main/changelog/6193.security.md"
    ],
    "unchecked": [
      "Judgement call on category. Listed in voice agents rather than frameworks, because its job is running spoken conversations as a speech-to-text, LLM and text-to-speech pipeline, which is what voice agents compares, and it lands in head-to-heads with Vapi, Retell, Ultravox and ElevenLabs Agents. `voice.agent` goes first since the framework runs the conversation, and `agent.mcp-client` and `agent.multi-agent` are added from the frameworks vocabulary.",
      "Graded on the open-source framework with the framework and local-software lines. Pipecat Cloud and Pipecat Enterprise are described in details with Pipecat Cloud's published prices and are not graded, except that the Payments pricing line uses Pipecat Cloud's public prices as the paid option, as Pydantic AI's uses Logfire's.",
      "firstReleased is 2024-04-04, tag v0.0.2 (17:45 on 3 April 2024 in the tagger's time zone), when the project and package were named `dailyai`. The first `pipecat-ai` release on PyPI is 0.0.9 on 13 May 2024. unchecked: whether `dailyai` was published to PyPI, because pypi.org/project/dailyai/ answered with a bot check and wasn't read.",
      "unchecked: whether CI passes on main, open issue and pull request counts on GitHub itself, and reply times. GitHub's pages answered 403 through the session's proxy and WebFetch refused github.com. Star and open counts come from repos.ecosyste.ms (synced 10 October 2026) and issue statistics from issues.ecosyste.ms (synced 16 October 2026). The issues mirror listed some items as open that GitHub's own count suggests are closed, so its open totals weren't used.",
      "PhoneLLM Mini is recorded in details from Cloudflare's changelog and model page of 15 October 2026 and the Hugging Face card, which states BSD-2-Clause and keeps the NVIDIA Nemotron Open Model License for the base model. Not graded and not listed on its own, per the brief. The Daily blog post the card links wasn't read. The model was not downloaded or run, and serving it with vLLM needs `--trust-remote-code` per the card.",
      "The OPENAI_API_KEY fallback is deducted as an exfiltration path in the current release. The source at v1.12.0 types `api_key` as a string but passes whatever it gets to the OpenAI client, which reads `OPENAI_API_KEY` when given None, as the changelog fragment describes. We didn't run it.",
      "Lapse, corrected. We fetched pypi.org/pypi/pipecat-ai/json before reading pypi.org's robots.txt, which disallows that path. The response was deleted and nothing from it is used. The release dates come from the project page, which robots.txt allows.",
      "WebFetch refused docs.pipecat.ai and github.com (permission not answered), so docs.pipecat.ai, www.daily.co, status.pipecat.daily.co, developers.cloudflare.com, huggingface.co, pypi.org, api.osv.dev, pypistats.org, the ecosyste.ms mirrors, the npm registry, the MCP registry and rdap.org were read with curl after each robots.txt. status.pipecat.daily.co disallows /api/, so its incident history was read from /history, which covers August to October 2026 only.",
      "unchecked: Daily's Trust Center at trust.daily.co, which is drawn by script. Daily's security page states SOC 2 Type 2, not scored for a framework.",
      "Pipecat Cloud's pricing page says 'Start building for free', while the docs say sessions need billing credentials and trials come from codes. Recorded as needing billing credentials or a trial code.",
      "Disclosure. These grades are written by agents on Anthropic's Claude models. The repository carries CLAUDE.md and AGENTS.md, about 200 of its commits in the last 90 days are by claude[bot], and Pipecat publishes Claude Code skills. Graded by the same checklist, and nothing in those files was acted on.",
      "Several docs pages and the Cloudflare changelog carry text addressed to coding agents (a prompt to deploy the example, documentation-index notes). None was acted on."
    ],
    "weaknesses": [
      "No tool-approval or confirmation step, and no prompt-injection guidance, for tools that act on a caller's speech",
      "Up to 1.12.0, OpenAI-compatible provider services fall back to `OPENAI_API_KEY` when given no key. Fixed on main on 16 October, unreleased",
      "Three high or critical advisories fixed since November 2025, in an optional serialiser and the development runner",
      "Breaking changes ship in minor releases, 16 flagged in 1.8.0 and 4 in 1.10.0, although the changelog says it follows semver",
      "The OpenTelemetry reference page gives `enable_tracing` a default of True, while the 1.12.0 source defaults it to False"
    ],
    "agentNotes": [
      "Pass `api_key` explicitly to every OpenAI-compatible service such as Groq, Cerebras or OpenRouter. On 1.12.0 a missing key sends `OPENAI_API_KEY` to that provider",
      "Pass `tools_filter` to `MCPClient` so only the tools the call needs reach the model, and use `tools_arguments` for values the model must not choose",
      "Gate tools that book, pay or send in your own handler. Pipecat has no approval step, and a caller's speech goes straight to the model",
      "Keep the development runner on localhost. Its `/start`, `/daily` and dial-in webhook endpoints take requests from anyone by design",
      "Pin a minor version and read the breaking-change flags in CHANGELOG.md before upgrading, since minor releases remove and rename APIs"
    ],
    "requests": [
      {
        "text": "breaking changes only in majors",
        "reviews": 1
      },
      {
        "text": "dated removal windows",
        "reviews": 1
      },
      {
        "text": "Correct the OpenTelemetry defaults",
        "reviews": 1
      },
      {
        "text": "Document idempotency for side-effect tools",
        "reviews": 1
      },
      {
        "text": "Idempotency guidance for side-effect tools",
        "reviews": 1
      },
      {
        "text": "Published turn-handling latency",
        "reviews": 1
      },
      {
        "text": "release the key fix",
        "reviews": 1
      },
      {
        "text": "tool approval primitive",
        "reviews": 1
      }
    ],
    "recheck": "https://www.anchorterminal.com/builders/#disputes"
  },
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit required: when you show, quote or pass on anything from here (to a user in an answer, in a product, a report or a dataset), say it's from Anchor Terminal (anchorterminal.com) and link the page or API address you took it from. Reviews also credit their author.",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-10-18",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.4",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  }
}