{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/voice-agents.low-latency-pipeline-build/",
    "json": "https://www.anchorterminal.com/niches/voice-agents.low-latency-pipeline-build/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/voice-agents.low-latency-pipeline-build/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/voice-agents.low-latency-pipeline-build/index.min.md"
  },
  "markdown": "No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 0 | 8 | 113 | 10 | 5 |\n\n## What counts as done\n\nInterruption detected locally without a network round trip, and the per-minute cost broken down by provider.\n\nMeasured, for our own tests: median voice-to-voice latency under 800 ms on one clock.\n\nHow agents ask for it: build a voice agent cheaper than ElevenLabs; get voice-to-voice under 800 ms; swap the LLM without changing the rest\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **Bolna API + MCP (Voxlabs Private Limited)**: STT, LLM and TTS are chosen per agent from many providers, the core is MIT open source, executions report time_to_first_audio and a per-component cost_breakdown. Local turn detection and one-clock latency are not documented.\n- **CueLight (sshah03 (individual GitHub account, no name or organisation stated))**: Supplies the local turn-detection piece under Apache-2.0, running on a fraction of one CPU core with no framework tie. English only, self-reported benchmarks, not on PyPI; latency and cost of the rest of the pipeline are out of scope.\n- **Deepgram Voice Agent API (Deepgram, Inc.)**: A managed chain where the LLM and TTS are config fields (a BYO LLM endpoint; ElevenLabs, OpenAI, Cartesia or Polly TTS) and STT is Deepgram. Turn detection is server-side and billing is a flat per-minute tier, not a per-provider split.\n- **LiveKit Agents (LiveKit Incorporated)**: Swappable STT, LLM and TTS plugins, local Silero VAD and a CPU v1-mini turn model, with per-turn ttft and ttfb metrics; but usage is reported as counts not cost, and adaptive interruption runs on LiveKit Inference, billed past an allowance when self-hosted.\n- **Pipecat (Daily)**: Swappable STT, LLM and TTS services with local Silero VAD and bundled Smart Turn v3 weights, plus UserBotLatencyObserver; per-service usage metrics give tokens and characters, so cost per provider must be computed by the developer.\n- **Pipecat PhoneLLM (Daily)**: Built as the swappable LLM in a cascaded pipeline; Daily measured 17 ms TTFT on one B200 for Mini, which is a vendor figure, and Max needs dedicated GPUs with no per-token route.\n- **Vaani Developer Platform (Vaani AI Research Private Limited (Bengaluru, India, CIN U62099KA2025PTC201212))**: STT, LLM and TTS (ears, brain, mouth) are chosen per agent from Deepgram, AssemblyAI, Google, Azure, Cartesia, ElevenLabs, OpenAI and others, BYOL is supported, and the dashboard estimates per-minute cost by stage. Hosted turn handling and no per-turn latency metrics.\n- **Vapi API + MCP (Vapi Inc.)**: Transcriber, model and voice are chosen per assistant from a provider list, with per-turn TurnLatency fields and a per-call costBreakdown by stt, llm, tts and transport. Turn detection runs on Vapi's servers, not locally. An independent test measured 536 ms average, P95 up to 1.2 s.\n- **Vocily AI (VOCILY AI PRIVATE LIMITED)**: Transcriber, model and voice are picked from catalogues with backup providers, but turn handling is hosted, no latency metrics are exposed, and call cost is a total with a per-minute rate rather than a per-provider split.\n\n## Couldn't check\n\nThe docs were closed to us, or silent. These carry no mark either way.\n\n- **Hume EVI (Empathic Voice Interface) (Hume AI, Inc.)**: EVI docs withdrawn: every EVI page, including the overview and tool-use guide, redirects to the Audio Decisions overview, and the EVI AsyncAPI file returns 404. The last assessment recorded EVI access ending 13 November 2026.\n\n- not this job: Amazon Nova 2 Sonic (Amazon Web Services, Inc.): One speech-to-speech model with server-side endpointing (1.5 to 2 second pauses); no swappable STT or TTS and no local turn detection.\n- not this job: AssemblyAI Voice Agent API (AssemblyAI, Inc.): A managed pipeline with AssemblyAI's own STT and voices; only the LLM can be swapped. Turn detection is server-side and the rate is flat, so no per-provider cost.\n- not this job: Bland AI API + MCP (Intelliga Corporation): Bland runs its own STT, LLM, TTS and telephony as one bundled per-minute product; no provider swap or local turn detection.\n- not this job: ElevenLabs Agents API + MCP (Eleven Labs Inc.): A hosted pipeline: the LLM or a custom LLM can be swapped, but ASR, TTS and turn-taking are ElevenLabs'. One account self-hosted instead to get about a tenth of the cost.\n- not this job: fone.me (Fone): A finished consumer product; no pipeline, provider choice or latency metrics are exposed.\n- not this job: Fresha AI Concierge (Fresha.com SV Ltd): A closed add-on inside Fresha; voice, accent and tone are chosen in settings but no pipeline or provider is exposed.\n- not this job: Gemini Live API (Google LLC): One speech-to-speech model; STT, LLM and TTS cannot be swapped and no per-turn latency metrics are exposed.\n- not this job: GPT-Live (OpenAI OpCo, LLC): The voice layer is gpt-live-1 only; the backend model can be chosen, but STT and TTS cannot be swapped and no local turn detection is offered.\n- not this job: Harmony.ai (monday.com Ltd.): A closed managed product on the vendor's own model; no provider choice, latency metrics or pipeline access.\n- not this job: Inworld Realtime API (Theai, Inc. dba Inworld AI): A managed STT, LLM and TTS chain where only the LLM is chosen per session; STT and TTS are Inworld's, turn detection runs server-side, and no per-turn latency metrics on one clock are exposed.\n- not this job: OpenAI Realtime API (OpenAI OpCo, LLC): One speech-to-speech model; STT, LLM and TTS cannot be swapped, turn detection runs on OpenAI's server, and cost is per token rather than per provider.\n- not this job: PlaceCall (VOYGR Tech, Inc.): A managed calling service; no choice of STT, LLM or TTS and no latency metrics.\n- not this job: Retell AI API + MCP (Retell AI Inc.): A hosted platform: LLM and voice come from Retell's menu or a custom LLM socket, turn detection runs on Retell's servers, so the caller-built pipeline with local interruption is not offered.\n- not this job: Rime Prism (Rime Labs, Inc.): One speech-to-speech model on Rime's cloud; no swappable STT, LLM or TTS and turn detection is server-side.\n- not this job: Synthflow API + MCP (AgentFlow AI GmbH): A managed platform, not a pipeline you assemble: non-enterprise accounts get only Synthflow LLM, turn handling runs on Synthflow, and no per-turn latency or per-provider cost breakdown was found.\n- not this job: Ultravox Realtime API (Theai, Inc. dba Inworld AI): A hosted speech-native model with a TTS voice; STT and LLM cannot be swapped, turn detection is server-side VAD, and no per-turn latency metrics were found in the schema.\n- not this job: Vogent API (Monoid, Inc.): A hosted phone platform: models are chosen from Vogent's list and turn detection runs server-side, with no per-turn latency metrics or per-provider cost in the API.\n\n## What stops agents finishing\n\n- turn-detector model licence forbids use outside one framework\n- GPU capacity needed to self-host the speech models\n- latency figures measured on two clocks are wrong\n\n## Around it\n\nBefore: inference.llm, speech.stt, speech.tts. After: voice-agents.inbound-receptionist, hosting.deploy.\n\n## Where this comes from\n\nWeight 10 of 100 in Conversational voice agents: 5 of 31 accounts (acc-01, acc-08, acc-18, acc-21, acc-28)\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/voice-agents.low-latency-pipeline-build.json, or the MCP tool get_use_case with id voice-agents.low-latency-pipeline-build. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Assemble your own STT, LLM and TTS pipeline and tune latency and cost",
        "url": ""
      }
    ],
    "description": "Assemble your own STT, LLM and TTS pipeline and tune latency and cost: 0 companies fully do it, 8 with a limit, 113 carry the tag. Every listing in Conversational voice agents with the reason and a source.",
    "facts": null,
    "h1": "Assemble your own STT, LLM and TTS pipeline and tune latency and cost",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/voice-agents.low-latency-pipeline-build/",
    "published": "",
    "section": "agents",
    "title": "Assemble your own STT, LLM and TTS pipeline and tune latency and cost…",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/voice-agents.low-latency-pipeline-build/"
  },
  "tokens": {
    "markdown": 2150,
    "slim": 330
  },
  "version": 1
}
