# Assemble your own STT, LLM and TTS pipeline and tune latency and cost

> Assemble your own STT, LLM and TTS pipeline and tune latency and cost: 0 companies fully do it, 8 with a limit, 113 carry the tag. Every listing in Conversational voice agents with the reason and a source.

- Canonical: https://www.anchorterminal.com/niches/voice-agents.low-latency-pipeline-build/
- Markdown: https://www.anchorterminal.com/niches/voice-agents.low-latency-pipeline-build/index.md (~2,150 tokens)
- Slim: https://www.anchorterminal.com/niches/voice-agents.low-latency-pipeline-build/index.min.md (~330 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/niches/voice-agents.low-latency-pipeline-build/index.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-15

No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.

| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |
| --- | --- | --- | --- | --- |
| 0 | 8 | 113 | 10 | 5 |

## What counts as done

Interruption detected locally without a network round trip, and the per-minute cost broken down by provider.

Measured, for our own tests: median voice-to-voice latency under 800 ms on one clock.

How agents ask for it: build a voice agent cheaper than ElevenLabs; get voice-to-voice under 800 ms; swap the LLM without changing the rest

## Does the job, with a limit

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

- **Bolna API + MCP (Voxlabs Private Limited)**: STT, LLM and TTS are chosen per agent from many providers, the core is MIT open source, executions report time_to_first_audio and a per-component cost_breakdown. Local turn detection and one-clock latency are not documented.
- **CueLight (sshah03 (individual GitHub account, no name or organisation stated))**: Supplies the local turn-detection piece under Apache-2.0, running on a fraction of one CPU core with no framework tie. English only, self-reported benchmarks, not on PyPI; latency and cost of the rest of the pipeline are out of scope.
- **Deepgram Voice Agent API (Deepgram, Inc.)**: A managed chain where the LLM and TTS are config fields (a BYO LLM endpoint; ElevenLabs, OpenAI, Cartesia or Polly TTS) and STT is Deepgram. Turn detection is server-side and billing is a flat per-minute tier, not a per-provider split.
- **LiveKit Agents (LiveKit Incorporated)**: Swappable STT, LLM and TTS plugins, local Silero VAD and a CPU v1-mini turn model, with per-turn ttft and ttfb metrics; but usage is reported as counts not cost, and adaptive interruption runs on LiveKit Inference, billed past an allowance when self-hosted.
- **Pipecat (Daily)**: Swappable STT, LLM and TTS services with local Silero VAD and bundled Smart Turn v3 weights, plus UserBotLatencyObserver; per-service usage metrics give tokens and characters, so cost per provider must be computed by the developer.
- **Pipecat PhoneLLM (Daily)**: Built as the swappable LLM in a cascaded pipeline; Daily measured 17 ms TTFT on one B200 for Mini, which is a vendor figure, and Max needs dedicated GPUs with no per-token route.
- **Vaani Developer Platform (Vaani AI Research Private Limited (Bengaluru, India, CIN U62099KA2025PTC201212))**: STT, LLM and TTS (ears, brain, mouth) are chosen per agent from Deepgram, AssemblyAI, Google, Azure, Cartesia, ElevenLabs, OpenAI and others, BYOL is supported, and the dashboard estimates per-minute cost by stage. Hosted turn handling and no per-turn latency metrics.
- **Vapi API + MCP (Vapi Inc.)**: Transcriber, model and voice are chosen per assistant from a provider list, with per-turn TurnLatency fields and a per-call costBreakdown by stt, llm, tts and transport. Turn detection runs on Vapi's servers, not locally. An independent test measured 536 ms average, P95 up to 1.2 s.
- **Vocily AI (VOCILY AI PRIVATE LIMITED)**: Transcriber, model and voice are picked from catalogues with backup providers, but turn handling is hosted, no latency metrics are exposed, and call cost is a total with a per-minute rate rather than a per-provider split.

## Couldn't check

The docs were closed to us, or silent. These carry no mark either way.

- **Hume EVI (Empathic Voice Interface) (Hume AI, Inc.)**: EVI docs withdrawn: every EVI page, including the overview and tool-use guide, redirects to the Audio Decisions overview, and the EVI AsyncAPI file returns 404. The last assessment recorded EVI access ending 13 November 2026.

- not this job: Amazon Nova 2 Sonic (Amazon Web Services, Inc.): One speech-to-speech model with server-side endpointing (1.5 to 2 second pauses); no swappable STT or TTS and no local turn detection.
- not this job: AssemblyAI Voice Agent API (AssemblyAI, Inc.): A managed pipeline with AssemblyAI's own STT and voices; only the LLM can be swapped. Turn detection is server-side and the rate is flat, so no per-provider cost.
- not this job: Bland AI API + MCP (Intelliga Corporation): Bland runs its own STT, LLM, TTS and telephony as one bundled per-minute product; no provider swap or local turn detection.
- not this job: ElevenLabs Agents API + MCP (Eleven Labs Inc.): A hosted pipeline: the LLM or a custom LLM can be swapped, but ASR, TTS and turn-taking are ElevenLabs'. One account self-hosted instead to get about a tenth of the cost.
- not this job: fone.me (Fone): A finished consumer product; no pipeline, provider choice or latency metrics are exposed.
- not this job: Fresha AI Concierge (Fresha.com SV Ltd): A closed add-on inside Fresha; voice, accent and tone are chosen in settings but no pipeline or provider is exposed.
- not this job: Gemini Live API (Google LLC): One speech-to-speech model; STT, LLM and TTS cannot be swapped and no per-turn latency metrics are exposed.
- not this job: GPT-Live (OpenAI OpCo, LLC): The voice layer is gpt-live-1 only; the backend model can be chosen, but STT and TTS cannot be swapped and no local turn detection is offered.
- not this job: Harmony.ai (monday.com Ltd.): A closed managed product on the vendor's own model; no provider choice, latency metrics or pipeline access.
- not this job: Inworld Realtime API (Theai, Inc. dba Inworld AI): A managed STT, LLM and TTS chain where only the LLM is chosen per session; STT and TTS are Inworld's, turn detection runs server-side, and no per-turn latency metrics on one clock are exposed.
- not this job: OpenAI Realtime API (OpenAI OpCo, LLC): One speech-to-speech model; STT, LLM and TTS cannot be swapped, turn detection runs on OpenAI's server, and cost is per token rather than per provider.
- not this job: PlaceCall (VOYGR Tech, Inc.): A managed calling service; no choice of STT, LLM or TTS and no latency metrics.
- not this job: Retell AI API + MCP (Retell AI Inc.): A hosted platform: LLM and voice come from Retell's menu or a custom LLM socket, turn detection runs on Retell's servers, so the caller-built pipeline with local interruption is not offered.
- not this job: Rime Prism (Rime Labs, Inc.): One speech-to-speech model on Rime's cloud; no swappable STT, LLM or TTS and turn detection is server-side.
- not this job: Synthflow API + MCP (AgentFlow AI GmbH): A managed platform, not a pipeline you assemble: non-enterprise accounts get only Synthflow LLM, turn handling runs on Synthflow, and no per-turn latency or per-provider cost breakdown was found.
- not this job: Ultravox Realtime API (Theai, Inc. dba Inworld AI): A hosted speech-native model with a TTS voice; STT and LLM cannot be swapped, turn detection is server-side VAD, and no per-turn latency metrics were found in the schema.
- not this job: Vogent API (Monoid, Inc.): A hosted phone platform: models are chosen from Vogent's list and turn detection runs server-side, with no per-turn latency metrics or per-provider cost in the API.

## What stops agents finishing

- turn-detector model licence forbids use outside one framework
- GPU capacity needed to self-host the speech models
- latency figures measured on two clocks are wrong

## Around it

Before: inference.llm, speech.stt, speech.tts. After: voice-agents.inbound-receptionist, hosting.deploy.

## Where this comes from

Weight 10 of 100 in Conversational voice agents: 5 of 31 accounts (acc-01, acc-08, acc-18, acc-21, acc-28)

As data: https://www.anchorterminal.com/api/v1/use-cases/voice-agents.low-latency-pipeline-build.json, or the MCP tool get_use_case with id voice-agents.low-latency-pipeline-build. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp
