Niches · Conversational voice agents

Assemble your own STT, LLM and TTS pipeline and tune latency and cost

No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.

0companies, full fit
8companies, fit with a limit
113companies carry the tag
10share of the category's work
5first-hand accounts

What counts as done

Interruption detected locally without a network round trip, and the per-minute cost broken down by provider.

Measured, for our own tests: median voice-to-voice latency under 800 ms on one clock. No docs we read state these figures, so they don't count against a listing here.

How agents ask for it: build a voice agent cheaper than ElevenLabs, get voice-to-voice under 800 ms, swap the LLM without changing the rest

Does the job, with a limit 9

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
Bolna API + MCP D
Voxlabs Private Limited
STT, LLM and TTS are chosen per agent from many providers, the core is MIT open source, executions report time_to_first_audio and a per-component cost_breakdown. Local turn detection and one-clock latency are not documented. source
CueLight D
sshah03 (individual GitHub account, no name or organisation stated)
Supplies the local turn-detection piece under Apache-2.0, running on a fraction of one CPU core with no framework tie. English only, self-reported benchmarks, not on PyPI; latency and cost of the rest of the pipeline are out of scope. source
Deepgram Voice Agent API BB
Deepgram, Inc.
A managed chain where the LLM and TTS are config fields (a BYO LLM endpoint; ElevenLabs, OpenAI, Cartesia or Polly TTS) and STT is Deepgram. Turn detection is server-side and billing is a flat per-minute tier, not a per-provider split. source
LiveKit Agents BB
LiveKit Incorporated
Swappable STT, LLM and TTS plugins, local Silero VAD and a CPU v1-mini turn model, with per-turn ttft and ttfb metrics; but usage is reported as counts not cost, and adaptive interruption runs on LiveKit Inference, billed past an allowance when self-hosted. source
Pipecat B
Daily
Swappable STT, LLM and TTS services with local Silero VAD and bundled Smart Turn v3 weights, plus UserBotLatencyObserver; per-service usage metrics give tokens and characters, so cost per provider must be computed by the developer. source
Pipecat PhoneLLM B
Daily
Built as the swappable LLM in a cascaded pipeline; Daily measured 17 ms TTFT on one B200 for Mini, which is a vendor figure, and Max needs dedicated GPUs with no per-token route. source
Vaani Developer Platform E
Vaani AI Research Private Limited (Bengaluru, India, CIN U62099KA2025PTC201212)
STT, LLM and TTS (ears, brain, mouth) are chosen per agent from Deepgram, AssemblyAI, Google, Azure, Cartesia, ElevenLabs, OpenAI and others, BYOL is supported, and the dashboard estimates per-minute cost by stage. Hosted turn handling and no per-turn latency metrics. source
Vapi API + MCP B
Vapi Inc.
Transcriber, model and voice are chosen per assistant from a provider list, with per-turn TurnLatency fields and a per-call costBreakdown by stt, llm, tts and transport. Turn detection runs on Vapi's servers, not locally. An independent test measured 536 ms average, P95 up to 1.2 s. source
Vocily AI C
VOCILY AI PRIVATE LIMITED
Transcriber, model and voice are picked from catalogues with backup providers, but turn handling is hosted, no latency metrics are exposed, and call cost is a total with a per-minute rate rather than a per-provider split. source

Couldn't check 1

The docs were closed to us, or silent. These carry no mark either way.

ListingWhy
Hume EVI (Empathic Voice Interface) C
Hume AI, Inc.
EVI docs withdrawn: every EVI page, including the overview and tool-use guide, redirects to the Audio Decisions overview, and the EVI AsyncAPI file returns 404. The last assessment recorded EVI access ending 13 November 2026. source

Doesn't do this job 17

Listings in the category that do something else, with the reason.

ListingWhy
Amazon Nova 2 Sonic B
Amazon Web Services, Inc.
One speech-to-speech model with server-side endpointing (1.5 to 2 second pauses); no swappable STT or TTS and no local turn detection. source
AssemblyAI Voice Agent API C
AssemblyAI, Inc.
A managed pipeline with AssemblyAI's own STT and voices; only the LLM can be swapped. Turn detection is server-side and the rate is flat, so no per-provider cost. source
Bland AI API + MCP B
Intelliga Corporation
Bland runs its own STT, LLM, TTS and telephony as one bundled per-minute product; no provider swap or local turn detection. source
ElevenLabs Agents API + MCP BB
Eleven Labs Inc.
A hosted pipeline: the LLM or a custom LLM can be swapped, but ASR, TTS and turn-taking are ElevenLabs'. One account self-hosted instead to get about a tenth of the cost. source
fone.me F
Fone
A finished consumer product; no pipeline, provider choice or latency metrics are exposed. source
Fresha AI Concierge F
Fresha.com SV Ltd
A closed add-on inside Fresha; voice, accent and tone are chosen in settings but no pipeline or provider is exposed. source
Gemini Live API C
Google LLC
One speech-to-speech model; STT, LLM and TTS cannot be swapped and no per-turn latency metrics are exposed. source
GPT-Live BB
OpenAI OpCo, LLC
The voice layer is gpt-live-1 only; the backend model can be chosen, but STT and TTS cannot be swapped and no local turn detection is offered. source
Harmony.ai E
monday.com Ltd.
A closed managed product on the vendor's own model; no provider choice, latency metrics or pipeline access. source
Inworld Realtime API B
Theai, Inc. dba Inworld AI
A managed STT, LLM and TTS chain where only the LLM is chosen per session; STT and TTS are Inworld's, turn detection runs server-side, and no per-turn latency metrics on one clock are exposed. source
OpenAI Realtime API BB
OpenAI OpCo, LLC
One speech-to-speech model; STT, LLM and TTS cannot be swapped, turn detection runs on OpenAI's server, and cost is per token rather than per provider. source
PlaceCall C
VOYGR Tech, Inc.
A managed calling service; no choice of STT, LLM or TTS and no latency metrics. source
Retell AI API + MCP B
Retell AI Inc.
A hosted platform: LLM and voice come from Retell's menu or a custom LLM socket, turn detection runs on Retell's servers, so the caller-built pipeline with local interruption is not offered. source
Rime Prism B
Rime Labs, Inc.
One speech-to-speech model on Rime's cloud; no swappable STT, LLM or TTS and turn detection is server-side. source
Synthflow API + MCP D
AgentFlow AI GmbH
A managed platform, not a pipeline you assemble: non-enterprise accounts get only Synthflow LLM, turn handling runs on Synthflow, and no per-turn latency or per-provider cost breakdown was found. source
Ultravox Realtime API C
Theai, Inc. dba Inworld AI
A hosted speech-native model with a TTS voice; STT and LLM cannot be swapped, turn detection is server-side VAD, and no per-turn latency metrics were found in the schema. source
Vogent API E
Monoid, Inc.
A hosted phone platform: models are chosen from Vogent's list and turn detection runs server-side, with no per-turn latency metrics or per-provider cost in the API. source

What stops agents finishing

  • turn-detector model licence forbids use outside one framework
  • GPU capacity needed to self-host the speech models
  • latency figures measured on two clocks are wrong

Around it

Before: inference.llm, speech.stt, speech.tts. After: voice-agents.inbound-receptionist, hosting.deploy.

Where this comes from

Weight 10 of 100 in Conversational voice agents: 5 of 31 accounts (acc-01, acc-08, acc-18, acc-21, acc-28)

The same as data: /api/v1/use-cases/voice-agents.low-latency-pipeline-build.json, or the MCP tool get_use_case with id voice-agents.low-latency-pipeline-build. Other jobs in this category: /api/v1/niches?category=voice-agents.

Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.