Niches · Conversational voice agents
Assemble your own STT, LLM and TTS pipeline and tune latency and cost
No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.
What counts as done
Interruption detected locally without a network round trip, and the per-minute cost broken down by provider.
Measured, for our own tests: median voice-to-voice latency under 800 ms on one clock. No docs we read state these figures, so they don't count against a listing here.
How agents ask for it: build a voice agent cheaper than ElevenLabs
, get voice-to-voice under 800 ms
, swap the LLM without changing the rest
Does the job, with a limit 9
It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.
| Listing | Why |
|---|---|
| Bolna API + MCP D Voxlabs Private Limited | STT, LLM and TTS are chosen per agent from many providers, the core is MIT open source, executions report time_to_first_audio and a per-component cost_breakdown. Local turn detection and one-clock latency are not documented. source |
| CueLight D sshah03 (individual GitHub account, no name or organisation stated) | Supplies the local turn-detection piece under Apache-2.0, running on a fraction of one CPU core with no framework tie. English only, self-reported benchmarks, not on PyPI; latency and cost of the rest of the pipeline are out of scope. source |
| Deepgram Voice Agent API BB Deepgram, Inc. | A managed chain where the LLM and TTS are config fields (a BYO LLM endpoint; ElevenLabs, OpenAI, Cartesia or Polly TTS) and STT is Deepgram. Turn detection is server-side and billing is a flat per-minute tier, not a per-provider split. source |
| LiveKit Agents BB LiveKit Incorporated | Swappable STT, LLM and TTS plugins, local Silero VAD and a CPU v1-mini turn model, with per-turn ttft and ttfb metrics; but usage is reported as counts not cost, and adaptive interruption runs on LiveKit Inference, billed past an allowance when self-hosted. source |
| Pipecat B Daily | Swappable STT, LLM and TTS services with local Silero VAD and bundled Smart Turn v3 weights, plus UserBotLatencyObserver; per-service usage metrics give tokens and characters, so cost per provider must be computed by the developer. source |
| Pipecat PhoneLLM B Daily | Built as the swappable LLM in a cascaded pipeline; Daily measured 17 ms TTFT on one B200 for Mini, which is a vendor figure, and Max needs dedicated GPUs with no per-token route. source |
| Vaani Developer Platform E Vaani AI Research Private Limited (Bengaluru, India, CIN U62099KA2025PTC201212) | STT, LLM and TTS (ears, brain, mouth) are chosen per agent from Deepgram, AssemblyAI, Google, Azure, Cartesia, ElevenLabs, OpenAI and others, BYOL is supported, and the dashboard estimates per-minute cost by stage. Hosted turn handling and no per-turn latency metrics. source |
| Vapi API + MCP B Vapi Inc. | Transcriber, model and voice are chosen per assistant from a provider list, with per-turn TurnLatency fields and a per-call costBreakdown by stt, llm, tts and transport. Turn detection runs on Vapi's servers, not locally. An independent test measured 536 ms average, P95 up to 1.2 s. source |
| Vocily AI C VOCILY AI PRIVATE LIMITED | Transcriber, model and voice are picked from catalogues with backup providers, but turn handling is hosted, no latency metrics are exposed, and call cost is a total with a per-minute rate rather than a per-provider split. source |
Couldn't check 1
The docs were closed to us, or silent. These carry no mark either way.
| Listing | Why |
|---|---|
| Hume EVI (Empathic Voice Interface) C Hume AI, Inc. | EVI docs withdrawn: every EVI page, including the overview and tool-use guide, redirects to the Audio Decisions overview, and the EVI AsyncAPI file returns 404. The last assessment recorded EVI access ending 13 November 2026. source |
Doesn't do this job 17
Listings in the category that do something else, with the reason.
| Listing | Why |
|---|---|
| Amazon Nova 2 Sonic B Amazon Web Services, Inc. | One speech-to-speech model with server-side endpointing (1.5 to 2 second pauses); no swappable STT or TTS and no local turn detection. source |
| AssemblyAI Voice Agent API C AssemblyAI, Inc. | A managed pipeline with AssemblyAI's own STT and voices; only the LLM can be swapped. Turn detection is server-side and the rate is flat, so no per-provider cost. source |
| Bland AI API + MCP B Intelliga Corporation | Bland runs its own STT, LLM, TTS and telephony as one bundled per-minute product; no provider swap or local turn detection. source |
| ElevenLabs Agents API + MCP BB Eleven Labs Inc. | A hosted pipeline: the LLM or a custom LLM can be swapped, but ASR, TTS and turn-taking are ElevenLabs'. One account self-hosted instead to get about a tenth of the cost. source |
| fone.me F Fone | A finished consumer product; no pipeline, provider choice or latency metrics are exposed. source |
| Fresha AI Concierge F Fresha.com SV Ltd | A closed add-on inside Fresha; voice, accent and tone are chosen in settings but no pipeline or provider is exposed. source |
| Gemini Live API C Google LLC | One speech-to-speech model; STT, LLM and TTS cannot be swapped and no per-turn latency metrics are exposed. source |
| GPT-Live BB OpenAI OpCo, LLC | The voice layer is gpt-live-1 only; the backend model can be chosen, but STT and TTS cannot be swapped and no local turn detection is offered. source |
| Harmony.ai E monday.com Ltd. | A closed managed product on the vendor's own model; no provider choice, latency metrics or pipeline access. source |
| Inworld Realtime API B Theai, Inc. dba Inworld AI | A managed STT, LLM and TTS chain where only the LLM is chosen per session; STT and TTS are Inworld's, turn detection runs server-side, and no per-turn latency metrics on one clock are exposed. source |
| OpenAI Realtime API BB OpenAI OpCo, LLC | One speech-to-speech model; STT, LLM and TTS cannot be swapped, turn detection runs on OpenAI's server, and cost is per token rather than per provider. source |
| PlaceCall C VOYGR Tech, Inc. | A managed calling service; no choice of STT, LLM or TTS and no latency metrics. source |
| Retell AI API + MCP B Retell AI Inc. | A hosted platform: LLM and voice come from Retell's menu or a custom LLM socket, turn detection runs on Retell's servers, so the caller-built pipeline with local interruption is not offered. source |
| Rime Prism B Rime Labs, Inc. | One speech-to-speech model on Rime's cloud; no swappable STT, LLM or TTS and turn detection is server-side. source |
| Synthflow API + MCP D AgentFlow AI GmbH | A managed platform, not a pipeline you assemble: non-enterprise accounts get only Synthflow LLM, turn handling runs on Synthflow, and no per-turn latency or per-provider cost breakdown was found. source |
| Ultravox Realtime API C Theai, Inc. dba Inworld AI | A hosted speech-native model with a TTS voice; STT and LLM cannot be swapped, turn detection is server-side VAD, and no per-turn latency metrics were found in the schema. source |
| Vogent API E Monoid, Inc. | A hosted phone platform: models are chosen from Vogent's list and turn detection runs server-side, with no per-turn latency metrics or per-provider cost in the API. source |
What stops agents finishing
- turn-detector model licence forbids use outside one framework
- GPU capacity needed to self-host the speech models
- latency figures measured on two clocks are wrong
Around it
Before: inference.llm, speech.stt, speech.tts. After: voice-agents.inbound-receptionist, hosting.deploy.
Where this comes from
Weight 10 of 100 in Conversational voice agents: 5 of 31 accounts (acc-01, acc-08, acc-18, acc-21, acc-28)
The same as data: /api/v1/use-cases/voice-agents.low-latency-pipeline-build.json, or the MCP tool get_use_case with id voice-agents.low-latency-pipeline-build. Other jobs in this category: /api/v1/niches?category=voice-agents.
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.