AssemblyAI Voice Agent API vs ElevenLabs Agents API + MCP

AssemblyAI Voice Agent API

One $0.075 a minute rate covers speech-to-text, the managed model, speech output, recordings and hosting, and the socket has an AsyncAPI file, typed error codes and a 30-second resume window. No concurrent-session limit is published, the status page has no Voice Agent component, and phone calls are inbound only through a Twilio trunk the customer owns.

ElevenLabs Agents API + MCP

API keys scoped to endpoint groups, with per-key credit limits and service accounts. $0.08 a minute excludes LLM tokens and carrier minutes.

AssemblyAI Voice Agent API

1. Send session.end before closing the socket. Without it the session stays resumable, and billed, for 30 seconds
2. Retry only at_capacity, concurrency_exceeded and internal_error, with backoff. Every other session error code is fatal
3. Send tool.result inside the reply.done handler, not straight after tool.call
4. Mint a one-time token with GET /v1/token for browser clients and never ship the API key to a page
5. Deduplicate webhooks on event_id, and verify X-AAI-Signature against the raw body bytes because a 401 is not retried

ElevenLabs Agents API + MCP

1. Create a key scoped to the agents endpoints with a credit limit before handing it to an agent
2. Back off exponentially on rate_limit_exceeded, and wait for calls to finish on concurrent_limit_exceeded
3. Use signed URLs or conversation tokens for private agents instead of exposing the API key
4. Set platform_settings.privacy.retention_days where you don't need 2 years of history
5. Pick a low-latency LLM, the pipeline waits on it every turn