ElevenLabs Agents API + MCP vs Ultravox Realtime API

ElevenLabs Agents API + MCP

API keys scoped to endpoint groups, with per-key credit limits and service accounts. $0.08 a minute excludes LLM tokens and carrier minutes.

Ultravox Realtime API

Flat $0.05 a minute including model and built-in voices. No MCP server.

ElevenLabs Agents API + MCP

1. Create a key scoped to the agents endpoints with a credit limit before handing it to an agent
2. Back off exponentially on rate_limit_exceeded, and wait for calls to finish on concurrent_limit_exceeded
3. Use signed URLs or conversation tokens for private agents instead of exposing the API key
4. Set platform_settings.privacy.retention_days where you don't need 2 years of history
5. Pick a low-latency LLM, the pipeline waits on it every turn

Ultravox Realtime API

1. Honour Retry-After on 429 and 503 when creating calls
2. Pin the model string (ultravox-v0.7 or ultravox-v0.6) rather than relying on the default
3. Hand clients the joinUrl, never the API key
4. Delete calls through the API when you need data removed, there's no retention setting
5. Use call stages for multi-step flows instead of one long prompt