ElevenLabs Agents API + MCP vs OpenAI Realtime API

ElevenLabs Agents API + MCP

API keys scoped to endpoint groups, with per-key credit limits and service accounts. $0.08 a minute excludes LLM tokens and carrier minutes.

OpenAI Realtime API

A generally available speech-to-speech API with WebRTC, WebSocket and SIP transports, a public OpenAPI file that types its events, and per-token prices. Per-model rate limits appear only in account settings, each turn re-bills the whole conversation, and openai.com refused our reader, so the terms, privacy policy and security pages were not read.

ElevenLabs Agents API + MCP

1. Create a key scoped to the agents endpoints with a credit limit before handing it to an agent
2. Back off exponentially on rate_limit_exceeded, and wait for calls to finish on concurrent_limit_exceeded
3. Use signed URLs or conversation tokens for private agents instead of exposing the API key
4. Set platform_settings.privacy.retention_days where you don't need 2 years of history
5. Pick a low-latency LLM, the pipeline waits on it every turn

OpenAI Realtime API

1. Create a client secret on a server with POST /v1/realtime/client_secrets and give browsers only the ek_ value. Never ship the API key.
2. After a response with MCP calls finishes, send another response.create. The API does not create the follow-up response itself.
3. Set truncation with a retention_ratio below 1 and token_limits.post_instructions to cap input tokens, and keep instructions and tools unchanged to keep the cache.
4. On a WebSocket, stop playback on input_audio_buffer.speech_started and send conversation.item.truncate with audio_end_ms. WebRTC and SIP truncate on the server.
5. Use gpt-realtime-2.1 or gpt-realtime-2.1-mini. gpt-realtime and gpt-realtime-mini shut down on 20 January 2027.