# AssemblyAI Voice Agent API vs OpenAI Realtime API

> AssemblyAI Voice Agent scores 64.4 (B) to OpenAI Realtime's 63.3 (B) for voice agents. Prices, MCP, x402, uptime and agent notes side by side.

- Canonical: https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-openai-realtime
- Markdown: https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-openai-realtime.md (~2,800 tokens)
- Slim: https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-openai-realtime.min.md (~780 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-openai-realtime.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (full text: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- Updated: 2026-10-09

AssemblyAI Voice Agent API scores 64.4 (B) on agent readiness against OpenAI Realtime API's 63.3 (B), and leads in 3 of 7 scored categories. OpenAI Realtime API leads on reliability and transparency & trust. Both do voice agents.

- AssemblyAI Voice Agent API: grade B, 64.4/100, rank #351 of 950. Markdown https://www.anchorterminal.com/tools/assemblyai-voice-agent.md · JSON https://www.anchorterminal.com/api/v1/tools/assemblyai-voice-agent.json
- OpenAI Realtime API: grade B, 63.3/100, rank #400 of 950. Markdown https://www.anchorterminal.com/tools/openai-realtime.md · JSON https://www.anchorterminal.com/api/v1/tools/openai-realtime.json
- Best conversational voice-agent APIs: https://www.anchorterminal.com/best/voice-agents/index.md
- All 91 voice agents comparisons: https://www.anchorterminal.com/compare/voice-agents/index.md

## Which one, for what

### AssemblyAI Voice Agent API (B)

Good for: A developer who wants one managed pipeline at a flat rate and is content with AssemblyAI's own speech models and voices, with an optional model of their own.

Ahead on:
- Payments & pricing, 40 against 20
- Maintenance & community, 64 against 52

Also in its favour:
- Free to start without a card

Watch for: No number is published for the concurrent-session limit that the `concurrency_exceeded` error enforces

### OpenAI Realtime API (B)

Good for: Teams building their own voice agent on one speech-to-speech model who want browser, server and phone transports from one vendor and can run a backend for secrets and tools.

Ahead on:
- Reliability, 67 against 60
- Transparency & trust, 71 against 66

Watch for: openai.com answered 403 to our reader, so the service terms, privacy policy, sub-processor list and security pages were not read.


## Score by category

| Category | Weight | AssemblyAI Voice Agent API | OpenAI Realtime API | Edge |
| --- | --- | --- | --- | --- |
| Reliability | 16% (20 this run) | 60 | 67 | OpenAI Realtime API +7 |
| Performance | 10%, pending | pending | pending | not scored in this run |
| Schema & documentation | 13% (16.2 this run) | 83 | 84 | OpenAI Realtime API +1 |
| Agent ergonomics | 13% (16.2 this run) | 72 | 76 | OpenAI Realtime API +4 |
| Security & auth | 14% (17.5 this run) | 62 | 61 | AssemblyAI Voice Agent API +1 |
| Payments & pricing | 10% (12.5 this run) | 40 | 20 | AssemblyAI Voice Agent API +20 |
| Task success | 10%, pending | pending | pending | not scored in this run |
| Maintenance & community | 7% (8.8 this run) | 64 | 52 | AssemblyAI Voice Agent API +12 |
| Transparency & trust | 7% (8.8 this run) | 66 | 71 | OpenAI Realtime API +5 |
| Negative events | ≤15 | 0 | 0 | |
| **Total** | | **64.4 · B** | **63.3 · B** | |

## Facts side by side

| Fact | AssemblyAI Voice Agent API | OpenAI Realtime API |
| --- | --- | --- |
| Kind | HTTP API | HTTP API |
| Vendor | AssemblyAI, Inc. | OpenAI |
| Hosted endpoint | `wss://agents.assemblyai.com/v1/ws` | `https://api.openai.com/v1/realtime` |
| Transports | HTTP | HTTP |
| Auth | API key | API key |
| Pricing | Pay per use | Pay per use |
| Price for voice agents | $0.075 per minute of call | not published |
| x402 | no | no |
| Licence | Proprietary service under AssemblyAI's Terms of Service. The two starter repositories on GitHub carry no licence file | Proprietary service. The service terms on openai.com were not read. The Agents SDK for TypeScript and the OpenAPI description are MIT |
| Read-only variant documented | no | no |
| llms.txt | yes | yes |
| Last release | 2026-08-26 | 2026-07-28 |
| Terms last updated | 2026-07-01 | couldn't be read |
| Privacy policy last updated | 2026-05-26 | couldn't be read |
| Customer content may train models | yes, with an opt-out | couldn't be read |
| Terms restrict automated access | yes | couldn't be read |
| Terms restrict benchmarking | yes | couldn't be read |
| Terms or service can change without notice | not found in the text | couldn't be read |
| Arbitration or class-action waiver | not found in the text | couldn't be read |

## Verdicts

**AssemblyAI Voice Agent API.** One $0.075 a minute rate covers speech-to-text, the managed model, speech output, recordings and hosting, and the socket has an AsyncAPI file, typed error codes and a 30-second resume window. No concurrent-session limit is published, the status page has no Voice Agent component, and phone calls are inbound only through a Twilio trunk the customer owns.

**OpenAI Realtime API.** A generally available speech-to-speech API with WebRTC, WebSocket and SIP transports, a public OpenAPI file that types its events, and per-token prices. Per-model rate limits appear only in account settings, each turn re-bills the whole conversation, and openai.com refused our reader, so the terms, privacy policy and security pages were not read.

## Before you call either

### AssemblyAI Voice Agent API

1. Send `session.end` before closing the socket. Without it the session stays resumable, and billed, for 30 seconds
2. Retry only `at_capacity`, `concurrency_exceeded` and `internal_error`, with backoff. Every other session error code is fatal
3. Send `tool.result` inside the `reply.done` handler, not straight after `tool.call`
4. Mint a one-time token with `GET /v1/token` for browser clients and never ship the API key to a page
5. Deduplicate webhooks on `event_id`, and verify `X-AAI-Signature` against the raw body bytes because a 401 is not retried

### OpenAI Realtime API

1. Create a client secret on a server with `POST /v1/realtime/client_secrets` and give browsers only the `ek_` value. Never ship the API key.
2. After a response with MCP calls finishes, send another `response.create`. The API does not create the follow-up response itself.
3. Set `truncation` with a `retention_ratio` below 1 and `token_limits.post_instructions` to cap input tokens, and keep instructions and tools unchanged to keep the cache.
4. On a WebSocket, stop playback on `input_audio_buffer.speech_started` and send `conversation.item.truncate` with `audio_end_ms`. WebRTC and SIP truncate on the server.
5. Use `gpt-realtime-2.1` or `gpt-realtime-2.1-mini`. `gpt-realtime` and `gpt-realtime-mini` shut down on 20 January 2027.

## Questions

### Which is better for AI agents, AssemblyAI Voice Agent API or OpenAI Realtime API?

AssemblyAI Voice Agent API scores 64.4 (B) on agent readiness against OpenAI Realtime API's 63.3 (B), and leads in 3 of 7 scored categories. OpenAI Realtime API leads on reliability and transparency & trust.

### Do AssemblyAI Voice Agent API and OpenAI Realtime API need an API key?

Both need an API key.

### Can an agent call AssemblyAI Voice Agent API and OpenAI Realtime API without installing anything?

Yes. AssemblyAI Voice Agent API has a hosted endpoint at wss://agents.assemblyai.com/v1/ws and OpenAI Realtime API at https://api.openai.com/v1/realtime.


## For agents

- This comparison as JSON: https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-openai-realtime.json, and with the fewest tokens: https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-openai-realtime.min.md
- Over MCP at https://www.anchorterminal.com/mcp (no key): `compare_tools {"a": "assemblyai-voice-agent", "b": "openai-realtime"}`. From a terminal: `anchor compare assemblyai-voice-agent openai-realtime`
- Each listing in full: https://www.anchorterminal.com/api/v1/tools/assemblyai-voice-agent.json and https://www.anchorterminal.com/api/v1/tools/openai-realtime.json

## Other comparisons with AssemblyAI Voice Agent API or OpenAI Realtime API

- [AssemblyAI Voice Agent API vs Bland AI API + MCP](https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-bland-ai.md)
- [AssemblyAI Voice Agent API vs Bolna API + MCP](https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-bolna.md)
- [AssemblyAI Voice Agent API vs Deepgram Voice Agent API](https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-deepgram-voice-agent.md)
- [AssemblyAI Voice Agent API vs ElevenLabs Agents API + MCP](https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-elevenlabs-agents.md)
- [AssemblyAI Voice Agent API vs Gemini Live API](https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-gemini-live.md)
- [AssemblyAI Voice Agent API vs Hume EVI (Empathic Voice Interface)](https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-hume-evi.md)
- [AssemblyAI Voice Agent API vs Retell AI API + MCP](https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-retell-ai.md)
- [AssemblyAI Voice Agent API vs Synthflow API + MCP](https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-synthflow.md)
- [AssemblyAI Voice Agent API vs Ultravox Realtime API](https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-ultravox.md)
- [AssemblyAI Voice Agent API vs Vapi API + MCP](https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-vapi.md)
- [AssemblyAI Voice Agent API vs Vocily AI](https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-vocily.md)
- [AssemblyAI Voice Agent API vs Vogent API](https://www.anchorterminal.com/compare/assemblyai-voice-agent-vs-vogent.md)
- [Bland AI API + MCP vs OpenAI Realtime API](https://www.anchorterminal.com/compare/bland-ai-vs-openai-realtime.md)
- [Bolna API + MCP vs OpenAI Realtime API](https://www.anchorterminal.com/compare/bolna-vs-openai-realtime.md)
- [Deepgram Voice Agent API vs OpenAI Realtime API](https://www.anchorterminal.com/compare/deepgram-voice-agent-vs-openai-realtime.md)
- [ElevenLabs Agents API + MCP vs OpenAI Realtime API](https://www.anchorterminal.com/compare/elevenlabs-agents-vs-openai-realtime.md)
- [Gemini Live API vs OpenAI Realtime API](https://www.anchorterminal.com/compare/gemini-live-vs-openai-realtime.md)
- [Hume EVI (Empathic Voice Interface) vs OpenAI Realtime API](https://www.anchorterminal.com/compare/hume-evi-vs-openai-realtime.md)
- [OpenAI Realtime API vs Retell AI API + MCP](https://www.anchorterminal.com/compare/openai-realtime-vs-retell-ai.md)
- [OpenAI Realtime API vs Synthflow API + MCP](https://www.anchorterminal.com/compare/openai-realtime-vs-synthflow.md)
- [OpenAI Realtime API vs Ultravox Realtime API](https://www.anchorterminal.com/compare/openai-realtime-vs-ultravox.md)
- [OpenAI Realtime API vs Vapi API + MCP](https://www.anchorterminal.com/compare/openai-realtime-vs-vapi.md)
- [OpenAI Realtime API vs Vocily AI](https://www.anchorterminal.com/compare/openai-realtime-vs-vocily.md)
- [OpenAI Realtime API vs Vogent API](https://www.anchorterminal.com/compare/openai-realtime-vs-vogent.md)