Baseten vs Beam

Baseten

Team API keys scoped to inference-only, metrics-only or a single environment or model, plus a Viewer role since 1 September 2026. H100 at $6.50 and A100 at $4.00 an hour, and start-up and idle replica time are billed.

Beam

Per-millisecond billing with cold starts and image pulls free, H100 PCIe at $3.50 and RTX 4090 at $0.69 an hour. No published request rate limits, 429 handling or SLA.

Baseten

1. Create a team key with inference-only permission for calling models and keep full-access keys out of the agent
2. Sleep for retry_after seconds on a 429 from api.baseten.co; the activate and deactivate endpoints allow 20 calls a minute
3. Retry 429, 503 and 529 with backoff, but treat 500 as a bug in your model code
4. Set scale_down_delay below the 900-second default or every burst bills 15 idle minutes
5. Send payloads over 256 KiB to /predict, not /async_predict, unless support has raised the async limit

Beam

1. Check the response body for ok: false on gateway calls; a failure can arrive as HTTP 200
2. Don't pipe beam deploy --format json output into CI logs, since it contains the workspace token
3. Route anything over 180 seconds to a task queue and poll the task instead of holding the endpoint request
4. Set keep_warm_seconds deliberately; the 180-second endpoint default bills three minutes of GPU after every call
5. Pass gpu=["RTX4090", "A10G"] so a job still schedules when one type is out