Baseten vs Modal

Baseten

Team API keys scoped to inference-only, metrics-only or a single environment or model, plus a Viewer role since 1 September 2026. H100 at $6.50 and A100 at $4.00 an hour, and start-up and idle replica time are billed.

Modal

Scale to zero by default, per-second billing and about one-second container boots. No REST API or OpenAPI spec for deploying or invoking Functions.

Baseten

1. Create a team key with inference-only permission for calling models and keep full-access keys out of the agent
2. Sleep for retry_after seconds on a 429 from api.baseten.co; the activate and deactivate endpoints allow 20 calls a minute
3. Retry 429, 503 and 529 with backoff, but treat 500 as a bug in your model code
4. Set scale_down_delay below the 900-second default or every burst bills 15 idle minutes
5. Send payloads over 256 KiB to /predict, not /async_predict, unless support has raised the async limit

Modal

1. Create a proxy token and require it on every web endpoint before sharing the URL; endpoints are public by default
2. Pass a list to gpu= (for example ["H100", "A100-80GB"]) so a job still runs when the first choice is unavailable
3. Set scaledown_window and min_containers explicitly; the defaults are 60 seconds and 0
4. Use .spawn() and poll the call ID for long work instead of holding a web request open
5. Keep web endpoint traffic under 200 requests a second or ask Modal to raise the limit