Baseten vs Replicate Deployments

Baseten

Team API keys scoped to inference-only, metrics-only or a single environment or model, plus a Viewer role since 1 September 2026. H100 at $6.50 and A100 at $4.00 an hour, and start-up and idle replica time are billed.

Replicate Deployments

OpenAPI file, llms.txt and an MCP server with a two-tool code mode. Private instances bill set-up and idle time, H100 at $5.49 an hour.

Baseten

1. Create a team key with inference-only permission for calling models and keep full-access keys out of the agent
2. Sleep for retry_after seconds on a 429 from api.baseten.co; the activate and deactivate endpoints allow 20 calls a minute
3. Retry 429, 503 and 529 with backoff, but treat 500 as a bug in your model code
4. Set scale_down_delay below the 900-second default or every burst bills 15 idle minutes
5. Send payloads over 256 KiB to /predict, not /async_predict, unless support has raised the async limit

Replicate Deployments

1. List GET /v1/hardware first and use the returned sku in the deployment body
2. Set min_instances to 0 for bursty work; a warm H100 bills $5.49 an hour whether called or not
3. Send Prefer: wait on deployment predictions to block instead of polling
4. Copy outputs within an hour; API prediction data is deleted after that
5. Wait for the reset time in the 429 body before retrying; prediction creates cap at 600 a minute