Niches · GPU & serverless compute
Serve my own model as an endpoint that scales to zero
No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.
What counts as done
A first request after scale to zero returns a correct answer including weight load, a warm request returns, and the endpoint bills nothing after the idle window with no replica left running.
Measured, for our own tests: the first request after scale to zero answers within 60 seconds, and a warm request within 2 seconds plus inference. No docs we read state these figures, so they don't count against a listing here.
How agents ask for it: deploy this model as an API I can call
, run Whisper or a diffusion model behind an endpoint that costs nothing when idle
, host my fine-tune so my app can call it
Does the job, with a limit 11
It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.
| Listing | Why |
|---|---|
| Baseten B Baseten Labs, Inc. | min_replica 0 scales to zero and a sync request is parked until a replica is ready (1200 s default, then 500). Replica time bills per minute from start-up, and scale_down_delay defaults to 900 s, so each burst bills 15 idle minutes. No cold-start figure is published. source |
| Beam C Smartshare, Inc. | Endpoints scale to zero and bill only while a container runs; cold start and image pull are free. The default keep_warm_seconds of 180 bills three idle minutes after every call unless lowered, the endpoint budget is capped at 600 s, and no cold-start figure is published. source |
| Cerebrium C Cerebrium Inc. | min_replicas 0 scales to zero, cooldown defaults to 60 s, and cold-start time is not billed while model initialisation is. A request queues until a container is ready. disable_auth defaults to true, so the endpoint is public until changed, and no cold-start figure is published. source |
| Cloud Run GPUs B Google LLC | GPU services scale to zero and the GPU is usable about 5 s after an instance starts, with weights loaded on top from Cloud Storage. Instance-based billing charges the whole instance lifetime, idle included, so a request every 15 minutes billed a user all day (acc-36). One L4 or RTX PRO 6000 per instance. source |
| Hugging Face Inference Endpoints B Hugging Face, Inc. | Scale to zero defaults to 1 hour idle (settable to 15 minutes) and bills nothing at zero replicas, but the docs say scaling from zero takes minutes, the proxy answers 503 unless X-Scale-Up-Timeout holds the request, and users reported endpoints stuck Initializing for a week (acc-39) and no 503 on wake (acc-40). source |
| Koyeb D Koyeb SAS | GPU services scale to zero (public preview) after a 5 minute idle period and wake on the next request with a 1 to 5 s VM cold start, plus model load. Light Sleep snapshots are CPU only, the idle period is only adjustable on Pro and above, and no model-level cold-start figure is published. source |
| Modal B Modal Labs, Inc. | Functions scale to zero by default with a 60 s scaledown_window and the web request waits through the cold start. Users measured 20 to 30 s for demucs (acc-13) and about a minute for ComfyUI (acc-12); GPU memory snapshots are alpha and one user saw 24 of 41 restores segfault (acc-18). Endpoints are public until a proxy token is added. source |
| Replicate Deployments B Replicate, LLC | POST /v1/deployments with hardware, min_instances 0 and max_instances gives a private autoscaling endpoint that scales to zero, but instances bill per second for set-up and idle time, prediction data is deleted after an hour, and no cold-start figure or scale-down window is published. source |
| Runpod C Runpod, Inc. | Workers scale to zero after an idle timeout (5 s console default) and FlashBoot caches starts, but workers bill during cold start and idle. Users measured 63 s engine-ready for Qwen3-8B (acc-11), 12.7 s to first byte for a TTS template (acc-05) and cold starts up to five minutes (acc-06). source |
| Vast.ai C Vast.ai Inc. | Serverless endpoints recruit marketplace workers from a template with min_workers, inactivity_timeout and cold_mult parameters and pay only for execution, but each new endpoint needs a benchmarking evaluation period across GPU classes, workers are peer hosts, and no cold-start figure is published. source |
| Verda C DataCrunch Oy | Serverless Containers scale to zero when idle with queue-based autoscaling and a vLLM quickstart, billing every minute a replica is up including spin-up and spin-down. The scale-down delay must be tuned by hand and no cold-start figure is published. source |
Doesn't do this job 10
Listings in the category that do something else, with the reason.
| Listing | Why |
|---|---|
| CoreWeave C CoreWeave, Inc. | Dedicated Inference autoscaling requires min at least 1 and states scale-to-zero is not supported; CKS node pools are whole 8-GPU nodes billed by the hour. Nothing bills zero when idle. source |
| Crusoe Cloud B Crusoe Technologies LLC | VMs bill per second while running with no autoscaling, and self-serve deployments reserve GPUs by the hour for Crusoe's catalogue models and LoRA adapters, not a user's container or handler. source |
| GPUsMarket E Tyga.Cloud Ltd | Rentals serve the host's Ollama or OpenAI-compatible models and bill per second until stopped; there is no custom handler or container with scale to zero. Cloud pods can be paused but not autoscaled. source |
| Hyperbolic E Hyperbolic Labs, Inc. | Hourly VM and bare-metal rentals only; no containers, endpoints, autoscaling or scale to zero. The separate Inference product serves Hyperbolic's hosted models, not the user's. source |
| Lambda Cloud C Lambda, Inc. | On-demand VMs billed by the minute until terminated; no containers, endpoints, autoscaling or scale to zero. source |
| Massed Compute E Massed Compute, Inc. | Hourly GPU VMs only; no containers, endpoints, autoscaling or scale to zero, and a stopped VM still bills the full hourly rate. source |
| Nebius AI Cloud BB Nebius B.V. | A Serverless AI endpoint is one container on one VM that runs until stopped; stopping bills nothing but there is no autoscaling or scale to zero, so an idle endpoint keeps paying GPU rates until a call stops it. source |
| Northflank C Northflank Ltd | GPU services autoscale between a minimum and maximum instance count but have no scale to zero; a service bills until paused or deleted, so an idle endpoint keeps paying. source |
| Thunder Compute D Thunder GPU, Inc. | Persistent instances billed per minute while they run and sandboxes with a fixed lifetime; no autoscaling endpoint or scale to zero. source |
| Together GPU Clusters C Together Computer, Inc. | Clusters bill hourly from creation until deleted and cannot be paused or scaled to zero; there is no endpoint product here. source |
What stops agents finishing
- cold start of a minute or more for a request-response app, so the user gives up before the first token
- endpoint stuck initialising with no logs and no 503, so the caller waits on nothing
- a warm replica or keep-warm window bills while idle, so a request every 15 minutes costs a full day
- snapshot restore that fails intermittently and falls back to a full cold start
- endpoint public by default until a token is added
Around it
Before: gpu-compute.package-and-deploy, open-weights.fine-tune-base. After: inference.tool-loop, speech-to-text.long-recording-batch, voice-agents.low-latency-pipeline-build.
Where this comes from
Weight 28 of 100 in GPU & serverless compute: 21 of 48 accounts are about an endpoint on serverless GPU: cold start, idle billing, wake from zero. The most written-about job on Hacker News and in the issue trackers read.
The same as data: /api/v1/use-cases/gpu-compute.serverless-model-endpoint.json, or the MCP tool get_use_case with id gpu-compute.serverless-model-endpoint. Other jobs in this category: /api/v1/niches?category=gpu-compute.
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.