Niches · GPU & serverless compute

Serve my own model as an endpoint that scales to zero

No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.

0companies, full fit
11companies, fit with a limit
24companies carry the tag
28share of the category's work
21first-hand accounts

What counts as done

A first request after scale to zero returns a correct answer including weight load, a warm request returns, and the endpoint bills nothing after the idle window with no replica left running.

Measured, for our own tests: the first request after scale to zero answers within 60 seconds, and a warm request within 2 seconds plus inference. No docs we read state these figures, so they don't count against a listing here.

How agents ask for it: deploy this model as an API I can call, run Whisper or a diffusion model behind an endpoint that costs nothing when idle, host my fine-tune so my app can call it

Does the job, with a limit 11

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
Baseten B
Baseten Labs, Inc.
min_replica 0 scales to zero and a sync request is parked until a replica is ready (1200 s default, then 500). Replica time bills per minute from start-up, and scale_down_delay defaults to 900 s, so each burst bills 15 idle minutes. No cold-start figure is published. source
Beam C
Smartshare, Inc.
Endpoints scale to zero and bill only while a container runs; cold start and image pull are free. The default keep_warm_seconds of 180 bills three idle minutes after every call unless lowered, the endpoint budget is capped at 600 s, and no cold-start figure is published. source
Cerebrium C
Cerebrium Inc.
min_replicas 0 scales to zero, cooldown defaults to 60 s, and cold-start time is not billed while model initialisation is. A request queues until a container is ready. disable_auth defaults to true, so the endpoint is public until changed, and no cold-start figure is published. source
Cloud Run GPUs B
Google LLC
GPU services scale to zero and the GPU is usable about 5 s after an instance starts, with weights loaded on top from Cloud Storage. Instance-based billing charges the whole instance lifetime, idle included, so a request every 15 minutes billed a user all day (acc-36). One L4 or RTX PRO 6000 per instance. source
Hugging Face Inference Endpoints B
Hugging Face, Inc.
Scale to zero defaults to 1 hour idle (settable to 15 minutes) and bills nothing at zero replicas, but the docs say scaling from zero takes minutes, the proxy answers 503 unless X-Scale-Up-Timeout holds the request, and users reported endpoints stuck Initializing for a week (acc-39) and no 503 on wake (acc-40). source
Koyeb D
Koyeb SAS
GPU services scale to zero (public preview) after a 5 minute idle period and wake on the next request with a 1 to 5 s VM cold start, plus model load. Light Sleep snapshots are CPU only, the idle period is only adjustable on Pro and above, and no model-level cold-start figure is published. source
Modal B
Modal Labs, Inc.
Functions scale to zero by default with a 60 s scaledown_window and the web request waits through the cold start. Users measured 20 to 30 s for demucs (acc-13) and about a minute for ComfyUI (acc-12); GPU memory snapshots are alpha and one user saw 24 of 41 restores segfault (acc-18). Endpoints are public until a proxy token is added. source
Replicate Deployments B
Replicate, LLC
POST /v1/deployments with hardware, min_instances 0 and max_instances gives a private autoscaling endpoint that scales to zero, but instances bill per second for set-up and idle time, prediction data is deleted after an hour, and no cold-start figure or scale-down window is published. source
Runpod C
Runpod, Inc.
Workers scale to zero after an idle timeout (5 s console default) and FlashBoot caches starts, but workers bill during cold start and idle. Users measured 63 s engine-ready for Qwen3-8B (acc-11), 12.7 s to first byte for a TTS template (acc-05) and cold starts up to five minutes (acc-06). source
Vast.ai C
Vast.ai Inc.
Serverless endpoints recruit marketplace workers from a template with min_workers, inactivity_timeout and cold_mult parameters and pay only for execution, but each new endpoint needs a benchmarking evaluation period across GPU classes, workers are peer hosts, and no cold-start figure is published. source
Verda C
DataCrunch Oy
Serverless Containers scale to zero when idle with queue-based autoscaling and a vLLM quickstart, billing every minute a replica is up including spin-up and spin-down. The scale-down delay must be tuned by hand and no cold-start figure is published. source

Doesn't do this job 10

Listings in the category that do something else, with the reason.

ListingWhy
CoreWeave C
CoreWeave, Inc.
Dedicated Inference autoscaling requires min at least 1 and states scale-to-zero is not supported; CKS node pools are whole 8-GPU nodes billed by the hour. Nothing bills zero when idle. source
Crusoe Cloud B
Crusoe Technologies LLC
VMs bill per second while running with no autoscaling, and self-serve deployments reserve GPUs by the hour for Crusoe's catalogue models and LoRA adapters, not a user's container or handler. source
GPUsMarket E
Tyga.Cloud Ltd
Rentals serve the host's Ollama or OpenAI-compatible models and bill per second until stopped; there is no custom handler or container with scale to zero. Cloud pods can be paused but not autoscaled. source
Hyperbolic E
Hyperbolic Labs, Inc.
Hourly VM and bare-metal rentals only; no containers, endpoints, autoscaling or scale to zero. The separate Inference product serves Hyperbolic's hosted models, not the user's. source
Lambda Cloud C
Lambda, Inc.
On-demand VMs billed by the minute until terminated; no containers, endpoints, autoscaling or scale to zero. source
Massed Compute E
Massed Compute, Inc.
Hourly GPU VMs only; no containers, endpoints, autoscaling or scale to zero, and a stopped VM still bills the full hourly rate. source
Nebius AI Cloud BB
Nebius B.V.
A Serverless AI endpoint is one container on one VM that runs until stopped; stopping bills nothing but there is no autoscaling or scale to zero, so an idle endpoint keeps paying GPU rates until a call stops it. source
Northflank C
Northflank Ltd
GPU services autoscale between a minimum and maximum instance count but have no scale to zero; a service bills until paused or deleted, so an idle endpoint keeps paying. source
Thunder Compute D
Thunder GPU, Inc.
Persistent instances billed per minute while they run and sandboxes with a fixed lifetime; no autoscaling endpoint or scale to zero. source
Together GPU Clusters C
Together Computer, Inc.
Clusters bill hourly from creation until deleted and cannot be paused or scaled to zero; there is no endpoint product here. source

What stops agents finishing

  • cold start of a minute or more for a request-response app, so the user gives up before the first token
  • endpoint stuck initialising with no logs and no 503, so the caller waits on nothing
  • a warm replica or keep-warm window bills while idle, so a request every 15 minutes costs a full day
  • snapshot restore that fails intermittently and falls back to a full cold start
  • endpoint public by default until a token is added

Around it

Before: gpu-compute.package-and-deploy, open-weights.fine-tune-base. After: inference.tool-loop, speech-to-text.long-recording-batch, voice-agents.low-latency-pipeline-build.

Where this comes from

Weight 28 of 100 in GPU & serverless compute: 21 of 48 accounts are about an endpoint on serverless GPU: cold start, idle billing, wake from zero. The most written-about job on Hacker News and in the issue trackers read.

The same as data: /api/v1/use-cases/gpu-compute.serverless-model-endpoint.json, or the MCP tool get_use_case with id gpu-compute.serverless-model-endpoint. Other jobs in this category: /api/v1/niches?category=gpu-compute.

Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.