# Serve my own model as an endpoint that scales to zero

> Serve my own model as an endpoint that scales to zero: 0 companies fully do it, 11 with a limit, 24 carry the tag. Every listing in GPU & serverless compute with the reason and a source.

- Canonical: https://www.anchorterminal.com/niches/gpu-compute.serverless-model-endpoint/
- Markdown: https://www.anchorterminal.com/niches/gpu-compute.serverless-model-endpoint/index.md (~2,150 tokens)
- Slim: https://www.anchorterminal.com/niches/gpu-compute.serverless-model-endpoint/index.min.md (~330 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/niches/gpu-compute.serverless-model-endpoint/index.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-15

No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.

| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |
| --- | --- | --- | --- | --- |
| 0 | 11 | 24 | 28 | 21 |

## What counts as done

A first request after scale to zero returns a correct answer including weight load, a warm request returns, and the endpoint bills nothing after the idle window with no replica left running.

Measured, for our own tests: the first request after scale to zero answers within 60 seconds, and a warm request within 2 seconds plus inference.

How agents ask for it: deploy this model as an API I can call; run Whisper or a diffusion model behind an endpoint that costs nothing when idle; host my fine-tune so my app can call it

## Does the job, with a limit

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

- **Baseten (Baseten Labs, Inc.)**: min_replica 0 scales to zero and a sync request is parked until a replica is ready (1200 s default, then 500). Replica time bills per minute from start-up, and scale_down_delay defaults to 900 s, so each burst bills 15 idle minutes. No cold-start figure is published.
- **Beam (Smartshare, Inc.)**: Endpoints scale to zero and bill only while a container runs; cold start and image pull are free. The default keep_warm_seconds of 180 bills three idle minutes after every call unless lowered, the endpoint budget is capped at 600 s, and no cold-start figure is published.
- **Cerebrium (Cerebrium Inc.)**: min_replicas 0 scales to zero, cooldown defaults to 60 s, and cold-start time is not billed while model initialisation is. A request queues until a container is ready. disable_auth defaults to true, so the endpoint is public until changed, and no cold-start figure is published.
- **Cloud Run GPUs (Google LLC)**: GPU services scale to zero and the GPU is usable about 5 s after an instance starts, with weights loaded on top from Cloud Storage. Instance-based billing charges the whole instance lifetime, idle included, so a request every 15 minutes billed a user all day (acc-36). One L4 or RTX PRO 6000 per instance.
- **Hugging Face Inference Endpoints (Hugging Face, Inc.)**: Scale to zero defaults to 1 hour idle (settable to 15 minutes) and bills nothing at zero replicas, but the docs say scaling from zero takes minutes, the proxy answers 503 unless X-Scale-Up-Timeout holds the request, and users reported endpoints stuck Initializing for a week (acc-39) and no 503 on wake (acc-40).
- **Koyeb (Koyeb SAS)**: GPU services scale to zero (public preview) after a 5 minute idle period and wake on the next request with a 1 to 5 s VM cold start, plus model load. Light Sleep snapshots are CPU only, the idle period is only adjustable on Pro and above, and no model-level cold-start figure is published.
- **Modal (Modal Labs, Inc.)**: Functions scale to zero by default with a 60 s scaledown_window and the web request waits through the cold start. Users measured 20 to 30 s for demucs (acc-13) and about a minute for ComfyUI (acc-12); GPU memory snapshots are alpha and one user saw 24 of 41 restores segfault (acc-18). Endpoints are public until a proxy token is added.
- **Replicate Deployments (Replicate, LLC)**: POST /v1/deployments with hardware, min_instances 0 and max_instances gives a private autoscaling endpoint that scales to zero, but instances bill per second for set-up and idle time, prediction data is deleted after an hour, and no cold-start figure or scale-down window is published.
- **Runpod (Runpod, Inc.)**: Workers scale to zero after an idle timeout (5 s console default) and FlashBoot caches starts, but workers bill during cold start and idle. Users measured 63 s engine-ready for Qwen3-8B (acc-11), 12.7 s to first byte for a TTS template (acc-05) and cold starts up to five minutes (acc-06).
- **Vast.ai (Vast.ai Inc.)**: Serverless endpoints recruit marketplace workers from a template with min_workers, inactivity_timeout and cold_mult parameters and pay only for execution, but each new endpoint needs a benchmarking evaluation period across GPU classes, workers are peer hosts, and no cold-start figure is published.
- **Verda (DataCrunch Oy)**: Serverless Containers scale to zero when idle with queue-based autoscaling and a vLLM quickstart, billing every minute a replica is up including spin-up and spin-down. The scale-down delay must be tuned by hand and no cold-start figure is published.

- not this job: CoreWeave (CoreWeave, Inc.): Dedicated Inference autoscaling requires min at least 1 and states scale-to-zero is not supported; CKS node pools are whole 8-GPU nodes billed by the hour. Nothing bills zero when idle.
- not this job: Crusoe Cloud (Crusoe Technologies LLC): VMs bill per second while running with no autoscaling, and self-serve deployments reserve GPUs by the hour for Crusoe's catalogue models and LoRA adapters, not a user's container or handler.
- not this job: GPUsMarket (Tyga.Cloud Ltd): Rentals serve the host's Ollama or OpenAI-compatible models and bill per second until stopped; there is no custom handler or container with scale to zero. Cloud pods can be paused but not autoscaled.
- not this job: Hyperbolic (Hyperbolic Labs, Inc.): Hourly VM and bare-metal rentals only; no containers, endpoints, autoscaling or scale to zero. The separate Inference product serves Hyperbolic's hosted models, not the user's.
- not this job: Lambda Cloud (Lambda, Inc.): On-demand VMs billed by the minute until terminated; no containers, endpoints, autoscaling or scale to zero.
- not this job: Massed Compute (Massed Compute, Inc.): Hourly GPU VMs only; no containers, endpoints, autoscaling or scale to zero, and a stopped VM still bills the full hourly rate.
- not this job: Nebius AI Cloud (Nebius B.V.): A Serverless AI endpoint is one container on one VM that runs until stopped; stopping bills nothing but there is no autoscaling or scale to zero, so an idle endpoint keeps paying GPU rates until a call stops it.
- not this job: Northflank (Northflank Ltd): GPU services autoscale between a minimum and maximum instance count but have no scale to zero; a service bills until paused or deleted, so an idle endpoint keeps paying.
- not this job: Thunder Compute (Thunder GPU, Inc.): Persistent instances billed per minute while they run and sandboxes with a fixed lifetime; no autoscaling endpoint or scale to zero.
- not this job: Together GPU Clusters (Together Computer, Inc.): Clusters bill hourly from creation until deleted and cannot be paused or scaled to zero; there is no endpoint product here.

## What stops agents finishing

- cold start of a minute or more for a request-response app, so the user gives up before the first token
- endpoint stuck initialising with no logs and no 503, so the caller waits on nothing
- a warm replica or keep-warm window bills while idle, so a request every 15 minutes costs a full day
- snapshot restore that fails intermittently and falls back to a full cold start
- endpoint public by default until a token is added

## Around it

Before: gpu-compute.package-and-deploy, open-weights.fine-tune-base. After: inference.tool-loop, speech-to-text.long-recording-batch, voice-agents.low-latency-pipeline-build.

## Where this comes from

Weight 28 of 100 in GPU & serverless compute: 21 of 48 accounts are about an endpoint on serverless GPU: cold start, idle billing, wake from zero. The most written-about job on Hacker News and in the issue trackers read.

As data: https://www.anchorterminal.com/api/v1/use-cases/gpu-compute.serverless-model-endpoint.json, or the MCP tool get_use_case with id gpu-compute.serverless-model-endpoint. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp
