{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/gpu-compute.package-and-deploy/",
    "json": "https://www.anchorterminal.com/niches/gpu-compute.package-and-deploy/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/gpu-compute.package-and-deploy/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/gpu-compute.package-and-deploy/index.min.md"
  },
  "markdown": "Four or more companies' docs show a full fit.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 5 | 6 | 24 | 8 | 6 |\n\n## What counts as done\n\nA build and deploy from the CLI or API completes, the worker reports ready with the engine loaded, logs of a failed start are readable, and no secret is printed in the deploy output.\n\nMeasured, for our own tests: the build and deploy complete within 20 minutes.\n\nHow agents ask for it: deploy this repo as a GPU function; push my Cog model to Replicate; build a vLLM worker for this model and deploy it\n\n## Does the job\n\nThe docs show it meets the test above, and nothing we read says otherwise.\n\n- **Baseten (Baseten Labs, Inc.)**: baseten model push builds and deploys from config.yaml; the Startup time card shows Failed and Stalled (30 minutes) outcomes with the phase reached and links to logs; team keys authenticate CI without a browser.\n- **Cerebrium (Cerebrium Inc.)**: cerebrium deploy runs headless with CEREBRIUM_SERVICE_ACCOUNT_TOKEN, builds are cached and billed at $0.08 a minute, a GPU outside the plan is rejected at deploy time, and build logs are readable through the REST API.\n- **Cloud Run GPUs (Google LLC)**: gcloud run deploy --source builds on an e2-highcpu-8 and deploys with --gpu 1 --gpu-type; a startup probe can be lengthened for slow model loads and logs are in Cloud Logging. A service account key or workload identity authenticates CI without a browser.\n- **Koyeb (Koyeb SAS)**: Services build from git or a registry image and the CLI authenticates headless with a token; dry_run validates a definition before deploying and instance statuses (Starting, Unhealthy, Error) with messages are exposed for failed starts.\n- **Northflank (Northflank Ltd)**: Services and jobs build from a linked git repository or deploy a registry image, API tokens come from roles and work headless, and build and job logs are API endpoints.\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **Beam (Smartshare, Inc.)**: beam deploy --json with BEAM_TOKEN works headless from CI and the platform API streams logs. The founder-reported issue 1828 showed --format json printing the workspace token into CI logs until PR 1886 (8 September 2026), so older CLIs still leak.\n- **Hugging Face Inference Endpoints (Hugging Face, Inc.)**: A custom container or an engine config is deployed through the UI, API or Python client with runtime logs available, but there is no build step: the image must already exist in a registry, and the FAQ covers endpoints stuck at initializing.\n- **Modal (Modal Labs, Inc.)**: modal deploy builds the image and deploys from a token in the environment; crash-looping containers appear in the dashboard. A deployed Function that crashes on start is retried indefinitely with backoff rather than stopped, and there is no REST deploy API.\n- **Replicate Deployments (Replicate, LLC)**: cog push r8.im/user/model builds and publishes with REPLICATE_API_TOKEN set headless, but a push of a granite-speech model failed with unknown blob for two months (acc-41) and Cog 0.23.1 flooded receivers with full-log webhooks that restarted the pod mid-prediction (acc-42).\n- **Runpod (Runpod, Inc.)**: Templates and endpoints are created through REST v2 or the CLI, and rp login takes RUNPOD_API_KEY headless since 2.1.0. A startup OOM in the vLLM worker exits 1 and restarts on the same footprint with no response-level sign of the eager relaunch (acc-09).\n- **Verda (DataCrunch Oy)**: Containers deploy from any registry through the REST API with Docker config.json or provider auth, but there is no build step: the image must exist first, and the ECR note warns pulls fail after 12 hours without the token permission.\n\n- not this job: CoreWeave (CoreWeave, Inc.): No build or push step; customers bring container images to Kubernetes or point Dedicated Inference at weights in object storage.\n- not this job: Crusoe Cloud (Crusoe Technologies LLC): No image build or push; Crusoe Container Registry stores images for CMK and VMs take custom images, but nothing deploys a worker from a repo.\n- not this job: GPUsMarket (Tyga.Cloud Ltd): No custom image build or push; instance types are the host's Ollama, a docker container with SSH, or a shell.\n- not this job: Hyperbolic (Hyperbolic Labs, Inc.): No image build, push or deploy step; the product is a raw VM with Docker installed.\n- not this job: Lambda Cloud (Lambda, Inc.): No image build, push or deploy; the product is a VM with Lambda Stack preinstalled.\n- not this job: Massed Compute (Massed Compute, Inc.): No image build, push or deploy; images are a fixed list of VM images.\n- not this job: Nebius AI Cloud (Nebius B.V.): Serverless AI takes a container image from a registry; there is no build or push step on the platform.\n- not this job: Thunder Compute (Thunder GPU, Inc.): Sandbox images can be prepared from a Dockerfile, but nothing deploys a worker or endpoint from them; instances are VMs with templates.\n- not this job: Together GPU Clusters (Together Computer, Inc.): No image build or push; workloads are Kubernetes manifests or Slurm scripts the user brings.\n- not this job: Vast.ai (Vast.ai Inc.): Templates reference an existing Docker image; there is no build or push step on the platform.\n\n## What stops agents finishing\n\n- engine defaults tuned for one GPU list OOM on a card not in the list, and the worker restarts on the same footprint\n- image push fails with an opaque registry error and the only workaround is retrying\n- deploy output in JSON mode contains the bearer token, so CI logs leak it\n- CLI login opens a browser, so a headless agent needs a separate service token\n\n## Around it\n\nBefore: open-weights.fine-tune-base, agent-harnesses.fix-failing-ci. After: gpu-compute.serverless-model-endpoint.\n\n## Where this comes from\n\nWeight 8 of 100 in GPU \u0026 serverless compute: 6 of 48 accounts are deploy-time failures: startup OOM with a default setting, a push that fails with unknown blob for two months, a client that ignores Retry-After, a deploy log that prints the workspace token.\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/gpu-compute.package-and-deploy.json, or the MCP tool get_use_case with id gpu-compute.package-and-deploy. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Package the model and get a deploy to come up",
        "url": ""
      }
    ],
    "description": "Package the model and get a deploy to come up: 5 companies fully do it, 6 with a limit, 24 carry the tag. Every listing in GPU \u0026 serverless compute with the reason and a source.",
    "facts": null,
    "h1": "Package the model and get a deploy to come up",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/gpu-compute.package-and-deploy/",
    "published": "",
    "section": "agents",
    "title": "Package the model and get a deploy to come up | Anchor Terminal",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/gpu-compute.package-and-deploy/"
  },
  "tokens": {
    "markdown": 1800,
    "slim": 280
  },
  "version": 1
}
