Niches · GPU & serverless compute

Run an open model as my own OpenAI-compatible server for a while

Two or three companies' docs show a full fit.

3companies, full fit
17companies, fit with a limit
24companies carry the tag
14share of the category's work
7first-hand accounts

What counts as done

An OpenAI-compatible /v1/chat/completions URL answers correctly after the launch call, with model weights loaded from a template or one install step, and the machine is destroyed when done.

Measured, for our own tests: the URL answers within 15 minutes of the launch call, and the machine is destroyed by one call. No docs we read state these figures, so they don't count against a listing here.

How agents ask for it: spin up gpt-oss-20b on an H100 with vLLM and give me the URL, run Llama 4 Scout on 4 GPUs behind an API, host this model privately for my agents instead of paying per token

Does the job 3

The docs show it meets the test above, and nothing we read says otherwise.

ListingWhy
Baseten B
Baseten Labs, Inc.
Model Library presets serve named Hub models through vLLM's OpenAI-compatible server on a stated GPU, and BIS-LLM deployments expose /v1/chat/completions; deployments are deleted through the Management API. BIS-LLM itself needs support to enable. source
Hugging Face Inference Endpoints B
Hugging Face, Inc.
Pick a Hub model, a cloud instance and the vLLM, SGLang or TGI engine and the endpoint serves an OpenAI-compatible API; tensor_parallel_size defaults to the instance's GPU count, instances run to 8x H200, and the management API deletes the endpoint. source
Koyeb D
Koyeb SAS
The vLLM One-Click app deploys an OpenAI-compatible server on a GPU instance with the Hub model set in the command args, HF_TOKEN and VLLM_API_KEY as environment variables; GPU instances run from 20 GB to 8x H200, and a service is deleted by one API call. source

Does the job, with a limit 17

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
Beam C
Smartshare, Inc.
The SGLang example serves Qwen2.5-7B-Instruct on an H100 Pod with an OpenAI-compatible API, but it is a code example, not a one-step template: the agent writes app.py with the image and server command. Pods stop after 600 s idle by default. source
Cerebrium C
Cerebrium Inc.
Every endpoint can answer /chat/completions and a vLLM example exists, but the agent writes the handler and image itself; there is no one-step template taking only a model name. A100 and above need the Standard plan. source
Cloud Run GPUs B
Google LLC
An Ollama or vLLM container on an L4 serves an OpenAI-compatible API and a user found spin-up fast and the service easy (acc-35), but the agent builds the image itself and only 24 GB or 96 GB single-GPU instances exist, so models above about 70B do not fit. source
CoreWeave C
CoreWeave, Inc.
Dedicated Inference deployments run dynamo-vllm or vllm behind an OpenAI-compatible endpoint, but weights must first be uploaded to an AI Object Storage bucket (no Hub pulls), the API is v1alpha1, and access is sales-gated. source
Crusoe Cloud B
Crusoe Technologies LLC
Self-serve deployments expose an OpenAI-compatible gateway for a listed base model or a Hugging Face LoRA adapter, billed per GPU-hour; an arbitrary Hub model goes on a VM where the agent installs vLLM over SSH. source
GPUsMarket E
Tyga.Cloud Ltd
Renting a listing or deploying a pod with models ["llama3.3"] returns a per-rental token and an OpenAI-compatible /v1/chat/completions proxy; stop ends billing. Prompts pass through the host's machine and the reference says not to send confidential data. source
Hyperbolic E
Hyperbolic Labs, Inc.
Ubuntu with drivers and Docker preinstalled; a user rented 4x H100, pip-installed vLLM over SSH and served Llama 4 Scout with TP=4 (acc-44). No vLLM or Ollama template, so the engine and weights are installed by hand, and the VM bills hourly until terminated. source
Lambda Cloud C
Lambda, Inc.
Lambda Stack ships CUDA, PyTorch and Docker and guides cover vLLM and Ollama, but the agent installs the engine and weights over SSH; there is no serving template and the VM bills per minute until terminated. source
Massed Compute E
Massed Compute, Inc.
A VM with a chosen image and SSH can run vLLM or Ollama, but no engine template or OpenAI-compatible preset appears in the API or MCP tool list; the agent installs the server itself and the VM bills until terminated. source
Modal B
Modal Labs, Inc.
The vLLM example serves a Hub model behind /v1/chat/completions on H200s with scaledown_window set to 15 minutes, but it is a Python app the agent copies and deploys, not a one-step template; cold starts are tens of seconds to minutes, about ten from cache. source
Nebius AI Cloud BB
Nebius B.V.
A Serverless AI endpoint runs a vLLM container on a gpu-h100-sxm preset behind a managed HTTPS URL with a token, and stop or delete ends charges; the agent supplies the container and model settings, and there is no engine template taking only a model name. source
Northflank C
Northflank Ltd
The host-a-model guide deploys any serving image (vLLM, Ollama) as a GPU service with ports, secrets and health checks, but the agent supplies the image and command; no engine template takes a model name. source
Runpod C
Runpod, Inc.
The vLLM worker serves a Hub model with OpenAI-compatible responses from a template, but its hub defaults (MAX_NUM_SEQS 256) made an 8B model OOM on an RTX 5090, relaunch in eager mode at half throughput and restart on the same footprint (acc-09). source
Thunder Compute D
Thunder GPU, Inc.
tnr create --gpu a100xl --template ollama gives an instance with Ollama preinstalled and guides run DeepSeek R1 70B and GPT-OSS 120B in minutes, but the model is pulled by hand over SSH and no vLLM template or OpenAI-compatible URL is documented. source
Together GPU Clusters C
Together Computer, Inc.
A Kubernetes cluster can run vLLM, but the agent writes the manifest, the smallest unit is 8 GPUs, and provisioning takes 10 minutes to an hour; no engine template or OpenAI endpoint is provided by the product. source
Vast.ai C
Vast.ai Inc.
A recommended vLLM template plus a search filter gets a model server on a rented H100, which a user did to measure gpt-oss-20b refusal rates (acc-21). Prompts pass through the host's machine unless Secure Cloud is filtered, and the instance bills until destroyed. source
Verda C
DataCrunch Oy
The Serverless Containers vLLM quickstart serves a Hub model behind an OpenAI-compatible endpoint from a container image on an L40S; on a VM the images are Ubuntu or JupyterLab and the engine is installed by hand. Deleting the deployment or instance stops charges. source

Doesn't do this job 1

Listings in the category that do something else, with the reason.

ListingWhy
Replicate Deployments B
Replicate, LLC
A deployment serves a Cog model behind Replicate's prediction API; no OpenAI-compatible /v1/chat/completions route or vLLM template appears in the docs index, so the success criterion cannot be met. source

What stops agents finishing

  • no template for the engine, so the agent must install CUDA, vLLM and the weights over SSH
  • peer host hardware with firmware-capped clocks or that cannot pull images
  • prompts and outputs pass through an individual host's machine, so private data is out
  • an idle machine bills by the hour until destroyed, with no scale to zero

Around it

Before: gpu-compute.find-and-provision, open-weights.big-model-on-big-box, open-weights.tool-calling-assistant. After: inference.tool-loop, inference.harness-compat, agent-harnesses.local-model, decision-models.run-on-own-hardware.

Where this comes from

Weight 14 of 100 in GPU & serverless compute: 7 of 48 accounts stand up vLLM or Ollama on a rented machine to serve a model to their own tools; two of them left for local hardware on cost.

The same as data: /api/v1/use-cases/gpu-compute.self-hosted-llm-server.json, or the MCP tool get_use_case with id gpu-compute.self-hosted-llm-server. Other jobs in this category: /api/v1/niches?category=gpu-compute.

Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.