Niches · GPU & serverless compute
Run an open model as my own OpenAI-compatible server for a while
Two or three companies' docs show a full fit.
What counts as done
An OpenAI-compatible /v1/chat/completions URL answers correctly after the launch call, with model weights loaded from a template or one install step, and the machine is destroyed when done.
Measured, for our own tests: the URL answers within 15 minutes of the launch call, and the machine is destroyed by one call. No docs we read state these figures, so they don't count against a listing here.
How agents ask for it: spin up gpt-oss-20b on an H100 with vLLM and give me the URL
, run Llama 4 Scout on 4 GPUs behind an API
, host this model privately for my agents instead of paying per token
Does the job 3
The docs show it meets the test above, and nothing we read says otherwise.
| Listing | Why |
|---|---|
| Baseten B Baseten Labs, Inc. | Model Library presets serve named Hub models through vLLM's OpenAI-compatible server on a stated GPU, and BIS-LLM deployments expose /v1/chat/completions; deployments are deleted through the Management API. BIS-LLM itself needs support to enable. source |
| Hugging Face Inference Endpoints B Hugging Face, Inc. | Pick a Hub model, a cloud instance and the vLLM, SGLang or TGI engine and the endpoint serves an OpenAI-compatible API; tensor_parallel_size defaults to the instance's GPU count, instances run to 8x H200, and the management API deletes the endpoint. source |
| Koyeb D Koyeb SAS | The vLLM One-Click app deploys an OpenAI-compatible server on a GPU instance with the Hub model set in the command args, HF_TOKEN and VLLM_API_KEY as environment variables; GPU instances run from 20 GB to 8x H200, and a service is deleted by one API call. source |
Does the job, with a limit 17
It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.
| Listing | Why |
|---|---|
| Beam C Smartshare, Inc. | The SGLang example serves Qwen2.5-7B-Instruct on an H100 Pod with an OpenAI-compatible API, but it is a code example, not a one-step template: the agent writes app.py with the image and server command. Pods stop after 600 s idle by default. source |
| Cerebrium C Cerebrium Inc. | Every endpoint can answer /chat/completions and a vLLM example exists, but the agent writes the handler and image itself; there is no one-step template taking only a model name. A100 and above need the Standard plan. source |
| Cloud Run GPUs B Google LLC | An Ollama or vLLM container on an L4 serves an OpenAI-compatible API and a user found spin-up fast and the service easy (acc-35), but the agent builds the image itself and only 24 GB or 96 GB single-GPU instances exist, so models above about 70B do not fit. source |
| CoreWeave C CoreWeave, Inc. | Dedicated Inference deployments run dynamo-vllm or vllm behind an OpenAI-compatible endpoint, but weights must first be uploaded to an AI Object Storage bucket (no Hub pulls), the API is v1alpha1, and access is sales-gated. source |
| Crusoe Cloud B Crusoe Technologies LLC | Self-serve deployments expose an OpenAI-compatible gateway for a listed base model or a Hugging Face LoRA adapter, billed per GPU-hour; an arbitrary Hub model goes on a VM where the agent installs vLLM over SSH. source |
| GPUsMarket E Tyga.Cloud Ltd | Renting a listing or deploying a pod with models ["llama3.3"] returns a per-rental token and an OpenAI-compatible /v1/chat/completions proxy; stop ends billing. Prompts pass through the host's machine and the reference says not to send confidential data. source |
| Hyperbolic E Hyperbolic Labs, Inc. | Ubuntu with drivers and Docker preinstalled; a user rented 4x H100, pip-installed vLLM over SSH and served Llama 4 Scout with TP=4 (acc-44). No vLLM or Ollama template, so the engine and weights are installed by hand, and the VM bills hourly until terminated. source |
| Lambda Cloud C Lambda, Inc. | Lambda Stack ships CUDA, PyTorch and Docker and guides cover vLLM and Ollama, but the agent installs the engine and weights over SSH; there is no serving template and the VM bills per minute until terminated. source |
| Massed Compute E Massed Compute, Inc. | A VM with a chosen image and SSH can run vLLM or Ollama, but no engine template or OpenAI-compatible preset appears in the API or MCP tool list; the agent installs the server itself and the VM bills until terminated. source |
| Modal B Modal Labs, Inc. | The vLLM example serves a Hub model behind /v1/chat/completions on H200s with scaledown_window set to 15 minutes, but it is a Python app the agent copies and deploys, not a one-step template; cold starts are tens of seconds to minutes, about ten from cache. source |
| Nebius AI Cloud BB Nebius B.V. | A Serverless AI endpoint runs a vLLM container on a gpu-h100-sxm preset behind a managed HTTPS URL with a token, and stop or delete ends charges; the agent supplies the container and model settings, and there is no engine template taking only a model name. source |
| Northflank C Northflank Ltd | The host-a-model guide deploys any serving image (vLLM, Ollama) as a GPU service with ports, secrets and health checks, but the agent supplies the image and command; no engine template takes a model name. source |
| Runpod C Runpod, Inc. | The vLLM worker serves a Hub model with OpenAI-compatible responses from a template, but its hub defaults (MAX_NUM_SEQS 256) made an 8B model OOM on an RTX 5090, relaunch in eager mode at half throughput and restart on the same footprint (acc-09). source |
| Thunder Compute D Thunder GPU, Inc. | tnr create --gpu a100xl --template ollama gives an instance with Ollama preinstalled and guides run DeepSeek R1 70B and GPT-OSS 120B in minutes, but the model is pulled by hand over SSH and no vLLM template or OpenAI-compatible URL is documented. source |
| Together GPU Clusters C Together Computer, Inc. | A Kubernetes cluster can run vLLM, but the agent writes the manifest, the smallest unit is 8 GPUs, and provisioning takes 10 minutes to an hour; no engine template or OpenAI endpoint is provided by the product. source |
| Vast.ai C Vast.ai Inc. | A recommended vLLM template plus a search filter gets a model server on a rented H100, which a user did to measure gpt-oss-20b refusal rates (acc-21). Prompts pass through the host's machine unless Secure Cloud is filtered, and the instance bills until destroyed. source |
| Verda C DataCrunch Oy | The Serverless Containers vLLM quickstart serves a Hub model behind an OpenAI-compatible endpoint from a container image on an L40S; on a VM the images are Ubuntu or JupyterLab and the engine is installed by hand. Deleting the deployment or instance stops charges. source |
Doesn't do this job 1
Listings in the category that do something else, with the reason.
| Listing | Why |
|---|---|
| Replicate Deployments B Replicate, LLC | A deployment serves a Cog model behind Replicate's prediction API; no OpenAI-compatible /v1/chat/completions route or vLLM template appears in the docs index, so the success criterion cannot be met. source |
What stops agents finishing
- no template for the engine, so the agent must install CUDA, vLLM and the weights over SSH
- peer host hardware with firmware-capped clocks or that cannot pull images
- prompts and outputs pass through an individual host's machine, so private data is out
- an idle machine bills by the hour until destroyed, with no scale to zero
Around it
Before: gpu-compute.find-and-provision, open-weights.big-model-on-big-box, open-weights.tool-calling-assistant. After: inference.tool-loop, inference.harness-compat, agent-harnesses.local-model, decision-models.run-on-own-hardware.
Where this comes from
Weight 14 of 100 in GPU & serverless compute: 7 of 48 accounts stand up vLLM or Ollama on a rented machine to serve a model to their own tools; two of them left for local hardware on cost.
The same as data: /api/v1/use-cases/gpu-compute.self-hosted-llm-server.json, or the MCP tool get_use_case with id gpu-compute.self-hosted-llm-server. Other jobs in this category: /api/v1/niches?category=gpu-compute.
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.