Niches · Model APIs & inference

Run thousands of requests overnight at half price

Two or three companies' docs show a full fit.

3companies, full fit
14companies, fit with a limit
42companies carry the tag
10share of the category's work
3first-hand accounts

What counts as done

The batch returns a result for every request within 24 hours at 50 per cent of the standard token price, each result is matched to its request id, and only succeeded requests are billed.

How agents ask for it: tag and summarise these 800 notes, run the eval set against the new model, classify the backlog, I don't need it now

Does the job 3

The docs show it meets the test above, and nothing we read says otherwise.

ListingWhy
Eden AI C
Eden AI SAS
Batches run on OpenAI, Anthropic, Mistral and Vertex within 24 hours at the model's batch rates, report status and request_counts, can be cancelled, and bill only succeeded requests. source
Fireworks AI Inference BB
Fireworks.ai, Inc.
Batch runs at 50 per cent off serverless prices with 12 to 72 hour expiry (24 default), progress states, prompt caching stacked, and completed rows saved and billed when a job expires. source
GroqCloud BB
Groq LLC
Batch is 50 per cent off with a 24-hour to 7-day window, request_counts with completed and failed, custom_id results, cancel and expired states, and no draw on standard rate limits; the discount does not stack with caching. source

Does the job, with a limit 15

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
Amazon Bedrock A
Amazon Web Services, Inc.
S3-based jobs with totalRecordCount, processedRecordCount and a stop call, but Converse-format records with toolConfig fail, the supported-model table lists Claude Opus 5 and 4.x and not Opus 5.5 or Sonnet 5.5, and the half-price rate sits on a script-drawn pricing page. source
Claude API A
Anthropic, PBC
50 per cent price, custom_id matching, request_counts and cancel, and errored, canceled or expired requests are not billed. Batches expire at 24 hours and the docs say demand can make more requests expire; acc-15 saw 782 notes succeed in 25 minutes but later 4-hour waits. source
Azure OpenAI in Foundry Models A
Microsoft Corporation
50 per cent off Global Standard with separate enqueued-token quota, custom_id results and cancel returning partial output, but 24 hours is a target not a guarantee, jobs are not expired, and the billing basis for failed lines is not stated. source
Cerebras Inference B
Cerebras Systems Inc.
Batch has request_counts, cancel, expired status and an error file, with completion guaranteed in 24 hours, but it is a Private Preview reached through sales and the discount is not stated. source
Cloudflare Workers AI BB
Cloudflare, Inc.
queueRequest=true returns a request_id to poll and a Workers binding takes arrays, but no discount, cancellation or per-item counts are stated. source
DeepInfra B
Deep Infra Inc.
Batch has custom_id matching, request_counts total, completed and failed, cancel and a 24-hour window, but the discount is 20 per cent below real-time, not half price. source
Gemini Developer API B
Google LLC
Batch runs at 50 per cent with inline or JSONL input and a 24-hour target, but a job that passes 48 hours expires with no results to retrieve, and per-item counts and the billing basis are not stated. source
LiteLLM B
Berrie AI Incorporated
Batch pass-through for OpenAI, Azure, Vertex, Bedrock, Mistral, vLLM and xAI in OpenAI file format; Anthropic is absent and cost tracking is Enterprise only. source
Mistral AI API B
Mistral AI (RCS Paris 952 418 325)
50% discount, succeeded and failed counts, a cancel endpoint, timeout_hours defaulting to 24 and results that don't expire; whether failed requests are billed isn't stated and batches may exceed the spend limit. source
Novita AI C
Novita AI
Batch API with a 24h completion window, statuses including EXPIRED and CANCELLED, a cancel endpoint and completed and failed counts; the 50% discount is introductory and billing of failed items isn't stated. source
OpenAI API AA
OpenAI OpCo, LLC
50% off, request_counts, cancel, a 24h window, expired items marked batch_expired and only completed requests charged; acc-22 and acc-23 report batches stuck at 0 for hours and a $212 charge with no output. source
Together AI Inference B
Together Computer, Inc.
Up to 50% off on selected models, 50,000 requests, a separate rate pool, cancel, custom_id reconciliation and billing only for successful responses; the 24-hour window is best effort and can be exceeded. source
Vercel AI Gateway B
Vercel Inc.
Batch in OpenAI Chat, Responses and Anthropic Messages formats at 50% of standard prices (80% for SpaceXAI), a 24-hour window, cancel, request_counts and webhooks; billing for failed items isn't stated. source
Vertex AI (Gemini Enterprise Agent Platform) AA
Google LLC
Batch inference at a 50% discount that stacks with caching, cancellable at any time and with no predefined quota for Gemini; completion target and billing of cancelled items weren't read. source
xAI API (Grok) BB
SpaceXSI (xAI)
A Batch API reports num_success, num_error and num_pending, supports cancel and a batchDiscountPercent, but the discount is 20% on listed models, below the 50% asked, and the window and billing of failures weren't read. source

Couldn't check 4

The docs were closed to us, or silent. These carry no mark either way.

ListingWhy
Glimpse F
Elorian AI, Inc.
OpenRouter has a batch quickstart that was not opened today; nothing read says whether Glimpse is eligible.
OpenRouter B
OpenRouter, Inc.
An asynchronous Batch API appears in the SDK index (batch-quickstart) but its docs page wasn't opened, so pricing, progress and billing are unknown.
StepFun API E
STEPFUN TECHNOLOGY (HK) LIMITED
The host platform.stepfun.ai closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read.
Tencent Cloud TokenHub C
Tencent Cloud Computing (Beijing) Co., Ltd. (腾讯云计算(北京)有限责任公司)
The host cloud.tencent.com closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read.

Doesn't do this job 19

Listings in the category that do something else, with the reason.

ListingWhy
Antseed C
Antseed Foundation
No batch endpoint; the route table lists chat, responses, messages, images, audio and video only.
BlockRun.AI A
BlockRun, Inc.
No batch endpoint among the 102 OpenAPI paths; chat, messages and responses only.
Cloudflare AI Gateway B
Cloudflare, Inc.
No batch endpoint in the gateway docs.
Cohere Chat API (Command models) B
Cohere Inc.
No batch endpoint in the v2 API.
DeepSeek API D
Hangzhou DeepSeek Artificial Intelligence Co., Ltd.
No batch API; off-peak hours halve the price of ordinary calls instead.
Hugging Face Inference Providers C
Hugging Face, Inc.
No batch endpoint for the router.
Jais 3 C
Mohamed bin Zayed University of Artificial Intelligence (Institute of Foundation Models), with Inception42 (a G42 company)
No hosted endpoint and no batch service; Hugging Face lists no inference provider for any Jais 3 repository. source
Meta Model API D
Meta Platforms, Inc.
No batch endpoint; background responses exist at 600 submissions a minute but at full price.
MiniMax API C
Nanonoble Pte. Ltd.
No batch endpoint in the docs index; asynchronous task APIs exist only for video and speech. source
Muse Spark Open C
Meta Platforms, Inc.
No batch endpoint; the weights run on the operator's own server or a third-party host. source
Nemotron 3.5 Super VL C
NVIDIA Corporation
No batch endpoint in the API catalogue reference or quickstart. source
Prism Inference C
Prism Technologies Inc
No batch route; Responses runs no background jobs, and batch is sold through sales with no published price. source
Router.com (Ramp Router) C
Ramp Business Corporation
No batch route in the docs index or endpoint table. source
Beam 1.0 C
Reflection AI
Batch is listed as a not-supported endpoint. source
RouterPlus D
EVO HQ Inc
Batches, files, embeddings, audio and moderation paths return 404 not_found. source
SambaCloud B
SambaNova Systems, Inc.
No batch endpoint in the docs index. source
SiliconFlow E
SiliconFlow Labs Pte. Ltd.
No batch endpoint in the docs index. source
Simplismart F
Verute Technologies Private Limited
No batch endpoint in the docs index. source
SWE-2 Flash F
Cognition AI, Inc.
No API or batch endpoint. source

6 listings in the category were added after this reading and aren't marked yet.

What stops agents finishing

  • batch sits at 0 of N for hours with no error and expires after partial completion
  • no per-item progress or cancellation, so a stuck job cannot be salvaged
  • billing for requests that produced no output
  • five-minute cache lifetime expires inside a batch, so the shared prefix is paid in full

Around it

Before: document-extraction.pdf-to-text, web-scrapers.crawl-site. After: vector-db.store, evals.score.

Where this comes from

Weight 10 of 100 in Model APIs & inference: 3 first-hand accounts, one success and two failures with about a dozen users each in the forum threads

The same as data: /api/v1/use-cases/inference.batch-offline.json, or the MCP tool get_use_case with id inference.batch-offline. Other jobs in this category: /api/v1/niches?category=inference.

Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.