{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/inference.batch-offline/",
    "json": "https://www.anchorterminal.com/niches/inference.batch-offline/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/inference.batch-offline/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/inference.batch-offline/index.min.md"
  },
  "markdown": "Two or three companies' docs show a full fit.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 3 | 14 | 42 | 10 | 3 |\n\n## What counts as done\n\nThe batch returns a result for every request within 24 hours at 50 per cent of the standard token price, each result is matched to its request id, and only succeeded requests are billed.\n\nHow agents ask for it: tag and summarise these 800 notes; run the eval set against the new model; classify the backlog, I don't need it now\n\n## Does the job\n\nThe docs show it meets the test above, and nothing we read says otherwise.\n\n- **Eden AI (Eden AI SAS)**: Batches run on OpenAI, Anthropic, Mistral and Vertex within 24 hours at the model's batch rates, report status and request_counts, can be cancelled, and bill only succeeded requests.\n- **Fireworks AI Inference (Fireworks.ai, Inc.)**: Batch runs at 50 per cent off serverless prices with 12 to 72 hour expiry (24 default), progress states, prompt caching stacked, and completed rows saved and billed when a job expires.\n- **GroqCloud (Groq LLC)**: Batch is 50 per cent off with a 24-hour to 7-day window, request_counts with completed and failed, custom_id results, cancel and expired states, and no draw on standard rate limits; the discount does not stack with caching.\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **Amazon Bedrock (Amazon Web Services, Inc.)**: S3-based jobs with totalRecordCount, processedRecordCount and a stop call, but Converse-format records with toolConfig fail, the supported-model table lists Claude Opus 5 and 4.x and not Opus 5.5 or Sonnet 5.5, and the half-price rate sits on a script-drawn pricing page.\n- **Claude API (Anthropic, PBC)**: 50 per cent price, custom_id matching, request_counts and cancel, and errored, canceled or expired requests are not billed. Batches expire at 24 hours and the docs say demand can make more requests expire; acc-15 saw 782 notes succeed in 25 minutes but later 4-hour waits.\n- **Azure OpenAI in Foundry Models (Microsoft Corporation)**: 50 per cent off Global Standard with separate enqueued-token quota, custom_id results and cancel returning partial output, but 24 hours is a target not a guarantee, jobs are not expired, and the billing basis for failed lines is not stated.\n- **Cerebras Inference (Cerebras Systems Inc.)**: Batch has request_counts, cancel, expired status and an error file, with completion guaranteed in 24 hours, but it is a Private Preview reached through sales and the discount is not stated.\n- **Cloudflare Workers AI (Cloudflare, Inc.)**: queueRequest=true returns a request_id to poll and a Workers binding takes arrays, but no discount, cancellation or per-item counts are stated.\n- **DeepInfra (Deep Infra Inc.)**: Batch has custom_id matching, request_counts total, completed and failed, cancel and a 24-hour window, but the discount is 20 per cent below real-time, not half price.\n- **Gemini Developer API (Google LLC)**: Batch runs at 50 per cent with inline or JSONL input and a 24-hour target, but a job that passes 48 hours expires with no results to retrieve, and per-item counts and the billing basis are not stated.\n- **LiteLLM (Berrie AI Incorporated)**: Batch pass-through for OpenAI, Azure, Vertex, Bedrock, Mistral, vLLM and xAI in OpenAI file format; Anthropic is absent and cost tracking is Enterprise only.\n- **Mistral AI API (Mistral AI (RCS Paris 952 418 325))**: 50% discount, succeeded and failed counts, a cancel endpoint, timeout_hours defaulting to 24 and results that don't expire; whether failed requests are billed isn't stated and batches may exceed the spend limit.\n- **Novita AI (Novita AI)**: Batch API with a 24h completion window, statuses including EXPIRED and CANCELLED, a cancel endpoint and completed and failed counts; the 50% discount is introductory and billing of failed items isn't stated.\n- **OpenAI API (OpenAI OpCo, LLC)**: 50% off, request_counts, cancel, a 24h window, expired items marked batch_expired and only completed requests charged; acc-22 and acc-23 report batches stuck at 0 for hours and a $212 charge with no output.\n- **Together AI Inference (Together Computer, Inc.)**: Up to 50% off on selected models, 50,000 requests, a separate rate pool, cancel, custom_id reconciliation and billing only for successful responses; the 24-hour window is best effort and can be exceeded.\n- **Vercel AI Gateway (Vercel Inc.)**: Batch in OpenAI Chat, Responses and Anthropic Messages formats at 50% of standard prices (80% for SpaceXAI), a 24-hour window, cancel, request_counts and webhooks; billing for failed items isn't stated.\n- **Vertex AI (Gemini Enterprise Agent Platform) (Google LLC)**: Batch inference at a 50% discount that stacks with caching, cancellable at any time and with no predefined quota for Gemini; completion target and billing of cancelled items weren't read.\n- **xAI API (Grok) (SpaceXSI (xAI))**: A Batch API reports num_success, num_error and num_pending, supports cancel and a batchDiscountPercent, but the discount is 20% on listed models, below the 50% asked, and the window and billing of failures weren't read.\n\n## Couldn't check\n\nThe docs were closed to us, or silent. These carry no mark either way.\n\n- **Glimpse (Elorian AI, Inc.)**: OpenRouter has a batch quickstart that was not opened today; nothing read says whether Glimpse is eligible.\n- **OpenRouter (OpenRouter, Inc.)**: An asynchronous Batch API appears in the SDK index (batch-quickstart) but its docs page wasn't opened, so pricing, progress and billing are unknown.\n- **StepFun API (STEPFUN TECHNOLOGY (HK) LIMITED)**: The host platform.stepfun.ai closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read.\n- **Tencent Cloud TokenHub (Tencent Cloud Computing (Beijing) Co., Ltd. (腾讯云计算（北京）有限责任公司))**: The host cloud.tencent.com closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read.\n\n- not this job: Antseed (Antseed Foundation): No batch endpoint; the route table lists chat, responses, messages, images, audio and video only.\n- not this job: BlockRun.AI (BlockRun, Inc.): No batch endpoint among the 102 OpenAPI paths; chat, messages and responses only.\n- not this job: Cloudflare AI Gateway (Cloudflare, Inc.): No batch endpoint in the gateway docs.\n- not this job: Cohere Chat API (Command models) (Cohere Inc.): No batch endpoint in the v2 API.\n- not this job: DeepSeek API (Hangzhou DeepSeek Artificial Intelligence Co., Ltd.): No batch API; off-peak hours halve the price of ordinary calls instead.\n- not this job: Hugging Face Inference Providers (Hugging Face, Inc.): No batch endpoint for the router.\n- not this job: Jais 3 (Mohamed bin Zayed University of Artificial Intelligence (Institute of Foundation Models), with Inception42 (a G42 company)): No hosted endpoint and no batch service; Hugging Face lists no inference provider for any Jais 3 repository.\n- not this job: Meta Model API (Meta Platforms, Inc.): No batch endpoint; background responses exist at 600 submissions a minute but at full price.\n- not this job: MiniMax API (Nanonoble Pte. Ltd.): No batch endpoint in the docs index; asynchronous task APIs exist only for video and speech.\n- not this job: Muse Spark Open (Meta Platforms, Inc.): No batch endpoint; the weights run on the operator's own server or a third-party host.\n- not this job: Nemotron 3.5 Super VL (NVIDIA Corporation): No batch endpoint in the API catalogue reference or quickstart.\n- not this job: Prism Inference (Prism Technologies Inc): No batch route; Responses runs no background jobs, and batch is sold through sales with no published price.\n- not this job: Router.com (Ramp Router) (Ramp Business Corporation): No batch route in the docs index or endpoint table.\n- not this job: Beam 1.0 (Reflection AI): Batch is listed as a not-supported endpoint.\n- not this job: RouterPlus (EVO HQ Inc): Batches, files, embeddings, audio and moderation paths return 404 not_found.\n- not this job: SambaCloud (SambaNova Systems, Inc.): No batch endpoint in the docs index.\n- not this job: SiliconFlow (SiliconFlow Labs Pte. Ltd.): No batch endpoint in the docs index.\n- not this job: Simplismart (Verute Technologies Private Limited): No batch endpoint in the docs index.\n- not this job: SWE-2 Flash (Cognition AI, Inc.): No API or batch endpoint.\n\n6 listings in the category were added after this reading and aren't marked yet.\n\n## What stops agents finishing\n\n- batch sits at 0 of N for hours with no error and expires after partial completion\n- no per-item progress or cancellation, so a stuck job cannot be salvaged\n- billing for requests that produced no output\n- five-minute cache lifetime expires inside a batch, so the shared prefix is paid in full\n\n## Around it\n\nBefore: document-extraction.pdf-to-text, web-scrapers.crawl-site. After: vector-db.store, evals.score.\n\n## Where this comes from\n\nWeight 10 of 100 in Model APIs \u0026 inference: 3 first-hand accounts, one success and two failures with about a dozen users each in the forum threads\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/inference.batch-offline.json, or the MCP tool get_use_case with id inference.batch-offline. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Run thousands of requests overnight at half price",
        "url": ""
      }
    ],
    "description": "Run thousands of requests overnight at half price: 3 companies fully do it, 14 with a limit, 42 carry the tag. Every listing in Model APIs \u0026 inference with the reason and a source.",
    "facts": null,
    "h1": "Run thousands of requests overnight at half price",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/inference.batch-offline/",
    "published": "",
    "section": "agents",
    "title": "Run thousands of requests overnight at half price | Anchor Terminal",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/inference.batch-offline/"
  },
  "tokens": {
    "markdown": 2550,
    "slim": 430
  },
  "version": 1
}
