Niches · Model APIs & inference
Read a whole codebase or corpus in one call
Four or more companies' docs show a full fit.
What counts as done
A 300K-token request is accepted on the tier the caller is on.
Measured, for our own tests: it answers in under two minutes, and the response cites content from the end of the prompt as well as the start. No docs we read state these figures, so they don't count against a listing here.
How agents ask for it: load the entire repo and answer questions about it
, read these 300K tokens of history and summarise
, analyse this legacy monolith before we split it
Does the job 4
The docs show it meets the test above, and nothing we read says otherwise.
| Listing | Why |
|---|---|
| Claude API A Anthropic, PBC | 1M context is the default on current Opus, Sonnet, Fable and Haiku 5.5 with no beta header and standard pricing; Start tier allows 2,000,000 input tokens a minute on Opus and Sonnet (500,000 on Fable), so a 300K request fits. source |
| DeepInfra B Deep Infra Inc. | DeepSeek V4.1 Flash is listed with 1M context, the only limit is 200 concurrent requests per model, and requests queue at capacity rather than 429. source |
| DeepSeek API D Hangzhou DeepSeek Artificial Intelligence Co., Ltd. | 1M context with up to 384K output, limits are concurrency (2,500 and 500 per account) rather than tokens a minute, and off-peak pricing halves the cost of a large read. source |
| Meta Model API D Meta Platforms, Inc. | Muse Wonder and Muse Spark take 1,048,576 tokens with no long-context premium, and the per-model limit of 4,000,000 tokens a minute admits a 300K request. source |
Does the job, with a limit 25
It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.
| Listing | Why |
|---|---|
| Amazon Bedrock A Amazon Web Services, Inc. | Whether a 300K request passes depends on the per-model tokens-per-minute quota (input plus output) and the account's daily cross-model cap in Service Quotas; increases may be denied unless existing quota is being consumed. source |
| Azure OpenAI in Foundry Models A Microsoft Corporation | Per-deployment tokens-per-minute quota must cover the request, Tier 0 subscriptions get no quota increases, and long-context requests are priced at 2x input on a pricing page that needs JavaScript. source |
| BlockRun.AI A BlockRun, Inc. | No TPM or RPM cap on paid inference and 1M-context models are listed, but upstream calls time out at 120 seconds with a 504, which a 300K-token read can approach; max_tokens defaults to the smaller of model max and 8,192. source |
| Cloudflare AI Gateway B Cloudflare, Inc. | Pass-through with no stated request token limit; the cacheable request size is 25 MB and the provider's own quota decides acceptance. source |
| Eden AI C Eden AI SAS | Acceptance depends on the upstream model; Eden states no token limit of its own beyond 10 requests a second. source |
| Fireworks AI Inference BB Fireworks.ai, Inc. | Kimi K3 lists 1,040K context, but per-model tokens-per-minute limits start at a default and grow 25 per cent every 15 minutes toward a tier ceiling, so a fresh account's first 300K request can be a 429. source |
| Gemini Developer API B Google LLC | Gemini 4's 1M window is shared across input, thinking and output; acc-18 hit free-tier tokens-per-minute 429s past 300K and acc-14 found context preparation, not the call, was the bottleneck. source |
| Hugging Face Inference Providers C Hugging Face, Inc. | The router lists context per provider (1M on some hosts, 65K on others for the same model), but router rate limits and 429 behaviour are undocumented, so acceptance of a 300K request is unknown. source |
| Jais 3 C Mohamed bin Zayed University of Artificial Intelligence (Institute of Foundation Models), with Inception42 (a G42 company) | 512K context on 375B-A23B, but the reference deployment is 16 H100s or an 8-GPU H200 node, and high effort wants 32,768 output tokens on top. source |
| LiteLLM B Berrie AI Incorporated | Pass-through; context_window_fallbacks reroute a request the primary rejects for length, otherwise the provider's quota decides. source |
| MiniMax API C Nanonoble Pte. Ltd. | 1M context and a 10,000,000 TPM limit admit a 300K request, but thinking defaults to max effort and counts against max_tokens (maximum 512K), so set effort and budget; latency is unverified. source |
| Muse Spark Open C Meta Platforms, Inc. | 1,048,576-token window, but the server's context length is fixed at launch, the recipes assume 4 to 8 B300 GPUs, and the sample commands set 32768. source |
| Nemotron 3.5 Super VL C NVIDIA Corporation | The hosted page lists 262,144 tokens of context against the card's 1M and the hosted function caps completions at 131,072, so a 300K request fits only self-hosted. source |
| OpenAI API AA OpenAI OpCo, LLC | Reasoning tokens occupy the context window and the docs advise reserving 25,000 tokens; the pricing notes double input over 272K; acceptance of a 300K request depends on the tier's TPM shown in headers. source |
| OpenRouter B OpenRouter, Inc. | Depends on the routed model and host; max_tokens routes to hosts that fit it, and the in-flight credit budget can 429 a large request on a low-balance account. source |
| Prism Inference C Prism Technologies Inc | deepseek-v4.1-flash takes 1,000,000 tokens and oversize requests return context_length_exceeded rather than truncating; per-key limits aren't published, so a 300K call may hit 429. source |
| Router.com (Ramp Router) C Ramp Business Corporation | Models expose context_window (1,050,000 on some) and a long_context_token_threshold of 272000 with higher rates; a fallback list is bounded by its smallest window and rate limits are unpublished. source |
| Beam 1.0 C Reflection AI | Beam has a 1M context and 128K max output, but limits are per organisation in tokens per minute plus a daily token allowance renewing at 00:00 UTC, so a 300K call can spend the day's allowance. source |
| RouterPlus D EVO HQ Inc | context_overflow is a typed error that never fails over; trial keys run at 20 RPM and the balance hold assumes 32,768 output tokens, so send max_tokens with a large prompt. source |
| SambaCloud B SambaNova Systems, Inc. | Production models stop at 128K; only preview MiniMax-M3.1-Flash has 1M, and preview models have limited capacity and short-notice removal; Developer tier is capped at 20M tokens a day. source |
| SiliconFlow E SiliconFlow Labs Pte. Ltd. | Context lengths are per model (the reasoning table shows 65,536 for GLM-4.1V-9B-Thinking) and output past max_tokens is cut with finish_reason length; the rate-limits page returns 404. source |
| Together AI Inference B Together Computer, Inc. | Context depends on the model; serverless has no guaranteed capacity and bursts return 429 or 503, with provisioned throughput as the stated route. source |
| Vercel AI Gateway B Vercel Inc. | Depends on the routed model; rate limits aren't published, the free tier has lower per-model limits, and a 402 quota_for_entity_exceeded stops a spent budget. source |
| Vertex AI (Gemini Enterprise Agent Platform) AA Google LLC | Gemini ships a 1M-token window and thinking tokens share it under a soft budget; Standard PayGo has no fixed quota but shared-capacity 429s that acc-26 could not clear with 30 minutes of backoff; Claude has per-model QPM buckets. source |
| xAI API (Grok) BB SpaceXSI (xAI) | Grok 4.7 prices double above 200K prompt tokens for the whole request, and TPM caps by spend tier (Tier 0 at $0) can refuse a 300K request; context sizes are on the models page. source |
Couldn't check 6
The docs were closed to us, or silent. These carry no mark either way.
| Listing | Why |
|---|---|
| Antseed C Antseed Foundation | Context depends on the seller's model; the buyer waits 5 minutes for a non-streaming response by default and nothing states accepted input sizes. |
| Cloudflare Workers AI BB Cloudflare, Inc. | Context lengths live on each model page, none of which were read; the limits page lists request rates only. |
| Novita AI C Novita AI | Context lengths live on model pages drawn by script; the chat reference gives no context figure and per-model rate limits weren't readable. |
| Simplismart F Verute Technologies Private Limited | Context lengths aren't documented for shared endpoints. |
| StepFun API E STEPFUN TECHNOLOGY (HK) LIMITED | The host platform.stepfun.ai closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read. |
| Tencent Cloud TokenHub C Tencent Cloud Computing (Beijing) Co., Ltd. (腾讯云计算(北京)有限责任公司) | The host cloud.tencent.com closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read. |
Doesn't do this job 6
Listings in the category that do something else, with the reason.
| Listing | Why |
|---|---|
| Cerebras Inference B Cerebras Systems Inc. | Shared inference models stop at 65,536 (Qwen 3.8 27B) or 131,072 (GPT OSS 120B) tokens on the Hugging Face router's listing, so a 300K request is not accepted. source |
| Cohere Chat API (Command models) B Cohere Inc. | Command A's context is 256K and the trial key allows 100,000 tokens a minute, so a 300K-token request is not accepted. source |
| Glimpse F Elorian AI, Inc. | Context length is 262,144 tokens, below a 300K request. source |
| GroqCloud BB Groq LLC | Every self-serve model is served at 131,072 tokens of context (Hugging Face router listing for gpt-oss), so a 300K request is not accepted. source |
| Mistral AI API B Mistral AI (RCS Paris 952 418 325) | The largest context listed is 256K tokens and requests over the window return 400, so a 300K-token request can't be accepted. source |
| SWE-2 Flash F Cognition AI, Inc. | No context length published and no API. source |
6 listings in the category were added after this reading and aren't marked yet.
What stops agents finishing
- free or low tier tokens-per-minute cap below the context size, so a single large request is a 429
- the 1M window is shared with reasoning and output, so a long input leaves little room to answer
- context preparation takes longer than the call
Around it
Before: web-scrapers.crawl-site, document-extraction.pdf-to-text. After: inference.structured-extraction.
Where this comes from
Weight 8 of 100 in Model APIs & inference: 3 of 28 accounts; one team measured that 88 per cent of sessions used under 200K tokens, so the job is real but occasional
The same as data: /api/v1/use-cases/inference.long-context-read.json, or the MCP tool get_use_case with id inference.long-context-read. Other jobs in this category: /api/v1/niches?category=inference.
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.