Niches · Model APIs & inference
Pay once for a long static prefix across a session
Two or three companies' docs show a full fit.
What counts as done
From the second turn on, usage reports cache reads priced below the standard input rate, and a turn after an idle gap does not cost more than an uncached turn.
Measured, for our own tests: from the second turn on, at least 70 per cent of input tokens are reported as cache reads. No docs we read state these figures, so they don't count against a listing here.
How agents ask for it: stop paying for the system prompt and tool list every turn
, cut the cost of a 30-turn agent run
, why is my cache hit rate 7 per cent
Does the job 3
The docs show it meets the test above, and nothing we read says otherwise.
| Listing | Why |
|---|---|
| Amazon Bedrock A Amazon Web Services, Inc. | Explicit cache checkpoints on system, messages and tools with 512 to 4,096 token minimums, up to 4 checkpoints, 5-minute or 1-hour TTL, and separate cache-write and cache-read rates; implicit caching also exists on some models. source |
| Azure OpenAI in Foundry Models A Microsoft Corporation | Prefixes of at least 1,024 tokens cache automatically, reads are discounted, GPT-5.6 and later take prompt_cache_options.ttl of 30 minutes (12 hours on some Responses deployments) with cached_tokens in usage; older models clear after 5 to 10 idle minutes. source |
| DeepInfra B Deep Infra Inc. | Automatic prefix caching with cached_tokens in usage, a reduced cached rate, prompt_cache_key, and prompt_cache_options to retain a prefix for 5 minutes or 1 hour. source |
Does the job, with a limit 25
It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.
| Listing | Why |
|---|---|
| Claude API A Anthropic, PBC | cache_control on system and tools, reads at 0.1x (0.05x on Opus and Sonnet 5.5), usage reports cache_read_input_tokens. Default 5-minute lifetime means an idle gap rewrites at 1.25x unless the 1-hour TTL is bought; acc-25 found automatic caching's 20-block lookback made long loops worse. source |
| BlockRun.AI A BlockRun, Inc. | prompt_cache opt-in and cache_control on /v1/messages work for Anthropic models only, with cache_read_input_tokens and cache_creation_input_tokens in usage; no caching is described for other labs. source |
| Cerebras Inference B Cerebras Systems Inc. | Automatic prefix caching with prompt_cache_key and cached tokens reported in usage, but the guaranteed lifetime is 5 minutes, so an idle gap above that recomputes, and the cached price sits on model pages rather than the caching page. source |
| Cloudflare Workers AI BB Cloudflare, Inc. | Prefix caching is on by default for select models with a discounted cached rate and an x-session-affinity header to reach the same instance, but no minimum, lifetime or usage field is stated and pricing sits on each model page. source |
| DeepSeek API D Hangzhou DeepSeek Artificial Intelligence Co., Ltd. | Disk caching is automatic with cache hits at $0.044 per million on V4 Pro against $1.32 uncached and prompt_cache_hit_tokens in usage, but prefixes persist only at request boundaries, detected common prefixes or fixed intervals, and acc-09 saw zero cached tokens over 50 rounds. source |
| Eden AI C Eden AI SAS | A routing key keeps a conversation on the provider that holds its prompt cache, but Eden documents no cache fields, minimum or price of its own; savings are whatever the upstream gives. source |
| Glimpse F Elorian AI, Inc. | The endpoint reports implicit caching with cache reads at $0.06 per million, a tenth of input, but no lifetime or minimum and no account of hit rates. source |
| Fireworks AI Inference BB Fireworks.ai, Inc. | Automatic prefix caching, cached tokens discounted 50 per cent by default and reported as cache_read_input_tokens on the Anthropic route, with x-session-affinity to keep hits, but no cache lifetime or minimum length is stated. source |
| Gemini Developer API B Google LLC | Implicit caching reports total_cached_tokens at a tenth of input with a 4,096-token minimum, but Gemini 4 caches the previous request only when the new one adds 2,050 tokens or more and supports no explicit caching, and no lifetime is stated. source |
| GroqCloud BB Groq LLC | Automatic prefix caching halves the price of cached tokens, which also skip rate limits, and entries live 2 hours without use, but only a short list of models supports it and no minimum length is stated. source |
| LiteLLM B Berrie AI Incorporated | cache_control is forwarded for OpenAI, Anthropic, Gemini, Bedrock, Mantle and DeepSeek with provider minimums listed, but caching below the minimum is skipped silently and the caller must check cache_creation_input_tokens. source |
| Meta Model API D Meta Platforms, Inc. | Prefix caching is automatic, cached_tokens appears in usage and cached input is $0.20 against $4.00 on Muse Wonder, but no cache lifetime or minimum length is stated, so a long idle gap's cost is unknown. source |
| MiniMax API C Nanonoble Pte. Ltd. | Automatic prefix caching from 512 input tokens, reads at $0.015 against $0.15 input on M3.1 Flash, reported as cache_read_input_tokens or cached_tokens; no cache lifetime is stated, so the idle-gap case is unverified. source |
| Mistral AI API B Mistral AI (RCS Paris 952 418 325) | prompt_cache_key raises the chance of a hit but the docs say it doesn't guarantee one; cached tokens bill at 10% of input in 64-token blocks and show as cached_tokens; no lifetime stated. source |
| Muse Spark Open C Meta Platforms, Inc. | Caching comes from the serving stack (the deploy page enables SGLang's hierarchical cache) or a hosted provider's cached-input price; Meta's docs state no lifetime, minimum or usage field. source |
| Novita AI C Novita AI | Automatic prompt cache on supported models with cached_tokens in usage and a cache price on the pricing page; the guide states no minimum length, lifetime or discount figure. source |
| OpenAI API AA OpenAI OpCo, LLC | Minimum 1,024 tokens on GPT-5.6 and later, reads at 0.1x, writes at 1.25x for a 30-minute lifetime or 2.5x for a 12-hour ttl, cached_tokens in usage; a turn after the lifetime pays the write premium again. source |
| OpenRouter B OpenRouter, Inc. | Per-provider caching with a top-level cache_control for Anthropic, Bedrock, Vertex and Azure, cached_tokens and cache write tokens in usage, and ttl options from 5 minutes to 12 hours; a turn after the default 5-minute ttl rewrites at 1.25x. source |
| Prism Inference C Prism Technologies Inc | A cache-read price ($0.03 against $0.09 input) and cached_tokens appear in the model catalogue and OpenAPI, but no page says how caching triggers, its minimum or lifetime. source |
| RouterPlus D EVO HQ Inc | cache_control ephemeral passes through, with cached_tokens and cache_write_tokens in usage and cached_prompt and cache_write prices in the catalogue; lifetime and minimum follow the provider and aren't stated. source |
| SambaCloud B SambaNova Systems, Inc. | Automatic prefix caching from 4,096 tokens up to 192,000, cached_tokens in usage billed at a cached input rate; no lifetime stated. source |
| Together AI Inference B Together Computer, Inc. | Automatic prefix caching on serverless with cached_tokens in usage and a cached input price column; entries last about an hour on a best-effort basis and can be evicted under load; minimum length not stated. source |
| Vercel AI Gateway B Vercel Inc. | caching auto adds cache markers for Anthropic and relies on automatic caching for OpenAI and Google, with cache read tokens in usage and cache prices in /v1/models; lifetime and minimum follow the provider. source |
| Vertex AI (Gemini Enterprise Agent Platform) AA Google LLC | Implicit caching is on by default and explicit caches take a TTL; Gemini 3 and 4 need 4,096 tokens (6,144 on some Flash) for a 90% discount, cachedContentTokenCount is in usage, and explicit cache storage and 1.25x writes are billed. source |
| xAI API (Grok) BB SpaceXSI (xAI) | Cached input bills at a quarter of input ($0.50 against $2 on Grok 4.7) with cached_tokens in usage; caching is automatic with no minimum, lifetime or write price stated, and cached tokens still count toward TPM. source |
Couldn't check 5
The docs were closed to us, or silent. These carry no mark either way.
| Listing | Why |
|---|---|
| Antseed C Antseed Foundation | Sellers publish a cached-input price, but nothing read says whether the proxy keeps a seller's cache warm across turns or reports cache reads in usage. |
| Router.com (Ramp Router) C Ramp Business Corporation | A prompt caching page (cache-optimizations) is in the index but wasn't opened; the fallbacks guide notes a cold cache on a fallback model adds latency. |
| SiliconFlow E SiliconFlow Labs Pte. Ltd. | A cached price appears in the listing's pricing notes, but no caching page is in the docs index and the chat reference has no cached-token field. |
| StepFun API E STEPFUN TECHNOLOGY (HK) LIMITED | The host platform.stepfun.ai closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read. |
| Tencent Cloud TokenHub C Tencent Cloud Computing (Beijing) Co., Ltd. (腾讯云计算(北京)有限责任公司) | The host cloud.tencent.com closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read. |
Doesn't do this job 8
Listings in the category that do something else, with the reason.
| Listing | Why |
|---|---|
| Cloudflare AI Gateway B Cloudflare, Inc. | Gateway caching is an exact-match response cache keyed on the whole request body, not prefix caching; a growing conversation never hits it. source |
| Cohere Chat API (Command models) B Cohere Inc. | No prompt caching appears in the chat, tool, streaming or models pages read. |
| Hugging Face Inference Providers C Hugging Face, Inc. | No prompt caching, cache field or cached price appears in the router docs; any caching is the provider's and unreported. |
| Jais 3 C Mohamed bin Zayed University of Artificial Intelligence (Institute of Foundation Models), with Inception42 (a G42 company) | Caching is the serving stack's business; the cards say nothing about prefix caching or cache reporting. |
| Nemotron 3.5 Super VL C NVIDIA Corporation | No prompt caching parameter or cached-token usage field in the API reference, and the hosted trial has no token price to discount. source |
| Beam 1.0 C Reflection AI | The compatibility page's feature list has no prompt caching and no cached-token usage field, and the docs index has no caching page. source |
| Simplismart F Verute Technologies Private Limited | No caching parameter, price or usage field in the OpenAPI file or shared-endpoint page. source |
| SWE-2 Flash F Cognition AI, Inc. | Cache read and write prices are listed for use inside Devin ($0.01 and $0.125 per million) but there is no API to control or observe caching. source |
6 listings in the category were added after this reading and aren't marked yet.
What stops agents finishing
- prompt below the provider's minimum cacheable length caches nothing and says so nowhere
- a timestamp or session id at the top of the prompt breaks the prefix every turn
- automatic caching looks back a fixed number of blocks, so long loops write at a premium and never read
- gateways that rewrite prompts or drop the cache key
Around it
Before: inference.tool-loop. After: obs.cost-tracking.
Where this comes from
Weight 12 of 100 in Model APIs & inference: 3 first-hand accounts (two with production numbers) plus three secondary write-ups; every agent account resends the prefix, so the job is universal even where unnamed
The same as data: /api/v1/use-cases/inference.cached-long-session.json, or the MCP tool get_use_case with id inference.cached-long-session. Other jobs in this category: /api/v1/niches?category=inference.
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.