Niches · Model APIs & inference

Pay once for a long static prefix across a session

Two or three companies' docs show a full fit.

3companies, full fit
23companies, fit with a limit
42companies carry the tag
12share of the category's work
3first-hand accounts

What counts as done

From the second turn on, usage reports cache reads priced below the standard input rate, and a turn after an idle gap does not cost more than an uncached turn.

Measured, for our own tests: from the second turn on, at least 70 per cent of input tokens are reported as cache reads. No docs we read state these figures, so they don't count against a listing here.

How agents ask for it: stop paying for the system prompt and tool list every turn, cut the cost of a 30-turn agent run, why is my cache hit rate 7 per cent

Does the job 3

The docs show it meets the test above, and nothing we read says otherwise.

ListingWhy
Amazon Bedrock A
Amazon Web Services, Inc.
Explicit cache checkpoints on system, messages and tools with 512 to 4,096 token minimums, up to 4 checkpoints, 5-minute or 1-hour TTL, and separate cache-write and cache-read rates; implicit caching also exists on some models. source
Azure OpenAI in Foundry Models A
Microsoft Corporation
Prefixes of at least 1,024 tokens cache automatically, reads are discounted, GPT-5.6 and later take prompt_cache_options.ttl of 30 minutes (12 hours on some Responses deployments) with cached_tokens in usage; older models clear after 5 to 10 idle minutes. source
DeepInfra B
Deep Infra Inc.
Automatic prefix caching with cached_tokens in usage, a reduced cached rate, prompt_cache_key, and prompt_cache_options to retain a prefix for 5 minutes or 1 hour. source

Does the job, with a limit 25

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
Claude API A
Anthropic, PBC
cache_control on system and tools, reads at 0.1x (0.05x on Opus and Sonnet 5.5), usage reports cache_read_input_tokens. Default 5-minute lifetime means an idle gap rewrites at 1.25x unless the 1-hour TTL is bought; acc-25 found automatic caching's 20-block lookback made long loops worse. source
BlockRun.AI A
BlockRun, Inc.
prompt_cache opt-in and cache_control on /v1/messages work for Anthropic models only, with cache_read_input_tokens and cache_creation_input_tokens in usage; no caching is described for other labs. source
Cerebras Inference B
Cerebras Systems Inc.
Automatic prefix caching with prompt_cache_key and cached tokens reported in usage, but the guaranteed lifetime is 5 minutes, so an idle gap above that recomputes, and the cached price sits on model pages rather than the caching page. source
Cloudflare Workers AI BB
Cloudflare, Inc.
Prefix caching is on by default for select models with a discounted cached rate and an x-session-affinity header to reach the same instance, but no minimum, lifetime or usage field is stated and pricing sits on each model page. source
DeepSeek API D
Hangzhou DeepSeek Artificial Intelligence Co., Ltd.
Disk caching is automatic with cache hits at $0.044 per million on V4 Pro against $1.32 uncached and prompt_cache_hit_tokens in usage, but prefixes persist only at request boundaries, detected common prefixes or fixed intervals, and acc-09 saw zero cached tokens over 50 rounds. source
Eden AI C
Eden AI SAS
A routing key keeps a conversation on the provider that holds its prompt cache, but Eden documents no cache fields, minimum or price of its own; savings are whatever the upstream gives. source
Glimpse F
Elorian AI, Inc.
The endpoint reports implicit caching with cache reads at $0.06 per million, a tenth of input, but no lifetime or minimum and no account of hit rates. source
Fireworks AI Inference BB
Fireworks.ai, Inc.
Automatic prefix caching, cached tokens discounted 50 per cent by default and reported as cache_read_input_tokens on the Anthropic route, with x-session-affinity to keep hits, but no cache lifetime or minimum length is stated. source
Gemini Developer API B
Google LLC
Implicit caching reports total_cached_tokens at a tenth of input with a 4,096-token minimum, but Gemini 4 caches the previous request only when the new one adds 2,050 tokens or more and supports no explicit caching, and no lifetime is stated. source
GroqCloud BB
Groq LLC
Automatic prefix caching halves the price of cached tokens, which also skip rate limits, and entries live 2 hours without use, but only a short list of models supports it and no minimum length is stated. source
LiteLLM B
Berrie AI Incorporated
cache_control is forwarded for OpenAI, Anthropic, Gemini, Bedrock, Mantle and DeepSeek with provider minimums listed, but caching below the minimum is skipped silently and the caller must check cache_creation_input_tokens. source
Meta Model API D
Meta Platforms, Inc.
Prefix caching is automatic, cached_tokens appears in usage and cached input is $0.20 against $4.00 on Muse Wonder, but no cache lifetime or minimum length is stated, so a long idle gap's cost is unknown. source
MiniMax API C
Nanonoble Pte. Ltd.
Automatic prefix caching from 512 input tokens, reads at $0.015 against $0.15 input on M3.1 Flash, reported as cache_read_input_tokens or cached_tokens; no cache lifetime is stated, so the idle-gap case is unverified. source
Mistral AI API B
Mistral AI (RCS Paris 952 418 325)
prompt_cache_key raises the chance of a hit but the docs say it doesn't guarantee one; cached tokens bill at 10% of input in 64-token blocks and show as cached_tokens; no lifetime stated. source
Muse Spark Open C
Meta Platforms, Inc.
Caching comes from the serving stack (the deploy page enables SGLang's hierarchical cache) or a hosted provider's cached-input price; Meta's docs state no lifetime, minimum or usage field. source
Novita AI C
Novita AI
Automatic prompt cache on supported models with cached_tokens in usage and a cache price on the pricing page; the guide states no minimum length, lifetime or discount figure. source
OpenAI API AA
OpenAI OpCo, LLC
Minimum 1,024 tokens on GPT-5.6 and later, reads at 0.1x, writes at 1.25x for a 30-minute lifetime or 2.5x for a 12-hour ttl, cached_tokens in usage; a turn after the lifetime pays the write premium again. source
OpenRouter B
OpenRouter, Inc.
Per-provider caching with a top-level cache_control for Anthropic, Bedrock, Vertex and Azure, cached_tokens and cache write tokens in usage, and ttl options from 5 minutes to 12 hours; a turn after the default 5-minute ttl rewrites at 1.25x. source
Prism Inference C
Prism Technologies Inc
A cache-read price ($0.03 against $0.09 input) and cached_tokens appear in the model catalogue and OpenAPI, but no page says how caching triggers, its minimum or lifetime. source
RouterPlus D
EVO HQ Inc
cache_control ephemeral passes through, with cached_tokens and cache_write_tokens in usage and cached_prompt and cache_write prices in the catalogue; lifetime and minimum follow the provider and aren't stated. source
SambaCloud B
SambaNova Systems, Inc.
Automatic prefix caching from 4,096 tokens up to 192,000, cached_tokens in usage billed at a cached input rate; no lifetime stated. source
Together AI Inference B
Together Computer, Inc.
Automatic prefix caching on serverless with cached_tokens in usage and a cached input price column; entries last about an hour on a best-effort basis and can be evicted under load; minimum length not stated. source
Vercel AI Gateway B
Vercel Inc.
caching auto adds cache markers for Anthropic and relies on automatic caching for OpenAI and Google, with cache read tokens in usage and cache prices in /v1/models; lifetime and minimum follow the provider. source
Vertex AI (Gemini Enterprise Agent Platform) AA
Google LLC
Implicit caching is on by default and explicit caches take a TTL; Gemini 3 and 4 need 4,096 tokens (6,144 on some Flash) for a 90% discount, cachedContentTokenCount is in usage, and explicit cache storage and 1.25x writes are billed. source
xAI API (Grok) BB
SpaceXSI (xAI)
Cached input bills at a quarter of input ($0.50 against $2 on Grok 4.7) with cached_tokens in usage; caching is automatic with no minimum, lifetime or write price stated, and cached tokens still count toward TPM. source

Couldn't check 5

The docs were closed to us, or silent. These carry no mark either way.

ListingWhy
Antseed C
Antseed Foundation
Sellers publish a cached-input price, but nothing read says whether the proxy keeps a seller's cache warm across turns or reports cache reads in usage.
Router.com (Ramp Router) C
Ramp Business Corporation
A prompt caching page (cache-optimizations) is in the index but wasn't opened; the fallbacks guide notes a cold cache on a fallback model adds latency.
SiliconFlow E
SiliconFlow Labs Pte. Ltd.
A cached price appears in the listing's pricing notes, but no caching page is in the docs index and the chat reference has no cached-token field.
StepFun API E
STEPFUN TECHNOLOGY (HK) LIMITED
The host platform.stepfun.ai closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read.
Tencent Cloud TokenHub C
Tencent Cloud Computing (Beijing) Co., Ltd. (腾讯云计算(北京)有限责任公司)
The host cloud.tencent.com closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read.

Doesn't do this job 8

Listings in the category that do something else, with the reason.

ListingWhy
Cloudflare AI Gateway B
Cloudflare, Inc.
Gateway caching is an exact-match response cache keyed on the whole request body, not prefix caching; a growing conversation never hits it. source
Cohere Chat API (Command models) B
Cohere Inc.
No prompt caching appears in the chat, tool, streaming or models pages read.
Hugging Face Inference Providers C
Hugging Face, Inc.
No prompt caching, cache field or cached price appears in the router docs; any caching is the provider's and unreported.
Jais 3 C
Mohamed bin Zayed University of Artificial Intelligence (Institute of Foundation Models), with Inception42 (a G42 company)
Caching is the serving stack's business; the cards say nothing about prefix caching or cache reporting.
Nemotron 3.5 Super VL C
NVIDIA Corporation
No prompt caching parameter or cached-token usage field in the API reference, and the hosted trial has no token price to discount. source
Beam 1.0 C
Reflection AI
The compatibility page's feature list has no prompt caching and no cached-token usage field, and the docs index has no caching page. source
Simplismart F
Verute Technologies Private Limited
No caching parameter, price or usage field in the OpenAPI file or shared-endpoint page. source
SWE-2 Flash F
Cognition AI, Inc.
Cache read and write prices are listed for use inside Devin ($0.01 and $0.125 per million) but there is no API to control or observe caching. source

6 listings in the category were added after this reading and aren't marked yet.

What stops agents finishing

  • prompt below the provider's minimum cacheable length caches nothing and says so nowhere
  • a timestamp or session id at the top of the prompt breaks the prefix every turn
  • automatic caching looks back a fixed number of blocks, so long loops write at a premium and never read
  • gateways that rewrite prompts or drop the cache key

Around it

Before: inference.tool-loop. After: obs.cost-tracking.

Where this comes from

Weight 12 of 100 in Model APIs & inference: 3 first-hand accounts (two with production numbers) plus three secondary write-ups; every agent account resends the prefix, so the job is universal even where unnamed

The same as data: /api/v1/use-cases/inference.cached-long-session.json, or the MCP tool get_use_case with id inference.cached-long-session. Other jobs in this category: /api/v1/niches?category=inference.

Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.