# Pay once for a long static prefix across a session

> Pay once for a long static prefix across a session: 3 companies fully do it, 23 with a limit, 42 carry the tag. Every listing in Model APIs & inference with the reason and a source.

- Canonical: https://www.anchorterminal.com/niches/inference.cached-long-session/
- Markdown: https://www.anchorterminal.com/niches/inference.cached-long-session/index.md (~3,100 tokens)
- Slim: https://www.anchorterminal.com/niches/inference.cached-long-session/index.min.md (~580 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/niches/inference.cached-long-session/index.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-15

Two or three companies' docs show a full fit.

| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |
| --- | --- | --- | --- | --- |
| 3 | 23 | 42 | 12 | 3 |

## What counts as done

From the second turn on, usage reports cache reads priced below the standard input rate, and a turn after an idle gap does not cost more than an uncached turn.

Measured, for our own tests: from the second turn on, at least 70 per cent of input tokens are reported as cache reads.

How agents ask for it: stop paying for the system prompt and tool list every turn; cut the cost of a 30-turn agent run; why is my cache hit rate 7 per cent

## Does the job

The docs show it meets the test above, and nothing we read says otherwise.

- **Amazon Bedrock (Amazon Web Services, Inc.)**: Explicit cache checkpoints on system, messages and tools with 512 to 4,096 token minimums, up to 4 checkpoints, 5-minute or 1-hour TTL, and separate cache-write and cache-read rates; implicit caching also exists on some models.
- **Azure OpenAI in Foundry Models (Microsoft Corporation)**: Prefixes of at least 1,024 tokens cache automatically, reads are discounted, GPT-5.6 and later take prompt_cache_options.ttl of 30 minutes (12 hours on some Responses deployments) with cached_tokens in usage; older models clear after 5 to 10 idle minutes.
- **DeepInfra (Deep Infra Inc.)**: Automatic prefix caching with cached_tokens in usage, a reduced cached rate, prompt_cache_key, and prompt_cache_options to retain a prefix for 5 minutes or 1 hour.

## Does the job, with a limit

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

- **Claude API (Anthropic, PBC)**: cache_control on system and tools, reads at 0.1x (0.05x on Opus and Sonnet 5.5), usage reports cache_read_input_tokens. Default 5-minute lifetime means an idle gap rewrites at 1.25x unless the 1-hour TTL is bought; acc-25 found automatic caching's 20-block lookback made long loops worse.
- **BlockRun.AI (BlockRun, Inc.)**: prompt_cache opt-in and cache_control on /v1/messages work for Anthropic models only, with cache_read_input_tokens and cache_creation_input_tokens in usage; no caching is described for other labs.
- **Cerebras Inference (Cerebras Systems Inc.)**: Automatic prefix caching with prompt_cache_key and cached tokens reported in usage, but the guaranteed lifetime is 5 minutes, so an idle gap above that recomputes, and the cached price sits on model pages rather than the caching page.
- **Cloudflare Workers AI (Cloudflare, Inc.)**: Prefix caching is on by default for select models with a discounted cached rate and an x-session-affinity header to reach the same instance, but no minimum, lifetime or usage field is stated and pricing sits on each model page.
- **DeepSeek API (Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)**: Disk caching is automatic with cache hits at $0.044 per million on V4 Pro against $1.32 uncached and prompt_cache_hit_tokens in usage, but prefixes persist only at request boundaries, detected common prefixes or fixed intervals, and acc-09 saw zero cached tokens over 50 rounds.
- **Eden AI (Eden AI SAS)**: A routing key keeps a conversation on the provider that holds its prompt cache, but Eden documents no cache fields, minimum or price of its own; savings are whatever the upstream gives.
- **Glimpse (Elorian AI, Inc.)**: The endpoint reports implicit caching with cache reads at $0.06 per million, a tenth of input, but no lifetime or minimum and no account of hit rates.
- **Fireworks AI Inference (Fireworks.ai, Inc.)**: Automatic prefix caching, cached tokens discounted 50 per cent by default and reported as cache_read_input_tokens on the Anthropic route, with x-session-affinity to keep hits, but no cache lifetime or minimum length is stated.
- **Gemini Developer API (Google LLC)**: Implicit caching reports total_cached_tokens at a tenth of input with a 4,096-token minimum, but Gemini 4 caches the previous request only when the new one adds 2,050 tokens or more and supports no explicit caching, and no lifetime is stated.
- **GroqCloud (Groq LLC)**: Automatic prefix caching halves the price of cached tokens, which also skip rate limits, and entries live 2 hours without use, but only a short list of models supports it and no minimum length is stated.
- **LiteLLM (Berrie AI Incorporated)**: cache_control is forwarded for OpenAI, Anthropic, Gemini, Bedrock, Mantle and DeepSeek with provider minimums listed, but caching below the minimum is skipped silently and the caller must check cache_creation_input_tokens.
- **Meta Model API (Meta Platforms, Inc.)**: Prefix caching is automatic, cached_tokens appears in usage and cached input is $0.20 against $4.00 on Muse Wonder, but no cache lifetime or minimum length is stated, so a long idle gap's cost is unknown.
- **MiniMax API (Nanonoble Pte. Ltd.)**: Automatic prefix caching from 512 input tokens, reads at $0.015 against $0.15 input on M3.1 Flash, reported as cache_read_input_tokens or cached_tokens; no cache lifetime is stated, so the idle-gap case is unverified.
- **Mistral AI API (Mistral AI (RCS Paris 952 418 325))**: prompt_cache_key raises the chance of a hit but the docs say it doesn't guarantee one; cached tokens bill at 10% of input in 64-token blocks and show as cached_tokens; no lifetime stated.
- **Muse Spark Open (Meta Platforms, Inc.)**: Caching comes from the serving stack (the deploy page enables SGLang's hierarchical cache) or a hosted provider's cached-input price; Meta's docs state no lifetime, minimum or usage field.
- **Novita AI (Novita AI)**: Automatic prompt cache on supported models with cached_tokens in usage and a cache price on the pricing page; the guide states no minimum length, lifetime or discount figure.
- **OpenAI API (OpenAI OpCo, LLC)**: Minimum 1,024 tokens on GPT-5.6 and later, reads at 0.1x, writes at 1.25x for a 30-minute lifetime or 2.5x for a 12-hour ttl, cached_tokens in usage; a turn after the lifetime pays the write premium again.
- **OpenRouter (OpenRouter, Inc.)**: Per-provider caching with a top-level cache_control for Anthropic, Bedrock, Vertex and Azure, cached_tokens and cache write tokens in usage, and ttl options from 5 minutes to 12 hours; a turn after the default 5-minute ttl rewrites at 1.25x.
- **Prism Inference (Prism Technologies Inc)**: A cache-read price ($0.03 against $0.09 input) and cached_tokens appear in the model catalogue and OpenAPI, but no page says how caching triggers, its minimum or lifetime.
- **RouterPlus (EVO HQ Inc)**: cache_control ephemeral passes through, with cached_tokens and cache_write_tokens in usage and cached_prompt and cache_write prices in the catalogue; lifetime and minimum follow the provider and aren't stated.
- **SambaCloud (SambaNova Systems, Inc.)**: Automatic prefix caching from 4,096 tokens up to 192,000, cached_tokens in usage billed at a cached input rate; no lifetime stated.
- **Together AI Inference (Together Computer, Inc.)**: Automatic prefix caching on serverless with cached_tokens in usage and a cached input price column; entries last about an hour on a best-effort basis and can be evicted under load; minimum length not stated.
- **Vercel AI Gateway (Vercel Inc.)**: caching auto adds cache markers for Anthropic and relies on automatic caching for OpenAI and Google, with cache read tokens in usage and cache prices in /v1/models; lifetime and minimum follow the provider.
- **Vertex AI (Gemini Enterprise Agent Platform) (Google LLC)**: Implicit caching is on by default and explicit caches take a TTL; Gemini 3 and 4 need 4,096 tokens (6,144 on some Flash) for a 90% discount, cachedContentTokenCount is in usage, and explicit cache storage and 1.25x writes are billed.
- **xAI API (Grok) (SpaceXSI (xAI))**: Cached input bills at a quarter of input ($0.50 against $2 on Grok 4.7) with cached_tokens in usage; caching is automatic with no minimum, lifetime or write price stated, and cached tokens still count toward TPM.

## Couldn't check

The docs were closed to us, or silent. These carry no mark either way.

- **Antseed (Antseed Foundation)**: Sellers publish a cached-input price, but nothing read says whether the proxy keeps a seller's cache warm across turns or reports cache reads in usage.
- **Router.com (Ramp Router) (Ramp Business Corporation)**: A prompt caching page (cache-optimizations) is in the index but wasn't opened; the fallbacks guide notes a cold cache on a fallback model adds latency.
- **SiliconFlow (SiliconFlow Labs Pte. Ltd.)**: A cached price appears in the listing's pricing notes, but no caching page is in the docs index and the chat reference has no cached-token field.
- **StepFun API (STEPFUN TECHNOLOGY (HK) LIMITED)**: The host platform.stepfun.ai closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read.
- **Tencent Cloud TokenHub (Tencent Cloud Computing (Beijing) Co., Ltd. (腾讯云计算（北京）有限责任公司))**: The host cloud.tencent.com closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read.

- not this job: Cloudflare AI Gateway (Cloudflare, Inc.): Gateway caching is an exact-match response cache keyed on the whole request body, not prefix caching; a growing conversation never hits it.
- not this job: Cohere Chat API (Command models) (Cohere Inc.): No prompt caching appears in the chat, tool, streaming or models pages read.
- not this job: Hugging Face Inference Providers (Hugging Face, Inc.): No prompt caching, cache field or cached price appears in the router docs; any caching is the provider's and unreported.
- not this job: Jais 3 (Mohamed bin Zayed University of Artificial Intelligence (Institute of Foundation Models), with Inception42 (a G42 company)): Caching is the serving stack's business; the cards say nothing about prefix caching or cache reporting.
- not this job: Nemotron 3.5 Super VL (NVIDIA Corporation): No prompt caching parameter or cached-token usage field in the API reference, and the hosted trial has no token price to discount.
- not this job: Beam 1.0 (Reflection AI): The compatibility page's feature list has no prompt caching and no cached-token usage field, and the docs index has no caching page.
- not this job: Simplismart (Verute Technologies Private Limited): No caching parameter, price or usage field in the OpenAPI file or shared-endpoint page.
- not this job: SWE-2 Flash (Cognition AI, Inc.): Cache read and write prices are listed for use inside Devin ($0.01 and $0.125 per million) but there is no API to control or observe caching.

6 listings in the category were added after this reading and aren't marked yet.

## What stops agents finishing

- prompt below the provider's minimum cacheable length caches nothing and says so nowhere
- a timestamp or session id at the top of the prompt breaks the prefix every turn
- automatic caching looks back a fixed number of blocks, so long loops write at a premium and never read
- gateways that rewrite prompts or drop the cache key

## Around it

Before: inference.tool-loop. After: obs.cost-tracking.

## Where this comes from

Weight 12 of 100 in Model APIs & inference: 3 first-hand accounts (two with production numbers) plus three secondary write-ups; every agent account resends the prefix, so the job is universal even where unnamed

As data: https://www.anchorterminal.com/api/v1/use-cases/inference.cached-long-session.json, or the MCP tool get_use_case with id inference.cached-long-session. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp
