{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/inference.cached-long-session/",
    "json": "https://www.anchorterminal.com/niches/inference.cached-long-session/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/inference.cached-long-session/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/inference.cached-long-session/index.min.md"
  },
  "markdown": "Two or three companies' docs show a full fit.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 3 | 23 | 42 | 12 | 3 |\n\n## What counts as done\n\nFrom the second turn on, usage reports cache reads priced below the standard input rate, and a turn after an idle gap does not cost more than an uncached turn.\n\nMeasured, for our own tests: from the second turn on, at least 70 per cent of input tokens are reported as cache reads.\n\nHow agents ask for it: stop paying for the system prompt and tool list every turn; cut the cost of a 30-turn agent run; why is my cache hit rate 7 per cent\n\n## Does the job\n\nThe docs show it meets the test above, and nothing we read says otherwise.\n\n- **Amazon Bedrock (Amazon Web Services, Inc.)**: Explicit cache checkpoints on system, messages and tools with 512 to 4,096 token minimums, up to 4 checkpoints, 5-minute or 1-hour TTL, and separate cache-write and cache-read rates; implicit caching also exists on some models.\n- **Azure OpenAI in Foundry Models (Microsoft Corporation)**: Prefixes of at least 1,024 tokens cache automatically, reads are discounted, GPT-5.6 and later take prompt_cache_options.ttl of 30 minutes (12 hours on some Responses deployments) with cached_tokens in usage; older models clear after 5 to 10 idle minutes.\n- **DeepInfra (Deep Infra Inc.)**: Automatic prefix caching with cached_tokens in usage, a reduced cached rate, prompt_cache_key, and prompt_cache_options to retain a prefix for 5 minutes or 1 hour.\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **Claude API (Anthropic, PBC)**: cache_control on system and tools, reads at 0.1x (0.05x on Opus and Sonnet 5.5), usage reports cache_read_input_tokens. Default 5-minute lifetime means an idle gap rewrites at 1.25x unless the 1-hour TTL is bought; acc-25 found automatic caching's 20-block lookback made long loops worse.\n- **BlockRun.AI (BlockRun, Inc.)**: prompt_cache opt-in and cache_control on /v1/messages work for Anthropic models only, with cache_read_input_tokens and cache_creation_input_tokens in usage; no caching is described for other labs.\n- **Cerebras Inference (Cerebras Systems Inc.)**: Automatic prefix caching with prompt_cache_key and cached tokens reported in usage, but the guaranteed lifetime is 5 minutes, so an idle gap above that recomputes, and the cached price sits on model pages rather than the caching page.\n- **Cloudflare Workers AI (Cloudflare, Inc.)**: Prefix caching is on by default for select models with a discounted cached rate and an x-session-affinity header to reach the same instance, but no minimum, lifetime or usage field is stated and pricing sits on each model page.\n- **DeepSeek API (Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)**: Disk caching is automatic with cache hits at $0.044 per million on V4 Pro against $1.32 uncached and prompt_cache_hit_tokens in usage, but prefixes persist only at request boundaries, detected common prefixes or fixed intervals, and acc-09 saw zero cached tokens over 50 rounds.\n- **Eden AI (Eden AI SAS)**: A routing key keeps a conversation on the provider that holds its prompt cache, but Eden documents no cache fields, minimum or price of its own; savings are whatever the upstream gives.\n- **Glimpse (Elorian AI, Inc.)**: The endpoint reports implicit caching with cache reads at $0.06 per million, a tenth of input, but no lifetime or minimum and no account of hit rates.\n- **Fireworks AI Inference (Fireworks.ai, Inc.)**: Automatic prefix caching, cached tokens discounted 50 per cent by default and reported as cache_read_input_tokens on the Anthropic route, with x-session-affinity to keep hits, but no cache lifetime or minimum length is stated.\n- **Gemini Developer API (Google LLC)**: Implicit caching reports total_cached_tokens at a tenth of input with a 4,096-token minimum, but Gemini 4 caches the previous request only when the new one adds 2,050 tokens or more and supports no explicit caching, and no lifetime is stated.\n- **GroqCloud (Groq LLC)**: Automatic prefix caching halves the price of cached tokens, which also skip rate limits, and entries live 2 hours without use, but only a short list of models supports it and no minimum length is stated.\n- **LiteLLM (Berrie AI Incorporated)**: cache_control is forwarded for OpenAI, Anthropic, Gemini, Bedrock, Mantle and DeepSeek with provider minimums listed, but caching below the minimum is skipped silently and the caller must check cache_creation_input_tokens.\n- **Meta Model API (Meta Platforms, Inc.)**: Prefix caching is automatic, cached_tokens appears in usage and cached input is $0.20 against $4.00 on Muse Wonder, but no cache lifetime or minimum length is stated, so a long idle gap's cost is unknown.\n- **MiniMax API (Nanonoble Pte. Ltd.)**: Automatic prefix caching from 512 input tokens, reads at $0.015 against $0.15 input on M3.1 Flash, reported as cache_read_input_tokens or cached_tokens; no cache lifetime is stated, so the idle-gap case is unverified.\n- **Mistral AI API (Mistral AI (RCS Paris 952 418 325))**: prompt_cache_key raises the chance of a hit but the docs say it doesn't guarantee one; cached tokens bill at 10% of input in 64-token blocks and show as cached_tokens; no lifetime stated.\n- **Muse Spark Open (Meta Platforms, Inc.)**: Caching comes from the serving stack (the deploy page enables SGLang's hierarchical cache) or a hosted provider's cached-input price; Meta's docs state no lifetime, minimum or usage field.\n- **Novita AI (Novita AI)**: Automatic prompt cache on supported models with cached_tokens in usage and a cache price on the pricing page; the guide states no minimum length, lifetime or discount figure.\n- **OpenAI API (OpenAI OpCo, LLC)**: Minimum 1,024 tokens on GPT-5.6 and later, reads at 0.1x, writes at 1.25x for a 30-minute lifetime or 2.5x for a 12-hour ttl, cached_tokens in usage; a turn after the lifetime pays the write premium again.\n- **OpenRouter (OpenRouter, Inc.)**: Per-provider caching with a top-level cache_control for Anthropic, Bedrock, Vertex and Azure, cached_tokens and cache write tokens in usage, and ttl options from 5 minutes to 12 hours; a turn after the default 5-minute ttl rewrites at 1.25x.\n- **Prism Inference (Prism Technologies Inc)**: A cache-read price ($0.03 against $0.09 input) and cached_tokens appear in the model catalogue and OpenAPI, but no page says how caching triggers, its minimum or lifetime.\n- **RouterPlus (EVO HQ Inc)**: cache_control ephemeral passes through, with cached_tokens and cache_write_tokens in usage and cached_prompt and cache_write prices in the catalogue; lifetime and minimum follow the provider and aren't stated.\n- **SambaCloud (SambaNova Systems, Inc.)**: Automatic prefix caching from 4,096 tokens up to 192,000, cached_tokens in usage billed at a cached input rate; no lifetime stated.\n- **Together AI Inference (Together Computer, Inc.)**: Automatic prefix caching on serverless with cached_tokens in usage and a cached input price column; entries last about an hour on a best-effort basis and can be evicted under load; minimum length not stated.\n- **Vercel AI Gateway (Vercel Inc.)**: caching auto adds cache markers for Anthropic and relies on automatic caching for OpenAI and Google, with cache read tokens in usage and cache prices in /v1/models; lifetime and minimum follow the provider.\n- **Vertex AI (Gemini Enterprise Agent Platform) (Google LLC)**: Implicit caching is on by default and explicit caches take a TTL; Gemini 3 and 4 need 4,096 tokens (6,144 on some Flash) for a 90% discount, cachedContentTokenCount is in usage, and explicit cache storage and 1.25x writes are billed.\n- **xAI API (Grok) (SpaceXSI (xAI))**: Cached input bills at a quarter of input ($0.50 against $2 on Grok 4.7) with cached_tokens in usage; caching is automatic with no minimum, lifetime or write price stated, and cached tokens still count toward TPM.\n\n## Couldn't check\n\nThe docs were closed to us, or silent. These carry no mark either way.\n\n- **Antseed (Antseed Foundation)**: Sellers publish a cached-input price, but nothing read says whether the proxy keeps a seller's cache warm across turns or reports cache reads in usage.\n- **Router.com (Ramp Router) (Ramp Business Corporation)**: A prompt caching page (cache-optimizations) is in the index but wasn't opened; the fallbacks guide notes a cold cache on a fallback model adds latency.\n- **SiliconFlow (SiliconFlow Labs Pte. Ltd.)**: A cached price appears in the listing's pricing notes, but no caching page is in the docs index and the chat reference has no cached-token field.\n- **StepFun API (STEPFUN TECHNOLOGY (HK) LIMITED)**: The host platform.stepfun.ai closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read.\n- **Tencent Cloud TokenHub (Tencent Cloud Computing (Beijing) Co., Ltd. (腾讯云计算（北京）有限责任公司))**: The host cloud.tencent.com closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read.\n\n- not this job: Cloudflare AI Gateway (Cloudflare, Inc.): Gateway caching is an exact-match response cache keyed on the whole request body, not prefix caching; a growing conversation never hits it.\n- not this job: Cohere Chat API (Command models) (Cohere Inc.): No prompt caching appears in the chat, tool, streaming or models pages read.\n- not this job: Hugging Face Inference Providers (Hugging Face, Inc.): No prompt caching, cache field or cached price appears in the router docs; any caching is the provider's and unreported.\n- not this job: Jais 3 (Mohamed bin Zayed University of Artificial Intelligence (Institute of Foundation Models), with Inception42 (a G42 company)): Caching is the serving stack's business; the cards say nothing about prefix caching or cache reporting.\n- not this job: Nemotron 3.5 Super VL (NVIDIA Corporation): No prompt caching parameter or cached-token usage field in the API reference, and the hosted trial has no token price to discount.\n- not this job: Beam 1.0 (Reflection AI): The compatibility page's feature list has no prompt caching and no cached-token usage field, and the docs index has no caching page.\n- not this job: Simplismart (Verute Technologies Private Limited): No caching parameter, price or usage field in the OpenAPI file or shared-endpoint page.\n- not this job: SWE-2 Flash (Cognition AI, Inc.): Cache read and write prices are listed for use inside Devin ($0.01 and $0.125 per million) but there is no API to control or observe caching.\n\n6 listings in the category were added after this reading and aren't marked yet.\n\n## What stops agents finishing\n\n- prompt below the provider's minimum cacheable length caches nothing and says so nowhere\n- a timestamp or session id at the top of the prompt breaks the prefix every turn\n- automatic caching looks back a fixed number of blocks, so long loops write at a premium and never read\n- gateways that rewrite prompts or drop the cache key\n\n## Around it\n\nBefore: inference.tool-loop. After: obs.cost-tracking.\n\n## Where this comes from\n\nWeight 12 of 100 in Model APIs \u0026 inference: 3 first-hand accounts (two with production numbers) plus three secondary write-ups; every agent account resends the prefix, so the job is universal even where unnamed\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/inference.cached-long-session.json, or the MCP tool get_use_case with id inference.cached-long-session. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Pay once for a long static prefix across a session",
        "url": ""
      }
    ],
    "description": "Pay once for a long static prefix across a session: 3 companies fully do it, 23 with a limit, 42 carry the tag. Every listing in Model APIs \u0026 inference with the reason and a source.",
    "facts": null,
    "h1": "Pay once for a long static prefix across a session",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/inference.cached-long-session/",
    "published": "",
    "section": "agents",
    "title": "Pay once for a long static prefix across a session | Anchor Terminal",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/inference.cached-long-session/"
  },
  "tokens": {
    "markdown": 3100,
    "slim": 580
  },
  "version": 1
}
