{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/inference.failover-routing/",
    "json": "https://www.anchorterminal.com/niches/inference.failover-routing/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/inference.failover-routing/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/inference.failover-routing/index.min.md"
  },
  "markdown": "Two or three companies' docs show a full fit.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 2 | 10 | 44 | 16 | 8 |\n\n## What counts as done\n\nA failed primary call falls through to the next model inside one request or one harness retry, the response says which provider served it, and the agent run finishes without the raw error reaching the user.\n\nHow agents ask for it: if the model is rate limited, try another one; one key for several providers with fallback; don't let a 429 or overloaded_error kill the cron job\n\n## Does the job\n\nThe docs show it meets the test above, and nothing we read says otherwise.\n\n- **Eden AI (Eden AI SAS)**: A fallbacks list of up to 7 models retries inside one request on provider outage, rate limit or error, routing.allow_fallbacks switches provider for the same model, and provider/model ids pin a host.\n- **LiteLLM (Berrie AI Incorporated)**: fallbacks, context_window_fallbacks and content_policy_fallbacks run after num_retries with cooldowns, x-litellm-model-id names the deployment that answered, and acc-10 routes by failure class through it; a cross-provider fallback cannot decrypt the failed model's reasoning items.\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **Antseed (Antseed Foundation)**: Automatic routing fails over after retryable errors and a `pick: failover` option exists, but a pinned `\u003cpeerId\u003e@\u003cmodel\u003e` has no failover, 402 insufficient_deposits stops the run, and a model with no free seller fails with 502.\n- **BlockRun.AI (BlockRun, Inc.)**: The gateway fails over by itself and marks it with X-Fallback-Used, X-Original-Model and X-Fallback-Model on streaming and non-streaming responses, but the caller cannot set a model chain and the triggering errors are not listed; 503 says retry or fail over.\n- **Cloudflare AI Gateway (Cloudflare, Inc.)**: Universal endpoint fallbacks fire on any request error or timeout and cf-aig-step says which provider answered, and dynamic routes add rate and budget fallbacks; but the Universal endpoint is deprecated, dynamic routes need authentication plus stored BYOK keys, and mid-stream behaviour is not stated.\n- **DeepInfra (Deep Infra Inc.)**: No fallback list on the chat page read; fail_fast true turns a capacity queue into an immediate 429 so a client can fail over elsewhere, and service_tier in the response says which tier served.\n- **Glimpse (Elorian AI, Inc.)**: OpenRouter's models array falls back to another model on any error, but Glimpse has one endpoint, so provider failover has nowhere to go and a fallback means a different model.\n- **Hugging Face Inference Providers (Hugging Face, Inc.)**: :fastest, :cheapest and :preferred policies choose a live provider per request, but nothing read says a failed call falls through to another provider inside one request, and no 429 or error format is documented for the router.\n- **OpenRouter (OpenRouter, Inc.)**: models (or fallbacks, max 3) retries on any error including rate limits and moderation, and the chunk names the provider; once headers are committed errors arrive inside a 200 stream, and acc-01, acc-02 and acc-21 report failover not firing or shape differences.\n- **Router.com (Ramp Router) (Ramp Business Corporation)**: models (up to 15) on /v1/responses advances on unknown or retired model, 429, 5xx, network failure or pre-stream failure and names the provider; it stops once streaming starts, and truncated attempts are billed.\n- **RouterPlus (EVO HQ Inc)**: Fails over between providers of one model before output starts, with x-tm-attempts and x-tm-provider naming the server; it never fails over to another model, and after output starts a failure ends the stream.\n- **Vercel AI Gateway (Vercel Inc.)**: Default routing fails over across a model's providers and a models array tries backups in order, with modelAttempts and resolvedProvider naming what served; whether fallback continues after streaming starts isn't stated.\n\n## Couldn't check\n\nThe docs were closed to us, or silent. These carry no mark either way.\n\n- **StepFun API (STEPFUN TECHNOLOGY (HK) LIMITED)**: The host platform.stepfun.ai closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read.\n- **Tencent Cloud TokenHub (Tencent Cloud Computing (Beijing) Co., Ltd. (腾讯云计算（北京）有限责任公司))**: The host cloud.tencent.com closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read.\n\n- not this job: Amazon Bedrock (Amazon Web Services, Inc.): No fallback or models parameter; cross-Region inference profiles spread capacity for one model and do not fall through to another model.\n- not this job: Claude API (Anthropic, PBC): Single-provider API with no models array or fallback parameter; a spend-cap 429 carries no retry-after and SDK retries fail until the month resets.\n- not this job: Azure OpenAI in Foundry Models (Microsoft Corporation): No model fallback parameter; spillover only routes a provisioned deployment's overflow to a standard deployment of the same model in the same resource.\n- not this job: Cerebras Inference (Cerebras Systems Inc.): Single host with no fallback or models parameter.\n- not this job: Cloudflare Workers AI (Cloudflare, Inc.): No fallback parameter; failover is AI Gateway's job.\n- not this job: Cohere Chat API (Command models) (Cohere Inc.): Single-vendor API with no fallback parameter.\n- not this job: DeepSeek API (Hangzhou DeepSeek Artificial Intelligence Co., Ltd.): Single-vendor API with no fallback parameter; 429 and 503 carry no Retry-After.\n- not this job: Fireworks AI Inference (Fireworks.ai, Inc.): Single host with no fallback or models parameter.\n- not this job: Gemini Developer API (Google LLC): Single-vendor API with no fallback parameter; the Gemini CLI's Pro-to-Flash switch in acc-16 was inconsistent and lives in the harness.\n- not this job: GroqCloud (Groq LLC): Single host with no fallback or models parameter; 498 capacity_exceeded on Flex is a retry signal, not a fallback.\n- not this job: Jais 3 (Mohamed bin Zayed University of Artificial Intelligence (Institute of Foundation Models), with Inception42 (a G42 company)): Self-hosted weights; no router.\n- not this job: Meta Model API (Meta Platforms, Inc.): Single-vendor API with no fallback parameter; service_overloaded 429s carry Retry-After 60 and are retryable.\n- not this job: MiniMax API (Nanonoble Pte. Ltd.): Single-vendor API with no models list or fallback parameter; 429, 529 and the 1002 and 2064 retry-later codes are returned for the caller to handle.\n- not this job: Mistral AI API (Mistral AI (RCS Paris 952 418 325)): Single vendor; the error glossary gives Retry-After on 429 and backoff advice, with no models list or fallback parameter.\n- not this job: Muse Spark Open (Meta Platforms, Inc.): Self-hosted weights; no router, models list or fallback.\n- not this job: Nemotron 3.5 Super VL (NVIDIA Corporation): Single hosted endpoint with no fallback; the FAQ says request rates vary with other users and gives no 429 contract.\n- not this job: Novita AI (Novita AI): Single provider; errors come back as codes such as RATE_LIMIT_EXCEEDED with no fallback parameter.\n- not this job: OpenAI API (OpenAI OpCo, LLC): No fallback parameter; 429 slow_down and 503 server_is_overloaded carry Retry-After, and acc-03 and acc-10 describe rate-limit and billing failures killing agent runs until a harness-side fallback was added.\n- not this job: Prism Inference (Prism Technologies Inc): Single provider; a 429 covers both key limits and model capacity with Retry-After and a retryable flag, and there is no fallback list.\n- not this job: Beam 1.0 (Reflection AI): Single provider; 429 and 503 carry Retry-After and x-should-retry, with no fallback list.\n- not this job: SambaCloud (SambaNova Systems, Inc.): Single provider with headers for remaining requests; no fallback list.\n- not this job: SiliconFlow (SiliconFlow Labs Pte. Ltd.): Single provider; 429 messages name the limit type and 503 or 504 mean high load, with no fallback list.\n- not this job: Simplismart (Verute Technologies Private Limited): Single provider; no fallback, 429 format or retry guidance documented.\n- not this job: SWE-2 Flash (Cognition AI, Inc.): No API; model choice is a CLI flag or slash command.\n- not this job: Together AI Inference (Together Computer, Inc.): Single provider; 429 and 503 mean high demand with backoff advice and no fallback list.\n- not this job: Vertex AI (Gemini Enterprise Agent Platform) (Google LLC): No fallback inside the platform; a 429 'Resource exhausted' on Standard PayGo means shared capacity, and the guide's remedies are backoff, the global endpoint or Provisioned Throughput.\n- not this job: xAI API (Grok) (SpaceXSI (xAI)): Single provider; 429 is 'sending requests too frequently' with no Retry-After documented and no fallback list.\n\n6 listings in the category were added after this reading and aren't marked yet.\n\n## What stops agents finishing\n\n- router returns a generic error string the harness cannot classify, so no fallback is tried\n- fallback stops once a stream has started\n- spend cap or billing failure returns an error the harness retries instead of stopping\n- free-tier models all return 429 under a burst, so a chain of free models is no chain\n\n## Around it\n\nBefore: inference.tool-loop. After: obs.gateway, obs.cost-tracking.\n\n## Where this comes from\n\nWeight 16 of 100 in Model APIs \u0026 inference: 8 of 28 accounts are about fallback that did or did not fire; OpenClaw's issue tracker lists broken failover as the top OpenRouter complaint\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/inference.failover-routing.json, or the MCP tool get_use_case with id inference.failover-routing. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Keep an agent running through provider errors, rate limits and outages",
        "url": ""
      }
    ],
    "description": "Keep an agent running through provider errors, rate limits and outages: 2 companies fully do it, 10 with a limit, 44 carry the tag. Every listing in Model APIs \u0026 inference with the reason and a source.",
    "facts": null,
    "h1": "Keep an agent running through provider errors, rate limits and outages",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/inference.failover-routing/",
    "published": "",
    "section": "agents",
    "title": "Keep an agent running through provider errors, rate limits and…",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/inference.failover-routing/"
  },
  "tokens": {
    "markdown": 2700,
    "slim": 380
  },
  "version": 1
}
