Niches · Model APIs & inference

Keep an agent running through provider errors, rate limits and outages

Two or three companies' docs show a full fit.

2companies, full fit
10companies, fit with a limit
44companies carry the tag
16share of the category's work
8first-hand accounts

What counts as done

A failed primary call falls through to the next model inside one request or one harness retry, the response says which provider served it, and the agent run finishes without the raw error reaching the user.

How agents ask for it: if the model is rate limited, try another one, one key for several providers with fallback, don't let a 429 or overloaded_error kill the cron job

Does the job 2

The docs show it meets the test above, and nothing we read says otherwise.

ListingWhy
Eden AI C
Eden AI SAS
A fallbacks list of up to 7 models retries inside one request on provider outage, rate limit or error, routing.allow_fallbacks switches provider for the same model, and provider/model ids pin a host. source
LiteLLM B
Berrie AI Incorporated
fallbacks, context_window_fallbacks and content_policy_fallbacks run after num_retries with cooldowns, x-litellm-model-id names the deployment that answered, and acc-10 routes by failure class through it; a cross-provider fallback cannot decrypt the failed model's reasoning items. source

Does the job, with a limit 10

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
Antseed C
Antseed Foundation
Automatic routing fails over after retryable errors and a `pick: failover` option exists, but a pinned `<peerId>@<model>` has no failover, 402 insufficient_deposits stops the run, and a model with no free seller fails with 502. source
BlockRun.AI A
BlockRun, Inc.
The gateway fails over by itself and marks it with X-Fallback-Used, X-Original-Model and X-Fallback-Model on streaming and non-streaming responses, but the caller cannot set a model chain and the triggering errors are not listed; 503 says retry or fail over. source
Cloudflare AI Gateway B
Cloudflare, Inc.
Universal endpoint fallbacks fire on any request error or timeout and cf-aig-step says which provider answered, and dynamic routes add rate and budget fallbacks; but the Universal endpoint is deprecated, dynamic routes need authentication plus stored BYOK keys, and mid-stream behaviour is not stated. source
DeepInfra B
Deep Infra Inc.
No fallback list on the chat page read; fail_fast true turns a capacity queue into an immediate 429 so a client can fail over elsewhere, and service_tier in the response says which tier served. source
Glimpse F
Elorian AI, Inc.
OpenRouter's models array falls back to another model on any error, but Glimpse has one endpoint, so provider failover has nowhere to go and a fallback means a different model. source
Hugging Face Inference Providers C
Hugging Face, Inc.
:fastest, :cheapest and :preferred policies choose a live provider per request, but nothing read says a failed call falls through to another provider inside one request, and no 429 or error format is documented for the router. source
OpenRouter B
OpenRouter, Inc.
models (or fallbacks, max 3) retries on any error including rate limits and moderation, and the chunk names the provider; once headers are committed errors arrive inside a 200 stream, and acc-01, acc-02 and acc-21 report failover not firing or shape differences. source
Router.com (Ramp Router) C
Ramp Business Corporation
models (up to 15) on /v1/responses advances on unknown or retired model, 429, 5xx, network failure or pre-stream failure and names the provider; it stops once streaming starts, and truncated attempts are billed. source
RouterPlus D
EVO HQ Inc
Fails over between providers of one model before output starts, with x-tm-attempts and x-tm-provider naming the server; it never fails over to another model, and after output starts a failure ends the stream. source
Vercel AI Gateway B
Vercel Inc.
Default routing fails over across a model's providers and a models array tries backups in order, with modelAttempts and resolvedProvider naming what served; whether fallback continues after streaming starts isn't stated. source

Couldn't check 2

The docs were closed to us, or silent. These carry no mark either way.

ListingWhy
StepFun API E
STEPFUN TECHNOLOGY (HK) LIMITED
The host platform.stepfun.ai closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read.
Tencent Cloud TokenHub C
Tencent Cloud Computing (Beijing) Co., Ltd. (腾讯云计算(北京)有限责任公司)
The host cloud.tencent.com closed the connection on every attempt (curl exit 56, no HTTP status) from this session, so no page was read.

Doesn't do this job 27

Listings in the category that do something else, with the reason.

ListingWhy
Amazon Bedrock A
Amazon Web Services, Inc.
No fallback or models parameter; cross-Region inference profiles spread capacity for one model and do not fall through to another model.
Claude API A
Anthropic, PBC
Single-provider API with no models array or fallback parameter; a spend-cap 429 carries no retry-after and SDK retries fail until the month resets.
Azure OpenAI in Foundry Models A
Microsoft Corporation
No model fallback parameter; spillover only routes a provisioned deployment's overflow to a standard deployment of the same model in the same resource. source
Cerebras Inference B
Cerebras Systems Inc.
Single host with no fallback or models parameter.
Cloudflare Workers AI BB
Cloudflare, Inc.
No fallback parameter; failover is AI Gateway's job.
Cohere Chat API (Command models) B
Cohere Inc.
Single-vendor API with no fallback parameter.
DeepSeek API D
Hangzhou DeepSeek Artificial Intelligence Co., Ltd.
Single-vendor API with no fallback parameter; 429 and 503 carry no Retry-After.
Fireworks AI Inference BB
Fireworks.ai, Inc.
Single host with no fallback or models parameter.
Gemini Developer API B
Google LLC
Single-vendor API with no fallback parameter; the Gemini CLI's Pro-to-Flash switch in acc-16 was inconsistent and lives in the harness.
GroqCloud BB
Groq LLC
Single host with no fallback or models parameter; 498 capacity_exceeded on Flex is a retry signal, not a fallback.
Jais 3 C
Mohamed bin Zayed University of Artificial Intelligence (Institute of Foundation Models), with Inception42 (a G42 company)
Self-hosted weights; no router.
Meta Model API D
Meta Platforms, Inc.
Single-vendor API with no fallback parameter; service_overloaded 429s carry Retry-After 60 and are retryable.
MiniMax API C
Nanonoble Pte. Ltd.
Single-vendor API with no models list or fallback parameter; 429, 529 and the 1002 and 2064 retry-later codes are returned for the caller to handle. source
Mistral AI API B
Mistral AI (RCS Paris 952 418 325)
Single vendor; the error glossary gives Retry-After on 429 and backoff advice, with no models list or fallback parameter. source
Muse Spark Open C
Meta Platforms, Inc.
Self-hosted weights; no router, models list or fallback. source
Nemotron 3.5 Super VL C
NVIDIA Corporation
Single hosted endpoint with no fallback; the FAQ says request rates vary with other users and gives no 429 contract. source
Novita AI C
Novita AI
Single provider; errors come back as codes such as RATE_LIMIT_EXCEEDED with no fallback parameter. source
OpenAI API AA
OpenAI OpCo, LLC
No fallback parameter; 429 slow_down and 503 server_is_overloaded carry Retry-After, and acc-03 and acc-10 describe rate-limit and billing failures killing agent runs until a harness-side fallback was added. source
Prism Inference C
Prism Technologies Inc
Single provider; a 429 covers both key limits and model capacity with Retry-After and a retryable flag, and there is no fallback list. source
Beam 1.0 C
Reflection AI
Single provider; 429 and 503 carry Retry-After and x-should-retry, with no fallback list. source
SambaCloud B
SambaNova Systems, Inc.
Single provider with headers for remaining requests; no fallback list. source
SiliconFlow E
SiliconFlow Labs Pte. Ltd.
Single provider; 429 messages name the limit type and 503 or 504 mean high load, with no fallback list. source
Simplismart F
Verute Technologies Private Limited
Single provider; no fallback, 429 format or retry guidance documented. source
SWE-2 Flash F
Cognition AI, Inc.
No API; model choice is a CLI flag or slash command. source
Together AI Inference B
Together Computer, Inc.
Single provider; 429 and 503 mean high demand with backoff advice and no fallback list. source
Vertex AI (Gemini Enterprise Agent Platform) AA
Google LLC
No fallback inside the platform; a 429 'Resource exhausted' on Standard PayGo means shared capacity, and the guide's remedies are backoff, the global endpoint or Provisioned Throughput. source
xAI API (Grok) BB
SpaceXSI (xAI)
Single provider; 429 is 'sending requests too frequently' with no Retry-After documented and no fallback list. source

6 listings in the category were added after this reading and aren't marked yet.

What stops agents finishing

  • router returns a generic error string the harness cannot classify, so no fallback is tried
  • fallback stops once a stream has started
  • spend cap or billing failure returns an error the harness retries instead of stopping
  • free-tier models all return 429 under a burst, so a chain of free models is no chain

Around it

Before: inference.tool-loop. After: obs.gateway, obs.cost-tracking.

Where this comes from

Weight 16 of 100 in Model APIs & inference: 8 of 28 accounts are about fallback that did or did not fire; OpenClaw's issue tracker lists broken failover as the top OpenRouter complaint

The same as data: /api/v1/use-cases/inference.failover-routing.json, or the MCP tool get_use_case with id inference.failover-routing. Other jobs in this category: /api/v1/niches?category=inference.

Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.