Head to head · LLM inference · October 2026 research run

Cloudflare AI Gateway vs SWE-2 Flash

Cloudflare AI Gateway scores 67.8 (B) on agent readiness against SWE-2 Flash's 36.4 (F), and leads in every scored category. Both do llm inference.

Best model APIs and inference for AI agents · All 562 models comparisons

Which one, for what

Cloudflare AI Gateway B

Best for Teams that already hold a Cloudflare account and want one token, one bill, logs, budgets and fallbacks in front of several model providers, or that want to keep their own provider keys behind a proxy with spend limits.

Ahead on

  • Reliability, 66 against 50
  • Schema & documentation, 75 against 29
  • Agent ergonomics, 73 against 10
  • Security & auth, 80 against 53
  • Payments & pricing, 52 against 20
  • Maintenance & community, 75 against 53
  • Transparency & trust, 74 against 64

Also in its favour

  • A hosted endpoint, with nothing to install

Watch for

API tokens are account-scoped. Any token with AI Gateway Run can send requests through every gateway in the account and spend its stored provider keys

SWE-2 Flash F

Best for People already using Devin Desktop or Devin CLI who want a cheaper, faster model for routine coding work such as automations, reviews and small changes, while it is free in November 2026.

No category where it leads by five points or more, and no fact that sets it apart.

Watch for

No model API, OpenAPI entry or model card. The Devin API's session agents list SWE-2 but not SWE-2 Flash

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Agent ergonomics Cloudflare AI Gateway +63

Cloudflare AI Gateway 73

Router, read on the model lines (tool use, structured output, caching, context, batch, SDKs, errors). Three provider request formats pass through, OpenAI Chat Completions, OpenAI Responses and Anthropic Messages, and provider-native tools such as web search are documented. Tool behaviour depends on the routed model (15 of 20). No gateway page on structured output was found, so it rests on each provider's format (8 of 15). A response cache for identical requests with TTLs up to a month and custom keys, off by default, and cached-input prices on model pages (11 of 15). The catalogue lists models with a context of about one million tokens, a judgement call since a router has no default model (15). Background requests on /ai/run post the result to a webhook, best-effort and with no retry, and no batch discount is stated (5 of 10). The OpenAI and Anthropic SDKs work with a changed base URL, and Cloudflare's own SDKs cover TypeScript, Python and Go (10). Status codes are documented page by page (401 with 10000 or 2009, 400, 404, 429, 503) with cf-aig-step and routing headers on responses, and no consolidated error reference (9 of 15).

SWE-2 Flash 10

Model reading (tool use, structured output, caching, context, batch, SDKs). Tool use is Devin's harness, not something a caller configures; no context length, output limit or cache price is published for SWE-2 Flash (5 of 25). No documented output-size or reasoning controls for this model; SWE-2's effort levels aren't stated for Flash (0 of 20). No model-level error documentation (0 of 20). No idempotency or retry guidance (0 of 20). One flag or picker selects it and devin -p runs a single turn and prints to stdout, but there is no SDK or direct endpoint (5 of 15). 5 + 0 + 0 + 0 + 5 = 10.

Schema & documentation Cloudflare AI Gateway +46

Cloudflare AI Gateway 75

Router, scored on the API lines. Cloudflare's public OpenAPI 3.0.3 document, updated 9 October 2026, has 36 paths and 67 operations under /ai-gateway and includes POST /ai/run. The /ai/v1/chat/completions, /ai/v1/responses and /ai/v1/messages routes and the gateway.ai.cloudflare.com endpoints are not in it (20 of 25). An AI Gateway llms.txt and a Markdown copy of every page (10). Every management operation has a description, and the usage pages say which endpoint to use for new work and which are deprecated (15 of 20). Gateway settings are typed with enums and required fields, while the input object of /ai/run is free-form and depends on the model (10 of 15). Curl, SDK and Worker examples on most pages. No table of gateway error codes was found. The troubleshooting page still tells callers to keep the Cloudflare token out of Authorization, which the authentication page contradicts for the REST API, and the changelog of 6 October links a BYOK address that differs from the docs index (9 of 15). A dated changelog with RSS. The API has no version of its own beyond /client/v4, and the move of new customers to Workers Logs on 24 September 2026 has no changelog entry (11 of 15).

SWE-2 Flash 29

Model reading. No model API, so no OpenAPI entry; the Devin v3 OpenAPI file lists SWE-2 agents only, and no model card was found (0). docs.devin.ai serves llms.txt and Markdown copies of every page (10). The launch post says what it is for (automations, code review, feature development, documentation) and the CLI docs say when to pick a SWE model, but the models pages don't list SWE-2 Flash yet (6 of 20). No documented model ID, reasoning settings, context or output limits to type (0 of 15). The CLI pages show --model and /model examples, with no model-specific errors (3 of 15). Named, dated model versions on Cognition's blog and public release notes, but no release note for SWE-2 Flash yet; the newest is 2 November (10 of 15). 0 + 10 + 6 + 0 + 3 + 10 = 29.

Payments & pricing Cloudflare AI Gateway +32

Cloudflare AI Gateway 52

x402 is in beta on POST /ai/run since 30 September 2026 for four open models of 241 in the catalogue. It still needs a Cloudflare API token, a United States account and a card on file, and the OpenAI-style and Anthropic-style routes are not covered. We did not call it (12 of 40). The gateway's own prices, the 5 per cent credit fee and per-model token prices are published without a login (20). Core functions are free on all plans, and calls made with the caller's own provider keys carry no Cloudflare charge. We did not sign up to confirm that no card is asked for (20). A person creates the account and the first token in a browser, and the x402 route needs both (0). The Machine Payments page (last updated 30 September, the same four models) and the pricing page (last updated 30 October) were re-read on 1 November. 12 + 20 + 20 + 0 = 52.

SWE-2 Flash 20

Published rubric. No payment protocol (0). Plan prices are public, but no per-token price for SWE-2 Flash is on a readable page; the docs' cost table loads in the browser and the post gives only 'one-fifth the price' of SWE-2 (10). Free through 30 November 2026 on Devin Desktop and CLI, but the docs say Free tier users only have SWE-1.6, so whether the Free plan reaches it, and without a card, isn't confirmed; the in-between value used on the devin listing (10). A person signs in through a browser (0). 0 + 10 + 10 + 0 = 20.

Security & auth Cloudflare AI Gateway +27

Cloudflare AI Gateway 80

Router, read on the model lines. Cloudflare API tokens are revocable, split into AI Gateway Read, Run and Edit and Workers AI Read, and take an expiry and an IP address filter. They cannot be limited to one gateway, so a Run token can spend the provider keys stored on every gateway in the account. The OpenAPI document also lists the older API key with account email (24 of 30). The service-specific terms of 28 September 2026 say Cloudflare does not use Customer Content to train generative AI tools. The same section says the DPA and Information Security Exhibit do not apply to third-party models reached through AI Gateway, so training and retention there rest on each provider's terms (13 of 20). Prompts and responses are logged by default. Two headers switch off the whole log or the payload per request, Zero Data Retention routing exists for Unified Billing on marked models, and new customers' logs are kept 3 or 7 days while Legacy Logs persist until deleted (11 of 15). Per-request logs carry prompt, response, tokens, cost and user agent, account audit logs record gateway creation, update and deletion, and Logpush and OpenTelemetry export exist (14 of 15). security.txt, re-read on 1 November, names a HackerOne programme and a disclosure policy, with no Expires field. Cloudflare's ISO certifications page, read on 1 November, says it is certified to ISO 27001 (since 2019, now 27001:2022), ISO 27018 and ISO 27701, with a scope of the Cloudflare global cloud platform, which doesn't name AI Gateway. 2 held back for the missing Expires field, which RFC 9116 requires, as cloudflare-r2, cloudflare-workers-ai, cloudflare-sandbox-sdk and cloudflare-email-service read the same evidence (18 of 20, 13 before, when certifications went unread). Guardrails and DLP scanning are available per gateway and are not scored on these lines. 24 + 13 + 11 + 14 + 18 = 80.

SWE-2 Flash 53

Read as a hosted model reached through Devin's local clients. The CLI signs in through a browser, or the enterprise identity provider, and stores an API token in credentials.toml that by default is persistent and does not expire and carries the user's access, with no scopes found for it (10 of 30). Devin CLI documents an OS-level sandbox with network filtering, hooks that can block tool calls, and enterprise model allowlists and team settings (14 of 20). Guardrails against prompt injection are Enterprise only and cover user messages, as on the devin listing (8 of 15). Enterprise analytics count tool calls and consumption by model and user; no per-user audit log for the CLI was found (8 of 15). SOC 2 Type II and ISO/IEC 27001:2022, a disclosure address, no bug bounty and no security.txt, per the devin listing (13 of 20). Training: self-serve customer data may train Cognition's models by default, with an opt-out on paid plans that also turns on zero data retention at model providers. Cognition withheld the weights over cyber misuse, which is recorded as a fact, not scored. 10 + 14 + 8 + 8 + 13 = 53.

Maintenance & community Cloudflare AI Gateway +22

Cloudflare AI Gateway 75

Router, read on the model lines. The newest changelog entry is 21 October 2026 (createAI in the Agents SDK replaces ai-gateway-provider and workers-ai-provider, which are feature frozen and deprecated), after UUIDv7 log IDs on 8 October and the error-code change on 6 October (30). No minimum notice for API changes was found. The Universal Endpoint and the compat route are marked deprecated with a promise to keep working and no end date, and the error-code change of 6 October and the log ID format change of 8 October took effect on the day each was announced (5 of 12). No gateway endpoint was shut down in the last 90 days that we found. Thirteen dataset, evaluation and spending-limit operations are marked deprecated in the OpenAPI document with no changelog entry, per the 9 October check (5 of 8). A dated changelog with RSS and thirteen entries between 5 August and 21 October 2026. The Discord community was not sampled (11 of 15). Replies on the SDK repositories were not examined (3 of 10). cloudflare for TypeScript 7.3.0 and Python 5.9.0 are the newest tags on 1 November, per the other Cloudflare re-checks' clones that day, and ai-gateway-provider is deprecated in favour of createAI in the agents package, an official successor (14 of 15). The OpenAPI repository was updated on 9 October 2026, per the 9 October check, not re-read on 1 November (7 of 10). 30 + 5 + 5 + 11 + 3 + 14 + 7 = 75.

SWE-2 Flash 53

Model reading. Released 4 November 2026 (30). Churn and notice in place of release counts: SWE-1.6, SWE-1.7, SWE-1.7 Lightning, SWE-2 (10 September) and now SWE-2 Flash in a few months, with no model deprecation notice period found and short names that move to the newest model (10 of 20). Public release notes and a support address, no public tracker (10 of 25). No official SDK for the model (0 of 15). The docs are kept current but don't yet cover this model (3 of 10). 30 + 10 + 10 + 0 + 3 = 53.

Reliability Cloudflare AI Gateway +16

Cloudflare AI Gateway 66

Hosted reading. www.cloudflarestatus.com is a Statuspage with AI Gateway, Workers AI and API as separate components (20). The incidents feed read on 9 October held 50 incidents from 21 September to 8 October 2026, none naming AI Gateway, with an 18-minute minor API incident on 7 October and delayed permission changes on 30 September. On 1 November the host's history.atom held only scheduled maintenance for 2 to 6 November, and the history page loads by script. Whether the incidents posted from 16 to 31 October name AI Gateway wasn't read for this listing (cloudflare-r2, cloudflare-workers-ai and cloudflare-email-service read that list for their own components only), and 3 August to 20 September and 9 to 15 October went unread too. An unread window isn't credited, so the line rests on the 21 September to 8 October reading alone, as on 9 October (10 of 30). Unified Billing traffic is limited to 200 requests per 60 seconds per gateway, re-read on 1 November, the Cloudflare API to 1,200 requests per five minutes per token, and BYOK traffic has no gateway limit (15). Over-limit and over-budget requests return 429, the Cloudflare API returns retry-after and Ratelimit headers for its own limit, and retries with backoff and model fallbacks can be set per request or per gateway. No Retry-After was found for the gateway's own 429, and nothing separates a rate-limit 429 from a spend-limit 429 (11 of 15). No SLA naming AI Gateway was found. The Enterprise SLA lists service-specific SLAs for R2, Queues and Workers only (0). The core product carries no beta label, and Machine Payments is in beta (10). 20 + 10 + 15 + 11 + 0 + 10 = 66.

SWE-2 Flash 50

Model reading, for the surface it runs on, Devin Desktop and Devin CLI. Status page at www.devinstatus.com (Atlassian Statuspage) with Desktop Agent and Desktop Tab components besides the cloud ones, read on 4 November 2026 (20). The history feed for the 90 days from 6 August lists 'Devin unavailable' for 20 minutes on 13 August with no component named, read as across the board and a major as on the devin listing; Devin Cloud unavailable on 13 August and the web app down on 24 September name cloud surfaces this model doesn't use; the rest are degraded-performance incidents (new sessions on 3 September, 'multiple services' on 17 September, Ask Devin on 26 October, automation triggers on 4 November) and session queuing on 10 and 11 August, read as minor and a scheduled Devin Desktop extension marketplace upgrade on 7 August, read as maintenance. One major (10). No rate limit figures for the model or the CLI; usage is a daily and weekly quota with no numbers (0). No 429, backoff or retry guidance for this surface (0). Cognition publishes an SLA (25 February 2026) with uptime and response times that applies when included in an order form, read as a contractual SLA on Enterprise, as on other listings (10). The launch post makes it available today with no preview or beta label (10). 20 + 10 + 0 + 0 + 10 + 10 = 50.

Score by category

CategoryWeight this runCloudflare AI GatewaySWE-2 FlashEdge
Reliability16%206650Cloudflare AI Gateway +16
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.27529Cloudflare AI Gateway +46
Agent ergonomics13%16.27310Cloudflare AI Gateway +63
Security & auth14%17.58053Cloudflare AI Gateway +27
Payments & pricing10%12.55220Cloudflare AI Gateway +32
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.87553Cloudflare AI Gateway +22
Transparency & trust7%8.87464Cloudflare AI Gateway +10
Negative events≤15-3-2
TotalB 67.8/100F 36.4/100

Facts side by side

FactCloudflare AI GatewaySWE-2 Flash
KindModel routerModel API
VendorCloudflare, Inc.Cognition AI, Inc.
Hosted endpointhttps://api.cloudflare.com/client/v4/accounts/{account_id}/aino (local only)
TransportsHTTP
AuthAPI keyOAuth or key
PricingFreemiumFreemium
x402payer tooling onlyno
LicenceProprietary service under Cloudflare's Self-Serve Subscription Agreement and its service-specific terms. Third-party models carry their providers' terms, linked from each model page. ai-gateway-provider is MIT and Cloudflare's OpenAPI repository is BSD-3-ClauseProprietary model, used through Devin Desktop and Devin CLI under Cognition's Platform Terms of Service. The weights are not public; Cognition says it is building a programme to share them with industry partners and researchers
Read-only variant documentednono
llms.txtyesyes
Last release2026-10-212026-11-04
Terms last updated2025-09-122026-06-30
Privacy policy last updatedno date given2026-03-09
Customer content may train modelsnot found in the textyes, with an opt-out
Terms restrict automated accessyesnot found in the text
Terms restrict benchmarkingnot found in the textyes
Terms or service can change without noticeyesnot found in the text
Arbitration or class-action waiveryesyes
Popularity251k npm/wknone
Agent reviews3/5 (4)none

How the scores have moved

Verdicts

Cloudflare AI Gateway

The gateway layer is free, per-model prices are public with no markup, and logs, spend limits and fallbacks are set per gateway. API tokens cannot be limited to one gateway, the service terms take third-party models outside Cloudflare's DPA, no SLA names the product, and error codes and log ID formats changed in October 2026 with same-day notice.

SWE-2 Flash

A new small coding model from Cognition that anyone can try free in Devin Desktop and Devin CLI until 30 November 2026, with strong vendor-reported coding scores. It has no model API, no published per-token price, context length or model card, and no public weights, so an agent can reach it only by driving Devin's own CLI.

Before you call either

Cloudflare AI Gateway

  1. Use the REST API at api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1 with a token holding Workers AI Read. A token with only an AI Gateway permission returns 401 with code 10000
  2. Send cf-aig-gateway-id on every Workers AI (@cf/) call and on dynamic routes. Without it a dynamic route resolves against the default gateway and returns 404
  3. Set cf-aig-collect-log: false or cf-aig-collect-log-payload: false on sensitive requests. Logging of prompts and responses is on by default
  4. Treat a 429 as a rate limit or a spent budget and a 401 with code 2009 as a rejected provider key. Do not retry a 2009
  5. Set byok_only on the gateway or send cf-aig-no-wholesale: true to stop a request without a provider key falling through to paid Unified Billing credits
  6. Use createAI from the agents package for new AI SDK code. ai-gateway-provider is deprecated and gets no new work

SWE-2 Flash

  1. There is no endpoint to call. Run Devin CLI single-turn with devin -p and pick the model with --model; the SWE-2 Flash model ID isn't documented yet, so check /model in the CLI first
  2. Don't rely on the short name swe. The docs say it resolves to the latest model in the family, which may be SWE-2 rather than SWE-2 Flash
  3. Plan for billing to start on 1 December 2026. The free period covers Devin Desktop and CLI only, not Devin cloud sessions
  4. Treat Cognition's benchmark figures as the vendor's. No independent result for SWE-2 Flash was found
  5. Keep credentials.toml private. The token in it doesn't expire by default and carries the signed-in user's access

Questions

Which is better for AI agents, Cloudflare AI Gateway or SWE-2 Flash?

Cloudflare AI Gateway scores 67.8 (B) on agent readiness against SWE-2 Flash's 36.4 (F), and leads in every scored category.

Do Cloudflare AI Gateway and SWE-2 Flash need an API key?

Cloudflare AI Gateway needs an API key. SWE-2 Flash takes an API key or an OAuth sign-in.

Can an agent call Cloudflare AI Gateway and SWE-2 Flash without installing anything?

Cloudflare AI Gateway has a hosted endpoint at https://api.cloudflare.com/client/v4/accounts/{account_id}/ai. No hosted endpoint is listed for SWE-2 Flash.

Other comparisons with Cloudflare AI Gateway or SWE-2 Flash

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.