Fireworks AI Inference vs SWE-2 Flash

Fireworks AI Inference 80

Read by the HTTP API checklist, as for cerebras-inference and together-inference. Compact model API with bounded output, max_tokens and streaming (25). Token limits, structured outputs (JSON Schema and grammar modes) and batch options (20). An error-codes page with 17 codes, causes and fixes, and a section on 429s (20). Retry guidance without a general deduplication guarantee (5 of 20). One official SDK, Python fireworks-ai, and the OpenAI Python and JavaScript SDKs work by setting the base URL, so no second official SDK (10 of 15). Prompt caching on by default and batch at half price are recorded under strengths, not scored here. 25+20+20+5+10 = 80.

SWE-2 Flash 10

Model reading (tool use, structured output, caching, context, batch, SDKs). Tool use is Devin's harness, not something a caller configures; no context length, output limit or cache price is published for SWE-2 Flash (5 of 25). No documented output-size or reasoning controls for this model; SWE-2's effort levels aren't stated for Flash (0 of 20). No model-level error documentation (0 of 20). No idempotency or retry guidance (0 of 20). One flag or picker selects it and devin -p runs a single turn and prints to stdout, but there is no SDK or direct endpoint (5 of 15). 5 + 0 + 0 + 0 + 5 = 10.

Fireworks AI Inference 93

Public chat and completion OpenAPI 3.1 at text-completion.openapi.yaml, re-read on 1 November (25). llms.txt with Markdown twins of every page (10). Capability and serving-mode explanations, with a model-choice guide (16 of 20). Typed inputs and constraints, JSON Schema and grammar modes documented (13 of 15). Examples and an error-codes page with resolutions (14 of 15). Versioned endpoints and a dated changelog, newest entry 27 October (15).

SWE-2 Flash 29

Model reading. No model API, so no OpenAPI entry; the Devin v3 OpenAPI file lists SWE-2 agents only, and no model card was found (0). docs.devin.ai serves llms.txt and Markdown copies of every page (10). The launch post says what it is for (automations, code review, feature development, documentation) and the CLI docs say when to pick a SWE model, but the models pages don't list SWE-2 Flash yet (6 of 20). No documented model ID, reasoning settings, context or output limits to type (0 of 15). The CLI pages show --model and /model examples, with no model-specific errors (3 of 15). Named, dated model versions on Cognition's blog and public release notes, but no release note for SWE-2 Flash yet; the newest is 2 November (10 of 15). 0 + 10 + 6 + 0 + 3 + 10 = 29.

Fireworks AI Inference 77

Read by the HTTP API checklist, as for cerebras-inference and together-inference. Service accounts with API keys issued on their behalf, roles assigned at creation and changeable (30). An inference-user role can call models but not create or modify resources (20). No inference prompt-injection mitigation established (0). Enterprise audit logs with a REST API, and cost exports per API key (15). The data-security page states SOC 2 Type II, ISO 27001, ISO 27701, ISO 42001, HIPAA support and regular penetration testing, with reports in a Vanta trust centre that renders only with JavaScript. fireworks.ai/.well-known/security.txt answered 404 on 1 November, as on 19 October, and no disclosure policy or bug bounty was found. That matches people-data-labs (12) and launchdarkly (12), which state SOC 2 Type II and ISO 27001 with no security.txt, disclosure policy or bounty, so the 20 the 19 October check gave is corrected (12). The terms (section 3.6) rule out training on customer content and inference has zero retention by default, recorded for reviewers and not scored on this checklist. 30+20+0+15+12 = 77.

SWE-2 Flash 53

Read as a hosted model reached through Devin's local clients. The CLI signs in through a browser, or the enterprise identity provider, and stores an API token in credentials.toml that by default is persistent and does not expire and carries the user's access, with no scopes found for it (10 of 30). Devin CLI documents an OS-level sandbox with network filtering, hooks that can block tool calls, and enterprise model allowlists and team settings (14 of 20). Guardrails against prompt injection are Enterprise only and cover user messages, as on the devin listing (8 of 15). Enterprise analytics count tool calls and consumption by model and user; no per-user audit log for the CLI was found (8 of 15). SOC 2 Type II and ISO/IEC 27001:2022, a disclosure address, no bug bounty and no security.txt, per the devin listing (13 of 20). Training: self-serve customer data may train Cognition's models by default, with an opt-out on paid plans that also turns on zero data retention at model providers. Cognition withheld the weights over cyber misuse, which is recorded as a fact, not scored. 10 + 14 + 8 + 8 + 13 = 53.

Fireworks AI Inference 70

Public status page at status.fireworks.ai with a component per serverless model (20). The history page, read on 1 November, lists 236 incidents from 3 August to 31 October, nearly all short degradations of a single serverless model. Two hit many serverless models at once. On 11 August a cloud provider outage was 'affecting many of our serverless and dedicated deployments', with five serverless components (GPT OSS 120B, MiniMax M3 and the three Kimi K3 endpoints) in partial outage from 17:15 to 20:55 UTC and again from 01:36 to 01:49 UTC on 12 August, resolved 13 August. On 14 October a cloud provider outage set 18 serverless components to degraded performance, GPT OSS 120B, Kimi K3 and GLM 5.3 among them, from 23:06 UTC, with requests to them that 'may fail', recovery 'across the board' at 00:59 UTC on 15 October and the incident resolved at 01:53. Each is an hour or more of errors across most of the serverless catalogue, so two majors. Two majors sit between the checklist's one major (10) and several (0), so 5, as deepseek-api, black-forest-labs and telnyx-voice are read (5). The 19 October check gave the same 5 because no 90-day history could be read. A 30 October incident on deployment creation, dedicated training and batch inference was still open on 1 November. Numeric adaptive ceilings by model size, 216M total prompt, 54M uncached prompt and 2.16M generated tokens a minute for Small models (15). The rate-limits page says to honour Retry-After on 429 and otherwise back off with jitter, and the reliability guide covers timeouts and retries (15). Reserved throughput is 'covered by a throughput SLA' with no figure published (5 of 10). Serverless is generally available (10). 20+5+15+15+5+10 = 70.

SWE-2 Flash 50

Model reading, for the surface it runs on, Devin Desktop and Devin CLI. Status page at www.devinstatus.com (Atlassian Statuspage) with Desktop Agent and Desktop Tab components besides the cloud ones, read on 4 November 2026 (20). The history feed for the 90 days from 6 August lists 'Devin unavailable' for 20 minutes on 13 August with no component named, read as across the board and a major as on the devin listing; Devin Cloud unavailable on 13 August and the web app down on 24 September name cloud surfaces this model doesn't use; the rest are degraded-performance incidents (new sessions on 3 September, 'multiple services' on 17 September, Ask Devin on 26 October, automation triggers on 4 November) and session queuing on 10 and 11 August, read as minor and a scheduled Devin Desktop extension marketplace upgrade on 7 August, read as maintenance. One major (10). No rate limit figures for the model or the CLI; usage is a daily and weekly quota with no numbers (0). No 429, backoff or retry guidance for this surface (0). Cognition publishes an SLA (25 February 2026) with uptime and response times that applies when included in an order form, read as a contractual SLA on Enterprise, as on other listings (10). The launch post makes it available today with no preview or beta label (10). 20 + 10 + 0 + 0 + 10 + 10 = 50.

Fireworks AI Inference 40

No machine payment protocol in the docs index or billing pages, re-read on 1 November (0). Public per-token prices by model and serving mode (20). The credits FAQ documents a $1 starter credit that runs without a payment method, after which the account is suspended until one is added (20). Account signup and credit purchases need a person (0).

SWE-2 Flash 20

Published rubric. No payment protocol (0). Plan prices are public, but no per-token price for SWE-2 Flash is on a readable page; the docs' cost table loads in the browser and the post gives only 'one-fifth the price' of SWE-2 (10). Free through 30 November 2026 on Devin Desktop and CLI, but the docs say Free tier users only have SWE-1.6, so whether the Free plan reaches it, and without a card, isn't confirmed; the in-between value used on the devin listing (10). A person signs in through a browser (0). 0 + 10 + 10 + 0 = 20.

Fireworks AI Inference 65

Read by the HTTP API checklist, as for cerebras-inference and together-inference. Rune 26B-A4B joined serverless on 27 October, five days before this check (30). Dated changelog entries several times a month, among them 27, 21, 19 and 18 October (20). Support replies not tested (0). Official Python SDK current, v1.2.30 tagged on 21 October after eleven tags from 1 October, no official TypeScript SDK (15). Dependency health not audited (0). The serverless overview promises 'at least 2 weeks advance notice' before a model is removed, and the 27 August removals were posted the day they took effect (see the deduction). 30+20+0+15+0 = 65.

SWE-2 Flash 53

Model reading. Released 4 November 2026 (30). Churn and notice in place of release counts: SWE-1.6, SWE-1.7, SWE-1.7 Lightning, SWE-2 (10 September) and now SWE-2 Flash in a few months, with no model deprecation notice period found and short names that move to the newest model (10 of 20). Public release notes and a support address, no public tracker (10 of 25). No official SDK for the model (0 of 15). The docs are kept current but don't yet cover this model (3 of 10). 30 + 10 + 10 + 0 + 3 = 53.