Crawl4AI vs Scrapfly

Crawl4AI 35

Graded on the hosted lines, for Crawl4AI Cloud, which the README and llms.txt now lead with and which an agent calls through its MCP server; the library would read on the local lines. No status page is linked, and status.crawl4ai.com fails TLS (0). No incident history readable (5). Per-plan limits are published with numbers in /v1/prices, from 10 requests a minute and 2 at a time on Free to 1,500 and 50 on Scale (15). 429 carries X-RateLimit headers and a message, 503 busy says to resend after a few seconds, and MCP job submissions take an idempotency key, but no Retry-After or backoff guidance (10 of 15). No SLA found in the terms or pricing (0). Prices and the free credit are a soft launch, /serp is beta and /answer experimental, while scrape, extract and search carry no beta label (5 of 10). 0 + 5 + 15 + 10 + 0 + 5 = 35.

Scrapfly 85

scrapfly.io/status answers with Statuspage's status JSON, which names its page as scrapfly.statuspage.io. That page has 25 components and incident history back to 2022, read through its history feed on 1 November. The 1 October check read only the overall state and found no history (20, was 10). The newest incident is a Scrapium rendering fault on 21 May 2026, so nothing is posted in the 90 days from 3 August (30, was 5). Concurrency per plan is published (5 on Discovery to 100 on Enterprise) and each project has a configurable per-minute throttle (15). The error catalogue marks every code retryable or not and has three ERR::THROTTLE codes, but we found no Retry-After header or backoff guidance on the errors or throttling pages (10 of 15). The terms disclaim uptime and no uptime SLA is published, and the pricing page's SLA mentions cover support response on Enterprise (0). The Scrape API is GA. Vault and alerting are beta, outside the main surface (10). 20+30+15+10+0+10 = 85.

Crawl4AI 55

No OpenAPI or similar contract for the cloud API, and the MCP tool schemas are only in the live tools/list, which needs a key. /v1/prices is machine-readable but covers prices, not calls (0). llms.txt of 304 lines written for agents (10). Each endpoint says what it is for and when to choose another, such as trying without a proxy first, using browser only when auto misses content and /scrape/jobs for slow lists (16 of 20). Parameters are listed with types, defaults, allowed values (format both, md or html; proxy none, isp or residential) and limits, in prose (10 of 15). Curl examples per endpoint and documented error strings per status with whether each is charged (14 of 15). No changelog for the cloud API, only dated notes such as /serp 'added 8 Oct 2026'. The library has a dated CHANGELOG (5 of 15). 0 + 10 + 16 + 10 + 14 + 5 = 55.

Scrapfly 95

A Swagger reference per the 30 September check, and the MCP server exposes it as a web_scraping_openapi_specification resource. Every MCP tool has JSON Schema inputs generated from typed Go structs (25). llms.txt and Markdown twins such as pricing.md per the 30 September check. robots.txt still disallows /docs/llms.txt and the .md twins for crawlers it doesn't name, so they weren't re-read on 1 November (10). Descriptions say when to use a tool and when to switch, for example web_get_page falls back to web_scrape for tuning and to cloud_browser_open for interaction. A few are long and steer the model's next step (17 of 20). Typed inputs with enums, for example the vault service discriminator, and required fields (13 of 15). Error catalogue of 100+ ERR:: codes, each with HTTP status, retryable and billed flags and its own doc link (15). Dated release notes, newest 26 October 2026 (15). 25+10+17+13+15+15 = 95.

Crawl4AI 46

Bearer keys, shown once and regenerable, with 3 to 30 keys per plan. claude.ai and ChatGPT sign in through a flow that advertises one scope, mcp, and each app shows as a revocable key. No per-key scopes, and the docs don't offer the key in a URL (22 of 30). A key can call billing routes, and the MCP topup and recharge tools charge the saved card, though llms.txt tells agents to ask first and accounts can set a spend cap. No read-only key (6 of 20). Scraped pages are untrusted text and no prompt-injection guidance was found in llms.txt or the API docs (0). Per-call cost, balance and fetch path in response headers and fields, no access log documented (6 of 15). The security page (4 October 2026) gives a disclosure process with fix targets and says SOC 2 is in progress, and the library publishes GitHub advisories with reporter credits. security.txt redirects to a 404, and no bounty found (12 of 20). 22 + 6 + 0 + 6 + 12 = 46.

Scrapfly 75

OAuth 2 on the hosted MCP with dashboard revocation, open to all plans. API keys are per project, live or test, and since 15 April 2026 can be restricted to products (30). The canonical API form puts the key in the key query parameter and the MCP also accepts ?key=, so less 10 (20 of 30). Product-restricted keys, confirm=true on every mutating vault and alert tool, info_api_key off in server-key mode, and the self-hosted HTTP server binds loopback by default. Since v1.4.0 Cloud Browser sessions are scoped to the API key that opened them (15 of 20). The source gates mutating vault tools behind a preview step because prompt injection can reach them and redacts keys from CDP URLs. We found no user-facing guidance on untrusted page content (8 of 15). Monitoring shows every call as a log with its error, credit cost line by line and Unblocker state, with replay, per the 24 October release notes, as well as per-domain metrics and alerting. The 1 October check couldn't confirm per-request logs (15, was 10). security.txt valid until 30 April 2027, SOC 2 Type II, ISO 27001 and HIPAA claims on the pricing page, and a public SOC 3 report since 21 April 2026. No bug bounty found (17 of 20). 20+15+8+15+17 = 75.

Crawl4AI 67

16 MCP tools, six of them for billing and support (15 of 25). format md or html, parse opt-in blocks, batch limits and job results paged 500 at a time over HTTP and 50 over MCP (16 of 20). String error codes an agent can branch on (blocked, login-wall, no-answer, busy, no_credit, plan_cap, spend_cap), each with whether it is charged (18 of 20). Failed and cancelled calls aren't charged and jobs_submit takes an idempotency key, but no tool annotations are documented and two tools can charge the saved card (10 of 20). Only url or q is required. There is no official SDK for the cloud API; the docs give curl and standard-library Python and Node examples (8 of 15). 15 + 16 + 18 + 10 + 8 = 67.

Scrapfly 80

The open-source server registers 62 tools by default in v1.4.0 (5 web scraping, 2 account, 9 Cloud Browser sessions, 17 page interaction, 2 WebMCP, 15 credential vault, 12 alerts), all at once, and the docs list a smaller core. There's a code path to mount interaction tools per session, but registration is always-on for clients that don't refetch. On the API side, format=markdown and the extraction API keep responses small, and since v1.4.0 the scrape tools report api_cost and cost_details (10 of 25). Output format, extraction templates, crawler limits and 100-URL batches (15 of 20). Error codes an agent can branch on, with retryable and billed flags (20). Tools carry readOnlyHint, destructiveHint, idempotentHint and openWorldHint, and mutating vault and alert tools need confirm=true after a preview (20). web_get_page works with a URL alone, and official SDKs exist for Python, TypeScript, Go and Rust (15). 10+15+20+20+15 = 80.

Crawl4AI 45

No x402, MPP or L402 (0). Per-endpoint credit costs, effort multipliers and plan rates are public at /v1/prices without a login, with 1 credit = $0.001 (20). 1,000 free credits a month, no card (20). A 7-day pass key comes from a click on api.crawl4ai.com without an account, which is still a person in a browser, and a permanent key needs email verification (5 of 20). 0 + 20 + 20 + 5 = 45.

Scrapfly 40

No x402, MPP or L402 (0). Credit costs per request type (1 datacentre, 5 with rendering or Unblocker, 25 residential, 60 full-page screenshot) and plan prices are public. Unblocker surcharges per target aren't published, but cost_budget caps them (20). 1,000 free credits, no card, no time limit (20). A person signs up in a browser. OAuth still needs an account (0). Plan prices and the 1,000 free credits with no card re-read on the pricing page on 1 November. 0+20+20+0 = 40.

Crawl4AI 70

The library is Apache-2.0 and the cloud is closed under terms that name CONTEXT4AI PTE LTD (22 of 30). The privacy policy gives retention periods (account plus 90 days, usage records 13 months, logs 90 days), but none for crawled content, and llms.txt says repeated URLs are served from a shared archive while the security page says fetched pages are used only to answer the request (18 of 30). The legacy cloud API closed on 6 November 2026 with a migration guide but no dated notice found, and the terms promise notice of material changes only 'where reasonable' (8 of 20). Subprocessors named with their locations on the security page, with EU storage of account records (18 of 20). 22 + 18 + 8 + 18 = 66.

Scrapfly 75

Closed service with terms updated 2026-09-30 under US law. SDKs are BSD 2-Clause, and the MCP source is public with no LICENSE file and ISC in package.json (15 of 30). Privacy policy updated 14 May 2026 with a published DPA and HIPAA BAA, but retention is as long as necessary, and the retention of scraped content isn't stated (18 of 30). Release notes date deprecations and removals, for example Scrapium v153 dropping legacy network interception on 11 September. No standing deprecation policy found (15 of 20). Stripe, Google Analytics, Google Ads and Google Cloud are named, with SCC and Data Privacy Framework transfers, but there's no full subprocessor list (10 of 20). The release notes still date each CDP removal, most recently Storage.getRelatedWebsiteSets in Scrapium v156 on 26 October. Not otherwise re-read on 1 November. 15+18+15+10 = 58.