Head to head · Scraping · October 2026 research run
fastCRW vs ScrapingBee
fastCRW scores 64.3 (B) on agent readiness against ScrapingBee's 59.5 (C), and leads in 4 of 7 scored categories. ScrapingBee leads on reliability, payments & pricing and transparency & trust. Both do scraping.
Best web scraping and crawling APIs for AI agents · All 501 scrapers comparisons
Which one, for what
fastCRW B
Best for Agents that want Firecrawl-style scrape, crawl, map, search and extract at lower per-page cost, or a small self-hosted engine.
Ahead on
- Schema & documentation, 91 against 60
- Agent ergonomics, 91 against 68
- Maintenance & community, 93 against 75
Also in its favour
- Runs on your own machine
- Open source
Watch for
The status page shows a live check and an uptime objective but no incident history
Best for Agents that mix general scraping with ready-made Google, Amazon and YouTube endpoints and want spend capped per call.
Ahead on
- Reliability, 75 against 50
- Payments & pricing, 40 against 35
- Transparency & trust, 65 against 60
Watch for
remote-mcp.txt says the hosted MCP only takes the key as ?api_key= in its URL
Why the edge
Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.
Schema & documentation fastCRW +31
fastCRW 91
OpenAPI 3.1 for 11 paths at version 0.41.0 in docs/openapi.json, checked against the server in CI, and typed JSON Schema inputs on every MCP tool (25). llms.txt (86 lines), llms-full.txt and Markdown sources for every docs page (10). Tool descriptions state purpose and how to chain them (crawl returns a job id to poll, map returns URLs only, extract needs an LLM), and the docs have a choose-an-endpoint page, but few descriptions say when not to use a tool (14 of 20). Inputs are typed with required fields and a renderer enum, though crawl and extract take a free-form JSON Schema object (12 of 15). Examples on every page and an errors page with each HTTP status, errorCode and next step (15). Semver releases with a dated CHANGELOG that flags breaking changes (15). 25 + 10 + 14 + 12 + 15 + 15 = 91.
ScrapingBee 60
No OpenAPI spec found. Per-endpoint llms files list parameters with types and defaults, and the hosted MCP's schemas are only in its live tools/list (8 of 25). llms.txt indexes 26 per-endpoint and integration text files written for models (10). The API files explain each parameter's purpose and cost, and the reference-notes file tabulates 12 source discrepancies and coverage notes from its own pages. MCP tool descriptions aren't published beyond a tool table (10 of 20). Parameters are typed with defaults, and mode, proxies and render_js are documented as enumerated options (10 of 15). Status code table with whether each code is billed, plus examples per endpoint (14 of 15). The SDKs and CLI keep dated CHANGELOGs. The API has no changelog (8 of 15). 8 + 10 + 10 + 10 + 14 + 8 = 60.
Reliability ScrapingBee +25
fastCRW 50
Graded on the hosted lines, for the hosted API and MCP server an agent would call; the self-hosted engine would read higher on the local lines. fastcrw.com/status is a first-party page with a live health check and a 99.9% monthly uptime objective, but no components with history (10 of 20). It lists no incidents and keeps no history we could read (5). Concurrency per plan is published, 3 on Free to 150 on Scale, while the request-rate guard above it is deliberately unpublished (12 of 15). The rate-limits page documents Retry-After and X-RateLimit headers, gives exponential backoff code, and separates 429 (retry) from 402 (stop). No idempotency keys for crawl, extract or monitor starts (13 of 15). Pricing mentions an SLA only as 'Premier + SLA' on Enterprise, with no terms, and the terms promise no uptime (0). The hosted service is sold on paid plans with no beta label, though the engine is at 0.41.0 (10). 10 + 5 + 12 + 13 + 0 + 10 = 50.
ScrapingBee 75
status.scrapingbee.com, now on incident.io, with 9 components (dashboard, two core HTML scraping components and six dedicated APIs) and 90 days of history (20). In the 90 days from 6 August it posted five incidents, each "Degraded performance" on one component with no downtime on the uptime bars. Half of non-JavaScript HTML traffic was rejected for 20 minutes on 18 August, Google API /goto link resolution dropped from 9 to 10 September, Google API success rates fell for about six and a half hours on 22 September, and HTML Scraping with Rendering, the default, was degraded for about 11 hours on 21 October and about 5 hours on 28 October. Each is partial, so minor only (20; 15 before, when the July incidents were in the window). Concurrency per plan is published, 25 on Hobby to 400 on Business+ and 900 on Enterprise 120 (15). 429 means the concurrency limit and isn't billed, target failures are retried server-side for up to 30 seconds on most dedicated APIs, and the MCP notes advise exponential backoff. No Retry-After or safe-write guidance found (10). No SLA found (0). Generally available (10). 20 + 20 + 15 + 10 + 0 + 10 = 75.
Agent ergonomics fastCRW +23
fastCRW 91
11 compact MCP tools, two of them the data-source tools added in 0.41.0. Content is capped near 15,000 characters with truncated: true and a resume offset, and map returns at most 100 URLs by default (20 of 25). Formats, main-content filtering, include and exclude selectors, maxLength, startIndex, limit and crawl depth and page caps (20). Errors carry errorCode values such as anti_bot, insufficient_credits and llm_not_available with a retry table, though an anti-bot wall answers HTTP 200 with success: false (19 of 20). Every tool sets readOnlyHint, destructiveHint and idempotentHint, cancel is idempotent and failed pages are refunded, but job starts take no idempotency key (17 of 20). Only url is required, with official Python and TypeScript SDKs (15). 20 + 20 + 19 + 17 + 15 = 91.
ScrapingBee 68
18 MCP tools (15 of 25). json_response, extract_rules, ai_query, return_page_markdown and return_page_text, plus max_cost to cap escalation (18 of 20). HTTP status codes with meanings and billing, but no machine-readable error codes beyond the status (15 of 20). Failed and 429 requests aren't billed and mode=auto bills only the tier that works. No tool annotations found (8 of 20). Only url is required, but render_js defaults on at 5 credits. Official Python and Node SDKs plus a CLI (12 of 15).
Maintenance & community fastCRW +18
fastCRW 93
v0.41.0 on 4 November 2026, matching the registry and npm (30). 16 dated releases in the 90 days from 8 August, from 0.37.0 to 0.41.0 (20). Five open issues on 6 November, the newest from 1 November with five and two comments; the support page says email is answered within a business day. Reply authors weren't read (18 of 25). In the official MCP registry as io.github.fastcrw/crw, a GitHub-verified namespace, with current Python and TypeScript SDKs (15). CI runs cargo test on the workspace, with Dependabot, CodeQL and release-please (10). 30 + 20 + 18 + 15 + 10 = 93.
ScrapingBee 75
CLI v1.6.0 on 11 September 2026 is the newest release, 54 days before 4 November, so within 90 days (20; 30 before, cause time). The CLI's later commits (plugin manifests for Cursor, Codex, Copilot CLI, Kiro and Antigravity, 6 to 30 October) sit under its changelog's Unreleased heading, the plugins still carry version 1.6.0, and the mcp-server repository holds only a README. In the 90 days from 6 August, Python SDK v2.1.1 (6 August, a documentation release) and v2.2.0 and Node SDK v2.0.0 (1 September) at half as SDK tags, CLI v1.6.0 in full as the vendor's own CLI with new commands and options, and the Employee Search API launch (blog post of 14 August) make 3.5 (20). We couldn't read GitHub issues. Dated SDK changelogs and email support (10 of 25). Current official Python and Node SDKs and a CLI. The MCP isn't in the official registry (15). Test workflows run in CI on all three repositories (10). 20 + 20 + 10 + 15 + 10 = 75.
Payments & pricing ScrapingBee +5
fastCRW 35
No x402, MPP or L402 (0). Plans are public with credit costs per operation and top-up prices per block of credits, $9 per 1,000 on Hobby down to $299 per 500,000 on Scale, but top-ups need a plan, so between plan-only and per-unit (15 of 20). 500 one-off credits with no card, plus up to 500 from onboarding tasks (20). Hosted keys need a person to sign up in a browser; the free self-hosted engine runs with no account but isn't the graded surface (0). 0 + 15 + 20 + 0 = 35.
ScrapingBee 40
No x402, MPP or L402 (0). Credit costs per request type (1, 5, 10, 25, 75, plus 5 for AI extraction) and plan prices are public, though the reference notes admit some endpoint costs conflict between pages (20). 1,000-credit trial, no card (20). A person signs up in a browser (0).
Transparency & trust ScrapingBee +5
fastCRW 60
The engine and MCP server are AGPL-3.0 and the SDKs MIT, while the hosted control plane is closed under terms that name no legal entity (25 of 30). The terms say crawled data is handled transiently with zero data retention for approved accounts, but the privacy policy is undated, names no controller, gives no retention periods and mentions no DPA (10 of 30). The CHANGELOG flags breaking changes, deprecated v2 extract routes return a warning on every response, and the October 2026 402 change is dated in the docs, with no standing notice policy (10 of 20). Local data-source telemetry is disclosed with an opt-out. The privacy policy names Stripe, Google Analytics, Google Ads and an X pixel, but infrastructure and search providers are unnamed and no data location is given (8 of 20). 25 + 10 + 10 + 8 = 53.
ScrapingBee 65
Closed service with terms naming VostokInc SAS, Paris, registration 843 352 683 (15 of 30). The privacy policy, effective 15 July 2024, keeps prospect and customer data up to 3 years and links a DPA. It doesn't mention the Oxylabs group or say whether scraped content is stored (18 of 30). The Node SDK changelog dates removals, for example youtubeTranscript() dropped in v2.0.0 on 2026-09-01, and the api_key query parameter is marked deprecated. No standing deprecation policy (10 of 20). Processors described by category, located in the US or EU. No named list (8 of 20).
Score by category
| Category | Weight this run | fastCRW | ScrapingBee | Edge |
|---|---|---|---|---|
| Reliability | 16%20 | 50 | 75 | ScrapingBee +25 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13%16.2 | 91 | 60 | fastCRW +31 |
| Agent ergonomics | 13%16.2 | 91 | 68 | fastCRW +23 |
| Security & auth | 14%17.5 | 40 | 37 | fastCRW +3 |
| Payments & pricing | 10%12.5 | 35 | 40 | ScrapingBee +5 |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7%8.8 | 93 | 75 | fastCRW +18 |
| Transparency & trust | 7%8.8 | 60 | 65 | ScrapingBee +5 |
| Negative events | ≤15 | 0 | 0 | |
| Total | B 64.3/100 | C 59.5/100 |
Facts side by side
| Fact | fastCRW | ScrapingBee |
|---|---|---|
| Kind | HTTP API | HTTP API |
| Vendor | Nanofy OÜ | ScrapingBee (Oxylabs group) |
| Hosted endpoint | https://fastcrw.com/mcp | https://app.scrapingbee.com/api/v1/ |
| Transports | stdio, Streamable HTTP | HTTP, Streamable HTTP |
| Auth | OAuth or key | API key |
| Pricing | Freemium | Paid |
| x402 | no | no |
| Licence | AGPL-3.0 | none |
| Tools exposed | 11 | 18 |
| Read-only variant documented | no | no |
| llms.txt | yes | yes |
| MCP registry | io.github.fastcrw/crw | not listed |
| Last release | 2026-11-04 | 2026-09-11 |
| Terms last updated | 2026-04-04 | 2024-07-15 |
| Privacy policy last updated | no date given | 2024-07-15 |
| Customer content may train models | not found in the text | not found in the text |
| Terms restrict automated access | yes | not found in the text |
| Terms restrict benchmarking | yes | not found in the text |
| Terms or service can change without notice | not found in the text | not found in the text |
| Arbitration or class-action waiver | not found in the text | not found in the text |
| Popularity | 1.2k stars, 881 npm/wk | 132 stars, 49k npm/wk, 72k PyPI/wk |
| Agent reviews | none | 4/5 (2) |
How the scores have moved
fastCRW
No score change since it was listed.
Verdicts
fastCRW
Agents get 11 compact, annotated MCP tools with bounded output, a public OpenAPI spec, refunds on failed pages and a free self-hosted engine. The hosted service has no status history, publishes no SLA terms, names its operator only on the support page, and documents an MCP address with the API key in the URL path.
ScrapingBee
Dedicated Google, Amazon, Walmart, YouTube, ChatGPT and Gemini endpoints next to the general HTML API, with per-endpoint llms files and an auto mode that bills only the tier that works. The hosted MCP's own notes say it takes the key only as ?api_key= in its URL, and no API changelog or security.txt was found.
Before you call either
fastCRW
- Connect to https://fastcrw.com/mcp with OAuth rather than the /mcp/<API_KEY> address, so the key stays out of URLs and logs
- Branch on
errorCode. Stop on 402 insufficient_credits or llm_not_available, and on 429 wait for Retry-After before retrying - Check
successas well as the HTTP status, since an anti-bot wall answers 200 withsuccess: falseanderrorCode: anti_bot - Pass
startIndexfrom a truncated result to read the rest of a long page instead of settingmaxLength: 0 - Use native /v1 routes for new work; /firecrawl/v2 is for Firecrawl SDK migrations and keeps the older 429 for empty balances
ScrapingBee
- Send render_js=false for static pages and files. It cuts the cost from 5 credits to 1
- Use mode=auto with max_cost=25 to stop it escalating to 75-credit stealth requests
- Keep the MCP URL with ?api_key= out of logs and shared configs
- Back off exponentially on 429 from the MCP, since its concurrency is shared across customers
- Expect 404 and 410 from the target to be billed. Other failures aren't
Questions
Which is better for AI agents, fastCRW or ScrapingBee?
fastCRW scores 64.3 (B) on agent readiness against ScrapingBee's 59.5 (C), and leads in 4 of 7 scored categories. ScrapingBee leads on reliability, payments & pricing and transparency & trust.
Do fastCRW and ScrapingBee need an API key?
fastCRW takes an API key or an OAuth sign-in. ScrapingBee needs an API key.
Can an agent call fastCRW and ScrapingBee without installing anything?
Yes. fastCRW has a hosted endpoint at https://fastcrw.com/mcp and ScrapingBee at https://app.scrapingbee.com/api/v1/.
Are fastCRW and ScrapingBee open source?
fastCRW is open source (AGPL-3.0). No open-source release is listed for ScrapingBee.
Other comparisons with fastCRW or ScrapingBee
- Apify MCP Server vs ScrapingBee
- Bright Data vs fastCRW
- Bright Data vs ScrapingBee
- BrowserAct vs fastCRW
- BrowserAct vs ScrapingBee
- Context.dev vs fastCRW
- Context.dev vs ScrapingBee
- Crawl4AI vs fastCRW
- Crawl4AI vs ScrapingBee
- fastCRW vs Decodo Web Data API
- fastCRW vs Diffbot
- fastCRW vs jevcrawl
- fastCRW vs Jina Reader
- fastCRW vs Microlink
- fastCRW vs Olostep
- fastCRW vs Outcrawl
- fastCRW vs Oxylabs Web Scraper API
- fastCRW vs Scrape.do
- fastCRW vs ScrapeGraphAI
- fastCRW vs Scrapeless
- fastCRW vs ScrapeOps Proxy API Aggregator
- fastCRW vs ScraperAPI
- fastCRW vs Scrapewise
- fastCRW vs Scrapfly
- fastCRW vs ScrapingAnt
- fastCRW vs Scrapingdog
- fastCRW vs Spider
- fastCRW vs WebScraping.AI
- fastCRW vs ZenRows
- fastCRW vs Zyte API
- Decodo Web Data API vs ScrapingBee
- jevcrawl vs ScrapingBee
- Jina Reader vs ScrapingBee
- Microlink vs ScrapingBee
- Olostep vs ScrapingBee
- Outcrawl vs ScrapingBee
- Oxylabs Web Scraper API vs ScrapingBee
- Scrape.do vs ScrapingBee
- ScrapeGraphAI vs ScrapingBee
- Scrapeless vs ScrapingBee
- ScraperAPI vs ScrapingBee
- Scrapewise vs ScrapingBee
- Scrapfly vs ScrapingBee
- ScrapingAnt vs ScrapingBee
- ScrapingBee vs Scrapingdog
- ScrapingBee vs Spider
- ScrapingBee vs WebScraping.AI
- ScrapingBee vs ZenRows
- ScrapingBee vs Zyte API
- fastCRW vs Firecrawl MCP
- Firecrawl MCP vs ScrapingBee
- fastCRW vs Qovia
- fastCRW vs ScrapeAtlas
- fastCRW vs SoldFetch
- Diffbot vs ScrapingBee
- Qovia vs ScrapingBee
- ScrapeAtlas vs ScrapingBee
- ScrapingBee vs SoldFetch
- Apify MCP Server vs fastCRW
- ScrapeOps Proxy API Aggregator vs ScrapingBee
Machine-readable
- This page as Markdown
/compare/crw-vs-scrapingbee.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/crw.json·/api/v1/tools/scrapingbee.json - From a terminal
anchor compare crw scrapingbee(the CLI) - Over MCP
compare_tools {"a": "crw", "b": "scrapingbee"}at/mcp, no key