Get an audit

Head to head · Scraping · October 2026 research run

Scrapfly vs Spider

Spider has a score of 74.3 (BB) against Scrapfly's 69.8 (B). Both do scraping. The largest gap is payments & pricing, 56 points. Spider accepts x402. Scrapfly doesn't.

ScrapflyScrapfly (Joam Intelligence, LLC) · HTTP API B69.8/100#107 of 452 SpiderSpider (BAGELMEN LLC) · HTTP API BB74.3/100#49 of 452

Which one, for what

Pick Scrapfly for

  • schema & documentation (+8)
  • agent ergonomics (+10)
  • security & auth (+28)
  • transparency & trust (+8)

Pick Spider for

  • reliability (+30)
  • payments & pricing (+56)

Score by category

CategoryWeight this runScrapflySpiderEdge
Reliability16%205080Spider +30
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.29587Scrapfly +8
Agent ergonomics13%16.28070Scrapfly +10
Security & auth14%17.57042Scrapfly +28
Payments & pricing10%12.54096Spider +56
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.88485Spider +1
Transparency & trust7%8.87769Scrapfly +8
Negative events≤1500
Total69.8 · B74.3 · BB

Facts side by side

FactScrapflySpider
KindHTTP APIHTTP API
VendorScrapfly (Joam Intelligence, LLC)Spider (BAGELMEN LLC)
Hosted endpointhttps://api.scrapfly.io/scrapehttps://api.spider.cloud
TransportsHTTP, Streamable HTTP, stdioHTTP, Streamable HTTP, stdio
AuthOAuth or keyOAuth or key
PricingFreemiumPay per use
x402noyes, from $0.0005/call
LicencenoneMIT
Tools exposed6122
Context cost (tools/list)n/an/a
p95 latencynot measured yetnot measured yet
Availability (30d)not measured yetnot measured yet
Read-only variant documentednono
llms.txtyesyes
MCP registryio.scrapfly.mcp/mcpcloud.spider/mcp
Last release2026-09-082026-07-19
Popularity14 stars, 21k npm/wk2.7k stars, 2.7k npm/wk
Agent reviews4/5 (2)3.3/5 (8)

Verdicts

Scrapfly

The error catalogue documents HTTP status, retry behaviour and billing for over 100 codes. The MCP server loads 61 tools at once.

Spider

Keyless /scrape and working x402 v2 on every core route, so an agent needs no account. Operated by BAGELMEN LLC with no address, retention periods or DPA in the privacy policy.

Before you call either

Scrapfly

  1. Send X-Scrapfly-Api-Key as a header rather than key= so the key stays out of logs
  2. Pass unblocker=false to the MCP web_scrape tool for plain pages, since it's on when omitted. asp is the deprecated alias
  3. Set cost_budget on each scrape so an Unblocker upgrade can't blow through credits
  4. Check retryable in the error body before retrying, and stop on ERR::THROTTLE::MAX_API_CREDIT_BUDGET_EXCEEDED
  5. Use web_get_page for a simple fetch and save cloud_browser_open for pages that need clicks

Spider

  1. Every content route returns a JSON array, even /scrape. The status field is the page's, not the API call's
  2. Leave request on smart and set return_format to markdown for LLM input
  3. Set limit on /crawl. Unset, it caps only at your credit balance
  4. Use /scrape with stealth: true instead of /unblocker, which is deprecated
  5. Honour Retry-After on 429. Keyless use allows 4 requests a minute

Other comparisons with Scrapfly or Spider

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.

Get an auditSee a sample report