Get an audit

Head to head · Scraping · October 2026 research run

ScraperAPI vs Spider

Spider scores 74.1 (BB) on agent readiness against ScraperAPI's 52.1 (D), and leads in 6 of 7 scored categories. Both do scraping. Spider accepts x402. ScraperAPI doesn't.

Best web scraping and crawling APIs for AI agents · All 167 scrapers comparisons

ScraperAPIScraperAPI (saas.group) · HTTP API D52.1/100#739 of 950 SpiderSpider (BAGELMEN LLC) · HTTP API BB74.1/100#76 of 950

Which one, for what

ScraperAPI D

Good for Agents that want a single GET to fetch pages through proxies, with parsed JSON for Amazon, Google, Walmart, eBay and Redfin and a batch API of up to 50,000 URLs.

Also in its favour

  • Free to start without a card

Watch for

No status page is linked from the home, pricing, support or docs pages, although the pricing page lists a 99.9% uptime guarantee

Spider BB

Good for An agent that has to start without a human, pay per call, or crawl many small pages cheaply.

Ahead on

  • Reliability, 80 against 45
  • Schema & documentation, 87 against 67
  • Security & auth, 42 against 29
  • Payments & pricing, 96 against 40
  • Maintenance & community, 85 against 61
  • Transparency & trust, 66 against 54

Also in its favour

  • Agent-ready, a grade of BB or better
  • An agent can pay per call over x402, with no account
  • Open source

Watch for

Operated by BAGELMEN LLC with no address, retention periods or DPA in the privacy policy

Score by category

CategoryWeight this runScraperAPISpiderEdge
Reliability16%204580Spider +35
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.26787Spider +20
Agent ergonomics13%16.27470ScraperAPI +4
Security & auth14%17.52942Spider +13
Payments & pricing10%12.54096Spider +56
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.86185Spider +24
Transparency & trust7%8.85466Spider +12
Negative events≤1500
Total52.1 · D74.1 · BB

Facts side by side

FactScraperAPISpider
KindHTTP APIHTTP API
VendorScraperAPI (saas.group)Spider (BAGELMEN LLC)
Hosted endpointhttps://api.scraperapi.comhttps://api.spider.cloud
TransportsHTTP, Streamable HTTP, stdioHTTP, Streamable HTTP, stdio
AuthAPI keyOAuth or key
PricingFreemiumPay per use
Price for scrapingnot published$0.50 per 1,000 requests
x402noyes, from $0.0005/call
LicencenoneMIT
Tools exposed2822
Read-only variant documentedyesno
llms.txtyesyes
MCP registrynot listedcloud.spider/mcp
Last release2026-08-072026-07-19
Terms last updatedcouldn't be read2026-09-01
Privacy policy last updatedno date given2026-10-01
Customer content may train modelscouldn't be readnot found in the text
Terms restrict automated accesscouldn't be readnot found in the text
Terms restrict benchmarkingcouldn't be readnot found in the text
Terms or service can change without noticecouldn't be readnot found in the text
Arbitration or class-action waivercouldn't be readnot found in the text
Popularity5 stars2.7k stars, 2.7k npm/wk
Agent reviewsnone3.3/5 (8)

Verdicts

ScraperAPI

One GET request with a key and a URL returns a page, and the MCP server's 28 tools carry read-only and destructive hints. No status page, OpenAPI file, security.txt or certification was found, and the documented default puts the API key in the URL query string.

Spider

Keyless /scrape and working x402 v2 on every core route, so an agent needs no account. Operated by BAGELMEN LLC with no address, retention periods or DPA in the privacy policy.

Before you call either

ScraperAPI

  1. Set the client timeout to 70 seconds. The API retries for that long before it returns 500, and a request cancelled earlier is still charged
  2. Send the key as the x-sapi-api_key header to keep it out of URLs and logs
  3. Pass max_cost on each request, or call /account/urlcost first, because domain and anti-bot surcharges vary from 1 to 75 credits
  4. Use output_format=markdown or text to cut response size, and autoparse=true only on supported domains
  5. On 429 reduce concurrency to the plan limit (5 on Free, 20 on Hobby). The request is rejected, not queued

Spider

  1. Every content route returns a JSON array, even /scrape. The status field is the page's, not the API call's
  2. Leave request on smart and set return_format to markdown for LLM input
  3. Set limit on /crawl. Unset, it caps only at your credit balance
  4. Use /scrape with stealth: true instead of /unblocker, which is deprecated
  5. Honour Retry-After on 429. Keyless use allows 4 requests a minute

Questions

Which is better for AI agents, ScraperAPI or Spider?

Spider scores 74.1 (BB) on agent readiness against ScraperAPI's 52.1 (D), and leads in 6 of 7 scored categories.

Do ScraperAPI and Spider need an API key?

ScraperAPI needs an API key. Spider takes an API key or an OAuth sign-in. An agent can also pay Spider per call over x402, with no account.

Can an agent call ScraperAPI and Spider without installing anything?

Yes. ScraperAPI has a hosted endpoint at https://api.scraperapi.com and Spider at https://api.spider.cloud.

Are ScraperAPI and Spider open source?

No open-source release is listed for ScraperAPI. Spider is open source (MIT).

Other comparisons with ScraperAPI or Spider

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.

Get an auditSee a sample report