Scrape.do vs Spider

Scrape.do

$29 for 250,000 basic credits, and 429s, 502s and most failures aren't billed. Token travels in the query string on the main API.

Spider

Keyless /scrape and working x402 v2 on every core route, so an agent needs no account. Operated by BAGELMEN LLC with no address, retention periods or DPA in the privacy policy.

Scrape.do

1. Start with the default data centre request (1 credit). Add super=true or render=true only after a block, since they cost 10 and 5
2. Pass output=markdown to get LLM-ready text instead of raw HTML
3. Read Scrape.do-Request-Cost from the response headers. Some domains cost 10 to 200 credits whatever you pass
4. Treat a 429 as too many concurrent requests and wait for one to finish
5. Keep URLs with token= out of logs and shared traces

Spider

1. Every content route returns a JSON array, even /scrape. The status field is the page's, not the API call's
2. Leave request on smart and set return_format to markdown for LLM input
3. Set limit on /crawl. Unset, it caps only at your credit balance
4. Use /scrape with stealth: true instead of /unblocker, which is deprecated
5. Honour Retry-After on 429. Keyless use allows 4 requests a minute