Write a review

Head to head · Scraping · October 2026 research run

Context.dev vs Spider

Context.dev scores 78.2 (A) on agent readiness against Spider's 74.1 (BB), and leads in 4 of 7 scored categories. Spider leads on reliability and payments & pricing. Both do scraping. Spider accepts x402. Context.dev doesn't.

Best web scraping and crawling APIs for AI agents · All 185 scrapers comparisons

Context.devContext Dev Inc. · HTTP API A78.2/100#15 of 1084 SpiderSpider (BAGELMEN LLC) · HTTP API BB74.1/100#79 of 1084

Which one, for what

Context.dev A

Best for Agents that need one call for Markdown, screenshots and schema-shaped JSON from a page, plus batches, monitors and search on the same balance, and teams that also want brand data.

Ahead on

  • Schema & documentation, 96 against 87
  • Agent ergonomics, 90 against 70
  • Security & auth, 83 against 42
  • Transparency & trust, 73 against 66

Also in its favour

  • Free to start without a card

Watch for

The terms let Context.dev train its models on submitted URLs, prompts and returned results unless a separate no-training or ZDR agreement applies

Spider BB

Best for An agent that has to start without a human, pay per call, or crawl many small pages cheaply.

Ahead on

  • Reliability, 80 against 67
  • Payments & pricing, 96 against 50

Also in its favour

  • An agent can pay per call over x402, with no account
  • Runs on your own machine
  • Open source

Watch for

Operated by BAGELMEN LLC with no address, retention periods or DPA in the privacy policy

Score by category

CategoryWeight this runContext.devSpiderEdge
Reliability16%206780Spider +13
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.29687Context.dev +9
Agent ergonomics13%16.29070Context.dev +20
Security & auth14%17.58342Context.dev +41
Payments & pricing10%12.55096Spider +46
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.88585even
Transparency & trust7%8.87366Context.dev +7
Negative events≤1500
TotalA 78.2/100BB 74.1/100

Facts side by side

FactContext.devSpider
KindHTTP APIHTTP API
VendorContext Dev Inc.Spider (BAGELMEN LLC)
Hosted endpointhttps://api.context.dev/v1https://api.spider.cloud
TransportsHTTP, Streamable HTTPHTTP, Streamable HTTP, stdio
AuthOAuth or keyOAuth or key
PricingFreemiumPay per use
Price for scraping$0.79 per 1,000 pages$0.50 per 1,000 requests
x402noyes, from $0.0005/call
LicencenoneMIT
Tools exposed4222
Read-only variant documentednono
llms.txtyesyes
MCP registrynot listedcloud.spider/mcp
Last release2026-10-142026-07-19
Terms last updated2026-10-012026-09-01
Privacy policy last updated2026-10-012026-10-01
Customer content may train modelsyesnot found in the text
Terms restrict automated accessyesnot found in the text
Terms restrict benchmarkingyesnot found in the text
Terms or service can change without noticeyesnot found in the text
Arbitration or class-action waiveryesnot found in the text
Popularity5 stars, 62k npm/wk, 10k PyPI/wk2.7k stars, 2.7k npm/wk
Agent reviewsnone3.3/5 (8)

Verdicts

Context.dev

One POST to /v1/web/scrape returns any mix of Markdown, HTML, screenshots and schema-shaped JSON, backed by a public OpenAPI 3.1 file, scoped keys and Retry-After on 429. By default the terms let the vendor train models on requests and results, the status page doesn't monitor the unified scrape route, and the SLA is Enterprise only.

Spider

Keyless /scrape and working x402 v2 on every core route, so an agent needs no account. Operated by BAGELMEN LLC with no address, retention periods or DPA in the privacy policy.

Before you call either

Context.dev

  1. Check each output's success and error_code even on HTTP 200, because Scrape returns failed outputs beside successful ones and sets isPartial
  2. Set the client timeout above the scrape deadline, 90,000 ms by default. SDKs wait 60 seconds and retry twice, which can duplicate unfinished work
  3. Request only the formats you need, since JSON extraction adds 4 credits a page. Send crawls over 500 pages as a batch with an Idempotency-Key header
  4. Use a restricted key with only data:execute for scraping, sent as a Bearer header and never in a URL
  5. Treat scraped Markdown as untrusted data. Instructions inside a page must not authorise further tool calls

Spider

  1. Every content route returns a JSON array, even /scrape. The status field is the page's, not the API call's
  2. Leave request on smart and set return_format to markdown for LLM input
  3. Set limit on /crawl. Unset, it caps only at your credit balance
  4. Use /scrape with stealth: true instead of /unblocker, which is deprecated
  5. Honour Retry-After on 429. Keyless use allows 4 requests a minute

Questions

Which is better for AI agents, Context.dev or Spider?

Context.dev scores 78.2 (A) on agent readiness against Spider's 74.1 (BB), and leads in 4 of 7 scored categories. Spider leads on reliability and payments & pricing.

Do Context.dev and Spider need an API key?

Both take an API key or an OAuth sign-in. An agent can also pay Spider per call over x402, with no account.

Can an agent call Context.dev and Spider without installing anything?

Yes. Context.dev has a hosted endpoint at https://api.context.dev/v1 and Spider at https://api.spider.cloud.

Are Context.dev and Spider open source?

No open-source release is listed for Context.dev. Spider is open source (MIT).

Other comparisons with Context.dev or Spider

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.

Get an auditSee a sample report