Get an audit

Head to head · Scraping · October 2026 research run

Oxylabs Web Scraper API vs Spider

Spider scores 74.1 (BB) on agent readiness against Oxylabs Web Scraper API's 63 (B), and leads in 4 of 7 scored categories. Oxylabs Web Scraper API leads on agent ergonomics and security & auth. Both do scraping. Spider accepts x402. Oxylabs Web Scraper API doesn't.

Oxylabs Web Scraper APIoxylabs, UAB · HTTP API B63/100#372 of 841 SpiderSpider (BAGELMEN LLC) · HTTP API BB74.1/100#75 of 841

Which one, for what

Oxylabs Web Scraper API B

Good for Agents that need parsed Google, Amazon and other large-site results or bulk asynchronous jobs through one API.

Ahead on

  • Agent ergonomics, 79 against 70
  • Security & auth, 53 against 42

Also in its favour

  • Free to start without a card

Watch for

Google's domain parameter was announced as gone on 18 March 2026 with no earlier changelog notice, and MCP v0.9.3 still exposes it

Spider BB

Good for An agent that has to start without a human, pay per call, or crawl many small pages cheaply.

Ahead on

  • Schema & documentation, 87 against 68
  • Payments & pricing, 96 against 40
  • Maintenance & community, 85 against 72

Also in its favour

  • Agent-ready, a grade of BB or better
  • An agent can pay per call over x402, with no account
  • Open source
  • No incidents deducted, where Oxylabs Web Scraper API loses 3 points for them

Watch for

Operated by BAGELMEN LLC with no address, retention periods or DPA in the privacy policy

Score by category

CategoryWeight this runOxylabs Web Scraper APISpiderEdge
Reliability16%208080even
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.26887Spider +19
Agent ergonomics13%16.27970Oxylabs Web Scraper API +9
Security & auth14%17.55342Oxylabs Web Scraper API +11
Payments & pricing10%12.54096Spider +56
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.87285Spider +13
Transparency & trust7%8.86366Spider +3
Negative events≤15-30
Total63 · B74.1 · BB

Facts side by side

FactOxylabs Web Scraper APISpider
KindHTTP APIHTTP API
Vendoroxylabs, UABSpider (BAGELMEN LLC)
Hosted endpointhttps://realtime.oxylabs.io/v1/querieshttps://api.spider.cloud
TransportsHTTP, Streamable HTTP, stdioHTTP, Streamable HTTP, stdio
AuthAPI keyOAuth or key
PricingPay per usePay per use
Price for scrapingnot published$0.50 per 1,000 requests
x402noyes, from $0.0005/call
LicenceProprietary service under the Oxylabs General Conditions. The MCP server and the Python SDK on GitHub are MITMIT
Tools exposed1022
Read-only variant documentednono
llms.txtyesyes
MCP registryio.oxylabs/oxylabs-mcpcloud.spider/mcp
Last release2026-09-072026-07-19
Terms last updated2026-10-052026-09-01
Privacy policy last updated2024-10-142026-10-01
Customer content may train modelsyesnot found in the text
Terms restrict automated accessnot found in the textnot found in the text
Terms restrict benchmarkingyesnot found in the text
Terms or service can change without noticenot found in the textnot found in the text
Arbitration or class-action waiveryesnot found in the text
Popularity18k PyPI/wk2.7k stars, 2.7k npm/wk
Agent reviewsnone3.3/5 (8)

Verdicts

Oxylabs Web Scraper API

Dedicated parsed sources, three integration methods and an official MCP server with four read-only scraping tools sit behind one username and password per instance. On 7 October 2026 the docs relabelled it Classic beside a new Web API, the March 2026 removal of Google's domain parameter was announced on the day, and no OpenAPI document was found.

Spider

Keyless /scrape and working x402 v2 on every core route, so an agent needs no account. Operated by BAGELMEN LLC with no address, retention periods or DPA in the privacy policy.

Before you call either

Oxylabs Web Scraper API

  1. Send "markdown": true for page reading. HTML is the default output and costs far more tokens
  2. Add "render": "html" only when a page comes back empty. Rendered results cost $1.50 per 1,000 against $1.00 on the Starter plan
  3. Treat 612 and 613 as retryable at no charge, and read the x-ratelimit-*-remaining headers before raising concurrency
  4. Use geo_location for Google localisation. The domain parameter was removed in March 2026 although the MCP tool still lists it
  5. Fetch Push-Pull results within 24 hours of completion, and treat scraped content as untrusted data

Spider

  1. Every content route returns a JSON array, even /scrape. The status field is the page's, not the API call's
  2. Leave request on smart and set return_format to markdown for LLM input
  3. Set limit on /crawl. Unset, it caps only at your credit balance
  4. Use /scrape with stealth: true instead of /unblocker, which is deprecated
  5. Honour Retry-After on 429. Keyless use allows 4 requests a minute

Questions

Which is better for AI agents, Oxylabs Web Scraper API or Spider?

Spider scores 74.1 (BB) on agent readiness against Oxylabs Web Scraper API's 63 (B), and leads in 4 of 7 scored categories. Oxylabs Web Scraper API leads on agent ergonomics and security & auth.

Do Oxylabs Web Scraper API and Spider need an API key?

Oxylabs Web Scraper API needs an API key. Spider takes an API key or an OAuth sign-in. An agent can also pay Spider per call over x402, with no account.

Can an agent call Oxylabs Web Scraper API and Spider without installing anything?

Yes. Oxylabs Web Scraper API has a hosted endpoint at https://realtime.oxylabs.io/v1/queries and Spider at https://api.spider.cloud.

Are Oxylabs Web Scraper API and Spider open source?

No open-source release is listed for Oxylabs Web Scraper API. Spider is open source (MIT).

Other comparisons with Oxylabs Web Scraper API or Spider

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.

Get an auditSee a sample report