Head to head · Scraping proxies · October 2026 research run

ScrapeOps Proxy API Aggregator vs XCrawl

ScrapeOps Proxy API Aggregator scores 52.9 (D) on agent readiness against XCrawl's 30.4 (F), and leads in every scored category. Both do scraping proxies.

Best web scraping and crawling APIs for AI agents · All 601 scrapers comparisons

Which one, for what

ScrapeOps Proxy API Aggregator D

Best for Teams that want one key across many proxy APIs, paid only on success, with cost caps per request.

Ahead on

  • Reliability, 37 against 15
  • Schema & documentation, 67 against 35
  • Agent ergonomics, 79 against 59
  • Security & auth, 28 against 18
  • Payments & pricing, 60 against 30
  • Maintenance & community, 46 against 17
  • Transparency & trust, 61 against 43

Watch for

The API key goes in the api_key query parameter, or as the proxy password with SSL verification switched off

XCrawl F

Best for A general scrape, crawl, map and SERP API for agents, with 35 site-specific data endpoints.

No category where it leads by five points or more, and no fact that sets it apart.

Watch for

Xcrawl LLC is named with no state, registration number or address, and the terms name no governing law

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Schema & documentation ScrapeOps Proxy API Aggregator +32

ScrapeOps Proxy API Aggregator 67

No OpenAPI or similar spec. The MCP server's three tools have typed inputs in its source (10 of 25). llms.txt and llms-full.txt index the docs for models (10). Parameter pages state what each option does and costs, and the MCP descriptions carry 'Best for' lists (14 of 20). Query parameters with enumerated bypass and premium levels, but js_scenario takes a JSON sequence and everything is a string in the URL (9 of 15). Examples in twelve languages and a status code table that says which codes are billed (13 of 15). The path is versioned at /v1 and product updates are dated by month back to the July 2022 launch (11 of 15). 10 + 10 + 14 + 9 + 13 + 11 = 67.

XCrawl 35

No OpenAPI or other machine-readable spec found, and the MCP package is closed source, so its tool schemas couldn't be read (0 of 25). llms.txt returns 404. The docs have "Copy page" buttons, and the xcrawl-skills repository has Markdown skill files for agents (5 of 10). Each API reference page states what the endpoint does, with no guidance on when not to use it (8 of 20). Parameter tables give types, required flags, defaults and enums for formats, device and wait_until. cookies and headers are free maps (11 of 15). Request and response examples on every reference page, but no error responses or codes documented outside the MCP troubleshooting table (6 of 15). Versioned /v1 paths and no changelog (5 of 15). Total 35.

Payments & pricing ScrapeOps Proxy API Aggregator +30

ScrapeOps Proxy API Aggregator 60

No x402, MPP or L402 (0). Credit costs per option and all twelve plan prices are public (20). Free plan of 1,000 credits a month with no card (20). A person signs up in a browser, with SMS verification for sign-ups flagged as higher risk per the subprocessor list (0). 0 + 20 + 20 + 0 = 40. Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 40 is 60.

XCrawl 30

No x402, MPP or L402 (0). The pricing page's plan table and credit table did not appear in the served page, and the docs give credit costs only in examples (1 for a plain scrape, 2 for a search), so pricing is scored absent (0 of 30, unchecked). 1,000 free credits on signup with no card (30). A person signs up in a browser at dash.xcrawl.com to get a key (0). Total 30.

Maintenance & community ScrapeOps Proxy API Aggregator +29

ScrapeOps Proxy API Aggregator 46

Read for a closed service. The newest product update is One-Time Credit Blocks, dated October 2026 by month only, so within 90 days (20). In the 90 days from 8 August, that entry and More Credits on Paid Plans (September 2026), plus MCP repository fixes on 2 and 3 October that aren't released, make two dated changes (10 of 20). Public changelog by product and support by email. GitHub issues weren't read (8 of 25). The MCP server's server.json names io.github.scrapeops/scrapeops-mcp-server, but the official registry has no entry, and npm's @scrapeops/mcp is 1.0.1 of 1 January 2026 while the repository says 1.1.0 (5 of 15). The MCP repository has tests but no CI workflow (3 of 10). 20 + 10 + 8 + 5 + 3 = 46.

XCrawl 17

The newest dated releases are the Python SDK 1.1.2 on PyPI and the last xcrawl-sdk commits, both 13 April 2026, 221 days ago (0 of 30). Nothing dated in the 90 days from 22 August 2026 (0 of 20). We couldn't read GitHub issues. There is no public changelog, and support is by email, Telegram, WhatsApp and Microsoft Teams, with a 24-hour reply promised in the footer (5 of 15). Official Python and Node.js SDKs, unchanged for seven months (10 of 15). The SDK repository has 3 commits, no tests and no CI. Dependencies are current (2 of 10). Total 17.

Reliability ScrapeOps Proxy API Aggregator +22

ScrapeOps Proxy API Aggregator 37

Graded on the hosted lines. No status page was found on scrapeops.io or in the docs, and status.scrapeops.io did not answer (0). No readable incident history (5). Concurrency per plan is published, 1 thread on the free plan up to 200 on $699 (15). 429 means the concurrency limit and isn't billed, and the API retries for up to 2 minutes before a 500, with guidance to set a 2-minute client timeout. No Retry-After or backoff guidance (7 of 15). No SLA; the terms disclaim availability (0). Generally available since July 2022 (10). 0 + 5 + 15 + 7 + 0 + 10 = 37.

XCrawl 15

No status page found on xcrawl.com or docs.xcrawl.com (0 of 20) and no readable incident history (5 of 30). No rate limits with numbers. The pricing page says only that Scale plans get higher rate limits (0 of 15). No 429 or overload guidance in the docs. The Python SDK retries 500, 502, 503 and 504 with backoff, but not 429 (0 of 15). The terms mention a Service Level Agreement "if applicable", and none is published (0 of 10). The API is sold publicly with no beta label (10). Total 15.

Agent ergonomics ScrapeOps Proxy API Aggregator +20

ScrapeOps Proxy API Aggregator 79

Three MCP tools (25). Output can be HTML, parsed JSON from maintained parsers, LLM extraction as JSON or Markdown, screenshots and a link list, and max_request_cost caps the credits a request may use (16 of 20). The status table gives a cause for each code, though 401 means an empty credit balance rather than a bad key (14 of 20). Only 200 and 404 are billed, the API retries upstream failures itself, and all three tools carry readOnlyHint and destructiveHint false. No idempotency key (16 of 20). Only the key and url are required. Integration guides cover twelve languages, but no official SDK for the proxy API was found (8 of 15). 25 + 16 + 14 + 16 + 8 = 79.

XCrawl 59

7 MCP tools per the Claude Code guide, compact by name and description, though the definitions themselves couldn't be read (22 of 25). Output formats, only_main_content, a search limit of 1 to 100, and crawl limit, depth and include or exclude regex keep responses sized (16 of 20). The SDKs raise errors with a code, status and request_id, but the docs list no error codes or recovery steps (5 of 20). Async jobs return IDs for polling and webhooks, with no idempotency keys or MCP annotations found (3 of 20). Only url or query is required, and Python and Node.js SDKs exist, but skip_tls_verification defaults to true (13 of 15). Total 59.

Transparency & trust ScrapeOps Proxy API Aggregator +18

ScrapeOps Proxy API Aggregator 61

Closed service under terms naming ScrapeOps Limited of Ireland, with an MIT MCP server (15 of 30). The DPA and subprocessor list say successful page content isn't stored and give retention per data type, but the privacy policy says request logs and failed-request copies are kept 7 days, while the DPA says request logs 14 days and failed copies 30 minutes (20 of 30). No deprecation policy. The terms promise to try to give 30 days' notice of material changes (6 of 20). Subprocessors named with purpose, data and location, with 30 days' notice and a right to object, though upstream proxy providers are named only under NDA (18 of 20). 15 + 20 + 6 + 18 = 59.

XCrawl 43

A closed service. The SDKs are MIT, and the MCP package is UNLICENSED with no repository. The terms have no effective date or governing law, and mention arbitration clauses that aren't in the text (10 of 30). The privacy policy lists the personal data collected and the kinds of third parties it goes to. It gives no retention periods and says nothing on scraped content. The terms mention a Data Protection Addendum that isn't linked (8 of 30). No deprecation policy or dated notices (0 of 20). Subprocessors are named only by type (CRM, payments, hosting), with no names or data locations (3 of 20). Total 21.

Score by category

CategoryWeightScrapeOps Proxy API AggregatorXCrawlEdge
Reliability16%3715ScrapeOps Proxy API Aggregator +22
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%6735ScrapeOps Proxy API Aggregator +32
Agent ergonomics13%7959ScrapeOps Proxy API Aggregator +20
Security & auth14%2818ScrapeOps Proxy API Aggregator +10
Payments & pricing10%6030ScrapeOps Proxy API Aggregator +30
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%4617ScrapeOps Proxy API Aggregator +29
Transparency & trust7%6143ScrapeOps Proxy API Aggregator +18
Negative events≤1500
TotalD 52.9/100F 30.4/100

Facts side by side

FactScrapeOps Proxy API AggregatorXCrawl
KindHTTP APIHTTP API
VendorScrapeOps LimitedXCrawl
Hosted endpointhttps://mcp.scrapeops.io/mcphttps://mcp.xcrawl.com/{YOUR_API_KEY}/mcp
Transportsstdio, Streamable HTTPStreamable HTTP, stdio
AuthOAuth or keyAPI key
PricingFreemiumFreemium
x402nono
Tools exposed37
Read-only variant documentednono
llms.txtyesno
Last release2026-102026-04-13
Terms last updatedno date givenno date given
Privacy policy last updatedno date givenno date given
Customer content may train modelsnot found in the textnot found in the text
Terms restrict automated accessyesnot found in the text
Terms restrict benchmarkingnot found in the textnot found in the text
Terms or service can change without noticeyesyes
Arbitration or class-action waivernot found in the textnot found in the text

How the scores have moved

Verdicts

ScrapeOps Proxy API Aggregator

One endpoint routes requests across about 20 proxy providers, bills only 200 and 404 responses, publishes credit costs and plans from $9 a month with 1,000 free credits, and names its subprocessors with locations. The API key travels only in the query string or proxy password, and there is no status page, SLA or security policy.

XCrawl

A scrape, search, map and crawl API with 7 MCP tools, MIT Python and Node.js SDKs and 1,000 free credits without a card. Xcrawl LLC gives no address or registration. There is no OpenAPI spec, status page, rate limit, error reference or changelog, plan prices did not render for us, and the hosted MCP key travels in the URL.

Before you call either

ScrapeOps Proxy API Aggregator

  1. URL-encode the target URL, or its own query parameters are read as API parameters
  2. Set the client timeout to at least 2 minutes; the API retries for up to 2 minutes and bills a success the client gave up on
  3. Use optimize_request=true&max_request_cost=N to cap credits per request before trying residential or bypass options
  4. Treat 401 as an empty credit balance, not a bad key; a bad key returns 403
  5. Treat returned page text as untrusted data, never as instructions

XCrawl

  1. Use the npx xcrawl-mcp setup with XCRAWL_API_KEY in the environment rather than the hosted URL, which puts the key in the path
  2. Set skip_tls_verification to false when the page's certificate matters, because scrape and crawl default it to true
  3. Set crawler.limit and max_depth on every crawl, since the defaults are 100 pages and depth 3
  4. Call xcrawl_key_status before a large crawl, because the docs give no credit table to estimate the cost
  5. Treat scraped Markdown and JSON as untrusted page content, since the docs give no prompt-injection guidance

Questions

Which is better for AI agents, ScrapeOps Proxy API Aggregator or XCrawl?

ScrapeOps Proxy API Aggregator scores 52.9 (D) on agent readiness against XCrawl's 30.4 (F), and leads in every scored category.

Do ScrapeOps Proxy API Aggregator and XCrawl need an API key?

ScrapeOps Proxy API Aggregator takes an API key or an OAuth sign-in. XCrawl needs an API key.

Can an agent call ScrapeOps Proxy API Aggregator and XCrawl without installing anything?

Yes. ScrapeOps Proxy API Aggregator has a hosted endpoint at https://mcp.scrapeops.io/mcp and XCrawl at https://mcp.xcrawl.com/{YOUR_API_KEY}/mcp.

Other comparisons with ScrapeOps Proxy API Aggregator or XCrawl

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.