Head to head · Scraping · October 2026 research run

Scrapewise vs XCrawl

Scrapewise scores 62.5 (B) on agent readiness against XCrawl's 30.4 (F), and leads in every scored category. Both do scraping.

Best web scraping and crawling APIs for AI agents · All 601 scrapers comparisons

Which one, for what

Scrapewise B

Best for Agents that need e-commerce prices, Amazon and Google data, or recurring price monitoring with cleaning and product matching, paid per page in euros.

Ahead on

  • Reliability, 45 against 15
  • Schema & documentation, 79 against 35
  • Agent ergonomics, 72 against 59
  • Security & auth, 56 against 18
  • Payments & pricing, 60 against 30
  • Maintenance & community, 79 against 17
  • Transparency & trust, 54 against 43

Watch for

No status page, SLA, security.txt, disclosure policy or certification found

XCrawl F

Best for A general scrape, crawl, map and SERP API for agents, with 35 site-specific data endpoints.

Also in its favour

  • Runs on your own machine

Watch for

Xcrawl LLC is named with no state, registration number or address, and the terms name no governing law

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Maintenance & community Scrapewise +62

Scrapewise 79

Read as a closed service. Changelog items dated 18, 20, 21, 22 and 23 October 2026 (30). At least three dated changes in 90 days, the September entry and the October items among them (20). A public changelog, a support address and site chat. Two outside issues on the MCP repository from 11 and 15 October had no reply per the 15 October check; GitHub's issue pages couldn't be read on 2 November (8 of 15). Verified namespace in the official MCP registry, version 0.1.1 still the latest on 2 November (15). The Python SDK has CI and trusted publishing at version 0.1.0, Beta, with no commit since 7 October (6 of 10). 30 + 20 + 8 + 15 + 6 = 79.

XCrawl 17

The newest dated releases are the Python SDK 1.1.2 on PyPI and the last xcrawl-sdk commits, both 13 April 2026, 221 days ago (0 of 30). Nothing dated in the 90 days from 22 August 2026 (0 of 20). We couldn't read GitHub issues. There is no public changelog, and support is by email, Telegram, WhatsApp and Microsoft Teams, with a 24-hour reply promised in the footer (5 of 15). Official Python and Node.js SDKs, unchanged for seven months (10 of 15). The SDK repository has 3 commits, no tests and no CI. Dependencies are current (2 of 10). Total 17.

Schema & documentation Scrapewise +44

Scrapewise 79

Public OpenAPI 3.1 for the whole REST surface (247 operations on 2 November, 224 on 15 October) and an MCP subset (114 operations, 93 on 15 October), and MCP tools are generated from it with JSON Schema inputs (25). llms.txt, agents.md and pricing.md on scrapewise.ai, though docs.scrapewise.ai has no llms.txt (10). The data tools' descriptions say when to use each, what it returns, its price and that it is free or paid. Setup tools carry one-line descriptions (16 of 20). Required fields are marked and there are 55 enums in the MCP subset, but limits such as 1 to 20 results sit in prose, not as constraints, and paid tools wrap arguments in a body object (10 of 15). Request examples throughout, error codes listed with fixes, but the docs show two error envelope shapes and the MCP subset documents only 200, 204, 400 and 402 (10 of 15). A changelog with six entries, the newest items dated by day up to 23 October 2026, one evolving spec with no version prefix, and a one-release deprecation rule (8 of 15).

XCrawl 35

No OpenAPI or other machine-readable spec found, and the MCP package is closed source, so its tool schemas couldn't be read (0 of 25). llms.txt returns 404. The docs have "Copy page" buttons, and the xcrawl-skills repository has Markdown skill files for agents (5 of 10). Each API reference page states what the endpoint does, with no guidance on when not to use it (8 of 20). Parameter tables give types, required flags, defaults and enums for formats, device and wait_until. cookies and headers are free maps (11 of 15). Request and response examples on every reference page, but no error responses or codes documented outside the MCP troubleshooting table (6 of 15). Versioned /v1 paths and no changelog (5 of 15). Total 35.

Security & auth Scrapewise +38

Scrapewise 56

Scoped keys (USER for REST, LLM_READ and LLM_FULL for MCP) stored as hashes, revocable at once, several per account with last-used time, rotated by mint and revoke. No expiry and no finer scopes (26 of 30). LLM_READ cannot run paid tools or write, and deletes need a preview token committed within 5 minutes. Since 22 October the setup and destructive tools are no longer opt-in for a Full key, and LLM_FULL has no per-tool consent (16 of 20). A documented prompt-injection contract and guidance, but it rests on a type: "scraped" content block that the MCP specification of 25 November 2025 does not define, and the docs' statement that Anthropic's clients honour it by default was not confirmed (8 of 15). Every call is logged with a correlation id and key prefix and the wallet ledger itemises charges, but the operator sees logs only through support (6 of 15). No security.txt, disclosure policy, bug bounty or certification found, security.txt still 404 on 2 November (0 of 20).

XCrawl 18

A plain bearer API key from the dashboard with no scopes, and the MCP guide says to reissue an invalid key there (20). The recommended hosted MCP setup puts the key in the URL path, so less 10, read the way the checklist reads a key in a query string (10 of 30). The API only reads third-party pages and has no destructive account actions, but there is no read-only or least-privilege key (5 of 20). No prompt-injection guidance in the docs or the skills (0 of 15). Each response reports credits used, and xcrawl_key_status gives the balance, with no request log found (3 of 15). No security.txt, disclosure policy or bug bounty. The ISO 27001 badge on the home page links to no certificate or security page (0 of 20). Total 18.

Reliability Scrapewise +30

Scrapewise 45

Read with the hosted lines. No status page found on scrapewise.ai, the docs, llms.txt or agents.md on 2 November 2026 (0). No readable incident history (5). Rate limits documented with numbers, per account per second and per day by plan (30 a second and 50,000 a day on STARTER up to 1,000 and 100,000,000 on BUSINESS, 2 a second unauthenticated), and 30 requests a second per key on the MCP gateway, plus an hourly cost ceiling there (15). 429 carries Retry-After and X-RateLimit-Remaining, Idempotency-Key is supported on scraper create and run, and paid MCP tools take an idempotency_key remembered for 24 hours (15). No SLA found (0). The API and MCP server are not marked beta, though AI control is in a test phase (10). 0 + 5 + 15 + 15 + 0 + 10 = 45.

XCrawl 15

No status page found on xcrawl.com or docs.xcrawl.com (0 of 20) and no readable incident history (5 of 30). No rate limits with numbers. The pricing page says only that Scale plans get higher rate limits (0 of 15). No 429 or overload guidance in the docs. The Python SDK retries 500, 502, 503 and 504 with backoff, but not 429 (0 of 15). The terms mention a Service Level Agreement "if applicable", and none is published (0 of 10). The API is sold publicly with no beta label (10). Total 15.

Payments & pricing Scrapewise +30

Scrapewise 60

No machine payment protocol (0). Per-page and per-call prices in euros published without login on the pricing page, pricing.md and each data API page (20). 5 free requests with no card (20). Signup is a browser flow with email or Google, keys are minted in the portal, and top-ups go through Stripe Checkout, so a person is needed (0). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 40 is 60.

XCrawl 30

No x402, MPP or L402 (0). The pricing page's plan table and credit table did not appear in the served page, and the docs give credit costs only in examples (1 for a plain scrape, 2 for a search), so pricing is scored absent (0 of 30, unchecked). 1,000 free credits on signup with no card (30). A person signs up in a browser at dash.xcrawl.com to get a key (0). Total 30.

Agent ergonomics Scrapewise +13

Scrapewise 72

Since 22 October 2026 a Full (LLM_FULL) key lists every tool, 113 in the tool catalogue and 114 operations in the MCP subset of the spec, and the ?toolsets= address that kept the setup tools out of the default list is no longer documented, so more than 30 tools (5). A Read (LLM_READ) key lists only the read tools, a read-only subset of 48 (5 added back). Cursor pagination with limit up to 1,000, q, sort, from and to filters, and job results in pages of 100 (20). Error codes an agent can branch on, such as WALLET_INSUFFICIENT_BALANCE, scope_rejected, cost_exceeded, SCRAPER_RUN_ACTIVE and MCP_SHOP_LIMIT, with a troubleshooting table, though two envelope shapes are documented (16 of 20). Idempotency keys on paid tools and REST create and run, a same-name retry rule for list runs, and preview tokens before deletes. Annotations are documented but only some appear in the spec, and tools/list was not read (16 of 20). Few required fields and a free quote_only check, but only a Python SDK (10 of 15). 10 + 20 + 16 + 16 + 10 = 72.

XCrawl 59

7 MCP tools per the Claude Code guide, compact by name and description, though the definitions themselves couldn't be read (22 of 25). Output formats, only_main_content, a search limit of 1 to 100, and crawl limit, depth and include or exclude regex keep responses sized (16 of 20). The SDKs raise errors with a code, status and request_id, but the docs list no error codes or recovery steps (5 of 20). Async jobs return IDs for polling and webhooks, with no idempotency keys or MCP annotations found (3 of 20). Only url or query is required, and Python and Node.js SDKs exist, but skip_tls_verification defaults to true (13 of 15). Total 59.

Score by category

CategoryWeightScrapewiseXCrawlEdge
Reliability16%4515Scrapewise +30
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%7935Scrapewise +44
Agent ergonomics13%7259Scrapewise +13
Security & auth14%5618Scrapewise +38
Payments & pricing10%6030Scrapewise +30
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%7917Scrapewise +62
Transparency & trust7%5443Scrapewise +11
Negative events≤1500
TotalB 62.5/100F 30.4/100

Facts side by side

FactScrapewiseXCrawl
KindHTTP APIHTTP API
VendorBEBOTECH OÜXCrawl
Hosted endpointhttps://mcp.scrapewise.ai/mcphttps://mcp.xcrawl.com/{YOUR_API_KEY}/mcp
TransportsStreamable HTTPStreamable HTTP, stdio
AuthAPI keyAPI key
PricingPay per useFreemium
x402nono
Tools exposed1137
Read-only variant documentednono
llms.txtyesno
MCP registryai.scrapewise/scrapewisenot listed
Last release2026-10-232026-04-13
Terms last updated2025-10-29no date given
Privacy policy last updated2025-10-29no date given
Customer content may train modelsnot found in the textnot found in the text
Terms restrict automated accessnot found in the textnot found in the text
Terms restrict benchmarkingnot found in the textnot found in the text
Terms or service can change without noticenot found in the textyes
Arbitration or class-action waivernot found in the textnot found in the text
Popularity1 stars, 30 PyPI/wknone
Agent reviews3/5 (4)none

How the scores have moved

Verdicts

Scrapewise

Paid MCP tools each take a free quote_only price check and idempotency keys, keys are read-only or full-access, and euro prices start at €0.15 per 1,000 plain pages. A Full key now lists about 113 tools, and there is no status page, SLA or disclosure policy. The terms claim ownership of all scraped data.

XCrawl

A scrape, search, map and crawl API with 7 MCP tools, MIT Python and Node.js SDKs and 1,000 free credits without a card. Xcrawl LLC gives no address or registration. There is no OpenAPI spec, status page, rate limit, error reference or changelog, plan prices did not render for us, and the hosted MCP key travels in the URL.

Before you call either

Scrapewise

  1. Call each paid tool with quote_only: true first, then repeat with the same idempotency_key so a retry is charged once
  2. Expect about 113 tools from a Full key; use a Read key, which lists 48 read tools, when the task only reads saved rows or job results
  3. Use an LLM_READ key unless the task needs paid tools or writes, since LLM_FULL has no per-tool consent
  4. Treat 402 WALLET_INSUFFICIENT_BALANCE as final and ask the user to top up; on 429 wait for Retry-After, on 429 cost_exceeded wait for the next hour, and stop a run before deleting its scraper (409 SCRAPER_RUN_ACTIVE)
  5. Scrape many addresses with one scrapewise_run_scraper_url_list call (up to 5,000) and retry with the same name, never one scraper per address

XCrawl

  1. Use the npx xcrawl-mcp setup with XCRAWL_API_KEY in the environment rather than the hosted URL, which puts the key in the path
  2. Set skip_tls_verification to false when the page's certificate matters, because scrape and crawl default it to true
  3. Set crawler.limit and max_depth on every crawl, since the defaults are 100 pages and depth 3
  4. Call xcrawl_key_status before a large crawl, because the docs give no credit table to estimate the cost
  5. Treat scraped Markdown and JSON as untrusted page content, since the docs give no prompt-injection guidance

Questions

Which is better for AI agents, Scrapewise or XCrawl?

Scrapewise scores 62.5 (B) on agent readiness against XCrawl's 30.4 (F), and leads in every scored category.

Do Scrapewise and XCrawl need an API key?

Both need an API key.

Can an agent call Scrapewise and XCrawl without installing anything?

Yes. Scrapewise has a hosted endpoint at https://mcp.scrapewise.ai/mcp and XCrawl at https://mcp.xcrawl.com/{YOUR_API_KEY}/mcp.

Other comparisons with Scrapewise or XCrawl

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.