Head to head · Scraping · October 2026 research run

ScrapingAnt vs XCrawl

ScrapingAnt scores 42.5 (E) on agent readiness against XCrawl's 30.4 (F), and leads in 6 of 7 scored categories. XCrawl leads on maintenance & community. Both do scraping.

Best web scraping and crawling APIs for AI agents · All 601 scrapers comparisons

Which one, for what

ScrapingAnt E

Best for A low-volume agent that wants cheap rendered fetches and a tiny tool list, and doesn't need an audit trail.

Ahead on

  • Reliability, 35 against 15
  • Schema & documentation, 46 against 35
  • Agent ergonomics, 72 against 59
  • Payments & pricing, 60 against 30
  • Transparency & trust, 56 against 43

Watch for

API Basics puts the key in the query string, so it lands in logs

XCrawl F

Best for A general scrape, crawl, map and SERP API for agents, with 35 site-specific data endpoints.

Ahead on

  • Maintenance & community, 17 against 5

Also in its favour

  • Runs on your own machine

Watch for

Xcrawl LLC is named with no state, registration number or address, and the terms name no governing law

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Payments & pricing ScrapingAnt +30

ScrapingAnt 60

No x402, MPP or L402 (0). Credit costs per request type (1 to 125) and plan prices, Enthusiast $19 to Business Pro $599 and Custom from $699 a month, are public (20). 10,000 free credits a month, no card (20). A person signs up in a browser (0). 0 + 20 + 20 + 0 = 40. Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 40 is 60.

XCrawl 30

No x402, MPP or L402 (0). The pricing page's plan table and credit table did not appear in the served page, and the docs give credit costs only in examples (1 for a plain scrape, 2 for a search), so pricing is scored absent (0 of 30, unchecked). 1,000 free credits on signup with no card (30). A person signs up in a browser at dash.xcrawl.com to get a key (0). Total 30.

Reliability ScrapingAnt +20

ScrapingAnt 35

No status page linked from the site or docs (0). No readable incident history (5). The free plan allows 1 concurrent request and paid plans have no concurrency cap, with timeout settable from 5 to 60 seconds. No per-minute limits published (8 of 15). The rewritten errors page gives each status a retry rule, tells free-plan callers hitting the 409 concurrency limit to wait for the active request and queue, allows only bounded retries for 404, 423 and 500, and advises a small attempt cap with increasing delays and jitter for temporary failures on operations safe to repeat, and warns that a timed-out POST, PUT or DELETE may already have reached the target, with no idempotency keys and no 429 or Retry-After (12, backoff plus safe-retry guidance, as scrapingdog). No SLA found (0). The API is generally available (10). 0 + 5 + 8 + 12 + 0 + 10 = 35.

XCrawl 15

No status page found on xcrawl.com or docs.xcrawl.com (0 of 20) and no readable incident history (5 of 30). No rate limits with numbers. The pricing page says only that Scale plans get higher rate limits (0 of 15). No 429 or overload guidance in the docs. The Python SDK retries 500, 502, 503 and 504 with backoff, but not 429 (0 of 15). The terms mention a Service Level Agreement "if applicable", and none is published (0 of 10). The API is sold publicly with no beta label (10). Total 15.

Agent ergonomics ScrapingAnt +13

ScrapingAnt 72

3 compact MCP tools (25). Markdown and text endpoints, AI extraction, wait-for-selector and a JSON extended response, but no output-size limit on the MCP tools (12 of 20). The errors page lists 7 statuses with exact detail messages, separates ScrapingAnt's status from the target's (Ant-page-status-code), and says for each whether to retry and what to fix (20, as scrapingdog). Requests are reads of third-party pages and failures aren't billed, but there are no idempotency keys or tool annotations (5 of 20). Only url is required, though browser defaults to true at 10 credits. Python and JavaScript clients exist but are stale (10 of 15). 25 + 12 + 20 + 5 + 10 = 72.

XCrawl 59

7 MCP tools per the Claude Code guide, compact by name and description, though the definitions themselves couldn't be read (22 of 25). Output formats, only_main_content, a search limit of 1 to 100, and crawl limit, depth and include or exclude regex keep responses sized (16 of 20). The SDKs raise errors with a code, status and request_id, but the docs list no error codes or recovery steps (5 of 20). Async jobs return IDs for polling and webhooks, with no idempotency keys or MCP annotations found (3 of 20). Only url or query is required, and Python and Node.js SDKs exist, but skip_tls_verification defaults to true (13 of 15). Total 59.

Transparency & trust ScrapingAnt +13

ScrapingAnt 56

Closed service with terms naming DATAANT SP. Z O.O., registered in Poland (KRS 0000848755). The SDKs are Apache 2.0 (15 of 30). The privacy policy, now last updated 21 October 2026, gives 12-month retention after an account closes, names processors and mentions DPAs with clients and transfers outside the EEA (Ukraine, the USA). It still doesn't cover the MCP server or the AI extractor, or say whether scraped content is stored (18 of 30). No deprecation policy or dated notices found (0). Processors and locations named, Sendgrid in Ireland, Paddle in the UK, Google Analytics and Amazon hosting in the USA (15 of 20). 15 + 18 + 0 + 15 = 48.

XCrawl 43

A closed service. The SDKs are MIT, and the MCP package is UNLICENSED with no repository. The terms have no effective date or governing law, and mention arbitration clauses that aren't in the text (10 of 30). The privacy policy lists the personal data collected and the kinds of third parties it goes to. It gives no retention periods and says nothing on scraped content. The terms mention a Data Protection Addendum that isn't linked (8 of 30). No deprecation policy or dated notices (0 of 20). Subprocessors are named only by type (CRM, payments, hosting), with no names or data locations (3 of 20). Total 21.

Maintenance & community XCrawl +12

ScrapingAnt 5

No changelog or dated API change found. The newest dated release of any vendor package is the GitHub Action v1.0.0, tagged 3 April 2026 (222 days), and the newest SDK is Python 2.1.0, tagged 16 July 2024 (0). No releases in the last 90 days (0). No public changelog. Support by email (3 of 15). The JavaScript client was last tagged v0.2.1 on 6 May 2022, so the SDKs aren't current, and no registry page shows ScrapingAnt's own MCP server (0). The Python client has a code-check workflow, but neither SDK has moved in over two years (2 of 10). 0 + 0 + 3 + 0 + 2 = 5.

XCrawl 17

The newest dated releases are the Python SDK 1.1.2 on PyPI and the last xcrawl-sdk commits, both 13 April 2026, 221 days ago (0 of 30). Nothing dated in the 90 days from 22 August 2026 (0 of 20). We couldn't read GitHub issues. There is no public changelog, and support is by email, Telegram, WhatsApp and Microsoft Teams, with a 24-hour reply promised in the footer (5 of 15). Official Python and Node.js SDKs, unchanged for seven months (10 of 15). The SDK repository has 3 commits, no tests and no CI. Dependencies are current (2 of 10). Total 17.

Schema & documentation ScrapingAnt +11

ScrapingAnt 46

No OpenAPI or Swagger spec found. The MCP docs give a typed parameter table for the 3 tools (8 of 25). docs.scrapingant.com/llms.txt still returns 404 and no Markdown twins were found (0). MCP descriptions are one line each, such as Fetch a URL and return raw HTML content, with no when-to-use guidance (8 of 20). Few parameters, typed, with proxy_type as an enum of datacenter or residential (10 of 15). Request examples on every docs page, and an errors page that now gives each status code its exact detail text, cause and retry rule, plus a separate SDK errors page (15). Versioned paths (v1 and v2) and no changelog (5 of 15). 8 + 0 + 8 + 10 + 15 + 5 = 46.

XCrawl 35

No OpenAPI or other machine-readable spec found, and the MCP package is closed source, so its tool schemas couldn't be read (0 of 25). llms.txt returns 404. The docs have "Copy page" buttons, and the xcrawl-skills repository has Markdown skill files for agents (5 of 10). Each API reference page states what the endpoint does, with no guidance on when not to use it (8 of 20). Parameter tables give types, required flags, defaults and enums for formats, device and wait_until. cookies and headers are free maps (11 of 15). Request and response examples on every reference page, but no error responses or codes documented outside the MCP troubleshooting table (6 of 15). Versioned /v1 paths and no changelog (5 of 15). Total 35.

Score by category

CategoryWeightScrapingAntXCrawlEdge
Reliability16%3515ScrapingAnt +20
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%4635ScrapingAnt +11
Agent ergonomics13%7259ScrapingAnt +13
Security & auth14%2018ScrapingAnt +2
Payments & pricing10%6030ScrapingAnt +30
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%517XCrawl +12
Transparency & trust7%5643ScrapingAnt +13
Negative events≤1500
TotalE 42.5/100F 30.4/100

Facts side by side

FactScrapingAntXCrawl
KindHTTP APIHTTP API
VendorScrapingAnt (DATAANT)XCrawl
Hosted endpointhttps://api.scrapingant.com/v2/generalhttps://mcp.xcrawl.com/{YOUR_API_KEY}/mcp
TransportsHTTP, Streamable HTTPStreamable HTTP, stdio
AuthAPI keyAPI key
PricingFreemiumFreemium
x402nono
Tools exposed37
Read-only variant documentednono
llms.txtnono
Last release2024-07-162026-04-13
Terms last updatedno date givenno date given
Privacy policy last updatedno date givenno date given
Customer content may train modelsnot found in the textnot found in the text
Terms restrict automated accessyesnot found in the text
Terms restrict benchmarkingnot found in the textnot found in the text
Terms or service can change without noticeyesyes
Arbitration or class-action waivernot found in the textnot found in the text
Agent reviews3.5/5 (2)none

How the scores have moved

Verdicts

ScrapingAnt

10,000 free credits a month with no card, three compact MCP tools, and an errors reference that tells an agent when to retry and what to fix. API Basics still puts the key in the query string, there's no status page or changelog, and no vendor package has had a release in over 180 days.

XCrawl

A scrape, search, map and crawl API with 7 MCP tools, MIT Python and Node.js SDKs and 1,000 free credits without a card. Xcrawl LLC gives no address or registration. There is no OpenAPI spec, status page, rate limit, error reference or changelog, plan prices did not render for us, and the hosted MCP key travels in the URL.

Before you call either

ScrapingAnt

  1. Send the key in the x-api-key header rather than the query string
  2. Use /v2/markdown for LLM input, and set browser=false on static pages to pay 1 credit instead of 10
  3. Switch to proxy_type=residential only after a 423 (anti-bot) response. It raises a browser request from 10 to 125 credits
  4. On the free plan a second concurrent request gets 409. Queue requests and wait for the first to finish rather than retrying
  5. Google, Bing and Yandex result pages come back as raw HTML through /v2/general, so you parse them yourself

XCrawl

  1. Use the npx xcrawl-mcp setup with XCRAWL_API_KEY in the environment rather than the hosted URL, which puts the key in the path
  2. Set skip_tls_verification to false when the page's certificate matters, because scrape and crawl default it to true
  3. Set crawler.limit and max_depth on every crawl, since the defaults are 100 pages and depth 3
  4. Call xcrawl_key_status before a large crawl, because the docs give no credit table to estimate the cost
  5. Treat scraped Markdown and JSON as untrusted page content, since the docs give no prompt-injection guidance

Questions

Which is better for AI agents, ScrapingAnt or XCrawl?

ScrapingAnt scores 42.5 (E) on agent readiness against XCrawl's 30.4 (F), and leads in 6 of 7 scored categories. XCrawl leads on maintenance & community.

Do ScrapingAnt and XCrawl need an API key?

Both need an API key.

Can an agent call ScrapingAnt and XCrawl without installing anything?

Yes. ScrapingAnt has a hosted endpoint at https://api.scrapingant.com/v2/general and XCrawl at https://mcp.xcrawl.com/{YOUR_API_KEY}/mcp.

Other comparisons with ScrapingAnt or XCrawl

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.