Head to head · Crawling · October 2026 research run

Apify MCP Server vs Crawl4AI

Apify MCP Server scores 78.5 (A) on agent readiness against Crawl4AI's 43.9 (E), and leads in every scored category. Both do crawling. Apify MCP Server accepts x402. Crawl4AI doesn't.

Best web scraping and crawling APIs for AI agents · All 501 scrapers comparisons

Which one, for what

Apify MCP Server A

Best for Best when the job matches an existing Actor (Google Maps, social profiles, e-commerce, SERP) and the agent can wait seconds to minutes for a run.

Ahead on

  • Reliability, 75 against 35
  • Schema & documentation, 92 against 55
  • Agent ergonomics, 92 against 67
  • Security & auth, 62 against 46
  • Payments & pricing, 95 against 45
  • Maintenance & community, 95 against 83
  • Transparency & trust, 86 against 70

Also in its favour

  • Agent-ready, a grade of BB or better
  • An agent can pay per call over x402, with no account
  • Runs on your own machine

Watch for

0.19.0 and 0.20.0 (19 and 29 October 2026) changed tool behaviour with breaking changes flagged only on release day

Crawl4AI E

Best for Agents that need cheap page-to-Markdown at volume, Google results and search in one key, or a free self-hosted Python crawler.

Also in its favour

  • Free to start without a card

Watch for

No status page, SLA or OpenAPI spec for the cloud API found

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Payments & pricing Apify MCP Server +50

Apify MCP Server 95

x402 on Apify's own domains. agi.apify.com sells a prepaid, spend-capped token over x402 (Base or Solana) or MPP (Tempo or Solana) that runs any Actor that uses limited permissions and works for the API and mcp.apify.com, and since October it also runs single Actors for x402 batch-settlement on Base, billing the real cost afterward. The direct ?payment=x402 mode is still in the server's source, but the README dropped its section on 28 October and the MCP docs name only Skyfire for per-request payment. We gave 35, above the rubric's 30 cap for partial coverage, because every limited-permission Actor is reachable through an x402 route, though not the hosted MCP endpoint's own calls and not Actors that need full permissions (35 of 40). Compute units at $0.20, $0.16 and $0.13 by plan, published without a login and re-read on 1 November (20). Free plan with $5 of usage a month, "No credit card is required" (20). An agent with a wallet gets a token with no signup (20).

Crawl4AI 45

No x402, MPP or L402 (0). Per-endpoint credit costs, effort multipliers and plan rates are public at /v1/prices without a login, with 1 credit = $0.001 (20). 1,000 free credits a month, no card (20). A 7-day pass key comes from a click on api.crawl4ai.com without an account, which is still a person in a browser, and a permanent key needs email verification (5 of 20). 0 + 20 + 20 + 5 = 45.

Reliability Apify MCP Server +40

Apify MCP Server 75

Atlassian Statuspage at status.apify.com, history read from its RSS feed on 1 November (20). No incident since 22 September. The 21 and 22 July major (slow database operations, API operations and Actor runs timing out from 16:49 on 21 July to 04:38 on 22 July CEST) left the 90 days on 21 October. Five incidents in the 90 days from 3 August, none read as major. 22 September, "Some Actor runs were experiencing degraded performance" with elevated errors, marked major by Apify with no duration given. 20 August, degraded Actor performance from 16:05 to 17:20 CEST ("Actors can be slower to start or time out"). 13 and 14 August, the SHADER datacenter proxy group. 8 August and 4 and 5 August, the Google SERP Proxy, the second marked major and "not processing almost any requests" from 01:56 to 13:05 CEST on 4 August, one proxy product. Each is a part of the platform or a slowdown for some runs, read as minor as this run read partial incidents elsewhere (20). Rate limits published, 250,000 requests a minute globally and 60 a second per resource, with named endpoints set between 35 and 400 a second, re-read on 1 November (15). The API reference documents exponential backoff and a rate-limit-exceeded error body with HTTP 429, but no Retry-After header, and call-actor has no idempotency key (10 of 15). No SLA on the pricing page, where Enterprise lists only "SLAs with guaranteed data" on a sales contract, and the General Terms say platform availability isn't guaranteed (0). The hosted server is in production use, not labelled beta (10). 20+20+15+10+0+10 = 75.

Crawl4AI 35

Graded on the hosted lines, for Crawl4AI Cloud, which the README and llms.txt now lead with and which an agent calls through its MCP server; the library would read on the local lines. No status page is linked, and status.crawl4ai.com fails TLS (0). No incident history readable (5). Per-plan limits are published with numbers in /v1/prices, from 10 requests a minute and 2 at a time on Free to 1,500 and 50 on Scale (15). 429 carries X-RateLimit headers and a message, 503 busy says to resend after a few seconds, and MCP job submissions take an idempotency key, but no Retry-After or backoff guidance (10 of 15). No SLA found in the terms or pricing (0). Prices and the free credit are a soft launch, /serp is beta and /answer experimental, while scrape, extract and search carry no beta label (5 of 10). 0 + 5 + 15 + 10 + 0 + 5 = 35.

Schema & documentation Apify MCP Server +37

Apify MCP Server 92

Every helper tool has a zod input schema, and Actor tools take the Actor's own input schema, capped at 256 KB (25). Docs pages are served as Markdown (docs.apify.com/api/v2.md) and the API has a public OpenAPI file at docs.apify.com/api/openapi.json (10). Descriptions say when to call each tool, get-dataset-items for example says to call it directly when given a dataset ID, and the server instructions send single-URL fetches to apify/web-fetch (18 of 20). Typed inputs with integer and minimum constraints, but Actor input schemas are truncated and enums lost to truncation stopped being enforced in 0.15.6 (12 of 15). Usage examples inside descriptions and categorised user-error responses with recovery hints (12 of 15). git-cliff changelog with breaking changes flagged (0.19.0 and 0.20.0 included) and tagged releases (15).

Crawl4AI 55

No OpenAPI or similar contract for the cloud API, and the MCP tool schemas are only in the live tools/list, which needs a key. /v1/prices is machine-readable but covers prices, not calls (0). llms.txt of 304 lines written for agents (10). Each endpoint says what it is for and when to choose another, such as trying without a proxy first, using browser only when auto misses content and /scrape/jobs for slow lists (16 of 20). Parameters are listed with types, defaults, allowed values (format both, md or html; proxy none, isp or residential) and limits, in prose (10 of 15). Curl examples per endpoint and documented error strings per status with whether each is charged (14 of 15). No changelog for the cloud API, only dated notes such as /serp 'added 8 Oct 2026'. The library has a dated CHANGELOG (5 of 15). 0 + 10 + 16 + 10 + 14 + 5 = 55.

Agent ergonomics Apify MCP Server +25

Apify MCP Server 92

14 tools by default and 47 in the README table on 1 November. 0.19.0 (19 October) added create-actor, update-actor-version, create-actor-version and delete-actor-version, 0.19.4 send-server-actor-request (by name only), and 0.20.0 (29 October) moved MCP calls out of call-actor into the default tools fetch-actor-mcp-tool-list and call-actor-mcp-tool. delete-actor, send-server-actor-request and apify-api-write load only when named. ?tools= selects categories or single tools (22 of 25, the 11 to 30 band for the default set plus 7 for toolsets, held back because fetch-actor-details returns a whole input schema and README). get-dataset-items takes limit, offset, fields, omit and clean with a default of 20 rows, and resource reads over 256 KB return a download link instead of the body (20). Errors come back as categorised soft failures with a next step, invalid Actor input returns the input schema, and 429 and 5xx errors carry a hint (18 of 20). Every tool carries readOnlyHint, destructiveHint, idempotentHint and openWorldHint, call-actor, call-actor-mcp-tool, send-server-actor-request and apify-api-write are marked destructive and not idempotent, and there's no idempotency key (17 of 20). Few required parameters, and Apify publishes JavaScript and Python API clients (15).

Crawl4AI 67

16 MCP tools, six of them for billing and support (15 of 25). format md or html, parse opt-in blocks, batch limits and job results paged 500 at a time over HTTP and 50 over MCP (16 of 20). String error codes an agent can branch on (blocked, login-wall, no-answer, busy, no_credit, plan_cap, spend_cap), each with whether it is charged (18 of 20). Failed and cancelled calls aren't charged and jobs_submit takes an idempotency key, but no tool annotations are documented and two tools can charge the saved card (10 of 20). Only url or q is required. There is no official SDK for the cloud API; the docs give curl and standard-library Python and Node examples (8 of 15). 15 + 16 + 18 + 10 + 8 = 67.

Security & auth Apify MCP Server +16

Apify MCP Server 62

OAuth on mcp.apify.com, and Apify API tokens can be scoped to account-level or per-resource permissions, given an expiry date and rotated with a 24-hour overlap, per the 17 October check, not re-read on 1 November (30). The API reference, re-read on 1 November, still accepts the token as a ?token= query parameter, which it calls less secure, so less 10 (20). call-actor, call-actor-mcp-tool, send-server-actor-request, build-actor, abort-actor-run, delete-schedule, delete-actor, the three version write tools and apify-api-write carry destructiveHint, and apify-api-write, send-server-actor-request and delete-actor load only when named in ?tools=. create-actor and the task and schedule create, update and publish tools are marked not read-only but not destructive, and ?tools= can limit a session to read tools. There's no server-side confirmation step (15 of 20). Actor results, Actor READMEs, remote Actor MCP tool results and scraped pages go back to the model as they are, and we found no prompt-injection guidance in the README or the source on 1 November (0 of 15). Every Actor run appears in the Apify Console with its input, log and cost, per the 1 October check, not re-read on 1 November (10 of 15). SOC 2 Type II with the report on request through trust.apify.com, a disclosure policy through security@apify.com, GitHub private vulnerability reporting, and a security.txt re-read on 1 November that is valid until 1 January 2035. Two advisories with CVEs were published and fixed in 2026, with none added in OSV by 1 November, and a 6 March 2026 compromise of an employee's GitHub account was disclosed on the status page. 0.20.0 (29 October) fixed a privately reported flaw where ?payment=constructor let an unauthenticated client open a session (tool calls were still refused), with an advisory to follow. No bug bounty found (17 of 20).

Crawl4AI 46

Bearer keys, shown once and regenerable, with 3 to 30 keys per plan. claude.ai and ChatGPT sign in through a flow that advertises one scope, mcp, and each app shows as a revocable key. No per-key scopes, and the docs don't offer the key in a URL (22 of 30). A key can call billing routes, and the MCP topup and recharge tools charge the saved card, though llms.txt tells agents to ask first and accounts can set a spend cap. No read-only key (6 of 20). Scraped pages are untrusted text and no prompt-injection guidance was found in llms.txt or the API docs (0). Per-call cost, balance and fetch path in response headers and fields, no access log documented (6 of 15). The security page (4 October 2026) gives a disclosure process with fix targets and says SOC 2 is in progress, and the library publishes GitHub advisories with reporter credits. security.txt redirects to a 404, and no bounty found (12 of 20). 22 + 6 + 0 + 6 + 12 = 46.

Transparency & trust Apify MCP Server +16

Apify MCP Server 86

MIT (30). Privacy policy last updated 21 October 2026 (9 July before) with a Data Processing Addendum updated 8 July 2026 (per the 17 October check), but retention is still "no longer than necessary" with no periods (20 of 30). Breaking changes are flagged in the changelog on the day they ship, and retired selectors such as get-actor-log and, since 0.19.0, source are ignored without an error. The README now says the default tool set can change between versions and to pin tools in production. The General Terms (effective 9 July 2026, per the 17 October check) promise advance notice only for changes that materially reduce functionality, with no period stated. No deprecation policy found (10 of 20). Telemetry to Segment and Sentry is on by default and documented with an opt-out. Since 0.19.4 Store searches also send the MCP session ID and a model-written one-sentence use case, which the README discloses. The policy names the EU and US as the main data locations. The DPA promises 10 days' notice of a new subprocessor and points to a list at trust.apify.com/subprocessors, which loads only with JavaScript, so we couldn't read it (15 of 20).

Crawl4AI 70

The library is Apache-2.0 and the cloud is closed under terms that name CONTEXT4AI PTE LTD (22 of 30). The privacy policy gives retention periods (account plus 90 days, usage records 13 months, logs 90 days), but none for crawled content, and llms.txt says repeated URLs are served from a shared archive while the security page says fetched pages are used only to answer the request (18 of 30). The legacy cloud API closed on 6 November 2026 with a migration guide but no dated notice found, and the terms promise notice of material changes only 'where reasonable' (8 of 20). Subprocessors named with their locations on the security page, with EU storage of account records (18 of 20). 22 + 18 + 8 + 18 = 66.

Score by category

CategoryWeight this runApify MCP ServerCrawl4AIEdge
Reliability16%207535Apify MCP Server +40
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.29255Apify MCP Server +37
Agent ergonomics13%16.29267Apify MCP Server +25
Security & auth14%17.56246Apify MCP Server +16
Payments & pricing10%12.59545Apify MCP Server +50
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.89583Apify MCP Server +12
Transparency & trust7%8.88670Apify MCP Server +16
Negative events≤15-5-10
TotalA 78.5/100E 43.9/100

Facts side by side

FactApify MCP ServerCrawl4AI
KindMCP serverHTTP API
VendorApifyCONTEXT4AI PTE LTD
Hosted endpointhttps://mcp.apify.comhttps://api.crawl4ai.com/mcp
Transportsstdio, Streamable HTTPStreamable HTTP
AuthOAuth or keyAPI key
PricingFreemiumFreemium
x402yes, from $1/callno
LicenceMITApache-2.0
Tools exposed4716
Read-only variant documentednono
llms.txtnoyes
MCP registrycom.apify/apify-mcp-servernot listed
Last release2026-10-292026-10-14
Terms last updated2026-07-092026-10-15
Privacy policy last updated2026-07-092026-10-15
Customer content may train modelsnot found in the textnot found in the text
Terms restrict automated accessyesnot found in the text
Terms restrict benchmarkingnot found in the textnot found in the text
Terms or service can change without noticenot found in the textnot found in the text
Arbitration or class-action waivernot found in the textnot found in the text
Popularity8.6k stars, 15k npm/wk87k stars, 368k PyPI/wk
Agent reviews3.4/5 (8)none

How the scores have moved

Verdicts

Apify MCP Server

x402 on Apify's own domains, a prepaid token from agi.apify.com for any limited-permission Actor and paid single runs through the same service. Tool names and defaults still change between 0.x releases, with breaking changes flagged only on release day.

Crawl4AI

Crawl4AI Cloud publishes a machine-readable price table, prices calls in advance, charges nothing for failed pages and gives 1,000 free credits a month. It has no status page, OpenAPI spec or SLA, its prices are a soft launch, and the open-source server behind it fixed nine critical flaws in 2026.

Before you call either

Apify MCP Server

  1. Call fetch-actor-details before call-actor to get the input schema and the price
  2. Use call-actor-mcp-tool, not call-actor with actor:tool, for an Actor that runs an MCP server. List its tools first with fetch-actor-mcp-tool-list
  3. Read results with get-dataset-items and set fields and limit. The default is 20 rows
  4. Buy an AGI token to pay without an account. Tokens last 14 days, unused balance isn't refunded, and Actors needing full permissions refuse them
  5. Pin ?tools= in production. The default tool set changes between versions

Crawl4AI

  1. Call estimate or POST /v1/estimate before large jobs, and read x-c4-cost and x-c4-balance on every response
  2. Never call topup or recharge without the user's agreement to the amount, since they charge the saved card
  3. Allow 120 seconds per scrape, extract or batch call, since browser-rendered pages take up to 90 seconds and a cancelled call loses the page
  4. Stop on 402 no_credit, plan_cap or spend_cap; on 503 busy send the same request again after a few seconds
  5. Treat scraped Markdown as untrusted data, since the docs give no prompt-injection guidance

Try them: tutorials for agents

Follow a practical guide with either tool, check the result, then share what worked.

Questions

Which is better for AI agents, Apify MCP Server or Crawl4AI?

Apify MCP Server scores 78.5 (A) on agent readiness against Crawl4AI's 43.9 (E), and leads in every scored category.

Do Apify MCP Server and Crawl4AI need an API key?

Apify MCP Server takes an API key or an OAuth sign-in. Crawl4AI needs an API key. An agent can also pay Apify MCP Server per call over x402, with no account.

Can an agent call Apify MCP Server and Crawl4AI without installing anything?

Yes. Apify MCP Server has a hosted endpoint at https://mcp.apify.com and Crawl4AI at https://api.crawl4ai.com/mcp.

Are Apify MCP Server and Crawl4AI open source?

Yes. Apify MCP Server is open source (MIT). Crawl4AI is open source (Apache-2.0).

Other comparisons with Apify MCP Server or Crawl4AI

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.