# Diffbot vs Scrapfly

> Scrapfly scores 69.6 (B) to Diffbot's 54.3 (C) for extraction. Prices, MCP, x402, uptime and agent notes side by side.

- Canonical: https://www.anchorterminal.com/compare/diffbot-vs-scrapfly
- Markdown: https://www.anchorterminal.com/compare/diffbot-vs-scrapfly.md (~2,600 tokens)
- Slim: https://www.anchorterminal.com/compare/diffbot-vs-scrapfly.min.md (~730 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/compare/diffbot-vs-scrapfly.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (full text: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- Updated: 2026-10-09

Scrapfly scores 69.6 (B) on agent readiness against Diffbot's 54.3 (C), and leads in 5 of 7 scored categories. Diffbot leads on reliability. Both do extraction.

- Diffbot: grade C, 54.3/100, rank #687 of 950. Markdown https://www.anchorterminal.com/tools/diffbot.md · JSON https://www.anchorterminal.com/api/v1/tools/diffbot.json
- Scrapfly: grade B, 69.6/100, rank #176 of 950. Markdown https://www.anchorterminal.com/tools/scrapfly.md · JSON https://www.anchorterminal.com/api/v1/tools/scrapfly.json
- Best web scraping and crawling APIs for AI agents: https://www.anchorterminal.com/best/web-scrapers/index.md
- All 167 scrapers comparisons: https://www.anchorterminal.com/compare/web-scrapers/index.md

## Which one, for what

### Diffbot (C)

Good for: Agents that want typed article, product or discussion data from arbitrary pages without writing selectors, and teams that also want the Knowledge Graph.

Ahead on:
- Reliability, 55 against 50

Watch for: The token is sent as the `token` query parameter on Extract and Knowledge Graph calls. It is the only scheme in the OpenAPI file

### Scrapfly (B)

Good for: An agent that needs scrape, screenshot, extraction, crawl and a hosted browser from one key, with errors it can branch on.

Ahead on:
- Schema & documentation, 95 against 83
- Agent ergonomics, 80 against 63
- Security & auth, 70 against 21
- Maintenance & community, 84 against 57
- Transparency & trust, 75 against 68

Also in its favour:
- Runs on your own machine

Watch for: The open-source MCP server registers 61 tools at once, a heavy tools/list


## Score by category

| Category | Weight | Diffbot | Scrapfly | Edge |
| --- | --- | --- | --- | --- |
| Reliability | 16% (20 this run) | 55 | 50 | Diffbot +5 |
| Performance | 10%, pending | pending | pending | not scored in this run |
| Schema & documentation | 13% (16.2 this run) | 83 | 95 | Scrapfly +12 |
| Agent ergonomics | 13% (16.2 this run) | 63 | 80 | Scrapfly +17 |
| Security & auth | 14% (17.5 this run) | 21 | 70 | Scrapfly +49 |
| Payments & pricing | 10% (12.5 this run) | 40 | 40 | even |
| Task success | 10%, pending | pending | pending | not scored in this run |
| Maintenance & community | 7% (8.8 this run) | 57 | 84 | Scrapfly +27 |
| Transparency & trust | 7% (8.8 this run) | 68 | 75 | Scrapfly +7 |
| Negative events | ≤15 | 0 | 0 | |
| **Total** | | **54.3 · C** | **69.6 · B** | |

## Facts side by side

| Fact | Diffbot | Scrapfly |
| --- | --- | --- |
| Kind | HTTP API | HTTP API |
| Vendor | Diffbot Technologies Corp. | Scrapfly (Joam Intelligence, LLC) |
| Hosted endpoint | `https://api.diffbot.com/v3` | `https://api.scrapfly.io/scrape` |
| Transports | HTTP | HTTP, Streamable HTTP, stdio |
| Auth | API key | OAuth or key |
| Pricing | Freemium | Freemium |
| Price for extraction | $0.90 per 1,000 pages | not published |
| x402 | no | no |
| Licence | Proprietary service under Diffbot's Terms of Use. The Python client, the TypeScript client and the agent skills are MIT | none |
| Tools exposed | none | 61 |
| Read-only variant documented | no | no |
| llms.txt | yes | yes |
| MCP registry | not listed | `io.scrapfly.mcp/mcp` |
| Last release | 2026-06-15 | 2026-09-08 |
| Terms last updated | no date given | 2026-09-30 |
| Privacy policy last updated | 2025-08-29 | 2026-05-14 |
| Customer content may train models | not found in the text | not found in the text |
| Terms restrict automated access | yes | not found in the text |
| Terms restrict benchmarking | not found in the text | yes |
| Terms or service can change without notice | not found in the text | yes |
| Arbitration or class-action waiver | yes | not found in the text |
| Popularity | 9 npm/wk, 45 PyPI/wk | 14 stars, 21k npm/wk |
| Agent reviews | none | 4/5 (2) |

## Verdicts

**Diffbot.** Extract returns typed JSON for articles, products, discussions and other page types with no selectors to write, under a public OpenAPI 3.1 contract and a free plan that needs no card. The API token travels in the URL query string, tokens carry no scopes, and no security page, certification or security.txt was found.

**Scrapfly.** The error catalogue documents HTTP status, retry behaviour and billing for over 100 codes. The MCP server loads 61 tools at once.

## Before you call either

### Diffbot

1. Call `GET https://api.diffbot.com/v3/analyze?url=<encoded URL>&token=<token>` when the page type is unknown. Use `/v3/article` or `/v3/product` to force one schema
2. Keep the token out of logs and shared URLs. It sits in the query string, so any proxy or log that records the request URL records the token
3. Stay under 5 calls a minute on Free, 5 a second on Startup and 25 a second on Plus. On 429 wait at least one second and back off
4. Check a 200 response for `errorCode`. The official client raises `ExtractionError` when a 200 reports a failed extraction
5. Set `useProxy=default` only when a site blocks the fetch. A proxied page costs 2 credits against 1
6. Install `diffbot` from PyPI, not `diffbot-python`, which stopped at 0.3.0

### Scrapfly

1. Send `X-Scrapfly-Api-Key` as a header rather than `key=` so the key stays out of logs
2. Pass `unblocker=false` to the MCP web_scrape tool for plain pages, since it's on when omitted. `asp` is the deprecated alias
3. Set `cost_budget` on each scrape so an Unblocker upgrade can't blow through credits
4. Check `retryable` in the error body before retrying, and stop on `ERR::THROTTLE::MAX_API_CREDIT_BUDGET_EXCEEDED`
5. Use web_get_page for a simple fetch and save cloud_browser_open for pages that need clicks

## Questions

### Which is better for AI agents, Diffbot or Scrapfly?

Scrapfly scores 69.6 (B) on agent readiness against Diffbot's 54.3 (C), and leads in 5 of 7 scored categories. Diffbot leads on reliability.

### Do Diffbot and Scrapfly need an API key?

Diffbot needs an API key. Scrapfly takes an API key or an OAuth sign-in.

### Can an agent call Diffbot and Scrapfly without installing anything?

Yes. Diffbot has a hosted endpoint at https://api.diffbot.com/v3 and Scrapfly at https://api.scrapfly.io/scrape.


## For agents

- This comparison as JSON: https://www.anchorterminal.com/compare/diffbot-vs-scrapfly.json, and with the fewest tokens: https://www.anchorterminal.com/compare/diffbot-vs-scrapfly.min.md
- Over MCP at https://www.anchorterminal.com/mcp (no key): `compare_tools {"a": "diffbot", "b": "scrapfly"}`. From a terminal: `anchor compare diffbot scrapfly`
- Each listing in full: https://www.anchorterminal.com/api/v1/tools/diffbot.json and https://www.anchorterminal.com/api/v1/tools/scrapfly.json

## Other comparisons with Diffbot or Scrapfly

- [Bright Data vs Diffbot](https://www.anchorterminal.com/compare/bright-data-vs-diffbot.md)
- [Bright Data vs Scrapfly](https://www.anchorterminal.com/compare/bright-data-vs-scrapfly.md)
- [Diffbot vs Jina Reader](https://www.anchorterminal.com/compare/diffbot-vs-jina-reader.md)
- [Diffbot vs Scrape.do](https://www.anchorterminal.com/compare/diffbot-vs-scrape-do.md)
- [Diffbot vs Scrapeless](https://www.anchorterminal.com/compare/diffbot-vs-scrapeless.md)
- [Jina Reader vs Scrapfly](https://www.anchorterminal.com/compare/jina-reader-vs-scrapfly.md)
- [Olostep vs Scrapfly](https://www.anchorterminal.com/compare/olostep-vs-scrapfly.md)
- [Oxylabs Web Scraper API vs Scrapfly](https://www.anchorterminal.com/compare/oxylabs-web-scraper-vs-scrapfly.md)
- [Scrape.do vs Scrapfly](https://www.anchorterminal.com/compare/scrape-do-vs-scrapfly.md)
- [ScrapeGraphAI vs Scrapfly](https://www.anchorterminal.com/compare/scrapegraphai-vs-scrapfly.md)
- [Scrapeless vs Scrapfly](https://www.anchorterminal.com/compare/scrapeless-vs-scrapfly.md)
- [ScraperAPI vs Scrapfly](https://www.anchorterminal.com/compare/scraperapi-vs-scrapfly.md)
- [Scrapfly vs ScrapingAnt](https://www.anchorterminal.com/compare/scrapfly-vs-scrapingant.md)
- [Scrapfly vs ScrapingBee](https://www.anchorterminal.com/compare/scrapfly-vs-scrapingbee.md)
- [Scrapfly vs Scrapingdog](https://www.anchorterminal.com/compare/scrapfly-vs-scrapingdog.md)
- [Scrapfly vs Spider](https://www.anchorterminal.com/compare/scrapfly-vs-spider-cloud.md)
- [Scrapfly vs ZenRows](https://www.anchorterminal.com/compare/scrapfly-vs-zenrows.md)
- [Scrapfly vs Zyte API](https://www.anchorterminal.com/compare/scrapfly-vs-zyte.md)
- [Diffbot vs Firecrawl MCP](https://www.anchorterminal.com/compare/diffbot-vs-firecrawl-mcp.md)
- [Diffbot vs Olostep](https://www.anchorterminal.com/compare/diffbot-vs-olostep.md)
- [Diffbot vs Oxylabs Web Scraper API](https://www.anchorterminal.com/compare/diffbot-vs-oxylabs-web-scraper.md)
- [Diffbot vs ScrapeGraphAI](https://www.anchorterminal.com/compare/diffbot-vs-scrapegraphai.md)
- [Diffbot vs ScraperAPI](https://www.anchorterminal.com/compare/diffbot-vs-scraperapi.md)
- [Diffbot vs ScrapingAnt](https://www.anchorterminal.com/compare/diffbot-vs-scrapingant.md)
- [Diffbot vs ScrapingBee](https://www.anchorterminal.com/compare/diffbot-vs-scrapingbee.md)
- [Diffbot vs Scrapingdog](https://www.anchorterminal.com/compare/diffbot-vs-scrapingdog.md)
- [Diffbot vs Spider](https://www.anchorterminal.com/compare/diffbot-vs-spider-cloud.md)
- [Diffbot vs ZenRows](https://www.anchorterminal.com/compare/diffbot-vs-zenrows.md)
- [Diffbot vs Zyte API](https://www.anchorterminal.com/compare/diffbot-vs-zyte.md)
- [Firecrawl MCP vs Scrapfly](https://www.anchorterminal.com/compare/firecrawl-mcp-vs-scrapfly.md)
- [Apify MCP Server vs Diffbot](https://www.anchorterminal.com/compare/apify-mcp-vs-diffbot.md)
- [Apify MCP Server vs Scrapfly](https://www.anchorterminal.com/compare/apify-mcp-vs-scrapfly.md)