Niches · Web scraping & crawling
Watch pages on a schedule and report what changed
Four or more companies' docs show a full fit.
What counts as done
Each run returns the same schema for every URL, failed fetches are flagged rather than silently empty, and the per-run cost is known in advance.
How agents ask for it: tell me when this price drops
, track competitor pricing pages weekly
, alert me when this page changes
Does the job 6
The docs show it meets the test above, and nothing we read says otherwise.
| Listing | Why |
|---|---|
| Context.dev A Context Dev Inc. | Monitors watch a page, sitemap or extract target on an interval schedule, run a baseline, report changes by signed webhook or polling, and cost 1 credit per page run (10 for extract) with failed and skipped runs free. source |
| fastCRW B Nanofy OÜ | Monitors at fastcrw.com/api take cron or text schedules (15-minute minimum), scrape or crawl targets, classify pages same, new, changed, removed or error, diff tracked JSON fields, and cost 1 credit per URL per check plus 1 per judged change. source |
| Firecrawl MCP A SideGuide Technologies, Inc. (d/b/a Firecrawl) | /monitor runs cron or text schedules (5-minute minimum), returns estimatedCreditsPerMonth on create, records same, new, changed, removed or error per page, and JSON-mode changeTracking diffs named fields at 1 credit a page per check plus 1 per judged change. source |
| ScrapeGraphAI B ScrapeGraphAI, Inc. | monitor takes a cron interval, formats including json for the same typed payload each run, records change diffs, adds 5 credits only on a change and can post to a webhook. source |
| Scrapewise B BEBOTECH OÜ | Scrapers run on a schedule (daily to monthly) and return the same rows each run; failed pages aren't charged, quote_only gives the cost first and idempotency_key stops double billing. Built for price tracking. source |
| ZenRows A ZENROWS, S.L. | Batch scheduled jobs run on an interval (60 minutes minimum) or calendar, each task carries a status, only successful tasks (plus 404 and 410) are billed, and cost is estimable from the fixed rate card. source |
Does the job, with a limit 15
It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.
| Listing | Why |
|---|---|
| Apify MCP Server A Apify Technologies s.r.o. | The schedules category creates cron schedules that run Actors and acc-25 runs rank and price tracking in production, but a run can report SUCCEEDED with an empty dataset and still bill, and per-run cost is only known once the Actor's pricing is read. source |
| BrowserAct D Hongkong Ziniao Technology Co., Limited | Schedules run a published bot hourly, daily or weekly, email when results change or meet a condition, keep retry settings, and tasks report failed status, but per-run cost is an estimate of 10 to 30 credits until the task record shows it. source |
| Crawl4AI D CONTEXT4AI PTE LTD | POST /price reads prices from named sites and GET /price/history keeps past readings, but nothing runs on a schedule; the agent must call it each time. The price table lists a monitor kind the docs don't describe. source |
| Outcrawl B Aeonmind LLC (trading as Outcrawl) | Monitors take a cron schedule and notify Discord, Slack or a webhook at 1 credit a check with an optional judge, but notifications are sent once and never retried and there is no call that reads past checks or diffs. source |
| Scrape.do E PACKEND, LLC | No monitor or schedule endpoint; the agent runs the schedule itself. Amazon plugin gives the same JSON schema each run and failed requests cost nothing, but 404 and target 400 are billed. source |
| ScrapeAtlas B OD. Media & Technology LLC | No schedule or monitor endpoint; the terms say your application owns storage and scheduling. Each run returns the same per-platform schema at 1 credit and failures carry failure.code at no charge. source |
| Scrapeless C NST LABS TECH LTD. | No monitor or schedule endpoint. The Amazon and Google APIs give a stable schema per run, but failed-request billing is unstated outside the Google Search copy. source |
| ScrapeOps Proxy API Aggregator D ScrapeOps Limited | No page monitor or schedule endpoint (the Monitoring product watches your own scrapers). auto_extract gives a stable schema and only 200 and 404 are billed, so the agent's own schedule can work. source |
| ScraperAPI D ScraperAPI LLC | DataPipeline schedules projects with notifications and error reports at standard request cost (July 2026 release note), but no change diff; those pages were not opened this pass. source |
| Scrapfly BB Joam Intelligence, LLC | Crawler refresh (refresh_interval 1 hour to 90 days) re-scrapes a crawl's URLs and re-indexes only changed pages, billing each page; no diff report or alert and no typed parsers. source |
| ScrapingBee B VostokInc SAS | No monitor or schedule endpoint. The Amazon Pricing and Product APIs give the same fields each run and only 200, 404 and 410 are billed, so the agent's own schedule works. source |
| Scrapingdog E OpenData Labs Private Limited | No monitor or schedule endpoint. Amazon and Google endpoints return the same schema per run and failures aren't billed, so the agent's own schedule works. source |
| SoldFetch E SoldFetch | No schedule; the observe-prices guide tells you to run observations yourself with a fresh Idempotency-Key each time. Same schema per run, failures free and flagged, 429 and 503 named. source |
| WebScraping.AI B Urlooker LLC | No monitor or schedule endpoint. /data returns the same snake_case fields per page kind each run and failed fetches are free, so the agent's own schedule works. source |
| Zyte API BB Zyte Group Limited | No schedule or monitor in Zyte API; acc-20 ran product extraction on a schedule of their own with IFTTT alerts. Same schema per run and failed requests free, but the schedule is yours. source |
Couldn't check 1
The docs were closed to us, or silent. These carry no mark either way.
| Listing | Why |
|---|---|
| Oxylabs Web Scraper API B oxylabs, UAB | The Web API pages read have no monitor; the docs index lists a Scheduler under the Classic API that wasn't read. Dedicated scrapers return the same schema and 5xx failures are unbilled. |
Doesn't do this job 10
Listings in the category that do something else, with the reason.
| Listing | Why |
|---|---|
| Bright Data C Bright Data Ltd. | No monitor or schedule tool or endpoint in the MCP or the Web Unlocker and Fetch API docs; acc-26 notes multi-platform monitoring drains the 5,000 free requests fast. source |
| Decodo Web Data API B UAB "Data troops" | No monitor, schedule or change-tracking endpoint; templates return the same schema each call but the agent must schedule and diff itself. source |
| Diffbot C Diffbot Technologies Corp. | No monitor or schedule endpoint; Product API gives a stable schema per call but the agent must schedule and diff. source |
| jevcrawl B Rayen Mlayeh | No scheduler, monitor or diff; each call is a one-off crawl. source |
| Jina Reader C Jina AI GmbH | No monitor, schedule or change-tracking endpoint on r.jina.ai or s.jina.ai; the agent would re-fetch and diff itself. source |
| Microlink B Microlink Labs, S.L. | No monitor or schedule endpoint; ttl and staleTtl control caching, not change detection. source |
| Olostep B Olostep Technologies Inc. | No monitor, schedule or change-tracking endpoint; the agent must re-scrape and compare. source |
| Qovia D Qovia | No monitor or schedule; Shopee price history is listed as coming soon. source |
| ScrapingAnt E DATAANT SP. Z O.O. | No schedule or monitor endpoint and no typed parsers; each run is a fresh scrape whose cost depends on page length. source |
| Spider BB BAGELMEN LLC | No monitor or schedule endpoint and no typed parsers; the price-monitoring use-case page is marketing. source |
1 listings in the category were added after this reading and aren't marked yet.
What stops agents finishing
- retailer bot check blocks the datacenter pool every run
- credits expire at month end so a steady schedule still wastes them
- no idempotency so a retried schedule double-bills
Around it
Before: web-scrapers.platform-structured-data. After: notifications.push, spreadsheets.append-row.
Where this comes from
Weight 8 of 100 in Web scraping & crawling: 4 of 35 accounts; price alert, Amazon price tracking, market-intelligence hourly trigger and the Apify production list
The same as data: /api/v1/use-cases/web-scrapers.price-and-change-monitoring.json, or the MCP tool get_use_case with id web-scrapers.price-and-change-monitoring. Other jobs in this category: /api/v1/niches?category=web-scrapers.
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.