Batch Scraping & Scheduled Jobs
Batch scraping and scheduled scraping are now available, letting you scrape thousands of URLs on a schedule with webhook delivery and full billing integration.
New Features
4Batch scraping endpoint
You can now submit thousands of URLs in a single POST /v1/batch request. Track progress, get webhook notifications on completion, and view itemized billing for each URL.
Scheduled scraping with cron
Set up recurring scrape jobs using cron expressions. Schedules run automatically and apply your BYOP discount when configured.
Batch and schedule dashboards
New dashboard pages for managing batch jobs and schedules, with progress tracking, error details, and usage breakdowns.
Data Processing Agreement page
A new DPA page is now available for enterprise customers who need a formal data processing agreement.
Bug Fixes
2Usage chart restored
Fixed an issue where the usage chart on the dashboard was showing a placeholder instead of real data.
Failed requests in detailed logs
Failed scrape requests now appear in your detailed usage logs, giving you full visibility into what went wrong.
Plus 53 internal changes for stability and performance.