Bug Fixes & Stability
This update introduces a major overhaul of the administration experience and expands geographic coverage to 195 countries. We have also refined data parsing accuracy and strengthened integration options for external monitoring systems.
The platform now periodically tests alternative scraping approaches for domains you frequently request. This breaks out of suboptimal patterns and automatically discovers faster, more reliable strategies over time.
When browser-based scraping capacity is under heavy load, requests are now intelligently routed to lighter processing methods instead of queuing. This reduces wait times and improves throughput during peak periods.
Infrastructure-related failures are now tracked separately from site-level blocks. This prevents temporary server issues from incorrectly influencing the platform's strategy choices for future requests.
New Features
6Adaptive Strategy Exploration
The platform now periodically tests alternative scraping approaches for domains you frequently request. This breaks out of suboptimal patterns and automatically discovers faster, more reliable strategies over time.
Smart Capacity Routing
When browser-based scraping capacity is under heavy load, requests are now intelligently routed to lighter processing methods instead of queuing. This reduces wait times and improves throughput during peak periods.
Discord notifications for Stripe payment events
- Creates `services/api/app/services/stripe_notifications.
Production Intelligence Audit
- 10 review findings from validated and fixed across the milestone branch.
Modernized Admin Experience
Experience a completely redesigned dashboard for managing your scraping operations. The new interface provides better visibility into your account and simplifies common administrative tasks.
Global Coverage Expanded
You can now target 195 different countries for localized scraping results, ensuring accurate data from virtually any region. This includes new priority regions for more precise fingerprinting and geo-targeting.
Improvements
8Faster Strategy Adaptation
The system now refreshes its domain performance data more frequently, allowing it to respond to site changes within days rather than months. Your requests benefit from up-to-date routing decisions.
General bug fixes and improvements
Plus 10 internal improvements for better reliability and performance.
Optimized enhanced mode Execution
We improved the reliability and speed of stealth execution by preventing redundant script injections. This leads to faster page loads and more consistent success rates on high-security targets.
Refined Search Result Parsing
Our search engine parsing logic now more accurately distinguishes between advertisements and legitimate content. This prevents false positives and ensures cleaner data delivery for your analysis.
Flexible Performance Tier Controls
You now have clearer control over forcing requests to specific performance tiers, whether you need higher capabilities or want to optimize for cost. This gives you more granular control over how your resources are utilized.
Faster System Recovery
System recovery processes are now significantly faster thanks to optimized database query patterns. This reduces downtime and ensures your jobs are resumed more quickly after maintenance.
Minimized Update Disruptions
Platform updates now feature significantly reduced connection interruptions for critical stateful services. You will experience smoother transitions and higher availability during our routine platform maintenance.
Enhanced Developer Documentation
The API reference now includes dedicated sections for authentication, scraping workflows, and session management. These updates provide clearer guidance for integrating core platform features.
Bug Fixes
28Accurate Failure Classification
Infrastructure-related failures are now tracked separately from site-level blocks. This prevents temporary server issues from incorrectly influencing the platform's strategy choices for future requests.
Isolated Subdomain Statistics
Performance data is now tracked independently for each subdomain. A single high-traffic subdomain can no longer skew the strategy used for other subdomains on the same root domain.
Improved Crawl Mode Reliability
Crawl-mode requests now benefit from the same adaptive exploration and capacity-aware routing as standard requests. This closes a gap where bulk crawling could get stuck using suboptimal strategies.
filter DuckDuckGo ad/tracking URLs from search results
py` to filter DuckDuckGo ad/tracking URLs (`duckduckgo.
Proactive daily Gemini call budget cap for AI engine
The budget is enforced via an atomic Redis INCR counter (`AI engine:gemini:calls_today`) that expires at midnight UTC.
reject unknown search request fields with 422 instead of silently ignoring
`SearchRequest` and `BatchSearchRequest` had no `model_config`, so Pydantic v2 used its default `extra="ignore"` — unknown fields like `domain_filter` were silently dropped instead of returning a vali.
AI engine quality scheduler — per-domain cooldown and empty-context guard
- Added empty-context guard: `OutputQualityAgent.
Gemini quota circuit-breaker in AI engine base agent
Adds a Redis circuit-breaker to `BaseAgent.
exclude infra failures from antibot fail_rate denominator
Antibot failure rate was computed as `antibot_fail / t_total` where `t_total` includes infra failures.
accept datetime for date_from/date_to in session-ingest list_sessions
- Fixes the alerts page 422 storm: every ~500ms poll was failing because the frontend sent a full ISO datetime but the backend only accepted `date` type.
show empty state for zero-event sessions in replay player
Zero-event sessions (61% of all sessions) showed a blank white rectangle in the replay player with no feedback.
remove explicit saturation flag delete — TTL-only expiry
With 9+ worker replicas, any replica dropping below the 80% utilisation threshold was clearing the global `worker:browser:saturated` key — neutralizing the backpressure signal while other replicas rem.
raise rrweb per-event size limit from 10KB to 512KB
rrweb full DOM snapshot events (type 2) are routinely 20–200KB but the per-event validator was capped at 10KB, causing 61% of sessions to receive zero events.
narrow exception clause to except ValueError in _BlankListEnvSource
Replace `except (ValueError, Exception):` with `except ValueError:` in `_BlankListEnvSource.
forward t4_degraded through crawl-intel fast path
- `services/worker/worker/consumers/unified_consumer.
Web PG pool idle timeout race causing log floods
The web container was logging ~8 `[PG Pool] Unexpected error on idle client: terminating connection due to idle-session timeout` errors/hour (384 in 48h).
surface Zoho error body and return 503 on invalid credentials
Zoho OAuth token refresh fails silently when the refresh token is expired/revoked.
notify wait-path callers before LRU close I/O in get_browser
Waiters should be woken then, not after close I/O completes.
add missing browser session vars to .env.production.example
Mirrors 7 browser session management env vars from `.
pool_metrics overflow critical threshold unreachable
Derives DB pool overflow alert thresholds from the actual `max_overflow` pool config instead of hardcoded constants.
wire promo_row.code into notify_deposit
code` when a promo was applied.
Gemini budget counter increments before limit check
Decrement the Gemini daily call counter when a call is rejected for exceeding the budget limit.
mask Discord webhook token in non-2xx error log
Discord webhook URLs embed a secret token as the final URL path segment.
export_session event_chunks missing deleted_at IS NULL filter
Add soft-delete filter to the `event_chunks` query in `export_session`.
Restored Node SDK Capabilities
The extraction template field is now fully restored for crawl requests in the Node SDK. This ensures consistency across all request types and simplifies template-based data extraction.
Consistent Crawl Behavior
The default crawl depth strategy has been restored to its intended sequential behavior. This ensures your crawl jobs follow a predictable path and resource usage aligns with your configurations.
Reliable Feedback Analysis
Manual feedback triggers now bypass internal activity filters, ensuring that even low-volume domains receive immediate analysis. Your input will always be processed regardless of recent traffic levels.
Accurate Usage Insights
We corrected a data filtering issue in the usage dashboard to ensure performance insights are accurately reported. You can now rely on precise metrics when analyzing resource limits and success rates.
Security
1Secure Monitoring Access
You can now use bearer token authentication to securely integrate your external monitoring tools with our metrics endpoint. This eliminates the need for complex network allowlists while keeping your data protected.
Plus 45 internal changes for stability and performance.