Files
libnovel/scraper/internal
Admin cf0c0dfaaf fix(scraper): replace browserless with direct HTTP and fix presign 404 race
- Fix audio presign 404: MinIO upload is now synchronous before response is sent,
  eliminating the race where presign was called before the file landed in MinIO
- Replace all Browserless usage with direct HTTP client across catalogue, metadata,
  ranking, and browse — novelfire.net pages are server-rendered and don't need a
  headless browser; direct is faster and more reliable
- Harden handleBrowse with 3-attempt retry loop, proper backoff, and full
  browser-like headers to reduce 502s from novelfire.net bot detection
- Remove Browserless env vars (BROWSERLESS_URL/TOKEN/STRATEGY) from main.go;
  add SCRAPER_TIMEOUT as a single timeout knob
- Clean up now-dead rejectResourceTypes var and Browserless-specific WaitFor/
  RejectResourceTypes/GotoOptions fields from scraper calls
2026-03-04 17:32:18 +05:00
..