Admin e9f880f7f7 fix: make ranking refresh async and use JS-rendered client for ScrapeRanking ...

Two bugs caused the 'Refresh Rankings' button to silently fail in production:

1. ScrapeRanking was using the plain HTTP client (s.client) instead of the
   browserless content client (s.urlClient). The /ranking page requires
   JavaScript rendering, so a plain fetch returned HTML without any novel
   entries. Now uses s.urlClient so the page is fully rendered before scraping.

2. handleRankingRefresh was synchronous, holding the HTTP connection open for
   up to 60 s while scraping. Reverse proxies and HTMX timeouts closed the
   connection before the scrape finished. Rewritten to the same async pattern
   used for book scraping: POST /ranking/refresh returns immediately with a
   polling badge; the browser polls GET /ui/ranking/status every 3 s; when
   the goroutine finishes the status endpoint sends HX-Redirect to /ranking.

2026-03-01 16:52:03 +05:00

5.1 KiB

Raw Blame History

View Raw