- ScrapeRanking now accepts a maxPages int parameter (0 = all pages).
Each page is fetched strictly sequentially; the next page is only
requested after every entry from the current page has been sent,
so there is no pre-fetching or look-ahead.
Pagination stops automatically when no next-page link is present
or when the rank-novels container is absent/empty.
- The ranking URL pattern follows the existing catalogue convention:
/ranking?page=N (next-page link detection as the stop condition).
- Server: handleRankingRefresh reads an optional 'pages' form field
and passes it to ScrapeRanking. Timeout scales at 90 s/page.
- UI: Refresh Rankings button is now a small form with a numeric
'Pages' input (default 1), letting the user choose how many pages
to pull in one refresh without touching the server config.
Two bugs caused the 'Refresh Rankings' button to silently fail in production:
1. ScrapeRanking was using the plain HTTP client (s.client) instead of the
browserless content client (s.urlClient). The /ranking page requires
JavaScript rendering, so a plain fetch returned HTML without any novel
entries. Now uses s.urlClient so the page is fully rendered before scraping.
2. handleRankingRefresh was synchronous, holding the HTTP connection open for
up to 60 s while scraping. Reverse proxies and HTMX timeouts closed the
connection before the scrape finished. Rewritten to the same async pattern
used for book scraping: POST /ranking/refresh returns immediately with a
polling badge; the browser polls GET /ui/ranking/status every 3 s; when
the goroutine finishes the status endpoint sends HX-Redirect to /ranking.
- Add Kokoro-FastAPI TTS integration to the chapter reader UI:
- Browser-side MSE streaming with paragraph-level click-to-start
- Voice selector, speed slider, auto-next with prefetch of the next chapter
- New GET /ui/chapter-text endpoint that strips Markdown and serves plain text
- Add ranking page (novelfire /ranking scraper, WriteRanking/ReadRankingItems
in writer, GET /ranking + POST /ranking/refresh + GET /ranking/view routes)
with local-library annotation and one-click scrape buttons
- Add StrategyDirect (plain HTTP client) as a new browser strategy; the
default strategy is now 'direct' for chapter fetching and 'content'
for chapter-list URL retrieval (split via BROWSERLESS_URL_STRATEGY)
- Fix chapter numbering bug: numbers are now derived from the URL path
(/chapter-N) rather than list position, correcting newest-first ordering
- Add 'refresh <slug>' CLI sub-command to re-scrape a book from its saved
source_url without knowing the original URL
- Extend NovelScraper interface with RankingProvider (ScrapeRanking)
- Tune scraper timeouts: wait-for-selector reduced to 5 s, GotoOptions
timeout set to 60 s, content/scrape client defaults raised to 90 s
- Add cover extraction fix (figure.cover > img rather than bare img.cover)
- Add AGENTS.md and .aiignore for AI tooling context
- Add integration tests for browser client and novelfire scraper (build
tag: integration) and unit tests for chapterNumberFromURL and pagination
