feat: add Kokoro TTS, ranking page, direct HTTP strategy, and chapter-number fix
- Add Kokoro-FastAPI TTS integration to the chapter reader UI: - Browser-side MSE streaming with paragraph-level click-to-start - Voice selector, speed slider, auto-next with prefetch of the next chapter - New GET /ui/chapter-text endpoint that strips Markdown and serves plain text - Add ranking page (novelfire /ranking scraper, WriteRanking/ReadRankingItems in writer, GET /ranking + POST /ranking/refresh + GET /ranking/view routes) with local-library annotation and one-click scrape buttons - Add StrategyDirect (plain HTTP client) as a new browser strategy; the default strategy is now 'direct' for chapter fetching and 'content' for chapter-list URL retrieval (split via BROWSERLESS_URL_STRATEGY) - Fix chapter numbering bug: numbers are now derived from the URL path (/chapter-N) rather than list position, correcting newest-first ordering - Add 'refresh <slug>' CLI sub-command to re-scrape a book from its saved source_url without knowing the original URL - Extend NovelScraper interface with RankingProvider (ScrapeRanking) - Tune scraper timeouts: wait-for-selector reduced to 5 s, GotoOptions timeout set to 60 s, content/scrape client defaults raised to 90 s - Add cover extraction fix (figure.cover > img rather than bare img.cover) - Add AGENTS.md and .aiignore for AI tooling context - Add integration tests for browser client and novelfire scraper (build tag: integration) and unit tests for chapterNumberFromURL and pagination
This commit is contained in:
@@ -27,6 +27,8 @@ type BookMeta struct {
|
||||
TotalChapters int `yaml:"total_chapters,omitempty"`
|
||||
// SourceURL is the canonical URL of the book's landing page.
|
||||
SourceURL string `yaml:"source_url"`
|
||||
// Ranking is the rank number from ranking pages.
|
||||
Ranking int `yaml:"ranking,omitempty"`
|
||||
}
|
||||
|
||||
// CatalogueEntry is a lightweight reference returned by CatalogueProvider.
|
||||
@@ -108,6 +110,13 @@ type ChapterTextProvider interface {
|
||||
ScrapeChapterText(ctx context.Context, ref ChapterRef) (Chapter, error)
|
||||
}
|
||||
|
||||
// RankingProvider can enumerate novels from a ranking page.
|
||||
type RankingProvider interface {
|
||||
// ScrapeRanking pages through the ranking list, sending BookMeta values
|
||||
// (with basic info like title, cover, genres, status, sourceURL) to the returned channel.
|
||||
ScrapeRanking(ctx context.Context) (<-chan BookMeta, <-chan error)
|
||||
}
|
||||
|
||||
// NovelScraper is the full interface that a concrete novel source must implement.
|
||||
// It composes all four provider interfaces.
|
||||
type NovelScraper interface {
|
||||
@@ -115,6 +124,7 @@ type NovelScraper interface {
|
||||
MetadataProvider
|
||||
ChapterListProvider
|
||||
ChapterTextProvider
|
||||
RankingProvider
|
||||
|
||||
// SourceName returns the human-readable name of this scraper, e.g. "novelfire.net".
|
||||
SourceName() string
|
||||
|
||||
Reference in New Issue
Block a user