feat(e2e): use direct HTTP for chapter scraping, cap TTS at 200 chars

- e2e fixture: replace single contentClient with directClient (plain HTTP)
  for chapter/metadata/ranking + contentClient (Browserless) for urlClient
  only — matches production wiring and is significantly faster
- server: add max_chars field to audio request body; truncates stripped text
  to N runes before sending to Kokoro (used by e2e for quick TTS tests)
- fix: move RankingItem to scraper package to break novelfire→storage import
  cycle; storage.RankingItem is now a type alias for backward compat
- fix: update stale New() call in novelfire integration test (missing args)
- fix: replace removed blob-ranking methods in storage integration test with
  current per-item API (UpsertRankingItem/ListRankingItems/RankingLastUpdated)
- justfile: add test-e2e and e2e tasks
This commit is contained in:
Admin
2026-03-03 20:02:15 +05:00
parent b8d4d94b18
commit af3c487afb
8 changed files with 105 additions and 83 deletions

View File

@@ -3,7 +3,10 @@
// wires them together without knowing anything about the concrete provider.
package scraper
import "context"
import (
"context"
"time"
)
// ─── Domain types ────────────────────────────────────────────────────────────
@@ -58,6 +61,19 @@ type Chapter struct {
Text string
}
// RankingItem represents a single entry in the novel ranking list.
type RankingItem struct {
Rank int `json:"rank"`
Slug string `json:"slug"`
Title string `json:"title"`
Author string `json:"author,omitempty"`
Cover string `json:"cover,omitempty"`
Status string `json:"status,omitempty"`
Genres []string `json:"genres,omitempty"`
SourceURL string `json:"source_url,omitempty"`
Updated time.Time `json:"updated,omitempty"`
}
// ─── Scraping selector descriptors ───────────────────────────────────────────
// Selector describes how to locate an element in an HTML document.