Commit Graph

10 Commits

Author SHA1 Message Date
Admin
acbfafb8cd fix(audio): remove speed from TTS/cache/MinIO keys; fix presigned URL host rewrite
- Speed is no longer part of Kokoro generation, in-memory cache keys, or
  MinIO object keys — audio is always generated at 1.0 and playback speed
  is applied client-side via audioEl.playbackRate
- presignAudio() now calls rewriteHost() so chapter audio URLs use the
  public MinIO endpoint (same as presignVoiceSample already did)
- docker-compose.yml: rename MINIO_PUBLIC_ENDPOINT → PUBLIC_MINIO_PUBLIC_URL
  for the ui service so SvelteKit's $env/dynamic/public picks it up
2026-03-04 19:30:46 +05:00
Admin
0d7b985469 fix(audio): return 404 from presign when audio object not yet uploaded
handlePresignAudio now checks AudioExists before presigning; previously it
would generate a valid-looking signed URL for a non-existent object, causing
the browser to get 404 from MinIO directly.

- Add AudioExists to Store interface and HybridStore
- Guard handlePresignAudio with AudioExists check, return 404 if missing
- Propagate 404 through presignAudio() in minio.ts (typed error.status=404)
- SvelteKit presign route forwards 404 to the browser instead of 500
2026-03-04 15:21:17 +05:00
Admin
11a846d043 feat(scraper): rewrite browse storage to domain/html+assets structure, populate ranking from snapshot
- Replace BrowsePageKey(genre/sort/status/type/page) with BrowseHTMLKey(domain, page) -> {domain}/html/page-{n}.html
- Add BrowseCoverKey(domain, slug) -> {domain}/assets/book-covers/{slug}.jpg
- Add SaveBrowseAsset/GetBrowseAsset for binary assets in browse bucket
- Rewrite triggerBrowseSnapshot: after storing HTML, parse it, upsert ranking records with MinIO cover keys, fire per-novel cover download goroutines
- Add handleGetCover endpoint (GET /api/cover/{domain}/{slug}) to proxy cover images from MinIO
- handleGetRanking rewrites MinIO cover keys to /api/cover/... proxy URLs
- Update save-browse CLI to use BrowseHTMLKey, populate ranking, and download covers
2026-03-04 15:13:09 +05:00
Admin
555973c053 feat(browse): add SingleFile browse-page snapshot cache via MinIO
- New MinIO bucket 'libnovel-browse' (MINIO_BUCKET_BROWSE env) for storing
  self-contained HTML snapshots of novelfire browse pages
- Store interface gains SaveBrowsePage / GetBrowsePage / BrowsePageKey methods
- handleBrowse is now cache-first: serves from MinIO snapshot when available,
  then fires a background triggerBrowseSnapshot goroutine to populate cache
  on live-fetch (de-duplicated, 90s timeout)
- New 'save-browse' CLI subcommand to bulk-capture pages via SingleFile CLI
- Dockerfile: downloads pinned single-file-x86_64-linux binary (v2.0.83),
  adds gcompat + libstdc++ to Alpine runtime for glibc compatibility
- docker-compose: adds libnovel-browse bucket init and SINGLEFILE_PATH env
- .gitignore: exclude scraper/scraper build artifact
2026-03-04 10:42:26 +05:00
Admin
f80b83309a fix(ui): install prod deps in Docker runtime; add reindex endpoint for chapters_idx
- ui/Dockerfile: copy package-lock.json and run npm ci --omit=dev in the
  runtime stage so marked (and other runtime deps) are available to
  adapter-node at startup — fixes ERR_MODULE_NOT_FOUND for 'marked'
- storage: add ReindexChapters to Store interface and HybridStore — walks
  MinIO objects for a slug, reads chapter titles, upserts chapters_idx
- server: add POST /api/reindex/{slug} to rebuild chapters_idx from MinIO
2026-03-04 00:59:36 +05:00
Admin
041099598b feat(storage): add scraping_tasks collection and Store interface methods
Adds a scraping_tasks PocketBase collection with fields for kind, status,
progress counters, timestamps, and error info. Exposes CreateScrapeTask,
UpdateScrapeTask, and ListScrapeTasks on the Store interface with
implementations in HybridStore and PocketBaseStore.
2026-03-04 00:40:17 +05:00
Admin
af3c487afb feat(e2e): use direct HTTP for chapter scraping, cap TTS at 200 chars
- e2e fixture: replace single contentClient with directClient (plain HTTP)
  for chapter/metadata/ranking + contentClient (Browserless) for urlClient
  only — matches production wiring and is significantly faster
- server: add max_chars field to audio request body; truncates stripped text
  to N runes before sending to Kokoro (used by e2e for quick TTS tests)
- fix: move RankingItem to scraper package to break novelfire→storage import
  cycle; storage.RankingItem is now a type alias for backward compat
- fix: update stale New() call in novelfire integration test (missing args)
- fix: replace removed blob-ranking methods in storage integration test with
  current per-item API (UpsertRankingItem/ListRankingItems/RankingLastUpdated)
- justfile: add test-e2e and e2e tasks
2026-03-03 20:02:15 +05:00
Admin
b8d4d94b18 refactor(ranking): replace blob cache with per-item PocketBase storage
- Replace SetRanking/GetRanking/SetRankingPageHTML/GetRankingPageHTML blob methods
  with WriteRankingItem/ReadRankingItems/RankingFreshEnough per-item operations
- Add 24h staleness gate in ScrapeRanking to skip re-scraping fresh data
- Add GET /api/ranking endpoint returning []RankingItem sorted by rank
- Remove RankingPageCacher interface and rankingCacheAdapter adapter
- Update integration tests to use new per-item upsert semantics
- Include e2e test suite (scraper/internal/e2e/)
2026-03-03 19:37:49 +05:00
Admin
33e2a4dc01 feat: add MinIO presign endpoints to scraper API and SvelteKit presign helper 2026-03-02 21:35:27 +05:00
Admin
9add9033b9 feat(v2): add internal/storage package with MinIO + PocketBase clients and Store interface 2026-03-02 14:34:25 +05:00