Commit Graph

2 Commits

Author SHA1 Message Date
Admin
cff0c78b4f perf(scraper): use direct HTTP for chapter text fetching, bypass Browserless
novelfire.net chapter content is server-rendered, so Browserless is not
needed. Add a dedicated chapterClient (always StrategyDirect) to Scraper
and use it in ScrapeChapterText, removing the now-irrelevant WaitFor /
RejectResourceTypes / GotoOptions fields from the ContentRequest.
2026-03-03 20:40:01 +05:00
Admin
2f0857be45 test: add integration tests for storage, server, and scrape+store flows
- scraper/internal/storage/integration_test.go: MinioClient and PocketBaseStore
  integration tests covering chapter/audio round-trips, presign URLs, book
  metadata, chapter index, ranking, progress, and audio cache CRUD
- scraper/internal/storage/hybrid_integration_test.go: HybridStore end-to-end
  tests for metadata, chapters, ranking, progress, presign, and audio cache
- scraper/internal/storage/scrape_integration_test.go: live Browserless + storage
  tests that scrape book metadata and first 3 chapters, store them, and verify
  the round-trip via HybridStore
- scraper/internal/server/integration_test.go: HTTP server functional tests for
  health, scrape status, presign chapter, reading progress, and chapter-text
  endpoints against real MinIO + PocketBase backends
- justfile: task runner at repo root with recipes for build, test, lint, UI,
  docker-compose, and individual service management
2026-03-03 14:54:43 +05:00