libnovel

kamil/libnovel

Fork 0

Commit Graph

Author	SHA1	Message	Date
Admin	fb6b364382	refactor: audit, split server.go, add unit tests, and fix latent bugs - Remove dead code: browser cdp/content_scrape strategies, writer package, printUsage, downloadAndStoreCoverCLI in main.go - Fix bugs: defer-in-loop in pocketbase deleteWhere, listAll() pagination hard cap removed, splitChapterTitle off-by-one in date extraction - Split server.go (~1700 lines) into focused handler files: handlers_audio, handlers_browse, handlers_progress, handlers_ranking, handlers_scrape - Export htmlutil.AttrVal/TextContent/ResolveURL; add storage/coverutil.go to consolidate duplicate helpers - Flatten deeply nested conditionals: voices() early-return guards, ScrapeCatalogue next-link double attr scan, chapterNumberFromKey dead strings.Cut line, splitChapterTitle double-nested unit/suffix loop - Add unit tests: htmlutil (9 funcs), novelfire ScrapeMetadata (3 cases), orchestrator Run (5 cases), storage chapterNumberFromKey/splitChapterTitle (22 cases); all pass with go build/vet/test clean	2026-03-04 22:14:23 +05:00
Admin	b8d4d94b18	refactor(ranking): replace blob cache with per-item PocketBase storage - Replace SetRanking/GetRanking/SetRankingPageHTML/GetRankingPageHTML blob methods with WriteRankingItem/ReadRankingItems/RankingFreshEnough per-item operations - Add 24h staleness gate in ScrapeRanking to skip re-scraping fresh data - Add GET /api/ranking endpoint returning []RankingItem sorted by rank - Remove RankingPageCacher interface and rankingCacheAdapter adapter - Update integration tests to use new per-item upsert semantics - Include e2e test suite (scraper/internal/e2e/)	2026-03-03 19:37:49 +05:00
Admin	2f0857be45	test: add integration tests for storage, server, and scrape+store flows - scraper/internal/storage/integration_test.go: MinioClient and PocketBaseStore integration tests covering chapter/audio round-trips, presign URLs, book metadata, chapter index, ranking, progress, and audio cache CRUD - scraper/internal/storage/hybrid_integration_test.go: HybridStore end-to-end tests for metadata, chapters, ranking, progress, presign, and audio cache - scraper/internal/storage/scrape_integration_test.go: live Browserless + storage tests that scrape book metadata and first 3 chapters, store them, and verify the round-trip via HybridStore - scraper/internal/server/integration_test.go: HTTP server functional tests for health, scrape status, presign chapter, reading progress, and chapter-text endpoints against real MinIO + PocketBase backends - justfile: task runner at repo root with recipes for build, test, lint, UI, docker-compose, and individual service management	2026-03-03 14:54:43 +05:00

Author

SHA1

Message

Date

Admin

fb6b364382

refactor: audit, split server.go, add unit tests, and fix latent bugs

- Remove dead code: browser cdp/content_scrape strategies, writer package,
  printUsage, downloadAndStoreCoverCLI in main.go
- Fix bugs: defer-in-loop in pocketbase deleteWhere, listAll() pagination
  hard cap removed, splitChapterTitle off-by-one in date extraction
- Split server.go (~1700 lines) into focused handler files:
  handlers_audio, handlers_browse, handlers_progress, handlers_ranking,
  handlers_scrape
- Export htmlutil.AttrVal/TextContent/ResolveURL; add storage/coverutil.go
  to consolidate duplicate helpers
- Flatten deeply nested conditionals: voices() early-return guards,
  ScrapeCatalogue next-link double attr scan, chapterNumberFromKey dead
  strings.Cut line, splitChapterTitle double-nested unit/suffix loop
- Add unit tests: htmlutil (9 funcs), novelfire ScrapeMetadata (3 cases),
  orchestrator Run (5 cases), storage chapterNumberFromKey/splitChapterTitle
  (22 cases); all pass with go build/vet/test clean

2026-03-04 22:14:23 +05:00

Admin

b8d4d94b18

refactor(ranking): replace blob cache with per-item PocketBase storage

- Replace SetRanking/GetRanking/SetRankingPageHTML/GetRankingPageHTML blob methods
  with WriteRankingItem/ReadRankingItems/RankingFreshEnough per-item operations
- Add 24h staleness gate in ScrapeRanking to skip re-scraping fresh data
- Add GET /api/ranking endpoint returning []RankingItem sorted by rank
- Remove RankingPageCacher interface and rankingCacheAdapter adapter
- Update integration tests to use new per-item upsert semantics
- Include e2e test suite (scraper/internal/e2e/)

2026-03-03 19:37:49 +05:00

Admin

2f0857be45

test: add integration tests for storage, server, and scrape+store flows

- scraper/internal/storage/integration_test.go: MinioClient and PocketBaseStore
  integration tests covering chapter/audio round-trips, presign URLs, book
  metadata, chapter index, ranking, progress, and audio cache CRUD
- scraper/internal/storage/hybrid_integration_test.go: HybridStore end-to-end
  tests for metadata, chapters, ranking, progress, presign, and audio cache
- scraper/internal/storage/scrape_integration_test.go: live Browserless + storage
  tests that scrape book metadata and first 3 chapters, store them, and verify
  the round-trip via HybridStore
- scraper/internal/server/integration_test.go: HTTP server functional tests for
  health, scrape status, presign chapter, reading progress, and chapter-text
  endpoints against real MinIO + PocketBase backends
- justfile: task runner at repo root with recipes for build, test, lint, UI,
  docker-compose, and individual service management

2026-03-03 14:54:43 +05:00

3 Commits