refactor: audit, split server.go, add unit tests, and fix latent bugs

- Remove dead code: browser cdp/content_scrape strategies, writer package,
  printUsage, downloadAndStoreCoverCLI in main.go
- Fix bugs: defer-in-loop in pocketbase deleteWhere, listAll() pagination
  hard cap removed, splitChapterTitle off-by-one in date extraction
- Split server.go (~1700 lines) into focused handler files:
  handlers_audio, handlers_browse, handlers_progress, handlers_ranking,
  handlers_scrape
- Export htmlutil.AttrVal/TextContent/ResolveURL; add storage/coverutil.go
  to consolidate duplicate helpers
- Flatten deeply nested conditionals: voices() early-return guards,
  ScrapeCatalogue next-link double attr scan, chapterNumberFromKey dead
  strings.Cut line, splitChapterTitle double-nested unit/suffix loop
- Add unit tests: htmlutil (9 funcs), novelfire ScrapeMetadata (3 cases),
  orchestrator Run (5 cases), storage chapterNumberFromKey/splitChapterTitle
  (22 cases); all pass with go build/vet/test clean
This commit is contained in:
Admin
2026-03-04 22:14:23 +05:00
parent 7b48707cd9
commit fb6b364382
26 changed files with 2359 additions and 2392 deletions

View File

@@ -3,10 +3,8 @@ module github.com/libnovel/scraper
go 1.25.0
require (
github.com/gorilla/websocket v1.5.3
github.com/minio/minio-go/v7 v7.0.98
golang.org/x/net v0.51.0
gopkg.in/yaml.v3 v3.0.1
)
require (
@@ -27,4 +25,5 @@ require (
golang.org/x/crypto v0.48.0 // indirect
golang.org/x/sys v0.41.0 // indirect
golang.org/x/text v0.34.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)