feat(v3): Caddy hardening, Watchtower, scrape fixes, docs diagrams

- Caddy: custom image with caddy-ratelimit plugin, security headers
  (X-Frame-Options, HSTS, CSP-adjacent, etc.), per-IP rate limiting on
  auth/scrape/global zones, static error pages (502/503/504), fix routing
  to remove /api/scrape/* and /api/chapter-text-preview/* direct-to-backend
  (were bypassing SvelteKit auth middleware)
- docker-compose: Caddy build context + error volume, Watchtower service
  (label-enable mode, 5 min poll), watchtower labels on backend/runner/ui
- Scraper: ScrapeChapterList uses retryGet (9 attempts, Retry-After backoff)
  to fix 429-induced chapter list failures; upTo param stops pagination early
  for range scrapes
- UI: Browse→Catalogue rename (routes, API, links), admin scrape page
  Continue/Retry buttons, +error.svelte branded error page, type cleanup
  (removed dead exports, added BookPreviewMeta/BookPreviewResponse to scraper.ts)
- Meilisearch: meta_updated field, sort=update fix, facet distribution
- Docs: reorganise into docs/d2/ and docs/mermaid/ subdirectories, update
  all diagrams to reflect Caddy/Watchtower/routing changes, add
  api-routing.d2 ownership map with auth-level colour coding, regenerate SVGs
This commit is contained in:
Admin
2026-03-22 21:10:38 +05:00
parent a85636d5db
commit 5b27d501af
60 changed files with 2485 additions and 908 deletions

View File

@@ -20,8 +20,10 @@ type MetadataProvider interface {
}
// ChapterListProvider can enumerate all chapters of a book.
// upTo > 0 stops pagination once at least upTo chapter numbers have been
// collected (early-exit optimisation for range scrapes). upTo == 0 fetches all pages.
type ChapterListProvider interface {
ScrapeChapterList(ctx context.Context, bookURL string) ([]domain.ChapterRef, error)
ScrapeChapterList(ctx context.Context, bookURL string, upTo int) ([]domain.ChapterRef, error)
}
// ChapterTextProvider can extract the readable text from a single chapter page.