Files

Admin 5b27d501af feat(v3): Caddy hardening, Watchtower, scrape fixes, docs diagrams

- Caddy: custom image with caddy-ratelimit plugin, security headers
  (X-Frame-Options, HSTS, CSP-adjacent, etc.), per-IP rate limiting on
  auth/scrape/global zones, static error pages (502/503/504), fix routing
  to remove /api/scrape/* and /api/chapter-text-preview/* direct-to-backend
  (were bypassing SvelteKit auth middleware)
- docker-compose: Caddy build context + error volume, Watchtower service
  (label-enable mode, 5 min poll), watchtower labels on backend/runner/ui
- Scraper: ScrapeChapterList uses retryGet (9 attempts, Retry-After backoff)
  to fix 429-induced chapter list failures; upTo param stops pagination early
  for range scrapes
- UI: Browse→Catalogue rename (routes, API, links), admin scrape page
  Continue/Retry buttons, +error.svelte branded error page, type cleanup
  (removed dead exports, added BookPreviewMeta/BookPreviewResponse to scraper.ts)
- Meilisearch: meta_updated field, sort=update fix, facet distribution
- Docs: reorganise into docs/d2/ and docs/mermaid/ subdirectories, update
  all diagrams to reflect Caddy/Watchtower/routing changes, add
  api-routing.d2 ownership map with auth-level colour coding, regenerate SVGs

2026-03-22 21:10:38 +05:00

5.0 KiB

Raw Blame History

API Endpoint Reference

Routing ownership map: see docs/d2/api-routing.svg (source: docs/d2/api-routing.d2) for a visual overview of which paths Caddy sends to the backend directly vs. through SvelteKit, with auth levels colour-coded.

All traffic enters through Caddy :443. Caddy routes a subset of paths directly to the Go backend (bypassing SvelteKit); everything else goes to SvelteKit, which enforces auth before proxying onward.

Health / Version

Method	Path	Auth	Description
`GET`	`/health`	—	Liveness probe. Returns `{"ok":true}`.
`GET`	`/api/version`	—	Build version + commit hash.

Scrape Jobs (admin)

Method	Path	Auth	Description
`POST`	`/scrape`	admin	Enqueue full catalogue scrape.
`POST`	`/scrape/book`	admin	Enqueue single-book scrape `{url}`.
`POST`	`/scrape/book/range`	admin	Enqueue range scrape `{url, from, to?}`.
`GET`	`/api/scrape/status`	admin	Current job status.
`GET`	`/api/scrape/tasks`	admin	All scrape task records.
`POST`	`/api/cancel-task/{id}`	admin	Cancel a pending task.

Browse / Catalogue

Method	Path	Auth	Description
`GET`	`/api/browse`	—	Live novelfire.net browse (MinIO page-1 cache). Legacy — used by save-browse subcommand.
`GET`	`/api/catalogue`	—	Primary browse endpoint. Meilisearch-backed, paginated. Params: `q`, `page`, `limit`, `genre`, `status`, `sort` (`popular`\|`new`\|`update`\|`rank`\|`top-rated`). Falls back to empty when Meilisearch is not configured.
`GET`	`/api/search`	—	Full-text search: Meilisearch local results merged with live novelfire.net remote results. Param: `q` (≥ 2 chars). Used by iOS app.
`GET`	`/api/ranking`	—	Top-ranked novels from PocketBase.
`GET`	`/api/cover/{domain}/{slug}`	—	Proxy cover image from MinIO (redirect to presigned URL).

Book / Chapter Content

Method	Path	Auth	Description
`GET`	`/api/book-preview/{slug}`	—	Returns stored metadata + chapter list, or enqueues a scrape task (202) if unknown.
`GET`	`/api/chapter-text/{slug}/{n}`	—	Chapter content as plain text (markdown stripped).
`GET`	`/api/chapter-markdown/{slug}/{n}`	—	Chapter content as raw markdown from MinIO.
`POST`	`/api/reindex/{slug}`	admin	Rebuild `chapters_idx` from MinIO objects.

Audio

Method	Path	Auth	Description
`POST`	`/api/audio/{slug}/{n}`	—	Trigger Kokoro TTS generation. Body: `{voice?}`. Returns `200 {status:"done"}` if cached, `202 {task_id, status}` if enqueued.
`GET`	`/api/audio/status/{slug}/{n}`	—	Poll audio generation status. Param: `voice`. Returns `{status, task_id?, error?}`.
`GET`	`/api/audio-proxy/{slug}/{n}`	—	Redirect to presigned MinIO audio URL.
`GET`	`/api/voices`	—	List available Kokoro voices. Returns `{voices:[]}` on error.

Presigned URLs

All presign endpoints return a 302 redirect to a short-lived MinIO presigned URL. The URL is cached in Valkey (TTL ~55 min) to avoid regenerating on every request.

Method	Path	Auth	Description
`GET`	`/api/presign/chapter/{slug}/{n}`	—	Presigned URL for chapter markdown object.
`GET`	`/api/presign/audio/{slug}/{n}`	—	Presigned URL for audio MP3. Param: `voice`.
`GET`	`/api/presign/voice-sample/{voice}`	—	Presigned URL for voice sample MP3.
`GET`	`/api/presign/avatar-upload/{userId}`	user	Presigned PUT URL for avatar upload.
`GET`	`/api/presign/avatar/{userId}`	—	Presigned GET URL for avatar image.

Reading Progress

Session-scoped (anonymous via cookie session ID, or tied to authenticated user).

Method	Path	Auth	Description
`GET`	`/api/progress`	—	Get all reading progress for the current session/user.
`POST`	`/api/progress/{slug}`	—	Set progress. Body: `{chapter}`.
`DELETE`	`/api/progress/{slug}`	—	Delete progress for a book.

Notes

Auth: The backend does not enforce auth itself — the SvelteKit UI layer enforces admin/user guards before proxying requests. The backend trusts all incoming requests.
/api/catalogue vs /api/browse: /api/catalogue is the primary UI endpoint (Meilisearch, always-local, fast). /api/browse hits or caches the live novelfire.net browse page and is only used internally by the save-browse subcommand.
Meilisearch fallback: When MEILI_URL is unset, /api/catalogue returns {books:[], has_next:false} and /api/search falls back to a PocketBase substring scan.
BACKEND_API_URL: The SvelteKit UI reads this env var (default http://localhost:8080) to reach the backend server-side. In docker-compose it is set to http://backend:8080.

5.0 KiB Raw Blame History