chore: migrate to v3, Doppler secrets, clean up legacy code

- Remove all pre-v3 code: scraper, ui-v2, backend v1, ios v1+v2, legacy CI workflows - Flatten v3/ contents to repo root - Add Doppler secrets management (project=libnovel, config=prd) - Add justfile with doppler run wrappers for all docker compose commands - Strip hardcoded env fallbacks from docker-compose.yml - Add minimal README.md - Clean up .gitignore
2026-03-23 17:21:12 +05:00
parent 1118392811
commit 59e8cdb19a
522 changed files with 5259 additions and 80365 deletions
--- a/docs/api-endpoints.md
+++ b/docs/api-endpoints.md
@@ -0,0 +1,82 @@
+# API Endpoint Reference
+
+> **Routing ownership map**: see [`docs/d2/api-routing.svg`](d2/api-routing.svg) (source: [`docs/d2/api-routing.d2`](d2/api-routing.d2)) for a visual overview of which paths Caddy sends to the backend directly vs. through SvelteKit, with auth levels colour-coded.
+
+All traffic enters through **Caddy :443**. Caddy routes a subset of paths directly to the Go backend (bypassing SvelteKit); everything else goes to SvelteKit, which enforces auth before proxying onward.
+
+## Health / Version
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| `GET` | `/health` | — | Liveness probe. Returns `{"ok":true}`. |
+| `GET` | `/api/version` | — | Build version + commit hash. |
+
+## Scrape Jobs (admin)
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| `POST` | `/scrape` | admin | Enqueue full catalogue scrape. |
+| `POST` | `/scrape/book` | admin | Enqueue single-book scrape `{url}`. |
+| `POST` | `/scrape/book/range` | admin | Enqueue range scrape `{url, from, to?}`. |
+| `GET` | `/api/scrape/status` | admin | Current job status. |
+| `GET` | `/api/scrape/tasks` | admin | All scrape task records. |
+| `POST` | `/api/cancel-task/{id}` | admin | Cancel a pending task. |
+
+## Browse / Catalogue
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| `GET` | `/api/browse` | — | Live novelfire.net browse (MinIO page-1 cache). Legacy — used by save-browse subcommand. |
+| `GET` | `/api/catalogue` | — | **Primary browse endpoint.** Meilisearch-backed, paginated. Params: `q`, `page`, `limit`, `genre`, `status`, `sort` (`popular`\|`new`\|`update`\|`rank`\|`top-rated`). Falls back to empty when Meilisearch is not configured. |
+| `GET` | `/api/search` | — | Full-text search: Meilisearch local results merged with live novelfire.net remote results. Param: `q` (≥ 2 chars). Used by iOS app. |
+| `GET` | `/api/ranking` | — | Top-ranked novels from PocketBase. |
+| `GET` | `/api/cover/{domain}/{slug}` | — | Proxy cover image from MinIO (redirect to presigned URL). |
+
+## Book / Chapter Content
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| `GET` | `/api/book-preview/{slug}` | — | Returns stored metadata + chapter list, or enqueues a scrape task (202) if unknown. |
+| `GET` | `/api/chapter-text/{slug}/{n}` | — | Chapter content as plain text (markdown stripped). |
+| `GET` | `/api/chapter-markdown/{slug}/{n}` | — | Chapter content as raw markdown from MinIO. |
+| `POST` | `/api/reindex/{slug}` | admin | Rebuild `chapters_idx` from MinIO objects. |
+
+## Audio
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| `POST` | `/api/audio/{slug}/{n}` | — | Trigger Kokoro TTS generation. Body: `{voice?}`. Returns `200 {status:"done"}` if cached, `202 {task_id, status}` if enqueued. |
+| `GET` | `/api/audio/status/{slug}/{n}` | — | Poll audio generation status. Param: `voice`. Returns `{status, task_id?, error?}`. |
+| `GET` | `/api/audio-proxy/{slug}/{n}` | — | Redirect to presigned MinIO audio URL. |
+| `GET` | `/api/voices` | — | List available Kokoro voices. Returns `{voices:[]}` on error. |
+
+## Presigned URLs
+
+All presign endpoints return a `302` redirect to a short-lived MinIO presigned
+URL. The URL is cached in Valkey (TTL ~55 min) to avoid regenerating on every
+request.
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| `GET` | `/api/presign/chapter/{slug}/{n}` | — | Presigned URL for chapter markdown object. |
+| `GET` | `/api/presign/audio/{slug}/{n}` | — | Presigned URL for audio MP3. Param: `voice`. |
+| `GET` | `/api/presign/voice-sample/{voice}` | — | Presigned URL for voice sample MP3. |
+| `GET` | `/api/presign/avatar-upload/{userId}` | user | Presigned PUT URL for avatar upload. |
+| `GET` | `/api/presign/avatar/{userId}` | — | Presigned GET URL for avatar image. |
+
+## Reading Progress
+
+Session-scoped (anonymous via cookie session ID, or tied to authenticated user).
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| `GET` | `/api/progress` | — | Get all reading progress for the current session/user. |
+| `POST` | `/api/progress/{slug}` | — | Set progress. Body: `{chapter}`. |
+| `DELETE` | `/api/progress/{slug}` | — | Delete progress for a book. |
+
+## Notes
+
+- **Auth**: The backend does not enforce auth itself — the SvelteKit UI layer enforces admin/user guards before proxying requests. The backend trusts all incoming requests.
+- **`/api/catalogue` vs `/api/browse`**: `/api/catalogue` is the primary UI endpoint (Meilisearch, always-local, fast). `/api/browse` hits or caches the live novelfire.net browse page and is only used internally by the `save-browse` subcommand.
+- **Meilisearch fallback**: When `MEILI_URL` is unset, `/api/catalogue` returns `{books:[], has_next:false}` and `/api/search` falls back to a PocketBase substring scan.
+- **`BACKEND_API_URL`**: The SvelteKit UI reads this env var (default `http://localhost:8080`) to reach the backend server-side. In docker-compose it is set to `http://backend:8080`.
--- a/docs/architecture.svg
+++ b/docs/architecture.svg
--- a/docs/d2/api-routing.d2
+++ b/docs/d2/api-routing.d2
@@ -0,0 +1,201 @@
+direction: right
+
+# ─── Legend ───────────────────────────────────────────────────────────────────
+
+legend: Legend {
+  style.fill: "#fafafa"
+  style.stroke: "#d4d4d8"
+
+  pub: public {
+    style.fill: "#f0fdf4"
+    style.font-color: "#15803d"
+    style.stroke: "#86efac"
+  }
+  user: user auth {
+    style.fill: "#eff6ff"
+    style.font-color: "#1d4ed8"
+    style.stroke: "#93c5fd"
+  }
+  adm: admin only {
+    style.fill: "#fff7ed"
+    style.font-color: "#c2410c"
+    style.stroke: "#fdba74"
+  }
+}
+
+# ─── Client ───────────────────────────────────────────────────────────────────
+
+client: Browser / iOS App {
+  shape: person
+  style.fill: "#fff9e6"
+}
+
+# ─── Caddy ────────────────────────────────────────────────────────────────────
+
+caddy: Caddy :443 {
+  shape: rectangle
+  style.fill: "#f1f5f9"
+  label: "Caddy :443\ncustom build · caddy-ratelimit\nsecurity headers · rate limiting\nstatic error pages"
+}
+
+# ─── SvelteKit UI ─────────────────────────────────────────────────────────────
+# Handles: auth enforcement, session, all /api/* routes that have SK counterparts
+
+sk: SvelteKit UI :3000 {
+  style.fill: "#fef3c7"
+
+  auth: Auth {
+    style.fill: "#fde68a"
+    style.stroke: "#f59e0b"
+    label: "POST /api/auth/login\nPOST /api/auth/register\nPOST /api/auth/change-password\nGET  /api/auth/session"
+  }
+
+  catalogue_sk: Catalogue {
+    style.fill: "#f0fdf4"
+    style.stroke: "#86efac"
+    label: "GET /api/catalogue-page\nGET /api/search"
+  }
+
+  book_sk: Book {
+    style.fill: "#f0fdf4"
+    style.stroke: "#86efac"
+    label: "GET /api/book/{slug}\nGET /api/chapter/{slug}/{n}\nGET /api/chapter-text-preview/{slug}/{n}"
+  }
+
+  scrape_sk: Scrape (admin) {
+    style.fill: "#fff7ed"
+    style.stroke: "#fdba74"
+    label: "GET  /api/scrape/status\nGET  /api/scrape/tasks\nPOST /api/scrape\nPOST /api/scrape/range\nPOST /api/scrape/cancel/{id}"
+  }
+
+  audio_sk: Audio {
+    style.fill: "#f0fdf4"
+    style.stroke: "#86efac"
+    label: "POST /api/audio/{slug}/{n}\nGET  /api/audio/status/{slug}/{n}\nGET  /api/voices"
+  }
+
+  presign_sk: Presigned URLs {
+    style.fill: "#f0fdf4"
+    style.stroke: "#86efac"
+    label: "GET /api/presign/chapter/{slug}/{n}\nGET /api/presign/audio/{slug}/{n}\nGET /api/presign/voice-sample/{voice}"
+  }
+
+  presign_user: Presigned URLs (user) {
+    style.fill: "#eff6ff"
+    style.stroke: "#93c5fd"
+    label: "GET /api/presign/avatar-upload/{userId}\nGET /api/presign/avatar/{userId}"
+  }
+
+  progress_sk: Progress {
+    style.fill: "#f0fdf4"
+    style.stroke: "#86efac"
+    label: "GET    /api/progress\nPOST   /api/progress/{slug}\nDELETE /api/progress/{slug}"
+  }
+
+  library_sk: Library {
+    style.fill: "#f0fdf4"
+    style.stroke: "#86efac"
+    label: "GET    /api/library\nPOST   /api/library/{slug}\nDELETE /api/library/{slug}"
+  }
+
+  comments_sk: Comments {
+    style.fill: "#f0fdf4"
+    style.stroke: "#86efac"
+    label: "GET  /api/comments/{slug}\nPOST /api/comments/{slug}"
+  }
+}
+
+# ─── Go Backend ───────────────────────────────────────────────────────────────
+# Caddy proxies these paths directly — no SvelteKit auth layer
+
+be: Backend API :8080 {
+  style.fill: "#eef3ff"
+
+  health_be: Health {
+    style.fill: "#f0fdf4"
+    style.stroke: "#86efac"
+    label: "GET /health\nGET /api/version"
+  }
+
+  scrape_be: Scrape admin (direct) {
+    style.fill: "#fff7ed"
+    style.stroke: "#fdba74"
+    label: "POST /scrape\nPOST /scrape/book\nPOST /scrape/book/range"
+  }
+
+  catalogue_be: Catalogue {
+    style.fill: "#f0fdf4"
+    style.stroke: "#86efac"
+    label: "GET /api/browse\nGET /api/catalogue\nGET /api/ranking\nGET /api/cover/{domain}/{slug}"
+  }
+
+  book_be: Book / Chapter {
+    style.fill: "#f0fdf4"
+    style.stroke: "#86efac"
+    label: "GET  /api/book-preview/{slug}\nGET  /api/chapter-text/{slug}/{n}\nGET  /api/chapter-markdown/{slug}/{n}\nPOST /api/reindex/{slug}  ⚠ admin"
+  }
+
+  audio_be: Audio {
+    style.fill: "#f0fdf4"
+    style.stroke: "#86efac"
+    label: "GET /api/audio-proxy/{slug}/{n}\nGET /api/voices"
+  }
+}
+
+# ─── Storage ──────────────────────────────────────────────────────────────────
+
+storage: Storage {
+  style.fill: "#eaf7ea"
+
+  pb: PocketBase :8090 {
+    shape: cylinder
+    label: "auth · books · progress\ncomments · library\nscrape_jobs · audio_cache"
+  }
+  mn: MinIO :9000 {
+    shape: cylinder
+    label: "chapters · audio\navatars · browse"
+  }
+  ms: Meilisearch :7700 {
+    shape: cylinder
+    label: "index: books"
+  }
+  vk: Valkey :6379 {
+    shape: cylinder
+    label: "presign URL cache"
+  }
+}
+
+# ─── Caddy routing ────────────────────────────────────────────────────────────
+
+client -> caddy: HTTPS :443
+
+caddy -> sk: "/* (catch-all)\n→ SvelteKit handles auth"
+caddy -> be: "/health  /scrape*\n/api/browse  /api/book-preview/*\n/api/chapter-text/*  /api/chapter-markdown/*\n/api/reindex/*  /api/cover/*\n/api/audio-proxy/*  /api/catalogue  /api/ranking"
+caddy -> storage.mn: "/avatars/*\n/audio/*\n/chapters/*\n(presigned MinIO GETs)"
+
+# ─── SvelteKit → Backend (server-side proxy) ──────────────────────────────────
+
+sk.catalogue_sk -> be.catalogue_be: internal proxy
+sk.book_sk -> be.book_be: internal proxy
+sk.audio_sk -> be.audio_be: internal proxy
+sk.presign_sk -> storage.vk: check cache
+sk.presign_sk -> storage.mn: generate presign
+sk.presign_user -> storage.mn: generate presign
+
+# ─── SvelteKit → Storage (direct) ────────────────────────────────────────────
+
+sk.auth -> storage.pb: sessions / users
+sk.scrape_sk -> storage.pb: scrape job records
+sk.progress_sk -> storage.pb
+sk.library_sk -> storage.pb
+sk.comments_sk -> storage.pb
+
+# ─── Backend → Storage ────────────────────────────────────────────────────────
+
+be.catalogue_be -> storage.ms: full-text search
+be.catalogue_be -> storage.pb: ranking records
+be.catalogue_be -> storage.mn: cover presign
+be.book_be -> storage.mn: chapter objects
+be.book_be -> storage.pb: book metadata
+be.audio_be -> storage.mn: audio presign
+be.audio_be -> storage.vk: presign cache
--- a/docs/d2/api-routing.svg
+++ b/docs/d2/api-routing.svg
--- a/docs/d2/architecture.d2
+++ b/docs/d2/architecture.d2
@@ -12,6 +12,11 @@ kokoro: Kokoro-FastAPI TTS {
  style.fill: "#f0f4ff"
 }

+letsencrypt: Let's Encrypt {
+  shape: cloud
+  style.fill: "#f0f4ff"
+}
+
 browser: Browser / iOS App {
  shape: person
  style.fill: "#fff9e6"
@@ -41,13 +46,23 @@ storage: Storage {

  minio: MinIO {
    shape: cylinder
-    label: "MinIO :9000\n\nbuckets:\n  libnovel-chapters\n  libnovel-audio\n  libnovel-avatars\n  libnovel-browse"
+    label: "MinIO :9000\n\nbuckets:\n  chapters\n  audio\n  avatars\n  catalogue"
  }

  pocketbase: PocketBase {
    shape: cylinder
    label: "PocketBase :8090\n\ncollections:\n  books  chapters_idx\n  audio_cache  progress\n  scrape_jobs  app_users\n  ranking"
  }
+
+  valkey: Valkey {
+    shape: cylinder
+    label: "Valkey :6379\n\n(presign URL cache\nTTL-based, shared)"
+  }
+
+  meilisearch: Meilisearch {
+    shape: cylinder
+    label: "Meilisearch :7700\n\nindices:\n  books"
+  }
 }

 # ─── Application ──────────────────────────────────────────────────────────────
@@ -55,6 +70,11 @@ storage: Storage {
 app: Application {
  style.fill: "#eef3ff"

+  caddy: caddy {
+    shape: rectangle
+    label: "Caddy :443 / :80\ncustom build + caddy-ratelimit\n\nfeatures:\n  auto-HTTPS (Let's Encrypt)\n  security headers\n  rate limiting (per-IP)\n  static error pages (502/503/504)"
+  }
+
  backend: backend {
    shape: rectangle
    label: "Backend API :8080\n(Go — HTTP API server)"
@@ -62,12 +82,23 @@ app: Application {

  runner: runner {
    shape: rectangle
-    label: "Runner\n(Go — background worker\nscraping + TTS jobs)"
+    label: "Runner :9091\n(Go — background worker\nscraping + TTS jobs\n/metrics endpoint)"
  }

  ui: ui {
    shape: rectangle
-    label: "SvelteKit UI :5252\n(adapter-node)"
+    label: "SvelteKit UI :3000\n(adapter-node)"
+  }
+}
+
+# ─── Ops ──────────────────────────────────────────────────────────────────────
+
+ops: Ops {
+  style.fill: "#fef9ec"
+
+  watchtower: Watchtower {
+    shape: rectangle
+    label: "Watchtower\n(containrrr/watchtower)\n\npolls every 5 min\nautopulls + redeploys:\n  backend · runner · ui"
  }
 }

@@ -80,20 +111,44 @@ init.pb-init -> storage.pocketbase: bootstrap schema {style.stroke-dash: 4}

 app.backend -> storage.minio: blobs (chapters, audio,\navatars, browse)
 app.backend -> storage.pocketbase: structured records\n(books, progress, jobs…)
+app.backend -> storage.valkey: cache presigned URLs\n(SET/GET with TTL)

 app.runner -> storage.minio: write chapter markdown\n& audio MP3s
 app.runner -> storage.pocketbase: read/update scrape jobs\nwrite book records
+app.runner -> storage.meilisearch: index books on\nscrape completion
+
+app.ui -> storage.valkey: read presigned URL cache
+app.ui -> storage.pocketbase: auth, progress,\ncomments, settings

 # ─── App internal ─────────────────────────────────────────────────────────────

-app.ui -> app.backend: REST API calls\n(server-side)
+app.ui -> app.backend: REST API calls (server-side)\n/api/catalogue /api/book-preview\n/api/chapter-text /api/audio etc.
+
+# ─── Caddy routing ────────────────────────────────────────────────────────────
+# Routes sent directly to backend (no SvelteKit counterpart):
+#   /health  /scrape*
+#   /api/browse  /api/book-preview/*  /api/chapter-text/*
+#   /api/reindex/*  /api/cover/*  /api/audio-proxy/*
+# Routes sent to MinIO:
+#   /avatars/*
+# Everything else → SvelteKit UI (including /api/scrape/*, /api/chapter-text-preview/*)
+
+app.caddy -> app.ui: "/* (catch-all)\n/api/scrape/*\n/api/chapter-text-preview/*\n→ SvelteKit (auth enforced)"
+app.caddy -> app.backend: "/health  /scrape*\n/api/browse  /api/book-preview/*\n/api/chapter-text/*\n/api/reindex/*  /api/cover/*\n/api/audio-proxy/*"
+app.caddy -> storage.minio: "/avatars/*\n/audio/*\n/chapters/*\n(presigned MinIO GETs)"

 # ─── External → App ───────────────────────────────────────────────────────────

 app.runner -> novelfire: scrape\n(HTTP GET)
 app.runner -> kokoro: TTS generation\n(HTTP POST)
+app.caddy -> letsencrypt: ACME certificate\n(TLS-ALPN-01)
+
+# ─── Ops → Docker socket ──────────────────────────────────────────────────────
+
+ops.watchtower -> app.backend: watch (label-enabled)
+ops.watchtower -> app.runner: watch (label-enabled)
+ops.watchtower -> app.ui: watch (label-enabled)

 # ─── Browser ──────────────────────────────────────────────────────────────────

-browser -> app.ui: HTTPS :5252
-browser -> storage.minio: presigned URLs\n(audio / chapter downloads)
+browser -> app.caddy: HTTPS :443\n(single entry point)
--- a/docs/d2/architecture.svg
+++ b/docs/d2/architecture.svg
--- a/docs/mermaid/architecture.mermaid.md
+++ b/docs/mermaid/architecture.mermaid.md
@@ -1,8 +1,11 @@
+# Architecture Overview
+
 ```mermaid
 graph LR
    %% ── External ──────────────────────────────────────────────────────────
    NF([novelfire.net])
    KK([Kokoro-FastAPI TTS])
+    LE([Let's Encrypt])
    CL([Browser / iOS App])

    %% ── Init containers ───────────────────────────────────────────────────
@@ -15,13 +18,21 @@ graph LR
    subgraph STORAGE["Storage"]
        MN[(MinIO :9000\nchapters · audio\navatars · browse)]
        PB[(PocketBase :8090\nbooks · chapters_idx\naudio_cache · progress\nscrape_jobs · app_users · ranking)]
+        VK[(Valkey :6379\npresign URL cache\nTTL-based · shared)]
+        MS[(Meilisearch :7700\nindex: books)]
    end

    %% ── Application ───────────────────────────────────────────────────────
    subgraph APP["Application"]
+        CD["Caddy :443/:80\ncustom build + caddy-ratelimit\nauto-HTTPS · security headers\nrate limiting · error pages"]
        BE[Backend API :8080\nGo HTTP server]
-        RN[Runner\nGo background worker]
-        UI[SvelteKit UI :5252]
+        RN[Runner :9091\nGo background worker\n/metrics endpoint]
+        UI[SvelteKit UI :3000\nadapter-node]
+    end
+
+    %% ── Ops ───────────────────────────────────────────────────────────────
+    subgraph OPS["Ops"]
+        WT[Watchtower\npolls every 5 min\nautopull + redeploy\nbackend · runner · ui]
    end

    %% ── Init → Storage ────────────────────────────────────────────────────
@@ -31,17 +42,31 @@ graph LR
    %% ── App → Storage ─────────────────────────────────────────────────────
    BE -->|blobs| MN
    BE -->|structured records| PB
+    BE -->|cache presigned URLs| VK
    RN -->|chapter markdown & audio| MN
    RN -->|read/update jobs & books| PB
+    RN -->|index books on scrape| MS
+    UI -->|read presign cache| VK
+    UI -->|auth · progress · comments| PB

    %% ── App internal ──────────────────────────────────────────────────────
-    UI -->|REST API| BE
+    UI -->|"REST API (server-side)\n/api/catalogue /api/book-preview\n/api/chapter-text /api/audio"| BE
+
+    %% ── Caddy routing ─────────────────────────────────────────────────────
+    CD -->|"/* catch-all\n/api/scrape/*\n/api/chapter-text-preview/*\n→ SvelteKit (auth enforced)"| UI
+    CD -->|"/health /scrape*\n/api/browse /api/book-preview/*\n/api/chapter-text/*\n/api/reindex/* /api/cover/*\n/api/audio-proxy/*"| BE
+    CD -->|/avatars/* presigned GETs| MN

    %% ── Runner → External ─────────────────────────────────────────────────
    RN -->|scrape HTTP GET| NF
    RN -->|TTS HTTP POST| KK
+    CD -->|ACME certificate| LE
+
+    %% ── Ops ───────────────────────────────────────────────────────────────
+    WT -->|watch label-enabled| BE
+    WT -->|watch label-enabled| RN
+    WT -->|watch label-enabled| UI

    %% ── Client ────────────────────────────────────────────────────────────
-    CL -->|HTTPS :5252| UI
-    CL -->|presigned URLs| MN
+    CL -->|HTTPS :443 single entry| CD
 ```
--- a/docs/mermaid/data-flow.mermaid.md
+++ b/docs/mermaid/data-flow.mermaid.md
@@ -0,0 +1,102 @@
+# Data Flow — Scrape & TTS Job Pipeline
+
+How content moves from novelfire.net through the runner into storage, and how
+audio is generated on-demand via the backend.
+
+## Catalogue Scrape Pipeline
+
+The runner performs a background catalogue walk on startup and then on a
+configurable interval (`RUNNER_CATALOGUE_REFRESH_INTERVAL`, default 24 h).
+
+```mermaid
+flowchart TD
+    A([Runner starts / refresh tick]) --> B[Walk novelfire.net catalogue\npages 1…N]
+    B --> C{Book already\nin PocketBase?}
+    C -- no --> D[Scrape book metadata\ntitle · author · genres\ncover · summary · status]
+    C -- yes --> E[Check for new chapters\ncompare total_chapters]
+    D --> F[Write BookMeta\nto PocketBase books]
+    E --> G{New chapters\nfound?}
+    G -- no --> Z([Done — next book])
+    G -- yes --> H
+    F --> H[Scrape chapter list with upTo limit\n→ chapters_idx in PocketBase\nretries on 429 with Retry-After backoff]
+    H --> I[Worker pool — N goroutines\nRUNNER_MAX_CONCURRENT_SCRAPE]
+    I --> J[For each missing chapter:\nGET chapter HTML from novelfire.net]
+    J --> K[Parse HTML → Markdown\nhtmlutil.NodeToMarkdown]
+    K --> L[PUT object to MinIO\nchapters/{slug}/{n}.md]
+    L --> M[Upsert book doc\nto Meilisearch index: books]
+    M --> Z
+    F --> M
+```
+
+## On-Demand Single-Book Scrape
+
+Triggered when a user visits `/books/{slug}` and the book is not in PocketBase.
+The UI calls `GET /api/book-preview/{slug}` → backend enqueues a scrape task.
+
+```mermaid
+sequenceDiagram
+    actor U as User
+    participant UI as SvelteKit UI
+    participant BE as Backend API
+    participant TQ as Task Queue (PocketBase)
+    participant RN as Runner
+    participant NF as novelfire.net
+    participant PB as PocketBase
+    participant MN as MinIO
+    participant MS as Meilisearch
+
+    U->>UI: Visit /books/{slug}
+    UI->>BE: GET /api/book-preview/{slug}
+    BE->>PB: getBook(slug) — not found
+    BE->>TQ: INSERT scrape_task (slug, status=pending)
+    BE-->>UI: 202 {task_id, message}
+    UI-->>U: "Scraping…" placeholder
+
+    RN->>TQ: Poll for pending tasks
+    TQ-->>RN: scrape_task (slug)
+    RN->>NF: GET novelfire.net/book/{slug}
+    NF-->>RN: HTML
+    RN->>PB: upsert book + chapters_idx
+    RN->>MN: PUT chapter objects
+    RN->>MS: UpsertBook doc
+    RN->>TQ: UPDATE task status=done
+
+    U->>UI: Poll GET /api/scrape/tasks/{task_id}
+    UI->>BE: GET /api/scrape/status
+    BE->>TQ: get task
+    TQ-->>BE: status=done
+    BE-->>UI: {status:"done"}
+    UI-->>U: Redirect to /books/{slug}
+```
+
+## TTS Audio Generation Pipeline
+
+Audio is generated lazily: on first request the job is enqueued; subsequent
+requests poll for completion and then stream from MinIO via presigned URL.
+
+```mermaid
+flowchart TD
+    A(["POST /api/audio/{slug}/{n}\nbody: voice=af_bella"]) --> B{Audio already\nin MinIO?}
+    B -- yes --> C[200 status: done]
+    B -- no --> D{Job already\nin queue?}
+    D -- "yes pending/generating" --> E[202 task_id + status]
+    D -- no --> F[INSERT audio_task\nstatus=pending\nin PocketBase]
+    F --> E
+
+    G([Runner polls task queue]) --> H[Claim audio_task\nstatus=generating]
+    H --> I["GET /api/chapter-text/{slug}/{n}\nfrom backend — plain text"]
+    I --> J[POST /v1/audio/speech\nto Kokoro-FastAPI\nbody: text + voice]
+    J --> K[Stream MP3 response]
+    K --> L[PUT object to MinIO\naudio/{slug}/{n}/{voice}.mp3]
+    L --> M[UPDATE audio_task\nstatus=done]
+
+    N(["Client polls\nGET /api/audio/status/{slug}/{n}"]) --> O{status?}
+    O -- "pending/generating" --> N
+    O -- done --> P["GET /api/presign/audio/{slug}/{n}"]
+    P --> Q{Valkey cache hit?}
+    Q -- yes --> R[302 → presigned URL]
+    Q -- no --> S[GeneratePresignedURL\nfrom MinIO — TTL 1h]
+    S --> T[Cache in Valkey\nTTL 3500s]
+    T --> R
+    R --> U([Client streams audio\ndirectly from MinIO])
+```
--- a/docs/mermaid/request-flow.mermaid.md
+++ b/docs/mermaid/request-flow.mermaid.md
@@ -0,0 +1,111 @@
+# Request Flow
+
+Two representative request paths through the stack: a **page load** (SSR) and a
+**media playback** (presigned URL → direct MinIO stream).
+
+## SSR Page Load — Catalogue / Book Detail
+
+```mermaid
+sequenceDiagram
+    actor C as Browser / iOS App
+    participant CD as Caddy :443
+    participant UI as SvelteKit UI :3000
+    participant BE as Backend API :8080
+    participant MS as Meilisearch :7700
+    participant PB as PocketBase :8090
+    participant VK as Valkey :6379
+
+    C->>CD: HTTPS GET /catalogue
+    CD->>UI: proxy /* (SvelteKit catch-all)
+    UI->>BE: GET /api/catalogue?page=1&sort=popular
+    BE->>MS: search(query, filters, sort)
+    MS-->>BE: [{slug, title, …}, …]
+    BE-->>UI: {books[], page, total, has_next}
+    UI-->>CD: SSR HTML
+    CD-->>C: 200 HTML
+
+    Note over C,UI: Infinite scroll — client fetches next page via SvelteKit API route
+    C->>CD: HTTPS GET /api/catalogue-page?page=2
+    CD->>UI: proxy /* (SvelteKit /api/catalogue-page server route)
+    UI->>BE: GET /api/catalogue?page=2
+    BE->>MS: search(…)
+    MS-->>BE: next page
+    BE-->>UI: {books[], …}
+    UI-->>C: JSON
+```
+
+## Audio Playback — Presigned URL Flow
+
+```mermaid
+sequenceDiagram
+    actor C as Browser / iOS App
+    participant CD as Caddy :443
+    participant UI as SvelteKit UI :3000
+    participant BE as Backend API :8080
+    participant VK as Valkey :6379
+    participant MN as MinIO :9000
+
+    C->>CD: GET /api/presign/audio/{slug}/{n}?voice=af_bella
+    CD->>UI: proxy /* (SvelteKit /api/presign/audio route)
+    UI->>BE: GET /api/presign/audio/{slug}/{n}?voice=af_bella
+    BE->>VK: GET presign:audio:{slug}:{n}:{voice}
+    alt cache hit
+        VK-->>BE: presigned URL (TTL remaining)
+        BE-->>UI: 302 redirect → presigned URL
+        UI-->>C: 302 redirect
+    else cache miss
+        BE->>MN: GeneratePresignedURL(audio-bucket, key, 1h)
+        MN-->>BE: presigned URL
+        BE->>VK: SET presign:audio:… EX 3500
+        BE-->>UI: 302 redirect → presigned URL
+        UI-->>C: 302 redirect
+    end
+    C->>MN: GET presigned URL (direct, no proxy)
+    MN-->>C: audio/mpeg stream
+```
+
+## Chapter Read — SSR + Content Fetch
+
+```mermaid
+sequenceDiagram
+    actor C as Browser / iOS App
+    participant CD as Caddy :443
+    participant UI as SvelteKit UI :3000
+    participant BE as Backend API :8080
+    participant PB as PocketBase :8090
+    participant MN as MinIO :9000
+
+    C->>CD: HTTPS GET /books/{slug}/chapters/{n}
+    CD->>UI: proxy /* (SvelteKit catch-all)
+    UI->>PB: getBook(slug) + listChapterIdx(slug)
+    PB-->>UI: book meta + chapter list
+    UI->>BE: GET /api/chapter-text/{slug}/{n}
+    BE->>MN: GetObject(chapters-bucket, {slug}/{n}.md)
+    MN-->>BE: markdown text
+    BE-->>UI: plain text (markdown stripped)
+    Note over UI: marked() → HTML
+    UI-->>CD: SSR HTML
+    CD-->>C: 200 HTML
+```
+
+## Caddy Request Lifecycle
+
+Shows how security hardening applies before a request reaches any upstream.
+
+```mermaid
+flowchart TD
+    A([Incoming HTTPS request]) --> B[TLS termination\nLet's Encrypt cert]
+    B --> C{Rate limit check\ncaddy-ratelimit}
+    C -- over limit --> D[429 Too Many Requests]
+    C -- ok --> E[Add security headers\nX-Frame-Options · X-Content-Type-Options\nReferrer-Policy · Permissions-Policy\nHSTS · X-XSS-Protection\nremove Server header]
+    E --> F{Route match}
+    F -- "/health /scrape*\n/api/browse /api/book-preview/*\n/api/chapter-text/*\n/api/reindex/* /api/cover/*\n/api/audio-proxy/*" --> G[reverse_proxy → backend:8080]
+    F -- "/avatars/*" --> H[reverse_proxy → minio:9000]
+    F -- "/* everything else\n(incl. /api/scrape/*\n/api/chapter-text-preview/*)" --> I[reverse_proxy → ui:3000\nSvelteKit auth middleware runs]
+    G --> J{Upstream healthy?}
+    H --> J
+    I --> J
+    J -- yes --> K([Response to client])
+    J -- "502/503/504" --> L[handle_errors\nstatic HTML from /srv/errors/]
+    L --> K
+```