Files

Admin 5b27d501af feat(v3): Caddy hardening, Watchtower, scrape fixes, docs diagrams

- Caddy: custom image with caddy-ratelimit plugin, security headers
  (X-Frame-Options, HSTS, CSP-adjacent, etc.), per-IP rate limiting on
  auth/scrape/global zones, static error pages (502/503/504), fix routing
  to remove /api/scrape/* and /api/chapter-text-preview/* direct-to-backend
  (were bypassing SvelteKit auth middleware)
- docker-compose: Caddy build context + error volume, Watchtower service
  (label-enable mode, 5 min poll), watchtower labels on backend/runner/ui
- Scraper: ScrapeChapterList uses retryGet (9 attempts, Retry-After backoff)
  to fix 429-induced chapter list failures; upTo param stops pagination early
  for range scrapes
- UI: Browse→Catalogue rename (routes, API, links), admin scrape page
  Continue/Retry buttons, +error.svelte branded error page, type cleanup
  (removed dead exports, added BookPreviewMeta/BookPreviewResponse to scraper.ts)
- Meilisearch: meta_updated field, sort=update fix, facet distribution
- Docs: reorganise into docs/d2/ and docs/mermaid/ subdirectories, update
  all diagrams to reflect Caddy/Watchtower/routing changes, add
  api-routing.d2 ownership map with auth-level colour coding, regenerate SVGs

2026-03-22 21:10:38 +05:00

3.8 KiB

Raw Blame History

Request Flow

Two representative request paths through the stack: a page load (SSR) and a media playback (presigned URL → direct MinIO stream).

SSR Page Load — Catalogue / Book Detail

sequenceDiagram
    actor C as Browser / iOS App
    participant CD as Caddy :443
    participant UI as SvelteKit UI :3000
    participant BE as Backend API :8080
    participant MS as Meilisearch :7700
    participant PB as PocketBase :8090
    participant VK as Valkey :6379

    C->>CD: HTTPS GET /catalogue
    CD->>UI: proxy /* (SvelteKit catch-all)
    UI->>BE: GET /api/catalogue?page=1&sort=popular
    BE->>MS: search(query, filters, sort)
    MS-->>BE: [{slug, title, …}, …]
    BE-->>UI: {books[], page, total, has_next}
    UI-->>CD: SSR HTML
    CD-->>C: 200 HTML

    Note over C,UI: Infinite scroll — client fetches next page via SvelteKit API route
    C->>CD: HTTPS GET /api/catalogue-page?page=2
    CD->>UI: proxy /* (SvelteKit /api/catalogue-page server route)
    UI->>BE: GET /api/catalogue?page=2
    BE->>MS: search(…)
    MS-->>BE: next page
    BE-->>UI: {books[], …}
    UI-->>C: JSON

Audio Playback — Presigned URL Flow

sequenceDiagram
    actor C as Browser / iOS App
    participant CD as Caddy :443
    participant UI as SvelteKit UI :3000
    participant BE as Backend API :8080
    participant VK as Valkey :6379
    participant MN as MinIO :9000

    C->>CD: GET /api/presign/audio/{slug}/{n}?voice=af_bella
    CD->>UI: proxy /* (SvelteKit /api/presign/audio route)
    UI->>BE: GET /api/presign/audio/{slug}/{n}?voice=af_bella
    BE->>VK: GET presign:audio:{slug}:{n}:{voice}
    alt cache hit
        VK-->>BE: presigned URL (TTL remaining)
        BE-->>UI: 302 redirect → presigned URL
        UI-->>C: 302 redirect
    else cache miss
        BE->>MN: GeneratePresignedURL(audio-bucket, key, 1h)
        MN-->>BE: presigned URL
        BE->>VK: SET presign:audio:… EX 3500
        BE-->>UI: 302 redirect → presigned URL
        UI-->>C: 302 redirect
    end
    C->>MN: GET presigned URL (direct, no proxy)
    MN-->>C: audio/mpeg stream

Chapter Read — SSR + Content Fetch

sequenceDiagram
    actor C as Browser / iOS App
    participant CD as Caddy :443
    participant UI as SvelteKit UI :3000
    participant BE as Backend API :8080
    participant PB as PocketBase :8090
    participant MN as MinIO :9000

    C->>CD: HTTPS GET /books/{slug}/chapters/{n}
    CD->>UI: proxy /* (SvelteKit catch-all)
    UI->>PB: getBook(slug) + listChapterIdx(slug)
    PB-->>UI: book meta + chapter list
    UI->>BE: GET /api/chapter-text/{slug}/{n}
    BE->>MN: GetObject(chapters-bucket, {slug}/{n}.md)
    MN-->>BE: markdown text
    BE-->>UI: plain text (markdown stripped)
    Note over UI: marked() → HTML
    UI-->>CD: SSR HTML
    CD-->>C: 200 HTML

Caddy Request Lifecycle

Shows how security hardening applies before a request reaches any upstream.

flowchart TD
    A([Incoming HTTPS request]) --> B[TLS termination\nLet's Encrypt cert]
    B --> C{Rate limit check\ncaddy-ratelimit}
    C -- over limit --> D[429 Too Many Requests]
    C -- ok --> E[Add security headers\nX-Frame-Options · X-Content-Type-Options\nReferrer-Policy · Permissions-Policy\nHSTS · X-XSS-Protection\nremove Server header]
    E --> F{Route match}
    F -- "/health /scrape*\n/api/browse /api/book-preview/*\n/api/chapter-text/*\n/api/reindex/* /api/cover/*\n/api/audio-proxy/*" --> G[reverse_proxy → backend:8080]
    F -- "/avatars/*" --> H[reverse_proxy → minio:9000]
    F -- "/* everything else\n(incl. /api/scrape/*\n/api/chapter-text-preview/*)" --> I[reverse_proxy → ui:3000\nSvelteKit auth middleware runs]
    G --> J{Upstream healthy?}
    H --> J
    I --> J
    J -- yes --> K([Response to client])
    J -- "502/503/504" --> L[handle_errors\nstatic HTML from /srv/errors/]
    L --> K

3.8 KiB Raw Blame History