architecture.d2:
- Split app into prod VPS (165.22.70.138) and homelab runner (192.168.0.109)
- Add CrowdSec, Dozzle agent, pocket-tts (voice samples)
- Valkey now shown as Asynq job queue in addition to presign cache
- Add caddy-l4 Redis TCP proxy (:6380) to Caddy label
- Add CI/CD node (Gitea Actions) with full job list incl. releases.json bake
- Remove runner from prod app group (it runs on homelab only)
- Watchtower: note runner is label-disabled on prod
api-routing.d2:
- Add /api/presign/* routes to backend (presign_be group)
- Add /api/audio POST + status GET to both sk and be
- Add /api/scrape/book and /api/scrape/book/range to scrape_sk
- Catalogue: annotate Meilisearch vs legacy browse
- Add Meilisearch filter/sort fields to storage node
- Add Asynq queue note to Valkey storage node
- Fix presign proxy: sk routes through be.presign_be, not directly to storage
curl -sf without -L silently wrote '301 Moved Permanently' to releases.json
instead of following the http→https redirect. Added -L to follow redirects,
set -euo pipefail, and jq type validation so the step fails hard on bad JSON.
Replace runtime Gitea API fetch with fs.readFileSync of releases.json,
which CI writes to ui/static/ before the Docker build context is sent.
Eliminates prod→homelab network dependency for the changelog page.
- Scrape page: replace three large cards + genre card with a compact
bordered table of rows (label + inline controls per action). Visually
much lighter, all controls visible without scrolling.
- Admin sidebar: add Changelog link after Audio.
- New /admin/changelog page: fetches releases from Gitea API and renders
them as a clean list (tag, title, date, body).
The bare { } block at the bottom was a second global options block which
Caddy's caddyfile adapter rejects on reload. Merged layer4 into the single
top-level global block. Changed listener from hostname (redis.libnovel.cc:6380)
to :6380 so Caddy binds to the local interface rather than the Cloudflare IP
that resolves for the hostname.
- Add caddy/errors/404.html (matches existing 502/503/504 style)
- Add handle_errors 404 block in Caddyfile
- Add active health checks (5s interval) and lb_try_duration 3s to the
ui reverse_proxy so Caddy detects Watchtower container replacements
quickly and serves the 502 maintenance page instead of a raw error
handlePresignVoiceSample generates voice samples on demand via pocket-tts,
which requires WAV→MP3 transcoding via ffmpeg. The backend was using
distroless/static (no ffmpeg) so all pocket-tts preview requests returned 500.
Switch backend stage to Alpine + ffmpeg, matching the runner image.
Move admin navigation from a two-row tab strip into a persistent left
sidebar with grouped sections (Pages / Tools). Consolidate Scrape and
Audio entries in the global top nav into a single Admin link.
Backend was missing POCKET_TTS_URL entirely — pocketTTSClient was nil
so voices() only returned 67 Kokoro voices. Runner already had the var
via Doppler but it was absent from the compose environment block.
Also fix stray leading space on backend environment: key (YAML parse error).
Verified: /api/voices now returns 87 voices (67 kokoro + 20 pocket-tts).
Removed onchange→navigateWithFilters from all three selects — the
immediate navigation on every change was the root cause of both bugs:
1. Filters applied before the user finished selecting all options.
2. Svelte 5 bind:value updates state after onchange fires, so the
navigateWithFilters call read stale values → wrong URL params → no results.
Renamed navigateWithFilters to applyFilters (no overrides arg needed).
Added an amber Apply button next to Reset; selects now only update local
state until the user presses Apply.
PUBLIC_BUILD_VERSION and PUBLIC_BUILD_COMMIT were set only in the builder
stage ENV — they were never re-declared in the runtime stage, so the Node
server started with them undefined and the badge always showed 'dev'.
Fix: re-declare all three ARGs after the second FROM and set runtime ENVs.
Add PUBLIC_BUILD_TIME (ISO timestamp from gitea.event.head_commit.timestamp)
injected via build-arg in release.yaml. Badge now shows e.g.:
v2.3.9+abc1234 · 28 Mar 2026 14:30 UTC
Add resolveAvatarUrl(userId, storedValue) helper that tries MinIO first,
then falls back to the stored HTTP URL for OAuth users (Google/GitHub)
who have never uploaded a custom avatar.
Add getUserById() to pocketbase helpers for batch avatar resolution in
comments. Update all 6 call sites to use the new helper.
Docker image builds belong to the release workflow (tag-triggered).
CI now runs: go vet, go build (backend/runner/healthcheck), go test,
svelte-check, and UI vite build — fast feedback without Docker overhead.
Also triggers on all branches, not just main/master.
- Catalogue filter selects: replace value= initializer with bind:value +
onchange goto() so filters navigate immediately on change (no Apply button)
- Add selected= on each <option> for correct DOM initialisation in Svelte 5
- Build badge: give distinct bg-zinc-800 pill, visible zinc-300/400/500 text
instead of zinc-700/zinc-800 which blended into the footer background
- scraper.go: ScrapeCatalogue now uses retryGet (9 attempts, 10s base) +
500–1500ms inter-page jitter instead of bare GetContent. ScrapeMetadata
also switched to retryGet so a single 429 on a book page is retried rather
than aborting the whole refresh.
- catalogue_refresh.go: per-book delay is now configurable
(RUNNER_CATALOGUE_REQUEST_DELAY, default 2s) + up to 50% random jitter
applied before every metadata fetch. Only metadata is scraped here —
chapters are fetched on-demand, not during catalogue refresh. Progress
logged every 50 books instead of 100.
- config.go / runner.go / main.go: add CatalogueRequestDelay field wired
from RUNNER_CATALOGUE_REQUEST_DELAY env var.
- release.yaml: comment out upload-sourcemaps job and remove it from the
release needs; GlitchTip auth token needs refreshing after DB wipe.
- docker-compose.yml: MEILI_URL in x-infra-env was hardcoded to the local
meilisearch container, ignoring the Doppler MEILI_URL secret entirely.
Changed to "${MEILI_URL:-http://meilisearch:7700}" so prod reads from
https://search.libnovel.cc while local dev keeps the container default.
- ui/+layout.svelte: rename nav + footer label from "Discover" to "Catalogue"
(desktop nav, mobile menu, footer — all three occurrences).
- Add POCKET_TTS_URL env to backend service in docker-compose.yml so
pocket-tts voices appear in the voice selector (Doppler secret existed
but the env var was never passed to the container)
- Fix GetAudioTask PocketBase filter using %q (double-quotes) instead of
single-quoted string, causing the duplicate-task guard to always miss
- Fix AudioPlayer double-POST: GET /api/presign/audio already enqueues
TTS internally on 404; AudioPlayer now skips the redundant POST and
polls directly, eliminating the 500 from the PB unique-key conflict
- Footer now displays build version tag + short commit SHA (from
PUBLIC_BUILD_VERSION / PUBLIC_BUILD_COMMIT env vars baked in at
image build time); falls back to 'dev' in local builds
- docker-compose.yml: add watchtower label to caddy service so it
auto-updates alongside backend/ui/runner on new image pushes
- homelab/docker-compose.yml: use locally-built kokoro-fastapi:latest
image (consistent with actual homelab setup)
Expose all available voices from both TTS engines via the /api/voices
endpoint. AudioPlayer and profile voice-selector now group voices by
engine and show a labelled optgroup. Voice type carries an engine field
so the chapter-reader can route synthesis to the correct backend.
WithEndpoint expects host[:port] with no scheme. When Doppler has
https://otel.libnovel.cc the backend was crashing with 'invalid port'.
Now strip the scheme and enable TLS when prefix is https://.
- pocket-tts has no prebuilt image on ghcr.io — must be built from source on homelab
- Updated image to pocket-tts:latest (local tag), add command with --host 0.0.0.0
- Add model cache volumes (pocket_tts_cache, hf_cache) so model weights survive restarts
- start_period increased to 120s (first startup downloads weights)
- otelsetup.Init now returns a *slog.Logger wired to the OTLP log exporter
so all slog output is shipped to Loki with embedded trace IDs
- backend and runner both adopt the new OTel-bridged logger
- runner.runScrapeTask and runAudioTask now emit structured OTel spans
- ui/hooks.server.ts adds BatchLogRecordProcessor alongside existing trace exporter
- homelab: add kokoro-fastapi GPU service (ghcr.io/remsky/kokoro-fastapi-gpu)
using deploy.resources.reservations for NVIDIA GPU, exposed internally on :8880
- homelab: add pocket-tts CPU service (ghcr.io/kyutai-labs/pocket-tts) on :8000
- runner KOKORO_URL hardcoded to http://kokoro-fastapi:8880 (fixes DNS failure
for the stale kokoro.kalekber.cc hostname)
Go backend:
- Add OTel SDK + otelhttp middleware deps (go.mod)
- New internal/otelsetup package: init OTLP/HTTP TracerProvider from env vars
- cmd/backend/main.go: call otelsetup.Init() after logger + ctx setup
- internal/backend/server.go: wrap mux with otelhttp.NewHandler() before
sentryhttp, so all HTTP spans are recorded
SvelteKit UI:
- Add @opentelemetry/sdk-node, exporter-trace-otlp-http, resources,
semantic-conventions
- hooks.server.ts: init NodeSDK when OTEL_EXPORTER_OTLP_ENDPOINT is set;
graceful shutdown on SIGTERM/SIGINT
Config:
- docker-compose.yml: pass OTEL_EXPORTER_OTLP_ENDPOINT + OTEL_SERVICE_NAME
to backend, runner, and ui services
- homelab/docker-compose.yml: fix runner OTel endpoint to HTTP port 4318
- Doppler prd: OTEL_EXPORTER_OTLP_ENDPOINT=https://otel.libnovel.cc
- Doppler prd_homelab: OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318
All services no-op gracefully when the env var is unset (local dev).
The home, library, and /books routes were fetching all 15k books from
PocketBase on every SSR request (31 sequential HTTP calls per request).
Changes:
- Add src/lib/server/cache.ts: generic Valkey JSON cache
- Add getBooksBySlugs(): single PB query fetching only requested slugs,
with fallback to the 5-min Valkey cache populated by listBooks()
- listBooks(): now caches results in Valkey for 5 min (safety net for
admin routes that still need the full list)
- Home + /api/home: replaced listBooks()+filter with getBooksBySlugs()
on progress slugs only — typically 1 PB request instead of 31
- /books + /api/library: same pattern using progress+saved slug union
@sentry/vite-plugin uses sentry-cli which creates release entries but
doesn't upload files to GlitchTip's API correctly. Switch to the native
glitchtip-cli which uses the debug ID inject+upload approach that
GlitchTip actually supports.
- Add search.libnovel.cc Caddy vhost proxying to meilisearch:7700
- Pass MEILI_URL + MEILI_API_KEY from Doppler into homelab runner
- Set GODEBUG=preferIPv4=1 to work around missing IPv6 route on homelab
- Update comments to reflect runner now indexes books into Meilisearch
- Enable sourcemap:true in vite.config.ts
- Add sentryVitePlugin: uploads maps to errors.libnovel.cc, deletes them post-upload so they never ship in the Docker image
- Wire release: PUBLIC_BUILD_VERSION in both hooks.client.ts and hooks.server.ts so events correlate to the correct artifact
- Add upload-sourcemaps CI job in release.yaml (parallel to docker-ui, uses GLITCHTIP_AUTH_TOKEN secret)
Two bugs caused audio tasks to loop endlessly:
1. claimRecord never set heartbeat_at — newly claimed tasks had
heartbeat_at=null, which matched the reaper's stale filter
(heartbeat_at=null || heartbeat_at<threshold). Tasks were reaped
and reset to pending within seconds of being claimed, before the
30s heartbeat goroutine had a chance to write a timestamp.
Fix: set heartbeat_at=now() in claimRecord alongside status=running.
2. Audio semaphore was checked AFTER claiming the task. When the
semaphore was full the select/break only broke the inner select,
not the for loop — the code fell through and launched an uncapped
goroutine that blocked forever on <-audioSem drain. The task also
stayed status=running with no heartbeat, feeding bug #1.
Fix: pre-acquire a semaphore slot BEFORE claiming the task; release
it immediately if the queue is empty or claim fails.
- Add email/email_verified/verification_token/verification_token_exp fields
to app_users PocketBase schema (pb-init-v3.sh)
- Add SMTP env vars to UI service in docker-compose.yml
- New email.ts: raw TLS SMTP mailer via Node tls module, sendVerificationEmail()
- createUser() now takes email param, stores verification token (24h TTL)
- loginUser() throws 'Email not verified' when email_verified is false
- New /verify-email route: validates token, verifies user, auto-logs in
- Login page: email field in register form, check-inbox state after register
- /api/auth/register (iOS): returns { pending_verification, email } instead of token
- Add pb.libnovel.cc and storage.libnovel.cc Caddy virtual hosts for homelab runner
- Add homelab runner docker-compose and libnovel.sh helper script
- Add @sentry/sveltekit: server-side init in hooks.server.ts, client-side in
hooks.client.ts; opt-in via PUBLIC_GLITCHTIP_DSN env var; handleError wired up
- Add Umami analytics: script tag in +layout.svelte (opt-in via PUBLIC_UMAMI_WEBSITE_ID);
track book_view, chapter_read, audio_played events via window.umami?.track
- Presign audio endpoint now auto-triggers Kokoro TTS generation when audio is
missing (404 → POST /api/audio → return 202); AudioPlayer handles 202 status
- Fix listAudioCache: was querying non-existent audio_cache collection; now projects
from audio_jobs where status='done'
- Add Feedback nav link (desktop + mobile + footer) pointing to feedback.libnovel.cc
- Admin scrape page: add full catalogue scrape button; fix retryTask for catalogue
kind; show retry button for all failed/cancelled tasks regardless of kind
