Expose all available voices from both TTS engines via the /api/voices
endpoint. AudioPlayer and profile voice-selector now group voices by
engine and show a labelled optgroup. Voice type carries an engine field
so the chapter-reader can route synthesis to the correct backend.
WithEndpoint expects host[:port] with no scheme. When Doppler has
https://otel.libnovel.cc the backend was crashing with 'invalid port'.
Now strip the scheme and enable TLS when prefix is https://.
- pocket-tts has no prebuilt image on ghcr.io — must be built from source on homelab
- Updated image to pocket-tts:latest (local tag), add command with --host 0.0.0.0
- Add model cache volumes (pocket_tts_cache, hf_cache) so model weights survive restarts
- start_period increased to 120s (first startup downloads weights)
- otelsetup.Init now returns a *slog.Logger wired to the OTLP log exporter
so all slog output is shipped to Loki with embedded trace IDs
- backend and runner both adopt the new OTel-bridged logger
- runner.runScrapeTask and runAudioTask now emit structured OTel spans
- ui/hooks.server.ts adds BatchLogRecordProcessor alongside existing trace exporter
- homelab: add kokoro-fastapi GPU service (ghcr.io/remsky/kokoro-fastapi-gpu)
using deploy.resources.reservations for NVIDIA GPU, exposed internally on :8880
- homelab: add pocket-tts CPU service (ghcr.io/kyutai-labs/pocket-tts) on :8000
- runner KOKORO_URL hardcoded to http://kokoro-fastapi:8880 (fixes DNS failure
for the stale kokoro.kalekber.cc hostname)
Go backend:
- Add OTel SDK + otelhttp middleware deps (go.mod)
- New internal/otelsetup package: init OTLP/HTTP TracerProvider from env vars
- cmd/backend/main.go: call otelsetup.Init() after logger + ctx setup
- internal/backend/server.go: wrap mux with otelhttp.NewHandler() before
sentryhttp, so all HTTP spans are recorded
SvelteKit UI:
- Add @opentelemetry/sdk-node, exporter-trace-otlp-http, resources,
semantic-conventions
- hooks.server.ts: init NodeSDK when OTEL_EXPORTER_OTLP_ENDPOINT is set;
graceful shutdown on SIGTERM/SIGINT
Config:
- docker-compose.yml: pass OTEL_EXPORTER_OTLP_ENDPOINT + OTEL_SERVICE_NAME
to backend, runner, and ui services
- homelab/docker-compose.yml: fix runner OTel endpoint to HTTP port 4318
- Doppler prd: OTEL_EXPORTER_OTLP_ENDPOINT=https://otel.libnovel.cc
- Doppler prd_homelab: OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318
All services no-op gracefully when the env var is unset (local dev).
The home, library, and /books routes were fetching all 15k books from
PocketBase on every SSR request (31 sequential HTTP calls per request).
Changes:
- Add src/lib/server/cache.ts: generic Valkey JSON cache
- Add getBooksBySlugs(): single PB query fetching only requested slugs,
with fallback to the 5-min Valkey cache populated by listBooks()
- listBooks(): now caches results in Valkey for 5 min (safety net for
admin routes that still need the full list)
- Home + /api/home: replaced listBooks()+filter with getBooksBySlugs()
on progress slugs only — typically 1 PB request instead of 31
- /books + /api/library: same pattern using progress+saved slug union
@sentry/vite-plugin uses sentry-cli which creates release entries but
doesn't upload files to GlitchTip's API correctly. Switch to the native
glitchtip-cli which uses the debug ID inject+upload approach that
GlitchTip actually supports.
- Add search.libnovel.cc Caddy vhost proxying to meilisearch:7700
- Pass MEILI_URL + MEILI_API_KEY from Doppler into homelab runner
- Set GODEBUG=preferIPv4=1 to work around missing IPv6 route on homelab
- Update comments to reflect runner now indexes books into Meilisearch
- Enable sourcemap:true in vite.config.ts
- Add sentryVitePlugin: uploads maps to errors.libnovel.cc, deletes them post-upload so they never ship in the Docker image
- Wire release: PUBLIC_BUILD_VERSION in both hooks.client.ts and hooks.server.ts so events correlate to the correct artifact
- Add upload-sourcemaps CI job in release.yaml (parallel to docker-ui, uses GLITCHTIP_AUTH_TOKEN secret)
Two bugs caused audio tasks to loop endlessly:
1. claimRecord never set heartbeat_at — newly claimed tasks had
heartbeat_at=null, which matched the reaper's stale filter
(heartbeat_at=null || heartbeat_at<threshold). Tasks were reaped
and reset to pending within seconds of being claimed, before the
30s heartbeat goroutine had a chance to write a timestamp.
Fix: set heartbeat_at=now() in claimRecord alongside status=running.
2. Audio semaphore was checked AFTER claiming the task. When the
semaphore was full the select/break only broke the inner select,
not the for loop — the code fell through and launched an uncapped
goroutine that blocked forever on <-audioSem drain. The task also
stayed status=running with no heartbeat, feeding bug #1.
Fix: pre-acquire a semaphore slot BEFORE claiming the task; release
it immediately if the queue is empty or claim fails.
- Add email/email_verified/verification_token/verification_token_exp fields
to app_users PocketBase schema (pb-init-v3.sh)
- Add SMTP env vars to UI service in docker-compose.yml
- New email.ts: raw TLS SMTP mailer via Node tls module, sendVerificationEmail()
- createUser() now takes email param, stores verification token (24h TTL)
- loginUser() throws 'Email not verified' when email_verified is false
- New /verify-email route: validates token, verifies user, auto-logs in
- Login page: email field in register form, check-inbox state after register
- /api/auth/register (iOS): returns { pending_verification, email } instead of token
- Add pb.libnovel.cc and storage.libnovel.cc Caddy virtual hosts for homelab runner
- Add homelab runner docker-compose and libnovel.sh helper script
- Add @sentry/sveltekit: server-side init in hooks.server.ts, client-side in
hooks.client.ts; opt-in via PUBLIC_GLITCHTIP_DSN env var; handleError wired up
- Add Umami analytics: script tag in +layout.svelte (opt-in via PUBLIC_UMAMI_WEBSITE_ID);
track book_view, chapter_read, audio_played events via window.umami?.track
- Presign audio endpoint now auto-triggers Kokoro TTS generation when audio is
missing (404 → POST /api/audio → return 202); AudioPlayer handles 202 status
- Fix listAudioCache: was querying non-existent audio_cache collection; now projects
from audio_jobs where status='done'
- Add Feedback nav link (desktop + mobile + footer) pointing to feedback.libnovel.cc
- Admin scrape page: add full catalogue scrape button; fix retryTask for catalogue
kind; show retry button for all failed/cancelled tasks regardless of kind
- Remove BrowseStore interface + all browse MinIO ops (PutBrowsePage, GetBrowsePage,
BrowseObjectKey, putBrowse, getBrowse); the /api/browse endpoint was replaced by
the Meilisearch-backed /api/catalogue route
- Remove browse_refresh.go and the 6h browse refresh ticker in runner
- Add Sentry/GlitchTip error tracking to both backend and runner binaries via
github.com/getsentry/sentry-go; opt-in via GLITCHTIP_DSN env var
- Wrap http.ServeMux with sentryhttp middleware for automatic panic recovery
- Rename MinIO bucket defaults: libnovel-chapters→chapters, libnovel-audio→audio,
libnovel-browse→catalogue (env vars still override)
- Remove staged compiled binaries (backend/bin/runner, backend/healthcheck)
- docker-compose: add dozzle, uptime-kuma, gotify services with Caddy subdomains
(logs/uptime/push.libnovel.cc); bind-mount dozzle/users.yml for simple auth
- Caddyfile: add site blocks for logs, uptime, push subdomains
- admin layout: add Logs, Uptime, Push tabs to external tools pill
- CatalogueEntry: add Slug field; populated by novelfire scraper
- meili.Client: add BookExists() to check if a book is already indexed
- catalogue_refresh: skip books already present in Meilisearch to prevent
redundant re-scraping and re-indexing on every runner restart
- docker-compose runner: set RUNNER_SKIP_INITIAL_CATALOGUE_REFRESH=true
- New Go backend binary (backend + runner) replacing old scraper/
- Rename SCRAPER_API_URL → BACKEND_API_URL in UI env and docker-compose
- Rename scraperFetch → backendFetch across all 19 UI server files
- Remove SCRAPER_PROXY env var and proxy transport from browser.Config
- Add Meilisearch, Valkey, Caddy to docker-compose
- Add docs/: api-endpoints.md, request-flow.mermaid.md, data-flow.mermaid.md
Adds docs/architecture.d2 and docs/architecture.mermaid.md showing the
docker-compose-new.yml service topology — storage, application, init
containers, and external dependencies with annotated connections.
Also includes the rendered docs/architecture.svg (D2 output).
View live: d2 --watch docs/architecture.d2
View in Gitea: navigate to docs/architecture.mermaid.md in the web UI.
novelfire.net responds with Content-Encoding: br when the scraper
advertises 'gzip, deflate, br'. The client only handled gzip, so
Brotli-compressed bytes were fed raw into the HTML parser producing
garbage — empty titles, zero chapters, and selector failures.
Added github.com/andybalholm/brotli and wired it into GetContent
alongside the existing gzip path.
Minimal PocketBase bootstrap for the v2 stack (backend + runner + ui-v2).
Creates only the 6 collections actually used by v2:
books, chapters_idx, ranking, progress, scraping_tasks, audio_jobs
Drops v1-only collections (app_users, user_settings, audio_cache,
book_comments, comment_votes, user_library, user_sessions,
user_subscriptions) and unused fields (date_label, user_id/audio_time
on progress). heartbeat_at is included in create_collection from the
start and also covered by ensure_field for existing instances.
docker-compose-new.yml pb-init service now mounts pb-init-v2.sh.
The sed-based collection id and fields extraction was greedy and broke on
collections with multiple fields (grabbed the last field id instead of the
top-level collection id → PATCH to wrong URL → 404).
Rewrite ensure_field to use python3 for reliable JSON parsing. Also adds the
missing heartbeat_at (date) field to scraping_tasks and audio_jobs which was
never applied on the initial deploy because the bug prevented the PATCH.
- Runner fetches 9 browse combos (genre×sort×status) every 6h and stores
JSON snapshots in MinIO libnovel-browse bucket (browse_refresh.go)
- Backend handleBrowse reads page-1 results from MinIO first; falls back
to live novelfire.net fetch; returns empty+cached:false on total failure
instead of 502
- Add BrowseStore interface (bookstore.go), MinIO put/get helpers (minio.go),
Store methods + compile-time assertion (store.go), BucketBrowse config,
wiring in cmd/backend and cmd/runner, docker-compose-new bucket init
- Fix ReapStaleTasks: PocketBase datetime fields require heartbeat_at=null
(not heartbeat_at="") in filter expressions, and nil (not "") in patch
payload — was causing 400 errors on every reap cycle
- backend/: Go API server and runner binaries with PocketBase + MinIO storage
- ui-v2/: SvelteKit frontend rewrite
- docker-compose-new.yml: compose file for the v2 stack
- .gitea/workflows/release-v2.yaml: CI/CD for backend, runner, and ui-v2 Docker Hub images
- scripts/pb-init.sh: migrate from wget to curl, add superuser bootstrap for fresh installs
- .env.example: document DOCKER_BUILDKIT=1 for Colima users
Upgrade DirectHTTPClient to send a full Chrome 124 header set
(Sec-Fetch-*, Accept-Encoding with gzip decompression, Referer,
Cache-Control) to reduce bot-detection false positives on WAFs.
Add SCRAPER_PROXY env var to route all outbound scrape requests
through a configurable proxy (residential or otherwise); falls back
to the standard HTTP_PROXY / HTTPS_PROXY env vars.
- Add UserProfileView and UserProfileViewModel for iOS
- Implement user library API endpoint (/api/users/[username]/library)
- Add DELETE /api/progress/[slug] endpoint for removing books from library
- Integrate subscription feed in home API
- Update Xcode project with new profile components
- PocketBase: new user_subscriptions collection (follower_id, followee_id)
- pocketbase.ts: subscribe/unsubscribe/getFollowingIds/getPublicProfile/
getUserPublicLibrary/getUserCurrentlyReading/getSubscriptionFeed helpers
- GET /api/users/[username] — public profile with subscription state
- POST/DELETE /api/users/[username]/subscribe — follow/unfollow
- /users/[username] — public profile page: avatar, stats, follow button,
currently reading grid, full library grid
- CommentsSection: usernames are now links to /users/[username]
- Home page: 'From People You Follow' section powered by subscription feed
iOS BookDetailView: replace paginated inline chapter list with a single
tappable 'Chapters' row (showing reading progress) that opens
BookChaptersSheet — a searchable full-screen sheet with jump-to-current.
ChapterReaderView: hide tab bar in reader, swap back/Aa/ToC button order
(Aa left, ToC right, X rightmost), remove mini-player spacer (tab bar
and player are hidden).
HomeView: remove large HeroContinueCard, promote all continue-reading
items into a single horizontal shelf (Apple Books style) with progress
bar below each cover. NavigationLink now goes directly to the chapter.
Web +page.svelte: replace inline paginated chapter list with a compact
'Chapters' row linking to /books/[slug]/chapters. Admin scrape controls
are now a collapsible row inside the same card.
