The home, library, and /books routes were fetching all 15k books from
PocketBase on every SSR request (31 sequential HTTP calls per request).
Changes:
- Add src/lib/server/cache.ts: generic Valkey JSON cache
- Add getBooksBySlugs(): single PB query fetching only requested slugs,
with fallback to the 5-min Valkey cache populated by listBooks()
- listBooks(): now caches results in Valkey for 5 min (safety net for
admin routes that still need the full list)
- Home + /api/home: replaced listBooks()+filter with getBooksBySlugs()
on progress slugs only — typically 1 PB request instead of 31
- /books + /api/library: same pattern using progress+saved slug union
@sentry/vite-plugin uses sentry-cli which creates release entries but
doesn't upload files to GlitchTip's API correctly. Switch to the native
glitchtip-cli which uses the debug ID inject+upload approach that
GlitchTip actually supports.
- Add search.libnovel.cc Caddy vhost proxying to meilisearch:7700
- Pass MEILI_URL + MEILI_API_KEY from Doppler into homelab runner
- Set GODEBUG=preferIPv4=1 to work around missing IPv6 route on homelab
- Update comments to reflect runner now indexes books into Meilisearch
- Enable sourcemap:true in vite.config.ts
- Add sentryVitePlugin: uploads maps to errors.libnovel.cc, deletes them post-upload so they never ship in the Docker image
- Wire release: PUBLIC_BUILD_VERSION in both hooks.client.ts and hooks.server.ts so events correlate to the correct artifact
- Add upload-sourcemaps CI job in release.yaml (parallel to docker-ui, uses GLITCHTIP_AUTH_TOKEN secret)
Two bugs caused audio tasks to loop endlessly:
1. claimRecord never set heartbeat_at — newly claimed tasks had
heartbeat_at=null, which matched the reaper's stale filter
(heartbeat_at=null || heartbeat_at<threshold). Tasks were reaped
and reset to pending within seconds of being claimed, before the
30s heartbeat goroutine had a chance to write a timestamp.
Fix: set heartbeat_at=now() in claimRecord alongside status=running.
2. Audio semaphore was checked AFTER claiming the task. When the
semaphore was full the select/break only broke the inner select,
not the for loop — the code fell through and launched an uncapped
goroutine that blocked forever on <-audioSem drain. The task also
stayed status=running with no heartbeat, feeding bug #1.
Fix: pre-acquire a semaphore slot BEFORE claiming the task; release
it immediately if the queue is empty or claim fails.
- Add email/email_verified/verification_token/verification_token_exp fields
to app_users PocketBase schema (pb-init-v3.sh)
- Add SMTP env vars to UI service in docker-compose.yml
- New email.ts: raw TLS SMTP mailer via Node tls module, sendVerificationEmail()
- createUser() now takes email param, stores verification token (24h TTL)
- loginUser() throws 'Email not verified' when email_verified is false
- New /verify-email route: validates token, verifies user, auto-logs in
- Login page: email field in register form, check-inbox state after register
- /api/auth/register (iOS): returns { pending_verification, email } instead of token
- Add pb.libnovel.cc and storage.libnovel.cc Caddy virtual hosts for homelab runner
- Add homelab runner docker-compose and libnovel.sh helper script
- Add @sentry/sveltekit: server-side init in hooks.server.ts, client-side in
hooks.client.ts; opt-in via PUBLIC_GLITCHTIP_DSN env var; handleError wired up
- Add Umami analytics: script tag in +layout.svelte (opt-in via PUBLIC_UMAMI_WEBSITE_ID);
track book_view, chapter_read, audio_played events via window.umami?.track
- Presign audio endpoint now auto-triggers Kokoro TTS generation when audio is
missing (404 → POST /api/audio → return 202); AudioPlayer handles 202 status
- Fix listAudioCache: was querying non-existent audio_cache collection; now projects
from audio_jobs where status='done'
- Add Feedback nav link (desktop + mobile + footer) pointing to feedback.libnovel.cc
- Admin scrape page: add full catalogue scrape button; fix retryTask for catalogue
kind; show retry button for all failed/cancelled tasks regardless of kind
- Remove BrowseStore interface + all browse MinIO ops (PutBrowsePage, GetBrowsePage,
BrowseObjectKey, putBrowse, getBrowse); the /api/browse endpoint was replaced by
the Meilisearch-backed /api/catalogue route
- Remove browse_refresh.go and the 6h browse refresh ticker in runner
- Add Sentry/GlitchTip error tracking to both backend and runner binaries via
github.com/getsentry/sentry-go; opt-in via GLITCHTIP_DSN env var
- Wrap http.ServeMux with sentryhttp middleware for automatic panic recovery
- Rename MinIO bucket defaults: libnovel-chapters→chapters, libnovel-audio→audio,
libnovel-browse→catalogue (env vars still override)
- Remove staged compiled binaries (backend/bin/runner, backend/healthcheck)
- docker-compose: add dozzle, uptime-kuma, gotify services with Caddy subdomains
(logs/uptime/push.libnovel.cc); bind-mount dozzle/users.yml for simple auth
- Caddyfile: add site blocks for logs, uptime, push subdomains
- admin layout: add Logs, Uptime, Push tabs to external tools pill
- CatalogueEntry: add Slug field; populated by novelfire scraper
- meili.Client: add BookExists() to check if a book is already indexed
- catalogue_refresh: skip books already present in Meilisearch to prevent
redundant re-scraping and re-indexing on every runner restart
- docker-compose runner: set RUNNER_SKIP_INITIAL_CATALOGUE_REFRESH=true
- New Go backend binary (backend + runner) replacing old scraper/
- Rename SCRAPER_API_URL → BACKEND_API_URL in UI env and docker-compose
- Rename scraperFetch → backendFetch across all 19 UI server files
- Remove SCRAPER_PROXY env var and proxy transport from browser.Config
- Add Meilisearch, Valkey, Caddy to docker-compose
- Add docs/: api-endpoints.md, request-flow.mermaid.md, data-flow.mermaid.md
Adds docs/architecture.d2 and docs/architecture.mermaid.md showing the
docker-compose-new.yml service topology — storage, application, init
containers, and external dependencies with annotated connections.
Also includes the rendered docs/architecture.svg (D2 output).
View live: d2 --watch docs/architecture.d2
View in Gitea: navigate to docs/architecture.mermaid.md in the web UI.
novelfire.net responds with Content-Encoding: br when the scraper
advertises 'gzip, deflate, br'. The client only handled gzip, so
Brotli-compressed bytes were fed raw into the HTML parser producing
garbage — empty titles, zero chapters, and selector failures.
Added github.com/andybalholm/brotli and wired it into GetContent
alongside the existing gzip path.
Minimal PocketBase bootstrap for the v2 stack (backend + runner + ui-v2).
Creates only the 6 collections actually used by v2:
books, chapters_idx, ranking, progress, scraping_tasks, audio_jobs
Drops v1-only collections (app_users, user_settings, audio_cache,
book_comments, comment_votes, user_library, user_sessions,
user_subscriptions) and unused fields (date_label, user_id/audio_time
on progress). heartbeat_at is included in create_collection from the
start and also covered by ensure_field for existing instances.
docker-compose-new.yml pb-init service now mounts pb-init-v2.sh.
The sed-based collection id and fields extraction was greedy and broke on
collections with multiple fields (grabbed the last field id instead of the
top-level collection id → PATCH to wrong URL → 404).
Rewrite ensure_field to use python3 for reliable JSON parsing. Also adds the
missing heartbeat_at (date) field to scraping_tasks and audio_jobs which was
never applied on the initial deploy because the bug prevented the PATCH.
- Runner fetches 9 browse combos (genre×sort×status) every 6h and stores
JSON snapshots in MinIO libnovel-browse bucket (browse_refresh.go)
- Backend handleBrowse reads page-1 results from MinIO first; falls back
to live novelfire.net fetch; returns empty+cached:false on total failure
instead of 502
- Add BrowseStore interface (bookstore.go), MinIO put/get helpers (minio.go),
Store methods + compile-time assertion (store.go), BucketBrowse config,
wiring in cmd/backend and cmd/runner, docker-compose-new bucket init
- Fix ReapStaleTasks: PocketBase datetime fields require heartbeat_at=null
(not heartbeat_at="") in filter expressions, and nil (not "") in patch
payload — was causing 400 errors on every reap cycle
- backend/: Go API server and runner binaries with PocketBase + MinIO storage
- ui-v2/: SvelteKit frontend rewrite
- docker-compose-new.yml: compose file for the v2 stack
- .gitea/workflows/release-v2.yaml: CI/CD for backend, runner, and ui-v2 Docker Hub images
- scripts/pb-init.sh: migrate from wget to curl, add superuser bootstrap for fresh installs
- .env.example: document DOCKER_BUILDKIT=1 for Colima users
Upgrade DirectHTTPClient to send a full Chrome 124 header set
(Sec-Fetch-*, Accept-Encoding with gzip decompression, Referer,
Cache-Control) to reduce bot-detection false positives on WAFs.
Add SCRAPER_PROXY env var to route all outbound scrape requests
through a configurable proxy (residential or otherwise); falls back
to the standard HTTP_PROXY / HTTPS_PROXY env vars.
- Add UserProfileView and UserProfileViewModel for iOS
- Implement user library API endpoint (/api/users/[username]/library)
- Add DELETE /api/progress/[slug] endpoint for removing books from library
- Integrate subscription feed in home API
- Update Xcode project with new profile components
- PocketBase: new user_subscriptions collection (follower_id, followee_id)
- pocketbase.ts: subscribe/unsubscribe/getFollowingIds/getPublicProfile/
getUserPublicLibrary/getUserCurrentlyReading/getSubscriptionFeed helpers
- GET /api/users/[username] — public profile with subscription state
- POST/DELETE /api/users/[username]/subscribe — follow/unfollow
- /users/[username] — public profile page: avatar, stats, follow button,
currently reading grid, full library grid
- CommentsSection: usernames are now links to /users/[username]
- Home page: 'From People You Follow' section powered by subscription feed
iOS BookDetailView: replace paginated inline chapter list with a single
tappable 'Chapters' row (showing reading progress) that opens
BookChaptersSheet — a searchable full-screen sheet with jump-to-current.
ChapterReaderView: hide tab bar in reader, swap back/Aa/ToC button order
(Aa left, ToC right, X rightmost), remove mini-player spacer (tab bar
and player are hidden).
HomeView: remove large HeroContinueCard, promote all continue-reading
items into a single horizontal shelf (Apple Books style) with progress
bar below each cover. NavigationLink now goes directly to the chapter.
Web +page.svelte: replace inline paginated chapter list with a compact
'Chapters' row linking to /books/[slug]/chapters. Admin scrape controls
are now a collapsible row inside the same card.
All four workflows (ci-scraper, ci-ui, release-scraper, release-ui) now pass
build-args to docker/build-push-action. Release workflows use the semver tag
from docker/metadata-action outputs.version; CI workflows use the git SHA.
The /api/comments/[id] delete route was never created; the deleteComment helper
in pocketbase.ts existed but was unreachable. Added DELETE /api/comment/[id]
route handler alongside the existing vote route. Updated CommentsSection.svelte
and iOS APIClient to use /api/comment/{id} for both delete and (already fixed)
vote, keeping all comment-mutation endpoints under the singular /api/comment/
prefix to avoid SvelteKit route conflicts with /api/comments/[slug].
- Go scraper: Version/Commit vars in main.go, injected via -ldflags; Server struct + New() updated; GET /health and new GET /api/version expose them
- UI Dockerfile: ARG BUILD_VERSION/BUILD_COMMIT → ENV PUBLIC_BUILD_VERSION/PUBLIC_BUILD_COMMIT for SvelteKit
- Footer: shows version+short commit when not 'dev' (text-zinc-800, subtle)
- docker-compose: args blocks for scraper and ui build sections pass $GIT_TAG/$GIT_COMMIT
/api/comments/[id] and /api/comments/[slug] were ambiguous dynamic segments at
the same path level, causing a build error. Moved the vote handler to the
singular /api/comment/ prefix and updated all callers (web + iOS).
- Batch-resolve avatar presign URLs server-side in GET /api/comments/[slug];
returns avatarUrls map alongside comments and myVotes
- CommentsSection.svelte: show avatar image or initials fallback (24px top-level,
20px replies) next to each comment/reply username
- iOS CommentsResponse gains avatarUrls field; CommentsViewModel stores and
populates it on load; CommentRow renders AsyncImage with initials fallback
- Also includes: comment replies (1-level nesting), delete, sort (Top/New),
parent_id schema migration, and AvatarCropModal cropperjs fix
Production Docker image has no node_modules at runtime; marked was being
externalized by adapter-node (it is in dependencies), causing ERR_MODULE_NOT_FOUND
when +page.server.ts and +server.ts imported it. Adding it to ssr.noExternal
forces Vite to inline it into the server bundle.
marked({ async: true }) triggers a dynamic internal require inside marked
that vite/rollup treats as external, causing ERR_MODULE_NOT_FOUND at runtime
in the adapter-node Docker image which ships no node_modules.
Switching to the synchronous marked() call makes rollup inline the full
library into the server chunk.
