Translation content is immutable once generated — add Cache-Control: public,
max-age=3600, stale-while-revalidate=86400 to handleTranslationRead so the
browser and any CDN reuse the response without hitting MinIO on repeat views.
Forward the header through the SvelteKit /api/translation/[slug]/[n] proxy
which previously stripped it by constructing a bare Content-Type-only Response.
Three bugs:
1. +page.server.ts returned an unawaited Promise — SvelteKit awaits it on
the server anyway so data.jobs arrived as a plain AIJob[] on the client,
not a Promise. The $effect calling .then() on an array silently failed,
leaving jobs=[] and the table empty. Fixed by awaiting in the load fn.
2. +page.svelte $effect updated to assign data.jobs directly (plain array)
instead of calling .then() on it.
3. handlers_textgen.go: final UpdateAIJob (payload+status write) used
r.Context() which is cancelled when the SSE client disconnects. If the
browser navigated away mid-job, results were silently dropped and the
payload stayed as the initial {pattern} stub with no results array.
Fixed by using context.Background() for the final write, matching the
pattern already used in handlers_image.go.
- Add bookstore.NotificationStore interface and wire it directly to *storage.Store
in Dependencies, bypassing the Asynq wrapper that caused Producer.(*storage.Store)
to fail with a 500 on every notification endpoint when Redis is configured
- Replace s.deps.Producer.(*storage.Store) type assertion in all 5 notification
handlers with s.deps.NotificationStore (nil-safe, always works)
- Fix PocketBase filter single-quote bug in ListNotifications, ClearAllNotifications,
and MarkAllNotificationsRead (PocketBase requires double quotes for string values)
The SSE (non-async) chapter-names handler streamed results to the client
but never wrote them into the PocketBase job payload — only the initial
{pattern} stub was stored. The Review button then fetched the job and
found no results, showing 'No results found in this job's payload.'
Fix: accumulate allResults across batches (same as the async handler) and
write the full {pattern, slug, results:[...]} payload when marking done.
- Replace bell dropdown with full-screen NotificationsModal (mirrors SearchModal pattern)
- Notifications visible to all logged-in users (not just admin)
- Admin users excluded from new-chapter fan-out (dedup vs Scrape Complete notification)
- Users with notify_new_chapters=false opted out of new-chapter in-app notifications
- Toggle in profile page to enable/disable in-app new-chapter notifications
- PATCH /api/profile endpoint to save notification preferences
- User-facing /notifications page (admin redirects to /admin/notifications)
- Service worker (src/service-worker.ts) handles push events and
notification clicks, navigating to the book page on tap
- Web app manifest (manifest.webmanifest) linked in app.html
- Profile page: push notification toggle (subscribe/unsubscribe)
using the browser Notification + PushManager API with VAPID
- API route POST/DELETE /api/push-subscription proxies to backend
- Go backend: push_subscriptions PocketBase collection storage
methods (SavePushSubscription, DeletePushSubscription,
ListPushSubscriptionsByBook) in storage/store.go
- handlers_push.go: GET vapid-public-key, POST/DELETE subscription
- webpush package: VAPID-signed sends via webpush-go, SendToBook
fans out to all users who have the book in their library
- Runner fires push to subscribers whenever ChaptersScraped > 0
after a successful book scrape
- Config: VAPID_PUBLIC_KEY, VAPID_PRIVATE_KEY, VAPID_SUBJECT env vars
- domain.ScrapeResult gets a Slug field; orchestrator populates it
Demote per-book and per-chapter-list-page Info logs to Debug — these
fire hundreds of times per catalogue run and drown out meaningful signals:
- orchestrator: RunBook starting (per book)
- metadata saved (per book)
- chapter list fetched (per book)
- scraping chapter list page N (per pagination page per book)
The 'book scrape finished' summary log (with scraped/skipped/errors
counters) remains at Info — it is the useful signal per book.
Two bugs fixed in runScrapeTask / runCatalogueTask:
1. FinishScrapeTask was called with the task's own context, which is
already cancelled when the task is stopped. The PATCH to PocketBase
failed silently, leaving all counters at their initial zero values.
Fix: use a fresh context.WithTimeout(Background, 15s) for the write.
2. BooksFound was double-counted: RunBook already sets BooksFound=1 on
success, but the accumulation loop added an extra +1 unconditionally,
reporting 2 books per successful scrape.
Fix: result.BooksFound += bookResult.BooksFound (drop the + 1).
- Add `archived` bool to domain.BookMeta, pbBook, and Meilisearch bookDoc
- ArchiveBook / UnarchiveBook patch the PocketBase record; ListBooks filters
archived=false so hidden books disappear from all public responses
- Meilisearch: add `archived` as a filterable attribute; Search and Catalogue
always prepend `archived = false` to exclude archived books from results
- DeleteBook permanently removes the PocketBase record, all chapters_idx rows,
MinIO chapter objects, cover image, and the Meilisearch document
- New BookAdminStore interface with ArchiveBook, UnarchiveBook, DeleteBook
- Admin HTTP endpoints: PATCH /api/admin/books/{slug}/archive|unarchive,
DELETE /api/admin/books/{slug}
- PocketBase schema: archived field added to live pb.libnovel.cc and to
pb-init-v3.sh (both create block and add_field migration)
Split chapter text into ~1000-char sentence-boundary chunks before sending
to kokoro-fastapi. Each chunk is generated individually and the raw MP3 bytes
are concatenated. This prevents the EOF / timeout failures that occur when
the server receives a very large single request (e.g. a full PDF 'Full Text'
chapter). chunkText() breaks at sentence endings (. ! ? newlines) to preserve
natural speech flow.
- parsePDF: return all text as single 'Full Text' chapter (admin splits manually)
- parseEPUB: fix chapter numbering to use sequential counter not spine index
- Remove dead code: chaptersFromBookmarks, cleanChapterText, extractChaptersFromText, chapterHeadingRE; drop pdfcpu alias and regexp imports
- Backend: POST /api/admin/books/:slug/split-chapters endpoint — splits text on '---' dividers, optional '## Title' headers, writes chapters via WriteChapter
- UI: admin panel now shows for all admin users regardless of source_url; chapter split tool shown when book has single 'Full Text' chapter, pre-fills from MinIO content
Three fixes to PDF chapter extraction quality:
1. Page ordering: parse page number from pdfcpu filename (out_Content_page_N.txt)
instead of using lexicographic sort index — fixes chapters bleeding into each
other (e.g. Prologue text appearing inside Chapter 1).
2. Windows-1252 chars: map bytes 0x91-0x9F to proper Unicode (curly quotes U+2018/
U+2019/U+201C/U+201D, em-dash U+2014, etc.) instead of raw Latin-1 control
bytes that rendered as ◆ in the browser.
3. Chapter header cleanup: skip the first page of each bookmark range (decorative
title art page) and strip any run-on title fragment at the start of the first
body page (e.g. 'for New Journeys!I stood atop...' → 'I stood atop...'). The
remaining sentence truncation is a fundamental limitation of this PDF's
PUA-encoded body font (C2_1/Literata) — those glyphs cannot be decoded without
the publisher's private ToUnicode mapping.
Use PDF outline (bookmarks) for chapter titles and page ranges, then
extract text from per-page content streams via pdfcpu ExtractContent.
This avoids the indefinite hang caused by dslipak/pdf trying to resolve
custom font ToUnicode CMaps on publisher PDFs.
- parsePDF: decrypt → ExtractContent → Bookmarks → chaptersFromBookmarks
- chaptersFromBookmarks: flatten bookmark tree, skip front/back matter
(Cover, Insert, Title Page, Copyright, Appendix), assign page ranges
- extractTextFromContentStream: handle TJ arrays (concat literal strings,
skip hex glyph arrays and kerning numbers) + single Tj strings
- Falls back to paragraph-splitting when no bookmarks present
- Build verified; test PDF produces 9 chapters with proper titles
- parsePDF function restored in import.go (body was orphaned outside function)
- ParseImportFile() called at upload time with 3-min timeout; chapters stored as JSON in MinIO
- runner.go: prefer ChaptersKey path (read pre-parsed JSON) over BookImport.Import()
- ImportChapterStore interface added; store wired in runner/main.go
- HeartbeatTask and ReapStaleTasks now include import_tasks collection
- parseImportTask now returns ChaptersKey in domain.ImportTask
- asynq_runner.go handleImportTask passes ChaptersKey
- pb-init-v3.sh: chapters_key field added to import_tasks schema
- parsePDF now attempts to strip encryption via pdfcpu (empty user password)
before handing bytes to dslipak/pdf — fixes '256-bit encryption key' error
on publisher PDFs that use owner-only encryption (copy/print restrictions)
- Add pdfcpu v0.11.1 as direct dependency (was already indirect)
- docker-compose.yml minio-init: add 'imports' and 'translations' buckets
so a fresh deploy creates all required buckets
- Add ImportFileStore interface to bookstore package
- Add ImportFileStore field to backend.Dependencies
- Wire ImportFileStore: store in cmd/backend/main.go
- handlers_import.go: use s.deps.ImportFileStore.PutImportFile instead of
broken s.deps.Producer.(*storage.Store) type assertion (fails when Asynq active)
- pb-init-v3.sh: add import_tasks and notifications collection definitions
- Import task: persist object_key, author, cover_url, genres, summary,
book_status in PocketBase so the runner can fetch the file and write
book metadata on completion
- Runner poll mode: pass task.ObjectKey instead of empty string
- Runner: write BookMeta + UpsertBook in Meilisearch after chapter ingest
so imported books appear in catalogue and search
- Import UI: add author, cover URL, genres, summary, status fields; add
AI tasks panel (chapter names, description, image gen, tagline) after
import completes; add AI tasks button on each done task in the list
- Admin nav: add Notifications entry to sidebar (all 5 locales)
- Logout: delete user_sessions row on sign-out so sessions don't
accumulate as phantoms after each login/logout cycle
- docker-compose.yml: BODY_SIZE_LIMIT=52428800 (50MB) on UI service
— adapter-node was rejecting PDFs >512KB with 'Content-length exceeds limit'
- storage/import.go: add AnalyzeFile() public function using real PDF/EPUB parsers
- handlers_import.go: analyzeImportFile() now calls storage.AnalyzeFile() instead
of the file-size stub, so chapter count is accurate in the preview
- NewBookImporter now takes *Store instead of raw *minio.Client (same package access)
- Runner Dependencies gains ChapterIngester interface; storeImportedChapters no-op removed
- store.IngestChapters is now actually called after PDF/EPUB extraction
- BookImport and ChapterIngester wired in runner main.go
- CreateImportTask interface gains initiatorUserID param; threaded through store/asynq/handler
- domain.ImportTask gains InitiatorUserID field; parseImportTask populates it
- Runner notifies initiator (falls back to 'admin' when empty)
- UI: import task list polls every 3s while any task is pending/running
- UI: label[for] + input[id] fix for a11y warnings in admin import page
- Import page: add file upload with review step before committing
- Backend: add analyze endpoint to preview chapters before import
- Add notifications collection + API for admin alerts
- Add bell icon in header with notification dropdown (admin only)
- Runner creates notification on import completion
- Notifications link to /admin/import for easy review
- Add ImportTask/ImportResult types to domain.go
- Add TypeImportBook to asynqqueue for task routing
- Add CreateImportTask to producer and storage layers
- Add ClaimNextImportTask/FinishImportTask to Consumer interfaces
- Add import task handling to runner (polling + Asynq handler)
- Add BookImporter interface to bookstore for PDF/EPUB parsing
- Add backend API endpoints: POST/GET /api/admin/import
- Add SvelteKit UI at /admin/import with task list
- Add nav link in admin layout
Note: PDF/EPUB parsing is a placeholder - needs external library integration.
Cloudflare Workers AI changed the API for flux-2-dev, flux-2-klein-4b,
and flux-2-klein-9b to require multipart/form-data (instead of JSON) and
now returns {"image":"<base64>"} instead of raw PNG bytes.
- Add requiresMultipart() helper for the three FLUX.2 models
- callImageAPI builds multipart body for those models, JSON for others
- Parse {"image":"<base64>"} JSON response; fall back to raw bytes for legacy models
- Use "steps" field name (not "num_steps") in multipart forms per CF docs
- Book page: capture and display actual backend error message instead of blank 'Error'
- Admin layout: SVG icons, active highlight, divider between nav sections
- Scrape page: status filter pills with counts, text + status combined search
- Audio page: status filter pills, cancel jobs, retry failed jobs, mobile cards for cache tab
- Translation page: status filter pills (incl. cancelled), cancel + retry jobs, mobile cancel/retry cards, i18n for all labels
- AI Jobs page: fix concurrent cancel (Set instead of single slot), per-job cancel errors inline, full mobile card layout, i18n title/heading
- Text-gen page: tagline editable input + copy, warnings copy, i18n title/heading
- Book page: chapter cover Save button, audio monitor link, currentShelf pre-populated from server
- pocketbase.ts: add getBookShelf(), shelf field on UserLibraryEntry
- New API route: POST /api/admin/translation/bulk (proxy for translation retry)
- i18n: 15 new admin_translation_*, admin_ai_jobs_*, admin_text_gen_* keys across all 5 locales
speechSynthesis is silently muted on iOS Safari and Chrome Android after
the audio session ends (onended), so chapter announcements never played.
Fix:
- Add GET /api/tts-announce backend endpoint: streams a short TTS clip
for arbitrary text without MinIO caching (backend/internal/backend/)
- Add GET /api/announce SvelteKit proxy route (no paywall)
- Add announceNavigatePending/announcePendingSlug/announcePendingChapter
to AudioStore
- Rewrite onended announce branch: sets audioStore.audioUrl to the
announcement clip URL so the persistent <audio> element plays it;
the next onended detects announceNavigatePending and navigates
- 10s safety timeout in case the clip fails to load/end
- Add POST /api/admin/image-gen/async: fire-and-forget image generation
that stores the result (base64) in an ai_job payload and returns 202
immediately — no more 60-120s blocking on FLUX models
- Add POST /api/admin/text-gen/description/async: same pattern for book
description generation
- Register both new routes in server.go
- Rewrite image-gen admin page to use the async path (submit → redirect
to AI Jobs for monitoring)
- Extend ai-jobs page with Review panels for image-gen jobs (show image,
Save as cover / Download / Discard) and description jobs (diff old vs
new, editable textarea, Apply / Discard)
- Add POST /api/admin/text-gen/chapter-names/async backend endpoint: fire-and-forget,
returns job_id immediately (HTTP 202), runs batch generation in background goroutine,
persists proposed titles in ai_job payload when done
- Register new route in server.go alongside existing SSE endpoint (backward compat)
- Add SvelteKit proxy at /api/admin/text-gen/chapter-names/async
- Add SvelteKit proxy for GET /api/admin/ai-jobs/[id] (job detail with payload)
- Add Review button on ai-jobs page for done chapter-names jobs; inline panel shows
editable title table (old to new) with Apply All button that POSTs to chapter-names/apply
- Backend: add GET /api/audio-preview/{slug}/{n} — generates first ~1800-char
chunk via CF AI so playback starts immediately; full chapter cached in MinIO
- Frontend: replace CF AI spinner with preview blob URL + background swap to
full presigned URL when runner finishes, preserving currentTime
- AudioPlayer: isPreview state + 'preview' badge in mini-bar during swap
- pocketbase.ts: fix 403 on stale token — reduce TTL to 50 min + retry once
on 401/403 with forced re-auth (was cached 12 h, PB tokens expire in 1 h)
- Footer build time now rendered in user's local timezone via toLocaleString()
- New `ai_jobs` PocketBase collection tracks all long-running AI tasks
(batch-covers, chapter-names) with status, progress, and cancellation
- `handlers_aijobs.go`: GET/cancel endpoints for ai_jobs; centralised
cancel registry (moved from handlers_catalogue)
- Batch-covers and chapter-names SSE handlers now create/resume ai_job
records, support from_item/to_item ranges, and resume from items_done
on restart via job_id
- New `POST /api/admin/image-gen/auto-prompt`: generates an image prompt
from book description (cover) or chapter title (chapter) via LLM
- image-gen page: "Auto-prompt" button calls auto-prompt API when a slug
is selected; falls back gracefully if TextGen not configured
- text-gen chapter-names: from/to chapter range inputs + job ID display
- catalogue-tools batch-covers: from/to item range + resume job ID input
- pb-init-v3.sh: adds ai_jobs collection (idempotent)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CF Workers AI has a ~4MB JSON body limit. Large cover images base64-encoded
can easily exceed this, causing Cloudflare to return a 502 Bad Gateway.
Added resizeRefImage() which scales the reference down so its longest side
≤ 768px (nearest-neighbour, re-encoded as JPEG) before passing it to the
GenerateImageFromReference call. Images already within the limit pass through
unchanged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Llama 4 Scout returns `result.response` as an array of objects
[{"generated_text":"..."}] instead of a plain string. Decode into
json.RawMessage and try both shapes; fall back to generated_text[0].
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Split chapter-name LLM requests into 100-chapter batches and stream
results back as SSE so large books (e.g. Shadow Slave: 2916 chapters)
never time out or truncate. Frontend shows live batch progress inline
and accumulates proposals as they arrive.
chapters_idx was missing created/updated columns (never defined in the
PocketBase schema), causing PocketBase to return 400 for any query
sorted by -created. recentlyUpdatedBooks() uses this sort.
- Add created date field to chapters_idx schema in pb-init-v3.sh
(also added via add_field for existing installations)
- Add idx_chapters_idx_created index for sort performance
- Set created timestamp on first insert in upsertChapterIdx so new
chapters are immediately sortable; existing records retain empty created
and will sort to the back (acceptable — only affects home page recency)
- Fix upsertChapterIdx race: use conflict-retry pattern (mirrors WriteMetadata)
so concurrent goroutines don't double-POST the same chapter number
- Add DeduplicateChapters to BookWriter interface and Store implementation;
keeps the latest record per (slug, number) and deletes extras
- Wire POST /api/admin/dedup-chapters/{slug} handler in server.go
The aura-2-en model enforces a hard 2 000-character limit per request.
Chapters routinely exceed this, producing 413 errors.
GenerateAudio now splits the stripped text into ≤1 800-char chunks at
paragraph → sentence → space → hard-cut boundaries, calls the API once
per chunk, and concatenates the MP3 frames. Callers (runner, streaming
handler) are unchanged. StreamAudioMP3/WAV inherit the fix automatically
since they delegate to GenerateAudio.
- Default max_tokens to 4096 for chapter-names so large chapter lists
are not cut off mid-JSON by the model's token limit
- Rewrite system prompt to clarify placeholder semantics ({n} = number,
{scene} = scene hint) and explicitly forbid echoing the number inside
the title field — prevents "Chapter 1 - 1: ..." style duplications
- UI: surface raw_response in the error area when chapters:[] is returned
so the admin can see what the model actually produced
Adds backend handlers and SvelteKit UI for an admin text generation tool.
The tool lets admins propose and apply AI-generated chapter titles and book
descriptions using Cloudflare Workers AI (12 LLM models, model selector shared
across both tabs).
**Ratings (1–5 stars)**
- New `book_ratings` PB collection (session_id, user_id, slug, rating)
- `getBookRating`, `getBookAvgRating`, `setBookRating` in pocketbase.ts
- GET/POST /api/ratings/[slug] API route
- StarRating.svelte component with hover, animated stars, avg display
- Star rating shown on book detail page (desktop + mobile)
**Plan-to-Read shelf**
- `shelf` field added to `user_library` (reading/plan_to_read/completed/dropped)
- `updateBookShelf`, `getShelfMap` in pocketbase.ts
- PATCH /api/library/[slug] for shelf updates
- Shelf selector dropdown on book detail page (only when saved)
- Shelf tabs on library page to filter by category
**Sleep timer**
- `sleepUntil` state added to AudioStore
- Layout handles timer lifecycle (survives chapter navigation)
- Cycles Off → 15m → 30m → 45m → 60m → Off
- Shows live countdown in AudioPlayer when active
**EPUB export**
- Go backend: GET /api/export/{slug}?from=N&to=N
- Generates valid EPUB2 zip (mimetype uncompressed, OPF, NCX, XHTML chapters)
- Markdown → HTML via goldmark
- SvelteKit proxy at /api/export/[slug]
- Download button on book detail page (only when in library)
**Fix TS errors**
- discover/+page.svelte: currentBook possibly undefined (use {@const book})
- cardEl now $state for reactive binding
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ClaimNextTranslationTask and HeartbeatTask were no-ops in the asynq
Consumer, so translation tasks created in PocketBase were never picked
up by the runner. Translation tasks live in PocketBase (not Redis),
so they must be claimed/heartbeated via the underlying pb consumer.
ReapStaleTasks is also delegated so stale translation tasks get reset.
Also removes the LibreTranslate healthcheck from homelab/runner
docker-compose.yml and relaxes depends_on to service_started — the
healthcheck was blocking runner startup until models loaded (~2 min)
and the models are already pre-downloaded in the volume.
- storage/pocketbase.go: replace http.DefaultClient (no timeout) with a
dedicated pbHTTPClient{Timeout: 30s} so a slow/hung PocketBase cannot
stall the backend or runner indefinitely
- runner/asynq_runner.go: heartbeat ticker was firing at StaleTaskThreshold
(2 min) == the Docker healthcheck deadline, so a single missed tick would
mark the container unhealthy; halved to StaleTaskThreshold/2 (1 min)
- New /discover page with swipe UI: left=skip, right=like, up=read now, down=nope
- Onboarding modal to collect genre/status preferences (persisted in localStorage)
- 3-card stack with pointer-event drag, CSS fly-out animation, 5 action buttons
- Tap card for preview modal; empty state with deck reset
- Like/read-now auto-saves book to user library
- POST /api/discover/vote + DELETE for deck reset
- Discovery vote persistence via PocketBase discovery_votes collection
- Fix duplicate books: dedup by slug in getBooksBySlugs
- Fix WriteMetadata TOCTOU race: conflict-retry on concurrent insert
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add StreamAudioWAV() to pocket-tts and Kokoro clients; pocket-tts streams
raw WAV directly (no ffmpeg), Kokoro requests response_format:wav with stream:true
- GET /api/audio-stream supports ?format=wav for lower-latency first-byte delivery;
WAV cached separately in MinIO as {slug}/{n}/{voice}.wav
- Add GET /api/admin/audio/jobs with optional ?slug filter
- Add POST /api/admin/audio/bulk {slug, voice, from, to, skip_existing, force}
where skip_existing=true (default) resumes interrupted bulk jobs
- Add POST /api/admin/audio/cancel-bulk {slug} to cancel all pending/running tasks
- Add CancelAudioTasksBySlug to taskqueue.Producer + asynqqueue implementation
- Add AudioObjectKeyExt to bookstore.AudioStore for format-aware MinIO keys
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- homelab/docker-compose.yml: add redis:7-alpine service (port 6379 bound to host
so Caddy TLS proxy on prod can reach it), add libretranslate service, add
redis_data and libretranslate_data volumes
- asynqqueue/producer.go: Asynq enqueue failures are now logged as warnings instead
of returned as errors — PB record already exists so runner picks it up via poll
- backend/main.go: pass logger to NewProducer
Root cause: Redis was not reachable at 192.168.0.109:6379 because the redis
container had no host port binding. Caddy TLS proxy terminates TLS but could
not TCP-connect to the backend Redis.
Add GET /api/audio-stream/{slug}/{n}?voice= that streams MP3 audio to the
client as TTS generates it, while simultaneously uploading to MinIO. On
subsequent requests the endpoint redirects to the presigned MinIO URL,
skipping generation entirely.
- PocketTTS: StreamAudioMP3 pipes live WAV response body through ffmpeg
(streaming transcode — no full-buffer wait)
- Kokoro: StreamAudioMP3 uses stream:true mode, returning MP3 frames
directly without the two-step download-link flow
- AudioStore: PutAudioStream added for multipart MinIO upload from reader
- WriteTimeout bumped 60s → 15min to accommodate full-chapter streams
- X-Accel-Buffering: no header disables Caddy/nginx response buffering
