Llama 4 Scout returns `result.response` as an array of objects
[{"generated_text":"..."}] instead of a plain string. Decode into
json.RawMessage and try both shapes; fall back to generated_text[0].
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Split chapter-name LLM requests into 100-chapter batches and stream
results back as SSE so large books (e.g. Shadow Slave: 2916 chapters)
never time out or truncate. Frontend shows live batch progress inline
and accumulates proposals as they arrive.
chapters_idx was missing created/updated columns (never defined in the
PocketBase schema), causing PocketBase to return 400 for any query
sorted by -created. recentlyUpdatedBooks() uses this sort.
- Add created date field to chapters_idx schema in pb-init-v3.sh
(also added via add_field for existing installations)
- Add idx_chapters_idx_created index for sort performance
- Set created timestamp on first insert in upsertChapterIdx so new
chapters are immediately sortable; existing records retain empty created
and will sort to the back (acceptable — only affects home page recency)
- Fix upsertChapterIdx race: use conflict-retry pattern (mirrors WriteMetadata)
so concurrent goroutines don't double-POST the same chapter number
- Add DeduplicateChapters to BookWriter interface and Store implementation;
keeps the latest record per (slug, number) and deletes extras
- Wire POST /api/admin/dedup-chapters/{slug} handler in server.go
The aura-2-en model enforces a hard 2 000-character limit per request.
Chapters routinely exceed this, producing 413 errors.
GenerateAudio now splits the stripped text into ≤1 800-char chunks at
paragraph → sentence → space → hard-cut boundaries, calls the API once
per chunk, and concatenates the MP3 frames. Callers (runner, streaming
handler) are unchanged. StreamAudioMP3/WAV inherit the fix automatically
since they delegate to GenerateAudio.
- Default max_tokens to 4096 for chapter-names so large chapter lists
are not cut off mid-JSON by the model's token limit
- Rewrite system prompt to clarify placeholder semantics ({n} = number,
{scene} = scene hint) and explicitly forbid echoing the number inside
the title field — prevents "Chapter 1 - 1: ..." style duplications
- UI: surface raw_response in the error area when chapters:[] is returned
so the admin can see what the model actually produced
Adds backend handlers and SvelteKit UI for an admin text generation tool.
The tool lets admins propose and apply AI-generated chapter titles and book
descriptions using Cloudflare Workers AI (12 LLM models, model selector shared
across both tabs).
**Ratings (1–5 stars)**
- New `book_ratings` PB collection (session_id, user_id, slug, rating)
- `getBookRating`, `getBookAvgRating`, `setBookRating` in pocketbase.ts
- GET/POST /api/ratings/[slug] API route
- StarRating.svelte component with hover, animated stars, avg display
- Star rating shown on book detail page (desktop + mobile)
**Plan-to-Read shelf**
- `shelf` field added to `user_library` (reading/plan_to_read/completed/dropped)
- `updateBookShelf`, `getShelfMap` in pocketbase.ts
- PATCH /api/library/[slug] for shelf updates
- Shelf selector dropdown on book detail page (only when saved)
- Shelf tabs on library page to filter by category
**Sleep timer**
- `sleepUntil` state added to AudioStore
- Layout handles timer lifecycle (survives chapter navigation)
- Cycles Off → 15m → 30m → 45m → 60m → Off
- Shows live countdown in AudioPlayer when active
**EPUB export**
- Go backend: GET /api/export/{slug}?from=N&to=N
- Generates valid EPUB2 zip (mimetype uncompressed, OPF, NCX, XHTML chapters)
- Markdown → HTML via goldmark
- SvelteKit proxy at /api/export/[slug]
- Download button on book detail page (only when in library)
**Fix TS errors**
- discover/+page.svelte: currentBook possibly undefined (use {@const book})
- cardEl now $state for reactive binding
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ClaimNextTranslationTask and HeartbeatTask were no-ops in the asynq
Consumer, so translation tasks created in PocketBase were never picked
up by the runner. Translation tasks live in PocketBase (not Redis),
so they must be claimed/heartbeated via the underlying pb consumer.
ReapStaleTasks is also delegated so stale translation tasks get reset.
Also removes the LibreTranslate healthcheck from homelab/runner
docker-compose.yml and relaxes depends_on to service_started — the
healthcheck was blocking runner startup until models loaded (~2 min)
and the models are already pre-downloaded in the volume.
- storage/pocketbase.go: replace http.DefaultClient (no timeout) with a
dedicated pbHTTPClient{Timeout: 30s} so a slow/hung PocketBase cannot
stall the backend or runner indefinitely
- runner/asynq_runner.go: heartbeat ticker was firing at StaleTaskThreshold
(2 min) == the Docker healthcheck deadline, so a single missed tick would
mark the container unhealthy; halved to StaleTaskThreshold/2 (1 min)
- New /discover page with swipe UI: left=skip, right=like, up=read now, down=nope
- Onboarding modal to collect genre/status preferences (persisted in localStorage)
- 3-card stack with pointer-event drag, CSS fly-out animation, 5 action buttons
- Tap card for preview modal; empty state with deck reset
- Like/read-now auto-saves book to user library
- POST /api/discover/vote + DELETE for deck reset
- Discovery vote persistence via PocketBase discovery_votes collection
- Fix duplicate books: dedup by slug in getBooksBySlugs
- Fix WriteMetadata TOCTOU race: conflict-retry on concurrent insert
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add StreamAudioWAV() to pocket-tts and Kokoro clients; pocket-tts streams
raw WAV directly (no ffmpeg), Kokoro requests response_format:wav with stream:true
- GET /api/audio-stream supports ?format=wav for lower-latency first-byte delivery;
WAV cached separately in MinIO as {slug}/{n}/{voice}.wav
- Add GET /api/admin/audio/jobs with optional ?slug filter
- Add POST /api/admin/audio/bulk {slug, voice, from, to, skip_existing, force}
where skip_existing=true (default) resumes interrupted bulk jobs
- Add POST /api/admin/audio/cancel-bulk {slug} to cancel all pending/running tasks
- Add CancelAudioTasksBySlug to taskqueue.Producer + asynqqueue implementation
- Add AudioObjectKeyExt to bookstore.AudioStore for format-aware MinIO keys
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- homelab/docker-compose.yml: add redis:7-alpine service (port 6379 bound to host
so Caddy TLS proxy on prod can reach it), add libretranslate service, add
redis_data and libretranslate_data volumes
- asynqqueue/producer.go: Asynq enqueue failures are now logged as warnings instead
of returned as errors — PB record already exists so runner picks it up via poll
- backend/main.go: pass logger to NewProducer
Root cause: Redis was not reachable at 192.168.0.109:6379 because the redis
container had no host port binding. Caddy TLS proxy terminates TLS but could
not TCP-connect to the backend Redis.
Add GET /api/audio-stream/{slug}/{n}?voice= that streams MP3 audio to the
client as TTS generates it, while simultaneously uploading to MinIO. On
subsequent requests the endpoint redirects to the presigned MinIO URL,
skipping generation entirely.
- PocketTTS: StreamAudioMP3 pipes live WAV response body through ffmpeg
(streaming transcode — no full-buffer wait)
- Kokoro: StreamAudioMP3 uses stream:true mode, returning MP3 frames
directly without the two-step download-link flow
- AudioStore: PutAudioStream added for multipart MinIO upload from reader
- WriteTimeout bumped 60s → 15min to accommodate full-chapter streams
- X-Accel-Buffering: no header disables Caddy/nginx response buffering
Two bugs prevented asynq mode from working correctly on the homelab runner:
1. No healthcheck file: asynq mode never writes /tmp/runner.alive, so
Docker healthcheck always fails. Added heartbeat goroutine that
writes the file every StaleTaskThreshold (30s).
2. Translation tasks not dispatched: translation uses ClaimNextTranslationTask
(PocketBase poll queue), not Redis/asynq. Audio + scrape use asynq mux,
but translation sits in PocketBase forever. Added pollTranslationTasks()
goroutine that polls PocketBase on the same PollInterval as the old
poll() loop.
All Go tests pass (go test ./... in backend/).
- LibreTranslate client (chunks on blank lines, ≤4500 chars, 3-goroutine semaphore)
- Runner translation task loop (OTel, heartbeat, MinIO storage)
- PocketBase translation_jobs collection support (create/claim/finish/list)
- Per-chapter language switcher on chapter reader (EN/RU/ID/PT/FR, polls until done)
- Admin /admin/translation page: bulk enqueue form + live-polling jobs table
- New backend routes: POST /api/translation/{slug}/{n}, GET /api/translation/status,
GET /api/translation/{slug}/{n}, GET /api/admin/translation/jobs,
POST /api/admin/translation/bulk
- ListTranslationTasks added to taskqueue.Reader interface + store impl
- All builds and tests pass; svelte-check: 0 errors
novelfire.net changed its book page structure. Old selectors produced empty
status and null genres for every book, causing all Meilisearch filters to
return zero results.
Old → new:
- status: <span class="status"> → <strong class="ongoing|completed|hiatus">
(text lowercased for consistent index values)
- genres: <div class="genres"> <a> → <div class="categories"> <a class="property-item">
(text lowercased for consistent index values)
Adds TestParseMetadataSelectors to guard against future regressions.
- scraper.go: ScrapeCatalogue now uses retryGet (9 attempts, 10s base) +
500–1500ms inter-page jitter instead of bare GetContent. ScrapeMetadata
also switched to retryGet so a single 429 on a book page is retried rather
than aborting the whole refresh.
- catalogue_refresh.go: per-book delay is now configurable
(RUNNER_CATALOGUE_REQUEST_DELAY, default 2s) + up to 50% random jitter
applied before every metadata fetch. Only metadata is scraped here —
chapters are fetched on-demand, not during catalogue refresh. Progress
logged every 50 books instead of 100.
- config.go / runner.go / main.go: add CatalogueRequestDelay field wired
from RUNNER_CATALOGUE_REQUEST_DELAY env var.
- release.yaml: comment out upload-sourcemaps job and remove it from the
release needs; GlitchTip auth token needs refreshing after DB wipe.
- Add POCKET_TTS_URL env to backend service in docker-compose.yml so
pocket-tts voices appear in the voice selector (Doppler secret existed
but the env var was never passed to the container)
- Fix GetAudioTask PocketBase filter using %q (double-quotes) instead of
single-quoted string, causing the duplicate-task guard to always miss
- Fix AudioPlayer double-POST: GET /api/presign/audio already enqueues
TTS internally on 404; AudioPlayer now skips the redundant POST and
polls directly, eliminating the 500 from the PB unique-key conflict
Expose all available voices from both TTS engines via the /api/voices
endpoint. AudioPlayer and profile voice-selector now group voices by
engine and show a labelled optgroup. Voice type carries an engine field
so the chapter-reader can route synthesis to the correct backend.
WithEndpoint expects host[:port] with no scheme. When Doppler has
https://otel.libnovel.cc the backend was crashing with 'invalid port'.
Now strip the scheme and enable TLS when prefix is https://.
- otelsetup.Init now returns a *slog.Logger wired to the OTLP log exporter
so all slog output is shipped to Loki with embedded trace IDs
- backend and runner both adopt the new OTel-bridged logger
- runner.runScrapeTask and runAudioTask now emit structured OTel spans
- ui/hooks.server.ts adds BatchLogRecordProcessor alongside existing trace exporter
- homelab: add kokoro-fastapi GPU service (ghcr.io/remsky/kokoro-fastapi-gpu)
using deploy.resources.reservations for NVIDIA GPU, exposed internally on :8880
- homelab: add pocket-tts CPU service (ghcr.io/kyutai-labs/pocket-tts) on :8000
- runner KOKORO_URL hardcoded to http://kokoro-fastapi:8880 (fixes DNS failure
for the stale kokoro.kalekber.cc hostname)
Go backend:
- Add OTel SDK + otelhttp middleware deps (go.mod)
- New internal/otelsetup package: init OTLP/HTTP TracerProvider from env vars
- cmd/backend/main.go: call otelsetup.Init() after logger + ctx setup
- internal/backend/server.go: wrap mux with otelhttp.NewHandler() before
sentryhttp, so all HTTP spans are recorded
SvelteKit UI:
- Add @opentelemetry/sdk-node, exporter-trace-otlp-http, resources,
semantic-conventions
- hooks.server.ts: init NodeSDK when OTEL_EXPORTER_OTLP_ENDPOINT is set;
graceful shutdown on SIGTERM/SIGINT
Config:
- docker-compose.yml: pass OTEL_EXPORTER_OTLP_ENDPOINT + OTEL_SERVICE_NAME
to backend, runner, and ui services
- homelab/docker-compose.yml: fix runner OTel endpoint to HTTP port 4318
- Doppler prd: OTEL_EXPORTER_OTLP_ENDPOINT=https://otel.libnovel.cc
- Doppler prd_homelab: OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318
All services no-op gracefully when the env var is unset (local dev).
Two bugs caused audio tasks to loop endlessly:
1. claimRecord never set heartbeat_at — newly claimed tasks had
heartbeat_at=null, which matched the reaper's stale filter
(heartbeat_at=null || heartbeat_at<threshold). Tasks were reaped
and reset to pending within seconds of being claimed, before the
30s heartbeat goroutine had a chance to write a timestamp.
Fix: set heartbeat_at=now() in claimRecord alongside status=running.
2. Audio semaphore was checked AFTER claiming the task. When the
semaphore was full the select/break only broke the inner select,
not the for loop — the code fell through and launched an uncapped
goroutine that blocked forever on <-audioSem drain. The task also
stayed status=running with no heartbeat, feeding bug #1.
Fix: pre-acquire a semaphore slot BEFORE claiming the task; release
it immediately if the queue is empty or claim fails.
- Runner fetches 9 browse combos (genre×sort×status) every 6h and stores
JSON snapshots in MinIO libnovel-browse bucket (browse_refresh.go)
- Backend handleBrowse reads page-1 results from MinIO first; falls back
to live novelfire.net fetch; returns empty+cached:false on total failure
instead of 502
- Add BrowseStore interface (bookstore.go), MinIO put/get helpers (minio.go),
Store methods + compile-time assertion (store.go), BucketBrowse config,
wiring in cmd/backend and cmd/runner, docker-compose-new bucket init
- Fix ReapStaleTasks: PocketBase datetime fields require heartbeat_at=null
(not heartbeat_at="") in filter expressions, and nil (not "") in patch
payload — was causing 400 errors on every reap cycle
- backend/: Go API server and runner binaries with PocketBase + MinIO storage
- ui-v2/: SvelteKit frontend rewrite
- docker-compose-new.yml: compose file for the v2 stack
- .gitea/workflows/release-v2.yaml: CI/CD for backend, runner, and ui-v2 Docker Hub images
- scripts/pb-init.sh: migrate from wget to curl, add superuser bootstrap for fresh installs
- .env.example: document DOCKER_BUILDKIT=1 for Colima users
