14 Commits

Author	SHA1	Message	Date
Admin	aaa008ac99	feat: add Cloudflare AI TTS engine (aura-2-en) with voice grouping in UI	2026-04-04 11:12:55 +05:00
Admin	8e611840d1	fix: add 30s timeout to PB HTTP client; halve heartbeat tick interval ... - storage/pocketbase.go: replace http.DefaultClient (no timeout) with a dedicated pbHTTPClient{Timeout: 30s} so a slow/hung PocketBase cannot stall the backend or runner indefinitely - runner/asynq_runner.go: heartbeat ticker was firing at StaleTaskThreshold (2 min) == the Docker healthcheck deadline, so a single missed tick would mark the container unhealthy; halved to StaleTaskThreshold/2 (1 min)	2026-04-02 18:49:04 +05:00
Admin	a771405db8	feat(audio): WAV streaming, bulk audio generation admin endpoints, cancel/resume ... - Add StreamAudioWAV() to pocket-tts and Kokoro clients; pocket-tts streams raw WAV directly (no ffmpeg), Kokoro requests response_format:wav with stream:true - GET /api/audio-stream supports ?format=wav for lower-latency first-byte delivery; WAV cached separately in MinIO as {slug}/{n}/{voice}.wav - Add GET /api/admin/audio/jobs with optional ?slug filter - Add POST /api/admin/audio/bulk {slug, voice, from, to, skip_existing, force} where skip_existing=true (default) resumes interrupted bulk jobs - Add POST /api/admin/audio/cancel-bulk {slug} to cancel all pending/running tasks - Add CancelAudioTasksBySlug to taskqueue.Producer + asynqqueue implementation - Add AudioObjectKeyExt to bookstore.AudioStore for format-aware MinIO keys Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-04-02 16:19:14 +05:00
Admin	4e7f8c6266	feat: streaming audio endpoint with MinIO write-through cache ... Add GET /api/audio-stream/{slug}/{n}?voice= that streams MP3 audio to the client as TTS generates it, while simultaneously uploading to MinIO. On subsequent requests the endpoint redirects to the presigned MinIO URL, skipping generation entirely. - PocketTTS: StreamAudioMP3 pipes live WAV response body through ffmpeg (streaming transcode — no full-buffer wait) - Kokoro: StreamAudioMP3 uses stream:true mode, returning MP3 frames directly without the two-step download-link flow - AudioStore: PutAudioStream added for multipart MinIO upload from reader - WriteTimeout bumped 60s → 15min to accommodate full-chapter streams - X-Accel-Buffering: no header disables Caddy/nginx response buffering	2026-03-30 22:02:36 +05:00
Admin	5da880d189	fix(runner): add heartbeat + translation polling to asynq mode ... Two bugs prevented asynq mode from working correctly on the homelab runner: 1. No healthcheck file: asynq mode never writes /tmp/runner.alive, so Docker healthcheck always fails. Added heartbeat goroutine that writes the file every StaleTaskThreshold (30s). 2. Translation tasks not dispatched: translation uses ClaimNextTranslationTask (PocketBase poll queue), not Redis/asynq. Audio + scrape use asynq mux, but translation sits in PocketBase forever. Added pollTranslationTasks() goroutine that polls PocketBase on the same PollInterval as the old poll() loop. All Go tests pass (go test ./... in backend/).	2026-03-29 20:22:37 +05:00
Admin	28ac8d8826	feat(translation): machine translation pipeline + admin bulk enqueue UI ... - LibreTranslate client (chunks on blank lines, ≤4500 chars, 3-goroutine semaphore) - Runner translation task loop (OTel, heartbeat, MinIO storage) - PocketBase translation_jobs collection support (create/claim/finish/list) - Per-chapter language switcher on chapter reader (EN/RU/ID/PT/FR, polls until done) - Admin /admin/translation page: bulk enqueue form + live-polling jobs table - New backend routes: POST /api/translation/{slug}/{n}, GET /api/translation/status, GET /api/translation/{slug}/{n}, GET /api/admin/translation/jobs, POST /api/admin/translation/bulk - ListTranslationTasks added to taskqueue.Reader interface + store impl - All builds and tests pass; svelte-check: 0 errors	2026-03-29 11:32:42 +05:00
Admin	aef9e04419	fix(runner): harden catalogue scrape against 429s; disable sourcemap upload ... - scraper.go: ScrapeCatalogue now uses retryGet (9 attempts, 10s base) + 500–1500ms inter-page jitter instead of bare GetContent. ScrapeMetadata also switched to retryGet so a single 429 on a book page is retried rather than aborting the whole refresh. - catalogue_refresh.go: per-book delay is now configurable (RUNNER_CATALOGUE_REQUEST_DELAY, default 2s) + up to 50% random jitter applied before every metadata fetch. Only metadata is scraped here — chapters are fetched on-demand, not during catalogue refresh. Progress logged every 50 books instead of 100. - config.go / runner.go / main.go: add CatalogueRequestDelay field wired from RUNNER_CATALOGUE_REQUEST_DELAY env var. - release.yaml: comment out upload-sourcemaps job and remove it from the release needs; GlitchTip auth token needs refreshing after DB wipe.	2026-03-28 16:28:36 +05:00
Admin	fe1a933fd0	feat(queue): replace PocketBase polling with Asynq + Redis ... Introduce a Redis-backed Asynq task queue so the runner consumes TTS jobs pushed by the backend instead of polling PocketBase. - backend/internal/asynqqueue: Producer and Consumer wrappers - backend/internal/runner: AsynqRunner mux, per-instance Prometheus registry (fixes duplicate-collector panic in tests), redisConnOpt - backend/internal/config: REDIS_ADDR / REDIS_PASSWORD env vars - backend/cmd/{backend,runner}/main.go: wire Redis when env set; fall back to legacy poll mode when unset - Caddyfile: caddy-l4 TCP proxy for redis.libnovel.cc:6380 → homelab - caddy/Dockerfile: add --with github.com/mholt/caddy-l4 - docker-compose.yml: Caddy exposes 6380, backend/runner get Redis env - homelab/runner/docker-compose.yml: Redis sidecar, runner depends_on - homelab/otel/grafana: Grafana dashboards (backend, catalogue, runner) and alerting rules / contact-points provisioning	2026-03-28 14:32:40 +05:00
Admin	fea09e3e23	feat(tts): add pocket-tts engine alongside kokoro-fastapi ... - New backend/internal/pockettts package: POST /tts client for kyutai-labs/pocket-tts; streams WAV response and transcodes to MP3 via ffmpeg so the rest of the pipeline stays format-agnostic - config.PocketTTS struct + POCKET_TTS_URL env var - runner.Dependencies.PocketTTS field; runAudioTask routes by voice name: pockettts.IsPocketTTSVoice (alba, marius, javert, …) → pocket-tts, everything else → kokoro-fastapi - Dockerfile: runner stage switched from distroless/static to alpine:3.21 with ffmpeg + ca-certificates installed - homelab compose: runner gets POCKET_TTS_URL=http://pocket-tts:8000 - Doppler prd + prd_homelab: KOKORO_URL=https://tts.libnovel.cc, POCKET_TTS_URL=https://pocket-tts.libnovel.cc	2026-03-27 16:17:13 +05:00
Admin	4831c74acc	feat(observability+tts): OTel logs for backend/runner/ui; add kokoro-fastapi (GPU) and pocket-tts (CPU) to homelab ... - otelsetup.Init now returns a *slog.Logger wired to the OTLP log exporter so all slog output is shipped to Loki with embedded trace IDs - backend and runner both adopt the new OTel-bridged logger - runner.runScrapeTask and runAudioTask now emit structured OTel spans - ui/hooks.server.ts adds BatchLogRecordProcessor alongside existing trace exporter - homelab: add kokoro-fastapi GPU service (ghcr.io/remsky/kokoro-fastapi-gpu) using deploy.resources.reservations for NVIDIA GPU, exposed internally on :8880 - homelab: add pocket-tts CPU service (ghcr.io/kyutai-labs/pocket-tts) on :8000 - runner KOKORO_URL hardcoded to http://kokoro-fastapi:8880 (fixes DNS failure for the stale kokoro.kalekber.cc hostname)	2026-03-27 16:05:22 +05:00
Admin	7e99fc6d70	fix(runner): fix audio task infinite loop and semaphore race ... Two bugs caused audio tasks to loop endlessly: 1. claimRecord never set heartbeat_at — newly claimed tasks had heartbeat_at=null, which matched the reaper's stale filter (heartbeat_at=null || heartbeat_at<threshold). Tasks were reaped and reset to pending within seconds of being claimed, before the 30s heartbeat goroutine had a chance to write a timestamp. Fix: set heartbeat_at=now() in claimRecord alongside status=running. 2. Audio semaphore was checked AFTER claiming the task. When the semaphore was full the select/break only broke the inner select, not the for loop — the code fell through and launched an uncapped goroutine that blocked forever on <-audioSem drain. The task also stayed status=running with no heartbeat, feeding bug #1. Fix: pre-acquire a semaphore slot BEFORE claiming the task; release it immediately if the queue is empty or claim fails.	2026-03-25 15:09:52 +05:00
Admin	59e8cdb19a	chore: migrate to v3, Doppler secrets, clean up legacy code ... - Remove all pre-v3 code: scraper, ui-v2, backend v1, ios v1+v2, legacy CI workflows - Flatten v3/ contents to repo root - Add Doppler secrets management (project=libnovel, config=prd) - Add justfile with doppler run wrappers for all docker compose commands - Strip hardcoded env fallbacks from docker-compose.yml - Add minimal README.md - Clean up .gitignore	2026-03-23 17:21:12 +05:00
Admin	3918bc8dc3	fix: serve browse pages from MinIO cache; fix ReapStaleTasks PocketBase filter ... - Runner fetches 9 browse combos (genre×sort×status) every 6h and stores JSON snapshots in MinIO libnovel-browse bucket (browse_refresh.go) - Backend handleBrowse reads page-1 results from MinIO first; falls back to live novelfire.net fetch; returns empty+cached:false on total failure instead of 502 - Add BrowseStore interface (bookstore.go), MinIO put/get helpers (minio.go), Store methods + compile-time assertion (store.go), BucketBrowse config, wiring in cmd/backend and cmd/runner, docker-compose-new bucket init - Fix ReapStaleTasks: PocketBase datetime fields require heartbeat_at=null (not heartbeat_at="") in filter expressions, and nil (not "") in patch payload — was causing 400 errors on every reap cycle	2026-03-15 21:19:28 +05:00
Admin	5825b859b7	feat: add v2 stack (backend, runner, ui-v2) with release workflow ... - backend/: Go API server and runner binaries with PocketBase + MinIO storage - ui-v2/: SvelteKit frontend rewrite - docker-compose-new.yml: compose file for the v2 stack - .gitea/workflows/release-v2.yaml: CI/CD for backend, runner, and ui-v2 Docker Hub images - scripts/pb-init.sh: migrate from wget to curl, add superuser bootstrap for fresh installs - .env.example: document DOCKER_BUILDKIT=1 for Colima users	2026-03-15 19:32:40 +05:00