19 Commits

Author	SHA1	Message	Date
root	fcd4b3ad7f	fix: wire import chapter ingestion, live task polling, a11y labels, notification user targeting ... - NewBookImporter now takes *Store instead of raw *minio.Client (same package access) - Runner Dependencies gains ChapterIngester interface; storeImportedChapters no-op removed - store.IngestChapters is now actually called after PDF/EPUB extraction - BookImport and ChapterIngester wired in runner main.go - CreateImportTask interface gains initiatorUserID param; threaded through store/asynq/handler - domain.ImportTask gains InitiatorUserID field; parseImportTask populates it - Runner notifies initiator (falls back to 'admin' when empty) - UI: import task list polls every 3s while any task is pending/running - UI: label[for] + input[id] fix for a11y warnings in admin import page	2026-04-09 11:00:01 +05:00
root	ab92bf84bb	feat: import review step + admin notifications ... - Import page: add file upload with review step before committing - Backend: add analyze endpoint to preview chapters before import - Add notifications collection + API for admin alerts - Add bell icon in header with notification dropdown (admin only) - Runner creates notification on import completion - Notifications link to /admin/import for easy review	2026-04-09 10:30:36 +05:00
root	e399b1ce01	feat: admin UX overhaul — status filters, retry/cancel, mobile cards, i18n, shelf pre-populate ... - Admin layout: SVG icons, active highlight, divider between nav sections - Scrape page: status filter pills with counts, text + status combined search - Audio page: status filter pills, cancel jobs, retry failed jobs, mobile cards for cache tab - Translation page: status filter pills (incl. cancelled), cancel + retry jobs, mobile cancel/retry cards, i18n for all labels - AI Jobs page: fix concurrent cancel (Set instead of single slot), per-job cancel errors inline, full mobile card layout, i18n title/heading - Text-gen page: tagline editable input + copy, warnings copy, i18n title/heading - Book page: chapter cover Save button, audio monitor link, currentShelf pre-populated from server - pocketbase.ts: add getBookShelf(), shelf field on UserLibraryEntry - New API route: POST /api/admin/translation/bulk (proxy for translation retry) - i18n: 15 new admin_translation_*, admin_ai_jobs_*, admin_text_gen_* keys across all 5 locales	2026-04-08 18:30:35 +05:00
Admin	9c79fd5deb	feat: AI job tracking, range support, auto-prompt, and resume ... - New `ai_jobs` PocketBase collection tracks all long-running AI tasks (batch-covers, chapter-names) with status, progress, and cancellation - `handlers_aijobs.go`: GET/cancel endpoints for ai_jobs; centralised cancel registry (moved from handlers_catalogue) - Batch-covers and chapter-names SSE handlers now create/resume ai_job records, support from_item/to_item ranges, and resume from items_done on restart via job_id - New `POST /api/admin/image-gen/auto-prompt`: generates an image prompt from book description (cover) or chapter title (chapter) via LLM - image-gen page: "Auto-prompt" button calls auto-prompt API when a slug is selected; falls back gracefully if TextGen not configured - text-gen chapter-names: from/to chapter range inputs + job ID display - catalogue-tools batch-covers: from/to item range + resume job ID input - pb-init-v3.sh: adds ai_jobs collection (idempotent) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-04-05 15:40:52 +05:00
Admin	0b82d96798	feat: admin Text Gen tool — chapter names + book description via CF Workers AI ... Adds backend handlers and SvelteKit UI for an admin text generation tool. The tool lets admins propose and apply AI-generated chapter titles and book descriptions using Cloudflare Workers AI (12 LLM models, model selector shared across both tabs).	2026-04-04 13:16:10 +05:00
Admin	d1b7d3e36c	feat: admin image generation via Cloudflare Workers AI ... - Add cfai/image.go: ImageGenClient with GenerateImage, GenerateImageFromReference, AllImageModels (9 models) - Add handlers_image.go: GET /api/admin/image-gen/models + POST /api/admin/image-gen (JSON + multipart) - Wire ImageGen client in main.go + server.go Dependencies - Add admin/image-gen SvelteKit page: type toggle, model selector, prompt, reference img2img, advanced options, result panel, history, save-as-cover, download - Add SvelteKit proxy route api/admin/image-gen forwarding to Go backend - Add admin_nav_image_gen message key to all 5 locale files - Add Image Gen nav link to admin layout	2026-04-04 11:46:22 +05:00
Admin	aaa008ac99	feat: add Cloudflare AI TTS engine (aura-2-en) with voice grouping in UI	2026-04-04 11:12:55 +05:00
Admin	a771405db8	feat(audio): WAV streaming, bulk audio generation admin endpoints, cancel/resume ... - Add StreamAudioWAV() to pocket-tts and Kokoro clients; pocket-tts streams raw WAV directly (no ffmpeg), Kokoro requests response_format:wav with stream:true - GET /api/audio-stream supports ?format=wav for lower-latency first-byte delivery; WAV cached separately in MinIO as {slug}/{n}/{voice}.wav - Add GET /api/admin/audio/jobs with optional ?slug filter - Add POST /api/admin/audio/bulk {slug, voice, from, to, skip_existing, force} where skip_existing=true (default) resumes interrupted bulk jobs - Add POST /api/admin/audio/cancel-bulk {slug} to cancel all pending/running tasks - Add CancelAudioTasksBySlug to taskqueue.Producer + asynqqueue implementation - Add AudioObjectKeyExt to bookstore.AudioStore for format-aware MinIO keys Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-04-02 16:19:14 +05:00
Admin	4e70a2981d	fix(pipeline): add redis+libretranslate to homelab, make Asynq enqueue errors non-fatal ... - homelab/docker-compose.yml: add redis:7-alpine service (port 6379 bound to host so Caddy TLS proxy on prod can reach it), add libretranslate service, add redis_data and libretranslate_data volumes - asynqqueue/producer.go: Asynq enqueue failures are now logged as warnings instead of returned as errors — PB record already exists so runner picks it up via poll - backend/main.go: pass logger to NewProducer Root cause: Redis was not reachable at 192.168.0.109:6379 because the redis container had no host port binding. Caddy TLS proxy terminates TLS but could not TCP-connect to the backend Redis.	2026-03-31 10:18:13 +05:00
Admin	4e7f8c6266	feat: streaming audio endpoint with MinIO write-through cache ... Add GET /api/audio-stream/{slug}/{n}?voice= that streams MP3 audio to the client as TTS generates it, while simultaneously uploading to MinIO. On subsequent requests the endpoint redirects to the presigned MinIO URL, skipping generation entirely. - PocketTTS: StreamAudioMP3 pipes live WAV response body through ffmpeg (streaming transcode — no full-buffer wait) - Kokoro: StreamAudioMP3 uses stream:true mode, returning MP3 frames directly without the two-step download-link flow - AudioStore: PutAudioStream added for multipart MinIO upload from reader - WriteTimeout bumped 60s → 15min to accommodate full-chapter streams - X-Accel-Buffering: no header disables Caddy/nginx response buffering	2026-03-30 22:02:36 +05:00
Admin	28ac8d8826	feat(translation): machine translation pipeline + admin bulk enqueue UI ... - LibreTranslate client (chunks on blank lines, ≤4500 chars, 3-goroutine semaphore) - Runner translation task loop (OTel, heartbeat, MinIO storage) - PocketBase translation_jobs collection support (create/claim/finish/list) - Per-chapter language switcher on chapter reader (EN/RU/ID/PT/FR, polls until done) - Admin /admin/translation page: bulk enqueue form + live-polling jobs table - New backend routes: POST /api/translation/{slug}/{n}, GET /api/translation/status, GET /api/translation/{slug}/{n}, GET /api/admin/translation/jobs, POST /api/admin/translation/bulk - ListTranslationTasks added to taskqueue.Reader interface + store impl - All builds and tests pass; svelte-check: 0 errors	2026-03-29 11:32:42 +05:00
Admin	aef9e04419	fix(runner): harden catalogue scrape against 429s; disable sourcemap upload ... - scraper.go: ScrapeCatalogue now uses retryGet (9 attempts, 10s base) + 500–1500ms inter-page jitter instead of bare GetContent. ScrapeMetadata also switched to retryGet so a single 429 on a book page is retried rather than aborting the whole refresh. - catalogue_refresh.go: per-book delay is now configurable (RUNNER_CATALOGUE_REQUEST_DELAY, default 2s) + up to 50% random jitter applied before every metadata fetch. Only metadata is scraped here — chapters are fetched on-demand, not during catalogue refresh. Progress logged every 50 books instead of 100. - config.go / runner.go / main.go: add CatalogueRequestDelay field wired from RUNNER_CATALOGUE_REQUEST_DELAY env var. - release.yaml: comment out upload-sourcemaps job and remove it from the release needs; GlitchTip auth token needs refreshing after DB wipe.	2026-03-28 16:28:36 +05:00
Admin	fe1a933fd0	feat(queue): replace PocketBase polling with Asynq + Redis ... Introduce a Redis-backed Asynq task queue so the runner consumes TTS jobs pushed by the backend instead of polling PocketBase. - backend/internal/asynqqueue: Producer and Consumer wrappers - backend/internal/runner: AsynqRunner mux, per-instance Prometheus registry (fixes duplicate-collector panic in tests), redisConnOpt - backend/internal/config: REDIS_ADDR / REDIS_PASSWORD env vars - backend/cmd/{backend,runner}/main.go: wire Redis when env set; fall back to legacy poll mode when unset - Caddyfile: caddy-l4 TCP proxy for redis.libnovel.cc:6380 → homelab - caddy/Dockerfile: add --with github.com/mholt/caddy-l4 - docker-compose.yml: Caddy exposes 6380, backend/runner get Redis env - homelab/runner/docker-compose.yml: Redis sidecar, runner depends_on - homelab/otel/grafana: Grafana dashboards (backend, catalogue, runner) and alerting rules / contact-points provisioning	2026-03-28 14:32:40 +05:00
Admin	fea09e3e23	feat(tts): add pocket-tts engine alongside kokoro-fastapi ... - New backend/internal/pockettts package: POST /tts client for kyutai-labs/pocket-tts; streams WAV response and transcodes to MP3 via ffmpeg so the rest of the pipeline stays format-agnostic - config.PocketTTS struct + POCKET_TTS_URL env var - runner.Dependencies.PocketTTS field; runAudioTask routes by voice name: pockettts.IsPocketTTSVoice (alba, marius, javert, …) → pocket-tts, everything else → kokoro-fastapi - Dockerfile: runner stage switched from distroless/static to alpine:3.21 with ffmpeg + ca-certificates installed - homelab compose: runner gets POCKET_TTS_URL=http://pocket-tts:8000 - Doppler prd + prd_homelab: KOKORO_URL=https://tts.libnovel.cc, POCKET_TTS_URL=https://pocket-tts.libnovel.cc	2026-03-27 16:17:13 +05:00
Admin	4831c74acc	feat(observability+tts): OTel logs for backend/runner/ui; add kokoro-fastapi (GPU) and pocket-tts (CPU) to homelab ... - otelsetup.Init now returns a *slog.Logger wired to the OTLP log exporter so all slog output is shipped to Loki with embedded trace IDs - backend and runner both adopt the new OTel-bridged logger - runner.runScrapeTask and runAudioTask now emit structured OTel spans - ui/hooks.server.ts adds BatchLogRecordProcessor alongside existing trace exporter - homelab: add kokoro-fastapi GPU service (ghcr.io/remsky/kokoro-fastapi-gpu) using deploy.resources.reservations for NVIDIA GPU, exposed internally on :8880 - homelab: add pocket-tts CPU service (ghcr.io/kyutai-labs/pocket-tts) on :8000 - runner KOKORO_URL hardcoded to http://kokoro-fastapi:8880 (fixes DNS failure for the stale kokoro.kalekber.cc hostname)	2026-03-27 16:05:22 +05:00
Admin	ee3ed29316	Add OTel distributed tracing to backend, ui, and runner ... Go backend: - Add OTel SDK + otelhttp middleware deps (go.mod) - New internal/otelsetup package: init OTLP/HTTP TracerProvider from env vars - cmd/backend/main.go: call otelsetup.Init() after logger + ctx setup - internal/backend/server.go: wrap mux with otelhttp.NewHandler() before sentryhttp, so all HTTP spans are recorded SvelteKit UI: - Add @opentelemetry/sdk-node, exporter-trace-otlp-http, resources, semantic-conventions - hooks.server.ts: init NodeSDK when OTEL_EXPORTER_OTLP_ENDPOINT is set; graceful shutdown on SIGTERM/SIGINT Config: - docker-compose.yml: pass OTEL_EXPORTER_OTLP_ENDPOINT + OTEL_SERVICE_NAME to backend, runner, and ui services - homelab/docker-compose.yml: fix runner OTel endpoint to HTTP port 4318 - Doppler prd: OTEL_EXPORTER_OTLP_ENDPOINT=https://otel.libnovel.cc - Doppler prd_homelab: OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318 All services no-op gracefully when the env var is unset (local dev).	2026-03-26 23:00:52 +05:00
Admin	59e8cdb19a	chore: migrate to v3, Doppler secrets, clean up legacy code ... - Remove all pre-v3 code: scraper, ui-v2, backend v1, ios v1+v2, legacy CI workflows - Flatten v3/ contents to repo root - Add Doppler secrets management (project=libnovel, config=prd) - Add justfile with doppler run wrappers for all docker compose commands - Strip hardcoded env fallbacks from docker-compose.yml - Add minimal README.md - Clean up .gitignore	2026-03-23 17:21:12 +05:00
Admin	3918bc8dc3	fix: serve browse pages from MinIO cache; fix ReapStaleTasks PocketBase filter ... - Runner fetches 9 browse combos (genre×sort×status) every 6h and stores JSON snapshots in MinIO libnovel-browse bucket (browse_refresh.go) - Backend handleBrowse reads page-1 results from MinIO first; falls back to live novelfire.net fetch; returns empty+cached:false on total failure instead of 502 - Add BrowseStore interface (bookstore.go), MinIO put/get helpers (minio.go), Store methods + compile-time assertion (store.go), BucketBrowse config, wiring in cmd/backend and cmd/runner, docker-compose-new bucket init - Fix ReapStaleTasks: PocketBase datetime fields require heartbeat_at=null (not heartbeat_at="") in filter expressions, and nil (not "") in patch payload — was causing 400 errors on every reap cycle	2026-03-15 21:19:28 +05:00
Admin	5825b859b7	feat: add v2 stack (backend, runner, ui-v2) with release workflow ... - backend/: Go API server and runner binaries with PocketBase + MinIO storage - ui-v2/: SvelteKit frontend rewrite - docker-compose-new.yml: compose file for the v2 stack - .gitea/workflows/release-v2.yaml: CI/CD for backend, runner, and ui-v2 Docker Hub images - scripts/pb-init.sh: migrate from wget to curl, add superuser bootstrap for fresh installs - .env.example: document DOCKER_BUILDKIT=1 for Colima users	2026-03-15 19:32:40 +05:00