17 Commits

Author SHA1 Message Date
Admin
e1621a3ec2 fix(infra): move Redis to prod, fix LibreTranslate config loading
All checks were successful
CI / Backend (push) Successful in 46s
CI / UI (push) Successful in 52s
Release / Docker / caddy (push) Successful in 34s
Release / Test backend (push) Successful in 48s
Release / Check ui (push) Successful in 49s
CI / Backend (pull_request) Successful in 25s
CI / UI (pull_request) Successful in 56s
Release / Docker / runner (push) Successful in 1m46s
Release / Docker / ui (push) Successful in 2m30s
Release / Docker / backend (push) Successful in 2m47s
Release / Gitea Release (push) Successful in 13s
- Add Redis sidecar to prod docker-compose; backend connects locally (redis:6379)
- Caddy layer4 now proxies redis.libnovel.cc:6380 → local redis:6379 (not homelab LAN)
- Remove HOMELAB_REDIS_ADDR; homelab runner connects out to prod Redis via rediss://
- Remove local Redis from homelab runner compose; drop redis_data volume
- Fix config.Load() missing LibreTranslate section — LIBRETRANSLATE_URL was never read
2026-03-31 18:26:32 +05:00
Admin
4e70a2981d fix(pipeline): add redis+libretranslate to homelab, make Asynq enqueue errors non-fatal
Some checks failed
CI / Backend (pull_request) Successful in 35s
CI / UI (push) Failing after 37s
CI / Backend (push) Successful in 53s
CI / UI (pull_request) Failing after 27s
- homelab/docker-compose.yml: add redis:7-alpine service (port 6379 bound to host
  so Caddy TLS proxy on prod can reach it), add libretranslate service, add
  redis_data and libretranslate_data volumes
- asynqqueue/producer.go: Asynq enqueue failures are now logged as warnings instead
  of returned as errors — PB record already exists so runner picks it up via poll
- backend/main.go: pass logger to NewProducer

Root cause: Redis was not reachable at 192.168.0.109:6379 because the redis
container had no host port binding. Caddy TLS proxy terminates TLS but could
not TCP-connect to the backend Redis.
2026-03-31 10:18:13 +05:00
Admin
8d95411139 fix(caddy): add SNI connection_policy to layer4 TLS block and anchor redis.libnovel.cc cert
Some checks failed
CI / Backend (pull_request) Successful in 30s
CI / UI (pull_request) Successful in 46s
Release / Test backend (push) Successful in 32s
CI / Backend (push) Successful in 49s
CI / UI (push) Successful in 57s
Release / Check ui (push) Successful in 31s
Release / Docker / caddy (push) Successful in 1m19s
Release / Docker / runner (push) Failing after 1m11s
Release / Docker / ui (push) Successful in 2m1s
Release / Docker / backend (push) Successful in 5m1s
Release / Gitea Release (push) Has been skipped
Without a connection_policy, Caddy resolved the TLS cert by the Docker
internal IP (172.18.0.5) instead of the hostname, causing TLS handshake
failures on :6380 (rediss:// from prod backend → homelab Redis / Asynq).

Changes:
- Caddyfile: add connection_policy { match { sni redis.libnovel.cc } } to
  the layer4 :6380 tls handler so Caddy picks the correct cert
- Caddyfile: add redis.libnovel.cc virtual-host block (respond 404) to
  force Caddy to obtain and cache a TLS cert for that hostname
- homelab/docker-compose.yml: add REDIS_ADDR, REDIS_PASSWORD,
  LIBRETRANSLATE_URL, LIBRETRANSLATE_API_KEY, and
  RUNNER_MAX_CONCURRENT_TRANSLATION to the runner service for parity with
  homelab/runner/docker-compose.yml
2026-03-31 00:02:01 +05:00
Admin
1b4835daeb fix(homelab): switch Fider SMTP to port 587 + STARTTLS
All checks were successful
CI / Backend (pull_request) Successful in 50s
CI / UI (pull_request) Successful in 1m10s
Port 465 (SMTPS) is blocked on the homelab server. Port 587 with STARTTLS
works. Updated FIDER_SMTP_PORT=587 and FIDER_SMTP_ENABLE_STARTTLS=true in
Doppler prd_homelab, and made EMAIL_SMTP_ENABLE_STARTTLS dynamic so it reads
from Doppler instead of being hardcoded.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 22:16:01 +05:00
Admin
c9c12fc4a8 fix(infra): correct doppler entrypoint for watchtower container
All checks were successful
CI / UI (pull_request) Successful in 39s
CI / Backend (pull_request) Successful in 45s
- Fix binary path: /usr/bin/doppler (not /usr/local/bin)
- Mount /root/.doppler config so the container can auth without DOPPLER_TOKEN env
- Set HOME=/root so doppler locates the mounted config directory
- Add explicit --project/--config flags to override directory-scope lookup
- Production: --project libnovel --config prd
- Homelab: --project libnovel --config prd_homelab

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 22:05:33 +05:00
Admin
dd35024d02 chore(infra): run watchtower via doppler for fresh secrets on restart
All checks were successful
CI / UI (pull_request) Successful in 39s
CI / Backend (pull_request) Successful in 47s
Mount the host doppler binary into the watchtower container and use it as
the entrypoint so WATCHTOWER_NOTIFICATION_URL and other secrets are fetched
from Doppler each time the container starts, rather than being baked in at
compose-up time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:52:32 +05:00
Admin
588e455aae chore(homelab): add LibreTranslate service to runner compose
Some checks failed
CI / Backend (pull_request) Failing after 11s
CI / UI (pull_request) Failing after 11s
- libretranslate/libretranslate:latest, internal Docker network only
- LT_LOAD_ONLY=en,ru,id,pt,fr (only pairs the runner needs)
- LT_API_KEYS=true, key stored in Doppler prd_homelab
- Runner depends_on libretranslate (service_healthy)
- LIBRETRANSLATE_URL=http://libretranslate:5000 (no tunnel needed)
- RUNNER_MAX_CONCURRENT_TRANSLATION wired from Doppler
2026-03-29 11:42:52 +05:00
Admin
f75292f531 fix(homelab): add Google + GitHub OAuth env vars to Fider service
All checks were successful
CI / Backend (pull_request) Successful in 45s
CI / UI (pull_request) Successful in 38s
2026-03-28 19:38:11 +05:00
Admin
4a00d953bb feat(ui): show build version in footer; enable watchtower for caddy
Some checks failed
CI / Check ui (pull_request) Successful in 50s
CI / Docker / ui (pull_request) Failing after 38s
CI / Test backend (pull_request) Successful in 3m31s
CI / Docker / backend (pull_request) Failing after 11s
CI / Docker / runner (pull_request) Failing after 11s
CI / Docker / caddy (pull_request) Successful in 11m0s
- Footer now displays build version tag + short commit SHA (from
  PUBLIC_BUILD_VERSION / PUBLIC_BUILD_COMMIT env vars baked in at
  image build time); falls back to 'dev' in local builds
- docker-compose.yml: add watchtower label to caddy service so it
  auto-updates alongside backend/ui/runner on new image pushes
- homelab/docker-compose.yml: use locally-built kokoro-fastapi:latest
  image (consistent with actual homelab setup)
2026-03-28 15:42:22 +05:00
Admin
fe1a933fd0 feat(queue): replace PocketBase polling with Asynq + Redis
Some checks failed
CI / Docker / caddy (pull_request) Failing after 50s
CI / Check ui (pull_request) Successful in 1m5s
Release / Check ui (push) Successful in 45s
Release / Test backend (push) Successful in 1m29s
CI / Docker / ui (pull_request) Successful in 1m28s
Release / Docker / backend (push) Successful in 3m14s
CI / Test backend (pull_request) Failing after 42s
CI / Docker / backend (pull_request) Has been skipped
CI / Docker / runner (pull_request) Has been skipped
Release / Docker / caddy (push) Successful in 6m48s
Release / Docker / ui (push) Successful in 2m8s
Release / Docker / runner (push) Successful in 2m51s
Release / Upload source maps (push) Failing after 53s
Release / Gitea Release (push) Has been skipped
Introduce a Redis-backed Asynq task queue so the runner consumes TTS
jobs pushed by the backend instead of polling PocketBase.

- backend/internal/asynqqueue: Producer and Consumer wrappers
- backend/internal/runner: AsynqRunner mux, per-instance Prometheus
  registry (fixes duplicate-collector panic in tests), redisConnOpt
- backend/internal/config: REDIS_ADDR / REDIS_PASSWORD env vars
- backend/cmd/{backend,runner}/main.go: wire Redis when env set; fall
  back to legacy poll mode when unset
- Caddyfile: caddy-l4 TCP proxy for redis.libnovel.cc:6380 → homelab
- caddy/Dockerfile: add --with github.com/mholt/caddy-l4
- docker-compose.yml: Caddy exposes 6380, backend/runner get Redis env
- homelab/runner/docker-compose.yml: Redis sidecar, runner depends_on
- homelab/otel/grafana: Grafana dashboards (backend, catalogue, runner)
  and alerting rules / contact-points provisioning
2026-03-28 14:32:40 +05:00
Admin
b30aa23d64 fix(homelab): pocket-tts uses locally-built image, correct start command and volumes
Some checks failed
CI / Check ui (pull_request) Successful in 35s
CI / Test backend (pull_request) Successful in 39s
CI / Docker / ui (pull_request) Successful in 1m27s
CI / Docker / backend (pull_request) Successful in 1m57s
CI / Docker / runner (pull_request) Failing after 35s
CI / Docker / caddy (pull_request) Successful in 6m14s
- pocket-tts has no prebuilt image on ghcr.io — must be built from source on homelab
- Updated image to pocket-tts:latest (local tag), add command with --host 0.0.0.0
- Add model cache volumes (pocket_tts_cache, hf_cache) so model weights survive restarts
- start_period increased to 120s (first startup downloads weights)
2026-03-27 16:29:36 +05:00
Admin
fea09e3e23 feat(tts): add pocket-tts engine alongside kokoro-fastapi
Some checks failed
Release / Test backend (push) Successful in 36s
Release / Check ui (push) Successful in 50s
Release / Docker / caddy (push) Failing after 45s
CI / Test backend (pull_request) Successful in 39s
CI / Check ui (pull_request) Failing after 11s
CI / Docker / ui (pull_request) Has been skipped
CI / Docker / caddy (pull_request) Failing after 11s
Release / Docker / backend (push) Failing after 48s
Release / Docker / runner (push) Failing after 58s
Release / Upload source maps (push) Failing after 39s
CI / Docker / backend (pull_request) Failing after 34s
CI / Docker / runner (pull_request) Failing after 30s
Release / Docker / ui (push) Successful in 2m42s
Release / Gitea Release (push) Has been skipped
- New backend/internal/pockettts package: POST /tts client for
  kyutai-labs/pocket-tts; streams WAV response and transcodes to MP3
  via ffmpeg so the rest of the pipeline stays format-agnostic
- config.PocketTTS struct + POCKET_TTS_URL env var
- runner.Dependencies.PocketTTS field; runAudioTask routes by voice name:
  pockettts.IsPocketTTSVoice (alba, marius, javert, …) → pocket-tts,
  everything else → kokoro-fastapi
- Dockerfile: runner stage switched from distroless/static to alpine:3.21
  with ffmpeg + ca-certificates installed
- homelab compose: runner gets POCKET_TTS_URL=http://pocket-tts:8000
- Doppler prd + prd_homelab: KOKORO_URL=https://tts.libnovel.cc,
  POCKET_TTS_URL=https://pocket-tts.libnovel.cc
2026-03-27 16:17:13 +05:00
Admin
4831c74acc feat(observability+tts): OTel logs for backend/runner/ui; add kokoro-fastapi (GPU) and pocket-tts (CPU) to homelab
Some checks failed
Release / Check ui (push) Successful in 47s
Release / Docker / caddy (push) Successful in 48s
Release / Test backend (push) Successful in 3m24s
CI / Check ui (pull_request) Successful in 52s
CI / Test backend (pull_request) Successful in 3m6s
CI / Docker / caddy (pull_request) Failing after 38s
Release / Upload source maps (push) Failing after 33s
Release / Docker / ui (push) Failing after 2m41s
Release / Docker / backend (push) Successful in 2m48s
Release / Docker / runner (push) Failing after 1m1s
Release / Gitea Release (push) Has been skipped
CI / Docker / ui (pull_request) Successful in 1m48s
CI / Docker / backend (pull_request) Successful in 1m56s
CI / Docker / runner (pull_request) Successful in 1m45s
- otelsetup.Init now returns a *slog.Logger wired to the OTLP log exporter
  so all slog output is shipped to Loki with embedded trace IDs
- backend and runner both adopt the new OTel-bridged logger
- runner.runScrapeTask and runAudioTask now emit structured OTel spans
- ui/hooks.server.ts adds BatchLogRecordProcessor alongside existing trace exporter
- homelab: add kokoro-fastapi GPU service (ghcr.io/remsky/kokoro-fastapi-gpu)
  using deploy.resources.reservations for NVIDIA GPU, exposed internally on :8880
- homelab: add pocket-tts CPU service (ghcr.io/kyutai-labs/pocket-tts) on :8000
- runner KOKORO_URL hardcoded to http://kokoro-fastapi:8880 (fixes DNS failure
  for the stale kokoro.kalekber.cc hostname)
2026-03-27 16:05:22 +05:00
Admin
ee3ed29316 Add OTel distributed tracing to backend, ui, and runner
Some checks failed
CI / Check ui (pull_request) Failing after 44s
CI / Docker / ui (pull_request) Has been skipped
CI / Test backend (pull_request) Successful in 3m30s
CI / Docker / backend (pull_request) Successful in 2m28s
CI / Docker / caddy (pull_request) Successful in 5m22s
CI / Docker / runner (pull_request) Successful in 1m52s
Go backend:
- Add OTel SDK + otelhttp middleware deps (go.mod)
- New internal/otelsetup package: init OTLP/HTTP TracerProvider from env vars
- cmd/backend/main.go: call otelsetup.Init() after logger + ctx setup
- internal/backend/server.go: wrap mux with otelhttp.NewHandler() before
  sentryhttp, so all HTTP spans are recorded

SvelteKit UI:
- Add @opentelemetry/sdk-node, exporter-trace-otlp-http, resources,
  semantic-conventions
- hooks.server.ts: init NodeSDK when OTEL_EXPORTER_OTLP_ENDPOINT is set;
  graceful shutdown on SIGTERM/SIGINT

Config:
- docker-compose.yml: pass OTEL_EXPORTER_OTLP_ENDPOINT + OTEL_SERVICE_NAME
  to backend, runner, and ui services
- homelab/docker-compose.yml: fix runner OTel endpoint to HTTP port 4318
- Doppler prd: OTEL_EXPORTER_OTLP_ENDPOINT=https://otel.libnovel.cc
- Doppler prd_homelab: OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318

All services no-op gracefully when the env var is unset (local dev).
2026-03-26 23:00:52 +05:00
Admin
a39f660a37 Migrate tooling to homelab; add OTel observability stack
All checks were successful
CI / Test backend (pull_request) Successful in 34s
CI / Check ui (pull_request) Successful in 33s
CI / Docker / backend (pull_request) Successful in 1m26s
CI / Docker / runner (pull_request) Successful in 1m50s
CI / Docker / ui (pull_request) Successful in 1m3s
CI / Docker / caddy (pull_request) Successful in 5m53s
- Remove GlitchTip, Umami, Fider, Gotify, Uptime Kuma, Dozzle, Watchtower
  from prod docker-compose (now run on homelab)
- Add dozzle-agent on prod (127.0.0.1:7007) for homelab Dozzle to connect to
- Remove corresponding subdomain blocks from Caddyfile (now routed via
  Cloudflare Tunnel from homelab)
- Add homelab/docker-compose.yml: unified homelab stack with all migrated
  tooling services plus full OTel stack (Tempo 2.6.1, Loki, Prometheus,
  OTel Collector, Grafana)
- Add homelab/otel/: Tempo, Loki, Prometheus, OTel Collector configs +
  Grafana provisioning (datasources + dashboards)
- Add homelab/dozzle/users.yml for Dozzle auth
2026-03-26 21:22:43 +05:00
Admin
a50b968b95 fix(infra): expose Meilisearch via search.libnovel.cc for homelab runner indexing
Some checks failed
CI / Test backend (pull_request) Has been cancelled
CI / Check ui (pull_request) Has been cancelled
CI / Docker / backend (pull_request) Has been cancelled
CI / Docker / runner (pull_request) Has been cancelled
CI / Docker / ui (pull_request) Has been cancelled
CI / Docker / caddy (pull_request) Has been cancelled
Release / Check ui (push) Failing after 22s
Release / Upload source maps (push) Has been skipped
Release / Docker / ui (push) Has been skipped
Release / Test backend (push) Successful in 36s
Release / Docker / caddy (push) Successful in 1m16s
Release / Docker / backend (push) Successful in 2m49s
Release / Docker / runner (push) Successful in 3m29s
Release / Gitea Release (push) Has been skipped
- Add search.libnovel.cc Caddy vhost proxying to meilisearch:7700
- Pass MEILI_URL + MEILI_API_KEY from Doppler into homelab runner
- Set GODEBUG=preferIPv4=1 to work around missing IPv6 route on homelab
- Update comments to reflect runner now indexes books into Meilisearch
2026-03-25 20:27:50 +05:00
Admin
920ac0d41b feat(auth): add email verification to registration flow
Some checks failed
CI / Test backend (pull_request) Failing after 11s
CI / Check ui (pull_request) Failing after 11s
CI / Docker / backend (pull_request) Has been skipped
CI / Docker / runner (pull_request) Has been skipped
CI / Docker / ui (pull_request) Has been skipped
CI / Docker / caddy (pull_request) Successful in 2m53s
Release / Test backend (push) Successful in 19s
Release / Check ui (push) Successful in 33s
Release / Docker / caddy (push) Failing after 1m26s
Release / Docker / ui (push) Failing after 11s
Release / Docker / backend (push) Successful in 2m2s
Release / Docker / runner (push) Successful in 2m24s
Release / Gitea Release (push) Has been skipped
- Add email/email_verified/verification_token/verification_token_exp fields
  to app_users PocketBase schema (pb-init-v3.sh)
- Add SMTP env vars to UI service in docker-compose.yml
- New email.ts: raw TLS SMTP mailer via Node tls module, sendVerificationEmail()
- createUser() now takes email param, stores verification token (24h TTL)
- loginUser() throws 'Email not verified' when email_verified is false
- New /verify-email route: validates token, verifies user, auto-logs in
- Login page: email field in register form, check-inbox state after register
- /api/auth/register (iOS): returns { pending_verification, email } instead of token
- Add pb.libnovel.cc and storage.libnovel.cc Caddy virtual hosts for homelab runner
- Add homelab runner docker-compose and libnovel.sh helper script
2026-03-24 20:18:24 +05:00