Compare commits

...

15 Commits

Author SHA1 Message Date
Admin
d3f06c5c40 fix(caddy): add 404 error page; add health checks and lb_try_duration to ui upstream
Some checks failed
Release / Test backend (push) Successful in 26s
CI / Backend (push) Successful in 42s
CI / UI (push) Successful in 47s
Release / Check ui (push) Successful in 26s
CI / UI (pull_request) Successful in 26s
CI / Backend (pull_request) Successful in 44s
Release / Docker / caddy (push) Successful in 53s
Release / Docker / backend (push) Failing after 1m4s
Release / Docker / runner (push) Failing after 1m3s
Release / Docker / ui (push) Successful in 1m54s
Release / Gitea Release (push) Has been skipped
- Add caddy/errors/404.html (matches existing 502/503/504 style)
- Add handle_errors 404 block in Caddyfile
- Add active health checks (5s interval) and lb_try_duration 3s to the
  ui reverse_proxy so Caddy detects Watchtower container replacements
  quickly and serves the 502 maintenance page instead of a raw error
2026-03-28 21:32:04 +05:00
Admin
e71ddc2f8b fix(backend): add ffmpeg to backend image for pocket-tts voice sample generation
Some checks failed
CI / Backend (push) Successful in 29s
CI / UI (push) Successful in 27s
Release / Test backend (push) Successful in 37s
CI / Backend (pull_request) Failing after 11s
Release / Check ui (push) Successful in 49s
Release / Docker / caddy (push) Successful in 57s
CI / UI (pull_request) Successful in 56s
Release / Docker / runner (push) Failing after 1m22s
Release / Docker / backend (push) Failing after 1m46s
Release / Docker / ui (push) Successful in 2m25s
Release / Gitea Release (push) Has been skipped
handlePresignVoiceSample generates voice samples on demand via pocket-tts,
which requires WAV→MP3 transcoding via ffmpeg. The backend was using
distroless/static (no ffmpeg) so all pocket-tts preview requests returned 500.
Switch backend stage to Alpine + ffmpeg, matching the runner image.
2026-03-28 21:24:59 +05:00
Admin
b783dae5f4 refactor(admin): replace tab bar with sidebar layout
Some checks failed
CI / Backend (push) Successful in 38s
CI / UI (push) Successful in 42s
Release / Test backend (push) Successful in 40s
Release / Check ui (push) Successful in 33s
CI / Backend (pull_request) Successful in 28s
Release / Docker / caddy (push) Successful in 1m11s
CI / UI (pull_request) Successful in 29s
Release / Docker / ui (push) Successful in 1m54s
Release / Docker / runner (push) Successful in 4m27s
Release / Docker / backend (push) Failing after 5m11s
Release / Gitea Release (push) Has been skipped
Move admin navigation from a two-row tab strip into a persistent left
sidebar with grouped sections (Pages / Tools). Consolidate Scrape and
Audio entries in the global top nav into a single Admin link.
2026-03-28 21:15:49 +05:00
Admin
dcf40197d4 fix(ui): brighten footer link text for readability on dark background
Some checks failed
CI / Backend (push) Successful in 51s
CI / UI (push) Successful in 28s
Release / Test backend (push) Successful in 50s
Release / Docker / caddy (push) Successful in 1m0s
Release / Check ui (push) Successful in 1m15s
CI / Backend (pull_request) Successful in 41s
CI / UI (pull_request) Successful in 32s
Release / Docker / ui (push) Successful in 2m2s
Release / Docker / backend (push) Successful in 4m9s
Release / Docker / runner (push) Successful in 5m46s
Release / Gitea Release (push) Failing after 1s
2026-03-28 21:12:46 +05:00
Admin
9dae5e7cc0 fix(infra): add POCKET_TTS_URL to backend and runner services
Some checks failed
CI / Backend (push) Successful in 27s
Release / Test backend (push) Successful in 39s
CI / UI (push) Successful in 48s
Release / Check ui (push) Successful in 25s
CI / UI (pull_request) Successful in 25s
CI / Backend (pull_request) Successful in 45s
Release / Docker / caddy (push) Successful in 1m7s
Release / Docker / backend (push) Successful in 2m17s
Release / Docker / ui (push) Successful in 2m9s
Release / Docker / runner (push) Failing after 3m34s
Release / Gitea Release (push) Has been skipped
Backend was missing POCKET_TTS_URL entirely — pocketTTSClient was nil
so voices() only returned 67 Kokoro voices. Runner already had the var
via Doppler but it was absent from the compose environment block.

Also fix stray leading space on backend environment: key (YAML parse error).

Verified: /api/voices now returns 87 voices (67 kokoro + 20 pocket-tts).
2026-03-28 20:54:04 +05:00
Admin
908f5679fd fix(ui): defer catalogue filter navigation to explicit Apply button
Some checks failed
CI / Backend (push) Failing after 11s
Release / Check ui (push) Successful in 34s
Release / Test backend (push) Successful in 52s
CI / UI (push) Successful in 55s
Release / Docker / caddy (push) Successful in 48s
CI / UI (pull_request) Successful in 39s
CI / Backend (pull_request) Successful in 43s
Release / Docker / runner (push) Failing after 38s
Release / Docker / ui (push) Successful in 1m55s
Release / Docker / backend (push) Successful in 3m30s
Release / Gitea Release (push) Has been skipped
Removed onchange→navigateWithFilters from all three selects — the
immediate navigation on every change was the root cause of both bugs:
1. Filters applied before the user finished selecting all options.
2. Svelte 5 bind:value updates state after onchange fires, so the
   navigateWithFilters call read stale values → wrong URL params → no results.

Renamed navigateWithFilters to applyFilters (no overrides arg needed).
Added an amber Apply button next to Reset; selects now only update local
state until the user presses Apply.
2026-03-28 19:44:36 +05:00
Admin
f75292f531 fix(homelab): add Google + GitHub OAuth env vars to Fider service
All checks were successful
CI / Backend (pull_request) Successful in 45s
CI / UI (pull_request) Successful in 38s
2026-03-28 19:38:11 +05:00
Admin
2cf0528730 fix(ui): show version+SHA+build time in footer; fix env not reaching runtime image
Some checks failed
CI / Backend (push) Successful in 51s
CI / UI (push) Successful in 27s
Release / Docker / caddy (push) Failing after 23s
Release / Test backend (push) Successful in 38s
Release / Check ui (push) Successful in 39s
CI / Backend (pull_request) Successful in 28s
CI / UI (pull_request) Successful in 36s
Release / Docker / backend (push) Failing after 1m25s
Release / Docker / runner (push) Successful in 2m25s
Release / Docker / ui (push) Successful in 1m51s
Release / Gitea Release (push) Has been skipped
PUBLIC_BUILD_VERSION and PUBLIC_BUILD_COMMIT were set only in the builder
stage ENV — they were never re-declared in the runtime stage, so the Node
server started with them undefined and the badge always showed 'dev'.

Fix: re-declare all three ARGs after the second FROM and set runtime ENVs.
Add PUBLIC_BUILD_TIME (ISO timestamp from gitea.event.head_commit.timestamp)
injected via build-arg in release.yaml. Badge now shows e.g.:
  v2.3.9+abc1234 · 28 Mar 2026 14:30 UTC
2026-03-28 19:33:49 +05:00
Admin
428b57732e fix(ui): resolve avatar URL from MinIO; fall back to OAuth provider URL
Some checks failed
CI / Backend (push) Successful in 50s
CI / UI (push) Successful in 34s
Release / Test backend (push) Successful in 37s
Release / Check ui (push) Successful in 45s
Release / Docker / caddy (push) Successful in 35s
CI / Backend (pull_request) Successful in 49s
CI / UI (pull_request) Successful in 40s
Release / Docker / backend (push) Failing after 52s
Release / Docker / runner (push) Failing after 58s
Release / Docker / ui (push) Failing after 55s
Release / Gitea Release (push) Has been skipped
Add resolveAvatarUrl(userId, storedValue) helper that tries MinIO first,
then falls back to the stored HTTP URL for OAuth users (Google/GitHub)
who have never uploaded a custom avatar.

Add getUserById() to pocketbase helpers for batch avatar resolution in
comments. Update all 6 call sites to use the new helper.
2026-03-28 19:23:30 +05:00
Admin
61e77e3e28 ci: remove Docker builds from CI; keep vet/build/test/type-check only
All checks were successful
CI / Backend (push) Successful in 43s
CI / UI (push) Successful in 37s
CI / UI (pull_request) Successful in 35s
CI / Backend (pull_request) Successful in 1m9s
Docker image builds belong to the release workflow (tag-triggered).
CI now runs: go vet, go build (backend/runner/healthcheck), go test,
svelte-check, and UI vite build — fast feedback without Docker overhead.
Also triggers on all branches, not just main/master.
2026-03-28 19:08:17 +05:00
Admin
b363c151a5 fix(ui): fix catalogue filters in Svelte 5; improve build badge visibility
Some checks failed
Release / Test backend (push) Successful in 40s
Release / Check ui (push) Successful in 49s
Release / Docker / caddy (push) Failing after 1m0s
CI / Test backend (pull_request) Successful in 40s
CI / Docker / caddy (pull_request) Failing after 21s
CI / Check ui (pull_request) Successful in 48s
Release / Docker / ui (push) Successful in 2m49s
Release / Docker / runner (push) Successful in 3m10s
Release / Docker / backend (push) Successful in 3m47s
Release / Gitea Release (push) Has been skipped
CI / Docker / ui (pull_request) Successful in 1m31s
CI / Docker / backend (pull_request) Successful in 3m13s
CI / Docker / runner (pull_request) Successful in 3m59s
- Catalogue filter selects: replace value= initializer with bind:value +
  onchange goto() so filters navigate immediately on change (no Apply button)
- Add selected= on each <option> for correct DOM initialisation in Svelte 5
- Build badge: give distinct bg-zinc-800 pill, visible zinc-300/400/500 text
  instead of zinc-700/zinc-800 which blended into the footer background
2026-03-28 19:01:58 +05:00
Admin
aef9e04419 fix(runner): harden catalogue scrape against 429s; disable sourcemap upload
Some checks failed
Release / Test backend (push) Successful in 29s
Release / Check ui (push) Successful in 43s
CI / Test backend (pull_request) Successful in 28s
Release / Docker / caddy (push) Successful in 1m8s
CI / Docker / caddy (pull_request) Failing after 39s
CI / Check ui (pull_request) Successful in 55s
Release / Docker / runner (push) Successful in 2m22s
Release / Docker / ui (push) Successful in 2m20s
CI / Docker / backend (pull_request) Successful in 2m32s
CI / Docker / ui (pull_request) Successful in 1m28s
CI / Docker / runner (pull_request) Successful in 2m25s
Release / Docker / backend (push) Successful in 2m33s
Release / Gitea Release (push) Failing after 2s
- scraper.go: ScrapeCatalogue now uses retryGet (9 attempts, 10s base) +
  500–1500ms inter-page jitter instead of bare GetContent. ScrapeMetadata
  also switched to retryGet so a single 429 on a book page is retried rather
  than aborting the whole refresh.
- catalogue_refresh.go: per-book delay is now configurable
  (RUNNER_CATALOGUE_REQUEST_DELAY, default 2s) + up to 50% random jitter
  applied before every metadata fetch. Only metadata is scraped here —
  chapters are fetched on-demand, not during catalogue refresh. Progress
  logged every 50 books instead of 100.
- config.go / runner.go / main.go: add CatalogueRequestDelay field wired
  from RUNNER_CATALOGUE_REQUEST_DELAY env var.
- release.yaml: comment out upload-sourcemaps job and remove it from the
  release needs; GlitchTip auth token needs refreshing after DB wipe.
2026-03-28 16:28:36 +05:00
Admin
58e78cd34d fix(infra): make MEILI_URL configurable; rename Discover → Catalogue in nav
Some checks failed
Release / Test backend (push) Failing after 11s
Release / Docker / caddy (push) Failing after 11s
Release / Check ui (push) Failing after 11s
Release / Docker / backend (push) Has been skipped
Release / Docker / runner (push) Has been skipped
Release / Upload source maps (push) Has been skipped
Release / Docker / ui (push) Has been skipped
Release / Gitea Release (push) Has been skipped
CI / Check ui (pull_request) Successful in 49s
CI / Docker / ui (pull_request) Successful in 1m37s
CI / Docker / caddy (pull_request) Successful in 4m4s
CI / Test backend (pull_request) Successful in 4m29s
CI / Docker / runner (pull_request) Failing after 1m27s
CI / Docker / backend (pull_request) Successful in 1m41s
- docker-compose.yml: MEILI_URL in x-infra-env was hardcoded to the local
  meilisearch container, ignoring the Doppler MEILI_URL secret entirely.
  Changed to "${MEILI_URL:-http://meilisearch:7700}" so prod reads from
  https://search.libnovel.cc while local dev keeps the container default.
- ui/+layout.svelte: rename nav + footer label from "Discover" to "Catalogue"
  (desktop nav, mobile menu, footer — all three occurrences).
2026-03-28 16:15:04 +05:00
Admin
c5c167035d fix(tts): fix pocket-tts voices missing in UI and 500 on first TTS enqueue
Some checks failed
CI / Test backend (pull_request) Successful in 40s
CI / Check ui (pull_request) Successful in 45s
Release / Test backend (push) Successful in 40s
Release / Check ui (push) Successful in 41s
Release / Docker / caddy (push) Successful in 1m8s
CI / Docker / runner (pull_request) Failing after 35s
CI / Docker / caddy (pull_request) Successful in 3m39s
CI / Docker / ui (pull_request) Successful in 1m12s
CI / Docker / backend (pull_request) Successful in 3m24s
Release / Upload source maps (push) Failing after 43s
Release / Docker / runner (push) Successful in 2m49s
Release / Docker / ui (push) Successful in 2m51s
Release / Docker / backend (push) Failing after 7m14s
Release / Gitea Release (push) Has been skipped
- Add POCKET_TTS_URL env to backend service in docker-compose.yml so
  pocket-tts voices appear in the voice selector (Doppler secret existed
  but the env var was never passed to the container)
- Fix GetAudioTask PocketBase filter using %q (double-quotes) instead of
  single-quoted string, causing the duplicate-task guard to always miss
- Fix AudioPlayer double-POST: GET /api/presign/audio already enqueues
  TTS internally on 404; AudioPlayer now skips the redundant POST and
  polls directly, eliminating the 500 from the PB unique-key conflict
2026-03-28 16:02:46 +05:00
Admin
4a00d953bb feat(ui): show build version in footer; enable watchtower for caddy
Some checks failed
CI / Check ui (pull_request) Successful in 50s
CI / Docker / ui (pull_request) Failing after 38s
CI / Test backend (pull_request) Successful in 3m31s
CI / Docker / backend (pull_request) Failing after 11s
CI / Docker / runner (pull_request) Failing after 11s
CI / Docker / caddy (pull_request) Successful in 11m0s
- Footer now displays build version tag + short commit SHA (from
  PUBLIC_BUILD_VERSION / PUBLIC_BUILD_COMMIT env vars baked in at
  image build time); falls back to 'dev' in local builds
- docker-compose.yml: add watchtower label to caddy service so it
  auto-updates alongside backend/ui/runner on new image pushes
- homelab/docker-compose.yml: use locally-built kokoro-fastapi:latest
  image (consistent with actual homelab setup)
2026-03-28 15:42:22 +05:00
26 changed files with 507 additions and 331 deletions

View File

@@ -2,20 +2,14 @@ name: CI
on:
push:
branches: ["main", "master"]
paths:
- "backend/**"
- "ui/**"
- "caddy/**"
- "docker-compose.yml"
- ".gitea/workflows/ci.yaml"
pull_request:
branches: ["main", "master"]
paths:
- "backend/**"
- "ui/**"
- "caddy/**"
- "docker-compose.yml"
- ".gitea/workflows/ci.yaml"
concurrency:
@@ -23,10 +17,13 @@ concurrency:
cancel-in-progress: true
jobs:
# ── backend: vet & test ───────────────────────────────────────────────────────
test-backend:
name: Test backend
# ── Go: vet + build + test ────────────────────────────────────────────────
backend:
name: Backend
runs-on: ubuntu-latest
defaults:
run:
working-directory: backend
steps:
- uses: actions/checkout@v4
@@ -36,16 +33,23 @@ jobs:
cache-dependency-path: backend/go.sum
- name: go vet
working-directory: backend
run: go vet ./...
- name: Build backend
run: go build -o /dev/null ./cmd/backend
- name: Build runner
run: go build -o /dev/null ./cmd/runner
- name: Build healthcheck
run: go build -o /dev/null ./cmd/healthcheck
- name: Run tests
working-directory: backend
run: go test -short -race -count=1 -timeout=60s ./...
# ── ui: type-check & build ────────────────────────────────────────────────────
check-ui:
name: Check ui
# ── UI: type-check + build ────────────────────────────────────────────────
ui:
name: UI
runs-on: ubuntu-latest
defaults:
run:
@@ -67,57 +71,3 @@ jobs:
- name: Build
run: npm run build
# ── docker: validate Dockerfiles build (no push) ──────────────────────────────
docker-backend:
name: Docker / backend
runs-on: ubuntu-latest
needs: [test-backend]
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- name: Build
uses: docker/build-push-action@v6
with:
context: backend
target: backend
push: false
docker-runner:
name: Docker / runner
runs-on: ubuntu-latest
needs: [test-backend]
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- name: Build
uses: docker/build-push-action@v6
with:
context: backend
target: runner
push: false
docker-ui:
name: Docker / ui
runs-on: ubuntu-latest
needs: [check-ui]
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- name: Build
uses: docker/build-push-action@v6
with:
context: ui
push: false
docker-caddy:
name: Docker / caddy
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- name: Build
uses: docker/build-push-action@v6
with:
context: caddy
push: false

View File

@@ -136,52 +136,51 @@ jobs:
cache-to: type=inline
# ── ui: source map upload ─────────────────────────────────────────────────────
# Builds the UI with source maps and uploads them to GlitchTip so that error
# stack traces resolve to original .svelte/.ts file names and line numbers.
# Runs in parallel with docker-ui (both need check-ui to pass first).
upload-sourcemaps:
name: Upload source maps
runs-on: ubuntu-latest
needs: [check-ui]
defaults:
run:
working-directory: ui
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "22"
cache: npm
cache-dependency-path: ui/package-lock.json
- name: Install dependencies
run: npm ci
- name: Build with source maps
run: npm run build
- name: Download glitchtip-cli
run: |
curl -L "https://gitlab.com/glitchtip/glitchtip-cli/-/jobs/artifacts/v0.1.0/raw/artifacts/glitchtip-cli-linux-x86_64?job=build-linux-x86_64" \
-o /usr/local/bin/glitchtip-cli
chmod +x /usr/local/bin/glitchtip-cli
- name: Inject debug IDs into build artifacts
run: glitchtip-cli sourcemaps inject ./build
env:
SENTRY_URL: https://errors.libnovel.cc/
SENTRY_AUTH_TOKEN: ${{ secrets.GLITCHTIP_AUTH_TOKEN }}
SENTRY_ORG: libnovel
SENTRY_PROJECT: libnovel-ui
- name: Upload source maps to GlitchTip
run: glitchtip-cli sourcemaps upload ./build --release ${{ gitea.ref_name }}
env:
SENTRY_URL: https://errors.libnovel.cc/
SENTRY_AUTH_TOKEN: ${{ secrets.GLITCHTIP_AUTH_TOKEN }}
SENTRY_ORG: libnovel
SENTRY_PROJECT: libnovel-ui
# Commented out: GlitchTip project/auth token needs to be recreated after
# the GlitchTip DB wipe. Re-enable once GLITCHTIP_AUTH_TOKEN is updated.
# upload-sourcemaps:
# name: Upload source maps
# runs-on: ubuntu-latest
# needs: [check-ui]
# defaults:
# run:
# working-directory: ui
# steps:
# - uses: actions/checkout@v4
#
# - uses: actions/setup-node@v4
# with:
# node-version: "22"
# cache: npm
# cache-dependency-path: ui/package-lock.json
#
# - name: Install dependencies
# run: npm ci
#
# - name: Build with source maps
# run: npm run build
#
# - name: Download glitchtip-cli
# run: |
# curl -L "https://gitlab.com/glitchtip/glitchtip-cli/-/jobs/artifacts/v0.1.0/raw/artifacts/glitchtip-cli-linux-x86_64?job=build-linux-x86_64" \
# -o /usr/local/bin/glitchtip-cli
# chmod +x /usr/local/bin/glitchtip-cli
#
# - name: Inject debug IDs into build artifacts
# run: glitchtip-cli sourcemaps inject ./build
# env:
# SENTRY_URL: https://errors.libnovel.cc/
# SENTRY_AUTH_TOKEN: ${{ secrets.GLITCHTIP_AUTH_TOKEN }}
# SENTRY_ORG: libnovel
# SENTRY_PROJECT: libnovel-ui
#
# - name: Upload source maps to GlitchTip
# run: glitchtip-cli sourcemaps upload ./build --release ${{ gitea.ref_name }}
# env:
# SENTRY_URL: https://errors.libnovel.cc/
# SENTRY_AUTH_TOKEN: ${{ secrets.GLITCHTIP_AUTH_TOKEN }}
# SENTRY_ORG: libnovel
# SENTRY_PROJECT: libnovel-ui
# ── docker: ui ────────────────────────────────────────────────────────────────
docker-ui:
@@ -219,6 +218,7 @@ jobs:
build-args: |
BUILD_VERSION=${{ steps.meta.outputs.version }}
BUILD_COMMIT=${{ gitea.sha }}
BUILD_TIME=${{ gitea.event.head_commit.timestamp }}
cache-from: type=registry,ref=${{ secrets.DOCKER_USER }}/libnovel-ui:latest
cache-to: type=inline
@@ -261,7 +261,7 @@ jobs:
release:
name: Gitea Release
runs-on: ubuntu-latest
needs: [docker-backend, docker-runner, docker-ui, docker-caddy, upload-sourcemaps]
needs: [docker-backend, docker-runner, docker-ui, docker-caddy]
steps:
- uses: actions/checkout@v4
with:

View File

@@ -170,12 +170,31 @@
# ── SvelteKit UI (catch-all — includes all remaining /api/* routes) ───────
handle {
reverse_proxy ui:3000 {
}
# Active health check: Caddy polls /health every 5 s and marks the
# upstream down immediately when it fails. Combined with
# lb_try_duration this means Watchtower container replacements
# show the maintenance page within a few seconds instead of
# hanging or returning a raw connection error to the browser.
health_uri /health
health_interval 5s
health_timeout 2s
health_status 200
# If the upstream is down, fail fast (don't retry for longer than
# 3 s) and let Caddy's handle_errors 502/503 take over.
lb_try_duration 3s
}
}
# ── Caddy-level error pages ───────────────────────────────────────────────
# These fire when the upstream (backend or ui) is completely unreachable.
# SvelteKit's own +error.svelte handles application-level errors (404, 500).
handle_errors 404 {
root * /srv/errors
rewrite * /404.html
file_server
}
handle_errors 502 {
root * /srv/errors
rewrite * /502.html
file_server

View File

@@ -30,9 +30,14 @@ RUN --mount=type=cache,target=/root/go/pkg/mod \
-o /out/healthcheck ./cmd/healthcheck
# ── backend service ──────────────────────────────────────────────────────────
FROM gcr.io/distroless/static:nonroot AS backend
# Uses Alpine (not distroless) so ffmpeg is available for on-demand voice
# sample generation via pocket-tts (WAV→MP3 transcoding).
FROM alpine:3.21 AS backend
RUN apk add --no-cache ffmpeg ca-certificates && \
addgroup -S appgroup && adduser -S appuser -G appgroup
COPY --from=builder /out/healthcheck /healthcheck
COPY --from=builder /out/backend /backend
USER appuser
ENTRYPOINT ["/backend"]
# ── runner service ───────────────────────────────────────────────────────────

View File

@@ -152,6 +152,7 @@ func run() error {
OrchestratorWorkers: workers,
MetricsAddr: cfg.Runner.MetricsAddr,
CatalogueRefreshInterval: cfg.Runner.CatalogueRefreshInterval,
CatalogueRequestDelay: cfg.Runner.CatalogueRequestDelay,
SkipInitialCatalogueRefresh: cfg.Runner.SkipInitialCatalogueRefresh,
RedisAddr: cfg.Redis.Addr,
RedisPassword: cfg.Redis.Password,

View File

@@ -126,6 +126,11 @@ type Runner struct {
// is already indexed and a 24h walk would be wasteful.
// Controlled by RUNNER_SKIP_INITIAL_CATALOGUE_REFRESH=true.
SkipInitialCatalogueRefresh bool
// CatalogueRequestDelay is the base delay inserted between per-book metadata
// requests during a catalogue refresh. A random jitter of up to 50% is added
// on top. Defaults to 2s. Increase to reduce 429 pressure on novelfire.net.
// Controlled by RUNNER_CATALOGUE_REQUEST_DELAY (e.g. "3s", "500ms").
CatalogueRequestDelay time.Duration
}
// Config is the top-level configuration struct consumed by both binaries.
@@ -196,6 +201,7 @@ func Load() Config {
MetricsAddr: envOr("RUNNER_METRICS_ADDR", ":9091"),
CatalogueRefreshInterval: envDuration("RUNNER_CATALOGUE_REFRESH_INTERVAL", 0),
SkipInitialCatalogueRefresh: envBool("RUNNER_SKIP_INITIAL_CATALOGUE_REFRESH", false),
CatalogueRequestDelay: envDuration("RUNNER_CATALOGUE_REQUEST_DELAY", 2*time.Second),
},
Meilisearch: Meilisearch{

View File

@@ -13,6 +13,7 @@ import (
"errors"
"fmt"
"log/slog"
"math/rand"
"net/url"
"path"
"strconv"
@@ -55,6 +56,9 @@ func (s *Scraper) SourceName() string { return "novelfire.net" }
// ── CatalogueProvider ─────────────────────────────────────────────────────────
// ScrapeCatalogue streams all CatalogueEntry values across all catalogue pages.
// Each page fetch uses retryGet with 429-aware exponential backoff.
// A small inter-page delay (cataloguePageDelay) is inserted between requests to
// avoid hammering the server when paging through hundreds of catalogue pages.
func (s *Scraper) ScrapeCatalogue(ctx context.Context) (<-chan domain.CatalogueEntry, <-chan error) {
entries := make(chan domain.CatalogueEntry, 64)
errs := make(chan error, 16)
@@ -73,8 +77,18 @@ func (s *Scraper) ScrapeCatalogue(ctx context.Context) (<-chan domain.CatalogueE
default:
}
// Polite inter-page delay — skipped on the very first page.
if page > 1 {
jitter := time.Duration(500+rand.Intn(1000)) * time.Millisecond
select {
case <-ctx.Done():
return
case <-time.After(jitter):
}
}
s.log.Info("scraping catalogue page", "page", page, "url", pageURL)
raw, err := s.client.GetContent(ctx, pageURL)
raw, err := retryGet(ctx, s.log, s.client, pageURL, 9, 10*time.Second)
if err != nil {
errs <- fmt.Errorf("catalogue page %d: %w", page, err)
return
@@ -139,10 +153,11 @@ func (s *Scraper) ScrapeCatalogue(ctx context.Context) (<-chan domain.CatalogueE
// ── MetadataProvider ──────────────────────────────────────────────────────────
// ScrapeMetadata fetches and parses book metadata from the book's landing page.
// Uses retryGet with 429-aware exponential backoff (up to 9 attempts).
func (s *Scraper) ScrapeMetadata(ctx context.Context, bookURL string) (domain.BookMeta, error) {
s.log.Debug("metadata fetch starting", "url", bookURL)
raw, err := s.client.GetContent(ctx, bookURL)
raw, err := retryGet(ctx, s.log, s.client, bookURL, 9, 10*time.Second)
if err != nil {
return domain.BookMeta{}, fmt.Errorf("metadata fetch %s: %w", bookURL, err)
}

View File

@@ -6,17 +6,20 @@ package runner
//
// Design:
// - Runs on its own ticker (CatalogueRefreshInterval, default 24h) inside Run().
// - Also fires once on startup.
// - ScrapeCatalogue streams CatalogueEntry values over a channel — we iterate
// and call ScrapeMetadata for each entry.
// - Per-request random jitter (13s) prevents hammering novelfire.net.
// - Cover images are fetched from the URL embedded in BookMeta.Cover and
// stored in MinIO (browse bucket, key: covers/{slug}.jpg).
// - WriteMetadata + UpsertBook are called for every successfully scraped book.
// - Errors for individual books are logged and skipped; the loop continues.
// - The cover URL stored in BookMeta.Cover is rewritten to the internal proxy
// path (/api/cover/novelfire.net/{slug}) so the UI always fetches via the
// backend, which will serve from MinIO.
// - Also fires once on startup (unless SkipInitialCatalogueRefresh is set).
// - ScrapeCatalogue streams CatalogueEntry values over a channel — already has
// its own inter-page jitter + retryGet (see scraper.go).
// - Per-book: only metadata is scraped here (not chapters). Chapters are scraped
// on-demand when a user opens a book or via an explicit scrape task.
// - Between each metadata request a configurable base delay plus up to 50%
// random jitter is applied (CatalogueRequestDelay, default 2s). This keeps
// the request rate well below novelfire.net's rate limit even for ~15k books.
// - ScrapeMetadata itself uses retryGet with 429-aware exponential backoff
// (up to 9 attempts), so transient rate limits are handled gracefully.
// - Cover images are fetched and stored in MinIO on first sight; subsequent
// refreshes skip covers that already exist (CoverExists check).
// - Books already present in Meilisearch are skipped entirely (fast path).
// - Errors for individual books are logged and skipped; the loop never aborts.
import (
"context"
@@ -29,7 +32,7 @@ import (
// runCatalogueRefresh performs one full catalogue walk: scrapes metadata for
// every book on novelfire.net, downloads covers to MinIO, and upserts to
// Meilisearch. Errors for individual books are logged and skipped.
// Meilisearch. Individual book failures are logged and skipped.
func (r *Runner) runCatalogueRefresh(ctx context.Context) {
if r.deps.Novel == nil {
r.deps.Log.Warn("runner: catalogue refresh skipped — Novel scraper not configured")
@@ -40,8 +43,9 @@ func (r *Runner) runCatalogueRefresh(ctx context.Context) {
return
}
delay := r.cfg.CatalogueRequestDelay
log := r.deps.Log.With("op", "catalogue_refresh")
log.Info("runner: catalogue refresh starting")
log.Info("runner: catalogue refresh starting", "request_delay", delay)
entries, errCh := r.deps.Novel.ScrapeCatalogue(ctx)
@@ -51,26 +55,26 @@ func (r *Runner) runCatalogueRefresh(ctx context.Context) {
break
}
// Skip books already present in Meilisearch — they were indexed on a
// previous run. Re-indexing only happens when a scrape task is
// explicitly enqueued (e.g. via the admin UI or API).
// Fast path: skip books already indexed in Meilisearch.
if r.deps.SearchIndex.BookExists(ctx, entry.Slug) {
skipped++
continue
}
// Random jitter between books to avoid rate-limiting.
jitter := time.Duration(1000+rand.Intn(2000)) * time.Millisecond
// Polite delay between metadata requests: base + up to 50% jitter.
// This applies before every fetch so we never fire bursts.
jitter := time.Duration(rand.Int63n(int64(delay / 2)))
select {
case <-ctx.Done():
break
case <-time.After(jitter):
case <-time.After(delay + jitter):
}
// ScrapeMetadata internally retries on 429 with exponential back-off.
meta, err := r.deps.Novel.ScrapeMetadata(ctx, entry.URL)
if err != nil {
log.Warn("runner: catalogue refresh: metadata scrape failed",
"url", entry.URL, "err", err)
log.Warn("runner: catalogue refresh: metadata scrape failed — skipping book",
"slug", entry.Slug, "url", entry.URL, "err", err)
errCount++
continue
}
@@ -81,35 +85,32 @@ func (r *Runner) runCatalogueRefresh(ctx context.Context) {
// Persist to PocketBase.
if err := r.deps.BookWriter.WriteMetadata(ctx, meta); err != nil {
log.Warn("runner: catalogue refresh: WriteMetadata failed",
log.Warn("runner: catalogue refresh: WriteMetadata failed — skipping book",
"slug", meta.Slug, "err", err)
errCount++
continue
}
// Index in Meilisearch.
// Index in Meilisearch (non-fatal).
if err := r.deps.SearchIndex.UpsertBook(ctx, meta); err != nil {
log.Warn("runner: catalogue refresh: UpsertBook failed",
"slug", meta.Slug, "err", err)
// non-fatal — continue
}
// Download and store cover image in MinIO if we have a cover URL
// and a CoverStore is wired in.
// Download cover to MinIO if not already cached (non-fatal).
if r.deps.CoverStore != nil && originalCover != "" {
if !r.deps.CoverStore.CoverExists(ctx, meta.Slug) {
if err := r.downloadCover(ctx, meta.Slug, originalCover); err != nil {
log.Warn("runner: catalogue refresh: cover download failed",
"slug", meta.Slug, "url", originalCover, "err", err)
// non-fatal
}
}
}
ok++
if ok%100 == 0 {
if ok%50 == 0 {
log.Info("runner: catalogue refresh progress",
"scraped", ok, "errors", errCount)
"scraped", ok, "skipped", skipped, "errors", errCount)
}
}

View File

@@ -62,6 +62,10 @@ type Config struct {
// scrapes per-book metadata, downloads covers, and re-indexes everything in
// Meilisearch. Defaults to 24h (expensive — full catalogue walk).
CatalogueRefreshInterval time.Duration
// CatalogueRequestDelay is the base inter-request pause during a catalogue
// refresh metadata walk. Jitter of up to 50% is added on top.
// Defaults to 2s. Set via RUNNER_CATALOGUE_REQUEST_DELAY.
CatalogueRequestDelay time.Duration
// SkipInitialCatalogueRefresh suppresses the immediate catalogue walk that
// otherwise fires at startup. The periodic ticker (CatalogueRefreshInterval)
// still fires normally. Set RUNNER_SKIP_INITIAL_CATALOGUE_REFRESH=true for
@@ -145,6 +149,9 @@ func New(cfg Config, deps Dependencies) *Runner {
if cfg.CatalogueRefreshInterval <= 0 {
cfg.CatalogueRefreshInterval = 24 * time.Hour
}
if cfg.CatalogueRequestDelay <= 0 {
cfg.CatalogueRequestDelay = 2 * time.Second
}
if cfg.MetricsAddr == "" {
cfg.MetricsAddr = ":9091"
}

View File

@@ -706,7 +706,7 @@ func (s *Store) ListAudioTasks(ctx context.Context) ([]domain.AudioTask, error)
}
func (s *Store) GetAudioTask(ctx context.Context, cacheKey string) (domain.AudioTask, bool, error) {
filter := fmt.Sprintf(`cache_key=%q`, cacheKey)
filter := fmt.Sprintf(`cache_key='%s'`, cacheKey)
items, err := s.pb.listAll(ctx, "audio_jobs", filter, "-started")
if err != nil || len(items) == 0 {
return domain.AudioTask{}, false, err

51
caddy/errors/404.html Normal file
View File

@@ -0,0 +1,51 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>404 — Page Not Found</title>
<style>
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
body {
min-height: 100svh;
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
gap: 1rem;
background: #09090b;
color: #a1a1aa;
font-family: ui-sans-serif, system-ui, sans-serif;
padding: 2rem;
text-align: center;
}
.code {
font-size: clamp(4rem, 20vw, 8rem);
font-weight: 800;
color: #27272a;
line-height: 1;
letter-spacing: -0.04em;
}
h1 { font-size: 1.25rem; font-weight: 600; color: #e4e4e7; }
p { font-size: 0.9rem; max-width: 36ch; line-height: 1.6; }
a {
margin-top: 0.5rem;
display: inline-block;
padding: 0.6rem 1.4rem;
border-radius: 0.5rem;
background: #f59e0b;
color: #000;
font-weight: 600;
font-size: 0.875rem;
text-decoration: none;
}
a:hover { background: #d97706; }
</style>
</head>
<body>
<div class="code">404</div>
<h1>Page Not Found</h1>
<p>The page you're looking for doesn't exist or has been moved.</p>
<a href="/">Go home</a>
</body>
</html>

View File

@@ -15,7 +15,7 @@ x-infra-env: &infra-env
POCKETBASE_ADMIN_EMAIL: "${POCKETBASE_ADMIN_EMAIL}"
POCKETBASE_ADMIN_PASSWORD: "${POCKETBASE_ADMIN_PASSWORD}"
# Meilisearch
MEILI_URL: "http://meilisearch:7700"
MEILI_URL: "${MEILI_URL:-http://meilisearch:7700}"
MEILI_API_KEY: "${MEILI_MASTER_KEY}"
# Valkey
VALKEY_ADDR: "valkey:6379"
@@ -160,6 +160,7 @@ services:
LOG_LEVEL: "${LOG_LEVEL}"
KOKORO_URL: "${KOKORO_URL}"
KOKORO_VOICE: "${KOKORO_VOICE}"
POCKET_TTS_URL: "${POCKET_TTS_URL}"
GLITCHTIP_DSN: "${GLITCHTIP_DSN}"
OTEL_EXPORTER_OTLP_ENDPOINT: "${OTEL_EXPORTER_OTLP_ENDPOINT}"
OTEL_SERVICE_NAME: "backend"
@@ -223,6 +224,7 @@ services:
# Kokoro-FastAPI TTS endpoint
KOKORO_URL: "${KOKORO_URL}"
KOKORO_VOICE: "${KOKORO_VOICE}"
POCKET_TTS_URL: "${POCKET_TTS_URL}"
GLITCHTIP_DSN: "${GLITCHTIP_DSN}"
OTEL_EXPORTER_OTLP_ENDPOINT: "${OTEL_EXPORTER_OTLP_ENDPOINT}"
OTEL_SERVICE_NAME: "runner"
@@ -366,6 +368,8 @@ services:
build:
context: ./caddy
dockerfile: Dockerfile
labels:
com.centurylinklabs.watchtower.enable: "true"
restart: unless-stopped
depends_on:
backend:

View File

@@ -222,6 +222,10 @@ services:
EMAIL_SMTP_USERNAME: "${FIDER_SMTP_USER}"
EMAIL_SMTP_PASSWORD: "${FIDER_SMTP_PASSWORD}"
EMAIL_SMTP_ENABLE_STARTTLS: "false"
OAUTH_GOOGLE_CLIENTID: "${OAUTH_GOOGLE_CLIENTID}"
OAUTH_GOOGLE_SECRET: "${OAUTH_GOOGLE_SECRET}"
OAUTH_GITHUB_CLIENTID: "${OAUTH_GITHUB_CLIENTID}"
OAUTH_GITHUB_SECRET: "${OAUTH_GITHUB_SECRET}"
# ── Dozzle ──────────────────────────────────────────────────────────────────
# Watches both homelab and prod containers.
@@ -394,7 +398,7 @@ services:
# Voices match existing IDs: af_bella, af_sky, af_heart, etc.
# The runner reaches it at http://kokoro-fastapi:8880 via the Docker network.
kokoro-fastapi:
image: ghcr.io/remsky/kokoro-fastapi-gpu:latest
image: kokoro-fastapi:latest
restart: unless-stopped
deploy:
resources:

View File

@@ -14,10 +14,12 @@ COPY . .
# Build-time version info — injected by docker-compose or CI via --build-arg.
ARG BUILD_VERSION=dev
ARG BUILD_COMMIT=unknown
ARG BUILD_TIME=unknown
# Expose as PUBLIC_ env vars so SvelteKit's $env/dynamic/public can read them.
ENV PUBLIC_BUILD_VERSION=$BUILD_VERSION
ENV PUBLIC_BUILD_COMMIT=$BUILD_COMMIT
ENV PUBLIC_BUILD_TIME=$BUILD_TIME
RUN npm run build
@@ -40,5 +42,16 @@ ENV NODE_ENV=production
ENV PORT=3000
ENV HOST=0.0.0.0
# Carry build-time metadata into the runtime image so the UI footer can
# display the version, commit SHA, and build timestamp.
# These must be re-declared after the second FROM — ARG values do not
# cross stage boundaries, but ENV values set here persist at runtime.
ARG BUILD_VERSION=dev
ARG BUILD_COMMIT=unknown
ARG BUILD_TIME=unknown
ENV PUBLIC_BUILD_VERSION=$BUILD_VERSION
ENV PUBLIC_BUILD_COMMIT=$BUILD_COMMIT
ENV PUBLIC_BUILD_TIME=$BUILD_TIME
EXPOSE $PORT
CMD ["node", "build"]

View File

@@ -343,23 +343,28 @@
// ── API helpers ────────────────────────────────────────────────────────────
type PresignResult =
| { ready: true; url: string }
| { ready: false; enqueued: boolean }; // enqueued=true → presign already POSTed
async function tryPresign(
targetSlug: string,
targetChapter: number,
targetVoice: string
): Promise<string | null> {
): Promise<PresignResult> {
const params = new URLSearchParams({
slug: targetSlug,
n: String(targetChapter),
voice: targetVoice
});
const res = await fetch(`/api/presign/audio?${params}`);
// 202: TTS was just enqueued by the presign endpoint — audio not ready yet.
// 202: presign endpoint already triggered TTS — skip the POST, go straight to polling.
// 404: legacy fallback (should no longer occur after endpoint change).
if (res.status === 202 || res.status === 404) return null;
if (res.status === 202) return { ready: false, enqueued: true };
if (res.status === 404) return { ready: false, enqueued: false };
if (!res.ok) throw new Error(`presign HTTP ${res.status}`);
const data = (await res.json()) as { url: string };
return data.url;
return { ready: true, url: data.url };
}
type AudioStatusResponse =
@@ -421,50 +426,52 @@
try {
// Fast path: already generated
const url = await tryPresign(slug, nextChapter, voice);
if (url) {
const presignResult = await tryPresign(slug, nextChapter, voice);
if (presignResult.ready) {
stopNextProgress();
audioStore.nextProgress = 100;
audioStore.nextAudioUrl = url;
audioStore.nextAudioUrl = presignResult.url;
audioStore.nextStatus = 'prefetched';
return;
}
// Slow path: trigger Kokoro generation (non-blocking POST), then poll.
const res = await fetch(`/api/audio/${slug}/${nextChapter}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ voice })
});
if (!res.ok) throw new Error(`Prefetch generation failed: HTTP ${res.status}`);
// Slow path: trigger generation (or skip POST if presign already enqueued).
if (!presignResult.enqueued) {
const res = await fetch(`/api/audio/${slug}/${nextChapter}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ voice })
});
if (!res.ok) throw new Error(`Prefetch generation failed: HTTP ${res.status}`);
// Whether the server returned 200 (already cached) or 202 (enqueued),
// always presign — the status endpoint no longer returns a proxy URL.
if (res.status === 200) {
// Body is { status: 'done' } — audio confirmed in MinIO. Presign it.
await res.body?.cancel();
}
// else 202: generation enqueued — fall through to poll.
if (res.status !== 200) {
// 202: poll until done.
const final = await pollAudioStatus(slug, nextChapter, voice);
stopNextProgress();
audioStore.nextProgress = 100;
if (final.status === 'failed') {
throw new Error(`Prefetch failed: ${(final as { error?: string }).error ?? 'unknown'}`);
if (res.status === 200) {
// Body is { status: 'done' } — audio confirmed in MinIO. Presign it.
await res.body?.cancel();
stopNextProgress();
audioStore.nextProgress = 100;
const doneUrl = await tryPresign(slug, nextChapter, voice);
if (!doneUrl.ready) throw new Error('Prefetch: audio done but presign returned 404');
audioStore.nextAudioUrl = doneUrl.url;
audioStore.nextStatus = 'prefetched';
return;
}
} else {
stopNextProgress();
audioStore.nextProgress = 100;
// 202: generation enqueued — fall through to poll.
}
// Poll until done (covers both: presign-enqueued and POST-enqueued paths).
const final = await pollAudioStatus(slug, nextChapter, voice);
stopNextProgress();
audioStore.nextProgress = 100;
if (final.status === 'failed') {
throw new Error(`Prefetch failed: ${(final as { error?: string }).error ?? 'unknown'}`);
}
// Audio is ready in MinIO — get a direct presigned URL.
const doneUrl = await tryPresign(slug, nextChapter, voice);
if (!doneUrl) throw new Error('Prefetch: audio done but presign returned 404');
if (!doneUrl.ready) throw new Error('Prefetch: audio done but presign returned 404');
audioStore.nextAudioUrl = doneUrl;
audioStore.nextAudioUrl = doneUrl.url;
audioStore.nextStatus = 'prefetched';
} catch {
stopNextProgress();
@@ -532,9 +539,9 @@
}
// Fast path B: audio already in MinIO (presign check).
const url = await tryPresign(slug, chapter, voice);
if (url) {
audioStore.audioUrl = url;
const presignResult = await tryPresign(slug, chapter, voice);
if (presignResult.ready) {
audioStore.audioUrl = presignResult.url;
audioStore.status = 'ready';
// Restore last saved position after the audio element loads
restoreSavedAudioTime();
@@ -547,33 +554,44 @@
audioStore.status = 'generating';
startProgress();
const res = await fetch(`/api/audio/${slug}/${chapter}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ voice })
});
if (!res.ok) throw new Error(`Generation failed: HTTP ${res.status}`);
// presignResult.enqueued=true means /api/presign/audio already POSTed on our
// behalf — skip the duplicate POST and go straight to polling.
if (!presignResult.enqueued) {
const res = await fetch(`/api/audio/${slug}/${chapter}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ voice })
});
if (!res.ok) throw new Error(`Generation failed: HTTP ${res.status}`);
if (res.status !== 200) {
// 202: generation enqueued — poll until done.
const final = await pollAudioStatus(slug, chapter, voice);
if (final.status === 'failed') {
throw new Error(
`Generation failed: ${(final as { error?: string }).error ?? 'unknown error'}`
);
if (res.status === 200) {
// Already cached — body is { status: 'done' }, no url needed.
await res.body?.cancel();
await finishProgress();
const doneUrl = await tryPresign(slug, chapter, voice);
if (!doneUrl.ready) throw new Error('Audio generated but presign returned 404');
audioStore.audioUrl = doneUrl.url;
audioStore.status = 'ready';
maybeStartPrefetch();
return;
}
} else {
// 200: already cached — body is { status: 'done' }, no url needed.
await res.body?.cancel();
// 202: fall through to polling below.
}
// Poll until the runner finishes generating.
const final = await pollAudioStatus(slug, chapter, voice);
if (final.status === 'failed') {
throw new Error(
`Generation failed: ${(final as { error?: string }).error ?? 'unknown error'}`
);
}
await finishProgress();
// Audio is ready in MinIO — always use a presigned URL for direct playback.
const doneUrl = await tryPresign(slug, chapter, voice);
if (!doneUrl) throw new Error('Audio generated but presign returned 404');
audioStore.audioUrl = doneUrl;
if (!doneUrl.ready) throw new Error('Audio generated but presign returned 404');
audioStore.audioUrl = doneUrl.url;
audioStore.status = 'ready';
// Don't restore time for freshly generated audio — position is 0
// Immediately start pre-generating the next chapter in background.

View File

@@ -40,8 +40,8 @@ export async function presignAvatarUploadUrl(userId: string, mimeType: string):
}
/**
* Returns a presigned GET URL for a user's avatar, rewritten to the public URL.
* Returns null if no avatar exists.
* Returns a presigned GET URL for a user's avatar from MinIO.
* Returns null if no avatar object exists in MinIO for this user.
*/
export async function presignAvatarUrl(userId: string): Promise<string | null> {
const res = await backendFetch(`/api/presign/avatar/${encodeURIComponent(userId)}`);
@@ -54,6 +54,42 @@ export async function presignAvatarUrl(userId: string): Promise<string | null> {
return data.url ? rewriteHost(data.url) : null;
}
/**
* Resolves the best available avatar URL for a user.
*
* Priority:
* 1. MinIO — if the user has uploaded a custom avatar it will be found here
* (presigned, short-lived GET URL).
* 2. OAuth provider URL — stored in avatar_url when the account was created
* via Google / GitHub OAuth (e.g. https://lh3.googleusercontent.com/...).
* Returned as-is; the browser fetches it directly.
*
* Pass the raw `avatar_url` field from the PocketBase record as `storedValue`
* so this function can distinguish between a MinIO key and a remote URL without
* an extra DB round-trip.
*
* Returns null when neither source yields an avatar.
*/
export async function resolveAvatarUrl(
userId: string,
storedValue: string | null | undefined
): Promise<string | null> {
// 1. Try MinIO first (custom upload takes priority over OAuth picture).
try {
const minioUrl = await presignAvatarUrl(userId);
if (minioUrl) return minioUrl;
} catch {
// MinIO unavailable — fall through to OAuth fallback.
}
// 2. Fall back to OAuth-provided picture URL if it looks like a remote URL.
if (storedValue && storedValue.startsWith('http')) {
return storedValue;
}
return null;
}
/**
* Rewrites the MinIO host in a presigned URL to the public-facing URL.
*

View File

@@ -541,6 +541,19 @@ export async function getUserByUsername(username: string): Promise<User | null>
return listOne<User>('app_users', `username="${username.replace(/"/g, '\\"')}"`);
}
/**
* Look up a user by their PocketBase record ID. Returns null if not found.
*/
export async function getUserById(id: string): Promise<User | null> {
const token = await getToken();
const res = await fetch(`${PB_URL}/api/collections/app_users/records/${encodeURIComponent(id)}`, {
headers: { Authorization: `Bearer ${token}` }
});
if (res.status === 404) return null;
if (!res.ok) return null;
return res.json() as Promise<User>;
}
/**
* Look up a user by email. Returns null if not found.
*/

View File

@@ -244,7 +244,7 @@
href="/catalogue"
class="hidden sm:block text-sm transition-colors {page.url.pathname.startsWith('/catalogue') ? 'text-zinc-100 font-medium' : 'text-zinc-400 hover:text-zinc-100'}"
>
Discover
Catalogue
</a>
<a
href="https://feedback.libnovel.cc"
@@ -257,20 +257,14 @@
<div class="ml-auto flex items-center gap-4">
<!-- Desktop: admin + profile + sign out (hidden on mobile) -->
{#if data.user?.role === 'admin'}
<a
href="/admin/scrape"
class="hidden sm:block text-sm transition-colors {page.url.pathname.startsWith('/admin/scrape') ? 'text-zinc-100 font-medium' : 'text-zinc-400 hover:text-zinc-100'}"
>
Scrape
</a>
<a
href="/admin/audio"
class="hidden sm:block text-sm transition-colors {page.url.pathname.startsWith('/admin/audio') ? 'text-zinc-100 font-medium' : 'text-zinc-400 hover:text-zinc-100'}"
>
Audio
</a>
{/if}
{#if data.user?.role === 'admin'}
<a
href="/admin/scrape"
class="hidden sm:block text-sm transition-colors {page.url.pathname.startsWith('/admin') ? 'text-zinc-100 font-medium' : 'text-zinc-400 hover:text-zinc-100'}"
>
Admin
</a>
{/if}
<a
href="/profile"
class="hidden sm:block text-sm transition-colors {page.url.pathname === '/profile' ? 'text-zinc-100 font-medium' : 'text-zinc-400 hover:text-zinc-100'}"
@@ -332,7 +326,7 @@
onclick={() => (menuOpen = false)}
class="px-3 py-2.5 rounded-lg text-sm font-medium transition-colors {page.url.pathname.startsWith('/catalogue') ? 'bg-zinc-800 text-zinc-100' : 'text-zinc-400 hover:bg-zinc-800 hover:text-zinc-100'}"
>
Discover
Catalogue
</a>
<a
href="https://feedback.libnovel.cc"
@@ -350,31 +344,17 @@
>
Profile <span class="text-zinc-500 font-normal">({data.user.username})</span>
</a>
{#if data.user?.role === 'admin'}
<div class="my-1 border-t border-zinc-700/60"></div>
<p class="px-3 pt-1 pb-0.5 text-xs text-zinc-600 uppercase tracking-widest">Admin</p>
<a
href="/admin/scrape"
onclick={() => (menuOpen = false)}
class="px-3 py-2.5 rounded-lg text-sm font-medium transition-colors {page.url.pathname.startsWith('/admin/scrape') ? 'bg-zinc-800 text-zinc-100' : 'text-zinc-400 hover:bg-zinc-800 hover:text-zinc-100'}"
>
Scrape tasks
</a>
<a
href="/admin/audio"
onclick={() => (menuOpen = false)}
class="px-3 py-2.5 rounded-lg text-sm font-medium transition-colors {page.url.pathname === '/admin/audio' ? 'bg-zinc-800 text-zinc-100' : 'text-zinc-400 hover:bg-zinc-800 hover:text-zinc-100'}"
>
Audio cache
</a>
<a
href="/admin/audio-jobs"
onclick={() => (menuOpen = false)}
class="px-3 py-2.5 rounded-lg text-sm font-medium transition-colors {page.url.pathname.startsWith('/admin/audio-jobs') ? 'bg-zinc-800 text-zinc-100' : 'text-zinc-400 hover:bg-zinc-800 hover:text-zinc-100'}"
>
Audio jobs
</a>
{/if}
{#if data.user?.role === 'admin'}
<div class="my-1 border-t border-zinc-700/60"></div>
<p class="px-3 pt-1 pb-0.5 text-xs text-zinc-600 uppercase tracking-widest">Admin</p>
<a
href="/admin/scrape"
onclick={() => (menuOpen = false)}
class="px-3 py-2.5 rounded-lg text-sm font-medium transition-colors {page.url.pathname.startsWith('/admin') ? 'bg-zinc-800 text-zinc-100' : 'text-zinc-400 hover:bg-zinc-800 hover:text-zinc-100'}"
>
Admin panel
</a>
{/if}
<div class="my-1 border-t border-zinc-700/60"></div>
<form method="POST" action="/logout">
<Button
@@ -396,16 +376,16 @@
</main>
<footer class="border-t border-zinc-800 mt-auto">
<div class="max-w-6xl mx-auto px-4 py-6 flex flex-col items-center gap-4 text-xs text-zinc-600">
<div class="max-w-6xl mx-auto px-4 py-6 flex flex-col items-center gap-4 text-xs text-zinc-500">
<!-- Top row: site links -->
<nav class="flex flex-wrap items-center justify-center gap-x-5 gap-y-2">
<a href="/books" class="hover:text-zinc-400 transition-colors">Library</a>
<a href="/catalogue" class="hover:text-zinc-400 transition-colors">Discover</a>
<a href="/books" class="hover:text-zinc-300 transition-colors">Library</a>
<a href="/catalogue" class="hover:text-zinc-300 transition-colors">Catalogue</a>
<a
href="https://feedback.libnovel.cc"
target="_blank"
rel="noopener noreferrer"
class="hover:text-zinc-400 transition-colors flex items-center gap-1"
class="hover:text-zinc-300 transition-colors flex items-center gap-1"
>
Feedback
<svg class="w-3 h-3" fill="none" stroke="currentColor" viewBox="0 0 24 24">
@@ -417,7 +397,7 @@
href="https://novelfire.net"
target="_blank"
rel="noopener noreferrer"
class="hover:text-zinc-400 transition-colors flex items-center gap-1"
class="hover:text-zinc-300 transition-colors flex items-center gap-1"
>
novelfire.net
<svg class="w-3 h-3" fill="none" stroke="currentColor" viewBox="0 0 24 24">
@@ -426,16 +406,35 @@
</svg>
</a>
</nav>
<!-- Bottom row: legal links + copyright -->
<div class="flex flex-wrap items-center justify-center gap-x-5 gap-y-2 text-zinc-700">
<a href="/disclaimer" class="hover:text-zinc-500 transition-colors">Disclaimer</a>
<a href="/privacy" class="hover:text-zinc-500 transition-colors">Privacy</a>
<a href="/dmca" class="hover:text-zinc-500 transition-colors">DMCA</a>
<span>&copy; {new Date().getFullYear()} libnovel</span>
{#if env.PUBLIC_BUILD_VERSION && env.PUBLIC_BUILD_VERSION !== 'dev'}
<span class="text-zinc-800">{env.PUBLIC_BUILD_VERSION}+{env.PUBLIC_BUILD_COMMIT?.slice(0, 7)}</span>
<!-- Bottom row: legal links + copyright -->
<div class="flex flex-wrap items-center justify-center gap-x-5 gap-y-2 text-zinc-500">
<a href="/disclaimer" class="hover:text-zinc-300 transition-colors">Disclaimer</a>
<a href="/privacy" class="hover:text-zinc-300 transition-colors">Privacy</a>
<a href="/dmca" class="hover:text-zinc-300 transition-colors">DMCA</a>
<span>&copy; {new Date().getFullYear()} libnovel</span>
</div>
<!-- Build version / commit SHA / build time -->
{#snippet buildTime()}
{#if env.PUBLIC_BUILD_TIME && env.PUBLIC_BUILD_TIME !== 'unknown'}
{@const d = new Date(env.PUBLIC_BUILD_TIME)}
<span class="text-zinc-500" title="Build time">
· {d.toUTCString().replace(' GMT', ' UTC').replace(/:\d\d /, ' ')}
</span>
{/if}
{/snippet}
<div class="text-xs tabular-nums font-mono px-2 py-0.5 rounded bg-zinc-800 border border-zinc-700">
{#if env.PUBLIC_BUILD_VERSION && env.PUBLIC_BUILD_VERSION !== 'dev'}
<span class="text-zinc-300" title="Build version">{env.PUBLIC_BUILD_VERSION}</span>
{#if env.PUBLIC_BUILD_COMMIT && env.PUBLIC_BUILD_COMMIT !== 'unknown'}
<span class="text-zinc-500 select-all" title="Commit SHA"
>+{env.PUBLIC_BUILD_COMMIT.slice(0, 7)}</span
>
{/if}
</div>
{@render buildTime()}
{:else}
<span class="text-zinc-400">dev</span>
{/if}
</div>
</div>
</footer>
</div>

View File

@@ -1,12 +1,12 @@
<script lang="ts">
import { page } from '$app/state';
const adminTabs = [
const internalLinks = [
{ href: '/admin/scrape', label: 'Scrape' },
{ href: '/admin/audio', label: 'Audio' }
];
const toolTabs = [
const externalLinks = [
{ href: 'https://feedback.libnovel.cc', label: 'Feedback' },
{ href: 'https://errors.libnovel.cc', label: 'Errors' },
{ href: 'https://analytics.libnovel.cc', label: 'Analytics' },
@@ -21,36 +21,51 @@
let { children }: Props = $props();
</script>
<!-- Admin nav: internal pages + external tools -->
<div class="mb-6 flex flex-wrap items-center gap-3">
<!-- Internal admin pages -->
<div class="flex gap-1 bg-zinc-800 rounded-lg p-1 border border-zinc-700">
{#each adminTabs as tab}
<a
href={tab.href}
class="px-4 py-1.5 rounded-md text-sm font-medium transition-colors
{page.url.pathname.startsWith(tab.href)
? 'bg-zinc-700 text-zinc-100'
: 'text-zinc-400 hover:text-zinc-200'}"
>
{tab.label}
</a>
{/each}
</div>
<div class="flex min-h-[calc(100vh-4rem)] gap-0">
<!-- Sidebar -->
<aside class="w-48 shrink-0 border-r border-zinc-800 px-3 py-6 flex flex-col gap-6">
<!-- Internal pages -->
<div>
<p class="px-2 mb-2 text-xs font-semibold text-zinc-600 uppercase tracking-widest">Pages</p>
<nav class="flex flex-col gap-0.5">
{#each internalLinks as link}
<a
href={link.href}
class="px-2 py-1.5 rounded-md text-sm font-medium transition-colors
{page.url.pathname.startsWith(link.href)
? 'bg-zinc-800 text-zinc-100'
: 'text-zinc-400 hover:bg-zinc-800/60 hover:text-zinc-200'}"
>
{link.label}
</a>
{/each}
</nav>
</div>
<!-- External tools (open in new tab) -->
<div class="flex gap-1 bg-zinc-800 rounded-lg p-1 border border-zinc-700">
{#each toolTabs as tool}
<a
href={tool.href}
target="_blank"
rel="noopener noreferrer"
class="px-4 py-1.5 rounded-md text-sm font-medium text-zinc-400 hover:text-zinc-200 transition-colors"
>
{tool.label}
</a>
{/each}
</div>
<!-- External tools -->
<div>
<p class="px-2 mb-2 text-xs font-semibold text-zinc-600 uppercase tracking-widest">Tools</p>
<nav class="flex flex-col gap-0.5">
{#each externalLinks as link}
<a
href={link.href}
target="_blank"
rel="noopener noreferrer"
class="px-2 py-1.5 rounded-md text-sm font-medium text-zinc-400 hover:bg-zinc-800/60 hover:text-zinc-200 transition-colors flex items-center justify-between"
>
{link.label}
<svg class="w-3 h-3 shrink-0 opacity-50" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2"
d="M10 6H6a2 2 0 00-2 2v10a2 2 0 002 2h10a2 2 0 002-2v-4M14 4h6m0 0v6m0-6L10 14" />
</svg>
</a>
{/each}
</nav>
</div>
</aside>
<!-- Main content -->
<main class="flex-1 min-w-0 px-8 py-6">
{@render children?.()}
</main>
</div>
{@render children?.()}

View File

@@ -1,6 +1,7 @@
import { json, error } from '@sveltejs/kit';
import type { RequestHandler } from './$types';
import { getUserByUsername } from '$lib/server/pocketbase';
import { resolveAvatarUrl } from '$lib/server/minio';
/**
* GET /api/auth/me
@@ -13,10 +14,11 @@ export const GET: RequestHandler = async ({ locals }) => {
}
// Fetch full record from PocketBase to get avatar_url
const record = await getUserByUsername(locals.user.username).catch(() => null);
const avatarUrl = await resolveAvatarUrl(locals.user.id, record?.avatar_url).catch(() => null);
return json({
id: locals.user.id,
username: locals.user.username,
role: locals.user.role,
avatar_url: record?.avatar_url ?? null
avatar_url: avatarUrl
});
};

View File

@@ -5,9 +5,10 @@ import {
listReplies,
createComment,
getMyVotes,
getUserById,
type CommentSort
} from '$lib/server/pocketbase';
import { presignAvatarUrl } from '$lib/server/minio';
import { resolveAvatarUrl } from '$lib/server/minio';
import { log } from '$lib/server/logger';
/**
@@ -38,13 +39,15 @@ export const GET: RequestHandler = async ({ params, url, locals }) => {
replies: repliesPerComment[i]
}));
// Batch-resolve avatar presign URLs for all unique user_ids
// Batch-resolve avatar URLs for all unique user_ids
// MinIO first (custom upload), fall back to OAuth provider picture.
const allComments = [...topLevel, ...allReplies];
const uniqueUserIds = [...new Set(allComments.map((c) => c.user_id).filter(Boolean))];
const avatarEntries = await Promise.all(
uniqueUserIds.map(async (userId) => {
try {
const url = await presignAvatarUrl(userId);
const user = await getUserById(userId);
const url = await resolveAvatarUrl(userId, user?.avatar_url);
return [userId, url] as [string, string | null];
} catch {
return [userId, null] as [string, null];

View File

@@ -1,6 +1,6 @@
import { json, error } from '@sveltejs/kit';
import type { RequestHandler } from './$types';
import { presignAvatarUrl } from '$lib/server/minio';
import { presignAvatarUrl, resolveAvatarUrl } from '$lib/server/minio';
import { updateUserAvatarUrl, getUserByUsername } from '$lib/server/pocketbase';
import { backendFetch } from '$lib/server/scraper';
@@ -63,10 +63,6 @@ export const GET: RequestHandler = async ({ locals }) => {
if (!locals.user) error(401, 'Not authenticated');
const record = await getUserByUsername(locals.user.username).catch(() => null);
if (!record?.avatar_url) {
return json({ avatar_url: null });
}
const avatarUrl = await presignAvatarUrl(locals.user.id);
const avatarUrl = await resolveAvatarUrl(locals.user.id, record?.avatar_url).catch(() => null);
return json({ avatar_url: avatarUrl });
};

View File

@@ -1,7 +1,7 @@
import { json, error } from '@sveltejs/kit';
import type { RequestHandler } from './$types';
import { getPublicProfile, getSubscription } from '$lib/server/pocketbase';
import { presignAvatarUrl } from '$lib/server/minio';
import { resolveAvatarUrl } from '$lib/server/minio';
import { log } from '$lib/server/logger';
/**
@@ -15,11 +15,9 @@ export const GET: RequestHandler = async ({ params, locals }) => {
const profile = await getPublicProfile(username);
if (!profile) error(404, `User "${username}" not found`);
// Resolve avatar presigned URL if set
// Resolve avatar — MinIO first, fall back to OAuth provider picture
let avatarUrl: string | null = null;
if (profile.avatar_url) {
avatarUrl = await presignAvatarUrl(profile.id).catch(() => null);
}
avatarUrl = await resolveAvatarUrl(profile.id, profile.avatar_url).catch(() => null);
// Is the current logged-in user subscribed?
let isSubscribed = false;

View File

@@ -1,5 +1,6 @@
<script lang="ts">
import { enhance } from '$app/forms';
import { goto } from '$app/navigation';
import { navigating } from '$app/state';
import { untrack } from 'svelte';
import type { PageData, ActionData } from './$types';
@@ -7,6 +8,29 @@
let { data, form }: { data: PageData; form: ActionData } = $props();
// ── Local filter state (mirrors URL params) ──────────────────────────────
// These are separate from data.* so we can bind them to selects and keep
// the DOM in sync. They sync back from data whenever a navigation completes.
let filterSort = $state(untrack(() => data.sort));
let filterGenre = $state(untrack(() => data.genre));
let filterStatus = $state(untrack(() => data.status));
// Keep local state in sync whenever SvelteKit re-runs the load (URL changed).
$effect(() => {
filterSort = data.sort;
filterGenre = data.genre;
filterStatus = data.status;
});
function applyFilters() {
const params = new URLSearchParams();
params.set('sort', filterSort);
params.set('genre', filterGenre);
params.set('status', filterStatus);
params.set('page', '1');
goto(`/catalogue?${params.toString()}`);
}
// Track which novel card is currently being navigated to
let loadingSlug = $state<string | null>(null);
@@ -389,11 +413,11 @@
<select
id="filter-sort"
name="sort"
value={data.sort}
bind:value={filterSort}
class="bg-zinc-900 border border-zinc-700 text-zinc-200 text-sm rounded px-3 py-2 focus:outline-none focus:border-amber-400 w-full"
>
{#each sorts as s}
<option value={s.value}>{s.label}</option>
<option value={s.value} selected={s.value === filterSort}>{s.label}</option>
{/each}
</select>
</div>
@@ -403,12 +427,12 @@
<select
id="filter-genre"
name="genre"
value={data.genre}
bind:value={filterGenre}
disabled={isRankView}
class="bg-zinc-900 border border-zinc-700 text-zinc-200 text-sm rounded px-3 py-2 focus:outline-none focus:border-amber-400 disabled:opacity-40 disabled:cursor-not-allowed w-full"
>
{#each genres as g}
<option value={g.value}>{g.label}</option>
<option value={g.value} selected={g.value === filterGenre}>{g.label}</option>
{/each}
</select>
</div>
@@ -418,12 +442,12 @@
<select
id="filter-status"
name="status"
value={data.status}
bind:value={filterStatus}
disabled={isRankView}
class="bg-zinc-900 border border-zinc-700 text-zinc-200 text-sm rounded px-3 py-2 focus:outline-none focus:border-amber-400 disabled:opacity-40 disabled:cursor-not-allowed w-full"
>
{#each statuses as st}
<option value={st.value}>{st.label}</option>
<option value={st.value} selected={st.value === filterStatus}>{st.label}</option>
{/each}
</select>
</div>
@@ -438,8 +462,8 @@
Reset
</a>
<button
type="submit"
onclick={() => (filtersOpen = false)}
type="button"
onclick={applyFilters}
class="px-4 py-2 rounded bg-amber-400 text-zinc-900 text-sm font-semibold hover:bg-amber-300 transition-colors"
>
Apply

View File

@@ -1,7 +1,7 @@
import { fail, redirect } from '@sveltejs/kit';
import type { Actions, PageServerLoad } from './$types';
import { changePassword, listUserSessions, getUserByUsername } from '$lib/server/pocketbase';
import { presignAvatarUrl } from '$lib/server/minio';
import { resolveAvatarUrl } from '$lib/server/minio';
import { log } from '$lib/server/logger';
export const load: PageServerLoad = async ({ locals }) => {
@@ -16,13 +16,11 @@ export const load: PageServerLoad = async ({ locals }) => {
log.warn('profile', 'listUserSessions failed (non-fatal)', { err: String(e) });
}
// Fetch avatar presigned URL if user has one
// Fetch avatar — MinIO first, fall back to OAuth provider picture
let avatarUrl: string | null = null;
try {
const record = await getUserByUsername(locals.user.username);
if (record?.avatar_url) {
avatarUrl = await presignAvatarUrl(locals.user.id);
}
avatarUrl = await resolveAvatarUrl(locals.user.id, record?.avatar_url);
} catch (e) {
log.warn('profile', 'avatar fetch failed (non-fatal)', { err: String(e) });
}

View File

@@ -6,7 +6,7 @@ import {
getUserPublicLibrary,
getUserCurrentlyReading
} from '$lib/server/pocketbase';
import { presignAvatarUrl } from '$lib/server/minio';
import { resolveAvatarUrl } from '$lib/server/minio';
import { log } from '$lib/server/logger';
export const load: PageServerLoad = async ({ params, locals }) => {
@@ -15,11 +15,9 @@ export const load: PageServerLoad = async ({ params, locals }) => {
const profile = await getPublicProfile(username).catch(() => null);
if (!profile) error(404, `User "${username}" not found`);
// Resolve avatar
// Resolve avatar — MinIO first, fall back to OAuth provider picture
let avatarUrl: string | null = null;
if (profile.avatar_url) {
avatarUrl = await presignAvatarUrl(profile.id).catch(() => null);
}
avatarUrl = await resolveAvatarUrl(profile.id, profile.avatar_url).catch(() => null);
// Subscription state for the logged-in visitor
let isSubscribed = false;