- Batch-resolve avatar presign URLs server-side in GET /api/comments/[slug];
returns avatarUrls map alongside comments and myVotes
- CommentsSection.svelte: show avatar image or initials fallback (24px top-level,
20px replies) next to each comment/reply username
- iOS CommentsResponse gains avatarUrls field; CommentsViewModel stores and
populates it on load; CommentRow renders AsyncImage with initials fallback
- Also includes: comment replies (1-level nesting), delete, sort (Top/New),
parent_id schema migration, and AvatarCropModal cropperjs fix
- Go scraper: add PresignAvatarUploadURL/PresignAvatarURL/DeleteAvatar to
Store interface, implement on HybridStore+MinioClient, register
GET /api/presign/avatar-upload/{userId} and /api/presign/avatar/{userId}
- SvelteKit: replace direct AWS S3 SDK in minio.ts with presign calls to
the Go scraper; rewrite avatar +server.ts (POST=presign, PATCH=record key)
- iOS: rewrite uploadAvatar() as 3-step presigned PUT flow; refactor
chip components into shared ChipButton in CommonViews.swift
Replace blocking POST /api/audio with a non-blocking 202 flow: the Go
handler immediately enqueues a job in a new `audio_jobs` PocketBase
collection and returns {job_id, status}. A background goroutine runs
the actual Kokoro TTS work and updates job status (pending → generating
→ done/failed). A new GET /api/audio/status/{slug}/{n} endpoint lets
clients poll progress. The SvelteKit proxy and AudioPlayer.svelte are
updated to POST, then poll the status route every 2s until done.
- Add user_sessions PocketBase collection (user_id, session_id, user_agent, ip, created/last_seen)
- Extend auth token format to include a per-login authSessionId (4th segment)
- Hook validates authSessionId against DB on each request; revoked sessions are cleared immediately
- Login/register create a session record capturing user-agent and IP
- Profile page shows all active sessions with current session highlighted; per-session End/Sign out buttons
- GET /api/sessions and DELETE /api/sessions/[id] endpoints for client-side revocation
- Backward compatible: legacy 3-segment tokens pass through without DB check
The MinIO cache key only encoded page number, so all filter combinations
hit the same cache entry and returned unfiltered results. Introduce
BrowseFilteredHTMLKey() that encodes sort/genre/status into the key
(e.g. novelfire.net/html/new-isekai-completed/page-1.html); falls back
to the original page-only key for default filters to preserve existing
cached pages.
When a book is not in the local DB and the preview endpoint is hit,
metadata and the chapter index are now saved to PocketBase in the
background. Subsequent visits load from the local store instead of
scraping live. Chapter text is not fetched until an explicit scrape job
is triggered.
- Speed is no longer part of Kokoro generation, in-memory cache keys, or
MinIO object keys — audio is always generated at 1.0 and playback speed
is applied client-side via audioEl.playbackRate
- presignAudio() now calls rewriteHost() so chapter audio URLs use the
public MinIO endpoint (same as presignVoiceSample already did)
- docker-compose.yml: rename MINIO_PUBLIC_ENDPOINT → PUBLIC_MINIO_PUBLIC_URL
for the ui service so SvelteKit's $env/dynamic/public picks it up
handlePresignAudio now checks AudioExists before presigning; previously it
would generate a valid-looking signed URL for a non-existent object, causing
the browser to get 404 from MinIO directly.
- Add AudioExists to Store interface and HybridStore
- Guard handlePresignAudio with AudioExists check, return 404 if missing
- Propagate 404 through presignAudio() in minio.ts (typed error.status=404)
- SvelteKit presign route forwards 404 to the browser instead of 500
On every scraper startup, EnsureMigrations fetches each collection schema
and PATCHes in any missing fields. This repairs the progress collection on
cloud deploys where pb-init ran before user_id was added to the schema.
No-ops when the field already exists.
- New MinIO bucket 'libnovel-browse' (MINIO_BUCKET_BROWSE env) for storing
self-contained HTML snapshots of novelfire browse pages
- Store interface gains SaveBrowsePage / GetBrowsePage / BrowsePageKey methods
- handleBrowse is now cache-first: serves from MinIO snapshot when available,
then fires a background triggerBrowseSnapshot goroutine to populate cache
on live-fetch (de-duplicated, 90s timeout)
- New 'save-browse' CLI subcommand to bulk-capture pages via SingleFile CLI
- Dockerfile: downloads pinned single-file-x86_64-linux binary (v2.0.83),
adds gcompat + libstdc++ to Alpine runtime for glibc compatibility
- docker-compose: adds libnovel-browse bucket init and SINGLEFILE_PATH env
- .gitignore: exclude scraper/scraper build artifact
Add optional user_id field to the progress collection. When a user is
authenticated, progress is keyed by user_id instead of session_id, making
it portable across devices and browsers. Anonymous reading still works via
session_id as before.
On login or registration, any progress accumulated under the anonymous
session is merged into the user's account (most-recent timestamp wins),
so chapters read before logging in are not lost.
- scripts/pb-init.sh: add user_id field to progress collection schema
- scraper/internal/storage/pocketbase.go: add user_id to EnsureCollections
- ui/src/lib/server/pocketbase.ts: Progress interface gains user_id;
getProgress/allProgress/setProgress accept optional userId and query/write
by user_id when present; add mergeSessionProgress() helper
- ui/src/routes/login/+page.server.ts: call mergeSessionProgress after
successful login and registration (fire-and-forget, non-fatal)
- ui/src/routes/api/progress/+server.ts: pass locals.user?.id to setProgress
- ui/src/routes/books/+page.server.ts: pass locals.user?.id to allProgress
- ui/src/routes/books/[slug]/+page.server.ts: pass locals.user?.id to getProgress
Add a second MinIO client (pub) initialized with MINIO_PUBLIC_ENDPOINT so
presigned audio URLs are signed against the public hostname from the start,
rather than signed internally and then rewritten. This avoids AWS4 signature
mismatch (SignatureDoesNotMatch 403) that occurred when the signed host was
substituted after signing.
- storage/minio.go: add PublicEndpoint/PublicUseSSL to MinioConfig; add pub
client field; NewMinioClient creates pub client when public endpoint differs;
PresignAudio uses pub, PresignChapter keeps internal client
- cmd/scraper/main.go: wire MINIO_PUBLIC_ENDPOINT and MINIO_PUBLIC_USE_SSL env vars
- docker-compose.yml: expose MINIO_PUBLIC_ENDPOINT and MINIO_PUBLIC_USE_SSL to scraper service
- ui/src/lib/server/minio.ts: remove rewriteHost() call from presignAudio
- ui/Dockerfile: copy package-lock.json and run npm ci --omit=dev in the
runtime stage so marked (and other runtime deps) are available to
adapter-node at startup — fixes ERR_MODULE_NOT_FOUND for 'marked'
- storage: add ReindexChapters to Store interface and HybridStore — walks
MinIO objects for a slug, reads chapter titles, upserts chapters_idx
- server: add POST /api/reindex/{slug} to rebuild chapters_idx from MinIO
Adds a scraping_tasks PocketBase collection with fields for kind, status,
progress counters, timestamps, and error info. Exposes CreateScrapeTask,
UpdateScrapeTask, and ListScrapeTasks on the Store interface with
implementations in HybridStore and PocketBaseStore.
- Inject *slog.Logger into HybridStore, PocketBaseStore, and pbClient
- Fix credential defaults in main.go (changeme123 / admin) to match docker-compose
- listOne/listAll/upsert/deleteWhere now return errors on non-2xx HTTP status
- WriteChapter: log warn instead of discarding UpsertChapterIdx error
- MetadataMtime, GetAudioCache, GetProgress: log warn on PocketBase failures
- EnsureCollections: log info/debug/warn per outcome instead of _ = err
- CountChapterIdx: log warn on failure instead of silently returning 0
- server: log warn when SetAudioCache fails after audio generation
- NewHybridStore: add explicit Ping() before EnsureCollections for fast-fail on bad credentials
novelfire.net chapter content is server-rendered, so Browserless is not
needed. Add a dedicated chapterClient (always StrategyDirect) to Scraper
and use it in ScrapeChapterText, removing the now-irrelevant WaitFor /
RejectResourceTypes / GotoOptions fields from the ContentRequest.
PocketBase v0.23+ requires 'Bearer <token>' — sending the raw JWT without
the prefix results in 403 'Only superusers can perform this action' on all
collection record endpoints. Fix applied in three places:
- ui/src/lib/server/pocketbase.ts (pbGet, pbPost, pbPatch helpers)
- scraper/internal/storage/pocketbase.go (pbClient.do)
- scripts/pb-init.sh (wget --header in create_collection)
- e2e fixture: replace single contentClient with directClient (plain HTTP)
for chapter/metadata/ranking + contentClient (Browserless) for urlClient
only — matches production wiring and is significantly faster
- server: add max_chars field to audio request body; truncates stripped text
to N runes before sending to Kokoro (used by e2e for quick TTS tests)
- fix: move RankingItem to scraper package to break novelfire→storage import
cycle; storage.RankingItem is now a type alias for backward compat
- fix: update stale New() call in novelfire integration test (missing args)
- fix: replace removed blob-ranking methods in storage integration test with
current per-item API (UpsertRankingItem/ListRankingItems/RankingLastUpdated)
- justfile: add test-e2e and e2e tasks
- Replace SetRanking/GetRanking/SetRankingPageHTML/GetRankingPageHTML blob methods
with WriteRankingItem/ReadRankingItems/RankingFreshEnough per-item operations
- Add 24h staleness gate in ScrapeRanking to skip re-scraping fresh data
- Add GET /api/ranking endpoint returning []RankingItem sorted by rank
- Remove RankingPageCacher interface and rankingCacheAdapter adapter
- Update integration tests to use new per-item upsert semantics
- Include e2e test suite (scraper/internal/e2e/)
- Replace deprecated /api/admins/auth-with-password with
/api/collections/_superusers/auth-with-password in both the
SvelteKit UI (pocketbase.ts) and Go scraper (pocketbase.go)
- Rename custom users collection to app_users to avoid name clash
with PocketBase's built-in users auth collection
- Fix docker-compose volume mount path pb/pb_data -> pb_data so
persisted data matches the entrypoint --dir flag
- Add PB_ADMIN_EMAIL/PB_ADMIN_PASSWORD env vars to pocketbase service
so the superuser is auto-created on first boot
- scraper/internal/storage/integration_test.go: MinioClient and PocketBaseStore
integration tests covering chapter/audio round-trips, presign URLs, book
metadata, chapter index, ranking, progress, and audio cache CRUD
- scraper/internal/storage/hybrid_integration_test.go: HybridStore end-to-end
tests for metadata, chapters, ranking, progress, presign, and audio cache
- scraper/internal/storage/scrape_integration_test.go: live Browserless + storage
tests that scrape book metadata and first 3 chapters, store them, and verify
the round-trip via HybridStore
- scraper/internal/server/integration_test.go: HTTP server functional tests for
health, scrape status, presign chapter, reading progress, and chapter-text
endpoints against real MinIO + PocketBase backends
- justfile: task runner at repo root with recipes for build, test, lint, UI,
docker-compose, and individual service management
- Add `users` PocketBase collection (username, password_hash, role, created)
- Implement HMAC-SHA256 signed cookie auth in hooks.server.ts; token payload is userId:username:role
- Add User type, getUserByUsername, createUser (scrypt), loginUser (timing-safe) to pocketbase.ts
- Add login/register page with tabbed form UI and server actions
- Add logout route that clears the auth cookie
- Add layout.server.ts auth guard: redirect unauthenticated users to /login
- Extend App.Locals and App.PageData with role field
- Add AUTH_SECRET, POCKETBASE_ADMIN_EMAIL/PASSWORD to .env.example
- Install @types/node for Node crypto/scrypt types
