fix(browse): replace SingleFile with direct Go HTTP fetch; fix ranking schema and cache-hit ranking gap

- Remove SingleFile CLI and Node.js from Dockerfile; runtime base reverted to alpine:3.21
- Replace triggerBrowseSnapshot with triggerDirectScrape: plain Go HTTP GET to novelfire.net
  (page is server-rendered, no browser needed)
- Fix Accept-Encoding bug: stop setting header manually so Go transport auto-decompresses gzip
- Add in-memory browse cache fallback (browseMemCache) for when MinIO and upstream both fail
- Add warmBrowseCache() startup goroutine
- Call triggerDirectScrape on MinIO cache hits too, so PocketBase ranking records are
  repopulated after a schema reset or fresh deploy without waiting for a cache miss
- Fix grid view cards: wrap in <a> instead of <div> so clicking navigates to book detail
- Remove SINGLEFILE_PATH and BROWSERLESS_URL env vars from Dockerfile
This commit is contained in:
Admin
2026-03-04 21:13:59 +05:00
parent b0547c1b43
commit 7b48707cd9
3 changed files with 127 additions and 86 deletions

View File

@@ -13,20 +13,12 @@ RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
go build -ldflags="-s -w" -o /scraper ./cmd/scraper
# ── Runtime stage ──────────────────────────────────────────────────────────────
# Use node:22-alpine so single-file-cli (npm package) runs natively without
# any glibc shims. The pre-compiled binary release requires glibc symbols
# (e.g. __res_init) that Alpine's gcompat shim does not provide.
FROM node:22-alpine
FROM alpine:3.21
# ca-certificates: HTTPS to novelfire.net
# tzdata: timezone data
RUN apk add --no-cache ca-certificates tzdata
# Install single-file-cli as a global npm package.
# It runs via Node.js (no Deno/glibc needed) and connects to an external
# Chromium via the CDP --browser-server flag.
RUN npm install -g single-file-cli
WORKDIR /app
COPY --from=builder /scraper /app/scraper
@@ -40,12 +32,9 @@ RUN chown -R scraper:scraper /app
USER scraper
# ── Configuration ─────────────────────────────────────────────────────────────
ENV BROWSERLESS_URL=http://browserless:3030
ENV BROWSERLESS_STRATEGY=content
ENV SCRAPER_WORKERS=0
ENV SCRAPER_STATIC_ROOT=/app/static/books
ENV SCRAPER_HTTP_ADDR=:8080
ENV SINGLEFILE_PATH=single-file
EXPOSE 8080