fix(browse): replace SingleFile with direct Go HTTP fetch; fix ranking schema and cache-hit ranking gap
- Remove SingleFile CLI and Node.js from Dockerfile; runtime base reverted to alpine:3.21 - Replace triggerBrowseSnapshot with triggerDirectScrape: plain Go HTTP GET to novelfire.net (page is server-rendered, no browser needed) - Fix Accept-Encoding bug: stop setting header manually so Go transport auto-decompresses gzip - Add in-memory browse cache fallback (browseMemCache) for when MinIO and upstream both fail - Add warmBrowseCache() startup goroutine - Call triggerDirectScrape on MinIO cache hits too, so PocketBase ranking records are repopulated after a schema reset or fresh deploy without waiting for a cache miss - Fix grid view cards: wrap in <a> instead of <div> so clicking navigates to book detail - Remove SINGLEFILE_PATH and BROWSERLESS_URL env vars from Dockerfile
This commit is contained in:
@@ -13,20 +13,12 @@ RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
|
||||
go build -ldflags="-s -w" -o /scraper ./cmd/scraper
|
||||
|
||||
# ── Runtime stage ──────────────────────────────────────────────────────────────
|
||||
# Use node:22-alpine so single-file-cli (npm package) runs natively without
|
||||
# any glibc shims. The pre-compiled binary release requires glibc symbols
|
||||
# (e.g. __res_init) that Alpine's gcompat shim does not provide.
|
||||
FROM node:22-alpine
|
||||
FROM alpine:3.21
|
||||
|
||||
# ca-certificates: HTTPS to novelfire.net
|
||||
# tzdata: timezone data
|
||||
RUN apk add --no-cache ca-certificates tzdata
|
||||
|
||||
# Install single-file-cli as a global npm package.
|
||||
# It runs via Node.js (no Deno/glibc needed) and connects to an external
|
||||
# Chromium via the CDP --browser-server flag.
|
||||
RUN npm install -g single-file-cli
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY --from=builder /scraper /app/scraper
|
||||
@@ -40,12 +32,9 @@ RUN chown -R scraper:scraper /app
|
||||
USER scraper
|
||||
|
||||
# ── Configuration ─────────────────────────────────────────────────────────────
|
||||
ENV BROWSERLESS_URL=http://browserless:3030
|
||||
ENV BROWSERLESS_STRATEGY=content
|
||||
ENV SCRAPER_WORKERS=0
|
||||
ENV SCRAPER_STATIC_ROOT=/app/static/books
|
||||
ENV SCRAPER_HTTP_ADDR=:8080
|
||||
ENV SINGLEFILE_PATH=single-file
|
||||
|
||||
EXPOSE 8080
|
||||
|
||||
|
||||
Reference in New Issue
Block a user