194 lines
7.2 KiB
Markdown
194 lines
7.2 KiB
Markdown
# LibNovel v2 — Agent Context
|
|
|
|
This file is the root-level knowledge base for LLM coding agents (OpenCode, Claude, Cursor, Copilot, etc.).
|
|
Sub-directories have their own `AGENTS.md` with deeper context (e.g. `ios/AGENTS.md`).
|
|
|
|
---
|
|
|
|
## Stack
|
|
|
|
| Layer | Technology |
|
|
|---|---|
|
|
| UI | SvelteKit 2 + Svelte 5, TypeScript, TailwindCSS |
|
|
| Backend / Runner | Go (single repo, two binaries: `backend`, `runner`) |
|
|
| iOS app | SwiftUI, iOS 17+, Swift 5.9+ |
|
|
| Database | PocketBase (SQLite) + MinIO (object storage) |
|
|
| Search | Meilisearch |
|
|
| Queue | Asynq over Redis (local) / Valkey (prod) |
|
|
| Scraping | Novelfire scraper in `backend/novelfire/` |
|
|
|
|
---
|
|
|
|
## Repository Layout
|
|
|
|
```
|
|
.
|
|
├── .gitea/workflows/ # CI/CD — Gitea Actions (NOT .github/)
|
|
├── .opencode/ # OpenCode agent config (memory, skills)
|
|
├── backend/ # Go backend + runner (single module)
|
|
├── caddy/ # Caddy reverse proxy Dockerfile
|
|
├── homelab/ # Homelab docker-compose + observability stack
|
|
├── ios/ # SwiftUI iOS app (see ios/AGENTS.md)
|
|
├── scripts/ # Utility scripts
|
|
├── ui/ # SvelteKit UI
|
|
├── docker-compose.yml # Prod compose (all services)
|
|
├── AGENTS.md # This file
|
|
└── opencode.json # OpenCode config
|
|
```
|
|
|
|
---
|
|
|
|
## CI/CD — Gitea Actions
|
|
|
|
- Workflows live in `.gitea/workflows/` — **not** `.github/workflows/`
|
|
- Self-hosted Gitea instance; use `gitea.ref_name` / `gitea.sha` (not `github.*`)
|
|
- Two workflows:
|
|
- `ci.yaml` — runs on every push to `main` (test + type-check)
|
|
- `release.yaml` — runs on `v*` tags (build Docker images, upload source maps, create Gitea release)
|
|
- Secrets: `DOCKER_USER`, `DOCKER_TOKEN`, `GITEA_TOKEN`, `GLITCHTIP_AUTH_TOKEN`
|
|
|
|
### Git credentials
|
|
|
|
Credentials are embedded in the remote URL — no `HOME=/root` or credential helper needed for push:
|
|
|
|
```
|
|
https://kamil:95782641Apple%24@gitea.kalekber.cc/kamil/libnovel.git
|
|
```
|
|
|
|
All git commands still use `HOME=/root` prefix for consistency (picks up `/root/.gitconfig` for user name/email), but push auth works without it.
|
|
|
|
### Releasing a new version
|
|
|
|
```bash
|
|
HOME=/root git tag v2.6.X -m "Short title"
|
|
HOME=/root git push origin v3-cleanup --tags
|
|
```
|
|
|
|
CI will build all Docker images, upload source maps to GlitchTip, and create a Gitea release automatically.
|
|
|
|
---
|
|
|
|
## GlitchTip Error Tracking
|
|
|
|
- Instance: `https://errors.libnovel.cc/`
|
|
- Org: `libnovel`
|
|
- Projects: `ui` (id/1), `backend` (id/2), `runner` (id/3)
|
|
- Tool: `glitchtip-cli` v0.1.0
|
|
|
|
### Per-service DSNs (stored in Doppler)
|
|
|
|
| Service | Doppler key | GlitchTip project |
|
|
|---|---|---|
|
|
| UI (SvelteKit) | `PUBLIC_GLITCHTIP_DSN` | ui (1) |
|
|
| Backend (Go) | `GLITCHTIP_DSN_BACKEND` | backend (2) |
|
|
| Runner (Go) | `GLITCHTIP_DSN_RUNNER` | runner (3) |
|
|
|
|
### Source map upload flow (release.yaml)
|
|
|
|
The correct order is **critical** — uploading before `releases new` results in 0 files shown in GlitchTip UI:
|
|
|
|
```
|
|
glitchtip-cli sourcemaps inject ./build # inject debug IDs
|
|
glitchtip-cli releases new <version> # MUST come before upload
|
|
glitchtip-cli sourcemaps upload ./build \
|
|
--release <version> # associate files with release
|
|
glitchtip-cli releases finalize <version> # mark release complete
|
|
```
|
|
|
|
---
|
|
|
|
## Infrastructure
|
|
|
|
| Environment | Host | Path | Doppler config |
|
|
|---|---|---|---|
|
|
| Prod | `165.22.70.138` | `/opt/libnovel/` | `prd` |
|
|
| Homelab runner | `192.168.0.109` | `/opt/libnovel-runner/` | `prd_homelab` |
|
|
|
|
### Docker Compose — always use Doppler
|
|
|
|
```bash
|
|
# Prod
|
|
doppler run --project libnovel --config prd -- docker compose <cmd>
|
|
|
|
# Homelab full-stack (runs from .bak file on server)
|
|
doppler run --project libnovel --config prd_homelab -- docker compose -f homelab/docker-compose.yml.bak <cmd>
|
|
|
|
# Homelab runner only
|
|
doppler run --project libnovel --config prd_homelab -- docker compose -f homelab/runner/docker-compose.yml <cmd>
|
|
```
|
|
|
|
- Prod runner has `profiles: [runner]` — `docker compose up -d` will NOT accidentally start it
|
|
- When deploying, always sync `docker-compose.yml` to the server before running `up -d`
|
|
- **Caddyfile is NOT in git** — lives at `/opt/libnovel/Caddyfile` on prod server only. Edit directly on the server and restart the `caddy` container.
|
|
|
|
---
|
|
|
|
## Observability
|
|
|
|
| Tool | Purpose |
|
|
|---|---|
|
|
| GlitchTip | Error tracking (UI + backend + runner) |
|
|
| Grafana Faro | RUM / Web Vitals (collector at `faro.libnovel.cc/collect`) → Alloy (port 12347) |
|
|
| OpenTelemetry | Distributed tracing (OTLP → cloudflared → OTel collector → Tempo) |
|
|
| Grafana | Dashboards at `https://grafana.libnovel.cc` |
|
|
|
|
### Grafana dashboards: `homelab/otel/grafana/provisioning/dashboards/`
|
|
|
|
Key dashboards:
|
|
- `backend.json` — Backend logs (Loki: `{service_name="backend"}`, plain text)
|
|
- `runner.json` — Runner logs (Loki: `{service_name="runner"}`) + Asynq Prometheus metrics
|
|
- `web-vitals.json` — Web Vitals (Loki: `{service_name="unknown_service"} kind=measurement` + pattern parser)
|
|
- `catalogue.json` — Scrape progress (Loki: `{service_name="runner"} | json | body="..."`)
|
|
|
|
### Data pipeline (2026-04-07 working state)
|
|
|
|
**Browser → Grafana Faro:**
|
|
Browser sends RUM data → `https://faro.libnovel.cc/collect` → **Alloy** `faro.receiver` (port 12347) → Loki (logs/exceptions) + OTel collector → **Tempo** (traces)
|
|
|
|
**Backend/Runner → OTel:**
|
|
Backend/Runner Go SDK → `https://otel.libnovel.cc` (cloudflared tunnel) → **OTel collector** (port 4318) → Tempo (traces) + Loki (logs via `otlphttp/loki` exporter)
|
|
Runner also sends to **Alloy** `otelcol.receiver.otlp` (port 4318) → `otelcol.exporter.loki` → Loki
|
|
|
|
### Loki log format per service
|
|
|
|
- `service_name="backend"`: Plain text (e.g. `backend: asynq task dispatch enabled`)
|
|
- `service_name="runner"`: JSON with `body`, `attributes{slug,chapters,page}`, `severity`
|
|
- `service_name="unknown_service"`: Faro RUM text format (e.g. `kind=measurement lcp=5428.0 ...`)
|
|
|
|
### OTel Collector ports (homelab)
|
|
|
|
- gRPC: `4317` — receives from cloudflared (`otel.libnovel.cc`)
|
|
- HTTP: `4318` — receives from cloudflared + Alloy
|
|
- Metrics: `8888`
|
|
|
|
### Known issues / pending fixes
|
|
|
|
- Web Vitals use `service_name="unknown_service"` (Faro SDK doesn't set service.name in browser) — works with `unknown_service` label
|
|
- Runner logs go to both Alloy→Loki AND OTel collector→Loki (dual pipeline — intentional for resilience)
|
|
|
|
---
|
|
|
|
## Go Backend
|
|
|
|
- Primary files: `orchestrator.go`, `server/handlers_*.go`, `novelfire/scraper.go`, `storage/hybrid.go`, `storage/pocketbase.go`
|
|
- Store interface: `store.go` — never touch MinIO/PocketBase clients directly outside `storage/`
|
|
- Two binaries built from the same module: `backend` (HTTP API) and `runner` (Asynq worker)
|
|
|
|
---
|
|
|
|
## SvelteKit UI
|
|
|
|
- Source: `ui/src/`
|
|
- i18n: Paraglide — translation files in `ui/messages/*.json` (5 locales)
|
|
- Auth debug bypass: `GET /api/auth/debug-login?token=<DEBUG_LOGIN_TOKEN>&username=<username>&next=<path>`
|
|
|
|
---
|
|
|
|
## iOS App
|
|
|
|
Full context in `ios/AGENTS.md`. Quick notes:
|
|
- SwiftUI, iOS 17+, `@Observable` for new types
|
|
- Download key separator: `::` (not `-`)
|
|
- Voice fallback: book override → global default → `"af_bella"`
|
|
- Offline pattern: `NetworkMonitor` env object + `OfflineBanner` + `ErrorAlertModifier`
|