kamil/libnovel
1
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases 162 Wiki Activity
Files
ce34d2c75fb206b117e6d3904f218bd7e1324dde
libnovel/AGENTS.md

194 lines
7.2 KiB
Markdown
Raw Blame History

# LibNovel v2 — Agent Context
This file is the root-level knowledge base for LLM coding agents (OpenCode, Claude, Cursor, Copilot, etc.).
Sub-directories have their own `AGENTS.md` with deeper context (e.g. `ios/AGENTS.md`).
---
## Stack
| Layer | Technology |
|---|---|
| UI | SvelteKit 2 + Svelte 5, TypeScript, TailwindCSS |
| Backend / Runner | Go (single repo, two binaries: `backend`, `runner`) |
| iOS app | SwiftUI, iOS 17+, Swift 5.9+ |
| Database | PocketBase (SQLite) + MinIO (object storage) |
| Search | Meilisearch |
| Queue | Asynq over Redis (local) / Valkey (prod) |
| Scraping | Novelfire scraper in `backend/novelfire/` |
---
## Repository Layout
```
.
├── .gitea/workflows/ # CI/CD — Gitea Actions (NOT .github/)
├── .opencode/ # OpenCode agent config (memory, skills)
├── backend/ # Go backend + runner (single module)
├── caddy/ # Caddy reverse proxy Dockerfile
├── homelab/ # Homelab docker-compose + observability stack
├── ios/ # SwiftUI iOS app (see ios/AGENTS.md)
├── scripts/ # Utility scripts
├── ui/ # SvelteKit UI
├── docker-compose.yml # Prod compose (all services)
├── AGENTS.md # This file
└── opencode.json # OpenCode config
```
---
## CI/CD — Gitea Actions
- Workflows live in `.gitea/workflows/` — **not** `.github/workflows/`
- Self-hosted Gitea instance; use `gitea.ref_name` / `gitea.sha` (not `github.*`)
- Two workflows:
- `ci.yaml` — runs on every push to `main` (test + type-check)
- `release.yaml` — runs on `v*` tags (build Docker images, upload source maps, create Gitea release)
- Secrets: `DOCKER_USER`, `DOCKER_TOKEN`, `GITEA_TOKEN`, `GLITCHTIP_AUTH_TOKEN`
### Git credentials
Credentials are embedded in the remote URL — no `HOME=/root` or credential helper needed for push:
```
https://kamil:95782641Apple%24@gitea.kalekber.cc/kamil/libnovel.git
```
All git commands still use `HOME=/root` prefix for consistency (picks up `/root/.gitconfig` for user name/email), but push auth works without it.
### Releasing a new version
```bash
HOME=/root git tag v2.6.X -m "Short title"
HOME=/root git push origin v3-cleanup --tags
```
CI will build all Docker images, upload source maps to GlitchTip, and create a Gitea release automatically.
---
## GlitchTip Error Tracking
- Instance: `https://errors.libnovel.cc/`
- Org: `libnovel`
- Projects: `ui` (id/1), `backend` (id/2), `runner` (id/3)
- Tool: `glitchtip-cli` v0.1.0
### Per-service DSNs (stored in Doppler)
| Service | Doppler key | GlitchTip project |
|---|---|---|
| UI (SvelteKit) | `PUBLIC_GLITCHTIP_DSN` | ui (1) |
| Backend (Go) | `GLITCHTIP_DSN_BACKEND` | backend (2) |
| Runner (Go) | `GLITCHTIP_DSN_RUNNER` | runner (3) |
### Source map upload flow (release.yaml)
The correct order is **critical** — uploading before `releases new` results in 0 files shown in GlitchTip UI:
```
glitchtip-cli sourcemaps inject ./build # inject debug IDs
glitchtip-cli releases new <version> # MUST come before upload
glitchtip-cli sourcemaps upload ./build \
--release <version> # associate files with release
glitchtip-cli releases finalize <version> # mark release complete
```
---
## Infrastructure
| Environment | Host | Path | Doppler config |
|---|---|---|---|
| Prod | `165.22.70.138` | `/opt/libnovel/` | `prd` |
| Homelab runner | `192.168.0.109` | `/opt/libnovel-runner/` | `prd_homelab` |
### Docker Compose — always use Doppler
```bash
# Prod
doppler run --project libnovel --config prd -- docker compose <cmd>
# Homelab full-stack (runs from .bak file on server)
doppler run --project libnovel --config prd_homelab -- docker compose -f homelab/docker-compose.yml.bak <cmd>
# Homelab runner only
doppler run --project libnovel --config prd_homelab -- docker compose -f homelab/runner/docker-compose.yml <cmd>
```
- Prod runner has `profiles: [runner]` — `docker compose up -d` will NOT accidentally start it
- When deploying, always sync `docker-compose.yml` to the server before running `up -d`
- **Caddyfile is NOT in git** — lives at `/opt/libnovel/Caddyfile` on prod server only. Edit directly on the server and restart the `caddy` container.
---
## Observability
| Tool | Purpose |
|---|---|
| GlitchTip | Error tracking (UI + backend + runner) |
| Grafana Faro | RUM / Web Vitals (collector at `faro.libnovel.cc/collect`) → Alloy (port 12347) |
| OpenTelemetry | Distributed tracing (OTLP → cloudflared → OTel collector → Tempo) |
| Grafana | Dashboards at `https://grafana.libnovel.cc` |
### Grafana dashboards: `homelab/otel/grafana/provisioning/dashboards/`
Key dashboards:
- `backend.json` — Backend logs (Loki: `{service_name="backend"}`, plain text)
- `runner.json` — Runner logs (Loki: `{service_name="runner"}`) + Asynq Prometheus metrics
- `web-vitals.json` — Web Vitals (Loki: `{service_name="unknown_service"} kind=measurement` + pattern parser)
- `catalogue.json` — Scrape progress (Loki: `{service_name="runner"} | json | body="..."`)
### Data pipeline (2026-04-07 working state)
**Browser → Grafana Faro:**
Browser sends RUM data → `https://faro.libnovel.cc/collect` → **Alloy** `faro.receiver` (port 12347) → Loki (logs/exceptions) + OTel collector → **Tempo** (traces)
**Backend/Runner → OTel:**
Backend/Runner Go SDK → `https://otel.libnovel.cc` (cloudflared tunnel) → **OTel collector** (port 4318) → Tempo (traces) + Loki (logs via `otlphttp/loki` exporter)
Runner also sends to **Alloy** `otelcol.receiver.otlp` (port 4318) → `otelcol.exporter.loki` → Loki
### Loki log format per service
- `service_name="backend"`: Plain text (e.g. `backend: asynq task dispatch enabled`)
- `service_name="runner"`: JSON with `body`, `attributes{slug,chapters,page}`, `severity`
- `service_name="unknown_service"`: Faro RUM text format (e.g. `kind=measurement lcp=5428.0 ...`)
### OTel Collector ports (homelab)
- gRPC: `4317` — receives from cloudflared (`otel.libnovel.cc`)
- HTTP: `4318` — receives from cloudflared + Alloy
- Metrics: `8888`
### Known issues / pending fixes
- Web Vitals use `service_name="unknown_service"` (Faro SDK doesn't set service.name in browser) — works with `unknown_service` label
- Runner logs go to both Alloy→Loki AND OTel collector→Loki (dual pipeline — intentional for resilience)
---
## Go Backend
- Primary files: `orchestrator.go`, `server/handlers_*.go`, `novelfire/scraper.go`, `storage/hybrid.go`, `storage/pocketbase.go`
- Store interface: `store.go` — never touch MinIO/PocketBase clients directly outside `storage/`
- Two binaries built from the same module: `backend` (HTTP API) and `runner` (Asynq worker)
---
## SvelteKit UI
- Source: `ui/src/`
- i18n: Paraglide — translation files in `ui/messages/*.json` (5 locales)
- Auth debug bypass: `GET /api/auth/debug-login?token=<DEBUG_LOGIN_TOKEN>&username=<username>&next=<path>`
---
## iOS App
Full context in `ios/AGENTS.md`. Quick notes:
- SwiftUI, iOS 17+, `@Observable` for new types
- Download key separator: `::` (not `-`)
- Voice fallback: book override → global default → `"af_bella"`
- Offline pattern: `NetworkMonitor` env object + `OfflineBanner` + `ErrorAlertModifier`
Reference in New Issue View Git Blame Copy Permalink