feat(queue): replace PocketBase polling with Asynq + Redis
Some checks failed
CI / Docker / caddy (pull_request) Failing after 50s
CI / Check ui (pull_request) Successful in 1m5s
Release / Check ui (push) Successful in 45s
Release / Test backend (push) Successful in 1m29s
CI / Docker / ui (pull_request) Successful in 1m28s
Release / Docker / backend (push) Successful in 3m14s
CI / Test backend (pull_request) Failing after 42s
CI / Docker / backend (pull_request) Has been skipped
CI / Docker / runner (pull_request) Has been skipped
Release / Docker / caddy (push) Successful in 6m48s
Release / Docker / ui (push) Successful in 2m8s
Release / Docker / runner (push) Successful in 2m51s
Release / Upload source maps (push) Failing after 53s
Release / Gitea Release (push) Has been skipped

Introduce a Redis-backed Asynq task queue so the runner consumes TTS
jobs pushed by the backend instead of polling PocketBase.

- backend/internal/asynqqueue: Producer and Consumer wrappers
- backend/internal/runner: AsynqRunner mux, per-instance Prometheus
  registry (fixes duplicate-collector panic in tests), redisConnOpt
- backend/internal/config: REDIS_ADDR / REDIS_PASSWORD env vars
- backend/cmd/{backend,runner}/main.go: wire Redis when env set; fall
  back to legacy poll mode when unset
- Caddyfile: caddy-l4 TCP proxy for redis.libnovel.cc:6380 → homelab
- caddy/Dockerfile: add --with github.com/mholt/caddy-l4
- docker-compose.yml: Caddy exposes 6380, backend/runner get Redis env
- homelab/runner/docker-compose.yml: Redis sidecar, runner depends_on
- homelab/otel/grafana: Grafana dashboards (backend, catalogue, runner)
  and alerting rules / contact-points provisioning
This commit is contained in:
Admin
2026-03-28 14:32:40 +05:00
parent 98e4a87432
commit fe1a933fd0
21 changed files with 1869 additions and 47 deletions

View File

@@ -234,3 +234,27 @@ search.libnovel.cc {
reverse_proxy meilisearch:7700
}
# ── Redis TCP proxy: exposes homelab Redis over TLS for Asynq ─────────────────
# The backend (prod) connects to rediss://redis.libnovel.cc:6380 to enqueue
# Asynq jobs. Caddy terminates TLS (Let's Encrypt cert for redis.libnovel.cc)
# and proxies the raw TCP stream to the homelab Redis via this reverse proxy.
#
# NOTE: Redis is NOT running on the prod server — it runs on the homelab
# (192.168.0.109:6379) and is exposed to the internet via this Caddy proxy.
# The homelab Redis is protected by REDIS_PASSWORD (requirepass).
#
# Caddy layer4 app handles this; requires the caddy-l4 module in the build.
{
layer4 {
redis.libnovel.cc:6380 {
route {
tls
proxy {
# Homelab Redis — replace with actual homelab IP or FQDN
upstream {$HOMELAB_REDIS_ADDR:192.168.0.109:6379}
}
}
}
}
}
}