Compare commits

...

54 Commits

Author SHA1 Message Date
Admin
8a5bee7c09 ci: pin staticcheck as go tool, use go tool runner
Some checks failed
CI / Lint (push) Failing after 1m9s
CI / Test (push) Successful in 42s
CI / Build (push) Has been skipped
2026-03-06 11:01:09 +05:00
Admin
09e90d2ab0 feat: play cached TTS audio immediately on auto-start if already generated
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-03 13:46:37 +05:00
Admin
f28de2195d fix: always autoplay after generation and resume TTS on HTMX chapter swap
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 19:43:48 +05:00
Admin
9388a355e5 fix: show paused state instead of error when autoplay policy blocks play()
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
When the browser blocks audio.play() with NotAllowedError (no user gesture),
the player was surfacing an error badge and the user had no hint it was ready.
Now onCanPlay catches NotAllowedError specifically and calls setPaused() so
the audio element keeps its src and the user just needs to tap play once.
2026-03-02 19:18:28 +05:00
Admin
1c547519e8 fix: speed controls local playbackRate only, TTS generation always uses speed 1
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
Speed buttons now set audio.playbackRate directly (on load and on change)
instead of passing the speed to Kokoro. The TTS generation request always
sends speed:1 so the cached audio is at normal pitch/tempo and the browser
handles tempo adjustment locally.
2026-03-02 18:50:46 +05:00
Admin
1536c7dad1 fix: use audio.autoplay flag to unblock autoplay policy on chapter auto-advance
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
When the auto-next feature navigated to a new chapter and startAudio was
called on page load (outside a user gesture), the browser's autoplay policy
blocked the audio.play() call inside onCanPlay. Fix by setting
audio.autoplay = true before audio.load() when started automatically, then
resetting it in onCanPlay so manual pause/play is unaffected.
2026-03-02 15:13:24 +05:00
Admin
093e627a0e fix: restore missing generateAudio, settingsPanel, chapterListPanel DOM refs removed in cleanup
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 14:28:10 +05:00
Admin
90a16a7223 feat: autoplay uses full page navigation, auto-starts TTS on arrival
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 14:19:21 +05:00
Admin
412b0fb478 fix: settings panel closed by default, closes on click outside
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 14:10:21 +05:00
Admin
0a479cfe22 feat: replace native voice select with custom styled dropdown
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 14:05:24 +05:00
Admin
92f900a8a9 fix: seek-track and seek-times match chapter width on all devices
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 14:01:34 +05:00
Admin
b61e1a4212 fix: align seek-track and seek-times with chapter content width on desktop
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 14:00:56 +05:00
Admin
6e0041ea01 feat: settings panel becomes native-style bottom sheet on mobile with drag-to-expand/dismiss
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 13:57:29 +05:00
Admin
3e74019a5e fix: seek-track 90% width, seek-times 95% width, both margin auto
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 13:54:47 +05:00
Admin
d300714333 feat: replace hourglass emoji with animated SVG spinner in generating state
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 13:46:09 +05:00
Admin
e0265db7a8 feat: replace speed slider with preset buttons, replace voice grid with dropdown
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 13:44:06 +05:00
Admin
1cdc82249c fix: replace unicode gear with SVG icon, increase chapter title size in header
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 13:19:41 +05:00
Admin
ce4b904e62 fix: slim seek bar, scrollable settings panel, speed+toggles first then voices
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 13:17:41 +05:00
Admin
c08da7ef54 fix: cap voice grid height so speed/toggles remain visible in settings panel
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 13:09:48 +05:00
Admin
9d01ea1ee3 fix: hide idle badge by default, remove duplicate next-idle badge, remove chapter numbers under nav arrows
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 13:07:00 +05:00
Admin
c717fcfe94 fix: settings panel flex layout — voice grid scrolls, speed/toggles always pinned at bottom
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 13:03:06 +05:00
Admin
82d78bedb2 fix: center settings panel vertically on mobile, scrollable with max-h cap
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 13:00:17 +05:00
Admin
1254e90a65 fix: remove duplicate inline chapter nav, upgrade player prev/next to SVG chevrons with chapter number
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 12:56:35 +05:00
Admin
da3370ba9b fix: strip 'Chapter N - N: ' prefix from chapter titles in content menu
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 12:55:08 +05:00
Admin
2546a5774e fix: increase settings panel bottom offset to clear full player height
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 12:52:16 +05:00
Admin
00a6d2ce43 fix: proper SVG play/pause icons, preserve progress on resume, fix badge row alignment
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 12:49:36 +05:00
Admin
98fd0de7bc fix: remove stale old-template fragment from chapterTmpl in ui.go
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 12:41:02 +05:00
Admin
1128e47f6b feat: redesign ranking page with list layout, sticky filter bar, top genres fix
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 12:25:01 +05:00
Admin
1caa1f7ab5 feat: homepage hero card, progress bars, unread badges, stats, sort, genre chips, recently added
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 11:31:31 +05:00
Admin
d9c69a089c fix: prevent home page book cards from overflowing viewport on mobile
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 11:24:44 +05:00
Admin
42d0d53b52 fix: chapter list toggle no longer overwrites title span with arrow
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 11:17:58 +05:00
Admin
5182f31518 fix: show TTS gen duration in idle badge (24s / 1m 4s), drop uppercase
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 11:17:15 +05:00
Admin
7ed5eb0d69 fix: remove duplicate chapter title h1, drop nav arrow, enlarge hamburger
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 11:15:21 +05:00
Admin
0503f69537 Fix settings panel height on mobile using fixed positioning
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 11:12:25 +05:00
Admin
8e1519234a Stretch settings panel to near full height on mobile
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 11:08:26 +05:00
Admin
1353e2f6e6 Increase voice grid height on mobile
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 11:00:17 +05:00
Admin
954200af3c Show live % progress in playing badge, generation time in idle badge
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 10:55:53 +05:00
Admin
42dc38d224 Fix seek bar: tall hit area, correct rail-based rect, prevent scroll during touch drag
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 10:53:57 +05:00
Admin
63aae93ee3 Play pre-generated next chapter audio directly without page navigation to avoid autoplay block on locked screen
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 10:51:50 +05:00
Admin
c3e5a57dab Stretch settings panel full width on small screens
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 10:42:38 +05:00
Admin
7589866965 Fetch voices from Kokoro API at runtime; replace select with styled voice card grid
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 10:18:43 +05:00
Admin
2107b6e6b8 Replace checkbox inputs with toggle switches in reader settings panel
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 10:11:31 +05:00
Admin
3b905e54fe Add auto-scroll toggle to reader settings, persisted in localStorage
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 10:10:41 +05:00
Admin
427e952253 Pause TTS auto-scroll on user interaction, resume after 3s; add book cover artwork to Media Session
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 10:06:59 +05:00
Admin
090d155c19 Custom seek bar, bigger nav buttons, Media Session API for background playback
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 09:55:01 +05:00
Admin
5cfb411530 Unify player into single compact row, remove orphaned circle button
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 09:38:54 +05:00
Admin
e4ab8664e9 Fix listener stacking across HTMX swaps and wire auto-next chapter playback
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 09:35:39 +05:00
Admin
decf22f258 Replace custom audio player with native HTML audio element and compact layout
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-02 09:24:40 +05:00
Admin
5db21a821a Simplify chapter page audio player UX
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
- Remove play button and gear icon from chapter nav bar
- Show bottom mini-player bar always (not slide-up on play)
- Remove duplicate collapsed-bar seek track; keep only expanded panel scrub bar
- Move gear/settings button into the bottom mini-player bar
- Move settings panel into mini-player aside, opening upward
- Clean up JS: remove navBtn/navIcon refs, showPlayer/hidePlayer, playerSeekTrack refs
2026-03-01 23:56:36 +05:00
Admin
dfb3f61812 Fix book page returning empty body due to malformed onclick attribute
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
html/template's context-aware parser was treating the backslash-escaped
quote in the inline onclick regex literal as an HTML attribute terminator,
causing template execution to produce an empty/broken output.

Move the summary paragraph-splitting logic into a named window.openSummary
function in the script block, and call it via a clean onclick=openSummary(this)
attribute with no inline regex literals.
2026-03-01 23:45:08 +05:00
Admin
db59d55f42 Fix summary overlay scrolling and paragraph formatting on mobile
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
- Make overlay scroll within the viewport instead of being clipped (overflow-y-auto on backdrop, items-start alignment)
- Split summary text into paragraphs in the zoom overlay (newline-split with sentence-boundary fallback)
- Add Summary heading in overlay for context
- Remove old merge-poller JS; simplify generateAudio to use data.url directly (Kokoro backend rewrite cleanup)
2026-03-01 23:37:05 +05:00
Admin
13a3eacca6 Use full page navigation for double-tap chapter switching
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-01 23:12:55 +05:00
Admin
398a4703eb Add double-tap left/right navigation on chapter content page
Some checks failed
CI / Lint (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Build (push) Has been cancelled
2026-03-01 23:06:49 +05:00
Admin
e4624d0a0c Refactor chapter page: semantic HTML, accessibility improvements, Tailwind hover classes, and book page summary zoom overlay 2026-03-01 23:03:18 +05:00
7 changed files with 1807 additions and 1332 deletions

View File

@@ -33,9 +33,7 @@ jobs:
run: go vet ./...
- name: staticcheck
run: |
go install honnef.co/go/tools/cmd/staticcheck@latest
staticcheck ./...
run: go tool staticcheck ./...
# ── tests ────────────────────────────────────────────────────────────────────
test:

View File

@@ -3,8 +3,16 @@ module github.com/libnovel/scraper
go 1.25.0
require (
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c // indirect
github.com/gorilla/websocket v1.5.3 // indirect
github.com/yuin/goldmark v1.7.16 // indirect
golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678 // indirect
golang.org/x/mod v0.31.0 // indirect
golang.org/x/net v0.51.0 // indirect
golang.org/x/sync v0.19.0 // indirect
golang.org/x/tools v0.40.1-0.20260108161641-ca281cf95054 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
honnef.co/go/tools v0.7.0 // indirect
)
tool honnef.co/go/tools/cmd/staticcheck

View File

@@ -1,9 +1,22 @@
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c h1:pxW6RcqyfI9/kWtOwnv/G+AzdKuy2ZrqINhenH4HyNs=
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/yuin/goldmark v1.7.16 h1:n+CJdUxaFMiDUNnWC3dMWCIQJSkxH4uz3ZwQBkAlVNE=
github.com/yuin/goldmark v1.7.16/go.mod h1:ip/1k0VRfGynBgxOz0yCqHrbZXhcjxyuS66Brc7iBKg=
golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa h1:FRnLl4eNAQl8hwxVVC17teOw8kdjVDVAiFMtgUdTSRQ=
golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678 h1:1P7xPZEwZMoBoz0Yze5Nx2/4pxj6nw9ZqHWXqP0iRgQ=
golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678/go.mod h1:AbB0pIl9nAr9wVwH+Z2ZpaocVmF5I4GyWCDIsVjR0bk=
golang.org/x/mod v0.31.0 h1:HaW9xtz0+kOcWKwli0ZXy79Ix+UW/vOfmWI5QVd2tgI=
golang.org/x/mod v0.31.0/go.mod h1:43JraMp9cGx1Rx3AqioxrbrhNsLl2l/iNAvuBkrezpg=
golang.org/x/net v0.51.0 h1:94R/GTO7mt3/4wIKpcR5gkGmRLOuE/2hNGeWq/GBIFo=
golang.org/x/net v0.51.0/go.mod h1:aamm+2QF5ogm02fjy5Bb7CQ0WMt1/WVM7FtyaTLlA9Y=
golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4=
golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/tools v0.40.1-0.20260108161641-ca281cf95054 h1:CHVDrNHx9ZoOrNN9kKWYIbT5Rj+WF2rlwPkhbQQ5V4U=
golang.org/x/tools v0.40.1-0.20260108161641-ca281cf95054/go.mod h1:Ik/tzLRlbscWpqqMRjyWYDisX8bG13FrdXp3o4Sr9lc=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.7.0 h1:w6WUp1VbkqPEgLz4rkBzH/CSU6HkoqNLp6GstyTx3lU=
honnef.co/go/tools v0.7.0/go.mod h1:pm29oPxeP3P82ISxZDgIYeOaf9ta6Pi0EWvCFoLG2vc=

View File

@@ -15,7 +15,6 @@ import (
"io"
"log/slog"
"net/http"
"os"
"strconv"
"strings"
"sync"
@@ -39,12 +38,17 @@ type Server struct {
kokoroURL string // Kokoro-FastAPI base URL, e.g. http://kokoro:8880
kokoroVoice string // default voice, e.g. af_bella
// audioMu guards audioInFlight.
// audioInFlight maps an audio cache key to a channel that is closed when
// the in-flight Kokoro request for that key finishes (successfully or not).
// This prevents duplicate concurrent TTS generation for the same file.
// voiceMu guards cachedVoices.
voiceMu sync.RWMutex
cachedVoices []string // populated on first request from Kokoro /v1/audio/voices
// audioMu guards audioCache and audioInFlight.
// audioCache maps a cache key to the Kokoro download filename returned by
// POST /v1/audio/speech with return_download_link=true.
// audioInFlight deduplicates concurrent generation requests for the same key.
audioMu sync.Mutex
audioInFlight map[string]chan struct{}
audioCache map[string]string // cacheKey → kokoro download filename
audioInFlight map[string]chan struct{} // cacheKey → closed when done
}
// New creates a new Server.
@@ -57,10 +61,50 @@ func New(addr string, oCfg orchestrator.Config, novel scraper.NovelScraper, log
writer: writer.New(oCfg.StaticRoot),
kokoroURL: kokoroURL,
kokoroVoice: kokoroVoice,
audioCache: make(map[string]string),
audioInFlight: make(map[string]chan struct{}),
}
}
// voices returns the list of available Kokoro voices. On the first call it
// fetches GET /v1/audio/voices from the Kokoro service and caches the result.
// If the fetch fails (Kokoro not up yet, network error, etc.) it falls back to
// the hardcoded kokoroVoices list so the UI is never empty.
func (s *Server) voices() []string {
s.voiceMu.RLock()
cached := s.cachedVoices
s.voiceMu.RUnlock()
if len(cached) > 0 {
return cached
}
if s.kokoroURL != "" {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, s.kokoroURL+"/v1/audio/voices", nil)
if err == nil {
req.Header.Set("Accept", "application/json")
resp, err := http.DefaultClient.Do(req)
if err == nil {
defer resp.Body.Close()
var payload struct {
Voices []string `json:"voices"`
}
if resp.StatusCode == http.StatusOK && json.NewDecoder(resp.Body).Decode(&payload) == nil && len(payload.Voices) > 0 {
s.voiceMu.Lock()
s.cachedVoices = payload.Voices
s.voiceMu.Unlock()
s.log.Info("fetched kokoro voices", "count", len(payload.Voices))
return payload.Voices
}
}
}
s.log.Warn("could not fetch kokoro voices, using built-in list")
}
return kokoroVoices
}
// ListenAndServe starts the HTTP server and blocks until the provided context
// is cancelled.
func (s *Server) ListenAndServe(ctx context.Context) error {
@@ -82,18 +126,18 @@ func (s *Server) ListenAndServe(ctx context.Context) error {
mux.HandleFunc("GET /ui/ranking/status", s.handleRankingStatus)
// Plain-text chapter content for browser-side TTS
mux.HandleFunc("GET /ui/chapter-text/{slug}/{n}", s.handleChapterText)
// Server-side audio generation and serving.
// Audio generation can take several minutes for long chapters, so wrap it
// in its own timeout handler instead of relying on the server WriteTimeout.
// Server-side audio generation via Kokoro /v1/audio/speech.
// Generation can take several minutes, so wrap in its own timeout handler.
audioGenHandler := http.TimeoutHandler(
http.HandlerFunc(s.handleAudioGenerate),
10*time.Minute,
`{"error":"audio generation timed out"}`,
)
mux.Handle("POST /ui/audio/{slug}/{n}", audioGenHandler)
mux.HandleFunc("GET /ui/audio/{slug}/{n}/status", s.handleAudioStatus)
mux.HandleFunc("GET /ui/audio-file/{slug}/{n}", s.handleAudioFile)
mux.HandleFunc("GET /ui/audio-file/{slug}/{n}/part/{p}", s.handleAudioFilePart)
// Status route: returns the proxy URL if audio was already generated, 404 otherwise.
mux.HandleFunc("GET /ui/audio/{slug}/{n}", s.handleAudioStatus)
// Proxy route: fetches the generated file from Kokoro /v1/download/{filename}.
mux.HandleFunc("GET /ui/audio-proxy/{slug}/{n}", s.handleAudioProxy)
srv := &http.Server{
Addr: s.addr,
@@ -142,25 +186,18 @@ func (s *Server) handleChapterText(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, stripMarkdown(raw))
}
// ─── Chunked audio generation ────────────────────────────────────────────────
// ─── Audio generation via Kokoro /v1/audio/speech ────────────────────────────
//
// handleAudioGenerate handles POST /ui/audio/{slug}/{n}.
//
// Flow:
// 1. If the merged MP3 already exists on disk → return it immediately.
// 2. Otherwise split the chapter text into up to audioParts equal-ish chunks
// (by paragraph), generate part 0 synchronously (so the browser can start
// playing right away), then launch a background goroutine that generates
// parts 1…N and merges them into the final file.
// 3. Response: {"url":"<part-0-url>","parts":N,"merged":false}
// (or {"url":"<final-url>","parts":1,"merged":true} on a cache hit).
// It calls Kokoro's POST /v1/audio/speech with return_download_link=true.
// Kokoro generates the audio, saves it to its own temp storage, and returns
// the download filename in the X-Download-Path response header.
// We cache that filename (in memory, keyed by slug/chapter/voice/speed) and
// return a proxy URL that the browser sets as audio.src.
//
// Deduplication: if another request is already generating the *merged* file
// for the same (slug,n,voice,speed) key, the new request blocks until it
// finishes and then serves the cached result.
const audioParts = 10
// handleAudioGenerate handles POST /ui/audio/{slug}/{n}.
// On a cache hit the proxy URL is returned immediately without re-generating.
// Concurrent requests for the same key are deduplicated.
func (s *Server) handleAudioGenerate(w http.ResponseWriter, r *http.Request) {
slug := r.PathValue("slug")
n, err := strconv.Atoi(r.PathValue("n"))
@@ -186,18 +223,17 @@ func (s *Server) handleAudioGenerate(w http.ResponseWriter, r *http.Request) {
speed = body.Speed
}
audioPath := s.writer.AudioPath(slug, n, voice, speed)
cacheKey := fmt.Sprintf("%s/%d/%s/%.2f", slug, n, voice, speed)
// Fast path: merged file already on disk.
if _, err := os.Stat(audioPath); err == nil {
s.writeAudioResponse(w, slug, n, voice, speed, 1, true)
// Fast path: already generated this session.
s.audioMu.Lock()
if filename, ok := s.audioCache[cacheKey]; ok {
s.audioMu.Unlock()
s.writeAudioResponse(w, slug, n, voice, speed, filename)
return
}
// Deduplicate concurrent generation requests for the same merged file.
cacheKey := fmt.Sprintf("%s/%d/%s/%.2f", slug, n, voice, speed)
s.audioMu.Lock()
// Deduplicate concurrent generation for the same key.
if ch, ok := s.audioInFlight[cacheKey]; ok {
s.audioMu.Unlock()
select {
@@ -206,8 +242,11 @@ func (s *Server) handleAudioGenerate(w http.ResponseWriter, r *http.Request) {
http.Error(w, `{"error":"request cancelled"}`, http.StatusServiceUnavailable)
return
}
if _, err := os.Stat(audioPath); err == nil {
s.writeAudioResponse(w, slug, n, voice, speed, 1, true)
s.audioMu.Lock()
filename, ok := s.audioCache[cacheKey]
s.audioMu.Unlock()
if ok {
s.writeAudioResponse(w, slug, n, voice, speed, filename)
} else {
http.Error(w, `{"error":"audio generation failed"}`, http.StatusInternalServerError)
}
@@ -240,207 +279,86 @@ func (s *Server) handleAudioGenerate(w http.ResponseWriter, r *http.Request) {
return
}
// Ensure audio dir exists.
if err := os.MkdirAll(s.writer.AudioDir(slug), 0o755); err != nil {
http.Error(w, `{"error":"failed to create audio dir"}`, http.StatusInternalServerError)
// Call Kokoro POST /v1/audio/speech with return_download_link=true.
// Kokoro saves the generated audio to its own temp storage and returns the
// download path in the X-Download-Path response header.
filename, err := s.generateSpeech(r.Context(), text, voice, speed)
if err != nil {
s.log.Error("kokoro speech generation failed", "slug", slug, "chapter", n, "err", err)
http.Error(w, `{"error":"speech generation failed"}`, http.StatusBadGateway)
return
}
parts := splitTextIntoParts(text, audioParts)
totalParts := len(parts)
s.audioMu.Lock()
s.audioCache[cacheKey] = filename
s.audioMu.Unlock()
// Generate part 0 synchronously so the browser can start playing immediately.
if err := s.generateAudioPart(r.Context(), slug, n, voice, speed, 0, parts[0]); err != nil {
s.log.Error("part 0 generation failed", "err", err)
http.Error(w, `{"error":"part 0 generation failed"}`, http.StatusBadGateway)
return
}
if totalParts == 1 {
// Only one part — rename it to the final path directly.
partPath := s.writer.AudioPartPath(slug, n, voice, speed, 0)
if err := os.Rename(partPath, audioPath); err != nil {
http.Error(w, `{"error":"failed to save audio"}`, http.StatusInternalServerError)
return
}
s.log.Info("audio generated (single part)", "slug", slug, "chapter", n)
s.writeAudioResponse(w, slug, n, voice, speed, 1, true)
return
}
// Return part-0 URL immediately; generate the rest in the background.
s.writeAudioResponse(w, slug, n, voice, speed, totalParts, false)
// Background: generate parts 1…N then merge.
go func() {
bgCtx := context.Background()
for p := 1; p < totalParts; p++ {
if err := s.generateAudioPart(bgCtx, slug, n, voice, speed, p, parts[p]); err != nil {
s.log.Error("background part generation failed", "part", p, "err", err)
return
}
}
if err := s.mergeAudioParts(slug, n, voice, speed, totalParts); err != nil {
s.log.Error("audio merge failed", "slug", slug, "chapter", n, "err", err)
return
}
s.log.Info("audio merged", "slug", slug, "chapter", n, "parts", totalParts)
}()
s.log.Info("audio generated", "slug", slug, "chapter", n, "filename", filename)
s.writeAudioResponse(w, slug, n, voice, speed, filename)
}
// splitTextIntoParts divides text (paragraphs separated by blank lines) into
// at most n equal-ish chunks. Returns at least 1 element.
func splitTextIntoParts(text string, n int) []string {
// Split into paragraphs on blank lines.
raw := strings.Split(text, "\n\n")
var paras []string
for _, p := range raw {
p = strings.TrimSpace(p)
if p != "" {
paras = append(paras, p)
}
}
if len(paras) == 0 {
return []string{text}
}
if n > len(paras) {
n = len(paras)
}
if n < 1 {
n = 1
}
chunks := make([]string, n)
chunkSize := (len(paras) + n - 1) / n // ceiling division
for i := 0; i < n; i++ {
start := i * chunkSize
end := start + chunkSize
if start >= len(paras) {
// Fewer paragraphs than requested parts: return what we have.
chunks = chunks[:i]
break
}
if end > len(paras) {
end = len(paras)
}
chunks[i] = strings.Join(paras[start:end], "\n\n")
}
if len(chunks) == 0 {
return []string{text}
}
return chunks
}
// generateAudioPart calls Kokoro for a single text chunk and writes the result
// atomically to AudioPartPath(…, part).
func (s *Server) generateAudioPart(ctx context.Context, slug string, n int, voice string, speed float64, part int, text string) error {
// generateSpeech calls POST /v1/audio/speech on Kokoro with return_download_link=true
// and returns the filename from the X-Download-Path response header.
func (s *Server) generateSpeech(ctx context.Context, text, voice string, speed float64) (string, error) {
reqBody, _ := json.Marshal(map[string]interface{}{
"model": "kokoro",
"input": text,
"voice": voice,
"response_format": "mp3",
"speed": speed,
"stream": false,
"model": "kokoro",
"input": text,
"voice": voice,
"response_format": "mp3",
"speed": speed,
"stream": false,
"return_download_link": true,
})
req, err := http.NewRequestWithContext(ctx, http.MethodPost,
s.kokoroURL+"/v1/audio/speech", bytes.NewReader(reqBody))
if err != nil {
return fmt.Errorf("build request: %w", err)
return "", fmt.Errorf("build request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
return fmt.Errorf("kokoro request: %w", err)
return "", fmt.Errorf("kokoro request: %w", err)
}
defer resp.Body.Close()
// Drain body so the connection can be reused.
_, _ = io.Copy(io.Discard, resp.Body)
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return fmt.Errorf("kokoro status %d: %s", resp.StatusCode, string(body))
return "", fmt.Errorf("kokoro status %d", resp.StatusCode)
}
partPath := s.writer.AudioPartPath(slug, n, voice, speed, part)
tmpPath := partPath + ".tmp"
f, err := os.Create(tmpPath)
if err != nil {
return fmt.Errorf("create temp file: %w", err)
// X-Download-Path is e.g. "/download/speech_abc123.mp3"
dlPath := resp.Header.Get("X-Download-Path")
if dlPath == "" {
return "", fmt.Errorf("kokoro did not return X-Download-Path header")
}
if _, err := io.Copy(f, resp.Body); err != nil {
f.Close()
os.Remove(tmpPath)
return fmt.Errorf("write audio: %w", err)
// Extract just the filename from the path.
filename := dlPath
if idx := strings.LastIndex(dlPath, "/"); idx >= 0 {
filename = dlPath[idx+1:]
}
f.Close()
if err := os.Rename(tmpPath, partPath); err != nil {
os.Remove(tmpPath)
return fmt.Errorf("rename temp file: %w", err)
if filename == "" {
return "", fmt.Errorf("empty filename in X-Download-Path: %q", dlPath)
}
return nil
return filename, nil
}
// mergeAudioParts concatenates totalParts part files in order into AudioPath,
// then removes the individual part files.
func (s *Server) mergeAudioParts(slug string, n int, voice string, speed float64, totalParts int) error {
audioPath := s.writer.AudioPath(slug, n, voice, speed)
tmpPath := audioPath + ".tmp"
out, err := os.Create(tmpPath)
if err != nil {
return fmt.Errorf("create merged temp: %w", err)
}
for p := 0; p < totalParts; p++ {
partPath := s.writer.AudioPartPath(slug, n, voice, speed, p)
data, err := os.ReadFile(partPath)
if err != nil {
out.Close()
os.Remove(tmpPath)
return fmt.Errorf("read part %d: %w", p, err)
}
if _, err := out.Write(data); err != nil {
out.Close()
os.Remove(tmpPath)
return fmt.Errorf("write merged part %d: %w", p, err)
}
}
out.Close()
if err := os.Rename(tmpPath, audioPath); err != nil {
os.Remove(tmpPath)
return fmt.Errorf("rename merged: %w", err)
}
// Clean up part files (best-effort).
for p := 0; p < totalParts; p++ {
os.Remove(s.writer.AudioPartPath(slug, n, voice, speed, p))
}
return nil
}
func (s *Server) writeAudioResponse(w http.ResponseWriter, slug string, n int, voice string, speed float64, parts int, merged bool) {
var url string
if merged {
url = fmt.Sprintf("/ui/audio-file/%s/%d?voice=%s&speed=%.1f", slug, n, voice, speed)
} else {
url = fmt.Sprintf("/ui/audio-file/%s/%d/part/0?voice=%s&speed=%.1f", slug, n, voice, speed)
}
// writeAudioResponse writes the JSON response for a generated audio chapter.
// The URL points to our proxy handler which fetches from Kokoro on demand.
func (s *Server) writeAudioResponse(w http.ResponseWriter, slug string, n int, voice string, speed float64, filename string) {
proxyURL := fmt.Sprintf("/ui/audio-proxy/%s/%d?voice=%s&speed=%.1f", slug, n, voice, speed)
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]interface{}{
"url": url,
"parts": parts,
"merged": merged,
"url": proxyURL,
"filename": filename,
})
}
// writeAudioURL is kept for backward compatibility (used by dedup waiters).
func (s *Server) writeAudioURL(w http.ResponseWriter, slug string, n int, voice string, speed float64) {
s.writeAudioResponse(w, slug, n, voice, speed, 1, true)
}
// handleAudioStatus handles GET /ui/audio/{slug}/{n}/status.
// Returns {"merged":true/false,"url":"..."} so the browser can poll for the
// merged file after receiving a parts response from handleAudioGenerate.
// handleAudioStatus handles GET /ui/audio/{slug}/{n}.
// Returns the proxy URL if audio was already generated this session, 404 otherwise.
func (s *Server) handleAudioStatus(w http.ResponseWriter, r *http.Request) {
slug := r.PathValue("slug")
n, err := strconv.Atoi(r.PathValue("n"))
@@ -460,23 +378,22 @@ func (s *Server) handleAudioStatus(w http.ResponseWriter, r *http.Request) {
}
}
audioPath := s.writer.AudioPath(slug, n, voice, speed)
merged := false
if _, err := os.Stat(audioPath); err == nil {
merged = true
}
cacheKey := fmt.Sprintf("%s/%d/%s/%.2f", slug, n, voice, speed)
s.audioMu.Lock()
filename, ok := s.audioCache[cacheKey]
s.audioMu.Unlock()
mergedURL := fmt.Sprintf("/ui/audio-file/%s/%d?voice=%s&speed=%.1f", slug, n, voice, speed)
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]interface{}{
"merged": merged,
"url": mergedURL,
})
if !ok {
http.Error(w, `{"error":"not generated"}`, http.StatusNotFound)
return
}
s.writeAudioResponse(w, slug, n, voice, speed, filename)
}
// handleAudioFile handles GET /ui/audio-file/{slug}/{n}.
// Serves the cached merged MP3 file identified by voice and speed query params.
func (s *Server) handleAudioFile(w http.ResponseWriter, r *http.Request) {
// handleAudioProxy handles GET /ui/audio-proxy/{slug}/{n}.
// It looks up the Kokoro download filename for this chapter (voice/speed) and
// proxies GET /v1/download/{filename} from the Kokoro server back to the browser.
func (s *Server) handleAudioProxy(w http.ResponseWriter, r *http.Request) {
slug := r.PathValue("slug")
n, err := strconv.Atoi(r.PathValue("n"))
if err != nil || n < 1 {
@@ -495,52 +412,41 @@ func (s *Server) handleAudioFile(w http.ResponseWriter, r *http.Request) {
}
}
audioPath := s.writer.AudioPath(slug, n, voice, speed)
if _, err := os.Stat(audioPath); err != nil {
http.NotFound(w, r)
cacheKey := fmt.Sprintf("%s/%d/%s/%.2f", slug, n, voice, speed)
s.audioMu.Lock()
filename, ok := s.audioCache[cacheKey]
s.audioMu.Unlock()
if !ok {
http.Error(w, "audio not generated yet", http.StatusNotFound)
return
}
kokoroURL := s.kokoroURL + "/v1/download/" + filename
req, err := http.NewRequestWithContext(r.Context(), http.MethodGet, kokoroURL, nil)
if err != nil {
http.Error(w, "failed to build proxy request", http.StatusInternalServerError)
return
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
http.Error(w, "kokoro download failed", http.StatusBadGateway)
return
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
http.Error(w, fmt.Sprintf("kokoro returned %d", resp.StatusCode), http.StatusBadGateway)
return
}
w.Header().Set("Content-Type", "audio/mpeg")
w.Header().Set("Cache-Control", "public, max-age=86400")
http.ServeFile(w, r, audioPath)
}
// handleAudioFilePart handles GET /ui/audio-file/{slug}/{n}/part/{p}.
// Serves a specific MP3 part file.
func (s *Server) handleAudioFilePart(w http.ResponseWriter, r *http.Request) {
slug := r.PathValue("slug")
n, err := strconv.Atoi(r.PathValue("n"))
if err != nil || n < 1 {
http.NotFound(w, r)
return
w.Header().Set("Cache-Control", "public, max-age=3600")
if cl := resp.Header.Get("Content-Length"); cl != "" {
w.Header().Set("Content-Length", cl)
}
p, err := strconv.Atoi(r.PathValue("p"))
if err != nil || p < 0 {
http.NotFound(w, r)
return
}
voice := r.URL.Query().Get("voice")
if voice == "" {
voice = s.kokoroVoice
}
speedStr := r.URL.Query().Get("speed")
speed := 1.0
if speedStr != "" {
if v, err := strconv.ParseFloat(speedStr, 64); err == nil && v > 0 {
speed = v
}
}
partPath := s.writer.AudioPartPath(slug, n, voice, speed, p)
if _, err := os.Stat(partPath); err != nil {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", "audio/mpeg")
w.Header().Set("Cache-Control", "no-store")
http.ServeFile(w, r, partPath)
_, _ = io.Copy(w, resp.Body)
}
func (s *Server) handleScrapeCatalogue(w http.ResponseWriter, r *http.Request) {

File diff suppressed because it is too large Load Diff

View File

@@ -85,6 +85,17 @@ func (w *Writer) ReadMetadata(slug string) (scraper.BookMeta, bool, error) {
return meta, true, nil
}
// MetadataMtime returns the modification time (Unix seconds) of the
// metadata.yaml file for slug, or 0 if the file cannot be stat'd.
func (w *Writer) MetadataMtime(slug string) int64 {
path := filepath.Join(w.bookDir(slug), "metadata.yaml")
fi, err := os.Stat(path)
if err != nil {
return 0
}
return fi.ModTime().Unix()
}
// ─── Chapters ─────────────────────────────────────────────────────────────────
// ChapterExists returns true if the markdown file for ref already exists on disk.
@@ -270,6 +281,11 @@ func SplitChapterTitle(raw string) (title, date string) {
}
}
// Strip "Chapter N - N: " prefix (novelfire double-number format).
// Also handles "Chapter N: " (single number) and "Chapter N - Title" without colon.
chNumRe := regexp.MustCompile(`(?i)^chapter\s+\d+(?:\s*-\s*\d+)?\s*:\s*`)
raw = strings.TrimSpace(chNumRe.ReplaceAllString(raw, ""))
// Match a trailing relative date: "<n> <unit>[s] ago"
dateRe := regexp.MustCompile(`\s*(\d+\s+(?:second|minute|hour|day|week|month|year)s?\s+ago)\s*$`)
if m := dateRe.FindStringSubmatchIndex(raw); m != nil {

BIN
scraper/scraper Executable file

Binary file not shown.