- parsePDF: return all text as single 'Full Text' chapter (admin splits manually)
- parseEPUB: fix chapter numbering to use sequential counter not spine index
- Remove dead code: chaptersFromBookmarks, cleanChapterText, extractChaptersFromText, chapterHeadingRE; drop pdfcpu alias and regexp imports
- Backend: POST /api/admin/books/:slug/split-chapters endpoint — splits text on '---' dividers, optional '## Title' headers, writes chapters via WriteChapter
- UI: admin panel now shows for all admin users regardless of source_url; chapter split tool shown when book has single 'Full Text' chapter, pre-fills from MinIO content
Three fixes to PDF chapter extraction quality:
1. Page ordering: parse page number from pdfcpu filename (out_Content_page_N.txt)
instead of using lexicographic sort index — fixes chapters bleeding into each
other (e.g. Prologue text appearing inside Chapter 1).
2. Windows-1252 chars: map bytes 0x91-0x9F to proper Unicode (curly quotes U+2018/
U+2019/U+201C/U+201D, em-dash U+2014, etc.) instead of raw Latin-1 control
bytes that rendered as ◆ in the browser.
3. Chapter header cleanup: skip the first page of each bookmark range (decorative
title art page) and strip any run-on title fragment at the start of the first
body page (e.g. 'for New Journeys!I stood atop...' → 'I stood atop...'). The
remaining sentence truncation is a fundamental limitation of this PDF's
PUA-encoded body font (C2_1/Literata) — those glyphs cannot be decoded without
the publisher's private ToUnicode mapping.
Use PDF outline (bookmarks) for chapter titles and page ranges, then
extract text from per-page content streams via pdfcpu ExtractContent.
This avoids the indefinite hang caused by dslipak/pdf trying to resolve
custom font ToUnicode CMaps on publisher PDFs.
- parsePDF: decrypt → ExtractContent → Bookmarks → chaptersFromBookmarks
- chaptersFromBookmarks: flatten bookmark tree, skip front/back matter
(Cover, Insert, Title Page, Copyright, Appendix), assign page ranges
- extractTextFromContentStream: handle TJ arrays (concat literal strings,
skip hex glyph arrays and kerning numbers) + single Tj strings
- Falls back to paragraph-splitting when no bookmarks present
- Build verified; test PDF produces 9 chapters with proper titles
- parsePDF function restored in import.go (body was orphaned outside function)
- ParseImportFile() called at upload time with 3-min timeout; chapters stored as JSON in MinIO
- runner.go: prefer ChaptersKey path (read pre-parsed JSON) over BookImport.Import()
- ImportChapterStore interface added; store wired in runner/main.go
- HeartbeatTask and ReapStaleTasks now include import_tasks collection
- parseImportTask now returns ChaptersKey in domain.ImportTask
- asynq_runner.go handleImportTask passes ChaptersKey
- pb-init-v3.sh: chapters_key field added to import_tasks schema
- parsePDF now attempts to strip encryption via pdfcpu (empty user password)
before handing bytes to dslipak/pdf — fixes '256-bit encryption key' error
on publisher PDFs that use owner-only encryption (copy/print restrictions)
- Add pdfcpu v0.11.1 as direct dependency (was already indirect)
- docker-compose.yml minio-init: add 'imports' and 'translations' buckets
so a fresh deploy creates all required buckets
- docker-compose.yml: BODY_SIZE_LIMIT=52428800 (50MB) on UI service
— adapter-node was rejecting PDFs >512KB with 'Content-length exceeds limit'
- storage/import.go: add AnalyzeFile() public function using real PDF/EPUB parsers
- handlers_import.go: analyzeImportFile() now calls storage.AnalyzeFile() instead
of the file-size stub, so chapter count is accurate in the preview
- NewBookImporter now takes *Store instead of raw *minio.Client (same package access)
- Runner Dependencies gains ChapterIngester interface; storeImportedChapters no-op removed
- store.IngestChapters is now actually called after PDF/EPUB extraction
- BookImport and ChapterIngester wired in runner main.go
- CreateImportTask interface gains initiatorUserID param; threaded through store/asynq/handler
- domain.ImportTask gains InitiatorUserID field; parseImportTask populates it
- Runner notifies initiator (falls back to 'admin' when empty)
- UI: import task list polls every 3s while any task is pending/running
- UI: label[for] + input[id] fix for a11y warnings in admin import page
- Add ImportTask/ImportResult types to domain.go
- Add TypeImportBook to asynqqueue for task routing
- Add CreateImportTask to producer and storage layers
- Add ClaimNextImportTask/FinishImportTask to Consumer interfaces
- Add import task handling to runner (polling + Asynq handler)
- Add BookImporter interface to bookstore for PDF/EPUB parsing
- Add backend API endpoints: POST/GET /api/admin/import
- Add SvelteKit UI at /admin/import with task list
- Add nav link in admin layout
Note: PDF/EPUB parsing is a placeholder - needs external library integration.
