Offline rag library

Offline RAG Library

Generate a zero-server reading-library site: a static single-page app (SPA) that hosts EPUB books as markdown, with an on-device AI chatbot that answers questions using cross-book RAG — all running in the browser via transformers.js + WebGPU. No backend, no API keys, no external LLM server.

What this skill produces

A site with:

• A portal (catalog grid with search, tag filters, sort, light/dark theme)
• A reader (markdown books rendered to HTML, section-tracked via IntersectionObserver)
• An AI assistant (Gemma 4 E2B via WebGPU + embeddinggemma-300m for RAG)
• Scope-aware retrieval — ask about the current chapter, current book, or all books

The chatbot downloads model weights on first use (~3.1 GB generation + ~300 MB embeddings) and caches them in IndexedDB; subsequent visits are instant.

Prerequisites

• Python 3.10+ with uv — uv sync installs beautifulsoup4, lxml, markdownify automatically
• Node.js 18+ — only needed to build the embedding cache (scripts/package.json)
• A WebGPU browser to use the chatbot (Chrome 113+, Edge, Firefox) — the site works anywhere

Skill file layout

offline-rag-library/
├── scripts/                          # Build pipeline (all run from repo root)
│   ├── convert_epub.py               # EPUB → books/<slug>/index.md (pure Python, no Calibre)
│   ├── build_catalog.py              # books/*/index.md front matter → books.json
│   ├── build_chatbot_index.py        # books/*/index.md → assets/chatbot_chunks.json
│   ├── build_chatbot_embeddings.mjs  # chunks.json → assets/chatbot_embeddings.bin (needs npm install)
│   ├── init_library.py               # One-command site scaffolder
│   ├── wire_chatbot.py               # Wire SPA+chatbot <link>/<script> into index.html
│   └── book_markdown.py              # Shared markdown helpers (section markers)
├── web_assets/                       # Source of truth for the SPA (copy to assets/ to publish)
│   ├── spa.js, spa.css               # Hash router, markdown loader, section tracker
│   ├── chatbot.js, chatbot.css       # On-device Gemma + RAG assistant
│   └── vendor/marked.esm.js          # Markdown renderer
├── template/index.html               # Generic portal template (init_library.py brands it)
└── docs/                             # Live demo (GitHub Pages) with 3 public-domain books

Quick start: scaffold a new library

# 1. Scaffold a fresh site into a target directory
uv run python scripts/init_library.py ./my-library --name "My Library"

# 2. Add books (EPUB only — pure Python, no external binaries)
uv run python scripts/convert_epub.py "path/to/book.epub" --slug book-slug --output-root ./my-library/books

# 3. Generate the catalog (books.json) from front matter
uv run python scripts/build_catalog.py --root ./my-library

# 4. Build the RAG chunk index
uv run python scripts/build_chatbot_index.py --root ./my-library

# 5. Build the embedding cache (first run downloads ~300 MB model)
cd scripts && npm install && cd ..
node scripts/build_chatbot_embeddings.mjs   # writes assets/chatbot_embeddings.bin
# Move the artifacts into the target:
cp assets/chatbot_chunks.json assets/chatbot_embeddings.bin ./my-library/assets/

# 6. Serve locally and test
cd my-library && python -m http.server 8000

Adding a book to an existing library

# Convert the EPUB (pure Python — stdlib zipfile + BeautifulSoup, zero external binaries)
uv run python scripts/convert_epub.py "book.epub" --slug the-book-title --output-root ./books

# Optional: add display metadata (description, tags, year, coverBg) as a sidecar JSON
# Create books/the-book-title/meta.json:
#   {"description": "...", "tags": ["Fiction", "Classic"], "year": 1865}

# Rebuild catalog + RAG artifacts (run in this order)
uv run python scripts/build_catalog.py
uv run python scripts/build_chatbot_index.py
node scripts/build_chatbot_embeddings.mjs   # after: cd scripts && npm install

# Sync web_assets to the served location and wire the SPA
cp web_assets/spa.css web_assets/spa.js web_assets/chatbot.css web_assets/chatbot.js assets/
cp web_assets/vendor/marked.esm.js assets/vendor/
uv run python scripts/wire_chatbot.py

Book markdown format

Each book is a single index.md inside books/<slug>/:

---
title: Book Title
author: Author Name
source: single-page EPUB conversion
---

# Book Title

*Author · single-page EPUB conversion*

<!-- rl-section id="section-1" class="epub-section" kicker="Section 1" title="Introduction" -->

## Introduction

Prose and images here…

• Section markers <!-- rl-section id="…" class="…" kicker="…" title="…" --> are how the SPA splits content into scroll-tracked <section> elements and how the chunker assigns sectionId.
• Section IDs must be stable across regenerations — the IntersectionObserver and the chatbot chunker both key off them.
• Images are strictly relative (assets/filename.png); the SPA rewrites them to resolve from the book's folder.
• convert_epub.py generates all of this automatically from a valid EPUB.

Architecture: how the RAG works

1. Chunk index (assets/chatbot_chunks.json): every book's text, split into ~1200-char overlapping chunks per section. Schema: {id, lang, url, title, author, sectionId, sectionTitle, text}.
2. Embedding cache (assets/chatbot_embeddings.bin): precomputed vectors (embeddinggemma-300m, 768-dim, q8, mean-pooled + L2-normalized). Binary format v2 with a SHA-256 of chunk texts + the model ID in the header — the browser refuses a stale or wrong-model bin.
3. Query time: the browser embeds the user's question with the same model, cosine-sorts against the cached vectors, and injects the top-5 chunks into the Gemma system prompt alongside the current page text.

The scope toggle (This chapter / This book / All books) filters which chunks are eligible for retrieval.

Configuration: branding per deployment

Set window.LIBRARY_CONFIG in the host index.html (the template and init_library.py handle this):

<script>
  window.LIBRARY_CONFIG = {
    name: "My Library",
    description: "a curated collection of books",
    slug: "my-library"   // namespaces IndexedDB + localStorage
  };
</script>

The slug is critical: it isolates each deployment's IndexedDB and localStorage keys so multiple libraries on the same origin don't collide.

Deploying to GitHub Pages

# The repo root (or docs/ folder) IS the site. Push and enable Pages:
# Settings → Pages → Source: Deploy from branch → main / (root) or /docs

The SPA is a static index.html + assets/ + books/ — no build step, no server-side rendering.

Troubleshooting

Symptom	Cause	Fix
Chatbot says "Index needs rebuild"	chatbot_embeddings.bin model ID ≠ browser's embedder	Rebuild: node scripts/build_chatbot_embeddings.mjs
Cross-book search disabled	chatbot_chunks.json hash ≠ bin's stored hash (content changed)	Rebuild index then embeddings (in that order)
"Device is lost" during generation	WebGPU OOM from large prompt	Use "This chapter" scope, or ask shorter questions
Book images broken	Non-relative image paths	Ensure assets/ prefix; the SPA only rewrites relative URLs
convert_epub.py fails	Corrupt/DRMed EPUB	The converter reads via stdlib zipfile; DRMed files will fail to parse

What this skill does NOT include

• No Calibre / AZW3 support — EPUB only. AZW3/MOBI decoding requires Calibre; convert to EPUB first if needed.
• No web scraping — bring your own EPUB files.
• No CLI chatbot — the browser assistant is the product (the original repo had a terminal CLI; this skill keeps the scope to the web app).