Notebooklm book organizer

NotebookLM Book Organizer

Organize a multi-drive book library into deduplicated, subject-categorized groups ready for NotebookLM upload and Obsidian wiki ingestion.

Overview

This skill operates in three phases:

1. Scan & Index -- Read existing book scan JSONs (or scan drives directly) to build a unified index of all files with metadata (path, size, filename, extension, drive).
2. Deduplicate -- Identify duplicates using filename similarity + file size matching. Flag exact dupes and near-dupes (same title, different edition/format). Produce a dedup report the user can review before any files are moved or deleted.
3. Categorize & Organize -- Auto-categorize books by subject using title/filename keyword analysis. Group into subject buckets targeting ~100 files each (leaves room for arxiv/github additions later, user-configurable). Output organized folder structure + Obsidian-compatible index files.

Phase 1: Scan & Index

If scan JSONs already exist

The user may have pre-scanned their drives. Look for JSON files matching the pattern *_books_*.json or *_drive_*.json in the working directory. These contain arrays of objects with FullName, Length, and LastWriteTime fields.

# Expected JSON structure per entry:
{
  "FullName": "C:\\path\\to\\book.pdf",
  "Length": 12345678,         # file size in bytes
  "LastWriteTime": "2026-04-12T13:53:08-05:00"
}

Load all JSONs and merge into a single index. Tag each entry with its source drive.

If no scan JSONs exist

Run the scripts/scan_drives.py script to scan specified directories:

python scripts/scan_drives.py --paths "C:\Books" "G:\My Drive" --extensions pdf,epub,mobi,djvu,md,txt --output scan_results.json

Code files and repos

For code files (.py, .js, .ts, .rs, .go, etc.) and repos:

• If the user wants them in NotebookLM: convert to consolidated markdown using scripts/code_to_md.py which walks a repo and produces a single .md file with file tree + concatenated source.
• If the user wants them on GitHub: note them for a separate GitHub push workflow. Do NOT auto-push anything -- just flag and organize.

Phase 2: Deduplicate

Run scripts/deduplicate.py against the unified index.

Deduplication strategy (in priority order):

1. Exact match: Same filename + same file size = definite duplicate
2. Fuzzy filename match: Normalize filenames (strip edition info, publishers, years, "copy", "- Copy", numbering) and compare. Same normalized name + size within 10% = likely duplicate.
3. Cross-drive detection: Flag files that appear on multiple drives (C vs G vs D). The user's D drive is often a backup/restore point of G.

Output: dedup_report.json

{
  "exact_duplicates": [
    {
      "group_id": 1,
      "files": ["C:\\path\\book.pdf", "G:\\path\\book.pdf", "D:\\path\\book.pdf"],
      "recommendation": "keep C:\\path\\book.pdf (most recent)",
      "savings_mb": 45.2
    }
  ],
  "likely_duplicates": [...],
  "unique_files": 4521,
  "total_duplicates": 3256,
  "potential_savings_gb": 42.1
}

IMPORTANT: Present the dedup report to the user BEFORE taking any action. The user decides which copy to keep. Never delete files -- only move or create symlinks.

Phase 3: Categorize & Organize

Run scripts/categorize.py against the deduplicated index.

Auto-categorization approach:

Extract subject keywords from filenames and paths. The categorizer uses a two-tier system:

Tier 1 -- Broad domains (detected from keywords in titles):

• blockchain / crypto / defi / bitcoin / ethereum
• quantum / quantum-computing / quantum-physics
• ai-ml / machine-learning / deep-learning / neural-networks
• cybersecurity / hacking / penetration-testing
• mathematics / linear-algebra / calculus / statistics
• physics / fluid-dynamics / thermodynamics
• software-engineering / agile / devops / docker
• programming / python / javascript / rust / go
• economics / finance / trading
• philosophy / consciousness / cognitive-science
• biology / bioinformatics / genomics
• networking / distributed-systems / protocols

Tier 2 -- Sub-categories (finer splits when a domain exceeds 150 files): The categorizer recursively splits large groups using secondary keywords until each group is near the target size (default: 150, configurable via --target-size).

Output structure:

organized_library/
+-- _index.md                          # Obsidian master index with [[wikilinks]]
+-- _dedup_report.md                   # Human-readable dedup summary
+-- blockchain/
|   +-- _index.md                      # Category index with file list
|   +-- bitcoin-fundamentals/
|   |   +-- _index.md
|   |   +-- ... (symlinks or file list)
|   +-- defi-protocols/
|       +-- _index.md
|       +-- ...
+-- quantum/
|   +-- _index.md
|   +-- ...
+-- ai-ml/
|   +-- _index.md
|   +-- deep-learning/
|   |   +-- ...
|   +-- reinforcement-learning/
|       +-- ...
+-- ...

Obsidian index files (_index.md)

Each _index.md uses YAML frontmatter and wikilinks for Obsidian compatibility:

---
title: "Blockchain"
type: library-category
book_count: 287
subcategories: [bitcoin-fundamentals, defi-protocols, smart-contracts]
last_updated: 2026-04-15
---

# Blockchain

## Subcategories
- [[bitcoin-fundamentals/_index|Bitcoin Fundamentals]] (142 books)
- [[defi-protocols/_index|DeFi Protocols]] (145 books)

## All Books
| Title | Format | Size | Source Drive | Path |
|-------|--------|------|-------------|------|
| [[Proof of Stake - Buterin]] | PDF | 8.5 MB | C | ... |

NotebookLM upload manifest

Also generate notebooklm_upload_manifest.json -- groups of files sized for NotebookLM notebooks (respecting the ~50 source limit per notebook, and file size limits). Each group becomes one NotebookLM notebook:

{
  "notebooks": [
    {
      "name": "Blockchain - Bitcoin Fundamentals",
      "files": ["path1.pdf", "path2.epub", ...],
      "file_count": 48,
      "total_size_mb": 312
    }
  ]
}

User Interaction Points

This skill has THREE mandatory user checkpoints:

1. After scanning: "Found X files across Y drives. Proceed to dedup?"
2. After dedup: "Found X duplicates (Y GB recoverable). Review the report before I organize."
3. After categorization: "Organized into X categories. Review the structure before I create the folders / upload to NotebookLM."

Configuration

The user can override defaults:

• --target-size 150 -- target files per NotebookLM notebook group
• --output-dir ./organized_library -- where to create the organized structure
• --mode symlink|copy|move -- how to handle files (default: symlink, safest)
• --obsidian-vault-path -- if set, generate index files directly in the vault

Dependencies

• Python 3.10+
• rapidfuzz (for fuzzy matching): pip install rapidfuzz
• No other external dependencies -- standard library for everything else

Known limitations

• scripts/scan_drives.py and scripts/code_to_md.py are referenced by this skill but are not included in this release. They must be written to match the JSON schema documented in Phase 1 above.
• Categorization uses keyword matching only (no embeddings). Accuracy degrades for ambiguously titled files. Review the categorization report before promoting to NotebookLM.