Obsidian wiki builder

Skill richfrem/agent-plugins-skills/plugins/obsidian-wiki-engine/skills/obsidian-wiki-builder

repo for reusable plugins and skills

Install

npx -y skills add richfrem/agent-plugins-skills --skill obsidian-wiki-builder

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

4 stars4 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Transforms raw source files registered in wiki_sources.json into Karpathy-style LLM wiki nodes inside the wiki-root. Generates concept pages, cluster pages, index, and table of contents. Use when building or rebuilding the wiki from raw content.

SKILL.md

5.3 KB, as published. Nobody here has run it

Dependencies

Requires pyyaml and Python 3.8+. Also requires rlm-factory plugin installed.

pip install -r requirements.txt

Obsidian Wiki Builder

Status: Active Author: Richard Fremmerlid Domain: Obsidian Wiki Engine

Purpose

Transforms raw source documents (registered via wiki_sources.json) into Karpathy-style wiki nodes inside the designated wiki_root. Each concept gets its own {concept}.md node with wikilinks, cross-references, and an RLM summary layer.

Output Layout (Rigid)

{wiki_root}/
  wiki/
    _index.md          ← master concept index
    _toc.md            ← table of contents
    _{cluster}.md      ← per-topic cluster page
    {concept}.md       ← individual wiki node
  rlm/
    {concept}/
      summary.md       ← 1-5 sentence distilled summary
      bullets.md       ← key idea bullets
      deep.md          ← full multi-pass distillation
  meta/
    wiki_sources.json  ← raw source registry (created by wiki-init)
    config.yaml        ← wiki settings
    agent-memory.json  ← stale file tracking

Build Pipeline

wiki_builder.py runs a 3-step pipeline internally:

ingest.py  →  concept_extractor.py  →  wiki node formatting

ingest.py: Reads raw source files, normalizes content, hashes for staleness detection
concept_extractor.py: Groups records by concept slug, merges multi-source records into one authoritative node (the Karpathy "compile" step), improves cluster assignment via keyword extraction
wiki node formatting: Renders merged records into Karpathy-format .md files with wikilinks, cluster pages, _index.md, and _toc.md

Multi-Source Concept Merging

When two source files (e.g. arch-docs/auth.md and research/oauth.md) produce the same concept slug, they are merged into one wiki node that:

Combines content from all sources with attribution headers
Lists all source files in source_files frontmatter
Re-derives the cluster from combined content keywords
Sets multi_source: true for identification

This implements the core Karpathy "compile" metaphor: N raw files → M concept nodes (M ≤ N).

Usage

Note about repo-scoped inspection

The installer and helper agents should ask for and confirm the repository or filesystem scope before running discovery. By default prefer searching the project root only (e.g. /Users/you/projects/agent-plugins-skills). For broader searches (whole home or system), prompt the user and explain the expected runtime cost.

Build wiki nodes from all registered sources

python ./scripts/wiki_builder.py --wiki-root /path/to/wiki-root

Build from one named source only

python ./scripts/wiki_builder.py --wiki-root /path/to/wiki-root --source arch-docs

Use shared .agent/learning/ RLM cache for existing summaries

python ./scripts/wiki_builder.py --wiki-root /path/to/wiki-root \
    --rlm-cache-dir /path/to/project/.agent/learning/rlm_wiki_cache

Dry run (plan without writing)

python ./scripts/wiki_builder.py --wiki-root /path/to/wiki-root --dry-run

Karpathy Wiki Node Format

Each concept node uses this structure:

---
concept: {concept_name}
source: {source_label}
source_file: {relative_path}
wiki_root: {wiki_root}
generated_at: {timestamp}
cluster: {cluster_name}
---

# {Concept Name}

{1-sentence RLM summary}

## Key Ideas

- {bullet}
- {bullet}

## Details

{full content, truncated to 800 tokens if needed}

## See Also

- [[{related_concept_1}]]
- [[{related_concept_2}]]

## Raw Source

- `{source_label}` → `{source_file}`

Source Registry Format (`rlm_wiki_raw_sources_manifest.json`)

Canonical location: .agent/learning/rlm_wiki_raw_sources_manifest.json (mirrors rlm-factory's .agent/learning/rlm_profiles.json convention).

Managed by raw_manifest.py via the Guided Discovery agent during /wiki-init:

{
  "namespace": "my-project",
  "wiki_root": "/Users/me/vault/wiki-root",
  "sources": {
    "daily-notes": {
      "path": "/Users/me/vault/notes",
      "label": "daily-notes",
      "extensions": [".md"],
      "excludes": ["_archive", "*.tmp"],
      "description": "Daily journal and quick capture notes"
    },
    "arch-docs": {
      "path": "/Users/me/docs/architecture",
      "label": "arch-docs",
      "extensions": [".md", ".txt"],
      "excludes": [],
      "description": "Architecture decision records and design docs"
    }
  },
  "global_excludes": ["_archive", "*.tmp", "__pycache__"]
}

When to Use

After /wiki-init completes source registration
After adding new raw source documents
After running /wiki-distill to inject fresh RLM summaries
When the wiki index is stale (agent-memory.json shows diffs)

Related Scripts

raw_manifest.py — WikiSourceConfig loader (mirrors RLMConfig)
ingest.py — raw file parsing and normalization
wiki_builder.py — Karpathy node formatter and linker
distill_wiki.py — cheap-model distillation wrapper