Ingest

Skill raven-sourav/pmm-content-engine/skills/ingest

PMM Content Distribution Engine — Newsletter scraper, Obsidian knowledge brain, multi-format content generation powered by Claude Code

Install

npx -y skills add raven-sourav/pmm-content-engine --skill ingest

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
3 stars3 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

3.3 KB, as published. Nobody here has run it

Skill: Ingest New Content

Trigger

User provides new posts, writer content, URLs, or newsletter URLs to add to the system.

Process

For new user posts

Read the .docx or text content
Analyze for voice patterns, new mental models, topic evolution
Update data/brain/pmm_brain.json voice profile if needed
Store in ChromaDB for gap map retrieval

For new reference writer content

Read the .docx, URL, or text content
Analyze for mental models, argumentation patterns, thinking process
Update the PMM Brain's synthesized models if new patterns found
Store in ChromaDB for gap map retrieval

For newsletter/blog URLs

Fetch and read the content via WebFetch
Extract writer identity, key themes, mental models, unique perspectives
Add to newsletter_insights section of the Brain
Note any new emerging beliefs for the value landscape

For newsletter scraping (Substack / Beehiiv)

When user provides a newsletter URL for scraping (e.g., "scrape this newsletter", "ingest posts from https://..."):

Scrape — Run the newsletter scraper to pull posts:
```
python3 -m src.ingestion.newsletter_scraper \
  --url "https://newsletter-url.com" \
  --days 90 --full-content
```
This auto-detects the platform (Substack or Beehiiv), scrapes posts, stores them in SQLite + ChromaDB, and saves raw JSON to data/scraped/.

Generate brain brief — Get the synthesis brief:

python3 -m src.ingestion.newsletter_scraper \
  --url "https://newsletter-url.com" \
  --days 90 --full-content --output brain-brief

Synthesize into Brain — Read the brain brief and update data/brain/pmm_brain.json:
- newsletter_insights: Add/update entry with focus, key_frameworks, key_data_points, contrarian_insights, useful_for
- evidence_bank: Extract concrete stats, benchmarks, data points
- pmm_value_landscape: Note contested or emerging beliefs
- synthesized_mental_models: Extract new mental models or frameworks
- topic_depth_layers: Add practitioner wisdom, common mistakes, key debates
Verify — Confirm what was added to each Brain section.

CLI flags

Flag	Description
`--url`	Newsletter URL (required)
`--name`	Source name (auto-detected from author if omitted)
`--days N`	Only scrape last N days
`--max-posts N`	Limit total posts
`--keywords "a,b"`	Filter by keywords
`--full-content`	Fetch full post bodies (slower, richer for synthesis)
`--no-store`	Skip SQLite/ChromaDB storage, just preview
`--output`	`summary` (default), `json`, or `brain-brief`

Supported platforms

Substack — uses public /api/v1/archive JSON endpoint
Beehiiv — uses hidden /posts JSON endpoint
Both work with custom domains (auto-detected)

For visual illustrations

Read images to understand visual style
Update visual signature in voice profile if patterns have evolved

Data Flow

Newsletter URL
  → newsletter_scraper.py (detect platform → scrape → filter)
  → data/scraped/{source}_{timestamp}.json (audit trail)
  → SQLite posts table + ChromaDB embeddings (retrieval)
  → Brain synthesis brief (for Claude analysis)
  → pmm_brain.json updates (newsletter_insights, evidence_bank, etc.)

Output

Confirm what was ingested and what changed in the Brain.