Omega3 distill

omega3-distill

Meant to run autonomously (nightly, headless — see the repo README for cron/scheduler setup). Be conservative and auditable: never delete curated notes, never auto-approve insights, honor redaction, and log everything you do. OMEGA3=__OMEGA3_HOME__.

0. Setup

cd __OMEGA3_HOME__
git pull --rebase 2>/dev/null || true

Read .omega/last-distill — the ISO timestamp bounding "new since last run". Keep running counters for the digest (§8).

1. Drain inbox/ → systems/ + knowledge/ (dedup + merge)

For each inbox/*.md (skip README.md and the proposals/ subdir):

1. Read it; determine target folder from type: (system → systems/, knowledge → knowledge/).
2. Look for an existing note on the same subject (match by slug, title, or systems: tag; also try graphify query if you use the graph).
   • Exists → merge new info in, refresh updated:, add any missing [[links]]. Don't duplicate.
   • New → create from the matching templates/ file, filled out, with [[links]] to related notes.
3. Update the destination folder's INDEX.md (one dense line under its marker comment).
4. git rm/delete the drained inbox file. Leave anything ambiguous in inbox and flag it in the digest rather than guessing.

2. Harvest per-repo session buffers (optional supplement)

If you keep per-repo session buffers (e.g. .remember/ dirs under your repos root), scan them for entries modified after last-distill and pull only durable, high-signal items the session-end skill missed; treat each like an inbox item (§1). Best-effort — skip on any error. Skip this section entirely if you don't use per-repo buffers.

3. Refresh the graph (OPTIONAL — only if you use graphify)

This hub works fully as plain markdown + [[wikilinks]]; the knowledge graph is an optional accelerator. If you use graphify, refresh it incrementally:

graphify __OMEGA3_HOME__ --update

graphify reads its LLM backend from your environment (an API key or a cloud provider profile) — configure that per graphify's own docs; nothing about it is baked into this hub. If graphify is unavailable or errors, note it in the digest and continue — the rest of distill does not depend on it. Skip this section entirely if you don't use graphify.

4. Update journal rollups

Read journal/*.md entries since last-distill plus existing journal/rollups/*.md. For each initiatives: slug seen, create/update journal/rollups/<slug>.md: a running, promotion-ready narrative — situation, my role, drivers, actions to date, and quantified impact (aggregate results across entries). Update journal/rollups/INDEX.md.

5. Generative pass — propose insights (REVIEW-GATED)

Look for cross-system patterns, relations, contradictions, and recurring themes — use graphify query over the graph (if you use it) and the notes changed this run. For each candidate insight:

1. Skip if it's already covered by a note in insights/.
2. Skip if it matches anything in .omega/rejected.md (by slug or fingerprint/claim).
3. Otherwise write it to inbox/proposals/<slug>.md from templates/insight.md, filling confidence: and provenance: (which notes/edges it's built from; EXTRACTED vs INFERRED). Never write to insights/ directly — that only happens via /omega3-review. Be selective: a few high-value proposals beat many shallow ones.

6. Update indexes

Ensure every folder's INDEX.md reflects reality (added/merged/renamed notes).

7. Marker

Rewrite .omega/last-distill:

date -u +%Y-%m-%dT%H:%M:%SZ > .omega/last-distill

(keep the leading comment line).

8. Digest + commit + push

Append a dated block to .omega/distill-log.md (see its format) summarizing: inbox drained (counts + slugs), buffers harvested, graph status, rollups updated, insights proposed, and anything left unprocessed/flagged. Then:

git add -A
git commit -m "distill <YYYY-MM-DD>: <N inbox, M insights proposed, rollups K>"
git push

Guardrails

• Idempotent-ish: re-running the same night must not create duplicates (dedup in §1/§4).
• Never delete or overwrite curated systems//knowledge//insights/ content wholesale — merge additively.
• Honor the redaction rule on everything you write.
• If the working tree has unexpected uncommitted user edits, commit them separately or leave them — never discard.