Domain knowledge distiller

Domain Knowledge Distiller

Load Boundary

Load this skill when the user asks for durable learning or reusable knowledge construction, even if the final write decision is not yet clear.

• The user wants high-value resources transformed into reusable domain knowledge, not just summarized.
• The target output is a durable work_contexts/<domain_slug>/ mini wiki.
• The work needs first-principles analysis, reusable principles, patterns, case mechanisms, architecture, micro tactics, or future retrieval metadata.
• The user explicitly asks for future agent use, efficient recall, knowledge routing, or a Karpathy-style LLM wiki.

Do not load it when:

• The user only asks for a normal summary, translation, rewrite, excerpt, or temporary reading note.
• The source is low-value and the user has not asked to preserve it.
• The task is only factual verification, citation collection, or web research without durable distillation.
• The knowledge belongs in a repo-local AGENTS.md, a one-off prompt, or a deterministic script instead of a reusable domain context.

Execution Boundary

Loading the skill and deciding that a source is valuable do not authorize durable writes: source quality does not grant write authorization. A durable work-context update requires either:

• The user explicitly asks to save, absorb, internalize, distill, build a knowledge base, or put material into work_contexts.
• The active task already has explicit authorization to mutate the target memory or Work Context system, and the write stays within that declared scope.

When the user asks only to read, assess, or decide whether material deserves durable knowledge, perform the judgment pass and report the proposed write set without changing files. An explicit "do not save" or read-only boundary always wins. When write authorization exists but placement is ambiguous, make the best low-risk routing decision and ask only when a missing answer blocks safe execution, such as permission to store private material or a conflict with explicit constraints. When the material is noisy, weak, private, or source-bound, prefer a compact source card, low-confidence note, or open_questions.md over a high-confidence principle.

Stable Script Resolution

Resolve bundled helpers once before running them:

SKILL_DIR="${CODEX_HOME:-$HOME/.codex}/skills/domain-knowledge-distiller"

Never assume the current working directory contains this Skill's scripts/ directory.

Judgment Subagent Gate

Before promoting material into durable work-context knowledge, use an independent source-judgment subagent when the environment supports subagents. The main agent owns extraction and writing; the subagent owns judgment.

The subagent must not draft the final knowledge pages. It only returns:

• verdict: write, write_with_limits, defer, or reject.
• reasons: source quality, transferability, future-use value, conflict, privacy, and noise-risk rationale.
• confidence_ceiling: highest allowed confidence for promoted knowledge.
• required_boundaries: non-use cases, transfer limits, or open questions that must be preserved.

If the subagent says defer or reject, do not override silently. Either follow it, or explicitly report the disagreement and downgrade confidence. If subagent tooling is unavailable, perform a separate judgment pass before writing and label it as a fallback in the final report.

Procedure

1. Identify the target domain and source set.
   • Infer the domain from the source set, user language, and likely future retrieval path whenever reasonably possible.
   • Ask one concise question only when the domain or target path cannot be inferred without creating a wrong durable location.
   • Treat explicit requests like "save", "put into work_contexts", or "make this reusable later" as authorization to update that domain context.
2. Build a source overview before extracting claims.
   • Read references/dkd-tv-plus.md when the work includes multiple sources, source uncertainty, or durable writes.
   • Record source identity, domain scope, intended agent use, source quality, and extraction focus before summarizing.
   • Create or update sources/manifest.jsonl and, for new durable distillation runs, write run.json, judgment.json, and the evidence artifacts under sources/runs/<run_id>/.
   • Keep every manifest locator portable: use HTTP(S), a path relative to the owning context root, home://..., codex-home://..., or explicit cross-context work-contexts://...; reject raw absolute paths, ~, and file://. Keep one primary locator in url_or_path and use additional_locators instead of packing multiple URLs into one string.
3. Assess source quality and fit.
   • For external or current sources, verify with web access when needed and keep source links.
   • Read references/source-quality-rubric.md when source credibility, conflict, or copyright handling matters.
   • Read references/source-judgment-subagent.md and run the independent judgment gate before durable writes when source value, confidence, transferability, or noise risk is non-trivial.
   • Run the quality auditor when a manifest already exists, when promoting many sources at once, or when claims involve metrics, benchmarks, rankings, viral examples, small samples, or other noise-prone evidence. Use --allow-legacy only for a read-only inventory of historical unversioned runs; never use it to approve a new promotion.
4. Distill by mechanism, not by surface summary.
   • Use DKD-TV++: parallel candidate extraction, triple verification, accepted/downgraded/rejected decisions, and retrieval pressure checks.
   • Read references/first-principles-distillation.md for deep decomposition.
   • Extract goals, constraints, causal mechanisms, invariants, tradeoffs, failure boundaries, and transfer limits.
   • Keep every extracted item in the candidate pool; do not silently drop weak candidates.
5. Encode for future agent use.
   • Read references/agent-learning-loop.md when building durable knowledge cards.
   • Each important item needs future-use metadata: use cases, non-use cases, retrieval keys, source refs, confidence, and last reviewed date.
   • Accepted units should include Reference, Interpretation, Actual Case, Agent Trigger, Execution, and Boundary sections before being merged into domain pages.
6. Route into a domain mini wiki.
   • Resolve the domain context path in this order:
     1. User-provided target path.
     2. $CODEX_MEMORY_ROOT/work_contexts/<domain_slug>/.
     3. $CODEX_HOME/memories/work_contexts/<domain_slug>/.
     4. ~/.codex/memories/work_contexts/<domain_slug>/.
   • Read references/retrieval-routing.md before creating or restructuring the domain mini wiki.
   • Read references/work-context-root-maintenance.md when the target lives under a shared work_contexts/ root, when adding or reactivating a context, or when changing root-level routing, generated indexes, sync state, router evals, or duplicate source-derived cards.
   • Use templates from templates/ when creating new pages or cards.
7. Validate the result.
   • Run python3 "$SKILL_DIR/scripts/lint_domain_context.py" <domain_context_path> when files are created or edited.
   • For a new run, set RUN_ID to its direct child directory name and run python3 "$SKILL_DIR/scripts/audit_distillation_quality.py" <domain_context_path> --run-id "$RUN_ID" --strict. This current-run gate is mandatory before promotion. Whole-context legacy inventory must explicitly pass --allow-legacy; the default is fail-closed.
   • For shared-root validation, use the target repository's validator when <work_contexts_root>/scripts/validate_work_contexts.py exists. Use python3 "$SKILL_DIR/scripts/validate_work_contexts.py" <work_contexts_root> only as the portable fallback.
   • Run python3 "$SKILL_DIR/scripts/run_static_evals.py" before publishing Skill changes.
   • Perform one retrieval pressure check: given a future task prompt, verify that README.md -> index.md/retrieval.md -> target page finds the relevant knowledge.

Read Only If Needed

• references/first-principles-distillation.md: Use for deep decomposition and abstraction quality.
• references/dkd-tv-plus.md: Use when a durable run needs source overview, candidate pool, triple verification, accepted/rejected audit, and DKD-RIA++ units.
• references/agent-learning-loop.md: Use when converting sources into future-callable knowledge cards.
• references/retrieval-routing.md: Use when creating or maintaining the pages inside one work_contexts/<domain_slug>/.
• references/work-context-root-maintenance.md: Use when maintaining a shared work_contexts/ root, registry, router evals, open risks, portability, or canonical grouping.
• references/source-quality-rubric.md: Use when judging source quality, provenance, conflicts, or copyright boundaries.
• references/source-judgment-subagent.md: Use when delegating durable-write judgment to an independent subagent before the main agent writes or upgrades confidence.
• templates/: Use when creating a new domain context or adding structured knowledge cards.
• schemas/: Use when checking structured fields, source manifest entries, or frontmatter.
• evals/prompts.csv: Use when testing trigger behavior and retrieval quality.
• scripts/lint_domain_context.py: Run after writing or modifying a domain context.
• scripts/audit_distillation_quality.py: Run when checking source quality, future-use metadata, and noise-prone claims before promotion.
• scripts/validate_work_contexts.py: Run after modifying a shared work_contexts/ root, top-level registry, router evals, generated indexes, or canonical grouping indexes.
• scripts/run_static_evals.py: Run when changing the skill, linter, templates, schemas, or eval fixtures.

Output Contract

Return:

• Target domain and source set.
• What was absorbed, what was rejected or deferred, and why.
• Source manifest and run-audit artifacts: source overview, candidates, persisted independent judgment, accepted units, rejected/downgraded items, and source synthesis.
• New or updated reusable knowledge: principles, patterns, architecture, micro tactics, cases, open questions.
• Future-use routing: likely user triggers, non-triggers, retrieval keys, first page to read, registry updates, and pages updated.
• Judgment gate: subagent verdict, confidence ceiling, required boundaries, and any main-agent disagreement.
• Risk and validation surface: router eval cases, open risks, canonical grouping if needed, and portability decisions.
• Evidence: changed files, lint result, work-context root validation when applicable, and one retrieval pressure check.

Validation

• Explicit trigger: "Use domain-knowledge-distiller to absorb this article into work_contexts."
• Implicit trigger: "This failure case is valuable; turn it into principles I can use later."
• Negative control: "Summarize this article in 300 words."
• Evidence check: The output includes source provenance, candidate/rejected audit records, knowledge cards, retrieval keys, updated index/retrieval pages, and successful lint/static eval runs.
• Noise check: Metric-based, benchmark-based, anecdotal, or viral-source conclusions are downgraded, bounded, or routed to open_questions.md unless the evidence includes enough context to resist random-noise overfitting.
• Judgment check: Durable writes and high-confidence promotions have an independent subagent verdict, or a clearly labeled fallback judgment pass when subagents are unavailable.
• Root-system check: When a shared work_contexts/ root is touched, follow references/work-context-root-maintenance.md and run scripts/validate_work_contexts.py.

Common Mistakes

• Producing a polished summary but no reusable mechanism, boundary, or retrieval metadata.
• Copying whole articles into memory instead of storing compact source cards, links, short excerpts, and derived knowledge.
• Putting every domain into one large wiki instead of one work_contexts/<domain_slug>/ per domain.
• Hardcoding a personal absolute path in reusable skill instructions; resolve memory roots from user input or environment variables.
• Making README.md a tutorial; keep it an entry point with shortest read paths.
• Writing principles without non-use cases, source refs, or confidence.
• Promoting only the accepted pages and losing the candidate/rejected trail that explains why other material was discarded.
• Treating retrieval triggers as keyword stuffing instead of task-language, non-trigger, and boundary rules that can be tested.
• Promoting metric movement, benchmark deltas, rankings, or viral examples without checking baseline, sample size, time window, variance, selection bias, and transfer limits.
• Letting the same agent both advocate for the distillation and rubber-stamp its own promotion decision without an independent judgment gate.
• Adding or moving a shared-root context but skipping references/work-context-root-maintenance.md.
• Calling the knowledge base complete because domain pages lint, while root-level routing and portability checks were never run.