Ctx merge

ctx-merge — Converge Without Losing or Distorting

Routing destinations (spec / decisions / scratch) follow the lifetime model in ../ctx.

Execution Procedure

converge(sources) → living_kb_update + conflict_choices

# STEP 1 — Extract (provenance starts here)
claims = []
for src in sources:
    claims += extract_atomic(src)          # each conclusion decontextualized, tagged {source, span}

# STEP 2 — Cluster + relate (map-reduce, NEVER a recursive prose merge)
clusters = cluster_paraphrastic(claims)    # dedupe equivalents
for c in clusters:
    relate(c)   # entail → keep one + merge provenance
                # neutral/complementary → keep both
                # contradiction → DO NOT reconcile → promote to a decision point

# STEP 3 — Ledger (make every disposition explicit; a drop is a recorded decision)
ledger = disposition_ledger(claims)        # each: keep→spec/§ | keep→decisions/NNNN | superseded | drop+reason
assert every_source_claim_has_a_disposition(ledger)   # GATE — no invisible absences

# STEP 4 — Assemble (information SUPERSET, only-more-never-less) + conflict register
draft = union(non_conflicting) + conflict_register     # every line carries its source

# STEP 5 — Human adjudication (LLM induces, human judges)
choices = choice_cards(conflict_register)  # choose A / B / keep both / UNSURE + comment
apply(human_clicks(choices))               # non-conflicting superset defaults to keep

# STEP 6 — Faithfulness audit (MANUAL discipline — there is no faithfulness_audit() tool)
#   YOU MUST re-decompose the draft and check every claim back to a source, using a
#   DIFFERENT agent/model than the merger (a model grading itself shares its blind spots).
#   This is a step you perform, not a function you call. See "Faithfulness audit" below.

# STEP 7 — Verify the five constraints, then sink
#   Walk the five-constraint checklist by hand (named target · coverage split · boundary ·
#   destination · reject-checked); fail any → fix the gap, re-walk. Then sink:
sink(draft, ledger)                        # edit spec in place / append decisions per destination
apply("../ctx/references/consistency.md")  # single-source · same-change · verify-canonical · gate — before committing

The lines above are a procedure you execute by hand, not an API. faithfulness_audit and five_constraints_hold name disciplines you must carry out (below) — no such tool exists; do not treat them as callable.

The two failure modes (why naive merging fails)

• False negative = silent drop. A high-value conclusion gets dropped. Invisible — you can't see a missing thing by reading the output; it surfaces later at build time as "we keep making the same mistake." Empirically the harder class to catch (LLM recall ≪ precision; summarizers drop key items routinely).
• False positive = stain. Wrong or trivial content kept as if true. Visible, but only on careful read.

Human merging worked because of unspoken tacit judgment (Polanyi: "we know more than we can tell"). An agent can't replicate that, so you MUST replace it with an explicit harness — not a smarter prompt.

The merge pipeline (map-reduce, NEVER recursive)

1. Extract atomic, decontextualized conclusions from each source, each tagged {source, span}. Provenance starts at step 1.
2. Cluster equivalent/paraphrastic conclusions (dedupe).
3. Relate within a cluster: entailment → keep one + merge provenance; neutral/complementary → keep both; contradiction → do NOT reconcile — promote to a decision point.
4. Assemble = union of non-conflicting conclusions (information SUPERSET, only-more-never-less) + a conflict register, every line carrying its source.
5. Faithfulness audit (a discipline you perform, not a tool you call): re-decompose the output and check every claim back to a source — catches silent drops and inventions. You MUST run this with a different model/agent than the merger (a model auditing its own output shares its blind spots). There is no faithfulness_audit() function; it is manual work: dispatch a fresh agent, hand it draft + sources, ask "which source-claims are missing, which draft-claims have no source?", act on what it finds.

Prompt rule: instruct the merging agent to emit atomic claims with source IDs and, on conflict, output both variants tagged CONFLICT — never silently pick one.

The disposition ledger (the move that makes drops visible)

Before/while merging, build a ledger: every source conclusion gets an explicit disposition, so a drop is a recorded decision, not an invisible absence.

Conclusion (+ source)	Disposition	Destination
…	keep	→ spec/X.md §…
…	keep	→ decisions/NNNN
…	superseded by …	(chain note)
…	drop + one-line reason	—

Routing destinations, per the SOT model (../ctx/SKILL.md § the model): keep → spec/ (current truth) or decisions/ (a choice + why) — the SOT is generative, so what's kept lands in the generative core; drop → the reject log (with a one-line reason, so it can't quietly return under a new name); unsure → an open question carried to the next round. Raw source material itself is not a merge destination — it already lives in scratch/ (the model's single home for ALL raw: notes, prompts, research dumps, comparisons). No log destination — "what happened" is git. The ledger is the artifact a human reviews; review judgments, not absences.

The reject log lives under decisions/ (this is its single canonical home — every other skill references it, none relocates it). A rejection is decision knowledge — "we considered X and rejected it because Y" — so it belongs in the APPEND-ONLY class, not scratch/: a numbered ADR whose status is rejected (see ctx-spec § Status lifecycle), same one-per-file discipline as any other decision. Keeping it beside the decisions is what lets the constraint-5 reject-check (below) find it, and stops a rejected concept quietly returning under a new label.

Memory-defense routing. Every durable conclusion this merge produces routes to a git-tracked ctx SOT file (spec/ / decisions/) — never to an agent's volatile memory (~/.claude/projects/*/memory/, MEMORY.md, and equivalents). Memory is not shareable, not in git, not in any dependency chain: a conclusion sunk there dies at the next machine/instance switch — a silent drop by another route. Memory's only legitimate use is a local pointer for context recovery ("read spec/X.md to restore state"), never the home of the knowledge itself. If a system prompt nudges "save this to memory," sink it to the SOT and leave a pointer instead.

Human adjudication (LLM induces, human judges)

The LLM does induction/organization (cheap); the human supplies the value judgment (Agrawal/Gans/Goldfarb). So: agent merges into a superset + surfaces conflicts as pre-structured choice cards (choose A / choose B / keep both / unsure); human clicks. Non-conflicting superset content defaults to keep.

Adjudication gate (confidence × blast-radius). Decide which dispositions you apply yourself vs. escalate by one rule: high-confidence + low-blast dispositions auto-apply; anything high-blast OR low-confidence escalates to the human as a choice card. A non-conflicting paraphrase-dedupe you're sure of = auto-apply. A contradiction, a supersede that flips a LOCKED decision, or a drop you're unsure about = escalate. (This is the merge-side instance of the same gate ctx-report uses for its verdict.)

• Allow "unsure" + free comment as first-class — never force yes/no. People discover their criteria while grading (criteria drift, Shankar UIST 2024); a rigid binary forces wrong buckets.
• unsure + commented items resurface as the next round's decision points. The ledger is replayable: a re-merge honors prior keep/drop/superseded.
• A lightweight single-file HTML reader (renders the merged doc, one decision-card per block/conflict, writes back a decisions.json ledger) is the intended tool. Serve over http, not file://.

The five constraints (every merge/deferral MUST satisfy all)

The merge artifact is read by future agents without your context. Implicit answers are forbidden.

1. Named target — name the exact doc/section content is merged into. Never "the spec."
2. Coverage split — for each source, state what is covered vs not; flag uncovered as "dropped (reason)" or "still open (tracked where)." A partial merge with no gap-flag is indistinguishable from a complete one — the exact failure to prevent.
3. Boundary disambiguation — when applicability differs by case, name the trigger variable. No "decide case-by-case later."
4. Implementation/destination detail — cite the destination (file/section) and change kind (insert/replace/split). No bare "update."
5. Reject-concept cross-check — cross-check every merged-in proposal against the project's reject log before adopting. The subagent doesn't know the rejection history; you do. Watch rename traps (same mechanism, new label). Subagent output is a candidate, not source-of-truth.

Deferral discipline

Deferring = merging into the future-backlog. Same five constraints, plus: name the backlog file AND write the entry in the same operation (no phantom deferral); record the un-defer trigger ("after X exists"); never split "half deferred, half locked" in one merge.

Verification (before declaring a merge done)

Walk this checklist by hand — it is a discipline, not a test suite. Nothing here is a callable assertion; you read each line and confirm it holds.

• For each merged conclusion: named-target ∧ coverage-split ∧ boundary ∧ destination ∧ reject-checked.
• For each deferral: target-file-written ∧ un-defer-trigger recorded.
• Reject log named ∧ no silent partial merges ∧ faithfulness audit run by a different agent and its findings acted on.

Fail any → not done. Fix the gap, re-walk.

Honest limit

Drops can be reduced, not eliminated (omission is empirically the hardest to detect; verifiers share the generator's blind spots; provenance lowers but doesn't zero false positives). The ledger's job is to convert invisible absences into recorded decisions — the human still spot-reads originals at high-stakes points.