Meeting transcription

Meeting Transcription Pipeline

A pipeline that turns meeting recordings into clean, structured Markdown transcripts with named speakers. It is context-aware: each context (e.g. main, customer-x) has its own voice library, language default, and output routing, so the same system serves internal standups and customer calls in different languages without reconfiguration.

This skill is both the knowledge base (how the system works) and the operations manual (how to change it). The canonical scripts live in scripts/ here and are deployed to the worker host's ~/transcribe-pipeline/. It is the reference implementation of the bring-your-own-worker contract in docs/transcription-worker.md — /debrief talks to it only through scripts/debrief_sync.sh (pull / push).

Configuration — the yaml wiring

Three config surfaces, same pattern as every other capability:

1. bridge-config.yaml — the integration block (is the capability on, which script, which contexts — registration + routing):

integrations:
  transcription:
    enabled: true
    sync_script: "skills/meeting-transcription/scripts/debrief_sync.sh"
    default_context: main        # context for audio handed off without an explicit one
    contexts:
      main:                      # → workflow/contexts/main.yaml
        imports: work/imports    # where this context's transcripts land on pull

BRIDGE_IMPORTS / TRANSCRIBE_CONTEXTS env-override the imports dir + context list. /debrief degrades gracefully to its manual path when the block is absent (see the contract doc).

2. infra/transcriptions/topology.yaml — placement (WHERE the pipeline runs):

mode: remote                   # local | remote — THE topology switch (env TRANSCRIBE_MODE overrides)
worker:                        # consulted only when mode == remote
  host: worker-host            # SSH/Tailscale alias — see infra/remotes/
  launchd_label: com.openbridge.transcribe-worker
# local:                       # consulted only when mode == local; all keys optional
#   inbox_dir:       ~/transcribe-inbox        # defaults = the worker's own conventions
#   transcripts_dir: ~/Transcripts
#   library_dir:     ~/transcribe-pipeline/speaker-library
#   contexts_dir:    ~/transcribe-pipeline/contexts

Resolution: TRANSCRIBE_MODE / TRANSCRIBE_WORKER env > topology.yaml > inferred (a worker host resolves ⇒ remote, else local). remote runs every op over ssh+rsync; local runs plain filesystem cp/mv with no SSH (the compute stack is still required). An unknown mode fails loud. Schema + guide: infra/transcriptions/README.md.

3. workflow/contexts/<ctx>.yaml — per-context settings. A context "participates in transcription" iff it carries an integrations.transcription: block (schema: IntegrationTranscription in workflow/contexts/_schema.yaml):

integrations:
  transcription:
    language: auto        # de | en | auto (multi-window vote)
    library: main         # → speaker-library/<library>/*.npy on the worker
    output: main          # → ~/Transcripts/<output>/ on the worker
    notify: false         # completion ping (e.g. iMessage on macOS)
    # mic_speaker: Alice  # display name for the dual-track mic channel (default "Me")

The skill does not maintain its own context directory — two files joined-by-slug drift silently. Discovery rule (codified — run this whenever you need to know what contexts exist):

for f in workflow/contexts/*.yaml:
    data = yaml.safe_load(f)
    if data.get("integrations", {}).get("transcription"):
        ctx = data["id"]                              # slug, e.g. "main"
        settings = data["integrations"]["transcription"]
        # → this context is live; use settings

The worker doesn't parse the bridge yaml directly (its cfg() helper is awk-based and can't read nested keys). The deploy step closes that gap: scripts/extract_runtime_contexts.py reads the bridge yamls, flattens each integrations.transcription block into a flat runtime yaml per context (language, library, output, notify, mic_speaker), and rsyncs the result to ~/transcribe-pipeline/contexts/. The worker keeps its simple parser; the bridge keeps a single source of truth.

The 30-second model

capture machine (optional)     worker host                        Claude
──────────────────────────     ───────────                        ──────
2-track recorder or any  ──►   ffmpeg split (mic-L / meeting-R)
mixed recording                whisper.cpp large-v3 (Metal GPU)    ASR
~/Recordings/meetings/<ctx>/   + pyannote diarize on MPS (+embeddings)
        │                              │
launchd transcribe-bundler.sh  speaker_naming.py ◄── speaker-library/<ctx>/*.npy
   rsync per context                   │
        ▼                      merge_transcripts.py → transcript-raw.md
worker:~/transcribe-inbox/<ctx>/       │   (the NAKED transcript = worker output)
launchd transcribe-worker.sh   deliver ~/Transcripts/<ctx>/<ts>.md
                                       │
                                       ▼   NO summary on the worker
                          in-session /debrief → context-aware summary + tasks

Key idea — identity comes from voice, not text. Speakers are matched by pyannote voice-embedding (cosine ≥ 0.60) against a per-context library, so "Alice" is always labelled Alice even when Whisper mishears her name.

Second key idea — transcription is mechanical, interpretation is contextual. The worker produces ONLY the naked transcript (ASR + diarization + voice-match — no bridge context needed). The summary/tasks are made by the in-session /debrief, which has the bridge loaded (name conventions, ecosystem, board, open issues, contexts, prior meetings). A headless worker-side summarizer was removed for being context-blind: it distorted names and significance.

When to read which reference

This SKILL.md is the map. For anything beyond the quick operations below, open the matching reference — they hold the detail so this file stays scannable.

You need to…	Read
Understand data flow, file layout, every script's role	references/architecture.md
Add a context, add/re-train a speaker, reprocess, extend the pipeline	references/operations.md
Know what you need first (worker / capture / bridge prerequisites)	references/deployment.md § Prerequisites
Provision a worker host from scratch (venv, HF token, launchd)	references/deployment.md
Fix diarization bleed, OOM, language misdetect, delivery failures	references/troubleshooting.md

Core concepts

• Context = the inbox subfolder a recording lands in. Defined by the integrations.transcription block of workflow/contexts/<name>.yaml (see Configuration above). To add a context: add the yaml, re-deploy.
• Voice library = speaker-library/<context>/<name>.npy — 256-dim pyannote embeddings, one file per known speaker (can stack multiple samples). Bootstrapped once per speaker, then reused forever.
• Bilingual = Whisper auto-detects language (or the context forces it); the transcript frontmatter records it; the summary prompt (prompts/meeting-summary-{de,en}.md) is chosen by /debrief to match.
• Summary = NOT here. The worker does no summarizing — it delivers the naked transcript only. Summary + task-extraction happen in the in-session /debrief. summarize.py survives only as a manual headless fallback; prompts/meeting-summary-{de,en}.md is the /debrief summary template. The mechanical scripts/anchor_transcript.py (per-utterance anchors + .index.tsv, makes a transcript addressable) is owned here but invoked by /debrief's gold path; the interpretive ↪ evidence-linking stays in-session.
• Engine = hybrid (GPU). ASR runs on whisper.cpp large-v3 (Metal) via asr_whispercpp.py; diarization + the 256-d embeddings run on pyannote/MPS via diarize_assign.py — the same embedding space the library was built in, so libraries keep matching. A CPU-only fallback stays one env var away: TRANSCRIBE_ENGINE=whisperx. Rationale + measurements: references/architecture.md § Engine.

Quick operations

Most requests map to a playbook in references/operations.md. The headlines:

Transcribe a meeting — drop the recording into the context's inbox; the worker auto-runs. Manual: <worker>:~/transcribe-inbox/<ctx>/<ts>/meeting.mp3

• touch .READY. No 2-track? A mono/stereo-mix file still works (diarization separates by voice; set tracks: single in the manifest — the sync script's push does this automatically). Always confirm the right context first.

Add a context (bridge-side YAML + remote dirs + recording dir):

1. Add workflow/contexts/<name>.yaml (copy from _template.yaml — fill schema_version, scope, id, and an integrations.transcription block). Validate: check-jsonschema --schemafile workflow/contexts/_schema.yaml workflow/contexts/<name>.yaml.
2. Bootstrap the remote dirs + recording folder: bash skills/meeting-transcription/scripts/add_context.sh <name>.
3. Deploy: extract + rsync (see references/deployment.md § Redeploy).
4. Point your recorder at ~/Recordings/meetings/<name>/ (capture machine).

Add / re-train a speaker — after a meeting is diarized, show the user who each cluster is, then persist the voiceprint:

python bin/speaker_idcard.py --json <inbox>/<ctx>/<ts>/teams_out/teams.json
python bin/apply_speaker_names.py --json <that.json> --out teams-named.json \
   --map "SPEAKER_00=Alice,SPEAKER_01=Bob" \
   --library speaker-library/<ctx> --save-embeddings

Stacking more samples over time sharpens recognition.

Reprocess — delete the bundle's .PROCESSED flag and touch .READY, or re-run the steps manually. Cached teams.json means re-naming is cheap (no re-transcription). See operations.md § Reprocess.

Re-summarize only (manual fallback, e.g. after improving a prompt): python bin/summarize.py --raw <bundle>/transcript-raw.md --out <bundle>/transcript.md --prompt-dir prompts

Bridge to /debrief — scripts/debrief_sync.sh implements the worker contract (docs/transcription-worker.md):

• debrief_sync.sh pull — fetch finished transcripts into the bridge imports dir (prefixed <ctx>-…) where /debrief Phase 1 finds them.
• debrief_sync.sh push <audio> [ctx] — hand an audio file to the pipeline; the transcript arrives async and is collected on the next pull. /debrief runs pull at the start of Find and push for any audio without a transcript. Run it from the machine that can SSH into the worker.

Back up / sync voiceprints — debrief_sync.sh voiceprints pull syncs the per-context speaker-library/*.npy from the worker into the bridge under identity/voiceprints/<ctx>/; voiceprints push restores them onto a fresh worker. These are biometric embeddings (GDPR Art. 9), so identity/voiceprints/ is user data — it never promotes upstream. Git tracking is opt-in per context via your .gitignore; an instance handling customer meetings should route customer voiceprints to that customer's own repo rather than track them here. An optional weekly launchd job keeps the backup fresh (scripts/voiceprint_backup.sh + com.openbridge.voiceprint-backup.plist).

For anything structural (new pipeline stage, model swap, threshold tuning), read references/operations.md § Extend and references/troubleshooting.md — and explain the change to the user before deploying, since the worker runs unattended via launchd.

Hard rules

• Voice libraries are per-context. Never match one context's meeting against another context's voices — that's the whole point of the split. Only the recording user legitimately appears in multiple libraries.
• HF token lives in the worker's keychain (service hf-token-pyannote) or ~/.config/open-bridge/hf-token (chmod 600). Keychain writes over SSH fail (audit session) — use the file fallback when provisioning remotely.
• The worker runs unattended — validate script changes (bash -n, a manual bundle run) before relying on the launchd trigger.
• Customer transcripts route to the customer's repo at the /debrief stage; this pipeline only produces the Markdown.
• Voiceprints are biometric data — never commit them to a shared repo, never promote them upstream.