Jiang video e2e

Jiang Video E2E

Use this when testing or explaining the full path for one video:

Google Drive Colab artifacts
-> committed raw source artifacts
-> canonical source transcript
-> semantic packet outputs
-> internal semantic bundle
-> public source read
-> generated website episode or interview

This is a pipeline map, not a future autonomous-agent persona. Autonomous agents should normally run the narrower skill for their job. This skill is useful when a maintainer asks for one video end-to-end or when we need to test whether the narrower skills compose correctly.

Model Policy

Default to gpt-5.4 for first-pass video parsing, semantic packet completion, and public episode/interview read drafting. Scheduled production wakes should use low reasoning when supported; request escalation only when the source is dense, noisy, or conceptually consequential.

Escalate to gpt-5.5 for detailed QA, source ambiguity, contradiction, strong new Jiang formulations, or possible lens/atlas mutation. Do not use mini-class models for normal source parsing; they are for coordination and cheap comparison only.

The first pass is allowed to be a strong draft. It must preserve exact source refs, signature moments, questions, chronology, and enough evidence for a later strong-model QA or lens pass to improve it without rereading the whole pipeline from scratch.

Stage 0: Colab Has Produced Artifacts

Colab automation belongs to colab-video-pipeline. For normal content agents, assume artifacts already exist locally after Drive sync:

content/sources/raw/youtube/<channel>/<video-id>/
  metadata.youtube.json
  dump.json
  grouped.json
  transcription.json

content/sources/raw/youtube/Interviews/<host-channel-id>/<video-id>/
  metadata.youtube.json
  dump.json
  grouped.json
  transcription.json

If these are missing, stop and hand off to colab-video-pipeline.

Stage 1: Source Ingest

Use jiang-source-ingest.

Before import, check content/workflow/tasks/source-processing-policy.json. Known duplicate reuploads, empty artifacts, and archive-only raw folders should stay in the archive but must not become new website-visible episodes unless a maintainer explicitly requests --force-policy. The E2E and import scripts enforce this; treat a policy refusal as a completed archive decision, not as a metadata or Colab blocker.

Mechanical import creates:

content/sources/videos/<source-slug>/
content/workflow/tasks/<source-slug>/transcript-agent-packets.jsonl

The integration entry point remains:

node ops/scripts/process-video-e2e.mjs --video-id VIDEO_ID --channel @PredictiveHistory
# or, for interview-format sources:
node ops/scripts/process-video-e2e.mjs --video-id VIDEO_ID --channel Interviews/<host-channel-id>

If the orchestrator stops at source import, metadata, or packet preparation, resolve that under jiang-source-ingest.

Stage 2: Boundary Review

If the orchestrator reports pending-boundary-review, use jiang-transcript-boundary-review.

Expected review file:

content/workflow/reviews/<source-slug>/transcript-boundary-decisions.json

Then rerun:

node ops/scripts/process-video-e2e.mjs --video-id VIDEO_ID --channel @PredictiveHistory
# or the same interview command used at ingest

Stage 3: Semantic Transcript Pass

If the orchestrator reports pending-agent-packets, use jiang-agent-transcript-pass.

Expected outputs:

content/workflow/proposals/<source-slug>/packet-*.semantic.json

Validate packet outputs:

node ops/scripts/validate-agent-pass.mjs content/workflow/proposals/<source-slug>/*.semantic.json

Then rerun the orchestrator. When all packet outputs exist, it aggregates:

content/lens/evidence/videos/<source-slug>.semantic.json

Stage 4: Public Source Read

Use jiang-episode-read-writer.

Expected output:

content/lens/episodes/<source-slug>/read.json

The public source is not complete with only transcript, claims, glossary candidates, or semantic bundles. It needs a readable Jiang-voice distillation. Interview reads should preserve interviewer pressure, questions, and conversational context where those shape Jiang's answer.

Stage 5: Episode Publication

Use jiang-episode-publisher.

Expected generated output:

website/src/data/lens/episodes/<source-slug>.json
website/src/data/lens/interviews/<source-slug>.json

Expected routes:

/episodes/<source-slug>/
/episodes/<source-slug>/transcript/
/interviews/<source-slug>/
/interviews/<source-slug>/transcript/

Stage 6: Optional Existing Lens Links

During E2E, do not create or rewrite public lens docs unless explicitly asked. If the episode directly invokes an existing lens point, use jiang-provenance-linker to attach the existing lens-point:* ID to the relevant episode mark.

Required Validation

At the end of a successful E2E test:

node ops/scripts/compile-content.mjs
node ops/scripts/validate-content.mjs
cd website && npm run build

If website UI changed, inspect the rendered episode and transcript pages before handoff.

Boundary

Do not update these as part of ordinary video E2E unless the maintainer explicitly asks:

• website/src/content/docs/lens*.md
• content/lens/canon/
• content/lens/glossary/
• content/lens/ledger/
• content/workflow/proposals/<source-slug>/corpus-impact.json
• cross-episode concept pages

Those belong to corpus impact, concept writing, atlas maintenance, provenance linking, or canon promotion.