Youtube video context

YouTube Video Context

Turn a public YouTube video, playlist, or search topic into a local context pack before using it as coding, design, QA, or product context. This skill is evidence-oriented: agents should read generated codex_context.md files first, then inspect captions, manifests, or sampled frames only when a claim needs timestamp support.

How It Works

1. Resolve the source as a single video, playlist, or bounded search.
2. Capture metadata with yt-dlp when available.
3. Select one preferred caption track and normalize it into compact timestamped blocks unless captions are disabled.
4. Ask Gemini for schema-constrained analysis that separates observations from implementation hypotheses.
5. Optionally download bounded media frames and a contact-sheet preview when exact local visual evidence is required.
6. Write a portable context pack under .codex/video-context/.

Setup

Run the setup checker from this skill directory:

python scripts/setup_youtube_context.py

To install yt-dlp into the current Python user environment:

python scripts/setup_youtube_context.py --install-yt-dlp

For best visual/audio analysis, set a Gemini key in the shell before ingesting:

export GEMINI_API_KEY="..."

PowerShell:

$env:GEMINI_API_KEY = "..."

Do not paste API keys into prompts, SKILL.md, generated context packs, source files, or issue/PR comments.

Usage

Single video:

python scripts/ingest_youtube_video.py "https://www.youtube.com/watch?v=VIDEO_ID" --focus "extract implementation-relevant UI, mechanics, and timestamped evidence"

Playlist:

python scripts/ingest_youtube_video.py "https://www.youtube.com/playlist?list=PLAYLIST_ID" --max-videos 8 --focus "extract reusable UI and gameplay references"

Search topic:

python scripts/ingest_youtube_video.py "cozy factory automation UI references" --source-type search --max-videos 5 --focus "find implementation inspiration for management-game HUDs"

Fast Gemini-first pass:

python scripts/ingest_youtube_video.py "https://www.youtube.com/watch?v=VIDEO_ID" --skip-captions --json

Local frame evidence:

python scripts/ingest_youtube_video.py "https://www.youtube.com/watch?v=VIDEO_ID" --download-video --max-frames 12 --sample-interval 5

Arguments

• source: Public YouTube URL, playlist URL, or search topic.
• --source-type auto|video|playlist|search: Override source detection. Auto treats playlist URLs and list= watch URLs as playlists; non-URL text as search.
• --max-videos N: Bound playlist/search ingestion. Defaults to 5.
• --focus TEXT: Tell the analyzer what evidence the downstream task needs.
• --model MODEL: Gemini model. Defaults to GEMINI_MODEL or gemini-flash-latest.
• --no-gemini: Skip Gemini direct video analysis.
• --no-captions / --skip-captions: Skip caption download and transcript normalization for a quicker visual pass.
• --languages en,en-US,...: Set caption-language priority.
• --download-video: Download media for frame extraction.
• --sample-interval N: Seconds between sampled frames when downloading media.
• --max-frames N: Bound local frame extraction. Defaults to 12.
• --max-observations N: Bound Gemini evidence items. Defaults to 18.
• --refresh: Refresh cached metadata and transcript artifacts.
• --json: Print a machine-readable result object.

Output

Single-video packs:

.codex/video-context/<video-id>/
  codex_context.md          # first read; concise synthesis and deep links
  transcript.md            # normalized timestamp blocks
  transcript.json          # machine-readable transcript/status
  metadata.compact.json    # agent-friendly metadata
  metadata.json
  gemini_analysis.json     # observations, hypotheses, and status
  run_manifest.json        # options, cache hits, timing, and file list
  captions/
  preview.jpg              # optional contact sheet
  frames/                  # optional local evidence

Playlist and search runs also write:

.codex/video-context/_collections/<collection-slug>/
  collection_manifest.json
  codex_context.md

Present Results To User

Report the context pack path, the first-read codex_context.md file, whether captions/Gemini/frames were captured, and any missing dependency or rejected-key warnings. Keep design or implementation conclusions tied to timestamps when available.

Treat Observed Evidence as model-extracted evidence that still needs spot verification for exact values or labels. Treat Implementation Ideas (Inferred) as hypotheses. Never present an inferred transport, codec, backend, or algorithm as observed fact.

Troubleshooting

• yt-dlp is required for playlist expansion and search discovery: run python scripts/setup_youtube_context.py --install-yt-dlp.
• ffmpeg was not found: frame extraction needs ffmpeg, but normal metadata/caption/Gemini ingestion can continue without it.
• Gemini reports HTTP 400, 401, or 403: rotate or create a Google AI Studio key, set GEMINI_API_KEY, and rerun.
• No timestamps were produced: inspect captions or rerun with a narrower --focus; do not invent exact timing.

References

• references/context-pack-format.md documents the artifact contract.
• scripts/ingest_youtube_video.py is the implementation surface.
• scripts/test_ingest_youtube_video.py contains offline unit tests for discovery and context-pack behavior.
• scripts/benchmark_youtube_video.py scores a fixed public test video for regression checks.

Validate Changes

bun run test
bun run check
python scripts/benchmark_youtube_video.py --out .codex/video-context-benchmark

Use the live benchmark after changing prompts, schemas, transcript handling, or artifact layout. Compare its score, elapsed time, caption count, transcript blocks, timestamp links, concept coverage, and unsupported terms. Treat one model run as evidence rather than absolute proof.