Story cut editor skill

Story Cut Editor

End-to-end pipeline: long source → key moments identified → cut → captioned → mobile-cropped → joined into one story.

This skill is the composition of four things that work but were previously separate (youtube-downloader, video-analyzer, ai-viral-scorer, social-clip-captioner), plus a new join stage, built specifically to survive the failure modes each of those hit in practice. Read this whole file before running anything — the order of operations matters and skipping steps is exactly how the failure modes below happen again.

---

The pipeline, in order

1. Ownership check           (before touching anything)
2. Get the source locally    scripts/download_source.py
3. Transcribe ONCE           scripts/transcribe_full.py
4. Find the key moments      → references/scoring-rubric.md (this is YOUR judgment call, not a script)
5. Cut + caption + crop each scripts/cut_and_caption_clip.py  (per segment)
6. Join into one story       scripts/join_clips.py

Three ways to use this skill

Pick based on what the user actually asked for; combine freely (mobile-friendly reframing applies on top of either content mode below).

Mode A — Viral Highlights

"Find the best clips," "make this go viral," "cut the best parts." A handful (3-6) of the strongest self-contained moments, picked against references/scoring-rubric.md (emotional intensity, surprise/payoff, quotability, concrete value), optionally reordered for narrative flow, joined with hard cuts (--transition cut) to match snappy short-form pacing. This is the default workflow in Steps 1-6 below.

Mode B — Mobile-Friendly Reformat

"Make this vertical," "crop for TikTok/Reels/Shorts," "add captions." Orthogonal to content selection — applies via --aspect 9:16 (or 1:1/4:5) on cut_and_caption_clip.py regardless of whether segments came from Mode A or C. Uses face-priority crop with the blur-pad fallback (Step 5). If the user wants the whole video reframed with no cutting, treat that as Mode C with a single signal span covering the entire duration, plus --aspect.

Mode C — Full-Video Clean Edit (remove noise, keep signal)

"Remove the dead air," "cut the filler," "tighten this up but keep the whole story." A different judgment task from Mode A — see references/noise-vs-signal.md. Instead of picking a few best moments, go through the entire transcript and cut only what doesn't earn its place (silence, filler, false starts, redundant restatement, off-topic tangents), keeping everything else in chronological order. Use scripts/detect_noise_candidates.py for a first-pass draft of silence/filler spans, then read through and refine by hand — it doesn't catch fluently-spoken tangents or redundancy. Join with --transition crossfade (short, 0.3-0.5s) rather than hard cuts — a full-video edit has far more cut points than a highlight reel, and hard cuts at that frequency read as jarring.

---

Or run steps 2-3-5-6 in one call once you know the segments (step 4 always requires reading the transcript yourself first):

python3 scripts/build_story.py "SOURCE_URL_or_path" OUTPUT.mp4 \
    --segments segments.json \
    --aspect 9:16 --caption-style karaoke --transition cut

segments.json:

[
  {"start": 656, "end": 706, "label": "stat_reveal"},
  {"start": 396, "end": 460, "label": "user_story"},
  {"start": 90,  "end": 112, "label": "before_grade"}
]

---

Step 1 — Ownership check

Ask (once, briefly) whether this is the user's own content or something they have rights to repurpose, unless it's already obvious. Downloading and re-cutting someone else's video is a copyright question, not just a technical one — this isn't a check the download step should paper over. If they confirm rights, proceed.

Step 2 — Get the source

python3 scripts/download_source.py "URL_OR_PATH" /tmp/story/source.mp4

Accepts a yt-dlp-supported URL or a local file path (it'll just copy the local file). For YouTube specifically:

• --no-check-certificate is baked in (needed for sandboxed/proxied TLS interception).
• If you hit Sign in to confirm you're not a bot or repeated 429s: do not retry with spoofed clients or headers — that doesn't reliably work from data-center IPs and isn't worth escalating. Ask the user for a cookies.txt (references/cookie-export.md) or ask them to download the file themselves and upload it. The script detects this case and prints exactly this guidance.

Step 3 — Transcribe once

python3 scripts/transcribe_full.py /tmp/story/source.mp4 /tmp/story/master_transcript.json --model medium

This is the only transcription pass in the whole pipeline. Every clip's captions get sliced out of this one master transcript later — clips are never re-transcribed individually. This matters: re-transcribing a short isolated clip gives Whisper less context than the full pass and measurably produces more garbled/hallucinated words on the exact same audio. If you ever see a caption with a word in the wrong language or a sentence that doesn't parse, the fix is to re-check the master transcript and re-slice — not to re-transcribe that clip on its own.

--model medium is the floor for accented or noisy speech; small/base will drop or hallucinate words. Go to large-v3 if medium still isn't accurate enough and you can afford the extra time.

Step 4 — Find the key moments (your judgment, not a script)

For Mode A (viral highlights): read references/scoring-rubric.md, then read the full transcript end to end and pick 3-6 self-contained windows using the four-dimension rubric (emotional intensity, surprise/payoff, quotability, concrete value).

For Mode C (full-video clean edit): read references/noise-vs-signal.md instead, and run scripts/detect_noise_candidates.py first for a draft list of silence/filler spans to cut — then read through and refine, since it won't catch fluently-spoken tangents or redundant restatement. Keep the remaining spans in chronological order.

Either way, cross-check candidates against the visual scene map (sample frames every 20-40s across the source) — a quotable line over a dead screen reads worse than the transcript suggested it would.

Exclude anything where the speaker is narrating over someone else's copyrighted video/audio, or reading a third-party article at length — see the rubric doc for the full list of exclusions.

Write your picks to segments.json in narrative order (not necessarily chronological source order — a strong hook from later in the source can lead).

Step 5 — Cut, caption, crop each segment

python3 scripts/cut_and_caption_clip.py source.mp4 master_transcript.json clip_0.mp4 \
    --start 656 --end 706 --aspect 9:16 --caption-style karaoke

• --aspect 9:16 / 1:1 / 4:5 reframes for mobile using smart_crop.py, which tries face-priority cropping first and falls back to a blurred-background pillar/letterbox fill if no face is reliably detected in the segment (e.g. the source goes full-screen on an app/browser demo with no webcam bubble). This fallback exists because forcing a face-crop with no face just zooms into whatever happens to be in the center of the frame — on a real run this produced a clip that was just a blown-up chunk of unrelated background article text. Always spot-check a frame from any clip that used the blur-pad fallback.
• --aspect source skips reframing entirely — use this when a segment is genuinely better left in its native aspect (see the blur-pad note above; sometimes the honest answer is "don't force vertical here").
• --caption-style karaoke (word-highlight) or clean (sentence cards) or none.

Step 6 — Join into one story

python3 scripts/join_clips.py FINAL.mp4 clip_0.mp4 clip_1.mp4 clip_2.mp4 --mode cut

• Verifies every clip probes as a valid, complete file before concatenating, and fails loudly if one doesn't. This check exists because a clip truncated by a killed/timed-out render will otherwise make ffmpeg's concat demuxer silently stop early — the first time this pipeline was run by hand, exactly this happened: one clip got cut off mid-encode, and the "joined" output silently came out matching only the first clip's duration instead of the full combined length. Don't skip this verification even when you're confident the renders finished cleanly.
• --mode cut (hard cuts — matches snappy talking-head source material, use for Mode A highlight reels) or --mode crossfade (--transition-duration seconds of fade between each pair — use for Mode C full-video edits, where many more cut points make hard cuts feel jarring).
• All input clips must already share resolution — cut_and_caption_clip.py always targets 1080-width output regardless of which crop strategy it used, so clips coming out of Step 5 already match.

---

Known limitations

• Whisper accuracy on heavy accents or noisy rooms is still the main source of caption errors even with the chunking/context fixes here. Spot-check at least one frame per clip before delivering; don't present captions as verified without looking.
• Face detection is Haar-cascade based — fast, dependency-light, sometimes misses faces at extreme angles or poor lighting. The blur-pad fallback catches the "no face at all" case; it won't catch "wrong face" in a multi-person frame (it picks the largest detected face, which is usually but not always the presenter).
• YouTube bot-detection is IP-based and outside this skill's control — see Step 2.