Viral clip editor skill

Viral Clip Editor

Transform long-form videos into viral-ready vertical clips with word-by-word synced captions, smart cropping, and professional transitions. Works with local files or URLs (YouTube, Twitter/X, etc.).

How It Works

The pipeline is: Source → Transcript → Analyze → Crop → Cut → Caption → Join → Deliver

Every decision (where to cut, where to crop, what to caption) is driven by the actual spoken content, not by guessing from visual frames. This produces clips that tell a coherent story and feel natural.

Quick Start

User: "Turn this 30-min podcast into 5 viral clips"
User: "Download this YouTube video and make shorts from it"
User: "Cut my screen recording into key moments with captions"
User: "Make a vertical recap of this interview"

Prerequisites

Check and install dependencies before starting. Run once per session:

# Check yt-dlp
yt-dlp --version 2>/dev/null || pip install --break-system-packages yt-dlp

# Check faster-whisper (for transcription)
python3 -c "import faster_whisper" 2>/dev/null || pip install --break-system-packages faster-whisper

# Check opencv
python3 -c "import cv2" 2>/dev/null || pip install --break-system-packages opencv-python-headless

# Check ffmpeg
ffmpeg -version >/dev/null 2>&1 || echo "ERROR: ffmpeg not found"

# Check fonts (for captions)
fc-list | grep -i "dejavu" | grep -i bold

Pipeline Reference

Read the detailed pipeline guide for each step:

• {skill_path}/references/pipeline.md — Full pipeline architecture and decision logic

Scripts

The skill bundles reusable Python scripts in {skill_path}/scripts/:

Script	Purpose
pipeline.py	Main orchestrator — runs the full pipeline end-to-end
download.py	Download videos from URLs via yt-dlp
transcribe.py	Whisper transcription with word-level timestamps
analyze.py	Transcript analysis — find viral moments, score segments
smart_crop.py	OpenCV motion tracking + content-aware crop regions
cut_and_caption.py	ffmpeg-based clip extraction with phrase-synced captions
join_with_transitions.py	Concatenate clips with professional transitions
convert_format.py	Format conversion, resolution scaling, container changes

Workflow

Step 1: Get the Source Video

From URL (YouTube, Twitter/X, any yt-dlp-supported site):

python3 {skill_path}/scripts/download.py --url "https://youtube.com/watch?v=..." --output /path/to/output.mp4

From local file: Just use the file path directly. Supported formats: MP4, MKV, WebM, AVI, MOV.

Step 2: Transcribe

python3 {skill_path}/scripts/transcribe.py \
  --input /path/to/video.mp4 \
  --output-dir /path/to/workdir \
  --model base.en \
  --language en

Produces transcript.json with word-level timestamps:

[
  {
    "id": 0,
    "start": 1.4,
    "end": 11.5,
    "text": "Okay, so you have been working on...",
    "words": [
      {"word": "Okay,", "start": 1.4, "end": 1.8, "prob": 0.95},
      {"word": "so", "start": 1.9, "end": 2.0, "prob": 0.98}
    ]
  }
]

Step 3: Analyze & Plan Clips

Read the transcript and identify the best moments. The model should analyze the transcript using these scoring criteria (from references/pipeline.md):

1. Hook potential — Does this segment grab attention in the first 3 seconds?
2. Information density — How much value is packed per second?
3. Emotional resonance — Does it tell a story, surprise, or create urgency?
4. Self-contained — Can this segment stand alone and still make sense?
5. Visual action — Is something happening on screen (not just talking head)?

The model should produce a clip_plan.json:

[
  {
    "clip_num": 1,
    "name": "hook_problem",
    "label": "The Problem Hook",
    "start": 1.4,
    "end": 27.8,
    "reason": "Pattern-interrupt: relatable pain point that stops scrolling",
    "words": [...]
  }
]

Key rules for clip selection:

• Each clip should be 15-60 seconds (sweet spot for virality)
• The first clip MUST be a strong hook — no exceptions
• Clips should tell a story when viewed in sequence
• Avoid segments with lots of "um", "uh", dead air
• Prefer segments where spoken words match visible action

Step 4: Smart Crop Analysis

For 9:16 vertical output, use OpenCV to find the best crop region per clip.

python3 {skill_path}/scripts/smart_crop.py \
  --input /path/to/video.mp4 \
  --plan /path/to/workdir/clip_plan.json \
  --output /path/to/workdir/crop_data.json

Crop strategy matters. Use the right method per content type:

Content Type	Method	Why
Talking head / presenter	Motion tracking	Follows the person speaking
Screen recording / demo	Center-weighted	Keeps readable content in frame
Mixed (presenter + screen)	Motion + content blend	Tracks presenter but covers screen sections
Action / B-roll	Motion peaks	Follows where things are moving

Why not just center-crop? A blind center-crop often cuts off the presenter's face, misses the demo action, or shows empty space. Motion tracking finds where the viewer's attention should be and frames around that. This is the single biggest quality differentiator between amateur and professional vertical clips.

Step 5: Cut Clips with Captions

For each planned clip, extract it with smart cropping and burned-in captions:

python3 {skill_path}/scripts/cut_and_caption.py \
  --input /path/to/video.mp4 \
  --plan /path/to/workdir/clip_plan.json \
  --crops /path/to/workdir/crop_data.json \
  --output-dir /path/to/output/ \
  --style viral  # or: clean, minimal, bold

Caption rules:

• Sync to the actual transcript — words highlight as they're spoken
• Show 2 lines at a time (current phrase + next phrase)
• Active phrase in accent color, upcoming in white
• Semi-transparent dark backdrop behind caption area for readability
• NO text overlays at the top of the video (watermarks, titles) unless explicitly requested
• Font: DejaVu Sans Bold (or system default bold sans-serif)
• Font size: ~40pt after 1080x1920 upscale
• Caption area: bottom 300px of frame

Step 6: Join with Transitions (Optional)

To stitch clips into a single recap video:

python3 {skill_path}/scripts/join_with_transitions.py \
  --clips /path/to/output/Clip*.mp4 \
  --output /path/to/output/FinalRecap.mp4 \
  --transition fade \
  --transition-duration 0.5

Available transitions:

• fade — Quick crossfade (default, 0.5s)
• dip_black — Brief black dip between clips
• dip_white — Brief white flash between clips
• none — Hard cut (fastest, no re-encode of transitions)

Step 7: Format Conversion (Optional)

python3 {skill_path}/scripts/convert_format.py \
  --input /path/to/clip.mp4 \
  --output /path/to/clip_final.mp4 \
  --resolution 1080x1920 \
  --crf 24 \
  --preset medium

Output Specifications

Default output for social media virality:

• Resolution: 1080×1920 (9:16 vertical)
• Container: MP4 (H.264 + AAC)
• CRF: 24-26 (good quality, reasonable size)
• Captions: Burned-in (not soft subtitles) for maximum compatibility
• Audio: AAC 96-128kbps

Important Rules

1. Transcript-first, always. Never pick clip timestamps by guessing from visual frames. The transcript tells you what's being said and when. Use it as the source of truth.

2. Crop per clip, not per video. Different segments of the same video may have the presenter on different sides of the screen, or the demo may shift. Analyze each clip's time range independently.

3. Captions from transcript, not invented. The burned-in captions should match what's actually being said. Use Whisper's word timestamps for sync accuracy.

4. Respect the story. When selecting clips, think about narrative flow. The first clip hooks, the middle clips build, the last clip pays off. Even individual clips should have internal coherence — a beginning, middle, and end within 15-60 seconds.

5. No unnecessary overlays. Don't add watermarks, branding text, or decorative elements unless the user explicitly asks. Clean content > cluttered content.

6. Check memory constraints. Long videos with many drawtext filters can OOM. Process clips one at a time. For clips over 45 seconds, consider splitting into sub-clips and concatenating.

7. Horizontal option. If the user doesn't explicitly ask for vertical/mobile format, ask. Some content (screen recordings, presentations) may work better horizontal.

Full Pipeline (One Command)

For the full end-to-end pipeline in one go:

python3 {skill_path}/scripts/pipeline.py \
  --input /path/to/video.mp4 \
  --output-dir /path/to/output/ \
  --num-clips 5 \
  --style viral \
  --join \
  --join-transition fade

Or from a URL:

python3 {skill_path}/scripts/pipeline.py \
  --url "https://youtube.com/watch?v=..." \
  --output-dir /path/to/output/ \
  --num-clips 5 \
  --style viral

The pipeline script orchestrates all steps: download → transcribe → analyze → crop → cut → caption → (optionally) join.

When the Model Should Read Reference Files

• references/pipeline.md — Read this when you need to understand the detailed architecture, scoring criteria for clip selection, or crop strategy decisions. Read it before running the pipeline for the first time.

• scripts/pipeline.py — Read this when you need to understand command-line arguments, configuration options, or want to debug the pipeline.

Limitations

• Videos longer than 2 hours may need segment-by-segment transcription
• Very low audio quality or heavy music/noise reduces Whisper accuracy
• Screen recordings with no speech cannot be transcribed (use visual analysis instead)
• yt-dlp may not support every site — check with yt-dlp --list-extractors
• Memory-intensive operations (many caption filters) may need clips processed one at a time