Social clip

Social Clip

Create short-form social media videos optimized for TikTok, Instagram Reels, YouTube Shorts, and other platforms. Two modes: create from scratch or repurpose existing long-form video.

Prerequisites

Requires the video-toolkit skill to be installed for AI asset generation (voiceover, images, music).

TOOLKIT=~/.openclaw/workspace/claude-code-video-toolkit
cd $TOOLKIT && python3 tools/verify_setup.py

Install additional dependencies:

# Whisper for transcription (choose one)
pip3 install --break-system-packages openai-whisper
# OR use Remotion's built-in Whisper.cpp:
npx remotion install-whisper-cpp

# Auto-reframe (optional -- only for repurpose mode)
pip3 install --break-system-packages opencv-python mediapipe

# Clip extraction (optional -- only for repurpose mode)
pip3 install --break-system-packages librosa numpy

Platform Presets

Platform	Resolution	FPS	Duration	Aspect	Safe Zone (top/bottom/left/right)
TikTok	1080x1920	60	15-60s	9:16	150/270/40/40
Instagram Reel	1080x1920	30	15-90s	9:16	120/300/40/40
YouTube Short	1080x1920	60	15-60s	9:16	100/200/40/40
Instagram Story	1080x1920	30	1-15s	9:16	200/250/40/40
Square (Feed)	1080x1080	30	3-60s	1:1	0/0/0/0
Landscape (16:9)	1920x1080	30	3-600s	16:9	0/0/0/0

Each preset includes platform-specific safe zones (pixels) to avoid UI overlap with profile icons, captions area, and navigation. Keep all text and critical visual elements inside safe zones.

Codec and bitrate per platform:

Platform	Codec	Video Bitrate	Audio Bitrate
TikTok	H.264	8M	128k
Instagram Reel	H.264	6M	128k
YouTube Short	H.264	10M	128k
Instagram Story	H.264	6M	128k
Square (Feed)	H.264	5M	128k
Landscape (16:9)	H.264	8M	192k

Create Mode

Step 1: Create Project

cd $TOOLKIT
cp -r templates/social-clip projects/MY_CLIP
cd projects/MY_CLIP
npm install

Step 2: Write Config

Edit src/config/clip-config.ts:

import { SocialClipConfig } from './types';

export const clipConfig: SocialClipConfig = {
  title: 'Why devs are switching to terminal video',
  platforms: ['tiktok', 'instagram-reel', 'youtube-short'],
  primaryPlatform: 'tiktok',
  contentType: 'talking-head',
  durationSeconds: 45,

  hook: {
    style: 'bold-text',
    text: 'Stop using video editors.',
    soundEffect: 'audio/sfx/bass-drop.mp3',
    animation: 'zoom-in',
  },

  scenes: [
    {
      type: 'video',
      durationSeconds: 40,
      media: { src: 'video/talking-head.mp4', fit: 'cover' },
    },
    {
      type: 'text-only',
      durationSeconds: 5,
      text: { headline: 'Try it yourself', body: 'Link in bio' },
    },
  ],

  captions: {
    enabled: true,
    style: 'pop',
    highlightColor: '#FFD700',
    position: 'bottom',
    wordsPerLine: 4,
  },

  audio: {
    voiceover: 'audio/voiceover.mp3',
    backgroundMusic: 'audio/bg-lofi.mp3',
    backgroundMusicVolume: 0.12,
  },

  engagement: {
    cta: {
      text: 'Follow for dev tips',
      position: 'end-card',
      animation: 'slide-in',
    },
  },

  transitions: {
    default: 'zoom-rush',
    durationFrames: 8,
  },

  thumbnail: {
    frameSeconds: 1.5,
    textOverlay: 'Stop Using Video Editors',
    style: 'bold',
  },
};

Scene types: image, video, text-only, split.

Scene media animations: ken-burns-in, ken-burns-out, pan-left, pan-right, static, zoom-pulse.

Step 3: Generate Voiceover

cd $TOOLKIT
python3 tools/qwen3_tts.py \
  --text "Stop using video editors. Here's how I make viral content from my terminal..." \
  --speaker Ryan --tone excited \
  --output projects/MY_CLIP/public/audio/voiceover.mp3 \
  --cloud modal

Speakers: Ryan, Aiden, Vivian, Serena, Uncle_Fu, Dylan, Eric, Ono_Anna, Sohee Tones: neutral, warm, professional, excited, calm, serious, storyteller, tutorial

Step 4: Generate Background Music (optional)

cd $TOOLKIT
python3 tools/music_gen.py \
  --preset lofi \
  --duration 50 \
  --output projects/MY_CLIP/public/audio/bg-lofi.mp3 \
  --cloud modal

Presets: corporate-bg, upbeat-tech, ambient, dramatic, tension, hopeful, cta, lofi.

Step 5: Transcribe for Captions

cd $TOOLKIT
python3 tools/transcribe.py \
  --input projects/MY_CLIP/public/audio/voiceover.mp3 \
  --output projects/MY_CLIP/public/captions/transcript.json \
  --model base.en --word-timestamps

Output format (compatible with @remotion/captions):

{
  "segments": [
    {
      "text": "Stop using video editors",
      "start": 0.0,
      "end": 1.8,
      "words": [
        { "word": "Stop", "start": 0.0, "end": 0.3 },
        { "word": "using", "start": 0.35, "end": 0.6 },
        { "word": "video", "start": 0.62, "end": 0.9 },
        { "word": "editors", "start": 0.92, "end": 1.4 }
      ]
    }
  ]
}

The <CaptionOverlay> component uses createTikTokStyleCaptions() from @remotion/captions to render word-by-word animated captions synced to the transcript timestamps.

Step 6: Sync Timing

cd $TOOLKIT
ffprobe -v error -show_entries format=duration -of csv=p=0 \
  projects/MY_CLIP/public/audio/voiceover.mp3

Update durationSeconds in config to match actual audio: ceil(audio_duration + 3) (3s buffer for hook + CTA).

Asset Manifest

Each asset gets a manifest entry telling the skill what it contains, its role in the clip, and how prominently to feature it.

interface SocialAsset {
  path: string;                    // relative path to file
  type: 'photo' | 'video' | 'audio' | 'text-card' | 'product-shot' | 'screenshot';
  description: string;             // what this shows — Claude uses for placement
  tags: string[];                  // searchable: ['hook', 'highlight', 'b-roll', 'reaction']
  weight: 1 | 2 | 3 | 4 | 5;     // 1=b-roll flash, 3=standard, 5=hero/hook content
  section?: 'hook' | 'body' | 'cta' | 'outro';  // force into clip section
  platform?: string[];             // limit to: ['tiktok', 'reels'] or omit for all
  trimStart?: number;              // trim video from this second
  trimEnd?: number;                // trim video to this second
  beatSync?: boolean;              // true = cut on beat boundaries
  caption?: string;                // overlay text when this asset is shown
  duration?: number;               // suggested screen time
  notes?: string;                  // special instructions
}

interface SocialManifest {
  assets: SocialAsset[];
  defaults: {
    weight: number;
    durationByWeight: {
      1: number; // 0.5s — flash cut
      2: number; // 1s — quick show
      3: number; // 2s — standard beat
      4: number; // 3s — featured moment
      5: number; // 4s — hero content (hook or climax)
    };
  };
}

How Weights Work for Social Clips

Weight	Role	Screen Time	Position	Pacing
5 — Hook/Hero	Scroll-stopper	3-5s	First 3 seconds OR climax	Hold, dramatic entry
4 — Featured	Key moment	2-3s	Body highlights	Beat-synced emphasis
3 — Standard	Good content	1.5-2s	Body sequence	Standard beat cuts
2 — Supporting	Context	0.75-1.5s	Between key moments	Fast cuts
1 — Flash	Texture/energy	0.3-0.75s	Rapid montage	Flash cuts on beats

Smart Placement Rules

• weight: 5 + section: 'hook' → ALWAYS first 3 seconds. Non-negotiable.
• weight: 5 without section → hook if no other hook exists, else climax
• type: 'video' with beatSync: true → cuts land on beat boundaries
• type: 'text-card' → shown as animated overlay, not standalone frame
• weight: 1-2 assets fill rapid-fire montage sections between key moments
• trimStart/trimEnd → only use this portion of the video
• platform: ['tiktok'] → only included in TikTok renders (e.g., vertical-optimized clips)
• If total asset duration exceeds clip length, lower-weight assets get dropped first

Example

const manifest: SocialManifest = {
  assets: [
    {
      path: 'clips/reaction-wow.mp4',
      type: 'video',
      description: 'Friend reacting with genuine surprise and laughter',
      tags: ['hook', 'reaction'],
      weight: 5,
      section: 'hook',
      trimStart: 2.1,
      trimEnd: 4.8,
      notes: 'This is THE scroll-stopper. Use in first 3 seconds.',
    },
    {
      path: 'clips/cooking-montage.mp4',
      type: 'video',
      description: 'Quick cuts of cooking process — chopping, sauteing, plating',
      tags: ['process', 'montage'],
      weight: 3,
      section: 'body',
      beatSync: true,
    },
    {
      path: 'photos/finished-dish.jpg',
      type: 'photo',
      description: 'Beautifully plated final dish with garnish',
      tags: ['result', 'hero'],
      weight: 4,
      section: 'body',
      caption: 'The result speaks for itself',
    },
    {
      path: 'photos/kitchen-candid.jpg',
      type: 'photo',
      description: 'Candid shot laughing in the kitchen',
      tags: ['b-roll', 'personality'],
      weight: 1,
      beatSync: true,
    },
  ],
  defaults: {
    weight: 3,
    durationByWeight: { 1: 0.5, 2: 1, 3: 2, 4: 3, 5: 4 },
  },
};

Step 7: Preview and Render

cd $TOOLKIT/projects/MY_CLIP

# Preview hook frame
npx remotion still src/index.ts SocialClip-tiktok --frame=0 --output=/tmp/hook.png

# Preview mid-clip
npx remotion still src/index.ts SocialClip-tiktok --frame=900 --output=/tmp/mid.png

# Render primary platform
npm run render

# Batch render all platforms
bash ../../tools/batch_render.sh \
  --project . \
  --platforms tiktok,instagram-reel,youtube-short \
  --output out/

Output: out/tiktok/clip.mp4, out/instagram-reel/clip.mp4, out/youtube-short/clip.mp4 + thumbnails.

Repurpose Mode

Take an existing long-form video and extract the best segments as short-form social clips.

Step 1: Analyze Long-Form Video

cd $TOOLKIT
python3 tools/extract_clips.py \
  --input path/to/long-form-video.mp4 \
  --max-clips 5 \
  --target-duration 30-60 \
  --output projects/MY_CLIP/clips-manifest.json

The extraction tool:

1. Transcribes with Whisper to get content segments
2. Scores segments by audio energy (loud = engaging), speech density (words/second), and silence gaps (natural boundaries)
3. Identifies top N segments by engagement score
4. Outputs manifest with timestamps and suggested hook text

Step 2: Auto-Reframe (if source is 16:9)

cd $TOOLKIT
python3 tools/auto_reframe.py \
  --input path/to/long-form-video.mp4 \
  --output projects/MY_CLIP/public/focus-regions.json \
  --mode face-track

Reframe modes:

• center -- static center crop (no analysis needed)
• face-track -- OpenCV Haar cascades or MediaPipe face detection, tracks largest face
• motion-track -- optical flow to find area of highest motion
• rule-of-thirds -- positions crop at left or right third based on content density

The <AutoReframe> component reads focus-regions.json and applies an animated crop window that smoothly pans between focus regions using interpolate().

Step 3: Extract and Configure

Select clips from the manifest and create config with autoReframe.enabled: true:

autoReframe: {
  enabled: true,
  sourceAspectRatio: '16:9',
  focusMode: 'face-track',
},

Or provide manual focus region overrides:

autoReframe: {
  enabled: true,
  focusMode: 'manual',
  manualRegions: [
    { startSeconds: 0, endSeconds: 10, x: 660, y: 0, width: 608, height: 1080 },
    { startSeconds: 10, endSeconds: 25, x: 400, y: 0, width: 608, height: 1080 },
  ],
},

Step 4: Add Captions and Render

Same as Create Mode steps 5-7. Transcribe the extracted audio, enable captions, and batch render.

Caption Styles

Style	Effect	Best for
pop	Words scale up with spring() bounce when spoken	High energy, comedy
highlight	Current word changes to highlightColor, others stay white	Educational, tutorial
karaoke	Words fill with color left-to-right (gradient mask)	Music, storytelling
subtitle	Traditional subtitle bar at bottom, no per-word animation	Professional, clean
bounce	Words drop in from above with gravity-like spring	Playful, casual

Caption config defaults:

• fontSize: 64 for 9:16 platforms, 48 for 1:1
• fontFamily: Inter
• primaryColor: #FFFFFF
• highlightColor: #FFD700
• backgroundColor: rgba(0,0,0,0.6)
• position: bottom
• wordsPerLine: 4

Transition Library

Transition	Effect	Best for
swipe	Directional swipe (up/down/left/right)	Scene changes
zoom-rush	Rapid zoom in then out to next scene	High energy cuts
glitch	Digital glitch distortion	Tech content, edgy
flash	White flash between scenes	Photo reveals
whip-pan	Motion blur simulating camera whip (use @remotion/motion-blur <CameraMotionBlur>)	Fast pacing
fade	Standard opacity crossfade	Calm transitions
cut	Hard cut, no transition	Beat-synced montages

Import custom transitions from lib/transitions/presentations/ directly, never from barrel files. For additional variety, use GL Transitions (remotion-gl-transitions) for trend-matching styles.

Content Types

Type	Description	When to use
talking-head	Single speaker with captions, optional PiP	Advice, opinions, tutorials
montage	Multi-clip rapid cuts with even, custom, or beat-synced timing	Showcases, highlights, travel
before-after	Side-by-side or sequential comparison with split/wipe layout	Transformations, results
listicle	Numbered items with animated reveals and progress indicator	Tips, rankings, recommendations
quote-card	Styled quote with attribution and decorative quote marks	Motivation, testimonials
product-showcase	Product shots with feature callouts and optional rotation	Marketing, launches
tutorial-snippet	Quick how-to with step indicators and progress bar	Recipes, DIY, code tips

Hook Structure (First 3 Seconds)

Every clip must hook within 3 seconds. The hook config controls the opening:

• bold-text -- Large text slams in with spring() scale + shake. Pair with bass-drop SFX.
• question -- Typewriter question engages curiosity. "Did you know...?"
• statistic -- Animated counter rolls to a surprising number with interpolate(). "10x faster"
• motion-burst -- Flash + zoom + reveal. Maximum visual energy.
• sound-effect -- Audio sting with synced visual pulse. Whoosh, ding, dramatic.

Hook animations: zoom-in, shake, flash, scale-bounce.

Engagement Overlays

CTA (Call to Action)

engagement: {
  cta: {
    text: 'Follow for more',    // or "Link in bio", "Subscribe", etc.
    position: 'end-card',       // 'bottom' for persistent, 'end-card' for last 3-5s
    animation: 'pulse',         // 'pulse', 'slide-in', 'fade'
  },
},

Comment Prompt

engagement: {
  commentPrompt: {
    text: 'What do you think? Comment below',
    showAt: 20,                 // Seconds into clip
  },
},

Countdown

engagement: {
  countdown: {
    from: 5,
    showAt: 10,
    style: 'bold',              // 'minimal', 'bold', 'neon'
  },
},

Poll

engagement: {
  poll: {
    question: 'Which do you prefer?',
    options: ['Terminal', 'GUI Editor'],
    showAt: 15,
    durationSeconds: 5,
  },
},

Text Overlays

textOverlays: [
  {
    text: '#DevTips',
    startSeconds: 0,
    durationSeconds: 10,
    position: 'top',
    style: 'hashtag',           // 'title', 'subtitle', 'callout', 'label', 'hashtag'
    animation: 'fade-in',       // 'fade-in', 'slide-up', 'scale-in', 'typewriter', 'bounce-in'
  },
],

Platform-appropriate font sizes:

• Title: 72px (9:16), 56px (1:1), 48px (16:9)
• Subtitle: 48px (9:16), 36px (1:1), 32px (16:9)
• Callout: 56px with background pill
• Label: 32px uppercase tracking
• Hashtag: 36px with # prefix, accent color

Style Configuration

style: {
  colorScheme: 'dark',          // 'dark', 'light', 'vibrant', 'pastel', 'neon'
  accentColor: '#FFD700',
  fontFamily: 'Inter',
  borderRadius: 12,             // For image/video element corners
},

Batch Multi-Platform Export

Render one piece of content across all target platforms in a single command:

bash tools/batch_render.sh \
  --project projects/MY_CLIP \
  --platforms tiktok,instagram-reel,youtube-short,square \
  --output out/

The batch render script:

1. Reads clip-config.ts for the platform list
2. For each platform, runs npx remotion render with platform-specific composition ID
3. Post-processes with FFmpeg for codec/bitrate optimization per platform specs
4. Generates thumbnail stills for each platform
5. Outputs all files to out/{platform}/clip.mp4 + out/{platform}/thumbnail.png

Composition Patterns

Audio Layering

// Voiceover with 1s delay for hook
<Sequence from={fps * 3}>
  <Audio src={staticFile('audio/voiceover.mp3')} volume={1} />
</Sequence>

// Background music (full duration, low volume)
<Audio src={staticFile('audio/bg-lofi.mp3')} volume={0.12} />

// Sound effect at specific time
<Sequence from={0}>
  <Audio src={staticFile('audio/sfx/bass-drop.mp3')} volume={0.8} />
</Sequence>

Scene Transitions

import { TransitionSeries, linearTiming } from '@remotion/transitions';
import { fade } from '@remotion/transitions/fade';
// Custom transitions from lib/transitions/presentations/
import { swipe } from './lib/transitions/presentations/swipe';
import { zoomRush } from './lib/transitions/presentations/zoom-rush';

ALWAYS use <OffthreadVideo>, NEVER <video>.

Cost Estimates

Step	Cost	Notes
Voiceover (Qwen3-TTS)	~$0.01	Per clip, via Modal
Background music (MusicGen)	~$0.02-0.05	Duration-dependent
Image generation (FLUX.2)	~$0.01/image	If AI backgrounds needed
Whisper transcription	Free	Runs locally via whisper.cpp — preferred over cloud APIs
Remotion render	Free	Local render, no license for personal use
Auto-reframe analysis	Free	Local OpenCV processing

Total per clip: ~$0.05-0.10 for AI-generated assets. Free if using only user-provided media.

Batch export: Rendering 4 platform variants costs zero extra -- same Remotion render with different dimensions.

Remotion Ecosystem Packages

Install the official Remotion skills for best results: npx skills i remotion-dev/skills/skills/remotion. This provides 35 production rules covering animations, audio, transitions, text, 3D, and more.

Core Packages

Package	What it does
@remotion/transitions	fade, slide, wipe, flip, clockWipe, iris
@remotion/captions	TikTok-style word-by-word captions
@remotion/media-utils	visualizeAudio(), getAudioData(), getAudioDurationInSeconds()
@remotion/layout-utils	measureText(), fitText(), fillTextBox()
@remotion/motion-blur	<Trail> and <CameraMotionBlur> — use for whip-pan transitions
@remotion/noise	Procedural noise for film grain
@remotion/google-fonts	Google Fonts loading
@remotion/shapes	Geometric shape components
@remotion/paths	SVG path animation
remotion-animated	Declarative <Animated> with Move(), Scale(), Fade()

Community Packages

Package	What it does
Remotion Bits (npx remotion-bits find/fetch)	ParticleSystem (confetti, rain, starfields) for engagement effects, AnimatedText, StaggeredMotion
GL Transitions (remotion-gl-transitions)	Hundreds of GLSL shader transitions from gl-transitions.com — use for trend-matching transition styles
remotion-confetti	Canvas-based confetti with physics

AI Services

Service	What it does	Cost
fal.ai	Single API for video gen (Veo 3.1, Kling 3, Wan 2.2). One key for AI clip generation	$0.05-0.50/sec
whisper.cpp	LOCAL speech-to-text, zero cost. Use for transcription instead of cloud APIs	Free
Suno (via KIE)	AI music with vocals. Better than MusicGen for songs with lyrics	$0.03/song

Producer Intelligence

Audio ducking: Auto-lower music volume when voiceover speaks. Use Remotion's interpolate() on volume based on voiceover audio presence. Essential for voiceover clips.

Loudness normalization: Target LUFS per platform: YouTube -14, TikTok -14, Instagram -14, Podcast -16. Post-process:

ffmpeg -i input.mp4 -af loudnorm=I=-14:TP=-1.5:LRA=11 output.mp4

Film grain via @remotion/noise: Use noise2D() or noise3D() for procedural film grain instead of CSS overlay.

Light leaks via @remotion/light-leaks: WebGL overlays for transitions between scenes.

Tips

1. Hook is everything -- spend 80% of creative effort on the first 3 seconds. If the hook fails, nothing else matters.
2. Captions are mandatory -- 85% of social video is watched with sound off. Always enable captions.
3. Keep text in safe zones -- each platform overlays UI on the video. Use the preset safe zones to avoid text behind profile icons or navigation.
4. Match platform pacing -- TikTok favors cuts every 2-3 seconds. Instagram tolerates slower pacing. YouTube Shorts split the difference.
5. Use zoom-rush sparingly -- one zoom-rush per clip is impactful, five is nauseating.
6. End with CTA -- the last 3-5 seconds should always include a call to action.
7. Batch render always -- if you are making a TikTok, render Reel and Short variants too. Nearly free and doubles your reach.
8. Duration rule -- for scenes with voiceover, set durationSeconds to ceil(audio_duration + 3) (3s buffer for hook + CTA).
9. Test the hook first -- use npx remotion still --frame=0 to preview the opening frame before full render.
10. Sound effects sell it -- a bass-drop or whoosh on the hook frame increases perceived production quality dramatically.