Video streaming

Video Streaming for Mobile

Instructions

Mobile video requires adaptive bitrate (ABR) streaming -- the client switches between renditions based on bandwidth. Serve HLS on iOS (mandatory) and HLS or DASH on Android (HLS is simplest cross-platform).

1. Packaging

Source file -> transcoded into multiple renditions -> packaged as HLS/DASH -> stored on object storage -> served through a CDN.

Renditions for UGC-style content (rules of thumb):

Resolution	Bitrate (video)	Audio
240p	250 kbps	64 kbps
360p	500 kbps	64 kbps
480p	1.0 Mbps	96 kbps
720p	2.5 Mbps	128 kbps
1080p	4.5 Mbps	128 kbps

• Codec: H.264 (wide compatibility) + HEVC or AV1 for newer devices.
• Segment length: 4–6 seconds; shorter (2 s) only for low-latency HLS.
• Keyframe-aligned segments across renditions so the player can switch cleanly.

2. Pipeline

upload -> transcode worker (ffmpeg / AWS MediaConvert / Mux) -> HLS/DASH output
       -> object storage (media/<id>/hls/master.m3u8, media/<id>/hls/720p/*.ts)
       -> CDN

ffmpeg one-shot HLS (simplified):

ffmpeg -i input.mp4 \
  -filter_complex "[0:v]split=3[v1][v2][v3]; \
    [v1]scale=w=854:h=480[v1out]; \
    [v2]scale=w=1280:h=720[v2out]; \
    [v3]scale=w=1920:h=1080[v3out]" \
  -map "[v1out]" -c:v:0 libx264 -b:v:0 1000k \
  -map "[v2out]" -c:v:1 libx264 -b:v:1 2500k \
  -map "[v3out]" -c:v:2 libx264 -b:v:2 4500k \
  -map a:0 -map a:0 -map a:0 -c:a aac -b:a 128k \
  -var_stream_map "v:0,a:0 v:1,a:1 v:2,a:2" \
  -f hls -hls_time 4 -hls_playlist_type vod \
  -master_pl_name master.m3u8 \
  -hls_segment_filename "out/%v/seg_%03d.ts" "out/%v/index.m3u8"

For production, a managed service (AWS MediaConvert, Google Transcoder API, Mux, Bitmovin) saves a huge amount of ops work.

3. CMAF and fMP4

Prefer CMAF (fragmented MP4) segments. The same fMP4 files serve both HLS and DASH manifests; you halve storage and invalidation overhead. Required for low-latency HLS and for Widevine + FairPlay + PlayReady shared DRM.

4. Low-Latency HLS (LL-HLS)

Needed for live/near-live use cases (sub-5 s). Requires partial segments, blocking playlist reload, and CDN support. Most VoD apps do not need this; live-ish apps (sports, auctions, collab) do.

5. Adaptive Bitrate Clients

iOS (AVPlayer): HLS is native; set AVPlayerItem.preferredPeakBitRate to cap on cellular:

let asset = AVURLAsset(url: URL(string: "https://cdn/media/\(id)/hls/master.m3u8")!)
let item = AVPlayerItem(asset: asset)
if networkMonitor.isCellular { item.preferredPeakBitRate = 1_500_000 }
let player = AVPlayer(playerItem: item)

Android: ExoPlayer supports HLS and DASH:

val player = ExoPlayer.Builder(context).build()
val source = HlsMediaSource.Factory(DefaultHttpDataSource.Factory())
  .createMediaSource(MediaItem.fromUri("https://cdn/media/$id/hls/master.m3u8"))
player.setMediaSource(source)
player.prepare()

For Flutter / React Native, wrap the native players; do not roll your own.

6. DRM

Two tiers:

• AES-128 HLS encryption (signed key URL). Low-friction, blocks casual piracy. Not studio-grade.
• Widevine (Android) + FairPlay (iOS) + PlayReady (others) via a license server (AWS, DRMtoday, EZDRM, Axinom). Required for premium content and most licensed VOD.

CMAF + CENC + multi-DRM lets one set of files serve both. License exchange happens between the player and the license server; your backend signs a short-lived license request token.

7. Access Control

• Signed manifest URLs (CloudFront signed cookies or tokenized URLs) so the playlist is not guessable.
• Token bound to user_id + video_id + TTL.
• Segment URLs inherit the same signature (CDN path-based).
• Enforce geo / age / subscription checks at token-mint time, not in-flight.

8. Subtitles and Audio Tracks

• WebVTT sidecars referenced in the master playlist.
• Multi-language audio renditions flagged with LANGUAGE tags.
• Support forced captions for foreign-language segments.

9. Analytics

• QoE metrics: startup time, rebuffer ratio, average bitrate, error rates.
• Use CMCD (Common Media Client Data) headers so CDN logs carry player state.
• Alert on rebuffer ratio > 1% sustained -- likely a bitrate ladder or CDN issue.

10. Edge Cases

• iOS requires HLS for cellular-backed video playback in App Store policy; plan accordingly.
• Background audio: iOS needs audio background mode; the manifest must allow extraction.
• PiP: supported via native APIs; test with your chosen DRM.
• Chromecast / AirPlay: ensure the CDN URLs are reachable from the cast target (some DRM + signed cookies do not play well here).

Checklist

• Renditions cover 240p–1080p (or higher), keyframe-aligned, AAC audio.
• CMAF / fMP4 used where possible for HLS + DASH sharing.
• Segments 4–6 s for VoD; LL-HLS only if live/near-live.
• Client caps bitrate on cellular.
• Manifest and segments served via CDN with signed URLs/tokens.
• DRM tier chosen (none / AES-128 / multi-DRM) and documented.
• WebVTT subtitles and multi-language audio supported.
• QoE metrics (CMCD) collected and alerted on.