Audio music

Audio Music — local instrumental generation

Render a natural-language music prompt into one coherent track via MusicGen + primed continuation. You (the agent) are the composer: turn the prompt into a JSON score, then run the music CLI. Paths below are relative to this skill directory (prefix with the skill's absolute path if not cd'd in).

One-time setup

cd skills/audio-music
uv venv .venv                                # Python 3.11+
# NVIDIA GPU: install the CUDA torch build FIRST (see requirements.txt header), then:
uv pip install --python .venv -r requirements.txt

scripts/music.sh auto-uses .venv when present. First run downloads the MusicGen weights. Device defaults to MPS on macOS, CUDA elsewhere; override with --device {mps,cuda,cpu}.

Procedure

1. Gather inputs (ask only if truly ambiguous): the prompt (mood, instruments, genre, energy arc), duration in seconds (default 90), model (medium default / large), and stereo (default, lusher) vs mono (faster).
2. Compose the score (see references/composing.md). Write it to a file, e.g. scores/<slug>.json.
3. Validate fast: scripts/music.sh --score <file> -o /tmp/x.mp3 --dry-run. The engine prints VALID. or SCORE INVALID (N errors) with one actionable line per problem — fix the JSON and re-run until VALID.
4. Check the render-time budget (below). If it fits your time budget, render it; otherwise run it in the background or hand the user the command.
5. Render: scripts/music.sh --score <file> -o <out.mp3> --model <repo> -v
6. Verify with the audio-lens skill (silence, clipping, spectrogram) and report the path.

A <name>.score.json sidecar is written next to every render, recording exactly what was rendered.

Score schema

{
  "model": "facebook/musicgen-stereo-medium",  // set from the model/stereo choice (see table)
  "continuation_seconds": 5.0,    // tail of prev section used to prime next; MUST be < shortest section
  "guidance_scale": 3.0,          // classifier-free guidance; 3 is a good default (higher = more literal)
  "temperature": 1.0,             // sampling randomness
  "top_k": 250,
  "crossfade_ms": 250,            // overlap at each section join
  "fade_in_ms": 250,              // ramp up from silence at the very start
  "fade_out_ms": 2500,            // ramp down to silence at the end (graceful ending)
  "global": {
    "style": ["cinematic orchestral", "symphonic strings", "film score"],  // applied to EVERY section
    "avoid": ["percussion", "vocals with lyrics", "pop"]   // recorded only; NOT yet steered (see Limits)
  },
  "sections": [
    { "name": "Intro – …",  "seconds": 16, "style": ["…"] },
    { "name": "Build – …",  "seconds": 16, "style": ["…"] },
    { "name": "Climax – …", "seconds": 16, "style": ["…"] },
    { "name": "Resolve – …","seconds": 16, "style": ["…"] }
  ]
}

Each section's MusicGen prompt = global.style + section.style, comma-joined and de-duplicated. A complete worked example ships at examples/epic_fantasy.json (a "mournful violin building to a hopeful finale" arc) — read it to see the structure and vocabulary in practice. The full composing guide (arc shaping, prompt vocabulary, model selection) is in references/composing.md.

Model selection

Choice	Stereo (default, lusher)	Mono (faster)
medium (default)	facebook/musicgen-stereo-medium	facebook/musicgen-medium
large (best quality, slow)	facebook/musicgen-stereo-large	facebook/musicgen-large
small (quick tests / CPU)	facebook/musicgen-stereo-small	facebook/musicgen-small

Pick the final model up front and render the whole piece with it — you cannot prototype on small and finalize on large (different checkpoints, and continuation is per-model).

Render time & how to run it

On Apple Silicon (fp32, MPS): every run pays a one-time ~3–4 min cold warmup on section 1, then warm per-section gen ≈ mono-medium 2.5–3× the section's audio length, mono-large ~4×, and stereo ≈ ~2× the mono figure.

Model	~90 s song	~45 s song
mono-medium	~7–8 min	~5 min
stereo-medium	~13 min	~8 min
mono-large	~12–15 min	~9 min
stereo-large	~25 min+	~14 min

NVIDIA CUDA is typically much faster; CPU is far slower (use musicgen-small).

If a render exceeds your single-call time budget, run it in the background and redirect to a log — never pipe through grep/head/tail (a pipeline's exit status is the filter's, so an OOM-killed render can masquerade as success):

scripts/music.sh --score <file> -o <out.mp3> --model <repo> -v > render.log 2>&1

Then act on the real exit code: 0 = success (confirm the file exists); 137 = OOM SIGKILL → just retry (cheaper once the model is cached); 2 = invalid score → read the log, fix the JSON, re-run.

Limits (be honest with the user)

• avoid tags are recorded but not yet actively steered (MusicGen has no native negative prompt) — fold critical exclusions into positive phrasing.
• MusicGen won't compose a literal final cadence on command — the fade_out_ms tail is the ending.
• Coherence degrades over very long pieces; beyond ~2–3 min the mood holds but motifs drift.
• One render at a time — each music process loads a full model; two concurrent renders will OOM (especially large). Queue them sequentially.
• Non-lexical vocal pads (write "ethereal female vocal pads ahh ooh") are fine; MusicGen cannot sing actual words — never write lyrics.