Mk multimodal
Skill ngocsangyem/MeowKit/packages/mewkit/src/migrate/modules/codex/root/.agents/skills/mk-multimodal
Production ready. AI Agent Workflow System for Claude Code
npx -y skills add ngocsangyem/MeowKit --skill mk-multimodalAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 15 stars15 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Process images/video/audio/PDFs via Gemini API; generate images/video/speech/music; convert docs to Markdown. Multi-provider fallback. Use for analyze/transcribe/generate/convert on media files.
SKILL.md
5.7 KB, ~1.3k tokens by cl100k_base, as published. Nobody here has run it
Multimodal Analysis & Generation
Path convention: Commands below assume cwd is
$(git rev-parse --show-toplevel)(project root). Prefix paths with"$(git rev-parse --show-toplevel)/"when invoking from subdirectories.
Model selection and commands
Use the configured analysis default for media understanding. Generation routes Gemini → MiniMax → OpenRouter as keys permit. Provider model IDs, preview availability, and pricing change frequently: load references/models-and-pricing.md at execution time and verify preview IDs against the provider dashboard before relying on one.
| Task | Script |
|---|---|
| Analyze, transcribe, extract | scripts/gemini_analyze.py |
| Generate image or video with routing | scripts/gemini_generate.py |
| Generate MiniMax image, video, speech, music | scripts/minimax_generate.py |
| Convert document to Markdown | scripts/document_converter.py |
| Validate keys and dependencies | scripts/check_setup.py |
.agents/skills/.venv/bin/python3 .agents/skills/multimodal/scripts/gemini_analyze.py \
--files <path> --task <analyze|transcribe|extract> [--resolution low-res] [--json]
References
Load only when executing the corresponding step.
| Reference | When to load | Content |
|---|---|---|
| vision-understanding.md | Image analysis | Detection, segmentation, OCR |
| audio-processing.md | Audio/video transcription | Formats, splitting, timestamps |
| models-and-pricing.md | Model selection, cost | Full model table, pricing |
| image-generation.md | Image generation | Nano Banana 2, OpenRouter fallback |
| video-generation.md | Video generation | Veo 3.1, async polling |
| video-analysis.md | Video analysis | Resolution modes, cost math |
| minimax-generation.md | MiniMax generation | Image, video, TTS, music |
| document-conversion.md | Document conversion | Formats, batch mode |
When to Use
Auto-activate on these patterns:
- Task references image (.png, .jpg, .webp), video (.mp4, .mov), audio (.mp3, .wav), or document (.pdf, .docx) files
- Task asks to "analyze", "describe", "transcribe", "extract", "OCR"
- Task asks to "generate image", "generate video", "create image"
- Task asks to "generate speech", "text to speech", "TTS"
- Task asks to "generate music", "create music"
- Task asks to "convert document", "convert PDF to markdown"
- User references a non-text binary file for processing
Do NOT invoke when: text-only files (use Read), Gemini API docs (use mk:docs-finder), already-described image in context.
API key check
Use MEOWKIT_GEMINI_API_KEY (or legacy GEMINI_API_KEY); without it, generation may use
MEOWKIT_MINIMAX_API_KEY. If neither is present, stop with setup instructions.
Analysis Process
Detect media format → select model (default: gemini-2.5-flash) → run script. For >20MB files, use File API. Use --resolution low-res for video (62% savings, video only). Output capped at ~3000 tokens; prefer structured JSON/Markdown.
Generation Process
Provider router auto-selects best available provider by checking API keys:
- Image: Gemini → MiniMax → OpenRouter
- Video: Gemini → MiniMax
- TTS: MiniMax only
- Music: MiniMax only
Force a provider: --provider gemini|minimax|openrouter. Override chain order via MEOWKIT_IMAGE_PROVIDER_CHAIN etc.
Gotchas
- Python venv required: if you get
python3: command not foundor import errors, run.codex/scripts/bin/setup-workflowonce from the project root. - Audio >15 min: Gemini truncates silently. Split first.
- PDF >100 pages: Quality degrades. Process in 20-page chunks.
- Video cost: ~263 tokens/sec at default resolution. Use
--verbosefor cost estimate. - Image gen requires billing: Free tier = no gen. Use MiniMax/OpenRouter as fallback.
- MiniMax video timeout: Hailuo takes 60-180s. Max 600s.
- TTS voices: load the provider catalog at execution time; voice inventories change.
- Temperature: Keep Gemini at 1.0. Lowering causes degraded output.
Failure Handling
- Missing API key → STOP + setup instructions
- Invalid key (401) → STOP + re-check key
- Rate limit (429) → key rotation (
MEOWKIT_GEMINI_API_KEY_2/3/4), else wait 60s - Billing required → provider router tries MiniMax/OpenRouter fallback
Setup
Set MEOWKIT_GEMINI_API_KEY in env or .env file. Legacy GEMINI_API_KEY also works.
Optional: MEOWKIT_MINIMAX_API_KEY for TTS/music and alternative generation.
Budget rule: ≤3000 tokens inline. If response exceeds budget, summarize key findings before returning.