Vllm omni setup

Skill hsliuustc0106/vllm-omni-skills/skills/vllm-omni-setup

Install and configure vLLM-Omni for omni-modality model inference. Use when setting up vllm-omni, configuring the environment, installing dependencies, resolving GPU driver issues, or preparing a machine for model serving.From its SKILL.md

Install

npx -y skills add hsliuustc0106/vllm-omni-skills --skill vllm-omni-setup

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

SKILL.md

3.4 KB, as published. Nobody here has run it

vLLM-Omni Setup

Overview

vLLM-Omni extends vLLM to support omni-modality model inference (text, image, video, audio) with both autoregressive and diffusion architectures. This skill covers installation from source and environment configuration.

Version Variables

Set these before running any commands. Check the vllm-omni quickstart for currently recommended versions.

export VLLM_VERSION="0.20.0"           # vLLM pip package version
export VLLM_OMNI_VERSION="v0.20.0"     # vLLM-Omni release / Docker tag
export PYTHON_VERSION="3.12"           # Python version

Prerequisites

Linux OS (Ubuntu 20.04+ recommended)
Python $PYTHON_VERSION
GPU with appropriate drivers (NVIDIA CUDA, AMD ROCm, Huawei NPU, or Intel XPU)
uv package manager (recommended) or pip
Git

Installation Workflow

Step 1: Create Python Environment

uv venv --python $PYTHON_VERSION --seed
source .venv/bin/activate

Step 2: Install vLLM Base

Select the command matching your hardware:

NVIDIA GPU (CUDA):

uv pip install vllm==$VLLM_VERSION --torch-backend=auto

ROCm (AMD):

uv pip install vllm==$VLLM_VERSION --extra-index-url https://wheels.vllm.ai/rocm/$VLLM_VERSION/rocm722

Step 3: Install vLLM-Omni

git clone https://github.com/vllm-project/vllm-omni.git
cd vllm-omni
uv pip install -e .

Step 4: Verify Installation

from vllm_omni.entrypoints.omni import Omni
print("vLLM-Omni installed successfully")

Quick Smoke Test

Run a minimal text-to-image generation to verify the full stack:

from vllm_omni.entrypoints.omni import Omni

omni = Omni(model="Tongyi-MAI/Z-Image-Turbo")
outputs = omni.generate("a red circle on white background")
outputs[0].request_output[0].images[0].save("test.png")
print("Setup verified - test.png generated")

Docker Installation

For containerized deployment:

docker pull vllm/vllm-omni:$VLLM_OMNI_VERSION
docker run --gpus all -p 8091:8091 vllm/vllm-omni:$VLLM_OMNI_VERSION \
  vllm serve Tongyi-MAI/Z-Image-Turbo --omni --port 8091

Environment Variables

Variable	Purpose	Default
`VLLM_OMNI_LOG_LEVEL`	Logging verbosity	`INFO`
`HF_HOME`	Hugging Face cache directory	`~/.cache/huggingface`
`CUDA_VISIBLE_DEVICES`	GPU device selection	all GPUs
`VLLM_WORKER_MULTIPROC_METHOD`	Worker process method	`spawn`

Troubleshooting

CUDA version mismatch: Ensure your CUDA toolkit version matches the PyTorch build. Check with nvidia-smi and python -c "import torch; print(torch.version.cuda)".

Out of memory on model load: Use gpu_memory_utilization parameter to limit memory. Start with 0.8 and adjust:

omni = Omni(model="...", gpu_memory_utilization=0.8)

Model download failures: Set HF_HOME to a directory with sufficient disk space. Large models (e.g., Qwen3-Omni-30B) require 60GB+ of disk.

References

For GPU compatibility matrix and driver requirements, see references/gpu-compatibility.md