Gpu host tuning

Skill air-gapped/skills/.claude/skills/gpu-host-tuning

Audit AND tune Linux/GPU inference hosts — read-only host snapshot (CPU power state, C-states, NUMA topology, PCIe link state, GPU settings, kernel boot params, sysctl, ulimits, IRQ affinity, container runtime), optional pinned-host↔GPU memcpy bench (torch + numactl), and per-lever cheat-sheets to flip settings (governor, EPP, cpuidle, persistence, ECC, hugepages, intel_iommu, NCCL env, tuned-adm profiles, Dell/Supermicro/HPE BIOS guidance). Sits beneath any inference framework (vLLM, sglang, TensorRT-LLM) — about the host, not the framework.From its SKILL.md

Install

npx -y skills add air-gapped/skills --skill gpu-host-tuning

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

5 stars5 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
runs commandsInstructs the agent to run 3 commands, including `./scripts/collect.sh` and 2 more.

SKILL.md

5.9 KB, ~1.4k tokens by cl100k_base, as published. Nobody here has run it

gpu-host-tuning

Host-side tuning + audit for Linux GPU inference servers. Sits beneath any inference framework (vLLM, sglang, TensorRT-LLM, llama.cpp).

Assumes a working host. Driver, nvidia-fabricmanager, and NVLink/NVSwitch must already be healthy — if nvidia-smi doesn't list every GPU, or fabricmanager isn't active (running), that is a bring-up problem, not a tuning one: use nvidia-datacenter-bringup (same inference-host plugin) first. Tuning a half-initialised host produces numbers that describe the fault, not the hardware.

Three modes:

Audit — read-only snapshot
Bench — ground-truth pinned-host↔GPU memcpy ceiling
Tune — apply individual levers from the cheat-sheet

This file is a pointer map. The actual logic lives in scripts/ and the authoritative references in references/.

Quick start

# From the skill directory — typically ~/.claude/skills/gpu-host-tuning
# (personal) or .claude/skills/gpu-host-tuning (project install).

# Audit (read-only, ~60s)
./scripts/collect.sh

# Audit + pinned-memcpy bench (needs torch + CUDA, ~5 min)
./scripts/collect.sh --bench

The script prompts for the output parent dir on first interactive run and remembers the choice. Override via --out <dir> or HOST_AUDIT_DIR=<dir>. Default snapshot dirname is gpu-host-tuning-<host>-<UTC>.

What the snapshot captures

One file per probe, numbered by section. See references/probe-interpretation.md for the full file-by-file decoder.

Section	What
`00-09` meta	collector version, run timestamp, args
`10-19` system + firmware	dmidecode (BIOS, CPU, memory DIMMs), lshw, /sys/class/dmi
`20-29` CPU + power + C-states	governor, EPP, intel_pstate / amd_pstate, cpuidle states + disable mask, turbostat 5s residency, microcode, vulnerabilities, thermal zones
`30-39` memory + NUMA	numactl -H, /proc/meminfo, THP, numa_balancing, vm tunables, hugepages
`40-49` kernel + limits	uname, /etc/os-release, /proc/cmdline, sysctl -a, ulimit, /sys/devices/system/cpu/vulnerabilities, dmesg, IRQ affinity, env vars in vllm processes
`50-59` PCIe	lspci tree + verbose, AER counters, link width/speed for every NVIDIA device
`60-69` GPU	nvidia-smi -q full, topo -m, nvlink --status, clocks/power/ECC, dmon 5s, dcgmi diag
`70-79` network	NICs, IB (ibstat / ibv_devinfo), ethtool ring sizes, RDMA links
`80-89` storage	lsblk, NVMe id-ctrl, smartctl, mount flags, io scheduler
`90-99` container runtime	containerd version, CDI specs, cgroup v2, kubelet config, RKE2 config

Three modes — what each maps to

Mode	What	Reference
Audit	`./scripts/collect.sh` writes the snapshot directory	`references/probe-interpretation.md` decodes each numbered file
Bench	`./scripts/collect.sh --bench` adds the pinned-memcpy CSV	`references/session-findings.md` lists baselines per chassis
Tune	No script — apply individual levers from the cheat-sheet	`references/recommended-tunings.md` (lever-by-lever) and `references/tuned-profiles.md` (apply via `tuned-adm`)

When to use which reference

Goal	Read
Apply NVIDIA's stock DGX tunings	`references/tuned-profiles.md`
See exactly what NVIDIA's settings packages flip (per-platform JSON, GRUB drop-ins, sysctl, units)	`references/nvidia-dgx-config-decoder.md`
Run a proper bring-up flow	`references/bringup-recipe.md`
Find the lever the audit flagged	`references/recommended-tunings.md`
Decode an audit output file	`references/probe-interpretation.md`
Tune a Dell XE9680 (H100/H200, SPR/EMR)	`references/dell-xe9680.md`
Tune a Dell XE9780 / XE9780L (B200/B300, Granite Rapids)	`references/dell-xe9780.md`
Understand why cpufreq/cpuidle is empty inside a cloud VM	`references/virt-and-cloud-quirks.md`
See measured baselines from real boxes	`references/session-findings.md`

Comparing two snapshots

Two snapshots on the same host (e.g., pre-tune and post-tune) can be compared with diff -ruN snap_pre/ snap_post/. For a structured impact ranking, use references/probe-interpretation.md to interpret deltas.

Companion skills

vllm-nvidia-hardware — per-SKU specs (HBM, TDP, NVLink, PCIe gen)
vllm-deployment — K8s manifest authoring, cache mounts, probes
vllm-performance-tuning — vLLM-side knobs (above this skill's layer)

What ships with it: 13 files

172.2 KB alongside SKILL.md, 2 of them executable

references/

scripts/

bench_pinned_memcpy.pyruns5.8 KB
collect.shruns24.8 KB