Mirage graph coverage

Skill cfregly/gpu-perf-tune/plugins/profile-and-optimize/skills/mirage-graph-coverage

Read-only coverage auditor for the mirage / MPK persistent-megakernel's GENERATED task graph: cross-references DECLARED tensors (all_tensors[...] in kernel_N.cu) against tensors CONSUMED by tasks (inputs/outputs base_ptr in task_graph_N.json) and flags any tensor declared but never consumed as an input - the "feature defined in code but never wired into the task graph" bug class (e.g. RoPE cos/sin tables allocated but consumed by 0 tasks => position-blind attention => incoherent / degenerate-repetition output). A numerics / relL2 harness silently PASSES on this class, so run THIS first when a megakernel compiles + exits 0 but emits incoherent output. Distinct from inference-graph-diff (vLLM FX / Inductor graphs). Triggers on "graph coverage", "task graph audit", "megakernel wiring", "declared but unconsumed tensor", "rope not wired", "missing task", or "megakernel / mirage / MPK / task-graph" with "coverage / wiring / unconsumed / unwired / audit".From its SKILL.md

Install

npx -y skills add cfregly/gpu-perf-tune --skill mirage-graph-coverage

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

3.9 KB, 803 tokens by cl100k_base, as published. Nobody here has run it

mirage megakernel graph-coverage auditor

Catches the missing-wiring bug class in a mirage/MPK generated megakernel: a tensor (or whole feature) is defined and allocated but never wired into a task, so the kernel runs without it. The canonical case is GLM-5.1 on GB300 where the RoPE cos/sin tables were attached but consumed by 0 tasks, so attention ran position-blind and the model emitted fluent-but-incoherent, repetition-collapsing tokens - while every per-kernel numerics test passed.

Quick start

python3 assets/mk-graph-coverage-audit.py <gendir>
# <gendir> holds the generated kernel_<rank>.cu + task_graph_<rank>.json
#   (e.g. mirage demo/deepseek_v3/). Defaults to /work/mirage-perop2/demo/deepseek_v3
# options: --critical rope_cos,rope_sin,<scale-tensor>   --ranks 0,1,2,3

Exit 0 = PASS (every critical tensor consumed as input), 1 = FAIL (a critical tensor is declared but never consumed = missing wiring), 2 = no artifacts.

What it does

Per TP rank, it parses two generated artifacts and diffs them:

declared = all_tensors["NAME"] = NAME; lines in kernel_<rank>.cu
consumed = base_ptr of every all_tasks[].inputs[] / .outputs[] in task_graph_<rank>.json, mapped to the set of consuming task_types + counts

It then classifies each declared tensor and FAILs iff a critical tensor (rope_cos,rope_sin,cos_pos_embed,sin_pos_embed by default) has 0 input consumers.

Reading the output

[OK] <t>: input_refs=N consumer_task_types=[...] - wired (consumed by a task).
[MISSING-WIRING] <t>: input_refs=0 - declared but never read by any task => the bug.
INFO write-only - produced by a task but never read (often a final output like output_token. Suspicious only for an intermediate/feature tensor).
INFO dead - declared, 0 refs at all (allocated scratch / codegen leftover. The nullptr sentinel shows here as a benign false positive).

Worked example (GLM-5.1 on GB300)

Before the wiring fix the auditor FAILs: rope_cos input_refs=0 (rope absent from the graph). After the fix it PASSes: rope_cos/rope_sin input_refs=156, consumer_task_types=[278, 296] (296 = q-rope-apply, 278 = k_pe / KV-cache gather). Same artifacts a relL2 numerics harness would have called clean in both states - graph-coverage is what distinguishes them.

Triage rule

For a megakernel coherence bug (compiles, exits 0, incoherent output), run the graph-coverage audit before building a per-kernel numerics harness. A missing/mis-wired task makes a numerics harness pass while the model stays wrong, the audit names the unwired tensor directly. Only once coverage is clean does a relL2 harness make sense.

Caveat

Verifies structural wiring (is the tensor consumed by a task), not runtime numerical coherence. A PASS removes "feature absent" as the suspect. It does not by itself prove the output is correct.

What ships with it: 1 file

5.2 KB alongside SKILL.md, 1 of them executable

assets/

mk-graph-coverage-audit.pyruns5.2 KB