Kernel vuln analyzer
Skill winmin/kernel-vuln-analyzer/skills/kernel-vuln-analyzer
Analyze Linux kernel vulnerabilities from KASAN/UBSAN/BUG crash logs or CVE descriptions. Performs full root cause analysis, exploitability assessment, patch development, and verification. Use this skill whenever the user provides a kernel crash log, KASAN report, kernel panic trace, syzbot report, or asks to analyze/patch a kernel vulnerability. Also trigger when the user mentions kernel CVEs, kernel exploit analysis, kernel bug triage, or wants to understand if a kernel bug is exploitable. Even if the user just pastes a raw stack trace from dmesg, this skill applies.From its SKILL.md
npx -y skills add winmin/kernel-vuln-analyzer --skill kernel-vuln-analyzerAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
SKILL.md
93.5 KB, as published. Nobody here has run it
Kernel Vulnerability Analyzer
A comprehensive skill for analyzing Linux kernel vulnerabilities — from crash log triage through root cause analysis, exploitability assessment, patch development, and verified fix delivery.
This skill is designed around a hive-mode subagent architecture: break the analysis into parallel workstreams, plan before executing, and coordinate results across agents.
Core Workflow Overview
The analysis follows seven phases. Each phase builds on the previous, but many sub-tasks within a phase can run in parallel via subagents.
Phase 1: Triage & Planning ──→ Phase 2: Source Acquisition
│ │
│ [1.0: Source Version Check] │ [GATE: Source Version Verified]
↓ ↓
Phase 3: Root Cause Analysis ←── Phase 4: Dynamic Analysis (QEMU+GDB)
↓
Phase 5: Exploitability Assessment
↓
Phase 6: Patch Development & Verification
│
│ [6.1: Source Version Re-check]
│ [GATE: QEMU Evidence Required]
↓
Phase 7: Report Generation & Artifact Packaging
Phase 1: Triage & Planning
Goal: Understand what we're dealing with and plan the analysis strategy.
Before writing a single line of analysis, enter Plan mode and create a structured plan. This is non-negotiable — kernel bugs are complex and a wrong turn wastes significant time.
1.0 Verify Source Tree Is Current (FIRST — Before Anything Else)
This is a BLOCKING prerequisite. Before parsing the bug report, before reading source code, before planning — verify the kernel source tree is up-to-date. This takes 5 seconds and prevents all downstream version mismatch issues.
# Step 1: Fetch latest from remote
cd /path/to/kernel-src
git fetch origin --tags
# Step 2: Check how far behind we are
BEHIND=$(git log --oneline HEAD..origin/master | wc -l)
LATEST_TAG=$(git describe --tags --abbrev=0 origin/master 2>/dev/null || echo "unknown")
CURRENT=$(git describe --tags HEAD 2>/dev/null || echo "unknown")
echo "Current: $CURRENT | Latest upstream: $LATEST_TAG | Commits behind: $BEHIND"
# Step 3: If behind → update to latest tag
if [ "$BEHIND" -gt 0 ]; then
echo "WARNING: Local tree is $BEHIND commits behind upstream ($LATEST_TAG)"
echo "Updating to latest tag..."
git checkout "$LATEST_TAG"
fi
# Step 4: Record the base for the report
echo "Analysis base: $(git describe --tags HEAD)"
DO NOT proceed to Phase 1.1 until the source tree is on the latest upstream tag.
If git fetch fails (no network, wrong remote), warn the user and document that the
analysis is based on a potentially stale tree. But ALWAYS attempt the fetch first.
1.1 Parse the Input
The user may provide:
- A raw KASAN/UBSAN/BUG/panic log (most common)
- A syzbot report URL or crash description
- A CVE identifier
- A verbal description of a kernel bug
- A PoC (C code, syzkaller repro, etc.)
For crash logs, extract these key signals:
- Bug type: KASAN (UAF, OOB-read, OOB-write, double-free), UBSAN, BUG(), WARNING, NULL ptr deref, GPF, etc.
- Faulting address and access type: Read/Write, address pattern (NULL page, kernel text, slab, etc.)
- Call stack: The full decoded call trace — this is the most important piece
- Slab cache name: e.g.,
kmalloc-256,skbuff_head_cache,task_struct— hints at the object type - Allocated/Freed stacks: KASAN often shows where the object was allocated and freed
- Kernel version and config: What kernel is this running? What configs are relevant?
- Subsystem: Derived from the call stack — is this networking (net/), filesystem (fs/), drivers, etc.?
1.1.1 Decode the Stack Trace (Required)
If the crash log contains raw addresses (not resolved to source lines), decode it immediately:
./scripts/decode_stacktrace.sh vmlinux < crash.log > decoded_crash.log
If vmlinux is not available, use addr2line on individual addresses:
addr2line -e vmlinux -fip 0xffffffff81234567
The decoded backtrace is a critical artifact — it is needed for:
- The root cause analysis (Phase 3) — source file:line references
- The commit message (Phase 6) — upstream convention requires the decoded trace in the commit body
- The final report (Phase 7) — annotated trace with file:line
Save the decoded trace to logs/decoded_crash.log in the report directory. Keep both the
raw and decoded versions — raw for reproduction, decoded for analysis.
Read references/crash-log-analysis.md for detailed patterns and parsing guidance.
1.2 Acquire the PoC
If the user provides a crash log but no PoC, you need to obtain or create one.
From syzbot: Read references/syzbot-workflow.md for details.
# Download C reproducer from syzbot
curl -sL '<syzbot-repro-url>' -o poc.c
gcc -o poc -static -lpthread poc.c # always static-link for QEMU rootfs
From CVE databases: Search for public PoCs on GitHub, Exploit-DB, or the CVE references.
Write from scratch: If no PoC exists, write a minimal trigger based on the crash call trace:
- Identify the syscall entry point from the bottom of the stack trace
- Set up required preconditions (namespaces, sysctl, devices)
- Issue the triggering syscall sequence
- For race conditions: use pthreads to run concurrent paths
PoC validation checklist:
- Compiles with
gcc -static(needed for minimal QEMU rootfs) - Does it need root? Network? Specific sysctl?
- Does it need
unshare(CLONE_NEWUSER|CLONE_NEWNET)for namespaces? - Is the crash reliable or does it need loop/stress testing?
1.3 Identify the Kernel Subsystem and Source Tree
Do NOT guess the tree from the top-level directory name alone. Many subsystems have
independent maintainer trees even though their code lives under a shared parent directory.
The canonical source of truth is scripts/get_maintainer.pl and the MAINTAINERS file.
Step 1: Run get_maintainer.pl on the affected file
./scripts/get_maintainer.pl --scm --web <path/to/affected/file>
# The "SCM:" line tells you the correct git tree
# The "W:" line tells you the web page / mailing list
Step 2: Cross-reference with the subsystem tree mapping
Some paths are deceptive — always match most-specific path first:
| Source path | Subsystem | Git tree (fixes) | Prefix |
|---|---|---|---|
net/bluetooth/ | Bluetooth | bluetooth/bluetooth.git | PATCH bluetooth |
net/wireless/, drivers/net/wireless/ | WiFi | wireless/wifi.git | PATCH wifi |
net/mac80211/ | WiFi (mac80211) | wireless/wifi.git | PATCH wifi |
net/netfilter/, net/ipv4/netfilter/ | Netfilter | netfilter/nf.git | PATCH nf |
net/bridge/ | Bridge | netdev/net.git | PATCH net |
net/ipv4/, net/ipv6/, net/core/ | Networking core | netdev/net.git | PATCH net |
net/sctp/, net/dccp/, net/tipc/ | Networking | netdev/net.git | PATCH net |
net/can/ | CAN | linux-can/linux.git | PATCH can |
net/nfc/ | NFC | sameo/nfc.git | PATCH nfc |
kernel/bpf/, net/bpf/ | BPF | bpf/bpf.git | PATCH bpf |
drivers/net/ethernet/ | Network drivers | netdev/net.git | PATCH net |
drivers/bluetooth/ | Bluetooth drivers | bluetooth/bluetooth.git | PATCH bluetooth |
drivers/gpu/drm/ | DRM/GPU | drm/drm.git | PATCH drm |
drivers/usb/ | USB | usb/usb.git | PATCH usb |
sound/ | Sound/ALSA | tiwai/sound.git | PATCH sound |
fs/ext4/ | ext4 | tytso/ext4.git | PATCH ext4 |
fs/btrfs/ | Btrfs | kdave/btrfs.git | PATCH btrfs |
fs/xfs/ | XFS | djwong/xfs-linux.git | PATCH xfs |
mm/ | Memory management | akpm/mm.git | PATCH mm |
io_uring/ | io_uring | axboe/linux-block.git | PATCH io_uring |
security/apparmor/ | AppArmor | jj/linux-apparmor.git | PATCH apparmor |
| Others | General | torvalds/linux.git | PATCH |
The trap: net/bluetooth/ is under net/ but does NOT go to netdev/net.git.
Bluetooth patches go to bluetooth/bluetooth.git and are picked by the Bluetooth
maintainer (Luiz Augusto von Dentz). Eventually they flow through netdev/net.git
into mainline, but patches must be submitted to the Bluetooth tree directly.
WRONG: net/bluetooth/l2cap_core.c → "this is net/ → netdev/net.git → PATCH net"
RIGHT: net/bluetooth/l2cap_core.c → get_maintainer.pl → bluetooth.git → PATCH bluetooth
Another trap: Many subsystems also maintain a -next tree for development patches
(e.g., netfilter/nf-next.git, netdev/net-next.git, bpf/bpf-next.git).
A fix may exist in either the fixes tree or the -next tree but not yet in
torvalds/linux.git. You MUST fetch and search both subsystem trees during
deduplication — searching only origin (torvalds) will miss fixes that are
queued in the subsystem but not yet merged into mainline.
WRONG: git log origin/master --grep='nf_tables' -- net/netfilter/ ← only searches torvalds
RIGHT: git fetch nf; git fetch nf-next
git log nf/main nf-next/main origin/master --grep='nf_tables' -- net/netfilter/
Step 3: When in doubt, always trust get_maintainer.pl
# It handles all the edge cases in MAINTAINERS
./scripts/get_maintainer.pl --scm net/bluetooth/l2cap_core.c
# Output will show bluetooth.git, not net.git
Step 4: Identify the relevant kernel version
- Check if the bug exists in mainline, stable, or LTS
- Parse from crash log:
grep 'Not tainted' crash.log
1.4 Create the Analysis Plan
Use Plan mode to structure the work. A typical plan:
1. Clone source tree and checkout relevant version
2. [Parallel] Spawn subagents for:
a. Static analysis: read the vulnerable code path, trace data flow
b. Git archaeology: find the commit that introduced the bug (git log, git bisect)
c. Cross-reference: search kernelctf knowledge base for similar vulnerability patterns
3. Set up QEMU environment with matching kernel config
4. Reproduce the crash with PoC
5. Dynamic analysis with GDB to confirm root cause
6. Assess exploitability
7. Develop and test patch
8. Package report and artifacts
1.5 Subagent Dispatch Strategy
This skill makes heavy use of subagents (the Agent tool) to parallelize work. The guiding principle: plan centrally, execute in parallel, synthesize results.
Parallel-safe tasks (can run as concurrent subagents):
- Source code reading of different files/functions
- Git log / git blame on different paths
- Searching the kernelctf knowledge base
- Compiling kernel in a worktree
- Web research for related CVEs or patches
Sequential tasks (must wait for prior results):
- Dynamic analysis depends on QEMU environment being ready
- Patch writing depends on confirmed root cause
- Patch verification depends on patch being applied
When spawning subagents, always provide:
- Clear, self-contained task description
- All file paths and context needed (the subagent has no memory of your conversation)
- Expected output format
- Use
isolation: "worktree"for tasks that modify files (compilation, patching)
Phase 2: Source Acquisition & Static Analysis
2.1 Acquire and Update Source
In practice, the user usually already has a local kernel source tree. Do NOT blindly clone a fresh repo every time — check what's available first.
Case A: User already has a local kernel source tree (most common)
cd /path/to/existing/kernel-src
# 1. Check current state
git status # any uncommitted changes?
git describe --tags --abbrev=0 # what version is this?
git remote -v # what remote does it track?
# 2. Fetch latest from upstream — ALWAYS do this
git fetch origin
git fetch --tags origin
# 3. Check how far behind we are
git log --oneline HEAD..origin/master | head -20
# If significantly behind (>100 commits), strongly recommend updating
# 4. Update to latest (if tree is clean)
git pull --rebase origin master
# Or if on a specific branch:
git pull --rebase origin <branch>
If the user has uncommitted changes (their own annotations, previous patches, etc.):
git stashfirst, then fetch/pull, thengit stash popafter analysis- Or work on a detached HEAD at the latest tag:
git checkout <latest-tag>
Case B: No local source — clone fresh
# For the appropriate subsystem tree:
git clone <tree-url> /path/to/analysis/kernel-src
cd /path/to/analysis/kernel-src
git checkout <version-tag>
# For full history (needed for git bisect / git blame):
git clone <tree-url> /path/to/analysis/kernel-src
# Shallow clone is faster but limits git bisect:
git clone --depth=1 --branch <version-tag> <tree-url> /path/to/analysis/kernel-src
Case C: Local source exists but tracks a different tree
Sometimes the user has torvalds/linux.git but the bug is in a subsystem tree
(e.g., netdev/net.git). Add it as a second remote:
git remote add net git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
git fetch net
git log net/master --oneline | head -5
2.2 Verify Source Version (Required — Do This Before Any Analysis)
Writing a patch against stale source is a critical mistake: the patch may not apply to mainline, may fix an already-fixed bug, or may be semantically wrong due to context changes. Always verify the source is current before proceeding.
Step 1: Check if the local tree is up-to-date with remote AND fetch subsystem trees
git fetch origin
git log HEAD..origin/master --oneline | head -10
# If output is non-empty → the local tree is behind. Update it:
git pull --rebase origin master
# CRITICAL: Also fetch the subsystem tree for the affected file.
# Use the subsystem tree mapping table above, or run:
# ./scripts/get_maintainer.pl --scm <path/to/affected/file>
# to find the correct git tree URL.
#
# Example for net/netfilter/:
# git remote add nf https://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf.git 2>/dev/null
# git remote add nf-next https://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf-next.git 2>/dev/null
# git fetch nf
# git fetch nf-next
#
# Example for net/core/, net/ipv4/, drivers/net/:
# git remote add net https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git 2>/dev/null
# git remote add net-next https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git 2>/dev/null
# git fetch net
# git fetch net-next
#
# For other subsystems, follow the same pattern with the tree from the table.
# The 2>/dev/null on `git remote add` suppresses "already exists" errors on re-runs.
Step 2: Check if the bug is already fixed — search BOTH origin AND subsystem trees
This is the most important check — someone may have already submitted a fix. A fix may exist in the subsystem tree (e.g., nf.git, net-next.git) but not yet in torvalds/linux.git. Searching only origin will miss these. You MUST search all fetched remotes.
# Determine which refs to search (adjust remote names per subsystem)
# Example for netfilter — replace nf/main nf-next/main with your subsystem refs
SEARCH_REFS="origin/master nf/main nf-next/main"
# Search for patches touching the vulnerable function
git log $SEARCH_REFS --oneline -S '<vulnerable_function_name>' -- <file>
# Search for Fixes: tags referencing the introducing commit
git log $SEARCH_REFS --oneline --grep='Fixes: <introducing-commit-short-hash>'
# Search commit messages for the bug description keywords
git log $SEARCH_REFS --oneline --grep='<key_keyword>' -- <file>
# Check the linux-stable tree for backported fixes
git log --all --oneline --grep='<CVE-number>'
If a fix already exists (in ANY of the searched refs — origin OR subsystem tree):
- Report it — tell the user the bug is already fixed, with the fixing commit hash and which tree it's in
- Verify the fix — read the upstream fix to confirm it actually addresses the root cause
- Skip to Phase 7 — no need to write a new patch; document the existing fix in the report
Step 3: Two-Stage Workflow — Analyze on Crash Version, Patch on Latest
This is a critical distinction that the skill MUST enforce:
Extract the crash kernel version from the log first:
# Parse the crash log for the kernel version
CRASH_VERSION=$(grep -oP 'Not tainted \K\S+' crash.log)
# e.g., CRASH_VERSION="6.12.77"
# Find the closest git tag
CRASH_TAG=$(git tag -l "v${CRASH_VERSION}*" | sort -V | tail -1)
# Or for stable kernels: git tag -l "v$(echo $CRASH_VERSION | cut -d. -f1-2)*" | sort -V | tail -1
# Get current latest upstream
git fetch origin --tags
LATEST=$(git describe --tags --abbrev=0 origin/master)
┌────────────────────────────────────────────────────────────────────┐
│ Crash log says kernel $CRASH_VERSION (e.g., a stable/old release) │
│ Latest upstream is $LATEST (e.g., mainline HEAD) │
│ │
│ WRONG: Write the patch against $CRASH_VERSION and call it done. │
│ RIGHT: Analyze on $CRASH_VERSION, then rebase the fix onto │
│ $LATEST mainline/subsystem-tree HEAD before finalizing. │
└────────────────────────────────────────────────────────────────────┘
Stage 1 — Analyze & Reproduce on the crash version:
# Checkout the crash kernel version for analysis and QEMU reproduction
git checkout "$CRASH_TAG"
# Build, boot in QEMU, reproduce the crash, do root cause analysis
# This ensures you understand the bug in the exact context it was reported
Stage 2 — Fetch latest code and check before writing the patch:
# ALWAYS fetch BOTH origin AND the subsystem tree before writing ANY patch
git fetch origin
git fetch --tags origin
# Fetch the subsystem remote (must have been added in Phase 2.2 Step 1)
# Example for netfilter:
git fetch nf
git fetch nf-next
# Check: does the vulnerable code still exist in the latest version?
git show origin/master:<path/to/vulnerable/file> | grep '<vulnerable_function>'
# If the code has been refactored or removed → the bug may be moot in mainline
# Check: has someone already fixed this in the latest tree?
# CRITICAL: Search BOTH origin AND subsystem refs — a fix queued in the
# subsystem tree may not have reached torvalds/linux.git yet.
SEARCH_REFS="origin/master nf/main nf-next/main" # adjust per subsystem
git log $SEARCH_REFS --oneline -S '<vulnerable_function>' -- <file> | head -10
git log $SEARCH_REFS --oneline --grep='<key_keyword>' -- <file> | head -10
# If bug still exists in latest → write patch against latest subsystem HEAD:
git checkout origin/master
# Or for networking fixes:
git checkout net/main # netdev/net.git main branch
# Or for netfilter fixes:
git checkout nf/main # netfilter/nf.git main branch
Stage 3 — Write the patch against the latest code:
# The patch MUST be based on the latest subsystem tree HEAD
# NOT on the old crash kernel version
git diff > patch.diff # your fix, based on latest code
# Verify the fix also applies to the crash version (for QEMU testing)
git stash
git checkout "$CRASH_TAG"
git stash pop # if it applies cleanly
# Or: git cherry-pick / manual port if context differs
Why this matters:
- Upstream WILL NOT accept patches based on old stable kernels
- The code around the bug may have changed (variable renames, refactors, new callers)
- A patch against an old stable may not apply to the latest mainline at all
- Even if the patch applies, context lines may differ →
git amfails
| Crash kernel version | Analyze on | Write patch against |
|---|---|---|
| Stable (e.g., X.Y.Z) | v$CRASH_VERSION tag | Latest origin/master or subsystem HEAD |
| Mainline (e.g., X.Y-rcN) | v$CRASH_VERSION tag | Latest origin/master |
| Distro (e.g., X.Y.Z-distro) | Distro source | Upstream mainline HEAD |
| net-next / subsystem tree | Subsystem HEAD | Subsystem HEAD (already latest) |
If the local tree is old and behind upstream:
# You MUST update before writing the patch
git fetch origin
git log --oneline HEAD..origin/master | wc -l
# If significantly behind → pull or checkout latest
# Check if the vulnerable file has changed significantly
git diff "$CRASH_TAG"..origin/master -- <path/to/file> | diffstat
# If large diff → the patch context has changed, must write against latest
Step 4: Record the base commit in the report
Always document the exact commit your patch is based on:
echo "Patch base: $(git log --oneline -1 HEAD)" >> report_metadata.txt
# e.g., "Patch base: abc123def456 Merge tag 'net-6.12-rc4'"
This goes in the report's metadata section so anyone applying the patch knows exactly which tree state it was developed against.
If the source tree was already present (e.g., the user has a local clone):
- Still run Steps 1-3 — don't assume it's current
git fetch && git log HEAD..origin/master --onelineis fast and catches stale trees- If the tree is weeks/months behind, warn the user before proceeding
GATE CHECK: Before Entering Phase 3 — Source Version Verified
DO NOT proceed to Phase 3 (Root Cause Analysis) until the source tree version is verified. This gate ensures you are analyzing and will patch against the correct code.
Required Checks (must ALL pass)
echo "=== Source Version Gate Check ==="
# 1. Was remote fetched?
echo -n "1. Remote fetched: "
git fetch origin --tags 2>&1 | tail -1
echo "DONE"
# 1b. Was SUBSYSTEM remote fetched?
# Identify the subsystem remote from Phase 2.2 Step 1 (e.g., nf, nf-next, net, bpf)
echo -n "1b. Subsystem remote fetched: "
# Replace SUBSYS_REMOTE / SUBSYS_NEXT with the actual remote names
SUBSYS_REMOTE="nf" # adjust per subsystem
SUBSYS_NEXT="nf-next" # adjust per subsystem (may not exist for all)
git fetch "$SUBSYS_REMOTE" 2>&1 | tail -1 && echo " $SUBSYS_REMOTE DONE"
git fetch "$SUBSYS_NEXT" 2>&1 | tail -1 && echo " $SUBSYS_NEXT DONE"
# If `git remote` does not list the subsystem → go back to Phase 2.2 Step 1
# 2. What is the latest upstream tag?
echo -n "2. Latest upstream tag: "
LATEST=$(git describe --tags --abbrev=0 origin/master 2>/dev/null)
echo "$LATEST"
# 3. Are we on or ahead of the latest tag?
echo -n "3. Source version: "
BEHIND=$(git log --oneline HEAD..origin/master 2>/dev/null | wc -l)
if [ "$BEHIND" -eq 0 ]; then
echo "PASS — up to date ($(git describe --tags HEAD))"
else
echo "FAIL — $BEHIND commits behind $LATEST"
echo " → Run: git checkout $LATEST"
fi
# 4. Has the vulnerable file changed between current HEAD and latest?
echo -n "4. Vulnerable file changed since HEAD? "
VFILE="<path/to/vulnerable/file>"
CHANGES=$(git diff --stat HEAD..origin/master -- "$VFILE" 2>/dev/null | grep -c '|')
if [ "$CHANGES" -eq 0 ]; then
echo "PASS — no changes"
else
echo "WARNING — file changed, MUST patch against latest"
fi
# 5. Bug already fixed — search origin AND subsystem trees?
echo -n "5. Already fixed? "
SEARCH_REFS="origin/master"
# Add subsystem refs if they exist
git rev-parse --verify "$SUBSYS_REMOTE/main" &>/dev/null && SEARCH_REFS="$SEARCH_REFS $SUBSYS_REMOTE/main"
git rev-parse --verify "$SUBSYS_NEXT/main" &>/dev/null && SEARCH_REFS="$SEARCH_REFS $SUBSYS_NEXT/main"
echo " Searching: $SEARCH_REFS"
git log $SEARCH_REFS --oneline -S '<vulnerable_function>' -- "$VFILE" | head -5
If check 1b shows subsystem remote not configured → go back to Phase 2.2 Step 1. If check 3 shows FAIL → checkout the latest tag before proceeding. If check 5 shows a fix already exists (in ANY ref — origin OR subsystem tree) → report the existing fix, skip to Phase 7.
Common Mistakes This Gate Prevents
| Mistake | Consequence | This gate catches it |
|---|---|---|
| Patching against stale tree | Patch may not apply to mainline | Check 3 |
| Missing a context change | Patch applies but is semantically wrong | Check 4 |
| Duplicate work (mainline) | Bug already fixed in torvalds/linux.git | Check 5 (origin) |
| Duplicate work (subsystem) | Fix queued in subsystem tree, not yet in mainline | Check 5 (subsystem refs) |
| Subsystem remote not fetched | Miss fixes in nf.git / net-next.git / etc. | Check 1b |
| Wrong version in report | Report claims wrong "latest affected" | Check 2 |
2.3 Static Analysis (Spawn as Subagents)
Launch these in parallel:
Subagent A — Code Path Analysis:
- Read the functions in the call stack, starting from the crash point
- Trace the data flow: where does the faulting pointer come from?
- Identify the object lifecycle: allocation, use, free
- Look for missing locks, reference count issues, error path leaks
Subagent B — Git Archaeology:
git log --oneline <file>for recent changes to the affected filesgit blameon the vulnerable lines to find the introducing commit- Check if there are already patches in mainline or -next that fix this
- Look for related fixes in the same area (often bugs cluster)
Subagent C — Knowledge Base Cross-Reference:
- Search
references/kernelctf-knowledge-base.mdfor similar vulnerability patterns - Search
references/vuln-classification.mdto classify the bug type - Check if this subsystem has known exploit primitives
Phase 3: Root Cause Analysis
This is the most critical phase. The symptom (what the crash log shows) often differs from the actual bug.
Common Symptom-vs-Root-Cause Mismatches:
| Crash Symptom | Possible True Root Cause |
|---|---|
| NULL pointer dereference | UAF (object freed, memory reused/zeroed) |
| General Protection Fault | UAF (object freed, slab poisoned with 0x6b6b6b6b) |
| KASAN: slab-use-after-free | Straightforward UAF, but find the race condition |
| KASAN: slab-out-of-bounds | Off-by-one, integer overflow leading to undersized allocation |
| BUG: unable to handle page fault | UAF, double-free, or type confusion |
| WARNING in refcount_t | Reference count underflow — likely a UAF waiting to happen |
| UBSAN: shift-out-of-bounds | Integer handling bug, possibly exploitable for info leak |
Always ask: "What is the actual invariant violation, not just the symptom?"
Read references/vuln-classification.md for the full taxonomy of kernel vulnerability classes
and how to distinguish them.
3.1 Determine the True Bug Class
To identify the real root cause:
- Trace object lifetime: When was the object allocated? When freed? Who still holds a reference?
- Identify the race window: For concurrency bugs, what's the race between? (syscall vs IRQ, two CPUs, etc.)
- Check error paths: Many kernel bugs live in error handling — a
goto errthat forgets to unlock or drop a reference - Verify with KASAN alloc/free stacks: If KASAN provides them, the allocation and free call stacks tell you exactly who created and destroyed the object
3.2 Build the Bug Narrative — Source-Level Deep Trace (Required)
A shallow narrative ("Thread A frees, Thread B uses" or "missing NULL check") is NOT sufficient. After confirming the root cause, you must produce a source-level deep trace that combines the kernel source code and the PoC's behavior to explain exactly how the bug manifests.
This trace must be generated from the actual source code and PoC for each specific bug. The approach varies by vulnerability class — use the matching methodology below.
Common Steps (All Vulnerability Types)
Step A: Map PoC actions to kernel code paths
Read the PoC and trace what each part does in the kernel:
PoC line/action → syscall/packet → kernel entry function (file.c:line)- For multi-threaded PoCs: which thread does what, and what's the intended race
Step B: Read the relevant kernel source with file:line citations
For every function in the call chain from syscall entry to crash:
- What does it do? What data does it read/write?
- What validation/checks does it perform (or fail to perform)?
- What synchronization (locks, RCU, refcount, memory barriers) does it use?
Step B.1: Verify reachability — trace call-chain preconditions
For each code site identified as vulnerable, verify that the bad state can actually occur there. Trace the full call chain backwards and identify all preconditions that must hold for execution to reach that site: feature flags, device configuration constraints, creation-time validation, compile-time guards, caller-enforced invariants. If a precondition already guarantees the state is valid at that site, the site is not truly vulnerable — exclude it from the fix. Do NOT assume symmetry (e.g., "if the IPv6 path needs a fix, the IPv4 path must too") without proving each case independently.
Step C: Build a timeline/flow diagram from source
Produce a visual that shows the bug's progression. The format depends on the vulnerability class (see below). Use actual function names and file:line references from the source, not generic placeholders.
Step D: Explain why the bug exists
- What invariant is violated?
- What mechanism was supposed to prevent this? Why did it fail?
- Is this a design flaw or an implementation oversight?
Per-Vulnerability-Class Methodology
Choose the methodology that matches your bug's root cause. Not every bug involves refcounts or races — trace what's actually relevant.
UAF / Double-Free / Refcount bugs:
- Trace the object's full refcount lifecycle: creation (init → get) → normal state → destruction (put → release → free), with refcount value at each step
- Identify who holds each reference and when they release it
- Show the race timeline (side-by-side CPUs) with refcount transitions as
before→after - Highlight: missing
kref_get_unless_zero, prematureput, or unprotected reader
Race conditions (non-refcount):
- Identify the shared state being raced on (flag, pointer, list, counter)
- Show the TOCTOU window: what's checked, what changes, what's used
- Side-by-side CPU timeline showing interleaving that leads to the bug
- Highlight: missing lock, wrong lock scope, missing memory barrier
NULL pointer dereference (non-race):
- Trace the data flow: where does the NULL pointer originate?
- Is it from a failed allocation? A missing initialization? An error path that
skips setup? A sparse array lookup (like
inet_protos[])? - Show the call chain from the point where NULL enters to the crash dereference
- Highlight: what validation is missing and where it should be
Out-of-bounds (OOB) read/write:
- Trace the buffer allocation: what size, from where, based on what input?
- Trace the access: what index/offset, from where, based on what input?
- Show the arithmetic:
allocated_sizevsaccessed_offset— why does it overflow? - If integer overflow: show the multiplication/addition that wraps
- Highlight: missing bounds check, wrong size calculation, signedness confusion
Logic bugs / State machine errors:
- Map out the state machine: what states exist, what transitions are valid?
- Show the sequence of operations that reaches an "impossible" state
- Trace the error path that skips a required state transition
- Highlight: missing state check, wrong transition order, error path that forgets cleanup
Info leaks:
- Trace the data flow from kernel memory to user space
- Identify the uninitialized field, padding bytes, or stale pointer
- Show the struct layout with
pahole— which bytes are leaked? - Highlight: missing memset/initialization, struct padding, wrong copy size
Type confusion:
- Show the two types involved and their different layouts
- Trace how the object gets cast/reinterpreted as the wrong type
- Highlight which fields overlap incorrectly (especially function pointers vs data)
Step E: Visualize Your Analysis with ASCII Diagrams
Diagrams are NOT a separate step — they are the visual output of the source analysis above. After completing Steps A-D, produce diagrams that summarize what you found. The diagrams must reference actual function names and file:line from YOUR analysis, not generic templates.
Generate whichever diagram types are relevant to the bug:
- Call chain + data transformation: Show how data flows through each function with
skb->data/ pointer / buffer state at each layer. Each box = actual function (file:line). - Race timeline (for concurrency bugs): Side-by-side CPUs with actual function names, refcount/state transitions, and the race window marked.
- Struct layout (for OOB/type confusion/info leak): pahole-style field offsets showing which field is corrupted/leaked/confused. Use actual struct name from the source.
- Packet/data format (for protocol bugs): Byte-level layout of attacker input showing which fields are controlled and where validation is missing.
- Object lifecycle (for UAF): Allocation → use → free → use-after-free with refcount values at each step, referencing actual functions.
- State machine (for logic bugs): Valid vs actual state transitions.
- Memory/slab layout (for heap bugs): Slab page showing adjacent objects.
The diagram must reflect your source code analysis — not be a generic template.
For example, a call chain diagram should use the real function names you found in Step B,
not placeholder names like function_a().
Quality Criteria (All Types)
- Every source reference has a file:line citation
- The PoC's behavior is mapped to kernel code paths
- State changes (refcount, lock, flag, pointer) show before→after values
- The crash is traced to a specific struct field and offset
- Diagrams use actual function/struct names from the source analysis, not placeholders
- There's a clear explanation of what's broken and why
What to Avoid
- Generic descriptions without source references ("the object is freed then used")
- Drawing diagrams without doing the source analysis first (diagrams are OUTPUT, not INPUT)
- Using only one methodology for all bug types (not everything is a refcount race)
- Skipping the PoC→kernel mapping (the reader needs to understand HOW the bug triggers)
- Copying template diagrams instead of generating them from your analysis
3.3 Determine Affected Version Range
After identifying the introducing commit, determine the exact affected version range:
# Find the earliest release tag containing the introducing commit
git tag --contains <introducing-commit> | sort -V | head -5
# e.g., v3.13-rc1 → bug exists since v3.13
# If a fix already exists upstream, find when it landed
git tag --contains <fixing-commit> | sort -V | head -5
# e.g., v6.14-rc2 → fixed in v6.14
# Check which stable branches are affected
git branch -r --contains <introducing-commit> | grep 'stable'
# Check which stable branches have the fix backported
git branch -r --contains <fixing-commit> | grep 'stable'
Record in the report: Affected: v3.13 — v6.13 (fixed in v6.14-rc2)
For stable/LTS impact, check if the fix needs Cc: [email protected].
3.4 Diagram Reference (Format Examples)
When producing diagrams in Step E above, use these format conventions. These are formatting templates only — your actual diagrams must use real function names and data from your source analysis.
See references/crash-log-analysis.md for address interpretation patterns and
references/vuln-classification.md for the bug classification decision tree.
3.5 Validation-Layer Bugs: Comprehensive Verification Analysis
When this applies: the buggy code's job is to parse, decode, or deserialize untrusted or on-disk data into in-memory structures that later code trusts — log/journal recovery, on-disk metadata readers, netlink/attribute parsers, packet/protocol decoders, ELF/module loaders, ioctl argument unpacking, restore-from-checkpoint paths. For these bugs the single missing check the crash revealed is almost never the whole bug. The code has a contract — "produce only well-formed structures from arbitrary input" — and a crash means that contract is under-enforced in general.
Do NOT spot-fix. Adding the one check the crash exposed, and shuffling the existing checks into a helper, is the most common way to get a validation-layer patch rejected by a maintainer — it looks like refactoring, not hardening, and signals shallow analysis. The correct response is to analyze the entire layer and design verification that captures every way the input can be malformed, in one coherent abstraction.
The framing to apply to yourself (and to any Explore/analysis subagent you dispatch) — point it at the whole file/function, not the crash line:
Assume every byte this code reads from {disk/journal/socket/user} is attacker-controlled and may be arbitrarily malformed. The structures built here are later trusted and dereferenced. Enumerate every issue the current validation fails to catch — not just the one that crashed — and propose a layered plan to robustly verify the data before it is used.
This is the prompt that turns the model from a spot-fixer into an auditor.
Step 1: Map the trust boundary and the bootstrap fields
- Where does untrusted data enter, and at what point is the assembled structure first used/trusted (dereferenced, used to size an allocation, used as an index)? That transition is the trust boundary — verification belongs there, before first use, not scattered downstream.
- Identify the bootstrap / self-describing fields: the fields read first so the code knows how to parse the rest — a type tag, a region/element count, a length, an offset. These are the highest-value targets: everything downstream trusts them to build structures. A malformed count is how "read a count from disk → allocate/loop on it" becomes an OOB or overflow.
Step 2: Find the "commit point" — verify when you stop trusting more input
Incremental parsers accumulate input piece by piece (region by region, attribute by attribute, TLV by TLV). There is a specific point where the code decides it will consume no more input for the current item and finalizes it. That commit point is where whole-object verification belongs: the header, layout, counts, and total assembled size are all known there, and nothing downstream has consumed them yet. Verify the finishing object before starting the next one, and comprehensively verify the new object's header before trusting it to drive further parsing.
Step 3: Enumerate every missing check (be exhaustive, whole-layer)
For the entire layer, list what is currently trusted-without-checking. Typical categories:
- Known-type check — is the type tag a known value? (Reject unknown before using it to dispatch or size anything.)
- Count bounds — is the element/region count within the maximum valid for this type? (Not merely "nonzero" — the per-type maximum.)
- Per-element size — given the now-known type, is each element's size within the max any element of that type may have?
- Total-size / layout consistency — does the sum of parts match the declared whole? Do offsets stay in bounds? Is required alignment satisfied?
- Structural minimum — is each buffer at least
sizeof(the struct we cast it to)before the cast/field access? - Premature-terminator / out-of-sync detection — do we hit a new header or terminator before the current object was expected to end?
- Underflow / NULL on continuation — for incremental assembly, is the
accumulator non-empty and allocated before we append to
[count - 1]?
Produce this as a table (site → what's trusted → how it can be malformed → what check is missing), covering the whole file, not just the crash path.
Step 4: Design the layered abstraction (before any code)
Collapse the enumerated checks into a clean two-tier design — mixing the tiers is what makes verification code incomplete and unmaintainable:
- Generic / structural verification (type-agnostic): known-type, count vs per-type max, per-region size sanity, alignment, minimum sizes, out-of-sync detection. Runs once at the commit point for every item.
- Type-specific / semantic verification: each item/message type validates its
own region count and per-region sizes against what that type requires, before
its consumer casts and dereferences. Often best as a per-type
verify()callback in the existing ops table.
State explicitly which checks live in which tier and where each is called from. This design — not the individual checks — is the Phase 3 deliverable for a validation-layer bug. Confirm it at the high level (Phase 6.1 "Design before code") before writing a line.
What "good" looks like vs the rejected spot-fix
| Rejected (spot-fix) | Accepted (comprehensive) |
|---|---|
| Add the one missing check the crash exposed | Enumerate every way the input can be malformed across the whole layer |
| Move existing scattered checks into a helper, unchanged | Design two tiers (generic structural + per-type) placed at the trust/commit boundary |
| Fix only the crash path | Cover every path that builds structures from this input |
| "It no longer crashes on the PoC" | "Malformed input of any of these N forms is rejected before use" |
Read references/verification-hardening.md for the full methodology and a worked
example.
Phase 4: Dynamic Analysis (QEMU + GDB)
4.1 Set Up QEMU Environment
Read references/qemu-setup.md for detailed setup instructions.
Key requirements:
- Build kernel with:
CONFIG_KASAN=y,CONFIG_DEBUG_INFO=y,CONFIG_GDB_SCRIPTS=y,CONFIG_FRAME_POINTER=y,CONFIG_HARDENED_USERCOPY=y(and relevant subsystem configs) - Use
virtme-ngfor quick boot if applicable, or full QEMU with custom rootfs - Prepare a minimal rootfs with the PoC compiled and ready
# Example QEMU launch (adjust as needed)
qemu-system-x86_64 \
-kernel arch/x86/boot/bzImage \
-initrd rootfs.cpio.gz \
-append "console=ttyS0 root=/dev/ram rdinit=/init nokaslr" \
-nographic \
-m 2G \
-smp 2 \
-s -S # GDB stub on port 1234, halt at start
4.2 Reproduce and Debug
- Boot the vulnerable kernel in QEMU
- Run the PoC and confirm the crash reproduces
- Attach GDB:
gdb vmlinux -ex "target remote :1234" - Set breakpoints at key functions identified in static analysis
- Step through the vulnerable code path
- Confirm the root cause hypothesis from Phase 3
4.3 Key GDB Commands for Kernel Debugging
# Kernel-specific
lx-symbols # Load kernel module symbols
lx-dmesg # Print kernel log
lx-ps # List processes
lx-lsmod # List modules
# Analysis
p *(struct sk_buff *)$rdi # Print kernel structures
info threads # Check CPU/thread state
bt # Backtrace
watch *(int *)0xaddr # Hardware watchpoint on the vulnerable field
4.4 Handling Non-Deterministic Reproduction
Race conditions and timing-sensitive bugs may not crash on every run.
Increase reproduction rate:
# Loop the PoC — run 100 times and count crashes
for i in $(seq 1 100); do
timeout 5 ./poc 2>/dev/null
echo "Run $i: exit=$?"
done
# Increase CPU count to widen race windows
qemu-system-x86_64 ... -smp 4 # or -smp 8
# Add system stress to increase scheduling pressure
stress-ng --cpu 4 --io 2 --vm 2 --timeout 60 &
./poc
# Use taskset to pin PoC threads to specific CPUs
taskset -c 0,1 ./poc
Record reproduction rate in the report: e.g., "Triggers 30/100 runs with -smp 4"
If the PoC never crashes:
- Verify kernel config matches the crash environment (especially KASAN, PREEMPT, SMP)
- Check if the compiler version matters (see
references/syzbot-workflow.md) - Try the exact syzbot kernel config if available
- Add
usleep()delays in the PoC to manipulate race timing - Use
ftraceto confirm the race window exists even if it doesn't crash
Phase 5: Exploitability Assessment
After confirming the root cause, assess whether this bug is exploitable for privilege escalation, information leak, or denial of service.
Read references/exploitability-assessment.md for the full assessment framework.
5.1 Key Questions
-
What primitive does this bug give an attacker?
- UAF → potential arbitrary read/write via heap spray
- OOB write → adjacent object corruption
- Double-free → overlapping allocations
- Info leak → KASLR bypass
- Race condition → how wide is the window? Is it winnable?
-
What is the attack surface?
- Reachable from unprivileged user? Needs
CAP_NET_ADMIN? Needs user namespaces? - Reachable from network? From a container?
- Reachable from unprivileged user? Needs
-
What objects share the same slab cache?
- For UAF/OOB: what useful kernel objects (e.g.,
struct cred,struct file,msg_msg,pipe_buffer,sk_buff) live in the samekmalloc-*bucket? - Can the attacker control allocation/free timing?
- For UAF/OOB: what useful kernel objects (e.g.,
-
What mitigations apply?
- KASLR, SMEP, SMAP, CFI, RANDSTRUCT
CONFIG_SLAB_FREELIST_RANDOM,CONFIG_SLAB_FREELIST_HARDENEDCONFIG_HARDENED_USERCOPY,CONFIG_USERFAULTFDavailability
-
Is there precedent?
- Search the kernelctf knowledge base for exploits in the same subsystem or using similar primitives
- Reference known techniques:
msg_msgspray,pipe_bufferROP,io_uringprimitives, cross-cache attacks
5.2 Permission Gate Analysis (capable vs ns_capable) — Required
Before rating exploitability, you MUST trace the full permission check chain from syscall entry to the vulnerable function. This determines the true attack surface.
Why this matters: A bug gated by capable(CAP_NET_ADMIN) needs real root.
But the SAME capability checked via ns_capable() is obtainable by any unprivileged
user through unshare(CLONE_NEWUSER|CLONE_NEWNET). Many analysts miss this distinction,
leading to wrong severity assessments.
Step 1: Find all capability checks in the call path
# Trace from syscall entry to the vulnerable function
# and grep for every permission check along the way
grep -n 'capable\|ns_capable\|netlink_capable\|nfnl.*capable\|sk_net_capable' \
net/netfilter/nfnetlink.c net/netfilter/nfnetlink_osf.c
Step 2: Draw the permission gate diagram
For each layer, document: what check, namespace-aware or not, what happens on failure.
User-space syscall (sendmsg on netlink socket)
│
▼
Framework layer (e.g., nfnetlink_rcv)
│ netlink_net_capable(skb, CAP_NET_ADMIN) ← namespace-aware (ns_capable)
│ userns root CAN pass this gate
│ failure → -EPERM, never reaches callback
│
▼
Subsystem callback (e.g., nfnl_osf_add_callback)
│ capable(CAP_NET_ADMIN) ← init_user_ns ONLY
│ userns root CANNOT pass this gate
│ failure → -EPERM
│
▼
Vulnerable code path
Step 3: Determine effective privilege requirement
The effective requirement is the most restrictive check in the entire chain:
- If ANY layer uses
capable()→ needs real root (init_user_ns) - If ALL layers use
ns_capable()/netlink_capable()→ reachable via userns - Watch for framework vs callback mismatch (common pattern: framework is ns-aware
but specific callback adds a stricter
capable()check)
Step 4: Document in the report
Include the ASCII gate diagram in the report's exploitability section. State clearly:
- "Effective privilege: unprivileged (all checks are namespace-aware)" OR
- "Effective privilege: real root (callback uses
capable()at line X)" - Note if a future
capable()→ns_capable()change would expand the attack surface
Read references/exploitability-assessment.md § "Critical: capable() vs ns_capable()"
for the full reference on which subsystems use which check.
5.3 Challenge Your Initial Assessment (Required)
Do NOT stop at the first conclusion. The most common mistake in kernel vulnerability analysis is accepting the surface-level classification ("it's just a NULL deref / DoS") without probing deeper. Many high-severity CVEs were initially dismissed as DoS.
After forming your initial assessment, systematically challenge it by asking these questions:
"Is the crash symptom hiding a stronger primitive?"
| Initial symptom | Ask yourself | Deeper reality? |
|---|---|---|
| NULL ptr deref | Is KASAN enabled? Without KASAN, a UAF to zeroed memory looks like NULL deref | Possibly UAF |
| NULL ptr deref at offset N | Is N controllable by the attacker? If so, this may be an arbitrary-address read | Possible info leak |
| GPF / non-canonical addr | Is the address derived from attacker data + corruption? Could the attacker make it canonical? | Possible controlled dereference |
| Single-byte OOB write | What's adjacent in the slab? Even 1 byte can flip a boolean flag or corrupt a refcount | Possible privilege escalation |
| Refcount WARNING | Does this eventually lead to a UAF if triggered enough times? | Likely UAF with patience |
| DoS-only race | With different timing, does the race give a wider window? Can userfaultfd freeze the race? | Possible reliable UAF |
"Is the NULL deref masking a UAF?" — The RCU Lifetime Pattern
This is the most commonly missed upgrade path. When you see a NULL deref in RCU-protected code, ask: where did the NULL come from?
Pattern: RCU object has field cleared before grace period expires
Teardown path (writer): Read path (RCU reader):
────────────────────── ────────────────────────
hlist_del_rcu(&obj->node) rcu_read_lock()
obj = rcu_dereference(hash[idx])
obj->ops->destroy(obj) │
│ │ ← obj is valid (RCU protects it)
├─ resource_put(obj->ptr) │ but obj->ptr is being destroyed
├─ obj->ptr = NULL ◄── HERE │
│ │
└─ call_rcu(&obj->rcu, free_fn) obj->ptr->field ← NULL DEREF
rcu_read_unlock()
The critical question: What happens in the window BETWEEN resource_put(obj->ptr)
and obj->ptr = NULL?
Timeline:
T1: resource_put(obj->ptr) → obj->ptr's refcount drops to 0 → memory freed
T2: ───────────────────────── WINDOW: obj->ptr points to FREED MEMORY
T3: obj->ptr = NULL → now it's "safely" NULL
T4: call_rcu(free_fn) → obj itself deferred
If reader accesses obj->ptr between T1 and T3 → UAF (not NULL deref!)
If reader accesses obj->ptr after T3 → NULL deref (the "safe" crash)
The PoC that crashes with NULL deref is hitting the T3→T4 window. But the T1→T3 window is more dangerous — it's a real UAF where the reader follows a dangling pointer to freed memory.
How to evaluate this:
- Read the teardown code — is there a
put/release/freeBEFORE the= NULL? - If yes: the freed memory could be reallocated with attacker-controlled content
- What object is being freed? What slab cache? What size?
- Can the attacker spray that cache between T1 and T3?
- What fields does the reader dereference? Function pointers? Data?
Hypothetical example — generic subsystem teardown:
some_subsystem_destroy(obj):
dev_put(obj->netdev, ...) ← T1: netdev refcount → 0, memory freed
──────────────────────────────── ← WINDOW: obj->netdev is dangling pointer
obj->netdev = NULL ← T2: "safe" NULL
call_rcu(&obj->rcu, obj_free) ← T3: obj itself deferred
Reader hitting T1-T2 window: obj->netdev → freed memory
→ if reader dereferences function pointers through it → code execution
→ MUCH more dangerous than the NULL deref at T2+
Look for this pattern whenever you see a NULL deref in RCU-protected code where the NULL comes from an explicit assignment in a teardown/destroy path.
If you find this pattern, the bug upgrades from "DoS" to "Likely/Highly Exploitable".
"Can I get a different/stronger primitive from the same root cause?" — Root-Cause-Driven Path Enumeration (Required)
This is the most critical and most often skipped step in exploitability assessment.
The PoC crashes on ONE code path. But the root cause — the underlying invariant violation — may be reachable through MULTIPLE code paths, each yielding a different primitive. You MUST enumerate all paths, not just analyze the crash path.
The methodology is: Root Cause → All Affected Sites → Per-Site Primitive → Pick Strongest.
Step 1: Precisely define the root cause as a pattern
Do NOT define the root cause as "crash in function X". Define it as the abstract invariant violation that CAUSES the crash. Examples:
| Bad (crash-specific) | Good (root-cause pattern) |
|---|---|
"NULL deref in subsystem_periodic_scan" | "Any RCU traversal of the object hash that dereferences obj->parent without first taking a reference via kref_get_unless_zero() can observe an object with refcount=0 and cleared parent pointer" |
"UAF in subsystem_add_entry" | "Any path that holds a stale pointer to an entry across a transaction commit/abort boundary can access freed memory" |
"OOB write in subsystem_set_params" | "Any path that uses user-provided parameters without validating count * element_size against the allocated buffer can write past the buffer" |
The pattern definition determines your search scope. Be precise.
Step 2: Find ALL code sites matching the pattern
Use grep/Explore agents to systematically enumerate every site:
# Find all callers of the function that dereferences the unsafe pointer
grep -rn '<unsafe_function_name>' <subsystem_dir>/
# Find all RCU traversals of the hash that DON'T use kref_get_unless_zero
grep -rn 'hlist_for_each_entry_rcu.*<object_type>' <subsystem_dir>/
For each call site found, classify it:
| Site | Context | Has refcount guard? | Vulnerable? |
|---|---|---|---|
periodic_scanner() LNNN | workqueue (periodic) | No | YES |
hash_find_helper() LNNN | various callers | Yes (kref_get_unless_zero) | No |
packet_rx_handler() LNNN | packet RX path | Via hash_find | No |
| ... | ... | ... | ... |
Do NOT stop at the first vulnerable site. Exhaustively check every site.
Step 3: For each vulnerable site, analyze the exploitation primitive
For every site marked "Vulnerable" in step 2, answer:
-
What context does this code run in?
- Process context? → attacker can control scheduling, use userfaultfd
- softirq/IRQ? → preemption disabled, harder to race but no context switch
- Workqueue? → process context with possible sleep points
-
What operations does this code perform on the vulnerable object?
- Read a field → info leak primitive?
- Write a field → corruption primitive?
- Dereference a pointer → if controllable, arbitrary read/write?
- Call a function pointer → code execution?
- Compare fields → bypass of security check?
- Copy data to userspace → info leak to attacker?
-
What objects/resources does this code touch AFTER the vulnerable access?
- Even if the initial access is a crash, is there a code path where the vulnerable access succeeds (non-NULL, valid memory) and continues to do something useful to the attacker?
-
Can the attacker control what's in the freed/corrupted memory?
- What slab cache is the freed object in?
- What's the object size? → which kmalloc bucket?
- Can the attacker spray that cache via
msg_msg,sk_buff,setxattr, etc.? - If so, what fake field values would give the attacker a useful primitive?
-
What is the race window width?
- Back-to-back instructions → very narrow, but userfaultfd/FUSE can help
- Separated by lock acquire → lock contention can widen the window
- Separated by blocking operation → wide window, easy to exploit
Step 4: Cross-path comparison — find the strongest primitive
Build a summary table:
| Vulnerable path | Primitive | Controllability | Race window | Verdict |
|---|---|---|---|---|
| `periodic_scanner` (workqueue) | NULL deref → DoS | None | periodic interval | DoS only |
| `packet_handler` (if vulnerable) | Read parent fields → info leak? | Spray-dependent | Per-packet | Potential info leak |
| `teardown_helper` (put-before-del) | UAF if obj freed while in hash | Spray target cache | Lock-width | Potential UAF |
The strongest primitive across ALL paths is the bug's true exploitability ceiling.
Step 5: Analyze pre-crash windows on the strongest path
For the most promising path, trace the exact teardown sequence and identify ALL timing windows (not just the one the PoC hits):
Teardown sequence for the strongest vulnerable path:
T0: [precondition] — attacker sets up the race
T1: resource_put() — object freed, pointer dangling ← UAF WINDOW OPENS
T2: ptr = NULL — pointer cleared ← NULL WINDOW (PoC crashes here)
T3: call_rcu(free_fn) — parent object deferred free ← MEMORY FREED AFTER GRACE PERIOD
Attacker wants to hit T1→T2 (UAF), not T2→T3 (NULL deref).
For each window, assess:
- How wide is it? (instructions? microseconds? milliseconds?)
- Can it be widened by the attacker? (CPU pinning, interrupt flooding, lock contention)
- Can
userfaultfdorFUSEfreeze a page fault in the middle to hold the window open? - What specific slab object can be sprayed into the freed slot during this window?
The goal is not to confirm the PoC's crash — it's to find the strongest possible primitive from the root cause across all affected code paths.
"Can this be chained with other bugs?"
Even a "DoS-only" bug can be valuable in a chain:
- Info leak + this bug: If you have a separate KASLR bypass, does this bug become exploitable?
- This bug + another write: A read primitive from this bug + a write primitive from another bug = full exploit
- This bug enables another: Does crashing this specific code path leave the system in a state that makes another bug reachable?
"What happens on different kernel configurations?"
- Without
CONFIG_INIT_ON_FREE_DEFAULT_ON: freed memory retains old data → a NULL deref might become a valid-pointer dereference (UAF) - Without KASAN: UAF doesn't get caught immediately → attacker has more time to spray
- With
CONFIG_USERFAULTFD=y: race windows can be frozen → narrow races become reliable - With older kernels: NULL page may be mappable (
mmap_min_addr=0) → NULL deref becomes code execution
Document your reasoning
In the report, include a full path enumeration section like:
### Root-Cause Path Enumeration
Root cause pattern: {{precise description of the invariant violation}}
#### All affected code sites
| # | Function (Line) | Context | Refcount guard? | Vulnerable? |
|---|---|---|---|---|
| 1 | `func_a()` L123 | workqueue | No | **YES** |
| 2 | `func_b()` L456 | process ctx | Yes (kref_get_unless_zero) | No |
| 3 | `func_c()` L789 | softirq | No | **YES** |
#### Per-path primitive analysis
**Path 1: `func_a()` (workqueue context)**
- Operations after vulnerable access: {{what the code does with the object}}
- Primitive: {{DoS / info leak / UAF / arbitrary write / code exec}}
- Race window: {{width, widenable?}}
- Spray target: {{slab cache, object size, feasibility}}
**Path 3: `func_c()` (softirq context)**
- Operations after vulnerable access: {{...}}
- Primitive: {{...}}
- Race window: {{...}}
- Spray target: {{...}}
#### Strongest primitive across all paths
{{The most dangerous path is #N because... / All paths are DoS-only because...}}
### Alternative Exploitation Analysis
Initial assessment: {{initial primitive}} → {{initial impact}}
Challenges considered:
1. Could this be a UAF? — {{Yes/No: specific reasoning citing lock/teardown analysis}}
2. Is the offset controllable? — {{Yes/No: trace where the offset comes from}}
3. Different timing (pre-NULL vs post-NULL window)? — {{Yes/No: analyze teardown sequence}}
4. Different trigger path? — {{Yes/No: reference path enumeration above}}
5. Chain potential? — {{Yes/No: what would be needed}}
6. Different kernel config? — {{Yes/No: INIT_ON_FREE, USERFAULTFD, mmap_min_addr}}
Conclusion: {{Assessment stands / upgraded to X because path #N gives Y primitive}}
If you find a stronger primitive, update the rating and exploitation path accordingly. This step often upgrades "DoS" bugs to "Likely Exploitable" or higher.
5.4 Exploitability Rating
Rate as one of:
- Highly Exploitable: Reliable UAF/OOB-write with good heap spray target, reachable unprivileged
- Likely Exploitable: Bug gives useful primitive but exploitation has challenges (narrow race, limited control)
- Potentially Exploitable: Bug exists but exploitation path unclear or requires unusual conditions
- Unlikely Exploitable: DoS only, confirmed no stronger primitive after challenge analysis
- Not Exploitable: Theoretical bug with no practical trigger path
Phase 6: Patch Development & Verification
6.1 Write the Patch
STOP — Confirm source version before writing ANY patch code:
echo -n "Patch base: " && git describe --tags HEAD
LATEST=$(git describe --tags --abbrev=0 origin/master 2>/dev/null)
echo "Latest upstream: $LATEST"
BEHIND=$(git log --oneline HEAD..origin/master 2>/dev/null | wc -l)
[ "$BEHIND" -gt 0 ] && echo "ERROR: $BEHIND commits behind — checkout $LATEST first!" && exit 1
# Final dedup check against subsystem trees (nf/nf-next/net/bpf/etc.)
# A fix may have landed in the subsystem tree since Phase 2.2
SUBSYS_REMOTE="nf" # adjust per subsystem
SUBSYS_NEXT="nf-next" # adjust per subsystem
git fetch "$SUBSYS_REMOTE" 2>/dev/null; git fetch "$SUBSYS_NEXT" 2>/dev/null
SEARCH_REFS="origin/master"
git rev-parse --verify "$SUBSYS_REMOTE/main" &>/dev/null && SEARCH_REFS="$SEARCH_REFS $SUBSYS_REMOTE/main"
git rev-parse --verify "$SUBSYS_NEXT/main" &>/dev/null && SEARCH_REFS="$SEARCH_REFS $SUBSYS_NEXT/main"
echo "Dedup check against: $SEARCH_REFS"
git log $SEARCH_REFS --oneline -S '<vulnerable_function>' -- "$VFILE" | head -3
# If output is non-empty → someone already fixed this. STOP and verify the existing fix.
echo "OK — source is current, no existing fix found, safe to write patch"
If HEAD is behind origin/master → go back to Phase 2.2 and update the tree. If the dedup check finds a fix in the subsystem tree → STOP, report the existing fix. Do NOT write a patch against stale source or duplicate existing work. This check takes a few seconds and prevents rejected patches and wrong report metadata.
Read references/patch-writing-guide.md for Linux kernel patch conventions.
Fix Scope Decision (classify the bug before writing any code)
The correct shape of a fix depends on the bug class. Pick one — getting this wrong is exactly how a validation-layer patch earns the "you spot-fixed one check and moved the rest into a helper — that's not comprehensive verification" rejection.
-
Point fix — a specific lifetime, locking, refcount, or logic error in trusted internal code. → Minimal, targeted diff. Fix exactly the invariant violation. Prove each guard is reachable (Phase 3 Step B.1). Do NOT add symmetric or defensive checks elsewhere without independently proving reachability.
-
Validation-layer hardening — a bug in code that parses/decodes/deserializes untrusted or on-disk data (see Phase 3.5). → Comprehensive, layered verification. The one check the crash exposed is not the fix. Deliver the two-tier design from Phase 3.5 (generic structural + per-type). Here "minimal diff" means no unrelated refactoring — NOT "one check": every malformation class the untrusted input allows must be covered, because each is a real attack path. Reachability is satisfied by construction (the input is attacker- controlled); the only checks to omit are for states a caller already fully guarantees.
When unsure which mode applies, ask: is this code's contract to produce well-formed structures from arbitrary input? If yes → hardening mode.
Design before code (required for anything beyond a one-line fix)
Work the fix at the high level first, in prose, and confirm it before generating code. This is the step that most improves patch quality and is the one most often skipped — do not drop to "code level" until the design is settled.
- State the fix architecture in natural language: what invariant you enforce, where, and why that is the correct layer. For hardening bugs, this is the two-tier verification design from Phase 3.5.
- For any fix that touches more than one site or introduces a new abstraction, decompose into a single-change-per-patch series — each patch does exactly one thing and is independently reviewable and bisectable (e.g. 1: add the generic verifier; 2: wire it in at the commit point; 3: add per-type verify callbacks; 4: enforce them before commit). List the series before writing any hunk.
- Present the architecture + patch breakdown to the user and get agreement before coding. Then write the code one patch at a time.
Core principles:
- Fix the root cause, not the symptom (don't just add a NULL check if the real bug is a missing lock)
- Minimal diff — no unrelated refactoring or style churn. "Minimal" bounds scope, not thoroughness: for validation-layer bugs (see Fix Scope Decision above) the necessary change covers every malformation class of the untrusted input, not just the crash path.
- Follow the existing code style of the file you're modifying
- Comments only when necessary (非必要不要注释): match the file's existing comment
density; never comment what obvious code does, only the non-obvious why (subtle
locking, a spec/hardware quirk, an inferrable-nowhere invariant). Line-by-line
narration is an AI tell and gets rejected as noise. See
references/patch-writing-guide.md§ "Comments: Only When Necessary". - Add appropriate locking, reference counting, or lifetime management
- Consider all callers — your fix must not break other code paths
- Verify each fix point is necessary: For every guard you add, prove the bad state can actually reach that site (see Phase 3, Step B.1). For point fixes, do NOT add symmetric or defensive fixes "for completeness" without proving reachability independently — that wastes reviewer time and signals shallow analysis. For validation-layer hardening, covering every way the untrusted input can be malformed IS necessary (each is a real attack path) and is not "defensive completeness" — omit only checks for states a caller already fully guarantees.
Commit message format:
The commit message MUST include the decoded backtrace and follow upstream tag conventions.
Look at any KASAN/bug fix in mainline git log --grep='KASAN' for real examples.
Write it concisely — no AI tells. Terse, factual, human. Imperative subject with
no "This patch..." preamble; explain the bug and the why, don't narrate the diff;
cut filler ("In order to", "It is worth noting", "Additionally") and qualitative
fluff ("robust", "comprehensive", "properly", "gracefully"); prose paragraphs, not
bullet lists; don't hedge. If it reads like a blog post, cut it. See
references/patch-writing-guide.md § "Write Concisely — Avoid AI Tells" for
before/after examples.
subsystem: brief description of the fix
Longer explanation of what the bug is, how it manifests, and why
this patch fixes it. Use present tense. Include the root cause.
Weave in the trigger conditions naturally — what config is needed,
what privilege, whether it's reachable remotely or locally — so
reviewers and stable maintainers can assess severity from the
description itself.
Introduce the backtrace naturally (upstream convention):
syzbot reported a null-ptr-deref in icmp_unreach [1]:
BUG: KASAN: null-ptr-deref in icmp_unreach (net/ipv4/icmp.c:1085)
Call Trace:
<IRQ>
icmp_unreach (net/ipv4/icmp.c:1143)
icmp_rcv (net/ipv4/icmp.c:1524)
ip_protocol_deliver_rcu (net/ipv4/ip_input.c:205)
ip_local_deliver_finish (net/ipv4/ip_input.c:234)
ip_local_deliver (net/ipv4/ip_input.c:254)
ip_rcv (net/ipv4/ip_input.c:569)
</IRQ>
The root cause is that icmp_tag_validation() dereferences
inet_protos[proto] without checking for NULL...
<explanation of the fix>
[1] https://syzkaller.appspot.com/bug?extid=<hash>
Fixes: <12-char-hash> ("original commit title that introduced the bug")
Reported-by: <who reported> <email>
Closes: <bug report URL>
Link: <lore.kernel.org mail thread URL>
Reviewed-by: <reviewer> <email>
Cc: [email protected]
Signed-off-by: <your name> <email>
Commit message tag rules (order matters):
| Tag | Required? | Purpose |
|---|---|---|
Fixes: | Yes | 12-char hash of the introducing commit |
Reported-by: | Yes | Who found the bug |
Closes: | Yes (if URL exists) | URL of the bug report — required after Reported-by: per upstream convention |
Link: | Recommended | lore.kernel.org link to the mailing list discussion |
Tested-by: | Recommended | Who tested the patch (can be syzbot+<hash>@... if syzbot tested it) |
Reviewed-by: | If reviewed | Code reviewer's signoff |
Acked-by: | If acked | Subsystem maintainer acknowledgment |
Cc: [email protected] | If applicable | Request backport to stable trees |
Signed-off-by: | Yes (last) | Developer Certificate of Origin |
Trigger conditions (convey naturally in the commit body, not as a structured list):
The commit message should make clear what's needed to trigger the bug — required CONFIG options, privilege level, whether it's local or remote, and whether the default config is affected. This helps reviewers and stable maintainers assess severity and backport priority. Weave this information into the natural prose description of the bug, the way upstream commits do. Do NOT use a bullet-point "Trigger conditions:" section — that format is not used upstream.
Good (natural prose):
The bug is reachable from an unprivileged user namespace on any kernel
with CONFIG_NETFILTER=y (default). A crafted nfnetlink message triggers
a use-after-free in nf_tables_newrule().
Bad (structured list — not upstream style):
Trigger conditions:
- Required CONFIG: CONFIG_NETFILTER=y
- Required privilege: unprivileged via userns
- Attack vector: local (nfnetlink)
Backtrace guidelines:
- Introduce naturally:
"syzbot reported a <bug-type> in <function> [1]:"or just paste the first crash line, NOT"Decoded backtrace:"(not used upstream) - Use the DECODED trace (file:line), not raw hex addresses
- Trim to key frames: crash point + 5-10 relevant frames
- Remove noise: timestamps, registers,
?frames,Code:lines, module lists - Indent with single space
- If from syzbot, add footnote
[1]linking to the syzbot bug page
6.2 Validate the Patch
After git format-patch produces the .patch file, run ALL of these checks.
Do NOT skip any — a patch that fails checkpatch or doesn't apply cleanly will
be rejected by maintainers without review.
Format validation:
# 1. checkpatch — coding style and commit message format
./scripts/checkpatch.pl --strict 0001-*.patch
# Fix ALL errors and warnings. Common issues:
# - Line over 80/100 characters
# - Missing Signed-off-by
# - Trailing whitespace
# - Wrong indentation (spaces instead of tabs)
# 2. Patch applicability — verify it applies cleanly to the tree
git stash # save current state
git am --check 0001-*.patch # dry-run: does it apply?
# If "patch does not apply" → the patch context is wrong (wrong base commit)
git stash pop
# 3. Commit message format check
head -1 0001-*.patch | grep '^From ' # Must start with "From <hash>"
sed -n '4p' 0001-*.patch # Subject line
# Verify: "Subject: [PATCH <tag>] subsystem: description"
# Subject should be < 72 characters (after the [PATCH ...] prefix)
# Check body line length (should be < 75 chars, excluding backtrace)
awk '/^---$/{exit} /^$/{next} length>75{print NR": "length" chars: "$0}' 0001-*.patch
# 4. No MIME headers
grep -E 'MIME-Version|Content-Type|Content-Transfer-Encoding' 0001-*.patch
# Should find nothing. If present: commit message has non-ASCII, fix and regenerate.
# 5. Verify required tags are present
grep -E '^Fixes:|^Signed-off-by:|^Cc:' 0001-*.patch
# Must have at least Fixes: and Signed-off-by:
Build validation:
# 6. Compile test — apply patch and build
git am 0001-*.patch
make -j$(nproc) # Must compile without errors
# 7. Sparse (optional but recommended)
make C=1 <modified-file.o> # Static analysis for annotation issues
# 8. Subsystem selftests (if they exist)
make -C tools/testing/selftests/<subsystem> run_tests
Submission preparation:
# 9. Generate git send-email command with correct recipients
./scripts/get_maintainer.pl 0001-*.patch
Patch generation workflow — the commit message and code change must be bundled together
via git commit + git format-patch. Do NOT use git diff > patch — that produces a raw
diff without the commit message, tags, or authorship info.
# Step 1: Stage the fix
git add <modified-files>
# Step 2: Commit with the full message (body + backtrace + tags)
# Use a heredoc to preserve formatting
git commit -m "$(cat <<'COMMIT_EOF'
subsystem: brief description of the fix
Root cause explanation — naturally mention what config, privilege,
and attack vector are needed to trigger the bug, so reviewers can
assess severity from the description.
syzbot reported a <bug-type> in <function> [1]:
<decoded backtrace, indented with single space>
<explanation of the fix>
[1] https://syzkaller.appspot.com/bug?extid=...
Fixes: <12-char-hash> ("introducing commit title")
Reported-by: <name> <email>
Closes: <bug report URL>
Cc: [email protected]
Signed-off-by: <your name> <email>
COMMIT_EOF
)"
# Step 3: Generate the patch file with correct subject prefix
git format-patch -1 --subject-prefix="PATCH <tag>"
# This produces: 0001-subsystem-brief-description.patch
# The file contains: From/Date/Subject headers + commit message + diff
# Step 4: Verify the patch
head -30 0001-*.patch # Should show From, Date, Subject, blank line, body
grep -E 'MIME-Version|Content-Type' 0001-*.patch # Should find nothing
# Step 5: Get maintainers and build the send command
./scripts/get_maintainer.pl 0001-*.patch
# Auto-generate git send-email with correct recipients
git send-email \
$(./scripts/get_maintainer.pl --nogit --nogit-fallback --norolestats \
0001-*.patch | awk '{printf "--cc=\047%s\047 ", $0}') \
0001-*.patch
Read references/patch-writing-guide.md § "Find Maintainers and Generate git send-email Command"
for the full workflow including tocmd/cccmd auto-configuration.
6.3 Verify in QEMU — Two-Phase PoC Test (Required)
Both phases are mandatory. You need to prove: (A) the bug exists, (B) your patch fixes it. Save the QEMU serial output from both runs as log files for the report.
Phase A: Confirm crash on VULNERABLE kernel
# Build vulnerable kernel (patch NOT applied)
git stash # or checkout the un-patched commit
make -j$(nproc) bzImage
cp arch/x86/boot/bzImage report/kernel/test-bzImage
cp vmlinux report/kernel/test-vmlinux
# Boot and run PoC — capture output
timeout 60 qemu-system-x86_64 \
-kernel report/kernel/test-bzImage \
-initrd rootfs.cpio.gz \
-append "console=ttyS0 root=/dev/ram rdinit=/init nokaslr oops=panic panic=1" \
-nographic -m 2G -smp 2 -no-reboot \
2>&1 | tee report/logs/vuln-test.log
# Verify the crash happened:
grep -E 'BUG|KASAN|panic|Oops|NULL pointer' report/logs/vuln-test.log
# MUST find crash indicators. If not → PoC doesn't trigger, fix the PoC first.
Phase B: Confirm NO crash on PATCHED kernel
# Build patched kernel
git stash pop # or apply the patch
make -j$(nproc) bzImage
cp arch/x86/boot/bzImage report/kernel/patched-bzImage
cp vmlinux report/kernel/patched-vmlinux
# Boot and run PoC — capture output
timeout 60 qemu-system-x86_64 \
-kernel report/kernel/patched-bzImage \
-initrd rootfs.cpio.gz \
-append "console=ttyS0 root=/dev/ram rdinit=/init nokaslr oops=panic panic=1" \
-nographic -m 2G -smp 2 -no-reboot \
2>&1 | tee report/logs/patched-test.log
# Verify NO crash:
grep -E 'BUG|KASAN|panic|Oops|NULL pointer' report/logs/patched-test.log
# MUST find nothing. If crash indicators appear → patch is wrong, go back to 6.1.
# Also check for new warnings introduced by the patch:
grep -iE 'WARNING|lockdep|RCU' report/logs/patched-test.log
# Any new warnings → investigate. Your patch may have introduced a regression.
What to include in the report:
logs/vuln-test.log— full serial output from Phase A (shows crash)logs/patched-test.log— full serial output from Phase B (shows clean run)- Summary: "Vulnerable kernel: CRASH at <function>. Patched kernel: PoC ran, no crash, clean dmesg."
For race condition bugs: Run the PoC in a loop on the patched kernel to verify the fix is robust, not just lucky timing:
# Inside QEMU on patched kernel:
for i in $(seq 1 100); do ./poc; echo "Run $i: exit=$?"; done
# All 100 runs should complete without crash
Phase C: Unprivileged Trigger Test via unshare (if applicable)
If the Phase 5.2 permission gate analysis determined that ALL capability checks in the
call path use ns_capable() (not capable()), the bug is reachable from an unprivileged
user via unshare(CLONE_NEWUSER|CLONE_NEWNET). In this case, you MUST verify this
claim by actually running the PoC as a non-root user inside QEMU.
When to run Phase C: Only when the permission gate analysis concludes
"unprivileged via userns" — i.e., no capable() check blocks the path.
# Inside QEMU (on the VULNERABLE kernel), create a non-root user and test:
# 1. Create unprivileged user
adduser --disabled-password --gecos "" testuser 2>/dev/null || true
# 2. Copy PoC to user's home
cp /root/poc /home/testuser/poc
chmod +x /home/testuser/poc
# 3. Run PoC as unprivileged user with unshare
su testuser -c 'unshare -Urn /home/testuser/poc'
# ^^^^
# -U = CLONE_NEWUSER (new user namespace, get all caps)
# -r = map current user to root in new userns
# -n = CLONE_NEWNET (new network namespace)
# 4. Check result
dmesg | tail -20
# If kernel crashes → confirmed: unprivileged trigger via userns
# If EPERM or no crash → the permission analysis was wrong, re-check
If the unprivileged trigger succeeds: This is a significant finding. The report MUST include a dedicated section documenting this:
### Unprivileged Trigger Verification
The bug can be triggered by an unprivileged user via user namespaces:
- **Test**: PoC run as non-root user with `unshare -Urn ./poc`
- **Result**: Kernel crash confirmed — no root privileges required
- **Permission path**: All checks use `ns_capable()` / `netlink_capable()`
→ `unshare(CLONE_NEWUSER|CLONE_NEWNET)` grants sufficient capabilities
- **Impact upgrade**: This is NOT a "requires CAP_NET_ADMIN" bug —
it is an unprivileged local DoS/exploit on any system with
CONFIG_USER_NS=y and unprivileged_userns_clone=1 (Ubuntu default)
Tested command: `su testuser -c 'unshare -Urn /home/testuser/poc'`
Log: logs/unpriv-test.log
If the unprivileged trigger fails (EPERM): Document that too — it means
the permission analysis missed a capable() check somewhere. Go back to
Phase 5.2 and find the missing gate.
Save the log: report/logs/unpriv-test.log
6.4 Regression Check
- Ensure no new KASAN/UBSAN warnings in
patched-test.log - If the subsystem has selftests, run them on the patched kernel
- Test edge cases: what happens at the boundaries of your fix?
- For race condition fixes: verify with
-smp 4and stress testing
6.5 Restore Source Tree (Required)
After verification is complete, the kernel source tree MUST be restored to its original
(unpatched) state. The analysis process must be non-destructive — the patch lives only
in the report's patch/ directory as a standalone .patch file.
# Restore all modified source files
git checkout -- <modified-files>
# Verify clean state
git diff --stat # should show no changes
The correct workflow for the entire build-and-test cycle:
1. Save the patch: git diff > report/patch/0001-fix.patch
2. Stash changes: git stash
3. Build VULNERABLE kernel from clean source
4. Copy vulnerable bzImage/vmlinux to report/kernel/test-*
5. Pop stash: git stash pop
6. Build PATCHED kernel
7. Copy patched bzImage/vmlinux to report/kernel/patched-*
8. Test both in QEMU
9. RESTORE source: git checkout -- <files> ← DO NOT FORGET
If you skip step 9, the source tree is left in a modified state, which is messy and may confuse subsequent analyses or the user's own work.
GATE CHECK: Before Entering Phase 7 — Evidence Required
DO NOT proceed to Phase 7 until you have ACTUAL QEMU OUTPUT as evidence. A mental checklist is not enough. You must have run real commands and captured real output.
The gate check is not "did you think about testing" — it is "show me the QEMU log."
Required Evidence (must exist before report generation starts)
Before writing any report file, you must have ALL of these actual artifacts:
1. Vulnerable kernel crash log (logs/vuln-test.log)
- Must contain actual QEMU serial output from booting the vulnerable kernel
- Must show the PoC running and the kernel crashing
- Verify:
grep -E 'BUG|KASAN|panic|Oops' logs/vuln-test.logreturns matches
2. Patched kernel clean log (logs/patched-test.log)
- Must contain actual QEMU serial output from booting the patched kernel
- Must show the PoC running and the kernel NOT crashing
- Verify:
grep -E 'BUG|KASAN|panic|Oops' logs/patched-test.logreturns NOTHING
3. Patched kernel binaries (kernel/patched-bzImage, kernel/patched-vmlinux)
- Must be actual compiled kernel binaries, not empty files
- Verify:
file kernel/patched-bzImageshows "Linux kernel x86 boot executable"
4. checkpatch output
- Must have run
./scripts/checkpatch.pl --stricton the.patchfile - No errors allowed (warnings may be acceptable with justification)
How to verify the gate (run these commands)
# All 4 checks must pass:
echo "=== Gate Check ==="
echo -n "1. Vuln crash log: "
grep -cE 'BUG|KASAN|panic|Oops' report/logs/vuln-test.log 2>/dev/null && echo "PASS" || echo "FAIL — no crash log"
echo -n "2. Patched clean log: "
[ -f report/logs/patched-test.log ] && \
! grep -qE 'BUG|KASAN|panic|Oops' report/logs/patched-test.log && echo "PASS" || echo "FAIL — crash on patched kernel or no log"
echo -n "3. Patched kernel binary: "
file report/kernel/patched-bzImage 2>/dev/null | grep -q 'Linux kernel' && echo "PASS" || echo "FAIL — no patched kernel"
echo -n "4. Source restored: "
git diff --stat 2>/dev/null | grep -q '.' && echo "FAIL — uncommitted changes" || echo "PASS"
echo -n "5. Unpriv test (if ns_capable path): "
if [ -f report/logs/unpriv-test.log ]; then
echo "PASS — log exists"
else
echo "SKIP (not applicable, or FAIL if ns_capable path was identified)"
fi
If checks 1-4 show FAIL → you cannot proceed to Phase 7. Go back and fix it.
Check 5 is required only when Phase 5.2 identified an ns_capable() path — if so, FAIL = go back.
What happens if QEMU is unavailable
If qemu-system-x86_64 is not installed or the user's environment doesn't have it:
- Ask the user: "I need to verify the patch in QEMU. Where is your test environment?"
- Check for existing QEMU/rootfs:
which qemu-system-x86_64,ls *.cpio.gz - Build one: Run
scripts/setup_qemu_env.shto create the environment - If truly impossible: Tell the user explicitly: "I was unable to verify the patch in QEMU. The report is UNVERIFIED." Mark the report with a prominent warning banner.
NEVER silently skip testing and deliver a report as if it were verified.
Common excuses for skipping verification (all invalid)
| Excuse | Why it's wrong |
|---|---|
| "The patch is obviously correct" | Obvious patches have hidden bugs |
| "The prove log already showed it crashes" | That tested the VULNERABLE kernel, not the PATCHED one |
| "I'll test later" | The user will trust the report as-is |
| "QEMU isn't available" | Ask the user. Build one. Don't skip. |
| "It's just a NULL check" | Even a NULL check can be wrong scope/variable/return |
| "I already know it works" | You don't know until you see QEMU output with zero crash indicators |
Phase 7: Report Generation & Artifact Packaging
7.1 Output Directory Structure
Create a self-contained analysis folder. Generate two separate reports — one in English
(report_en.md) and one in Chinese (report_cn.md). Each report is a complete standalone
document (not a translation stub that references the other). Technical content (code snippets,
diffs, struct layouts, ASCII diagrams, shell commands) should be identical in both; only the
prose (explanations, analysis narrative, table headers) differs by language.
<CVE-or-bug-id>-analysis/
├── report_en.md # Full analysis report (English)
├── report_cn.md # Full analysis report (Chinese / 中文)
├── poc/
│ ├── poc.c # PoC source code
│ ├── Makefile # PoC build instructions
│ └── poc_binary # Compiled PoC (optional)
├── patch/
│ ├── 0001-fix-description.patch # The git format-patch output
│ └── patch_description.md # Patch explanation
├── kernel/
│ ├── test-vmlinux # Vulnerable kernel vmlinux (for GDB)
│ ├── test-bzImage # Vulnerable kernel bzImage
│ ├── patched-vmlinux # Patched kernel vmlinux
│ ├── patched-bzImage # Patched kernel bzImage
│ └── .config # Kernel config used
├── env/
│ ├── rootfs.cpio.gz # Root filesystem image
│ ├── run-vulnerable.sh # Script to boot vulnerable kernel
│ └── run-patched.sh # Script to boot patched kernel
└── logs/
├── crash.log # Original crash log
├── gdb-session.log # Key GDB session output
└── patch-verification.log # Log showing PoC no longer crashes
7.2 Report Templates
Two templates are provided:
assets/report_template.md— English templateassets/report_template_cn.md— Chinese (中文) template
Generate both report_en.md and report_cn.md using their respective templates.
Do NOT create a single bilingual report with interleaved EN/CN sections — each report must
be a complete, self-contained document in one language. Both reports must include:
- Executive Summary: One-paragraph overview — what the bug is, severity, exploitability, fix status
- Bug Classification: Type, affected subsystem, affected versions, CVE (if assigned)
- Crash Log Analysis: Annotated crash log with key signals highlighted
- Root Cause Analysis: Detailed technical analysis with code references
- Vulnerability Timeline: Object lifecycle, race window, or error path that leads to the bug
- Exploitability Assessment: Rating and justification, with references to similar known exploits
- Patch: The fix, with explanation of why it's correct
- Verification Results: Evidence that the patch fixes the bug without regressions
- References: Related CVEs, kernel commits, kernelctf entries
7.3 Run Scripts
Generate run-vulnerable.sh and run-patched.sh that boot QEMU with the correct parameters:
#!/bin/bash
# run-vulnerable.sh
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
qemu-system-x86_64 \
-kernel "$SCRIPT_DIR/../kernel/test-bzImage" \
-initrd "$SCRIPT_DIR/rootfs.cpio.gz" \
-append "console=ttyS0 root=/dev/ram rdinit=/init nokaslr" \
-nographic -m 2G -smp 2 \
-net nic -net user,hostfwd=tcp::10022-:22
Subagent Orchestration Patterns
Pattern 1: Parallel Static Analysis
Main Agent (Coordinator):
├── Subagent A: Read and analyze functions in the crash call stack
├── Subagent B: Git log/blame the affected files, find introducing commit
├── Subagent C: Search knowledge base for similar vulnerability patterns
└── Main Agent: Synthesize results → determine root cause hypothesis
Pattern 2: Build & Test Pipeline
Main Agent (Coordinator):
├── Subagent D [worktree]: Build vulnerable kernel with debug configs
├── Subagent E [worktree]: Build patched kernel
└── Main Agent: When both complete → verify with QEMU
Pattern 3: Cross-Reference Research
Main Agent (Coordinator):
├── Subagent F: Search kernelctf for exploits in same subsystem
├── Subagent G: Search for related CVEs and patches upstream
├── Subagent H: Analyze slab cache sharing for exploitability
└── Main Agent: Synthesize exploitability assessment
When to Use Subagents vs. Do It Yourself
- Use subagents when: tasks are independent, IO-bound (file reading, git operations, web fetch), or you need isolated worktrees (compilation)
- Do it yourself when: the task is a quick single-file read, or results of step N are needed for step N+1 with no parallelism opportunity
Important Considerations
Security & Ethics
This skill is for authorized security research, defensive analysis, and educational purposes. The goal is to understand vulnerabilities to fix them and improve kernel security.
- Always focus on developing correct patches, not weaponizing bugs
- If a PoC is needed for verification, keep it minimal — just enough to trigger the bug
- Report findings responsibly
Linux Coding Standards
When writing patches, strictly follow:
Documentation/process/coding-style.rstDocumentation/process/submitting-patches.rst- Run
scripts/checkpatch.pl --stricton all patches - Tabs for indentation, 80-column lines (soft limit, 100 hard)
- No unnecessary whitespace changes
- Commit messages: imperative mood, 72-char subject, wrapped body
Avoiding Patch Regressions
- Test with multiple kernel configs (defconfig, allyesconfig with relevant options)
- Consider all callers of modified functions
- If adding locking, verify no deadlock potential (lock ordering)
- If modifying reference counting, verify no leaks or double-frees introduced
- If changing error paths, verify all resources are properly cleaned up
Tools Reference
| Tool | Purpose |
|---|---|
scripts/decode_stacktrace.sh | Decode raw kernel stack traces |
scripts/get_maintainer.pl | Find subsystem maintainers |
scripts/checkpatch.pl | Validate patch coding style |
virtme-ng (vng) | Quick kernel boot for testing |
addr2line | Resolve addresses to source lines |
pahole | Inspect struct layouts and padding |
crash | Kernel crash dump analysis |
ftrace | In-kernel function tracer (no recompile needed) |
perf | Hardware performance counters, timing analysis |
objdump | Disassembly when source-level debugging isn't enough |
strace | Trace PoC syscall sequences |
slabinfo | Runtime slab cache analysis |
Reference Files
These files contain detailed guidance for specific phases. Read them when you reach the relevant phase — they contain information too detailed for the main skill but essential for correct analysis.
references/crash-log-analysis.md— How to parse KASAN, UBSAN, BUG, panic logs with examplesreferences/vuln-classification.md— Taxonomy of kernel vulnerability classes and identificationreferences/exploitability-assessment.md— Framework for assessing kernel bug exploitabilityreferences/patch-writing-guide.md— Linux kernel patch conventions and common patternsreferences/verification-hardening.md— Comprehensive verification analysis for parsers/decoders/deserializers of untrusted input (two-tier design, trust-boundary placement, single-change-per-patch decomposition)references/qemu-setup.md— Setting up QEMU+GDB kernel debugging environmentsreferences/kernelctf-knowledge-base.md— Exploit techniques and patterns from Google's kernelCTFreferences/syzbot-workflow.md— Syzbot interaction: reproducers,#syz test,#syz fixreferences/regression-testing.md— Kselftest, KUnit, LTP, multi-config testingassets/report_template.md— Template for the final analysis report
Quick Reference Index
| When you encounter... | Read this |
|---|---|
| KASAN crash log | references/crash-log-analysis.md § KASAN Reports |
| syzbot report URL | references/syzbot-workflow.md § Reading a Syzbot Report |
| UAF / double-free / OOB | references/vuln-classification.md |
| Bug in a parser / decoder / log-recovery / deserializer | references/verification-hardening.md + SKILL.md Phase 3.5 |
| "Is this exploitable?" | references/exploitability-assessment.md § Assessment Methodology |
| Writing a patch | references/patch-writing-guide.md § Common Fix Patterns |
| Setting up QEMU+GDB | references/qemu-setup.md § Full QEMU Setup |
| Similar known exploits | references/kernelctf-knowledge-base.md § Exploit Technique Catalog |
| Writing regression test | references/regression-testing.md § Writing a New Selftest |
| Submitting patch upstream | references/patch-writing-guide.md § Submitting Patches Upstream |
| Defensive blocking | references/exploitability-assessment.md § Syscalls to Block |