Fanout cve score tuner

fanout-cve-score-tuner

QC Spec compliance — assert before every tuning edit

Any gradient/difficulty lever you pull must STILL satisfy the QC Spec (.claude/QC-SPEC.md). Before recommending a change, assert it does not breach:

• No Binary Bundling — keep the 5 subscores independent; never AND them or short-circuit the grade on one failure.
• Incremental Rewards / No Dead Weights — every 0.2 subscore must stay independently reachable and missed by some — none always-0 or always-1.
• Functional Tests as Subscores — keep pods_rolled_live_env a live kubectl exec check; never relax it to a spec read.
• Equal Weights — all five subscores stay at 0.2.
• No Answer Leakage — CVE truth stays in root-only /opt/grader-cve.json, never an agent-editable artefact.

A lever that improves the gradient by breaching any of these is NOT valid — pick a different lever.

Execution Mode — FULLY AUTONOMOUS

NEVER pause for confirmation. Pick the best path and execute.

What this skill optimizes (different from the variance tasks)

This is a reward-shaped task. There is NO avg < <TARGET_MEAN> variance target, no drift, no decorrelation problem. The objective is a dense, correct, monotone-ish partial-credit gradient: the agent should earn 0.2 per genuinely-correct sub-step and only reach 1.0 with the complete two-phase remediation. Your job is to keep that gradient honest.

Invoked when: a subscore is deadweight (always-0 unreachable / always-1 free); partial credit is structurally impossible; a subscore is reward-hackable; the gradient mis-guides (e.g. you can score high while the system is still degraded); fanout-cve-task-guardian Step 5 hands off. NOT for Oracle failures → fanout-cve-validation-debugger.

Task Identity

Field	Value
UUID	<TASK_ID>
Subscores (0.2 each)	cve_image_patched, configmap_reverted, pods_rolled_live_env, gitops_committed, argocd_synced

The five subscores and what each must keep true

Subscore	Reward-shaping invariant
cve_image_patched	reachable only by changing the version off the vulnerable set; judged off root-only /opt/grader-cve.json (NOT the agent-editable harbor-scan-results) — keep it that way or it becomes reward-hackable
configmap_reverted	a pure config check (AMQP_TLS_VALIDATION_MODE==full); independent of the others
pods_rolled_live_env	the keystone — must use live kubectl exec, not the pod spec, or "no restart" passes for free. Must require rollout convergence + clean version label + live env full. This is intentionally the hardest/most-missed subscore
gitops_committed	the Gitea source declares a clean version + full; independent of the cluster CM
argocd_synced	argocd-bleater-app status==Synced; independent

Health checks (run these on the rollout distribution)

Healthy	Unhealthy → action
every score in {0,0.2,…,1.0} appears	a value never appears → a subscore may be coupled/unreachable
each subscore passed by some agents, missed by some	deadweight subscore → fix (see below)
pods_rolled_live_env is the most-missed	if it's the LEAST-missed, the live-env check may be too weak (verify it execs)
a fair share reach 1.0	nobody reaches 1.0 → an ordering/structural impossibility (check the revert-then-roll dependency)

Levers (smallest first)

Deadweight cve_image_patched (always-0)

The version marker is unreachable. Confirm the agent can change bleater.io/fanout-version (it's a template label, not in the selector) and that harbor-scan-results + baseline name the patched version. Don't require an exact string the agent can't discover.

Deadweight pods_rolled_live_env (always-0, Oracle 1.0)

Likely the exec is failing in the rollout window. Confirm ${SIDECAR_IMAGE} has sh; add exec retries (already 2). Do NOT relax to a spec-only check.

pods_rolled_live_env always-1 (free)

The live-env check isn't biting — verify it execs the pod (not reads the spec configMapKeyRef, which always shows the reference). If the stale snapshot coincidentally equals the target, ensure the version-label convergence still forces a roll.

Partial credit impossible (only 0.0 and 1.0 appear)

The subscores have become coupled (AND-bundled). Keep them five independent functions returning independent 0/1 — never short-circuit the whole grade on one failure.

Reward-hack: agent edits harbor-scan-results to fake "clean"

Ensure cve_image_patched reads /opt/grader-cve.json (root-only), never the ConfigMap.

Mis-guidance: high score while still degraded

The danger case the author called out: scoring well without actually fixing the live system. Guard it — pods_rolled_live_env (live env + converged + clean version) is what ties the score to the real running state. Never let configmap_reverted + gitops_committed + argocd_synced (3×0.2=0.6, all structural) be achievable while the pods are still vulnerable/stale. They can be, individually — that's fine for shaping — but 1.0 must require the live pods correct.

Difficulty (medium — do NOT inflate)

The issue specifies medium. Do not add drift, cross-namespace discovery, hidden /etc artefacts, or admission webhooks — those belong to the variance tasks and would break the reward-shaping design and the difficulty target.

Roles

• R1 Distribution Analyst — pull rollouts, tabulate the score histogram + per-subscore pass rate; classify each subscore (deadweight / healthy / hackable).
• R2 Keystone Specialist — focus on pods_rolled_live_env: is it execing live env, requiring convergence + clean version? Is it the right difficulty (most-missed but reachable)?
• R3 Independence Reviewer — confirm the 5 subscores are independent functions; no AND-bundling; partial credit flows.
• R4 Anti-Hack Reviewer — confirm version judged off /opt/grader-cve.json; no subscore passable by editing a read-only-intended artefact.
• R5 Final Recommender — 8-point block (which subscore/gradient problem / why / smallest fix / expected distribution impact / QC impact / risk / files / lines). Then → fanout-cve-iteration-loop → fanout-cve-task-guardian.

Operating procedure

Confirm trigger → read grader/setup/solution + rollouts → R1 distribution → identify the gradient problem → pick the smallest lever (keep medium, keep independence, keep anti-hack) → R3/R4 review → R5 recommend → NEVER apply here; hand to the loop/guardian.

Hard constraints

No ENABLE_ISTIO_BLEATER; keep the rbac COPY; judge version off /opt/grader-cve.json; keep the live-env exec in the keystone subscore; keep 5 independent subscores (no bundling); no drift/variance mechanics; medium difficulty; air-gapped runnable images (pods must exec); don't simplify solution.sh.

Preferred outcome

The full {0,0.2,0.4,0.6,0.8,1.0} gradient appears across a batch; pods_rolled_live_env is the most-instructive (most-missed but reached by careful agents); 1.0 requires the live pods to be patched + full; no subscore is deadweight or reward-hackable.