Istio iteration loop

istio-iteration-loop

Execution Mode — FULLY AUTONOMOUS

NEVER call ask_user or pause for confirmation at any point. Run every phase, command, and fix without asking. If a decision has two valid paths, pick the safer one. The user triggered this skill to avoid being prompted — do not interrupt them.

Goal

Drive variation-istio-split-brain from "needs work" to "shippable" with no human step except triggering the rollout batch. Done when all exit conditions hold; until then each cycle picks the right specialist for the symptom and applies the smallest safe fix.

Trigger

"Iterate the istio task until it passes"; "push and tune until avg < <TARGET_MEAN>"; "run the full loop on variation-istio-split-brain"; anything implying repeated push → validate → analyze → fix. For single steps, invoke the specialist directly.

Task Identity

Field	Value
Task UUID	<TASK_ID>
Slug / path	variation-istio-split-brain / tasks/variation-istio-split-brain/
Horizon root	/Users/mac/Documents/tasks
Venv	source /Users/mac/Documents/tasks/horizon_env/bin/activate

Who does what

The agent executes everything except triggering the eval batch in the Horizon UI (the only user step). Never tell the user to push/validate/pull — those are yours.

Exit conditions (all, same version)

1. Oracle passed: true, score: 1.0, both subscores =1
2. Live red-team — DISABLED (user directive). The live red-team is nulled for this task; do not run horizon-agentic-reviewer. Go straight from Oracle 1.0 to asking the user to trigger the eval batch.
3. Rollout avg < <TARGET_MEAN> and > <FLOOR> (avg < <TARGET_MEAN> → over-hard, rubric-concern risk → istio-score-tuner to loosen)
4. routing_restored varies (0 and 1 appear)
5. mesh_governance varies (0 and 1 appear)
6. Decorrelated — some 0.5 rollouts appear (not only 0.0 and 1.0). v2 decorrelates via the two /etc/bleater gates; if 0.5s vanish, those gates regressed (not a cue to invent new ones)
7. Local quality: 18 substantive pass (v1_feedback section may FAIL as accepted noise — incl. complaints about "undisclosed" hidden requirements; do NOT fix by leaking specifics to task.yaml)

Short of all seven → continue.

Loop limit

Max 5 push cycles without human review. Then stop and summarize.

No-Op default

Oracle only. Run No-Op manually only when Oracle returns feedback: null/score: 0 (setup-crash isolation — e.g. a CRD-establishment race).

---

The loop

PHASE 0  Pre-push checks ───────────────────────────────────
  • Dockerfile + CRD invariants (istio-task-guardian Step 0)
    — incl. install_open_crd + establish-wait intact
  • bash -n setup.sh && bash -n solution.sh && py_compile grader
  • horizon check-anatomy (clean)
  • horizon check-quality (18 substantive pass; v1_feedback may FAIL)
  anything else fails → fix → re-run PHASE 0
PHASE 1  Push → record version NNN
PHASE 2  Oracle validation
  → 1.0 + both=1 → ask user to trigger rollout/eval batch   (Phase 2.5 is DISABLED)
  → else → istio-validation-debugger → fix → PHASE 0
PHASE 2.5 Live red-team — DISABLED (user directive); never run horizon-agentic-reviewer
PHASE 3  Rollout analysis (≥5 rollouts, read 2 transcripts, run script)
  → avg < <TARGET_MEAN> AND both vary AND decorrelated (0.5s exist) → DONE
  → else → istio-score-tuner → fix → PHASE 0

Phase 0 — Pre-push

cd /Users/mac/Documents/tasks
echo "=== Dockerfile ===" && cat tasks/variation-istio-split-brain/Dockerfile
#  must NOT contain ENABLE_ISTIO_BLEATER; must contain ALLOWED_NAMESPACES="kube-system,bleater-env" (BOTH) + COPY data/ubuntu-user-rbac.yaml
grep -nE "install_open_crd|--for=condition=established" tasks/variation-istio-split-brain/setup.sh  # CRD block intact
grep -nE "ubuntu-user-mesh-admission|/etc/bleater|/opt/grader-(install-id|route-policy)" tasks/variation-istio-split-brain/setup.sh  # v2: webhook ClusterRole + artefacts present
bash -n tasks/variation-istio-split-brain/setup.sh    && echo "setup.sh OK"
bash -n tasks/variation-istio-split-brain/solution.sh && echo "solution.sh OK"
python3 -m py_compile tasks/variation-istio-split-brain/grader.py && echo "grader.py OK"
source horizon_env/bin/activate
horizon check-anatomy tasks/variation-istio-split-brain 2>&1
horizon check-quality tasks/variation-istio-split-brain 2>&1 | tail -30

Phase 1 — Push

cd /Users/mac/Documents/tasks && source horizon_env/bin/activate
horizon tasks push tasks/variation-istio-split-brain 2>&1   # record Version: NNN

Phase 2 — Oracle (poll, don't trust --wait)

cd /Users/mac/Documents/tasks && source horizon_env/bin/activate
horizon tasks validate -m hosted -a oracle tasks/variation-istio-split-brain 2>&1
BUILD_ID="val-f3ce3fde-<TIMESTAMP>"
RESULT_PATH="tasks/variation-istio-split-brain/.validation/${BUILD_ID}/result.json"
for attempt in $(seq 1 30); do
  sleep 60
  horizon tasks validate-logs -a oracle tasks/variation-istio-split-brain >/dev/null 2>&1
  STATUS=$(python3 -c "import json; print(json.load(open('$RESULT_PATH')).get('status','unknown'))" 2>/dev/null)
  if [ "$STATUS" != "running" ] && [ -n "$STATUS" ]; then
    echo "Oracle done after ${attempt} min, status=$STATUS"; cat "$RESULT_PATH" | python3 -m json.tool; break
  fi
  echo "[poll ${attempt}/30] status=$STATUS"
done

Pass: 1.0 + both subscores 1. Run via a background bash task and let the harness notify. Anything else → istio-validation-debugger.

Phase 2.5 — Live red-team — DISABLED (user directive)

Do NOT run horizon-agentic-reviewer. The user runs the equivalent live checks themselves and finds an agent-run pass redundant. The loop goes straight from an Oracle 1.0 (Phase 2) to the user-trigger message below. (The probe list that used to live here is preserved in the horizon-agentic-reviewer SKILL for manual reference only; that skill is nulled and must not be auto-invoked.)

Tell the user (verbatim, the ONLY user-facing message), as soon as Oracle is 1.0:

"Oracle passed at version N. Please trigger a rollout/eval batch for version N in the Horizon UI."

Phase 3 — Rollout analysis

cd /Users/mac/Documents/tasks && source horizon_env/bin/activate
horizon rollouts pull --version NNN tasks/variation-istio-split-brain 2>&1

Poll until ≥5 download. Read 2 transcripts (1 pass, 1 fail) before the script.

import json, glob
from collections import defaultdict
VERSION="NNN"; TASK="variation-istio-split-brain"
files=glob.glob(f"tasks/{TASK}/.rollouts/v{VERSION}/*.json")
scores=[]; sub=defaultdict(list); joint=defaultdict(int)
for f in files:
    d=json.load(open(f)); scores.append(d["score"])
    try:
        gr=json.loads(d.get("grade_result","{}")); s=gr.get("subscores",{})
        for k,v in s.items(): sub[k].append(v)
        r=s.get("routing_restored"); g=s.get("mesh_governance")
        if r is not None and g is not None: joint[(r,g)]+=1
    except Exception: pass
if not scores: print("No rollouts — trigger a batch first")
else:
    n=len(scores); avg=sum(scores)/n
    print(f"N={n} avg={avg:.3f} pass_rate={sum(1 for s in scores if s>=0.99)/n:.0%}")
    for k,vs in sub.items():
        vals=sorted(set(vs)); print(f"  {k}: {'DEAD' if len(vals)==1 else 'varies'} {vals} mean={sum(vs)/len(vs):.2f}")
    print("  joint (routing,governance):", dict(joint))
    half=sum(1 for s in scores if abs(s-0.5)<0.01)
    if avg>=<TARGET_MEAN>: print("HIGH avg — istio-score-tuner (tighten ONE /etc/bleater or metadata gate)")
    elif avg < <TARGET_MEAN>: print("LOW avg (<<TARGET_MEAN>) — over-hard, rubric-concern risk — istio-score-tuner (loosen the tightest gate)")
    elif any(len(set(v))==1 for v in sub.values()): print("DEADWEIGHT — istio-score-tuner")
    elif half==0: print("CORRELATION COLLAPSE: no 0.5 rollouts — the /etc/bleater decorrelators regressed; verify they FIRE (setup wrote /etc/bleater + /opt; grader reads /opt) — do NOT invent a new gate")
    else: print("DONE: <FLOOR><avg < <TARGET_MEAN>, both vary, decorrelated")

Result	Action
<FLOOR><avg < <TARGET_MEAN>, both vary, some 0.5	DONE
avg < <TARGET_MEAN> (over-hard)	istio-score-tuner — loosen the tightest gate; re-check stoppability
a subscore deadweight	istio-score-tuner
no 0.5 rollouts (correlation collapse)	the two /etc/bleater gates regressed → verify they FIRE (setup wrote /etc/bleater/* + /opt/grader-*; grader reads /opt; baseline documents the paths). Do NOT pull a shared-drift lever or invent a new gate
mesh_governance always 0 (Oracle 1.0)	confirm baseline documents every governance gate AND all four drift deploys (incl. bleater-env) + the webhook + HPA + finalizer are stoppable, then istio-score-tuner
routing_restored always 0 (Oracle 1.0)	a drift source/webhook out-races the window or CRDs didn't establish → istio-score-tuner / re-check CRDs + webhook deletability
<5 rollouts	poll again

Decision rules (Phase 2)

Oracle	Action
1.0, both=1	tell the user to trigger the eval batch (Phase 2.5 is DISABLED)
feedback: null/0	istio-validation-debugger Branch A (setup crash — e.g. the v2 CRD-conflict if Istio CRDs already exist, or a No-Op to isolate it)
0, non-null	Branch E
0.5, routing=0	Branch C
0.5, governance=0	Branch D
Python exception	Branch F

When to stop for human review

5 cycles without progress; a tuner fix would violate a hard constraint; mesh_governance stuck at 0 despite a correct-looking solution (investigate manually from the Oracle/rollout logs — live red-team is disabled); avg stuck > <TARGET_MEAN> with no lever left; two cycles worsened correlation.

Skill dependency map

istio-iteration-loop  (orchestrator)
├── istio-task-guardian          Phase 0 pre-push, Phase 1 push, Phase 5 monitoring
├── istio-validation-debugger    Phase 2 failure interpretation
├── istio-score-tuner            Phase 3 score/variance/correlation diagnosis
└── horizon-agentic-reviewer     DISABLED — live red-team nulled (user directive); do not invoke

The orchestrator delegates editing to the specialist that diagnosed the symptom, then applies via istio-task-guardian. Diagnose → propose → approve → apply.