K8s components checker

k8s-components-checker

Survey an RKE2 community cluster, cross-reference against the embedded compatibility registry, produce a verdict. Community editions only. Air-gapped at use time — references/ carries everything needed and the survey runs fully offline. Air-gap-complete is a floor, not a ceiling: when the workstation has internet + gh, grounding every specific version the verdict cites is mandatory (House Rule #8 · references/version-verification.md) — the registry's sifted patch numbers are methodology, not trusted release facts.

Surveys, never upgrades. This skill produces a verdict; it does not move a cluster. When the verdict says a component must move, hand off to the skill that owns that ladder: rancher-upgrade (Rancher + the downstream RKE2/K3s fleet, same k8s plugin), harvester-upgrade (Harvester HCI, harvester plugin), mimir-upgrade (Grafana Mimir, observability plugin). Those skills read this one's compatibility registry as input and treat it as the matrix authority — that split is deliberate, so keep upgrade procedure out of here and compatibility claims out of them.

Two registry components have no upgrade ladder because the answer is migration, not a version bump: a Zalando postgres-operator verdict hands off to postgres-operator-cloudnative-pg-migration (k8s plugin), and a rancher-logging verdict to rancher-logging-exit (observability).

The registry is methodology, not inventory. It encodes which versions of which components are compatible with which Kubernetes minors. It does not — and must not — record what's actually running where.

Quick decision guide

Task	Go to
Run a survey + answer an upgrade-readiness question	§ Survey workflow below
Look up one component's k8s support window	references/components.md
Read what changed for compat in a specific component minor	references/compat/<comp>.md
Which kubectl/helm/pluto commands to run, and how to parse output	references/cluster-survey.md
Which deprecated-API tools to trust and how	references/tooling.md
Verify a version exists / find the real latest patch (online)	references/version-verification.md
Source URLs + last-verified timestamps (read-only — freshen writes here)	references/sources.md

Survey workflow

The operator runs the skill against a kubeconfig pointing at one cluster at a time. Network access to the target cluster is assumed (direct or via VPN/bastion). The cluster survey itself makes no internet calls; the one sanctioned outbound path is version grounding — when internet + gh are available the verdict's specific version numbers MUST be confirmed against real releases (House Rule #8 · references/version-verification.md).

First identify the change set — what's being upgraded and from what to what. Five shapes the skill answers:

• anchor bump — k8s <current> → <target> (e.g. RKE2 1.32 → 1.34)
• single leaf — <component> <current> → <target> (e.g. Cilium 1.18 → 1.19)
• combined — multiple bumps in one upgrade window (e.g. k8s 1.32 → 1.34 + Argo CD 3.0 → 3.2)
• drift review — no specific target; report what's stale / EOL'd / unpatched
• feasibility — find the highest tolerable bump on one axis given the others are pinned (e.g. "highest Argo CD that supports our k8s 1.32")

Then run the survey and verdict per references/cluster-survey.md (Phases 1–5: cluster identity → component detection → deprecated-API liability via the apiserver metric + pluto → cross-reference against references/components.md and references/compat/<comp>.md → assemble the verdict per § Verdict format, ordering the action plan by the compat files' upgrade-ordering rules).

For a drift review (no target), the survey still runs; the verdict reports against the current k8s minor only, flagging components whose installed version is approaching or past upstream EOL.

Verdict format

The header carries the change set (see § Survey workflow) — anchor bump, single leaf, combined, drift review, or feasibility. The per-row reason at the end of each ✓ ready / ⚠ needs bump line is composed from the compat data and is change-set-specific (supports k8s 1.32..1.34 for an anchor bump, compat with target Cilium 1.19 for a leaf bump, current version EOL'd 2025-10-07 for drift, etc.).

<cluster name or kubeconfig context> — <change set>

✓ ready
   - <component> <version>                    <change-set-specific reason>

⚠ needs bump
   - <component> <version> → ≥ <target>       <reason — what blocks the change set>
     source: references/compat/<comp>.md § <version>
     # <target> is the furthest-coverage version (House Rule #9) — covers this hop
     # AND the operator's next known hop, not the bare immediate-hop minimum.
     # A row that satisfies the current target but sits at its own support ceiling
     # belongs HERE (with the forward-covering target named), not under ✓ ready.

⚠ ordering
   - <component A> must reach <version> BEFORE <component B> reaches <version>
     reason: <one-liner>
     source: references/compat/<a>.md § <version>

✗ blockers
   - <deprecated API>: still served (<N requests/day>) — source: <namespace/workload>
   - <component> <version>: <reason — current-state issue or change-set blocker>
     source: references/compat/<comp>.md § <version>

✗ out of registry scope
   - <component> <version>: below registry's min_tracked_version (<floor>);
     verdict abstained. Bump to a tracked version, or add an override in
     references/components.md.

action plan
   1. <ordered step>
   2. <ordered step>
   ...

✓ ready rows are noise after the first survey — collapse them by default in follow-up surveys unless the operator asks for the full list. ⚠ and ✗ rows always show.

Verdict vs report. The verdict above is the technical core, suitable for in-conversation use and runbook PRs. When the operator asks for a pre-upgrade report (for JIRA, change management, audit, mgmt review), wrap the verdict in the layout from references/report-format.md — same content, recognizable section skeleton across runs so prior reports compare cleanly.

House rules

These are non-negotiable; encode them into every verdict.

1. Community editions only. Any Prime / EE / paid-tier feature, version, or support window is out of scope. Ignore SUSE Prime backports. Ignore GitLab EE-only features (the operator runs the EE binary as CE; treat as CE).
2. Axis discipline. A component's axis_type is multi only when the operator picks two or more dimensions independently. Container runtime, driver version, GPU architecture — all derived from other choices and therefore NOT axes. RKE2 → containerd is fixed; don't treat it as a variable.
3. Min-tracked-version is overridable per component. Default floor is "current + prior 2 minors" (~18 months). Operator overrides set per-component min_tracked_version: in references/components.md; skill-improver freshen respects overrides and trims unset rows.
4. Apiserver metric is the truth source for deprecated APIs. It reports what the cluster has actually served. Pluto's bundled rule set goes stale; use it for manifest-side static scans, never as primary. Kubent is dead (rulesets stop at k8s 1.32); do not use.
5. Harvester ordering: any survey involving Harvester + RKE2 must check the Harvester compat file for ordering rules before emitting an action plan. Some Harvester↔RKE2 combinations require Harvester to upgrade first; missing this turns the upgrade plan into a cluster-rebuild.
6. Cite sources. Every ⚠ and ✗ row carries a source: line pointing at the exact references/compat/<comp>.md § <version> block that produced the finding. No unsourced verdicts.
7. Abstain when the registry is silent. If the registry doesn't carry enough signal to verdict a row, abstain on that component and recommend running skill-improver freshen <skill> from an internet-accessible client.
8. Never invent versions; ground or abstain. k8s support windows are registry methodology (cite the compat file). Specific version numbers — latest patch, newest minor, "CVE fixed in vX.Y.Z", a recommended target patch — are volatile and the #1 fabrication risk (a past verdict cited an Argo CD v3.2.10 that never existed; the 3.2 line ended at v3.2.6). State a specific release only if it is (a) cluster-reported, (b) grounded against a freshly fetched release listing, or (c) explicitly marked UNVERIFIED. When internet + gh are available, grounding is mandatory and uses the anti-confirmation method — enumerate the real tag list (naming no candidate) and derive the latest patch per minor line, and never ask "does vX exist?" (a named guess biases you toward confirming it). releases/latest is recency, not rank — it's the most-recently-published / maintainer-pinned release, NOT the highest version, so never reject a higher enumerated minor because it exceeds releases/latest (that misfire struck the real Harbor 2.15.x while releases/latest was 2.14.4 — a back-ported patch to an old line outranks a newer minor by date). Existence is not edition: for a vendor with a community/Prime (or OSS/EE) split that ships both to one feed (Rancher), a real, older patch can be Prime-only and will rubber-stamp as community — sort -V | tail -1 returns a Prime patch for any non-current minor. Apply the release-notes edition discriminator, don't trust version order. Full protocol + component→repo map + edition discrimination: references/version-verification.md.
9. Target furthest coverage, not the immediate-hop minimum. When a row needs a bump, recommend the lowest version whose support window also covers every known or queued next hop — never the bare minimum that only clears the current change set. If a newer minor already exists, is stable, and covers both the current target and the next planned minor, recommend it directly. A version that satisfies the current target but lands on its own support ceiling is a ⚠, not a ✓ — say so and name the forward-covering target, because shipping it forces an avoidable second bump one hop later. Step minor-by-minor ONLY when (a) mandatory sequential stops exist (GitLab app stops, Rancher CAPI conversion) or (b) skipping stacks breaking changes too aggressively — and even then, name the end-state target version, not just the next step. Always ask "what is the operator's next hop after this one, and does my recommended version already cover it?" before emitting a → ≥ <target>. (Encoded after a real miss: a k8s 1.32 → 1.33 handoff recommended KEDA 2.18.3 — CVE-patched and 1.33-valid but at the 2.18 ceiling, even though the handoff itself noted "1.34 would need ≥ 2.19 later" — while KEDA 2.19.0 (k8s 1.32–1.34, same CVE fix) already existed and would have covered the very next 1.33 → 1.34 hop in one bump. The look-ahead was known and not acted on.)

Truth-source-type — branch the lookup

Each component in the registry carries a truth_source_type field that says where the compat truth lives. The skill branches on it when reading references/compat/<comp>.md:

Type	What the file contains
published_matrix	Distilled rows extracted from the vendor's k8s support matrix, per-version.
release_notes	Sifted highlights from GitHub release notes — breaking, CRD, k8s floor, ordering.
chart_metadata	Helm Chart.yaml's kubeVersion: constraint per chart-release tag, plus chart→app mapping.

The branching matters at maintenance time (freshen probes different sources) and at use time (the verdict cites a different kind of evidence per row).

References

• references/components.md — the 19-entry registry (table for single-axis, stanzas for multi-axis). Carries axis_type, truth_source_type, source URL, min_tracked_version. The lookup table the survey reads.
• references/cluster-survey.md — the canonical command set: kubectl/helm/pluto/apiserver-metric. Detection patterns for mapping running workloads onto registry entries.
• references/tooling.md — apiserver apiserver_requested_deprecated_apis metric (primary), pluto (static manifest scan), kubent dead.
• references/compat/README.md — file-format spec for per-component compat files.
• references/compat/<comp>.md — one per component. The load-bearing per-version compatibility signal. Air-gap-complete.
• references/version-verification.md — anti-fabrication protocol (House Rule #8): how to ground every cited version against real releases via gh (the anti-confirmation method + component→repo map). Read whenever the workstation is online.
• references/sources.md — URL index with Last verified: timestamps. Maintained by skill-improver freshen; read at use time only to surface staleness in the verdict if a row is past 90 days.
• references/report-format.md — pre-upgrade report layout: fixed section skeleton (header table, exec summary, survey, verdict, diff vs prior, action plan, methodology) with dynamic content. Used when the operator asks for a report rather than a verdict.