Ai agents generation and release

ai-agents Generation and Release

<!-- vendor-portability: contributor-facing knowledge pack for the rjmurillo/ai-agents repo itself. It intentionally references .agents/architecture, .agents/retrospective, .claude/lib, scripts/hook_utilities, scripts/github_core, scripts/ai_review_common, scripts/sync_plugin_lib.py, scripts/validation, build/generate_agents.py, build/scripts, templates/agents, and templates/platforms because its audience is repo contributors, not plugin consumers. Issue #2050. -->

Runbook for the build, generation, mirroring, versioning, and release machinery of this repo. Every command below was executed or read from source on 2026-07-02/03 and re-verified against the working tree on 2026-07-29; re-verify with the one-liners in Provenance before trusting a volatile number.

Jargon, defined once:

• Generator: a Python function or script that reads a canonical tree and writes a derived (generated) tree.
• Drift: the generated tree no longer matches what the generators would produce from the canonical tree.
• Canonical: the tree you are allowed to edit. Generated trees are outputs; you never hand-edit them.
• Semver: semantic version string MAJOR.MINOR.PATCH. The plugin manifests carry no version at all (ADR-092); the npm package still does.

Triggers

• regenerate the mirrors
• run the drift checks
• why is the plugin version gate red
• release the npm cli
• why is the drift gate red

When NOT to Use This Skill

You want	Use instead
Set up the dev environment, uv, hooks	ai-agents-build-and-env
WHY the seam is shaped this way, invariants	ai-agents-architecture-contract
Triage a red CI gate you do not understand	ai-agents-debugging-playbook
Change classification, commit caps, review gates	ai-agents-change-control
Escape hatches like the skip-drift-check marker	ai-agents-config-catalog

Process

Phase 1: Know the Seam Before You Edit

The generation seam is ASYMMETRIC (ADR-072 is PROPOSED and refines this; the running system is what follows). Find your file in this table BEFORE editing. Editing a generated tree is the classic failure (see Anti-Patterns).

Tree	Role	Canonical source	Regenerated by
templates/agents/*.shared.md	CANONICAL for agents	itself	n/a
src/vs-code-agents/	generated	templates/agents	build/generate_agents.py
src/copilot-cli/agents/	generated	templates/agents	build/generate_agents.py
docs/agent-catalog.md	generated	templates/agents	build_all.py (agent-catalog)
.claude/ (rules, skills, hooks, commands, settings.json)	CANONICAL for everything else	itself	n/a (generators NEVER write here)
src/copilot-cli/{skills,instructions,lib,hooks}	generated	.claude/ trees	build/scripts/build_all.py
.github/instructions/	generated	.claude/rules/	build_all.py (rules)
scripts/{hook_utilities,github_core,ai_review_common}	CANONICAL for shared Python	itself	n/a
.claude/lib/	mirrored copy (relative imports)	scripts/ packages	scripts/sync_plugin_lib.py
src/claude/	MANUAL hand-synced exception (ADR-036, superseded in governance by ADR-052 2026-08-25, procedure still operative and unimplemented)	edited by hand	no generator; semantic drift CI only

Generator inventory inside build/scripts/build_all.py (list GENERATORS, build_all.py:435, order is load-bearing per the comment at build_all.py:429):

#	Generator	Reads	Writes
1	agents	templates/agents/*.shared.md + templates/platforms/*.yaml	src/copilot-cli/agents/*.agent.md, src/vs-code-agents/*.agent.md
2	agent-catalog	templates/agents/*.shared.md	docs/agent-catalog.md
3	skills	.claude/skills/*/SKILL.md	src/copilot-cli/skills/
4	commands	.claude/commands/*.md (top level, not CLAUDE.md)	src/copilot-cli/skills/ (command-bridge skills)
5	rules	.claude/rules/*.md	.github/instructions/*.instructions.md AND src/copilot-cli/instructions/
6	lib	.claude/lib/	src/copilot-cli/lib/ (must land before hooks)
7	hooks	.claude/settings.json + .claude/hooks/	src/copilot-cli/hooks/ + src/copilot-cli/hooks/hooks.json

Facts that prevent confusion:

• build_all.py enforces a no-write invariant on .claude/ (REQ-003-010): if any generator writes there, the run exits 2 with REQ-003-010 VIOLATION. .claude/ is input only.
• Generated-tree ownership is exactly OWNED_PREFIXES = ("src/", ".github/instructions/", "docs/agent-catalog.md") (build_all.py:765). --check only flags staleness inside those prefixes.
• The hooks generator maps Stop, SubagentStop, PermissionRequest, and PreCompact to their PascalCase compatibility names. Stop and SubagentStop remain direct host registrations because their structured decisions require host-level merging. A reported drop needs an identified source registration; never assume it is normal.
• Every run overwrites the generation audit log in the gitignored build audit directory. Read it to see what each generator did.

Phase 2: Regenerate After Editing a Canonical Surface

Always commit the canonical edit and the regenerated output TOGETHER, in the same commit set, or the drift gates go red.

Interpreter note: build/generate_agents.py and build/scripts/build_all.py both import PyYAML through build/scripts/yaml_loader.py. Run them as uv run python <script> locally so the project venv supplies it. CI can invoke bare python3 because .github/actions/setup-code-env installs the locked dependencies system-wide first.

You edited	Run	Then
templates/agents/*.shared.md	uv run python build/generate_agents.py then uv run python build/scripts/build_all.py (refreshes docs/agent-catalog.md)	commit template + all regenerated files. If the same agent exists in src/claude/agents/, hand-apply the equivalent edit there (ADR-036 manual sync; semantic drift CI is the only net).
.claude/skills/, .claude/commands/, .claude/rules/	uv run python build/scripts/build_all.py	commit source + generated. No manifest edit (Phase 4)
.claude/hooks/ or .claude/settings.json	uv run python build/scripts/build_all.py	same as above. The build-all-check pre-push job in lefthook.yml re-runs build_all.py --check at git push time and blocks if any generated output (including shims under src/copilot-cli/hooks/) drifts, so regenerate BEFORE pushing.
scripts/hook_utilities/, scripts/github_core/, scripts/ai_review_common/	python3 scripts/sync_plugin_lib.py (writes .claude/lib/) THEN uv run python build/scripts/build_all.py (writes src/copilot-cli/lib/)	BOTH are required; skipping either fails the Validate Generated Files CI. No manifest edit.
src/claude/ (deliberate manual change)	nothing to regenerate	nothing to bump; the manifest carries no version (Phase 4)

Useful flags, verified against source:

• build/generate_agents.py: --validate (CI compare mode), --what-if (dry run, writes nothing), --templates-path, --output-root. Exit codes: 0 ok, 1 logic error or drift, 2 config error (docstring, generate_agents.py:13-16).
• build/scripts/build_all.py: --check (staleness gate; snapshots and restores owned trees), --clean, --audit-format json, --platform copilot-cli. Exit codes: 0 ok, 1 generator error, 2 config error or staleness, 3 audit blocklist violation (docstring, build_all.py:16-20).
• scripts/sync_plugin_lib.py: no flag syncs, --check is the CI dry run (exit 1 when out of sync). It also rewrites from scripts.x import to relative imports; do not "fix" those imports in .claude/lib/ by hand.

Phase 3: Run the Drift Gates Locally Before Pushing

Drift-gate matrix (all local commands verified runnable, all green on 2026-07-29):

Gate	Catches	Local command	CI enforcement
Agent template drift	templates edited without regen (or vice versa)	uv run python build/generate_agents.py --validate	validate-generated-agents.yml, agent-drift-detection.yml
Full pipeline staleness	any canonical edit not mirrored to owned prefixes	uv run python build/scripts/build_all.py --check	validate-generated-agents.yml; named pre-push job in lefthook.yml
Lib mirror drift	scripts/ package edited without sync	python3 scripts/sync_plugin_lib.py --check	validate-generated-agents.yml
Install parity	plugin install layout broken	python3 scripts/validation/run_install_parity_ci.py	validate-generated-agents.yml
Manifest description parity	.claude vs src/copilot-cli plugin descriptions carry component counts	python3 build/scripts/check_plugin_manifest_parity.py	validate-generated-agents.yml, agent-drift-detection.yml (the version half was retired with ADR-092)
Plugin version field present	a manifest or marketplace entry carries version	pre-pr-validation job in lefthook.yml (scripts/validation/pre_pr.py)	validate-plugin-version-bump.yml
Semantic agent drift (src/claude)	hand-synced tree diverging in meaning	python3 build/scripts/detect_agent_drift.py	drift-detection.yml, weekly cron Monday 09:00 UTC (line 15); similarity threshold default 80 (detect_agent_drift.py:666-668), with a recorded-baseline floor so a clean checkout does not fail

When a drift gate is red, the output shows the DIFFERENCE, not the DIRECTION. Ask "which side is canonical?" using the Phase 1 table before touching anything. The 2025-12-15 incident (retro: .agents/retrospective/2025-12-15-drift-detection-disaster.md) happened because an agent edited the SOURCE to match the GENERATED tree; the commit was reverted. Fix is always: edit canonical, rerun generator, commit both.

An emergency bypass marker for the drift gate exists but requires a reason and approval; see ai-agents-config-catalog. Route the decision through ai-agents-change-control.

Run the installed Copilot hook E2E in an isolated home. This executes the marketplace layout and generated hook scripts without changing live plugin state:

E2E_HOME="$(mktemp -d)"
HOME="$E2E_HOME" COPILOT_HOME="$E2E_HOME/.copilot" \
  copilot plugin marketplace add "$PWD"
HOME="$E2E_HOME" COPILOT_HOME="$E2E_HOME/.copilot" \
  copilot plugin install project-toolkit@ai-agents
HOME="$E2E_HOME" COPILOT_HOME="$E2E_HOME/.copilot" \
  RUN_INSTALLED_PLUGIN_HOOK_E2E=1 uv run pytest \
  tests/e2e/test_installed_plugin_hook_e2e.py -q

Do not run this setup in the live Copilot home. Copilot CLI 1.0.72-1 was observed removing the marketplace registration for copilot plugin uninstall project-toolkit while leaving the _direct/project-toolkit cache. The active session then lost its hook scripts and denied matching tools until the marketplace plugin was reinstalled.

Phase 4: Plugin Manifests Carry No Version

Three plugin manifests exist (verify: find . -name plugin.json -path "*claude-plugin*"):

Tree	Manifest	Plugin name
.claude/	.claude/.claude-plugin/plugin.json	project-toolkit (Claude)
src/copilot-cli/	src/copilot-cli/.claude-plugin/plugin.json	project-toolkit (Copilot)
src/claude/	src/claude/.claude-plugin/plugin.json	claude-agents

The rule (docstring of build/scripts/validate_plugin_version_bump.py): none of those manifests may carry a version field, and neither may an entry in either marketplace.json. Claude Code then resolves freshness from the git commit SHA, which moves on every merge. ADR-092 records the reversal of ADR-079's hand-bumped rule after issue #4080 measured 14 of 22 conflicting PRs conflicting on that one line.

Practical consequences:

• Any .claude/ content edit regenerates src/copilot-cli/ too, but neither manifest changes: both are version-free, and check_plugin_manifest_parity.py now checks description component counts only.
• src/claude/ is the same: no version to move.
• .github/instructions/ and src/vs-code-agents/ carry no plugin.json at all.
• Marketplace: .claude-plugin/marketplace.json lists two plugins (claude-agents from ./src/claude, project-toolkit from ./.claude). The old marketplace count validator and its YAML config were retired, so treat description-count validation as currently unenforced and keep counts honest by hand.

Phase 5: npm Release Path

The npm surface is packages/ai-agents-cli (package @rjmurillo/ai-agents; read package.json for the current version). It vendors the Claude kit into consumer repos. Release procedure (source: RELEASING.md, repo root):

1. Bump packages/ai-agents-cli/package.json version. Tag/version mismatch is a listed failure mode.
2. Local sanity: cd packages/ai-agents-cli && bun run build (runs bun build src/cli.ts --outdir dist --target node), bun test, tsc --noEmit via bun run typecheck. As of 2026-07-29 bun run typecheck exits 1 on this package with 38 errors (36 TS2591, 2 TS2503). @types/node 26.0.0 is installed, but the package tsconfig.json sets no types field, and TypeScript 7.0.2 no longer auto-includes it the way 5.9.3 does, so Node builtins are absent from the compilation. Issue #3865 tracks the fix: add "types": ["node"] and a CI job that runs typecheck and bun test. Before blaming your change for a red typecheck, check whether that fix landed: grep -q '"types"' packages/ai-agents-cli/tsconfig.json.
3. Commit to main via normal PR flow (branch discipline still applies; see ai-agents-change-control).
4. git tag vX.Y.Z then git push origin main --tags.
5. .github/workflows/publish.yml fires on v* tags: validates package metadata, publishes with OIDC provenance (id-token: write; NPM_TOKEN is fallback only). workflow_dispatch offers a dry-run input defaulting to true.
6. Verify: registry version, provenance badge on npmjs.com, clean-environment install (commands in RELEASING.md Verification section).

Rollback is roll-FORWARD: npm unpublish is restricted; fix, bump patch, retag (RELEASING.md Rollback procedures).

Anti-Patterns

Anti-pattern	Why it burns you	Do instead
Hand-editing src/copilot-cli/, src/vs-code-agents/, .github/instructions/, .claude/lib/	Next regen silently overwrites your edit; ruff deliberately exempts generated Python (pyproject.toml [tool.ruff.lint.per-file-ignores] ignores all selected rule families under src/copilot-cli/{hooks,skills}/**/*.py), so lint will not even look at it	Edit the canonical tree (Phase 1 table), regenerate
Editing source to make a drift gate green	The 2025-12-15 disaster; drift output shows difference, not direction	Identify canonical side first, regenerate outward
Syncing lib with only one of the two steps	sync_plugin_lib.py feeds .claude/lib/; build_all.py feeds src/copilot-cli/lib/; missing either fails CI	Run both, in that order
Bumping the two project-toolkit manifests in lockstep to satisfy version parity	Version parity retired with the field (ADR-092); check_plugin_manifest_parity.py no longer compares versions, and the bump itself now fails the gate	Nothing to bump; leave both manifests version-free
Re-adding a version to a manifest to clear a red gate	The gate fails on the field's presence, so the red is permanent	Delete the field; freshness already tracks the commit SHA
Treating Dropped: N as normal without checking the source	Current source registrations all map to Copilot events; an unexplained drop means contract drift	Read the generation audit log, identify the source registration, and require an explicit eventDrop decision
Running build_all.py with bare python3 in a fresh shell	PyYAML lives in the venv; import fails	uv run python build/scripts/build_all.py
Running installed-plugin E2E against live ~/.copilot	Version skew and ambiguous uninstall mutate active hooks; a missing hook root can deny every matching tool	Set both HOME and COPILOT_HOME to one isolated directory and register the worktree marketplace there
Treating src/claude/ as generated	It is the manual exception (ADR-036); no generator will save you	Hand-apply changes there; its manifest carries no version to bump

Verification

Run this checklist before pushing any change that touched a canonical or generated surface:

• uv run python build/generate_agents.py --validate exits 0
• uv run python build/scripts/build_all.py --check exits 0 (2 means staleness: regenerate and stage)
• python3 scripts/sync_plugin_lib.py --check exits 0 (only relevant if scripts/ packages changed)
• python3 build/scripts/check_plugin_manifest_parity.py exits 0
• python3 build/scripts/validate_plugin_version_bump.py exits 0 (no manifest or marketplace entry carries a version)
• Canonical edit and regenerated output are staged in the same commit set
• For an npm release: tag matches packages/ai-agents-cli/package.json version exactly

Provenance and Maintenance

Verified 2026-07-29 against the working tree (re-verification pass; the 2026-07-03 pass had rotted for build/scripts/build_all.py, build/generate_agents.py, pyproject.toml, scripts/sync_plugin_lib.py, .github/workflows/publish.yml, and .github/workflows/validate-generated-agents.yml). Volatile facts and how to re-check them:

Fact	Source	Re-verify
7 generators and their order	build/scripts/build_all.py:435-443	grep -n -A9 "^GENERATORS" build/scripts/build_all.py
OWNED_PREFIXES trio	build/scripts/build_all.py:765	grep -n "OWNED_PREFIXES" build/scripts/build_all.py
.claude/ no-write invariant	build/scripts/build_all.py:674 (rule), :1013 (snapshot), :1076-1081 (enforcement)	grep -n "REQ-003-010" build/scripts/build_all.py
build_all exit codes 0/1/2/3	build/scripts/build_all.py:16-20	sed -n '16,20p' build/scripts/build_all.py
generate_agents flags and exit codes	build/generate_agents.py:13-16,460-487	uv run python build/generate_agents.py --help
sync pairs scripts to .claude/lib	scripts/sync_plugin_lib.py:27-31	grep -n -A4 "SYNC_PAIRS" scripts/sync_plugin_lib.py
Plugin manifest locations, all version-free	the three plugin.json files	python3 build/scripts/validate_plugin_version_bump.py
Version-field prohibition and the ADR-092 reversal	build/scripts/validate_plugin_version_bump.py docstring	grep -n "WHY THE FIELD MUST BE ABSENT" build/scripts/validate_plugin_version_bump.py
Parity gate #2222	build/scripts/check_plugin_manifest_parity.py:1-16	python3 build/scripts/check_plugin_manifest_parity.py
Drift CI wiring	.github/workflows/validate-generated-agents.yml:165,174,212,225,239; agent-drift-detection.yml:146,156,159,171	grep -n "uv run python" .github/workflows/validate-generated-agents.yml
Weekly semantic drift cron, threshold 80	.github/workflows/drift-detection.yml:13-15; build/scripts/detect_agent_drift.py:666-668	grep -n "cron" .github/workflows/drift-detection.yml
Git hook jobs, filters, and validators	lefthook.yml	uv run --frozen lefthook validate
Ruff exemption for generated Python	pyproject.toml [tool.ruff.lint.per-file-ignores]	grep -n "src/copilot-cli" pyproject.toml
npm package, bun build, tag flow	packages/ai-agents-cli/package.json; RELEASING.md:35-54; .github/workflows/publish.yml:13-16	grep -n "tags" .github/workflows/publish.yml
Marketplace count validator retired	no dedicated count validator or marketplace counter YAML should exist	find . -name "*marketplace*count*" -not -path "./.venv/*"
Audit log path, gitignored	.gitignore:70	grep -n "build/audit" .gitignore
2025-12-15 direction story	.agents/retrospective/2025-12-15-drift-detection-disaster.md	python3 -c "import pathlib;print([p.name for p in pathlib.Path('.agents/retrospective').glob('*drift*')])"
ADR-036 superseded by ADR-052 (2026-08-25, procedure still operative), ADR-052 accepted and unimplemented, ADR-072 Proposed	the ADR-036, ADR-052, and ADR-072 architecture records	head -12 .agents/architecture/ADR-072-jtbd-plugin-architecture.md && head -12 .agents/architecture/ADR-036-two-source-agent-template-architecture.md && grep -n "Still operative as procedure" .agents/architecture/ADR-036-two-source-agent-template-architecture.md && head -12 .agents/architecture/ADR-052-template-strategy.md (chained with && so a missing source fails the whole probe rather than being masked by a later successful command; the grep verifies the "still operative" claim, not just frontmatter status; row re-verified 2026-08-25; other rows in this table carry their 2026-07-29 verification)

Maintenance: when a generator is added or removed from GENERATORS, when a fourth plugin.json appears, or if a marketplace count validator is reintroduced to replace the retired one, update Phase 1/4 tables and re-run every re-verify command above.