agentsclimarketplace

Tdd bdd harness

Skill furkanzt/claude-tdd-bdd-harness/skills/tdd-bdd-harness

Bootstrap or maintain a self-enforcing TDD+BDD discipline in any repo — executable Gherkin scenario catalogue with machine-verified test citations, coverage matrix, CI meta-guard, gap-grinding workflow, adversarial test audits, and a quality-loop layer for behaviours only a live system can prove. Use when the user says "adopt TDD/BDD", "scenario catalogue", "spec harness", "test discipline", or wants this project's testing methodology ported to another repo.From its SKILL.md

Install
npx -y skills add furkanzt/claude-tdd-bdd-harness --skill tdd-bdd-harness

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

17.5 KB, as published. Nobody here has run it

TDD+BDD Harness — portable self-enforcing spec discipline

Battle-tested in Ayse_Hoca_baglam_v3 (529 scenarios / 0 gaps, pytest + Vitest, survived two adversarial audits). The core idea: every observable behaviour is a Gherkin scenario bound to a real test by a machine-verified citation — a promise ledger that CI refuses to let drift.

Reference implementation: any repo where this harness is already installed — its TDD.md, BDD.md, features/, meta-guard test and scripts/quality/ show every pattern in the flesh. The templates/ beside this SKILL.md are self-contained; nothing else is required.

The contract being installed

  1. One catalogue — features/*.feature (Gherkin, one file per domain), indexed by a generated Scenarios.md + docs/scenarios/coverage-matrix.md.
  2. Citations — every scenario carries # Backed by: <ref> naming its executable proof. Three ref forms, all machine-resolved:
    • unit/regression test: tests/test_x.py::test_fn (file + function checked)
    • file>title test (JS): web/src/x.test.ts > does the thing (file checked)
    • loop check (live-only behaviour): scripts/quality/checks.py::check_fn
  3. The ratchet — a scenario with no proof must be tagged @gap @wip (work-in-progress escape hatch). A bare @gap fails CI. Once BARE gaps hit 0, they stay 0 — @gap @wip is the only sanctioned way to carry an open one, and it clears before the feature ships.
  4. The meta-guard — a test file that fails CI on: unparseable Gherkin, unbacked scenarios, ghost/renamed citations, stale matrix totals. The guard's checkers are pure and have red-path proofs (synthetic bad inputs) so the guard itself is tested.
  5. Same-commit rule — scenario + test + code land together; behaviour changes update the scenario in the same commit; regenerate the matrix.
  6. The red-proof — a new test earns its citation by surviving mutation: invert the branch it guards, confirm the test fails, revert. Unconditional — always, solo or with a second reviewing model — it's what catches a vacuous fixture at write time instead of the next audit (Phase 4). Two carve-outs: tests that already existed when Phase 2 cited them are out of scope (Phase 5's adversarial audit covers that legacy set instead), and a @loop live check's red-proof attaches to its pure-core unit pin, never the live check itself (Phase 6).

Reviewer gate (optional — model-pairing mode)

Every phase below runs solo (one model, one context) or split across an implementer (writes, executes, has file access) and a reviewer (higher-judgment second opinion, no file access, sees only what's put in front of it) — e.g. Sonnet 5 as implementer with Opus 5 as advisor via a tool like Claude Code's advisor. Splitting isn't free — it only pays where gated below, and it fails quietly if these three rules are skipped:

  1. The artifact must be in front of the reviewer as literal text before the gate fires. If your reviewer only sees the running conversation (like advisor), that means pasted into the transcript — "I wrote features/chat.feature" gives it nothing, the scenario text does. If you invoke it with an explicit prompt instead (as in the refute-by-default subagent fallback below), the artifact goes in the prompt. Either way, a reviewer only sees what was actually handed to it.
  2. Fan-out phases (2, 4) push work into subagents whose output doesn't reach the main transcript by default. Gate the boundary, not the interior — before fan-out (the plan) or after it completes (the finished artifact), never mid-subagent. Phase 4 runs in two rounds specifically so a gate has a boundary to sit on: round one returns test specs AS DATA without implementing anything; the gate consumes that batch; round two implements. Phase 2's agents write straight to their own exclusive feature file instead — no merge needed, but nothing returned as data either — so its gate reads the finished file back once that domain is done. (Phase 5 isn't fan-out in model-pairing mode — the reviewer replaces the spawned skeptics outright, see Phase 5.)
  3. Batch within each gate point — one call per domain/package, not one per scenario or test. A transcript-forwarding reviewer re-sends the whole conversation every call; cost and signal dilution scale with call count either way. Gate the five phases below; don't gate every scenario.
Gate atReviewsWhy this point, not another
Phase 0domain/feature-file boundaries, before locking them innear-impossible to fix once hundreds of scenarios cite them
Phase 1the mocking-seam paragraph in the TDD.md draftthe wrong seam makes the whole suite dishonest
Phase 2each domain's Then-clauses, batched once that domain is done — cited ones against the source of the test they cite, @gap ones against what a test could realistically provecatches spec overstatement (and unrealistic @gap Thens) at write time, not audit time
Phase 4the package's batched test specs — name, assertion, seam — before any of them is implementedsee handoff contract below
Phase 5reviewer replaces the spawned skeptic subagents for this phase; implementer runs the refute pass and fixes survivorsthis phase already is a reviewer role

Phase 3 (copy templates, fill # EDIT ME, wire CI), matrix regeneration, apply_backed_by.py runs, and Phase 6 (its two honesty rules are self-checks, not reviewer judgment calls) are mechanical — no gate.

Handoff contract (Phase 4). "The reviewer writes the test" isn't literally achievable — it has no file access, and fan-out mode has no single implementer to hand off to across a pause. Phase 4 runs in two rounds instead: round one, each domain agent drafts and returns a test specification — exact test name, the precise assertion, the seam to mock — AS DATA, then exits without implementing anything. The batch is pasted for the reviewer. Round two, a fresh agent per domain receives its approved spec verbatim in the prompt — paraphrasing it is exactly where the reviewed quality leaks back out — writes whatever production code, if any, the test requires, and returns the scenario→test mapping for apply_backed_by.py same as any other gap-fill. The spec's mutation target isn't reviewed as a claim to be graded (a reviewer with no file access can't check it anyway) — the red-proof (below) verifies it by running it, which is strictly stronger than an opinion either way.

Why this beats solo — mutation, not opinion. A reviewer re-reading a test, even a stronger one, is still one perspective grading plausibility. The red-proof — mandatory in Phase 4, solo or paired, see there — isn't an opinion: invert the guarded production branch, run the test, it MUST fail, then revert. This, not gate placement, is what lets a split workflow match or exceed a solo strong model, and it doesn't depend on a reviewer tool existing at all.

No reviewer tool available? Degrade a gate to a refute-by-default subagent — spawn one with the artifact in its prompt, its only job to argue the artifact is wrong — or a manual pause for a human look. Rules 1 and 3 above already say how each applies either way. Nothing in the phases below assumes a specific tool exists — the gates are a workflow choice layered on top.

Phases (run what the situation needs; a fresh repo needs all)

Phase 0 — Assess (read-only)

Detect: language(s), test runner(s), existing test count/conventions, CI files, how the app boots. Decide the citation prefixes (UNIT_REF_PREFIXES, FILE_TITLE_REF_PREFIXES — loop-check refs ride under UNIT_REF_PREFIXES too, via its scripts/quality/ entry) and where things will live. Inventory the domains (5–12 feature files is the sweet spot). Gate: in model-pairing mode, paste the domain inventory for the reviewer before locking in feature-file boundaries (Reviewer gate above).

Phase 1 — Methodology docs

Write TDD.md (red→green→refactor manual: where tests live TABLE incl. the live-check layer, non-negotiable rules, mocking policy at the external-service seam, run commands, definition of done, anti-patterns) and BDD.md (ubiquitous-language glossary built from THIS project's real domain terms — never copy another project's; scenario format; the three citation forms; the @gap @wip workflow; an HONESTY NOTE stating exactly what executes where). Ground both in the target repo — file paths, real commands, real seams. Gate: in model-pairing mode, paste the mocking-seam paragraph for the reviewer before finalizing TDD.md — the wrong seam makes the whole suite dishonest.

Phase 2 — Scenario catalogue

Explore the codebase (parallel agents for big repos, one owner per domain file — exclusive namespaces, no write conflicts). Write scenarios grounded in REAL current behaviour: Given/When/Then, one behaviour each, deterministic Thens (contract, not prose), domain terms verbatim (keep native-language terms in step text; English Gherkin keywords). Cite existing tests where they already prove the invariant; tag the rest @gap @wip — the @wip comes off only when Phase 4 adds the citation. Gate: in model-pairing mode, once a domain's file is done, paste its drafted Then-clauses for the reviewer — cited ones next to the source of the test they cite, @gap ones on their own against what a test could realistically prove — one batch per domain, not per scenario. Catches spec overstatement (and unrealistic @gap Thens) at write time, not audit time. Install the tools (below), generate the matrix — the GAP list is now the backlog.

Phase 3 — Harness install

Copy from this skill's templates/ into the repo and fill the # EDIT ME config blocks:

  • templates/build_coverage_matrix.py → scripts/quality/
  • templates/apply_backed_by.py → scripts/quality/
  • templates/test_catalogue_integrity.py → tests/ (pytest projects; for JS-only repos port the four pure checkers to a *.test.ts — they are ~80 lines of regex walking, no Python magic). Install gherkin-official as a real dev dependency — the Gherkin-parse check importorskips it and degrades to a silent no-op without it, which would make the "fails CI on unparseable Gherkin" contract item false as shipped.
  • templates/live_checks_skeleton.py → scripts/quality/ (Phase 6, optional) Wire CI: the existing test job must run the meta-guard (it's just a test file) and any local gate script should too. Verify: whole suite green, then prove the guard BITES on all four failure modes, not just one — rename a citation (reference check), drop a # Backed by: line (traceability check), hand-edit the matrix total (freshness check), AND break a feature file's Gherkin syntax (parse check — the one that silently no-ops if gherkin-official didn't actually install) — revert each after confirming red.

Phase 4 — Gap grinding

Close @gaps in parallel packages (one agent per domain, exclusive test-file namespaces; agents return scenario→test mappings AS DATA, applied deterministically via apply_backed_by.py — never let agents hand-edit shared feature files concurrently). Gap-fills usually retro-fit a test to behaviour that already works (Phase 2 grounded scenarios in REAL current behaviour), so there's typically no red→green step — the red-proof below IS the red. Where a gap genuinely needs new production code, red→green applies first and the red-proof still runs after. Rules: tests NEVER hit the network/external services (mock at the client seam in the CONSUMER namespace); temp DB/dirs only; no watch-mode commands in agents. Red-proofs mutate shared production source, so they never run inside the parallel domain agents themselves: each agent returns its red-proof target (file, branch, expected-failing test) as part of its mapping data, and the orchestrating context — the one thread coordinating this fan-out, solo or paired — runs every red-proof serially, one at a time, once implementation is done.

Every new test earns its citation by surviving its red-proof: invert the branch it guards, confirm the test fails, revert. Unconditional — always, solo or with a second reviewing model — it's what catches a vacuous fixture at write time instead of the next audit. Tests that already existed when Phase 2 cited them are out of scope (Phase 5's adversarial audit covers that legacy set instead); a @loop live check's red-proof attaches to its pure-core unit pin instead of the live check (Phase 6); every other test authored in Phase 4 or later needs it.

Gate: in model-pairing mode, Phase 4 runs in two rounds so the gate has a real boundary (handoff contract above): round one, domain agents draft and return test specs AS DATA, then exit without implementing; paste the batch for the reviewer. Round two, fresh domain agents implement only reviewer-approved specs and return the scenario→test mapping — same as solo mode from there, including the serial red-proof pass above.

Phase 5 — Adversarial audit (do this; green lies)

Spawn skeptic subagents whose job is to REFUTE the suite. Gate: in model-pairing mode the reviewer replaces the spawned skeptics for this phase (not a fan-out here) — paste one domain's test file per call, bounded, and let it hunt the disease list below. The known disease catalogue (every one of these was found in the wild):

  • self-referential pin — test asserts a test-local string against itself (fix: constants live in production modules; tests import THEM)
  • seam-short pin — bare function tested, its registration/wiring unpinned (fix: assert on the app/router/registry object too)
  • source-text pin — asserts inspect.getsource() patterns instead of behaviour (fix: monkeypatch the collaborator to raise and drive the real path)
  • spec overstatement — scenario's Then promises more than the cited test proves (fix: strengthen the test or honestly weaken the wording)
  • vacuous fixture — the test input never reaches the guarded branch (fix: red-proof it — invert the production branch, test MUST fail, revert clean; a hit on a test written after Phase 4's rule took effect means the red-proof was skipped — treat it as a process failure, not just a test fix) Verify findings with refute-by-default second passes before fixing — run each pass from a context with file access, never by whatever raised the finding (a skeptic subagent, or in model-pairing mode, the reviewer); half of raw findings die under scrutiny. Fix survivors: production is usually already correct for these diseases, so there's no natural red — red-proof the repaired test the same way Phase 4 does.

Phase 6 — Loop layer (behaviours unit tests structurally cannot prove)

Real process boot, real files on disk, real HTTP headers, real browser CSP, real paid API calls: these live as named check_* functions in a scripts/quality/ script the project's recurring quality loop (or cron/CI nightly job) executes. Scenarios covering them are tagged @loop and cite the check function — the meta-guard resolves those citations on every CI run even though the checks execute on the loop's cadence. Two honesty rules: unit-provable behaviour never hides in the loop layer; every loop check's pure core (parsers, detectors) ALSO gets a normal unit pin, cited alongside. The red-proof rule attaches to that pure-core pin, not the live check itself — a live check can't be branch-inverted without triggering the real process/HTTP/paid call it exists to test.

Hard rules (carry into the target repo's CLAUDE.md)

  • Behaviour + scenario + test + matrix regen = ONE commit. A scenario that lies is worse than none.
  • ARMED means the cited test EXISTS (machine-verified). Whether it SUFFICES is guarded by audits — say so in the matrix header, never oversell.
  • Bare @gap cannot be committed; @gap @wip only while actively developing.
  • Unit tests never touch network/external services; the only sanctioned real calls live in loop-layer smokes that cost money knowingly.
  • Every production bug found later earns a regression test + scenario before the fix ships (the incident becomes a permanent pin).
  • Discover the project's REAL type-check/build command and pin it in TDD.md (e.g. tsc --noEmit can be a no-op under project references — verify it checks files before trusting it).
  • A new test earns its citation by surviving a red-proof: invert the branch it guards, confirm the test fails, revert. Mandatory, always — it's what catches a vacuous fixture at write time, not just at the next audit. A @loop live check's red-proof attaches to its pure-core unit pin, never the live check itself (it can't be branch-inverted without triggering the real process/HTTP/paid call it exists to test).

Maintenance loop (steady state)

New feature → scenarios first (@gap @wip) → red tests → green code → red-proof each new test → cite via apply_backed_by.py or by hand → build_coverage_matrix.py → commit. The meta-guard makes skipping any step a CI failure, not a code-review nag.

Keep looking

Skills are one crate of 608,352. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.