Full stack test gate

Full-stack test gate (unit → e2e, verified)

Prove an application works by running it: the project's own test suites, then a real browser that boots the app, drives every feature, screenshots each step, diffs the pixels, and names what nothing tested.

Core principle

"It works" is a claim; a green gate is evidence — and evidence has a scope. The loop is: run the suites → drive the flows → diff the screenshots → crawl for what no flow touches → fix → re-run.

Be honest about scope (the rule that keeps this skill correct): a green run means the flows you wrote passed. It does not mean the app works. Never report "fully tested" or "all features work". Report "N flows green, M% of controls untouched by any flow" — and the crawler exists precisely to produce that second number. → references/01-test-strategy-and-scope.md

When to use vs. not

• Use for: testing a web app end to end; writing/running unit, integration or regression tests; setting up Playwright; driving the UI and screenshotting it; visual regression; verifying a change didn't break anything; adding a test gate to CI; finding out what is untested.
• Not for: load/performance testing, security testing, or native mobile apps (different tooling). For accessibility, use the sibling a11y-gate skill.

Inputs to gather first

1. How to start the app — the command and the URL. Prefer the production build (npm run build && npm run preview); dev servers add overlays and dev-only warnings that produce false failures.
2. The existing test suites — how unit/integration tests run today. Don't guess: node run-suites.mjs --detect reads the repo and proposes them.
3. The critical journeys — the handful of things that must never break (sign in, search, checkout). Ask; don't infer from the routes.
4. A test account — a seeded user, never a real one. Credentials come from the environment (${TEST_PASSWORD}), never the config file.

Workflow

Load each reference when you reach its step.

1. Set the strategy and the honest scope. Decide what belongs in unit vs. e2e (if it can be a unit test, it must be). → references/01-test-strategy-and-scope.md

2. Install the harness and configure it.

   npm i -D @playwright/test playwright pixelmatch pngjs && npx playwright install --with-deps chromium
   cp scripts/e2e.config.example.json scripts/e2e.config.json   # edit baseUrl, webServer, suites, flows
   

   Merge scripts/package.snippet.json into package.json for the npm scripts.

3. Wire the existing suites and gate on them — cheapest first, so a broken build fails in seconds. → references/02-test-suites.md

   node scripts/run-suites.mjs --detect        # propose a suites[] block from the repo
   npm run test:suites                         # -> test-report/suites-report.md
   

4. Write the feature flows — one journey each, targeted by role + accessible name, asserting the user-visible outcome, including the unhappy paths. → references/03-authoring-flows.md

   npm run test:e2e -- --flow checkout --headed   # watch a single flow run
   npm run test:e2e                               # the gate -> gallery.html + e2e-report.md
   

5. Lock the look in place with visual regression on the named shot steps. → references/04-screenshots-and-visual-regression.md

   npm run test:visual          # fails on the first run: no baseline yet
   npm run test:visual:update   # review the screenshots, THEN accept them
   

6. Find what you didn't test. The crawler inventories the app and names the routes and controls no flow touches. → references/05-feature-coverage.md

   npm run test:coverage            # -> coverage-report.md
   npm run test:coverage -- --suggest   # stub flows for the gaps (assertions are on you)
   

7. Triage red, and treat flaky as broken. → references/06-debugging-and-flakes.md

8. Gate in CI, then complete the human sign-off for what automation can't reach (real emails, payments, migrations, exploratory). → references/07-ci-and-sign-off.md

   npm run test:ci        # Playwright, with retries + trace on failure
   npm run test:report    # open the HTML report
   

Try it on the bundled demo first

The skill ships a working app and a deliberately broken one, so you can watch the gate pass and then watch it fail. A gate you've never seen fail is not a gate.

node scripts/e2e-drive.mjs --config scripts/e2e.config.demo.json         # 7 flows, exit 0
node scripts/examples/serve-demo.mjs --port 5177 --broken &              # inject a real bug
node scripts/e2e-drive.mjs --config scripts/e2e.config.demo.json         # 4 flows fail, exit 1

What's in this skill

• scripts/run-suites.mjs — runs the project's unit/integration/regression suites (any language: a suite is a shell command + an exit code), parses JUnit XML for per-test detail, gates on required failures. --detect proposes a config from the repo.
• scripts/e2e-drive.mjs — the main gate. Boots the app, drives every flow in a real browser, screenshots each step, and watches the console, uncaught exceptions and the network the whole time. Writes e2e-report.md + a screenshot gallery.html.
• scripts/visual-diff.mjs — pixel-diffs the named shot screenshots against a committed baseline; --update to accept an intended change.
• scripts/crawl-features.mjs — crawls the running app and reports which routes/controls no flow touches. The honest half of "we tested everything".
• scripts/e2e.spec.ts + e2e.setup.ts + playwright.config.ts — the same flows under @playwright/test for CI: retries, traces, video, HTML report.
• scripts/lib/ — config (+ ${ENV} interpolation), server boot, the flow driver, the observers, the report writers.
• scripts/examples/ — the demo app, its --broken mode, and a fake unit suite.
• references/01–07 — strategy & honest scope, test suites, authoring flows, screenshots & visual regression, feature coverage, debugging & flakes, CI & the manual sign-off checklist.

Definition of done

• test:suites green — every required suite passes.
• test:e2e green — every critical journey runs, including the unhappy paths, with 0 console errors, 0 uncaught exceptions, 0 failed requests.
• test:visual green against a reviewed baseline (generated in the same environment CI uses).
• test:coverage read, not just run — every uncovered route/control is either now covered or explicitly, in writing, decided not to be.
• No flow is ⚠️ flaky. A flaky flow is an unfixed bug.
• Every bug fixed in this change has a regression test that fails on the old code.
• CI runs the gate; the manual sign-off checklist (07) is complete.

Guardrails — avoid these mistakes

• Don't claim the app works because the gate is green. It proves your flows passed. Say what's untested; the coverage report tells you.
• A flow with no assertion tests nothing. Clicking through the UI only proves the app didn't crash. State what "it worked" means.
• Target by role + accessible name, not CSS classes. A selector that breaks on a class rename teaches the team to ignore the suite.
• {"do":"wait","ms":500} is the #1 cause of flakes. Wait for a condition (waitFor, expect*), not for the clock.
• Flaky ≠ passing. Retries exist to surface flakes, not to hide them. Never fix one by raising a timeout.
• Never --update a baseline to make the build green. Look at the diff first; that is the moment the regression either gets caught or ships.
• Don't silence your own app's console errors. ignoreConsole is for third-party noise. An exception your code throws is a bug, not noise.
• Test the production build, and never against production data or a real user account.
• Don't put Playwright/Cypress in suites — the e2e layer is the other scripts; you'd run it twice.