agentsclimarketplace

Playwright e2e suite review

Skill Raishin/vanguard-frontier-agentic/skills/qa/playwright-e2e-suite-review

Use this skill when reviewing a Playwright end-to-end test suite for flakiness, selector brittleness, isolation defects, and CI reliability. Trigger when a user provides Playwright spec files, a playwright.config.ts/js, a CI workflow that runs Playwright, or asks why their E2E suite is flaky, slow, or fails intermittently in CI but passes locally. This skill reviews test artifacts statically; it does not execute the suite or launch browsers.From its SKILL.md

Install
npx -y skills add Raishin/vanguard-frontier-agentic --skill playwright-e2e-suite-review

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

SKILL.md

4.2 KB, 797 tokens by cl100k_base, as published. Nobody here has run it

Playwright E2E Suite Review

Purpose

This skill reviews a Playwright end-to-end test suite for the defects that destroy CI trust at scale: flakiness, brittle selectors, broken test isolation, and unreliable CI configuration. A flaky E2E suite is worse than no suite — engineers learn to re-run failures instead of reading them, real regressions ship behind a green-after-retry checkmark, and the suite stops gating anything. The review catches hard waits, manual non-retrying assertions, implementation-coupled selectors, shared mutable state across tests, and retry/sharding misconfiguration before they erode confidence in the deploy pipeline.

Lean operating rules

  • Treat any use of page.waitForTimeout / waitForTimeout in a spec (not a debugging branch) as HIGH — fixed sleeps are the single largest source of Playwright flakiness; they either race the app or pad every run.
  • Treat manual non-retrying assertions (expect(await locator.isVisible).toBe(true), expect(await locator.textContent).toBe(...)) as HIGH — they snapshot a single instant and lose Playwright's auto-retry; use web-first assertions (await expect(locator).toBeVisible).
  • Treat selectors bound to implementation detail — deep CSS chains, nth-child indexes, generated/hashed class names, raw XPath — as HIGH for brittleness; prefer role-, label-, text-, or data-testid-based locators.
  • Treat tests that depend on ordering or share mutable state (module-level variables mutated across test blocks, a record created in test A read in test B) as HIGH — they break under parallelism, sharding, and --shuffle, and produce non-reproducible failures.
  • Treat retries set greater than 0 in CI with no flaky-test surfacing (no trace-on-retry, no flaky reporter, no quarantine) as HIGH — retries then silently mask real flakiness instead of buying time to fix it.
  • Treat trace/screenshot/video all disabled in the CI project as HIGH — a CI-only failure with no trace is undebuggable and forces blind re-runs.
  • Treat absolute waits on network (waitForLoadState('networkidle')) used as a general synchronization crutch as MEDIUM — it is fragile under analytics/polling; wait on the specific element or response instead.
  • Treat shared storageState / auth fixtures mutated by tests, or login performed inside every test instead of via a setup project, as MEDIUM — slow and a cross-test contamination risk.
  • Treat a single un-sharded CI job for a large suite, or fullyParallel: false without a stated reason, as MEDIUM — wall-clock time blocks every deploy.
  • Treat expect timeouts or global timeout raised well above default to make a suite "pass" as MEDIUM — masks a real slow path or race.
  • Do not recommend deleting or .skip-ing a flaky test as the fix without a root-cause category and a quarantine/tracking path.
  • Label every finding with evidence basis: spec/config text provided, documentation-based, or inference from absent configuration.

References

Load these only when needed:

Response minimum

Return, at minimum:

  • Flakiness findings (hard waits, manual assertions, network-idle crutches)
  • Selector brittleness assessment (locator strategy per spec)
  • Test isolation findings (shared state, ordering dependence, auth contamination)
  • Retry and observability assessment (retries vs. trace/flaky surfacing)
  • CI configuration findings (sharding, parallelism, artifact capture, timeouts)
  • Severity-labelled finding list (critical / high / medium / low)
  • Safe next actions

What ships with it: 2 files

7.9 KB alongside SKILL.md

references/

Keep looking

Skills are one crate of 613,531. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.