Playwright e2e

Playwright E2E — Hybrid Agent Workflow

Core principle (the hybrid)

Browser MCP (Playwright MCP) and Playwright Test are two different layers, not interchangeable:

Layer	Role	Traits
Playwright MCP	Authoring & debugging	Interactive, explores live app, writes plan + specs, investigates failures. Token-heavy, non-deterministic.
Playwright Test	Execution	Queue, retries, JUnit/HTML artifacts, CI gate. Deterministic, cheap, headless.

NEVER run the suite "through MCP" as the executor. Generate/repair specs with MCP, then execute with npx playwright test. MCP cannot give CI a pass/fail gate, a reproducible suite, or detached runs.

When to use this skill

• User wants to create UI/E2E automated tests for a web app.
• User wants tests run as a queue (sequential, with retries), leaving Markdown artifacts.
• User wants a CI/CD setup (GitHub Actions, self-hosted Mac).
• User is driving the work with an AI agent + Playwright MCP.

Workflow

Phase 1 — Generate (MCP)

1. Connect Playwright MCP (npx @playwright/mcp).
2. Explore the target flow live; write a human-readable plan to specs/<flow>.md (steps as a numbered list — this is the "artifact" the user asked for).
3. Generate tests/<flow>.spec.ts following references/best-practices.md.
4. If a spec already exists but is flaky/broken, use MCP to open the app and inspect the real DOM/locators before editing.

Phase 2 — Execute (runner)

Run the queue via the bundled script:

bash scripts/run_queue.sh --rerun-failed
# or a subset:
bash scripts/run_queue.sh --grep "checkout"

The script runs npx playwright test with list + html + junit reporters, optionally reruns only failed specs once, and converts JUnit → Markdown.

Phase 3 — Artifacts (Markdown)

scripts/junit_to_md.py turns test-results/junit.xml into reports/run-<timestamp>.md with a pass/fail table, durations, and failure details + a verdict line (✅ ALL GREEN / ❌ HAS FAILURES). This is the per-run Markdown artifact.

Phase 4 — CI/CD

Drop templates/github-actions-mac.yml into .github/workflows/. It uses a self-hosted macos runner (for Safari/WebKit fidelity), installs deps, runs the queue with --rerun-failed, and uploads both playwright-report/ (HTML+trace) and reports/ (Markdown) as artifacts.

Scripts

• scripts/run_queue.sh — queue execution + rerun-failed + artifact conversion.
• scripts/junit_to_md.py — JUnit XML → Markdown report.

Templates

• templates/playwright.config.ts — reporters (list/html/junit), retries, trace: 'on-first-retry', screenshot/video on failure, single worker in CI.
• templates/github-actions-mac.yml — GitHub Actions self-hosted Mac workflow.

Anti-patterns (full list in references/best-practices.md)

1. No hard sleeps (waitForTimeout(2000)) — use web-first assertions / auto-wait.
2. No document.querySelector inside evaluate() — use locators.
3. No chained tests (test 2 depends on test 1) — each test self-contained.
4. Prefer getByRole / getByTestId over CSS/XPath.
5. Auth via fixtures / storageState, not re-login in every test.
6. Assert on behavior, not implementation; use expect(...).toBeVisible().

Pitfalls

• MCP-as-executor: no suite, no gate, token blow-up. Keep MCP for authoring.
• Flaky by design: fixed sleeps and chained tests produce false reds.
• Missing junit output: configure the junit reporter to a file in playwright.config.ts, not just --reporter=junit (which prints to stdout).
• Self-hosted Mac runner not registered: the workflow waits forever — verify gh runner is online before relying on CI.

Verification

• Local: bash scripts/run_queue.sh --rerun-failed → expect reports/run-*.md with a verdict line.
• CI: push a branch, open the Actions run, confirm both artifacts uploaded.
• Quality gate: fail the pipeline when junit_to_md.py verdict is ❌ HAS FAILURES (exit code from npx playwright test already does this).

Portability to Qwen Code

This skill uses the Agent Skills standard. To use it in Qwen Code:

# from the skill directory
cp -r . ~/.qwen/skills/playwright-e2e/   # or your Qwen skills path

SKILL.md format, scripts, and templates are tool-agnostic (bash + python3 + node). No Hermes-specific APIs are used.