Gavel analyze
Analyze test suite run results for any framework. Classify failures as env, seed, app bug, test bug, or expected fail. Produce pass-rate by area and ranked recommendations. Use after any test suite run.From its SKILL.md
npx -y skills add dsolisp/gavel --skill gavel-analyzeAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 2 stars2 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
SKILL.md
5.0 KB, as published. Nobody here has run it
Gavel Analyze
Post-run suite analysis. Framework-adaptive.
When to Use
- After running a full or partial test suite
- "Analyze results", "why did tests fail", "pass rate by area"
Failure Classification
| Category | Signals | Action |
|---|---|---|
| env | Connection refused, timeout, DNS failure | Fix env, re-run |
| seed | 404 on seeded data, empty grids, "not found" | Re-seed database |
| app bug | Assertion fails on correct test logic | File bug report |
| test bug | Wrong locator, stale auth, wrong config | Fix test |
| test-maintenance-drift | Clustered failures, same route/area, element-not-found or renamed-control pattern after recent deploy | gavel-impact → gavel-healer |
| app-regression | 5xx responses, unhandled exceptions, stack traces in test output | gavel-bug (confirm + report) |
| flake | Intermittent failures, race conditions, retry-dependent results | gavel-flake (triage) |
| expected fail | Marked expected failure (test.fail, @ExpectedFailure) | Document, link bug |
Detecting test-maintenance-drift
Flag when most of these are true:
- ≥3 failures in the same feature area, route, or spec directory
- Errors are locator timeouts, missing text/roles, or "not attached" — not 5xx/auth
- Pass rate was high recently; failures coincide with application merges
- Retries do not help (not flaky — consistently missing DOM)
Hand off to gavel-impact for commit correlation, then gavel-healer to update automation. Do not classify as app-bug without evidence the product regressed.
Output Template
## Suite Analysis -- [project] -- [date]
### Summary
| Metric | Count |
|--------|------:|
| Passed | |
| Failed (unexpected) | |
| Skipped | |
| Expected fail | |
| Duration | |
### Pass Rate by Area
| Area | Passed | Total | Rate |
|------|-------:|------:|-----:|
### Failures
| Test ID | File | Classification | Notes |
|---------|------|----------------|-------|
### Recommendations (ranked)
1. [Highest impact action]
Report Ingestion
When CI artifacts are available, parse them before manual triage:
# JUnit XML
node scripts/parsers/junit.js path/to/results.xml --json
# Allure results directory
node scripts/parsers/allure.js path/to/allure-results --json
# Auto-detect file or directory
node scripts/parsers/index.js path/to/report --json
# Cluster failures for drift vs env signals
node scripts/parsers/junit.js results.xml --json | node scripts/cluster-failures.js
# Playwright JSON report
node scripts/parsers/playwright.js playwright-report.json --json
# Cypress JSON report
node scripts/parsers/cypress.js cypress-results.json --json
# Full CI analysis with optional commit correlation (gavel-impact)
node scripts/analyze-ci.js path/to/report.json \
--app-repo path/to/app-repo \
--area-map fixtures/config/area-map.example.json \
--commits 15 --json
# Playwright HTML report directory
node scripts/parsers/playwright-html.js playwright-report/ --json
# One-shot: HTML report dir → parse + Gavel Result envelope
node scripts/analyze-ci.js playwright-report/ --envelope --project MySuite
# JSON envelope (schema-versioned, machine-readable)
node scripts/analyze-ci.js path/to/report.json --json-envelope --project MySuite
# Result envelope markdown (paste into gavel-analyze response)
node scripts/analyze-ci.js path/to/report.json --envelope --project MySuite
node scripts/analyze-ci.js path/to/report.json --app-repo ../app --area-map ./area-map.json --envelope
analyze-ci.js --envelope renders the standard Gavel Result block with
Lead Summary (one-line scan) and Worker Handoff (full table). Use --json-envelope
for machine-readable output with schema version gavel-result-envelope/1.1.0,
validated against schemas/result-envelope.schema.json (invalid output exits 2).
analyze-ci.js JSON includes per-cluster classification, suspectCommits (when
--app-repo is set), and nextAction. Commit correlation runs for every cluster
(not just test-maintenance-drift). Use suspect commits as the starting point
for manual gavel-impact validation — do not treat git keyword search as proof.
Use parser output to pre-fill Summary, Pass Rate by Area, and Failures tables.
When a cluster shows locator-timeout in the same area (≥3 failures), classify as
test-maintenance-drift and hand off to gavel-impact → gavel-healer.
Result Envelope
Analysis-only skill. Return templates/result-envelope.md when the report is complete.
- Status
DONE— summary, clusters, and ranked recommendations present - Status
INCOMPLETE— raw failures listed but clustering or classification unfinished - Omit Changes and Verification unless you also fixed tests
- Next Action must name
gavel-impact,gavel-healer,gavel-env, ornone