agentsclimarketplace

Gavel analyze

Skill dsolisp/gavel/skills/gavel-analyze

Analyze test suite run results for any framework. Classify failures as env, seed, app bug, test bug, or expected fail. Produce pass-rate by area and ranked recommendations. Use after any test suite run.From its SKILL.md

Install
npx -y skills add dsolisp/gavel --skill gavel-analyze

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 2 stars2 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

5.0 KB, as published. Nobody here has run it

Gavel Analyze

Post-run suite analysis. Framework-adaptive.

When to Use

  • After running a full or partial test suite
  • "Analyze results", "why did tests fail", "pass rate by area"

Failure Classification

CategorySignalsAction
envConnection refused, timeout, DNS failureFix env, re-run
seed404 on seeded data, empty grids, "not found"Re-seed database
app bugAssertion fails on correct test logicFile bug report
test bugWrong locator, stale auth, wrong configFix test
test-maintenance-driftClustered failures, same route/area, element-not-found or renamed-control pattern after recent deploygavel-impact → gavel-healer
app-regression5xx responses, unhandled exceptions, stack traces in test outputgavel-bug (confirm + report)
flakeIntermittent failures, race conditions, retry-dependent resultsgavel-flake (triage)
expected failMarked expected failure (test.fail, @ExpectedFailure)Document, link bug

Detecting test-maintenance-drift

Flag when most of these are true:

  • ≥3 failures in the same feature area, route, or spec directory
  • Errors are locator timeouts, missing text/roles, or "not attached" — not 5xx/auth
  • Pass rate was high recently; failures coincide with application merges
  • Retries do not help (not flaky — consistently missing DOM)

Hand off to gavel-impact for commit correlation, then gavel-healer to update automation. Do not classify as app-bug without evidence the product regressed.

Output Template

## Suite Analysis -- [project] -- [date]

### Summary
| Metric | Count |
|--------|------:|
| Passed | |
| Failed (unexpected) | |
| Skipped | |
| Expected fail | |
| Duration | |

### Pass Rate by Area
| Area | Passed | Total | Rate |
|------|-------:|------:|-----:|

### Failures
| Test ID | File | Classification | Notes |
|---------|------|----------------|-------|

### Recommendations (ranked)
1. [Highest impact action]

Report Ingestion

When CI artifacts are available, parse them before manual triage:

# JUnit XML
node scripts/parsers/junit.js path/to/results.xml --json

# Allure results directory
node scripts/parsers/allure.js path/to/allure-results --json

# Auto-detect file or directory
node scripts/parsers/index.js path/to/report --json

# Cluster failures for drift vs env signals
node scripts/parsers/junit.js results.xml --json | node scripts/cluster-failures.js

# Playwright JSON report
node scripts/parsers/playwright.js playwright-report.json --json

# Cypress JSON report
node scripts/parsers/cypress.js cypress-results.json --json

# Full CI analysis with optional commit correlation (gavel-impact)
node scripts/analyze-ci.js path/to/report.json \
  --app-repo path/to/app-repo \
  --area-map fixtures/config/area-map.example.json \
  --commits 15 --json

# Playwright HTML report directory
node scripts/parsers/playwright-html.js playwright-report/ --json

# One-shot: HTML report dir → parse + Gavel Result envelope
node scripts/analyze-ci.js playwright-report/ --envelope --project MySuite

# JSON envelope (schema-versioned, machine-readable)
node scripts/analyze-ci.js path/to/report.json --json-envelope --project MySuite

# Result envelope markdown (paste into gavel-analyze response)
node scripts/analyze-ci.js path/to/report.json --envelope --project MySuite
node scripts/analyze-ci.js path/to/report.json --app-repo ../app --area-map ./area-map.json --envelope

analyze-ci.js --envelope renders the standard Gavel Result block with Lead Summary (one-line scan) and Worker Handoff (full table). Use --json-envelope for machine-readable output with schema version gavel-result-envelope/1.1.0, validated against schemas/result-envelope.schema.json (invalid output exits 2).

analyze-ci.js JSON includes per-cluster classification, suspectCommits (when --app-repo is set), and nextAction. Commit correlation runs for every cluster (not just test-maintenance-drift). Use suspect commits as the starting point for manual gavel-impact validation — do not treat git keyword search as proof.

Use parser output to pre-fill Summary, Pass Rate by Area, and Failures tables. When a cluster shows locator-timeout in the same area (≥3 failures), classify as test-maintenance-drift and hand off to gavel-impact → gavel-healer.

Result Envelope

Analysis-only skill. Return templates/result-envelope.md when the report is complete.

  • Status DONE — summary, clusters, and ranked recommendations present
  • Status INCOMPLETE — raw failures listed but clustering or classification unfinished
  • Omit Changes and Verification unless you also fixed tests
  • Next Action must name gavel-impact, gavel-healer, gavel-env, or none

Keep looking

Skills are one crate of 609,689. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.