Analyze failures

Skill katalon-labs/true-skills/skills/analyze-failures

Install

npx -y skills add katalon-labs/true-skills --skill analyze-failures

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

3 stars3 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Triage Katalon True Platform/TestOps test failures and file defects. Use when you need to investigate failed test results, classify each failure as product defect vs automation defect vs environment/data issue, cluster failures by common signature, find likely root cause from execution data, and optionally create ALM-linked defects for real product bugs. This is failure diagnosis and defect filing; for the overall ship/no-ship release call use release-analyze, and for repairing the tests themselves use test-maintenance.

SKILL.md

3.9 KB, as published. Nobody here has run it

Katalon Analyze Failures

Use this skill for the failure-analysis part of the report/analysis stage: turn a set of failed results into a diagnosis and, when warranted, filed defects. The core value is classification — separating real product bugs from automation and environment noise.

Availability Boundary

Available via MCP: read results (read_test_result, read_execution_test_results, find_test_results, read_execution), defect context (fetch_defect_data), ALM discovery + filing (find_alm_integration_projects, create_defect).
Not directly available: AI root-cause summarization and automation-error-pattern analytics are TestOps/Studio product features, not MCP calls — narrate their availability, do not claim to call them. create_defect requires a known failed test result ID and ALM integration details; there is no ID-less defect creation.

Triage Workflow

+---------------------+     +----------------------+     +----------------------+
| Collect failures    | --> | Classify each        | --> | Cluster by signature |
| read results        |     | product/auto/env     |     |                      |
+---------------------+     +----------------------+     +----------------------+
                                                                   |
                                                                   v
                                                         +----------------------+
                                                         | File defects (asked) |
                                                         +----------------------+

Steps and tool rules

Collect the failures. find_test_results (recent/specific) or read_execution_test_results for a run; read_test_result per failed case for detail.
Classify each failure into one bucket:
- Product defect — the application behaved wrong (assertion on real behavior failed, unexpected error/state). Candidate for a filed defect.
- Automation defect — the test is wrong (bad locator, timing, stale data, broken step). Route to test-maintenance.
- Environment / data — infra, account, network, fixture, or AUT-state issue. Route to re-run after fix.
Cluster by signature. Group failures with the same error message / step / object so one root cause is not filed as N defects.
Check existing defects. fetch_defect_data to avoid duplicate filings.
File defects only when asked and only for product defects. find_alm_integration_projects -> create_defect with the failed result ID. Ask before creating unless the user explicitly requested defect filing.
Report. Per cluster: classification, likely cause, affected cases, and action (file / repair / re-run).

Prompt recipes

Triage the failures in execution 8842: which are product bugs vs flaky tests vs environment?
Cluster today's failed results by root cause and tell me what to file.
File defects for the confirmed product bugs in the checkout suite and link them to the failed results.

Hand-offs

Automation defects / flaky -> test-maintenance.
Ship decision from the failure picture -> release-analyze.
Coverage gap exposed by a failure -> test-plan.

Read references/failure-triage.md before classifying. Consult the orchestrator's references/unavailable-capabilities.md for defect-filing boundaries.