agentsclimarketplace
Sign in

Validating detections with atomic tests

Skill meltedinhex/analyst-ai-pack/skills/validating-detections-with-atomic-tests

Validates detection coverage by mapping Atomic Red Team tests to ATT&CK techniques, checking which of your detections cover each tested technique, and reporting coverage gaps. Activates for requests to validate detections with atomic tests, map Atomic Red Team to coverage, or find detection gaps against tested techniques.From its SKILL.md

Install
npx -y skills add meltedinhex/analyst-ai-pack --skill validating-detections-with-atomic-tests

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 21 stars21 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its file declares

Copied from the file, not written here

The file declares its own license as Apache-2.0. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.

SKILL.md

2.7 KB, 399 tokens by cl100k_base, as published. Nobody here has run it

Validating Detections With Atomic Tests

When to Use

  • You want to validate that your detections fire for specific ATT&CK techniques by correlating Atomic Red Team test definitions with your detection inventory.
  • You are measuring coverage and identifying gaps before or after running atomics.

Do not use the atomics themselves on production systems without authorization and isolation — they execute adversary behaviors. This skill performs mapping/coverage analysis offline.

Prerequisites

  • An export of Atomic test→technique mappings (CSV/JSON) and your detection inventory mapped to ATT&CK techniques.

Workflow

Step 1: Compute coverage

python scripts/analyst.py coverage --atomics atomics.json --detections detections.json

Joins atomic-tested techniques with your detections' covered techniques and reports covered vs uncovered techniques.

Step 2: Prioritize gaps

Rank uncovered techniques by the number of atomic tests (more atomics → more validated adversary behavior you currently miss).

Step 3: Validate live (authorized/isolated)

For covered techniques, run the corresponding atomic in an isolated range and confirm the detection fires; record results.

Step 4: Close gaps

Author detections for high-priority uncovered techniques and re-run coverage.

Validation

  • Coverage join keys on ATT&CK technique IDs (including sub-techniques).
  • Covered and uncovered technique lists are complete and disjoint.
  • Gap prioritization reflects atomic-test counts.

Pitfalls

  • Sub-technique vs parent-technique mismatch (T1059 vs T1059.001) skewing coverage.
  • Claiming coverage from a rule that exists but is disabled/untuned.
  • Running atomics outside an isolated, authorized range.

References

What ships with it: 3 files

4.1 KB alongside SKILL.md, 1 of them executable

references/

scripts/

Keep looking

Skills are one crate of 325,949. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.