Feedback loops
Cyber Threat Intelligence Skills for each stage of the CTI Lifecycle.
npx -y skills add Liberty91LTD/cti-skills --skill feedback-loopsAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 8 stars8 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Feedback loop implementation for continuous CTI improvement. Consumer feedback, analyst retrospectives, source quality tracking.
SKILL.md
3.4 KB, as published. Nobody here has run it
Feedback Loops
Feedback is Phase 6 of the CTI Hyperloop. Without it, the cycle is open-ended and quality degrades over time.
Types of Feedback
1. Consumer Feedback
Intelligence consumers (stakeholders) assess the value of products they receive.
Collection points:
- After every significant product delivery
- During quarterly PIR reviews
- During stakeholder briefings
Questions:
| Question | Response Options |
|---|---|
| Was this intelligence useful for your decision-making? | Very useful / Somewhat useful / Not useful |
| Was it delivered in time to act on? | Yes / Partially / No |
| Was the format appropriate for your needs? | Yes / Too technical / Too high-level / Wrong format |
| Was the confidence assessment helpful? | Yes / Unclear / Not included |
| What should we prioritise next? | Free text |
Actions:
- Useful + timely → Continue current approach, reinforce collection sources
- Useful + late → Improve detection/triage speed, adjust collection frequency
- Not useful → Review PIR alignment, stakeholder needs, product format
- Format wrong → Adjust per stakeholder-management skill
2. Source Quality Tracking
Track the reliability of intelligence sources over time.
For each source used in products:
- Was the information confirmed, partly confirmed, or disproven?
- Did the source's Admiralty rating need adjustment?
- Were there timeliness issues?
Actions:
- Consistently confirmed → Upgrade reliability rating (e.g., C → B)
- Inconsistent → Maintain or downgrade rating
- Repeatedly disproven → Downgrade to D or E
- Feed updates into source-assessment guidance
3. Analyst Retrospectives
After significant investigations or assessments:
Questions:
- What analytical techniques were applied? Were they effective?
- Were key assumptions validated or invalidated?
- Were there intelligence gaps that could have been filled?
- Were alternative hypotheses adequately considered?
- What would you do differently?
Actions:
- Update SOPs with lessons learned
- Adjust technique selection guidance
- Fill identified collection gaps
- Update knowledge cells with validated/invalidated assumptions
4. Detection Effectiveness
For detection rules produced by the platform:
Metrics:
- True positive rate (did the rule catch real threats?)
- False positive rate (did it generate noise?)
- Coverage (did it miss known instances?)
Actions:
- High FP → Refine rule, add exclusions
- Missed detections → Expand rule, add variants
- No hits → Verify rule logic, check data source availability
Feedback Integration
All feedback feeds back into Phase 1 (Planning & Direction):
Consumer says "not useful" → Review and adjust PIRs
Source proves unreliable → Update Admiralty ratings, adjust collection
Analyst retrospective finds gap → Create new collection tasking
Detection rule has high FP → Refine and redeploy
Tracking
The orchestrator maintains feedback state:
- Consumer feedback logged in stakeholder register
- Source quality tracked in knowledge cell Sources & References
- Retrospective outcomes logged in investigation workspace
- Detection effectiveness tracked in detection rule metadata