Aiml validation framework

Skill a5c-ai/babysitter/library/specializations/domains/science/biomedical-engineering/skills/aiml-validation-framework

Babysitter enforces obedience on agentic workforces and enables them to manage extremely complex tasks and workflows through deterministic, hallucination-free self-orchestration

Install

npx -y skills add a5c-ai/babysitter --skill aiml-validation-framework

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

What its author says it does

Copied from the file, not written here

AI/ML medical device validation skill implementing FDA's GMLP principles

SKILL.md

2.9 KB, as published. Nobody here has run it

AI/ML Validation Framework Skill

Purpose

The AI/ML Validation Framework Skill supports validation of AI/ML-enabled medical devices per FDA Good Machine Learning Practice (GMLP) principles, addressing data quality, model performance, and predetermined change control.

Capabilities

Training data quality assessment
Ground truth labeling validation
Model performance metrics calculation (AUC, sensitivity, specificity)
Subgroup performance analysis
Bias and fairness evaluation
Predetermined change control plan (PCCP) templates
Clinical validation study design
Locked algorithm vs. adaptive documentation
Model explainability documentation
Performance monitoring planning
Real-world performance tracking

Usage Guidelines

When to Use

Validating AI/ML algorithms
Assessing training data quality
Planning clinical validation studies
Preparing FDA AI/ML submissions

Prerequisites

Algorithm development complete
Training/test datasets curated
Ground truth established
Intended use clearly defined

Best Practices

Document data management practices
Validate on diverse populations
Plan for performance monitoring
Consider predetermined change control

Process Integration

This skill integrates with the following processes:

AI/ML Medical Device Development
Software Verification and Validation
Clinical Evaluation Report Development
Post-Market Surveillance System Implementation

Dependencies

FDA AI/ML guidance
GMLP principles
Fairness toolkits (AIF360, Fairlearn)
Statistical analysis tools
Clinical study resources

Configuration

aiml-validation-framework:
  algorithm-types:
    - locked
    - adaptive
    - continuously-learning
  performance-metrics:
    - AUC
    - sensitivity
    - specificity
    - PPV
    - NPV
  subgroup-categories:
    - age
    - sex
    - race
    - disease-severity

Output Artifacts

Data management documentation
Algorithm description documents
Performance reports
Bias/fairness assessments
PCCP documents
Clinical validation protocols
Monitoring plans
FDA submission sections

Quality Criteria

Training data quality documented
Ground truth methodology validated
Performance meets clinical requirements
Subgroup performance acceptable
Bias assessments completed
PCCP appropriate for algorithm type