agentsclimarketplace

Model validation

Skill aizech/clinical-skills/.cortex/skills/model-validation

A collection of AI agent skills focused on medical imaging and healthcare workflows. Built for radiologists, healthcare IT professionals, and researchers who want AI coding agents to help with imaging workflows, clinical documentation, AI integration, and medical research. Works with Claude Code, Codex, Cursor, Windsurf, and many other agents.

Install
npx -y skills add aizech/clinical-skills --skill model-validation

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 3 stars3 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Designs and executes validation studies for radiology AI models to ensure clinical reliability and regulatory compliance. Use when user mentions "validate model performance", "external validation", "statistical analysis", "clinical validation", or needs model evaluation.

SKILL.md

2.4 KB, as published. Nobody here has run it

Model Validation Skill

Triggers

  • "validate model performance"
  • "external validation"
  • "statistical analysis"
  • "clinical validation"
  • "model comparison"
  • "regulatory submission"
  • "performance benchmarking"
  • "fairness audit"

Parameters

  • validation_type (required): Type of validation needed
    • internal - Retrospective internal dataset
    • external - Prospective/out-of-distribution testing
    • prospective - Clinical deployment study
    • regulatory - FDA/EMA submission prep
    • fairness - Subgroup disparity analysis
    • comparison - Head-to-head model comparison
  • model_task (required): Model's intended use
    • detection - Sensitivity, specificity, PPV, NPV
    • segmentation - Dice, IoU, Hausdorff distance
    • classification - Accuracy, AUC, F1 score
    • regression - MAE, RMSE, correlation
  • modality (optional): Imaging modality
  • regulatory_path (optional): Target clearance pathway

Validation Framework

Performance Metrics

TaskPrimary MetricsSecondary
DetectionSensitivity, Specificity, AUCPPV, NPV, FROC
SegmentationDice, IoUHausdorff, ASD
ClassificationAccuracy, AUC, F1Sensitivity, Specificity
RegressionMAE, RMSECorrelation, Bland-Altman

Statistical Methods

  • Confidence intervals (bootstrap, binominal)
  • Significance testing (McNemar, DeLong for AUC)
  • Power analysis for sample sizing
  • Multiple comparison correction
  • Subgroup interaction testing

Regulatory Standards

  • FDA 510(k) predicate comparison
  • FDA De Novo requirements
  • EU MDR clinical evaluation
  • IMDRF clinical evidence framework
  • ACR-SIIM AI performance standards

Output Format

Returns structured JSON with:

  • Validation protocol and methodology
  • Required sample size with power analysis
  • Statistical test selection and rationale
  • Results template with standard metrics
  • Interpretation guidelines
  • Regulatory compliance checklist

Usage Examples

validation_type: external
model_task: detection
modality: CT

validation_type: regulatory
model_task: classification
regulatory_path: 510k

Gives 0 of the 12 instructions most quality gates skills give

Counted across 1,195 of the 2,094 authors here whose files we hold, read 2026-08-06

  • read the output and check the exit codein 55 of 1195, across 14 files
  • verify requirements using a line-by-line checklistin 53 of 1195, across 12 files
  • identify the verification command proving the claimin 53 of 1195, across 12 files
  • run the full verification commandin 51 of 1195, across 11 files
  • verify output confirms the claimin 49 of 1195, across 10 files
  • check version control diff after agent delegationin 45 of 1195, across 5 files
  • state claim with evidencein 43 of 1195, across 3 files
  • run the test suitein 32 of 1195, across 24 files
  • keep state in memory by defaultin 27 of 1195, across 6 files
  • make prototype runnable with one commandin 26 of 1195, across 5 files
  • detect the package manager from lockfilesin 24 of 1195, across 5 files
  • produce a verification reportin 23 of 1195, across 12 files

Said here and by no other author read

  • select performance metrics for the model task
  • compute confidence intervals
  • perform significance testing
  • conduct power analysis for sample sizing
  • apply multiple comparison correction
  • run subgroup interaction testing

Grouped from the skills themselves: near-identical wordings counted once, and counted by distinct author, so one author publishing three of these counts once.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.