agentsclimarketplace

Fairlearn bias audit starter

Skill ma-compbio-lab/SkillFoundry/skills/clinical-biomedical-data-science/fairlearn-bias-audit-starter

A framework for discovering, compiling, and validating reusable skills for scientific agents.

Install
npx -y skills add ma-compbio-lab/SkillFoundry --skill fairlearn-bias-audit-starter

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

SKILL.md

1.2 KB, as published. Nobody here has run it

Fairlearn Bias Audit Starter

Use this skill to run a deterministic group fairness audit over a tiny clinical toy cohort with fairlearn.

What it does

  • Loads a fixed tabular cohort with binary labels, binary predictions, and a sensitive group column.
  • Computes per-group accuracy, selection rate, true positive rate, and false positive rate with MetricFrame.
  • Summarizes demographic parity and equalized odds gaps and ratios.
  • Emits a compact JSON report and threshold-based audit flags for local debugging.

When to use it

  • You need a local starter for fairness or bias analysis in binary clinical risk models.
  • You want a small, reproducible example before auditing a real cohort.

Example

slurm/envs/statistics/bin/python skills/clinical-biomedical-data-science/fairlearn-bias-audit-starter/scripts/run_fairlearn_bias_audit.py \
  --input skills/clinical-biomedical-data-science/fairlearn-bias-audit-starter/examples/toy_fairness_cohort.tsv \
  --summary-out scratch/fairlearn/fairness_audit_summary.json

Verification

  • Skill-local tests: python3 -m unittest discover -s skills/clinical-biomedical-data-science/fairlearn-bias-audit-starter/tests -p 'test_*.py'

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.