Ai governance

AI Governance Audit

Systematic trust and control assessment for AI/ML systems. Maps the AI system boundary, evaluates risk controls, and identifies gaps against governance frameworks (EU AI Act, NIST AI RMF, ISO 42001).

Usage

/ai-governance                   # full audit of the current project
/ai-governance <path>            # audit a specific ML service or module
/ai-governance --framework eu    # focus on EU AI Act risk classification
/ai-governance --framework nist  # focus on NIST AI RMF (Govern/Map/Measure/Manage)

Behavior

Step 1 — Map the AI system boundary

# Identify model artifacts, training scripts, inference code
find . -name "*.pkl" -o -name "*.pt" -o -name "*.onnx" -o -name "model.*" 2>/dev/null | head -10
grep -rn "model.predict\|model.generate\|pipeline\|inference" --include="*.py" -l . | head -10

# Find dataset references
grep -rn "pd.read_csv\|load_dataset\|dataloader\|train_data" --include="*.py" -l . | head -10

Document:

• What decisions does the model make? (classification, ranking, generation, recommendation)
• Who is affected? (internal tooling vs customer-facing vs high-stakes)
• What data does it consume?

Step 2 — Risk classification

Classify the system under the EU AI Act tiers:

Risk tier	Criteria	Examples
Unacceptable	Social scoring, real-time biometric surveillance	Flag for legal review immediately
High	Employment, credit, health, safety, law enforcement	Full compliance controls required
Limited	Chatbots, emotion recognition disclosure	Transparency obligations only
Minimal	Spam filters, game AI, recommenders	Voluntary codes of practice

Flag the applicable tier and list required controls.

Step 3 — Evaluate bias and fairness controls

# Look for fairness tooling
grep -rn "fairlearn\|aif360\|what-if-tool\|demographic\|protected\|sensitive_attr\|equalized_odds" \
  --include="*.py" -l . 2>/dev/null

# Check if sensitive attributes are in training data
grep -rn "gender\|race\|ethnicity\|age\|religion\|nationality" \
  --include="*.py" --include="*.csv" --include="*.json" -l . 2>/dev/null | head -10

Check for:

• Fairness metric definition (demographic parity, equalized odds, calibration)
• Bias testing in CI pipeline
• Disparate impact analysis on test set
• Sensitive attribute handling (removed, proxied, or legitimately included)

Step 4 — Evaluate explainability

grep -rn "shap\|lime\|captum\|eli5\|anchors\|explanation\|interpret" --include="*.py" -l . 2>/dev/null

Check:

• Is there a method to explain individual predictions?
• Are explanations surfaced to affected users or decision-makers?
• Is there a global feature importance report?
• For LLMs: are citations / sources provided for generated claims?

Step 5 — Evaluate model risk management

Check for model risk controls:

Control	What to look for	Gap indicator
Model card	model_card.md, MODEL_CARD.md, README with model description	No documentation of intended use, limitations, training data
Version tracking	Model version in filename, MLflow, DVC, W&B	Model artifact with no version
Performance monitoring	Drift detection, accuracy tracking in production	No monitoring post-deployment
Human-in-the-loop	Approval step for high-stakes decisions	Fully automated decisions in high-risk domain
Rollback plan	Previous model artifact retained	Only latest model available
Incident process	Runbook for model failure	No defined response to model misbehavior

Step 6 — Evaluate data governance

# Check for data lineage documentation
find . -name "data_catalog*" -o -name "lineage*" -o -name "data_dictionary*" 2>/dev/null
grep -rn "consent\|gdpr\|pii\|personally_identifiable\|anonymize\|pseudonymize" --include="*.py" -l . 2>/dev/null

Check:

• Is training data provenance documented?
• Is PII in training data identified and handled?
• Is there a data retention / deletion policy?
• Can a subject request their data be removed from training?

Step 7 — Report findings

AI GOVERNANCE AUDIT — <system/path>

Risk Classification
  EU AI Act tier: LIMITED (chatbot with emotion recognition disclosure required)
  NIST AI RMF: Map phase incomplete — no impact assessment documented

Bias & Fairness (2 gaps)
  No fairness metrics defined — demographic parity not measured on test set
  gender column present in training features — document why or remove

Explainability (1 gap)
  No explanation method found — users cannot understand why they received a recommendation

Model Risk Management (3 gaps)
  No model card — intended use, limitations, and training data undocumented
  Model artifact (model.pkl) has no version tag — rollback impossible
  No drift monitoring configured — silent degradation in production undetected

Data Governance (1 gap)
  PII (email addresses) in training CSV with no anonymization documentation

Required actions for LIMITED tier compliance:
  [ ] Add disclosure that users are interacting with an AI system
  [ ] Document model limitations in model card
  [ ] Implement opt-out mechanism for data use in training

Edge Cases

• No ML code detected: Check for third-party model API calls (OpenAI, Bedrock, etc.) and apply governance review to the integration layer.
• Internal tooling only: Note reduced risk tier; focus on model risk management and explainability rather than regulatory compliance.
• Foundation model wrapper: Most governance obligations shift to the foundation model provider; focus audit on the application layer (context, output validation, misuse prevention).

Token Optimization

Expected range: 800–2,500 tokens (full audit); 300–600 tokens (single-framework focus)

Patterns used: Grep-before-Read, early exit (if no ML artifacts found, report and suggest scope), progressive disclosure (classification → gaps → required actions)

Caching: Caches system boundary map in .claude/cache/ai-governance/boundary.json (invalidated when model files or training scripts change).

Early exit: If no model artifacts, training scripts, or LLM client code found, report "No AI/ML system detected in this path" and exit.