agentsclimarketplace

Ai audit

Skill nrdxp/predicate/skills/ai-audit

SOP for auditing AI-generated code. Trigger when: - Reviewing, refactoring, or cleaning up AI-generated code to prevent regressions or hallucinated APIs. - Prompt contains: /ai-audit, code audit, AI cleanup, common flaws.From its SKILL.md

Install
npx -y skills add nrdxp/predicate --skill ai-audit

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 10 stars10 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

7.0 KB, as published. Nobody here has run it

AI-Generated Code Audit Workflow

Verification Dual — adversarial path. This skill is the applied methodology for the adversarial half of the Verification Dual: when no deterministic evaluator can be built for a condition in LLM-generated code, it is closed by decorrelated, context-free agents running this audit protocol from independent attractor basins. Load it whenever the symbolic path is unavailable and an adversarial review of AI-generated code is required.

A 4-layer framework for auditing LLM-generated code. Traditional SAST is insufficient—AI code is syntactically flawless but often logically "hollow."

Principle of Zero Trust: Treat every AI-generated line as a high-risk external contribution.


Layer 1: Inefficiency Taxonomy (ODC Framework)

LLMs prioritize token sequence probability over algorithmic optimization, creating systematic inefficiencies.

Research shows 0.74 correlation between General Logic failures and Readability/Maintainability issues.

Orthogonal Defect Classification (ODC)

CategoryTechnical TriggerThe AI-ismRemediation
AlgorithmPrime divisibility iterates to n instead of √nInefficient iterative blocks; overly broad loop conditionsNarrow loop constraints; implement early stopping
AlgorithmO(n²) logic where O(n log n) is standardSub-optimal complexity; prioritizes "plausible" over optimalReplace with standard library or optimized algorithms
AssignmentUsed-before-assignment; shadowing built-ins (dict = {})Shadowing & bloat; misuse of variable bindingRename shadowed variables; ensure proper initialization
InterfaceAccessing _internal members outside class scopeStructural incoherence; poor class hierarchy integrationEnforce encapsulation; refactor to public APIs
CheckingPasses happy path but lacks try/except or null checksPartially wrong logic; failure to address edge cases (CWE-754)Mandate input validation and exception traceability
MaintainabilityUnnecessary else after return or breakDefensive bloat; complex control flow without valueFlatten conditional logic; reduce cyclomatic complexity

Architectural Impact

AI code is logically simpler and more repetitive than human code. Any deviation into complexity without clear performance gain is a diagnostic marker of model failure.


Layer 2: Slopsquatting & Dependency Verification

"Slopsquatting" is a supply chain attack where adversaries register hallucinated package names.

Risk Statistics:

  • GPT-4 hallucinates packages ~20% of the time
  • Gemini reaches 64.5%
  • The "huggingface-cli" phantom received 30,000 downloads despite being empty

Verification Checklist

  • Registry Cross-Reference: Verify every import against official registries (npm, PyPI)
  • Version Hallucination Audit: Flag non-existent versions (e.g., pandas==2.5.0, tensorflow==3.2.1)
  • Ecosystem Integrity: Reject cross-ecosystem borrowing (e.g., @utils/helper in Python)

Remediation

If a phantom dependency is detected: Nuke and rebuild. Do not attempt to fix the import. Re-generate using only organization-approved, security-vetted libraries.


Layer 3: Stylistic Signature ("Transformer Cadence")

AI code exhibits "Verbosity Drift" and a specific rhythm. These stylistic markers often correlate with logic gaps.

Diagnostic Markers

  • Marker A — Robotic Documentation: Comments explain what (# increment x by 1) rather than why
  • Marker B — Defensive Bloat: Redundant null checks or wrappers masking shallow logical depth
  • Marker C — Context Collapse: Complexity increases across iterations without resolving root bug

Context Collapse Remediation

Stop and Restart Threshold: If code complexity (NLOC/CCN) increases over 3 iterations without resolving the primary defect, discard the session. The model has entered a hallucination loop—start fresh.


Layer 4: Instruction Adherence & Reasoning Failure

The "Curse of Instructions": failure rates increase exponentially with multiple constraints. Chain-of-Thought (CoT) reasoning can divert focus from simple constraints.

Constraint Attention Audit

  1. Constraint Mapping: Identify all negative constraints from the original prompt

    • "Do not use the requests library"
    • "Must be Python 3.9 compatible"
    • Format requirements
  2. Attention Analysis: Audit the reasoning trace

    • Flag where model acknowledges constraint in thinking but violates in code

Remediation

High Adherence Failure: If significant violations occur under high-constraint prompts, bypass LLM's internal CoT. Use an external constraint-validation classifier.


Audit Report Template

Critical Risk Scorecard

LayerODC MappingFocus AreaHigh-Risk Indicator
Layer 1Algorithm/AssignmentLogic & PerformanceO(n²) complexity; benchmark failures
Layer 2InterfaceSecurity & DependenciesHallucinated packages (20%+ risk); phantom versions
Layer 3Function/ClassStylistic SignatureRobotic comments; defensive bloat hiding shallow logic
Layer 4CheckingInstruction AdherenceNegative constraint violations; CoT-driven neglect

Report Format

AUDIT REPORT: [Component Name]
Date: YYYY-MM-DD
Auditor: [Name]

LAYER 1: Logic & Performance
  Status: [PASS/FAIL]
  Findings: ...
  Remediation: ...

LAYER 2: Dependencies
  Status: [PASS/FAIL]
  Findings: ...
  Remediation: ...

LAYER 3: Stylistic Signature
  Status: [PASS/FAIL]
  Findings: ...
  Remediation: ...

LAYER 4: Instruction Adherence
  Status: [PASS/FAIL]
  Findings: ...
  Remediation: ...

OVERALL: [PASS/FAIL/CONDITIONAL]
Priority Remediations:
1. ...
2. ...

Final Directive

Present report to Software Architect prioritizing remediability. High-risk artifacts must be rejected immediately. Highlight where code lacks "lexical diversity" or "structural depth" and provide ODC-mapped remediation steps.

Keep looking

Skills are one crate of 610,677. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.