Proofrail

ProofRail

Purpose

Protect an agent from unsafe skills, unexplained configuration drift, guess-and-patch debugging, and unsupported completion claims. Keep deterministic checks outside the model loop so routine monitoring does not consume model tokens.

Route the request

• For a new or changed skill, run the local audit before loading its instructions. Read references/security-policy.md when the verdict is REVIEW or BLOCK.
• For a failing or unreliable agent, follow references/debugging-workflow.md before editing anything.
• For scheduled health checks or updates, create a baseline and compare drift. Never auto-accept unexpected changes.
• Before saying fixed, healthy, safe, complete, or deployable, validate fresh evidence using references/evidence-contract.md.

Commands

Use Python 3.10 or newer. The script uses only the standard library.

python scripts/proofrail.py audit-skill /path/to/skill --json
python scripts/proofrail.py snapshot /path/to/agent --output proofrail-baseline.json
python scripts/proofrail.py check-drift /path/to/agent --baseline proofrail-baseline.json --json
python scripts/proofrail.py verify-evidence evidence.json --json

Exit codes are stable: 0 pass, 1 review or drift, 2 block or invalid evidence, 3 tool/runtime error.

Safety gates

1. Do not execute or load an untrusted skill before auditing it.
2. Treat static audit output as triage, not proof of safety.
3. Require user approval for permission expansion, credentials, external writes, destructive actions, or accepting unexpected drift.
4. Find a reproducible root cause before changing code. Test one hypothesis at a time.
5. After three failed fix attempts, stop patching and review assumptions, architecture, and observability.
6. A completion claim requires fresh evidence produced after the final change. Old logs and model narration are not evidence.
7. Keep secrets, private data, trading credentials, and live orders outside snapshots and reports.

Optional stronger scanner

If Cisco AI Defense Skill Scanner is installed, run it as an additional independent check. Do not downgrade its result because the local audit passed. ProofRail does not vendor or modify that project.

Output

Return a concise report containing verdict, concrete findings, root cause or remaining uncertainty, commands/evidence used, changed files, and next safe action. Never describe REVIEW as safe or BLOCK as usable.