Agent tester

Agent Tester

Tests a built agent: dry-run, unit tests, integration, compatibility with other agents.

When to use

• After Agent Builder has finished
• "test agent X"
• "check agent compatibility"

Input

• Agent from $AGENTS_PATH/[name]/
• Spec from $AGENTS_PATH/specs/[name].spec.md

How to execute

Step 1: Static analysis

Check the agent code:

• File exists and runs without syntax errors
• All imports resolve
• Config file is valid
• Paths in config exist
• Credentials are accessible
• Dry-run mode is implemented

Step 2: Dry-run test

Run the agent with --dry-run:

python3 $AGENTS_PATH/[name]/[name]_agent.py --dry-run

Check:

• Agent starts without errors
• Logs are clear
• Shows what it WOULD do (without real side effects)
• Execution time is reasonable

Step 3: Unit tests

Run tests:

python3 -m pytest $AGENTS_PATH/[name]/test_[name].py -v

Minimum tests:

• Input parsing works
• Business logic is correct on test data
• Error handling works (bad input, missing files, API timeout)
• Output format is correct

Step 4: Integration test (one run on real data)

WARNING: only with human approval!

1. Back up data that the agent modifies:

cp [target.csv] [target.csv.backup]

2. Run the agent once on real data

3. Check output:

   • Data was written correctly
   • Format matches schema.yaml
   • Nothing broke
   • Git commit was created (if needed)

4. If something is wrong -- rollback:

cp [target.csv.backup] [target.csv]

Step 5: Compatibility test

Check that the new agent does not conflict with existing ones:

## Compatibility Matrix

| Agent | Shared Files | Potential Conflict | Status |
|-------|-------------|-------------------|--------|
| Email Pipeline | activities.csv | Write conflict | ? |
| [other agents] | ... | ... | ? |

Specific checks:

• File locks: can two agents write to the same CSV simultaneously
• Data consistency: does the agent overwrite another agent's data
• ID generation: do IDs conflict (person_id, activity_id, etc.)
• Schedule overlap: do agents run at the same time
• Git conflicts: does auto-commit create merge conflicts

Step 6: Report

Create a test report file:

$AGENTS_PATH/specs/[name].test-report.md

Report structure:

# Test Report: [Agent Name]

## Date: YYYY-MM-DD
## Tester: Process Analyst Agent

## Results

| Test | Status | Notes |
|------|--------|-------|
| Static analysis | PASS/FAIL | |
| Dry-run | PASS/FAIL | |
| Unit tests | PASS/FAIL | X/Y passed |
| Integration | PASS/FAIL | |
| Compatibility | PASS/FAIL | |

## Issues Found
1. [Issue description + severity]

## Recommendation
- [ ] READY for production
- [ ] NEEDS FIXES (list what)
- [ ] BLOCKED (list why)

Output

• Test report in $AGENTS_PATH/specs/[name].test-report.md
• PASS/FAIL verdict
• List of issues if any

Related skills

• process-analyst — creates the spec
• agent-builder — builds the agent
• change-review — validates CRM/PM changes