Characterize

Characterization Test Generator

Generate characterization tests that document existing code behavior before modification. These tests are safety nets — they lock current behavior so refactoring or AI-assisted changes can be verified.

For background theory, read references/theory.md. For language-specific patterns, read references/language-patterns.md.

Workflow

Step 1: Identify targets

Read the file or directory specified by the user. Identify all public functions/methods that:

• Have business logic (not simple getters/setters)
• Accept parameters and return values
• Perform transformations, calculations, or orchestration

List targets and confirm with user before proceeding.

Step 2: Analyze each target

For each function:

1. Read the implementation thoroughly
2. Identify all code paths (if/else, switch, error handling)
3. Determine input types and realistic sample values
4. Identify output shape and unstable fields (timestamps, IDs, random values)
5. Note external dependencies that need mocking

Step 3: Generate characterization tests

Follow the Feathers Method for each function:

1. Create test with descriptive name: TestCharacterize_<FunctionName>_<Scenario>
2. Provide realistic input (prefer production-like data over trivial examples)
3. Call the function
4. Scrub unstable data — replace timestamps, UUIDs, random values with placeholders
5. Use golden file / snapshot pattern for the target language
6. Cover all code paths identified in Step 2

Critical rules:

• Do NOT fix bugs discovered during characterization — document them as comments
• Do NOT modify the production code being characterized
• Do NOT use trivial inputs — use realistic, production-like data
• DO scrub all unstable fields before comparison
• DO name tests to describe discovered behavior, not expected behavior
• DO aim for full branch coverage of the target function

Step 4: Verify with mutations

After generating tests, verify they catch changes:

1. Suggest 2-3 mutations the user should try (comment out key lines)
2. Explain what test should fail for each mutation
3. If a mutation doesn't cause a test failure, add more test cases

Step 5: Output

Generate files following project conventions:

• Place tests in the project's existing test directory structure
• Use the project's existing test framework and assertion library
• Create testdata/golden/ directory for golden files if applicable
• Include a helper function for scrubbing unstable data

Output a summary:

Characterization Tests Generated
=================================
Target: <file path>
Functions characterized: X
Test cases generated: Y
Estimated coverage: Z%

Files created:
- tests/characterization/test_char_<name>.py
- tests/characterization/testdata/golden/<name>.json

Next steps:
1. Run tests to capture golden files: <command>
2. Verify coverage: <command>
3. Try mutation: comment out line N in <file>, expect test X to fail

Language Detection

Detect language from file extension and project structure:

• .go + go.mod → Go (use testing + golden files)
• .py + requirements.txt/pyproject.toml → Python (use pytest + approvaltests)
• .ts/.tsx + package.json → TypeScript (use jest snapshots)
• .js/.jsx + package.json → JavaScript (use jest snapshots)
• .kt + build.gradle → Kotlin (use JUnit + ApprovalTests)
• .java + pom.xml/build.gradle → Java (use JUnit + ApprovalTests)

Scrubbing Strategy

Always scrub these fields before capturing golden output:

• Timestamps: created_at, updated_at, timestamp, date, time
• Identifiers: id, uuid, request_id, job_id, session_id
• Performance: execution_time, duration, elapsed
• System: hostname, pid, port
• Secrets: any field containing key, token, secret, password

Replace with stable placeholders: [TIMESTAMP], [ID], [DURATION], etc.