Quality gates

Quality Gates Skill

Use this skill to understand what each quality gate checks and how to ensure your workflow passes all gates.

Default Gates

1. validate - Structural Validation

What it checks:

• run.py exists and is valid Python
• PEP 723 metadata is present (# /// script)
• Required dependencies are declared (pydantic, rich)
• BaseWorkflow is subclassed
• run() method is implemented
• Imported tools exist in tools/ directory

Common failures:

✗ Missing PEP 723 metadata block
✗ run.py must subclass BaseWorkflow
✗ Workflow class must implement run() method
✗ Tool not found: yahoo_finance (imported from tools.yahoo_finance)

How to fix:

• Add PEP 723 block at top of run.py
• Ensure class inherits from BaseWorkflow[ParamsType]
• Implement def run(self) -> int: method
• Create missing tools in tools/ directory

2. dry - Dry Run

What it checks:

• Workflow executes with mock data (no real API calls)
• dry_run.py exists and provides mocks
• No crashes or unhandled exceptions
• Returns exit code 0

Common failures:

✗ dry_run.py not found
✗ KeyError: 'api_key' (accessing env var in dry run)
✗ ConnectionError: API not available (real network call in dry run)

How to fix:

• Create dry_run.py with mock functions
• Replace all external calls with mocks
• Use DryRunContext to log mock behavior
• Test locally: raw run workflow --dry

Optional Gates

3. pytest - Unit Tests

What it checks:

• All tests in test.py pass
• No test failures or errors

Enable in .raw/config.yaml:

builder:
  gates:
    optional:
      pytest:
        command: "pytest test.py -v"
        timeout_seconds: 60

Common failures:

✗ AssertionError: Expected 0 but got 1
✗ ModuleNotFoundError: No module named 'workflow_name'

How to fix:

• Write tests that match actual behavior
• Import workflow class correctly
• Run pytest locally to debug

4. ruff - Linting & Formatting

What it checks:

• Code follows Python style guide
• No unused imports
• Consistent formatting

Enable in .raw/config.yaml:

builder:
  gates:
    optional:
      ruff:
        command: "ruff check . && ruff format . --check"
        timeout_seconds: 30

Common failures:

✗ F401 'sys' imported but unused
✗ E501 Line too long (92 > 88 characters)

How to fix:

• Remove unused imports
• Run ruff format . to auto-format
• Run ruff check --fix . for auto-fixes

5. typecheck - Type Validation

What it checks:

• Type hints are correct
• No type mismatches

Enable in .raw/config.yaml:

builder:
  gates:
    optional:
      typecheck:
        command: "mypy run.py"
        timeout_seconds: 60

Gate Failure Strategy

When gates fail:

1. Read the error message carefully

   • Gate output is saved to .raw/builds/<build_id>/logs/<gate>.log
   • Contains full error details

2. Fix one gate at a time

   • Start with validate (structural issues)
   • Then dry (runtime issues)
   • Finally optional gates (quality issues)

3. Test locally before rebuilding

   raw validate workflow-id  # Check validation
   raw run workflow-id --dry # Check dry run
   pytest test.py            # Check tests
   

4. Common patterns:

   • validate fails → Fix run.py structure
   • dry fails → Add/fix mocks in dry_run.py
   • pytest fails → Fix tests or implementation
   • ruff fails → Run auto-formatters
   • typecheck fails → Add/fix type hints

Example: Fixing Validation Failure

✗ Validation failed
Errors:
  • Missing PEP 723 metadata block (# /// script)
  • Tool not found: yahoo_finance (imported from tools.yahoo_finance)

Steps to fix:

1. Add PEP 723 block to run.py:

   #!/usr/bin/env python3
   # /// script
   # requires-python = ">=3.10"
   # dependencies = ["pydantic>=2.0", "rich>=13.0", "yfinance>=0.2"]
   # ///
   

2. Create tools/yahoo_finance/:

   mkdir -p tools/yahoo_finance
   touch tools/yahoo_finance/__init__.py
   # Create tools/yahoo_finance/tool.py
   

3. Validate again:

   raw validate workflow-id