Corpus test

Skill erraggy/oastools/.claude/skills/corpus-test

Run corpus regression tests against real-world OpenAPI specs. Usage: /corpus-test [short]From its SKILL.md

Install

npx -y skills add erraggy/oastools --skill corpus-test

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

5 stars5 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

1.8 KB, as published. Nobody here has run it

corpus-test

Run integration tests against the corpus of real-world OpenAPI specifications.

Usage:

/corpus-test — run all corpus tests
/corpus-test short — exclude large specs (faster)

Step 1: Check Corpus Availability

Use the Glob tool to check for corpus files:

pattern: testdata/corpus/*.{json,yaml,yml}

If no corpus files found, download them:

make corpus-download

Step 2: Run Corpus Tests

Based on the mode argument:

Full mode (default):

make test-corpus

Short mode (argument = "short"):

make test-corpus-short

Capture and analyze the output.

Step 3: Analyze Results

Note: Corpus specs are real-world documents (Plaid, DigitalOcean, GitHub, etc.) that often contain validation errors intentionally. Validation failures are expected — the key metric is whether the fixer resolves them.

For each corpus spec, report:

✅ Parse/validate/fix succeeded
❌ Failures with error details (fixer failed to resolve validation errors)
⚠️ New warnings compared to expected behavior

Step 4: Report

## Corpus Test Results

| Spec | Parse | Validate | Fix | Notes |
|------|-------|----------|-----|-------|
| petstore.yaml | ✅ | ✅ | ✅ | |
| github-api.json | ✅ | ❌ | ⚠️ | [details] |
| ... | ... | ... | ... | ... |

### Summary
- **Passed**: N/M specs
- **Failed**: [list]
- **New issues**: [list]

Step 5: Offer Actions

If failures are found:

Investigate — deep dive into specific failures
Compare with main — check if failures are regressions
File issues — create GitHub issues for new failures