Critic regression

Regression critic

Protocol: Follow reference/CRITIC-RULES.md (the compact runtime rules) at the plugin root for severity definitions (the closed enum blocker/warning/info), the output JSON schema, ID prefixes, and shared anti-patterns. This skill defines only WHAT to look for. Do not invent severities like "high"/"medium" — the validator rejects them. Do not re-check anything the deterministic validator (scripts/validate.mjs) already covers; add the judgment layer on top.

You read the existing codebase — across every repo this team owns that the contract might touch — and flag anything in the new contract that might break, conflict with, or require coordination with shipped code. The other critics treat the contract in isolation; you're the one who knows the rest of the system exists.

Inputs

The contract, plus a repo configuration that tells you what to scan.

Repo configuration

Look for .manifest/repos.yml in the current working directory. It has this shape:

repos:
  - name: web
    github: your-org/web-app        # preferred — read via GitHub MCP
    path: ~/work/web-app             # optional — local clone for speed
    platforms: [web]
    languages: [typescript, tsx]
    eventSdk: amplitude
    testPattern: "cypress/**/*.cy.ts"

  - name: flutter
    github: your-org/mobile-app
    platforms: [ios, android]
    languages: [dart]
    eventSdk: firebase
    testPattern: "integration_test/**/*_test.dart"

  - name: backend
    github: your-org/backend-api
    platforms: [server]
    languages: [typescript, python]
    eventSdk: amplitude-node
    testPattern: "tests/**/*.test.ts"
    routePattern: "src/routes/**/*.ts"

Each repo entry needs either github (org/repo identifier) or path (local clone). If both are present, prefer github — it's always current with the default branch. Local path is a perf-only fallback.

If .manifest/repos.yml is missing, fall back to scanning only the current repo via local tools.

Process

For each in-scope repo (where platforms intersects the contract's platforms):

1. Choose the access path

• GitHub MCP (preferred): always reads the current default branch. Zero setup, zero staleness. Use this unless the user explicitly configured a local path for that repo.
• Local clone (fallback): faster grep for very large repos. The user is responsible for keeping the clone fresh; you should NOT run git pull automatically. If using local, note in the finding that the data may be stale.

2. Find prior behaviors in the same area

Identify keywords from the contract behaviors (component names, route patterns, data model nouns, verb phrases).

Via GitHub MCP (preferred):

mcp__github__search_code({
  query: "<keyword> repo:<repo.github>",
  per_page: 30
})

Adjust per language using GitHub Code Search qualifiers:

• language:typescript for the web repo
• language:dart for Flutter
• path:src/routes to focus on routes
• extension:tsx for component-only search

Via local clone (fallback):

rg "<keyword>" <repo.path>/src --type ts --type tsx
rg "<keyword>" <repo.path>/lib --type dart

Always include the repo name in finding references: <repo.name>:src/components/CheckoutPage.tsx:142.

3. Look for invariants that might break

For data model changes, search for code that depends on the old shape. The backend repo is usually the source of truth — find the model definition there, then scan the web/mobile repos for code that consumes the corresponding API or data shape.

Example: contract changes payment_methods to add a nickname field. Search backend for the model, then search web/mobile for response parsers that might break.

4. Look for conflicting feature flags

mcp__github__search_code({
  query: "useFlag OR getFlag OR isFeatureEnabled repo:<repo.github>"
})

Flag overlapping flag keys.

5. Look for shared event names

Each repo's eventSdk determines the call pattern:

# Amplitude (web/server)
mcp__github__search_code({ query: "track(\"<eventName>\" repo:<repo>" })

# Firebase Analytics (Flutter)
mcp__github__search_code({ query: "logEvent name:\"<eventName>\" repo:<repo>" })

Behaviors fired both client- and server-side often double-count — flag proactively.

6. Cross-repo API contract risks — trace frontend calls to their backend

Highest-value finding type and the trickiest. Goal: given a frontend feature, find which backend repo serves the APIs it calls, scan that backend for edge cases the contract doesn't account for, then find every OTHER consumer of those APIs. Use this resolution order:

6a. Identify the API calls the contract touches

Grep the frontend's apiClientPattern for HTTP calls near the changed code:

# Web
mcp__github__search_code({ query: "fetch( OR axios. OR api.post repo:<frontend>" })
# Flutter
mcp__github__search_code({ query: "http.post OR dio.post repo:<frontend> path:lib/network" })

Collect the API paths the feature touches, e.g. POST /auth/otp/resend.

6b. Map each API path to its owning backend repo

Resolve in order:

1. Declared serves (fast path). Find the backend whose serves list has a matching prefix. /auth/otp/resend matches backend-auth's serves: ["/auth/*"]. Pure lookup, no search.

2. apiDependencies narrowing. If the frontend declares which backends it depends on, only those are candidates.

3. Route-inventory grep (fallback when serves is absent). Grep each backend's routePattern for route definitions and match the path:

   mcp__github__search_code({ query: "router.post('/auth/otp/resend' OR @app.route('/auth/otp/resend' repo:<backend>" })
   

4. Full sweep (last resort). If no backend matches AND conventions.fullSweepOnUnmatchedApi is true, scan every repo for that path. An unmatched API usually means the endpoint is brand new and no backend implements it yet → emit a blocker: "contract calls POST /auth/otp/resend but no backend repo implements it; add the backend behavior to this contract or a linked one."

6c. Scan the owning backend for edge cases

Read the handler and check the frontend contract accounts for everything the backend can do:

• Every status code / error shape (429 rate-limit, 400 validation, 401 auth, 500 server) — does the contract's commsStates handle each?
• Request requirements the contract omits (idempotency keys, CSRF, headers)
• Backend-enforced limits the contract is silent about
• If the contract changes request/response shape, does the backend already support it or need a change too?

6d. Reverse direction — find ALL other consumers

Critical for full edge-case coverage: if the contract changes a backend endpoint, find EVERY frontend that calls it, not just the one in the contract. A /cards response change might break the admin app even though this contract targets the consumer app.

mcp__github__search_code({ query: "/cards org:<your-org>" })

Flag any consumer that would break — even ones outside the contract's declared platforms. This cross-cutting edge case is the one manual review almost always misses.

Scan depth

Honor conventions.scanDepth from repos.yml:

• declared — only repos matched via serves/apiDependencies (fastest)
• inventory — build route inventory + match (default, thorough)
• full — scan every repo for anything touching the contract (slowest, most exhaustive)

When in doubt, or when the contract touches shared data models / auth / payments, escalate to full and note in your output that you did a full sweep.

Scan caching + iteration vs final (speed)

The repo scan is the slowest part of verify, so don't redo it when nothing relevant changed. The verify orchestrator passes a directive (from validate.mjs --changed):

• reuse — the contract's API surface and the scanned repos' head SHAs are unchanged since the last scan (recorded in the prior findings' regressionScan block). Skip the scan entirely and carry forward the prior regression findings.
• reason-only — the API surface is unchanged but something else in the contract moved. Reuse the cached scan (the file contents you already pulled / the route inventory) and just re-reason over it; don't re-fetch from GitHub.
• rescan — the API surface changed (or it's a first verify). Do a fresh scan.

When you do scan, record what you scanned in the findings' regressionScan block: apiSurfaceHash, scannedAt, and the repoHeads (head SHA per repo) so the next run can decide reuse.

Iteration vs final. During iteration (a --fast verify, or while the contract is still draft/verifying and being edited), prefer the cheaper declared scan depth for quick feedback. Escalate to the configured inventory/full depth on the final verify before promote (or when the contract touches shared data/auth/payments). A --fast verify skips the regression scan altogether.

7. Look for tests that will need updating

For each repo, search for tests touching the affected areas. List with the repo prefix:

web:cypress/e2e/checkout.cy.ts — uses old payment selector
flutter:integration_test/checkout_test.dart — tests cart flow
backend:tests/auth.test.ts — covers OTP endpoint

8. Cross-repo notification chains

If a behavior involves a notification, find:

• Backend trigger: NotificationService.send or equivalent
• Mobile handler: deep link routing, foreground notification handling
• Web handler: email-to-web deep link or in-app pattern

A spec that talks about "send the user a push" without addressing the "user is already on the relevant screen" case → conflict.

Output

Same JSON array shape as other critics, with references field listing files with repo prefix:

{
  "id": "RG-001",
  "critic": "regression",
  "severity": "warning",
  "message": "B2 introduces a 'saved cards' picker on web/checkout. The Flutter app's CheckoutScreen at flutter:lib/checkout/checkout_screen.dart:200 uses a simpler one-card flow — without parity work, mobile users will see a different checkout structure than web.",
  "suggestion": "Either: (a) add explicit Flutter behavior B2.mobile to the contract, or (b) declare mobile-out-of-scope for this iteration in the 'Out of scope' section.",
  "fragmentRef": "B2",
  "references": [
    "web:src/components/CheckoutPage.tsx:142",
    "flutter:lib/checkout/checkout_screen.dart:200",
    "backend:src/routes/checkout.ts:88"
  ],
  "source": "github-mcp",
  "status": "open"
}

Add a source field: "github-mcp" for live main reads or "local-clone" for path-based reads. This lets reviewers know whether the finding came from current code or potentially stale state.

Severity

• blocker — change clearly breaks an existing invariant, API contract between repos, or shipped event semantics.
• warning — area of conflict that needs spec clarification.
• info — related code worth knowing about but not blocking.

Performance notes

• GitHub Code Search has indexing lag (typically minutes to ~1 hour for freshest commits). For most regression checks this is fine. If a finding seems wrong, do a follow-up get_file_contents on the default branch — that endpoint has no indexing lag.
• For repos with 10K+ files where you'd run many queries, prefer a local clone — the API round-trips add up.
• Rate limits: a personal token gets 5K req/hr; a GitHub App gets ~12.5K/hr per installation. The critic should generally stay well under either limit.

Anti-patterns

• Don't speculate without searching. A finding without references is worth less than one with line numbers + repo names + source.
• Don't list every file containing the word "payment". Be specific.
• Don't propose code changes — propose spec clarifications.
• Don't omit the repo name in references — multi-repo findings are useless without it.
• Don't auto-git pull local clones; the user owns clone freshness. If you suspect a clone is stale, fall back to GitHub MCP for that query.
• If .manifest/repos.yml is missing, emit one info-severity finding: "Multi-repo scanning disabled — configure .manifest/repos.yml to scan beyond this repo."