Architecture analysis

Architecture Analysis

Purpose

Judge and design system structure using observable evidence: where data changes shape (seams), where trust changes (boundaries), and how failures propagate (error taxonomy). The output is a set of explicit contracts and classifications — not diagrams for their own sake — sized to the system's written threat model, never larger.

When to Use / When NOT to Use

Use when:

• Designing or reviewing how phases, layers, or modules connect.
• Deciding whether an error should abort the run or degrade gracefully.
• Placing validation: which side of which boundary owns it.
• Evaluating whether a proposed module split or abstraction is justified.
• Assessing an unfamiliar system's structure after onboarding.

Do NOT use when:

• You have no map of the repo yet → load codebase-onboarding first; analysis without a map produces confident nonsense.
• You need historical why behind a structure → load failure-archaeology.
• The question is "is this too much structure?" as a general habit → load engineering-minimalism.
• You are auditing for vulnerabilities specifically → load security-review-playbook.
• The boundary in question is an LLM call (prompt/response contracts, malformed output) → load llm-integration-reliability for the LLM-specific tactics; this skill supplies the boundary framing.

Core Methodology

1. Identify the phases. Every system is a pipeline at some altitude: input → transform(s) → output. Name each phase in one sentence stating what it consumes and what it produces. If you cannot, the phase boundary is unclear — that is your first finding.
2. Find or define the contract at each seam. A contract is a typed, named shape (frozen dataclass, TypeScript interface, JSON schema) that fully describes what crosses the boundary. Test: could you replace either side's implementation without the other noticing? If the answer is no, the contract is leaky.
3. Classify every failure as fatal or recoverable. Fatal = the run's output would be wrong or meaningless without this; abort loudly. Recoverable = a degraded-but-honest output is still possible; skip, log, continue. Encode the classification in types (distinct exception classes), not in comments. This skill owns the general taxonomy; llm-integration-reliability step 10 specializes it for LLM backends.
4. Mark trust boundaries. A trust boundary is any point where data arrives from something you do not control: network requests, LLM output, user input, files, database blobs. Rule: validate on the receiving side, immediately at the boundary, before any other processing — cheapest checks first (size, signature, type), expensive parsing last.
5. Trace one datum end to end. Pick a representative input and follow it through every phase to output, writing one line per hop: phase (file) → shape. Any hop where you must read implementation internals to know the shape is a missing contract.
6. Apply the module-split test (see decision tree). Splits must be earned by a present need, not a predicted one.
7. Verify proportionality. Compare the structure to the written threat model and actual requirements. If no written threat model exists — the common case in an unfamiliar repository — that absence is itself your first finding: draft one before judging proportionality (see security-review-playbook Step 1). Structure that defends against threats outside the model, or abstracts for futures nobody scheduled, is a finding too — over-engineering is an architecture defect, not a virtue.

Decision tree: is this module split justified?

Do the two candidate pieces change for different reasons
(different external deps, different failure modes, different review needs)?
├─ no  → do not split; keep one module (a split here is pure overhead)
└─ yes → can you write the contract between them as a small typed shape today?
    ├─ no  → do not split yet; entanglement means you don't understand the
    │        seam — analyze more, split later
    └─ yes → does the split let you test or swap one side independently?
        ├─ yes → split, and make the contract a named type
        └─ no  → split only if a file-size/readability limit forces it

Decision tree: fatal or recoverable?

If this operation fails, is a correct-but-degraded output still possible?
├─ no  → FATAL: dedicated exception type, abort the run, actionable message
│        (tell the operator what to fix, e.g. "is the backend running?")
└─ yes → RECOVERABLE: skip the unit of work, record the skip, continue
         └─ but: would silent skipping mislead the consumer about coverage?
             ├─ yes → surface the skip in the output itself
             └─ no  → log and move on

Architecture evaluation checklist

• Every phase named with its input and output shape.
• Every seam has a written contract (typed shape or schema), or is flagged.
• Every exception path classified fatal or recoverable, encoded in types.
• Every trust boundary listed; validation confirmed on the receiving side, cheapest-first.
• One representative datum traced end to end with no "mystery hops".
• External-effect operations checked for idempotency (safe to re-run?).
• Unbounded work (loops, concurrency, payload sizes) checked for explicit caps.
• Structure justified against the written threat model — nothing defends imaginary threats.

Discovery Commands

All repo-agnostic; run from the repo root.

# Find contracts (typed shapes at seams)
grep -rn "dataclass" --include="*.py" .
grep -rn "frozen=True" --include="*.py" .
grep -rn "interface \|type .*=" --include="*.ts" . | head -30
grep -rn "struct \|trait " --include="*.rs" . | head -30

# Error taxonomy: custom exception types and how they're raised/caught
grep -rn "class .*Error\|class .*Exception" --include="*.py" .
grep -rn "raise \|except " --include="*.py" . | head -40
grep -rn "throw new\|catch" --include="*.ts" . | head -40

# Trust boundaries: external input entry points
grep -rn "json.loads\|request.\|input(" --include="*.py" .
grep -rn "JSON.parse\|req.body" --include="*.ts" .
grep -rn "os.environ\|getenv" --include="*.py" .        # config boundary

# Validation placement: is checking adjacent to the boundary?
grep -rn "isinstance\|compare_digest\|len(.*)\s*[<>]" --include="*.py" .

# Concurrency and resource caps
grep -rn "Semaphore\|timeout\|max_" --include="*.py" .

# Module dependency direction (who imports whom)
grep -rn "^from \|^import " --include="*.py" <pkg>/ | sort | uniq -c | sort -rn | head -20

# Size check: split pressure
find . -path ./node_modules -prune -o -name "*.py" -print | xargs wc -l | sort -n

Ecosystem variants: for Node use npm ls --depth=0 to see the dependency surface; for Rust cargo tree --depth 1; for Python pip list inside the venv.

Failure Modes & Anti-patterns

Symptom	Mistake	Correction
Swapping one phase breaks three others	Implementations coupled through internals, no contract	Define a typed shape at the seam; both sides depend only on it
One flaky unit kills the whole run	Every exception treated as fatal	Classify: degraded-but-honest output possible → skip and continue
Whole run "succeeds" but output is garbage	Every exception treated as recoverable	Missing precondition (backend down, bad config) → dedicated fatal type, abort loudly
Expensive parse of hostile input	Validation after processing, or on the wrong side	Validate at the receiving side, cheapest checks first (size/signature before parse)
Type errors deep inside business logic	Trusting deserialized data because "it parsed"	Parsing is not validation; type-guard fields immediately after parse
Re-running a job duplicates external effects	No idempotency design at effectful boundaries	Make effects idempotent (clean own prior output, or upsert)
A "measurement" that was never measured	One component both computes and interprets	Split compute from interpretation; the interpreter consumes only the computed contract
40 files for a 700-line problem	Splitting on predicted needs, speculative interfaces	Apply the split test: different change-reasons + contract writable today + independent testability
Runaway loop or resource exhaustion	Unbounded iteration/concurrency "because it usually terminates"	Explicit caps: iteration limits, semaphores, timeouts, payload maxima

Repository Examples

PRISM — error taxonomy, trust boundary, caps, idempotency (~/prism, as of 2026-07-04). Five modules, one concern each: prism/config.py, diff.py, github.py, reviewer.py, server.py. Fatal vs recoverable encoded in types: custom OllamaUnavailableError aborts the whole review (no backend → any "review" would be a lie), while a per-chunk timeout merely skips that chunk (degraded-but-honest coverage). Trust boundary ordering is textbook: HMAC-SHA256 webhook verification with hmac.compare_digest runs BEFORE JSON parsing, and a payload-size check runs before json.loads — cheapest checks first against unauthenticated input. After parsing LLM/webhook JSON, isinstance type-guards validate shapes before use. Resource caps are explicit: asyncio.Semaphore(5) concurrency cap, MAX_FILES_PER_PR=10, MAX_LINES_PER_CHUNK=120. Idempotency at the effectful boundary: prism deletes its own prior PR comments before posting, so re-running a review never duplicates output. Contracts are frozen dataclasses (Settings, Hunk, FileDiff, Chunk, InlineComment) crossing the seams between modules.

ORPHY — contracts-as-seams, compute/interpret split (~/orphy, as of 2026-07-04). Vite + React AI vocal coach with contract-first phase design: the DSP layer measures numbers (pitch, onsets, cents error); the LLM layer only interprets those numbers as language. The split exists specifically to prevent hallucinated measurements — the LLM can never claim a pitch it did not receive. Phases are connected by JSON contracts, so implementations swap with zero downstream change. This is the cleanest available example of step 2's substitution test.

AGENTIX — recoverable-by-design agent loop, bounded everything (~/agentix, as of 2026-07-04). ~700-line ReAct agent framework where tool exceptions are returned as observation strings so the loop survives any single tool failure — recoverable by construction, with the failure surfaced to the model rather than hidden. Bounds everywhere: hard cap of 10 loop iterations; 2 malformed-JSON strikes then surface the raw output; timeouts of shell 30s / python 15s / web 15s. LLM-output trust boundary handled by three-stage JSON extraction (whole string → ```json fence → first balanced braces respecting string escapes). Data trust boundary: np.load(..., allow_pickle=False) on the embedding store. All of it sized to a written threat model — README: "the security boundary is you and the model you point it at, not the code" — so the shell blocklist ["rm -rf /", "sudo", "mkfs"] is honestly documented as a guardrail, NOT a boundary. Proportionality, demonstrated.

NYTW — phase separation by cost and cacheability (~/NYTW, as of 2026-07-04). Next.js 16 quiz tool whose LLM-grading boundary is a strict contract: request JSON {"score":"correct|partial|incorrect","explanation"} and validate the enum on parse — the receiving side owns validation of LLM output. Question generation (one call, cacheable) is architecturally separated from grading (per attempt, fresh evidence: git diff + codebase evidence + user answer) — phases split along cost/frequency lines, a legitimate "different reasons to change." Mode variation (MCQ/matching/FRQ) lives in the prompt template (MODE_FORMATS), not in post-processing — variation pushed to the phase that owns it.

RAGIT — actionable fatal errors (~/ragit, as of 2026-07-04). Custom exceptions (IndexingError, RetrievalError, OllamaConnectionError, OllamaModelsError) whose messages tell the operator the fix, e.g. "Run ragit index {path} first" — the fatal branch of the taxonomy done well: abort, but with the remediation in the message. Embedding model fixed (nomic-embed-text) while the chat model is a ranked preference list — a deliberate asymmetry: the index's contract (embedding space) must be stable, while the interpreter can vary.

Validation Criteria

You applied this skill correctly if:

• You produced a phase list where every seam names its contract type, or flags its absence.
• Every custom exception in the system is placed in the fatal/recoverable taxonomy, and the placement matches runtime behavior (abort vs. skip).
• Every trust boundary has validation on the receiving side, ordered cheapest-first, and you can point to the line.
• Your end-to-end trace of one datum has zero mystery hops.
• Any split/abstraction you proposed passes the module-split decision tree, and you rejected at least as many structures as you added.
• Nothing you proposed defends a threat absent from the written threat model.

Provenance & Maintenance

• Sources: prism, agentix, ragit, orphy, NYTW at ~/<repo>, investigated 2026-07-04. Prism structural facts cross-checked against its live history (git -C ~/prism log --oneline, 2026-07-04); other repo facts from the same-day investigation fact pack.
• Doctrine encoded: local-first, minimal dependencies, security proportional to a written threat model, no over-engineering. These example repos embody the doctrine deliberately; the methodology's step 7 (proportionality) exists because of them.
• Assumptions: the described patterns (semaphore value, iteration caps, timeout values, exception names) are as recorded on 2026-07-04; any claim beyond the fact pack would be marked "(hypothesis — requires verification)". Repo facts are examples of the method — the method itself assumes nothing about any specific repo.
• Re-verification commands: grep -rn "OllamaUnavailableError\|Semaphore" ~/prism/prism/; grep -rn "compare_digest" ~/prism/prism/; grep -rn "allow_pickle" ~/agentix/agentix/; grep -rn "class .*Error" ~/ragit/ragit/; grep -rn "MODE_FORMATS" ~/NYTW/.
• Likely to drift: numeric caps (Semaphore(5), 10 iterations, timeout seconds, MAX_* values) are tuning knobs and will change; exception class names may be renamed; NYTW's prompt-template structure may move as the app grows.
• Maintenance checklist: quarterly — re-run re-verification greps; re-stamp examples; if a repo's threat model README changes, re-check the proportionality claims quoted here; confirm cross-referenced skills (codebase-onboarding, failure-archaeology, engineering-minimalism, security-review-playbook, llm-integration-reliability) still exist under ~/.claude/skills/.