Harness engineering

Harness Engineering

Harness engineering designs the operating environment around agents: the context they can read, the files they may edit, the feedback they receive, the state they preserve, the checks they must pass, and the actions that still require human judgment.

Use this skill to turn repeated agent confusion into reusable capability. Do not ask the agent to "try harder" when it fails. Ask what it was missing, then encode that missing piece as context, a principle, a linter, an eval, a script, a workflow, or an approval boundary.

Core Stance

• Treat the human as the harness architect: set goals, acceptance criteria, verification standards, and feedback loops.
• Treat the agent as an executor inside clear boundaries, not as an unbounded decision maker.
• When starting a new systematic project, design the harness before substantial implementation: context map, architecture boundaries, principles, checks, logs, and approval rules.
• Ask for missing context before giving strategic recommendations. Do not over-infer when the answer depends on organizational or product goals, user constraints, architecture, budget, safety, or workflow details.
• Prefer durable improvements over one-off fixes. If the agent repeats a mistake, turn the correction into documentation, a test, a linter rule, an eval, or a reusable skill.
• Keep loops small enough to inspect. A tiny reliable loop is better than a broad autonomous system with vague success criteria.
• Explain jargon briefly when the user appears non-technical.
• Respond in the user's language unless they request otherwise.

First Move

When this skill triggers:

1. State the user's objective in one sentence.
2. Identify the phase: scaffolding, feature work, review, eval design, cleanup, autonomous loop, or governance.
3. Look for existing context docs before inventing structure: AGENTS.md, ARCHITECTURE.md, docs/, .harness/, principles, plans, tests, CI, and recent logs.
4. If missing context could change the recommendation, ask concise questions first. If the risk is low, state assumptions and proceed.
5. Classify the surfaces: locked, editable, append-only, and human-controlled.
6. Choose the smallest feedback loop that proves whether the agent improved the system.

Human Judgment And Agent Execution

Use human attention for high-leverage creative judgment, not for mechanical participation. The human should own decisions whose correctness depends on their goals, taste, product intent, or definition of truth. The agent should own the complete execution burden after those decisions are clear.

Human-led decisions usually include:

• Core system architecture, domain boundaries, and source-of-truth choices.
• Core algorithm hypotheses, failure definitions, eval metrics, and acceptable trade-offs.
• Technical choices that materially change long-term cost, lock-in, privacy, safety, or reversibility.
• Product behavior, interaction priorities, feedback meaning, and experience quality.
• Changes to product direction, evidence authority, or durable governance.

Agent-led work usually includes:

• Gathering context and preparing a concise decision brief.
• Code, migrations, scripts, tests, eval plumbing, documentation, integration, debugging, browser verification, and version-control execution.
• Routine syntax, glue code, adapters, refactors inside approved boundaries, and mechanical asset production from an approved specification.
• End-to-end verification and a clear report of evidence, failures, and remaining uncertainty.

Trigger a human thinking gate only when an unresolved choice would materially change architecture, algorithmic behavior, evaluation truth, product experience, safety, cost, or long-term direction. Do not trigger one merely to make the human participate.

At a thinking gate:

1. Continue all safe discovery work until the actual decision is isolated.
2. Give the human a decision brief: problem, current evidence, constraints, two or three viable options, trade-offs, and a recommendation.
3. Request one bounded thinking deliverable, normally three to eight bullets. Ask for a decision, hypothesis, success definition, or experience intent, not code that the agent can write itself.
4. Check the response for contradictions, missing constraints, and technical feasibility. Explain concerns without silently replacing the human's judgment.
5. Record the approved decision in the project harness or design docs, then execute the implementation, tests, documentation, and integration end to end.
6. Return verification evidence and explain what the result teaches about the original decision.

Avoid fake participation. Do not ask the human to type routine code, syntax, CLI wiring, test boilerplate, or file edits unless they explicitly request coding practice. Do not offload research to the human when the agent can gather the context. Ask for only one thinking deliverable at a time, and do not pause unrelated safe execution while waiting for a decision.

Surface Model

Use four surface classes:

Surface	Examples	Rule
Locked	Eval metric, rubric, linter, merge policy, security rule	Agent may read and propose changes, but cannot use modified rules to approve itself
Editable	Skill draft, code under test, prompt, config, design doc	Agent may change these during the loop
Append-only	Decision log, experiment log, rejected ideas, run history	Agent may append, not rewrite history
Human-controlled	Production deploy, credentials, billing, destructive actions, merge authority	Requires explicit human approval

If a surface is ambiguous, make it explicit before the loop starts.

Context Management

Context management is usually the main bottleneck. Build a knowledge base that tells agents where to look and what not to rediscover.

Use this layout as a menu, not a requirement to create every file immediately:

AGENTS.md
ARCHITECTURE.md
docs/
  design-docs/
    index.md
    core-beliefs.md
  exec-plans/
    active/
    completed/
    tech-debt-tracker.md
  generated/
    db-schema.md
  product-specs/
    index.md
    new-user-onboarding.md
  references/
    design-system-reference-llms.txt
    framework-or-tool-llms.txt
  DESIGN.md
  FRONTEND.md
  PLANS.md
  PRODUCT_SENSE.md
  QUALITY_SCORE.md
  RELIABILITY.md
  SECURITY.md

Recommended meanings:

• AGENTS.md: agent onboarding map; where to start, what commands matter, what boundaries exist.
• ARCHITECTURE.md: dependency direction, core modules, allowed imports, and ownership boundaries.
• docs/design-docs/: durable product and architecture decisions.
• docs/exec-plans/: active work plans, completed plans, and debt tracker.
• docs/generated/: generated facts such as database schema, API surface, or command inventory.
• docs/product-specs/: user-facing behavior and acceptance criteria.
• docs/references/: external docs distilled for agent use.
• QUALITY_SCORE.md, RELIABILITY.md, SECURITY.md: review rubrics that agents can use but not silently weaken.

Keep docs navigable. Add an index before adding many files.

Architecture Constraints

Prefer architecture constraints over micro-management. A good constraint gives agents freedom inside a boundary.

Default layered dependency direction:

Types <- Config <- Repo <- Service <- Runtime <- UI

Layer meanings:

• Types: pure data definitions, schemas, interfaces, and domain vocabulary.
• Config: constants, environment-derived settings, feature flags, and static configuration.
• Repo: data access and persistence. It stores and retrieves data, but does not own business workflows.
• Service: business logic, policies, use cases, and workflows.
• Runtime: wires services, repos, providers, startup, shutdown, dependency injection, threads, and background jobs.
• UI: external interaction such as HTTP, WebSocket, CLI, or frontend handlers. It parses input, calls runtime/service entrypoints, and returns output. It should not contain business logic.

Rules:

• Lower layers must not import higher layers.
• External dependencies should enter through explicit providers or adapters.
• UI should stay thin; service should own behavior; repo should own storage mechanics.
• If the project uses a different architecture, document its dependency direction explicitly rather than forcing this exact chain.
• Convert repeated architecture drift into import lint rules or review checks.

Code Style For Reviewable Code

Write code so a reviewer who does not know the programming language syntax can still understand what each function is for.

For every new or changed function, add a function-level comment or docstring. At minimum, it must explain:

• Function: what the function does in plain language.
• Inputs: each parameter, expected shape/type, important units, and assumptions.
• Output: what the function returns or produces.
• Side effects: files, network calls, database writes, logs, external services, state changes, or user-visible effects.
• Errors or edge cases: what can fail and how the function responds.

Prefer comments that explain intent and review-relevant behavior. Avoid comments that only restate syntax.

Use the project's existing comment style. If there is no established style, use the language's normal function documentation format:

Python:

def build_report(user_id: str, include_private_notes: bool) -> Report:
    """Build the report shown to a user.

    Function: Collects the user's visible records and formats them into one Report object.
    Inputs:
      user_id: Stable user identifier used to find records.
      include_private_notes: Whether reviewer-only notes should be included.
    Output:
      A Report object ready for rendering or export.
    Side effects:
      Reads from the configured repository; does not write data.
    Errors:
      Raises UserNotFoundError when the user_id does not exist.
    """

TypeScript or JavaScript:

/**
 * Function: Builds the report shown to a user.
 * Inputs:
 * - userId: Stable user identifier used to find records.
 * - includePrivateNotes: Whether reviewer-only notes should be included.
 * Output:
 * - A Report object ready for rendering or export.
 * Side effects:
 * - Reads from the configured repository; does not write data.
 * Errors:
 * - Throws UserNotFoundError when the userId does not exist.
 */
function buildReport(userId: string, includePrivateNotes: boolean): Report {
  // ...
}

Review checks:

• Can a non-specialist understand the function's purpose without reading its implementation line by line?
• Are all inputs and outputs described clearly enough for audit?
• Are side effects and failure cases visible before reading the body?
• Do comments still match the code after edits?
• Did repeated comment failures become a lint rule, review checklist item, or golden principle?

Golden Principles

Golden principles are short, durable rules that encode human judgment.

Write each principle so it is:

• Specific enough for an agent to apply.
• Observable enough for a reviewer, linter, or test to check.
• Small enough that violating it points to a concrete fix.
• Paired with the reason behind it.

Template:

Principle:
Why it matters:
Rule:
How to check:
What to do when violated:

Examples:

Principle: Context before strategy
Why it matters: Recommendations become fantasy when the agent guesses hidden goals.
Rule: Ask for missing context before strategic recommendations when goals, constraints, or acceptance criteria are unclear.
How to check: Review strategic outputs for stated assumptions or clarifying questions.
What to do when violated: Add the missing context to AGENTS.md, a spec, or a principle.

Principle: UI is not business logic
Why it matters: Behavior becomes hard to test when it lives in request handlers.
Rule: UI handlers parse input and call runtime or service entrypoints; service owns workflow behavior.
How to check: Import linter plus review for workflow code in UI files.
What to do when violated: Move behavior into service and add a regression test.

Principle: New systems start with a harness
Why it matters: A project that begins as loose code quickly becomes hard for agents to extend, review, and clean up.
Rule: When starting a new systematic project, create or confirm the project harness before substantial implementation: context docs, architecture direction, golden principles, checks, logs, and approval boundaries.
How to check: Look for AGENTS.md or equivalent context map, documented architecture boundaries, a small validation command, and clear human approval rules.
What to do when violated: Pause feature work, add the missing harness pieces, then resume implementation inside those boundaries.

Self-Improvement Loop

Use this loop whenever an agent struggles repeatedly:

observe failure -> name missing capability -> encode durable fix
-> validate on a small case -> log the result -> update the knowledge base

Missing capability categories:

• Missing context: add docs, indexes, generated facts, or examples.
• Missing tool: add script, command, MCP tool, or deterministic helper.
• Missing abstraction: add module boundary, provider interface, template, or workflow.
• Missing feedback: add tests, lint, eval, rubric, review checklist, or screenshot check.
• Missing memory: add append-only logs, plans, state files, or handoff docs.
• Missing boundary: define locked/editable/human-controlled surfaces.
• Missing approval rule: state what requires human confirmation.

Vibe Coding Inside Boundaries

Use vibe coding as freedom inside a harness:

1. Scaffold the harness first: context docs, architecture boundaries, principles, CI, lint, and test commands.
2. Let the user describe outcomes in natural language instead of implementation details.
3. Let the agent choose implementation details within documented boundaries.
4. After each change, run checks and update docs only when the change alters durable behavior.
5. When a mistake repeats, add a principle, linter, eval, or cleanup task instead of relying on memory.

The goal is not to eliminate judgment. The goal is to move judgment into reusable structures.

Drift And Garbage Collection

Agents imitate existing patterns, including bad ones. Treat technical debt as something that compounds.

Use periodic cleanup agents or cleanup passes to:

• Find violations of golden principles.
• Detect dependency-direction drift.
• Reduce files that grew beyond local standards.
• Remove dead abstractions and stale generated docs.
• Prune stacked changes where a simpler version works equally well.
• Open small reviewable PRs instead of broad rewrites.

Cleanup agents should usually be read-only until a human asks for fixes or explicitly approves an edit loop.

PR And Review Loop

For agent-produced changes:

author agent -> local checks -> local review agent
-> fix feedback -> second independent review when risk is high
-> human review for merge/deploy/destructive approval

Review should prioritize:

• Behavioral regressions.
• Broken architecture boundaries.
• Missing tests or weak acceptance criteria.
• Security, privacy, cost, and irreversible actions.
• Drift away from golden principles.

Keep PRs small enough that a human can review them quickly.

Autonomous Loop Design

Use this checklist before starting a long-running or background agent:

1. Define the objective in one sentence.
2. Define success and stop conditions.
3. Classify locked, editable, append-only, and human-controlled surfaces.
4. Create a durable log before the loop starts.
5. Choose feedback: tests, lint, scalar metric, rubric, human review, or a combination.
6. Define keep, discard, crash, timeout, and review states.
7. Preserve rejected attempts so future agents do not repeat them.
8. Define approval boundaries for merge, deploy, billing, credentials, data deletion, or external messages.
9. Validate on one known-good and one known-bad case.

Minimal loop:

read locked context -> choose next attempt -> edit allowed surface
-> run feedback -> log result -> keep, discard, rollback, or route to review

Output Templates

Use these compact formats when helpful.

Harness diagnosis:

Objective:
Current phase:
Missing context or capability:
Locked surfaces:
Editable surfaces:
Append-only state:
Human approval boundary:
Smallest useful feedback loop:
Next action:

Harness change proposal:

Problem observed:
Durable fix:
Why this belongs in the harness:
Files or rules to change:
How to verify:
Risk:

High-leverage decision brief:

Decision needed:
Why it matters now:
Current evidence:
Constraints:
Options and trade-offs:
Recommendation:
One human thinking deliverable:
What the agent will execute after approval:

Human thinking deliverable:

Decision or hypothesis:
Reasoning:
Success definition:
Important boundary or non-goal:

Cleanup task:

Drift pattern:
Principle violated:
Candidate files:
Safe automated check:
Proposed small PR:
Human review needed:

Gotchas

• Mutable evaluator: if the agent can change the rule and then score itself, the loop is unsafe.
• Chat-only memory: plans disappear across compaction; write state to files.
• No rejected-attempt log: future agents rediscover failed ideas.
• Over-broad autonomy: vague objectives create vague work.
• Principle without check: a rule nobody can observe becomes decoration.
• Cleanup without restraint: broad refactors are hard to review; prefer small PRs.
• Approval ambiguity: "prepare a PR" is not "merge it"; "draft a deploy plan" is not "spend money."
• Over-context: too many docs without indexes makes context harder, not easier.