Prompt and context engineering

Prompt and Context Engineering

Purpose

The prompt is the program and the context is its input — and both are routinely written with less care than a shell script. This skill is the craft discipline for model input: structure prompts in a fixed order, show output shapes as literals, select few-shot examples that span the real variation, spend the context budget only on task-relevant evidence, and treat prompts as versioned code.

Metadata

• Prerequisites: llm-system-design (the model's role and output contract should be decided before the prompt is written).
• Related Skills: llm-integration-reliability (owns output contracts and parsing — this skill places the contract inside the prompt), evaluation-frameworks (measures whether a prompt change helped), agent-engineering (agent system prompts), change-control (prompts-are-code change discipline).
• Owns: prompt structure and instruction placement; few-shot example selection; context budget management (context engineering); prompt versioning; input chunking strategy.

When to Use / When NOT to Use

Use when:

• Writing a new prompt or system prompt, or restructuring an existing one.
• Choosing what evidence, retrieval results, or history to include in a call.
• A prompt has accreted instructions and behaves inconsistently.
• Selecting or refreshing few-shot examples.
• Splitting a long input across calls.

Do NOT use (load the sibling instead):

• The output comes back malformed / needs retries or type-guards → llm-integration-reliability.
• You're unsure the task should use a model at all → llm-system-design.
• You want to know if the new prompt is actually better → evaluation-frameworks (never ship a prompt change without it).
• The "prompt problem" is really an agent-loop problem (thrashing, wrong tool choice) → agent-engineering.

Definitions & Mental Model

• Prompt: everything you control in the model's input — system text, instructions, examples, evidence, and the question.
• Context budget: the finite token window; every included token competes with every other for the model's attention.
• Few-shot examples: worked input→output pairs included in the prompt to demonstrate the task.
• Ground-truth injection: supplying the facts to be judged inside the prompt (mechanics owned by llm-integration-reliability step 6).
• Prompt version: a diffable, dated revision of prompt text, treated exactly like a code revision.

Mental model: a prompt is a program written for an unreliable interpreter, and context is that program's working memory. Programs need structure (a fixed section order), explicit interfaces (a literal output shape), test cases (few-shot examples), and version control. Working memory is scarce: irrelevant context does not merely waste tokens — it actively degrades output, because the model attends to everything you include. The craft is therefore two disciplines in one: say exactly what you mean, in the right order (prompting) and include exactly what the task needs, nothing else (context engineering).

Core Methodology

1. Fix the section order. Assemble every prompt in this sequence:
   1. Role — one sentence: who the model is for this task.
   2. Task — one sentence: the job (from llm-system-design step 2).
   3. Constraints — the rules, as a short numbered list, most critical first.
   4. Output contract — the exact shape, shown as a LITERAL example (e.g. {"queue": "billing|shipping|returns|technical|account|other"}). Models copy shapes far more reliably than they follow prose descriptions. (Contract design: llm-integration-reliability step 1.)
   5. Few-shot examples — if used (step 3 below).
   6. Evidence — the injected facts/documents/diff the model must work from.
   7. The question/input — last, closest to the answer.
2. Place critical instructions twice in long prompts. Attention degrades over long spans: state the critical constraint early (in Constraints) and restate it in one line immediately before the input ("Remember: output ONLY the JSON object."). For short prompts (under a few hundred tokens), once is enough — duplication there is noise.
3. Select few-shot examples deliberately (decision tree):

Does the task need examples at all?
├─ Output contract is a simple enum/extraction and the model
│  complies without examples (test it) → ZERO examples. Cheapest.
└─ Model misformats or misjudges without demonstration →
    ├─ Choose 2–5 examples that SPAN the variation you expect:
    │   ├─ one typical case
    │   ├─ one boundary/hard case (the kind that fails today)
    │   └─ one "produce nothing / other / reject" case if the
    │       contract has a null path (silence must be demonstrated
    │       or the model will never choose it)
    ├─ Examples must be CORRECT — a wrong example teaches the
    │   error with authority. Verify each against the golden set.
    └─ Match example formatting EXACTLY to the output contract —
        the model copies the examples' shape over the prose spec.

4. Spend the context budget like money. For every candidate inclusion ask: does the task change if this is removed? If no, exclude it.
   • Include: the output contract, the evidence the task is about, the minimal necessary history.
   • Exclude: boilerplate courtesy, full documents when a section suffices, stale conversation turns, duplicate retrieval results, anything "just in case."
   • Order evidence by relevance, not arrival; put the most relevant material nearest the question.
   • Retrieval (RAG) is context engineering with a search engine: cap top-k, deduplicate, and validate that retrieved chunks are actually about the query before injecting (a deterministic relevance gate beats hoping).
5. Chunk long inputs with explicit bounds. When input exceeds the budget: split on semantic boundaries (files, sections, functions) not byte counts; set a hard per-chunk limit as a named constant; give every chunk the same full instruction block (chunks are processed independently — no chunk may depend on the model remembering another); design the merge step deterministically (code concatenates/dedupes results, the model does not "remember").
6. Version prompts as code. Prompts live in files (or named template constants), not inline string soup; changes go through diff review like any change (change-control); every change is measured before/after on the golden set (evaluation-frameworks — never ship a prompt change on vibes); per-mode variation lives in separate named templates, not runtime string surgery (template-vs-post-processing rule owned by llm-integration-reliability step 2).
7. Debug prompts by ablation, one variable at a time. When behavior is wrong: remove or change ONE section (an example, a constraint, an evidence block), re-run against a handful of golden cases, observe. Prompt debugging without a fixed comparison set is superstition.

Prompt review checklist

• Sections in the fixed order: role → task → constraints → contract → examples → evidence → input
• Output shape shown as a literal, not described in prose
• Critical constraint restated near the input (long prompts only)
• Few-shot examples: span the variation, include the null path, verified correct, shape-exact
• Every context inclusion justified by "task changes if removed"
• Retrieval capped, deduplicated, relevance-gated
• Chunking: semantic boundaries, named size constant, self-contained chunks, deterministic merge
• Prompt is in a versioned file; the change carries a before/after eval score
• No contradictory instructions (grep the prompt for "always"/"never" pairs and check them against each other)
• No begging ("please be very accurate") — replace with a checkable constraint or delete

Discovery & Audit Commands

Audit prompts and context handling in an existing codebase:

# Where do prompts live, and are they versioned files or inline strings?
grep -rln -iE "system_prompt|SYSTEM_PROMPT|prompt" --include="*.py" --include="*.ts" --include="*.js" --include="*.txt" --include="*.md" . | grep -v node_modules | head
git log --oneline -- "*prompt*" 2>/dev/null | head    # any change history at all?

# Is the output contract shown as a literal in the prompt?
grep -rn -E '\{"' --include="*.py" --include="*.ts" . | grep -iE "prompt|format" | head

# Few-shot examples present? (look for example/demonstration markers)
grep -rn -iE "example|few.shot|demonstration" --include="*.py" --include="*.ts" . | grep -v node_modules | head

# Chunking bounds (named constants, not magic numbers)
grep -rn -E "MAX_.*(CHUNK|TOKENS|LINES)|chunk_size" --include="*.py" --include="*.ts" . | grep -v node_modules | head

# Retrieval hygiene: top-k caps and dedup
grep -rn -iE "top_k|n_results|k=" --include="*.py" --include="*.ts" . | grep -v node_modules | head

Failure Modes & Anti-patterns

Symptom	Mistake	Correction
Model ignores the format spec	Contract described in prose, buried mid-prompt	Literal shape example, in the fixed section order (step 1)
Long prompt: early rules forgotten	Critical constraint stated once, far from the input	Restate the critical constraint just before the input (step 2)
Model never outputs "other"/empty	Null path absent from examples	Demonstrate the produce-nothing case explicitly (step 3)
New failure mode after adding examples	A few-shot example is wrong or off-shape	Verify every example; shape must match the contract exactly (step 3)
Quality degrades as features add context	"Just in case" inclusions	Removal test per inclusion; context is a budget (step 4)
RAG answers drift off-topic	Uncapped, undeduplicated retrieval injected raw	top-k cap + dedup + relevance gate before injection (step 4)
Chunk 7 references "the file above"	Chunks assumed to share memory	Self-contained chunks: full instructions each, deterministic merge (step 5)
"Improved" prompt, nobody can say from what	Inline string edited in place, no history	Prompts in versioned files; diff + eval per change (step 6)
Prompt says "always X" and "never X" in different paragraphs	Instruction accretion without review	Contradiction sweep in the checklist; delete or reconcile
"Please be extremely accurate and never hallucinate"	Begging instead of engineering	Ground-truth injection + contract + eval; delete the plea
Hours of prompt tweaking, no progress	Multi-variable edits against a single anecdote	Ablation: one section at a time against fixed golden cases (step 7)

Worked Example

Task: a prompt that classifies incoming emails as invoice, complaint, inquiry, or other.

Before (typical accreted prompt): a 40-line paragraph mixing role, warnings, format hints ("respond in JSON please"), and three contradictory tone instructions. Model returns prose 8% of the time, never uses other.

After, restructured by the methodology:

You are an email triage classifier.                                  # role
Classify the email into exactly one category.                        # task
Rules:                                                                # constraints
1. Output ONLY the JSON object — no prose, no markdown.
2. If no category clearly fits, use "other".
Output format: {"category": "invoice|complaint|inquiry|other"}        # literal contract
Examples:                                                             # few-shot spanning variation
Email: "Attached is bill #4411 for March services." → {"category": "invoice"}
Email: "Third time asking — my order arrived broken AGAIN." → {"category": "complaint"}
Email: "What are your support hours on weekends?" → {"category": "inquiry"}
Email: "FW: FW: FW: funny cat picture" → {"category": "other"}        # the null path, demonstrated
Remember: output ONLY the JSON object.                                # restated constraint
Email: {email_text}                                                   # input, last

The prompt moves into prompts/email_triage.txt (versioned); the change ships with a before/after score on a 100-email golden set (evaluation-frameworks): format compliance 92%→100%, other usage now non-zero and correct on 9/11 golden other cases.

Repository Examples

Repo facts below are point-in-time illustrations (as of 2026-07-04) — examples, never assumptions about your system.

• NYTW (~/NYTW) — per-mode templates done right: MODE_FORMATS prompt templates tell the model the exact output shape upfront for each quiz mode (MCQ carries an options array; FRQ carries a grader-only rubric; matching pairs are permuted) so the parser stays deterministic; grading prompts inject the actual git diff + codebase evidence + user answer (ground-truth injection) and demand literal enum JSON {"score":"correct|partial|incorrect","explanation"}; evidence is pre-fetched deterministically via a per-question perseus_query before the call — retrieval as a designed context step, not an afterthought.
• prism (~/prism) — context budgeting and the null path in production: diff input is chunked under a named constant (MAX_LINES_PER_CHUNK=120); the prompt scopes the reviewer to genuine bugs/security only and "post nothing" is the demonstrated-and-designed silence path.
• ragit (~/ragit) — retrieval-as-context pipeline: documents → chunks → embeddings (batched 64) → ChromaDB → top-k retrieval → prompt assembly, with the embedding model fixed because the vector space is a stable contract.

Validation Criteria

You applied this skill correctly when:

1. Any prompt you touched reads in the fixed section order, with the output contract as a literal.
2. Few-shot examples (if present) each map to a variation class, include the null path, and are verified correct.
3. For every piece of context you can state why removing it would change the task.
4. Chunked processing works when chunks are shuffled (proving self-containment).
5. The prompt lives in a versioned file and its last change has a before/after eval score attached.
6. A contradiction sweep of the prompt finds none.

Provenance & Maintenance

• Sources: ~/NYTW, ~/prism, ~/ragit — investigated 2026-07-04. Owner context: LLM output reliability is the owner's hardest problem; prompt/context craft is its input-side arm. Skill authored 2026-07-06; methodology is provider-agnostic.
• Assumptions: the attention-degradation guidance (restating constraints in long prompts) reflects observed 2026-era model behavior — Hypothesis for future model generations; re-check as models change. Repo constants are point-in-time.
• Re-verification commands:

  grep -rn "MODE_FORMATS\|perseus_query" ~/NYTW --include="*.js" | head
  grep -rn "MAX_LINES_PER_CHUNK" ~/prism/prism ~/prism/.env.example
  grep -rn "top_k\|n_results" ~/ragit/ragit | head
  

• Likely to drift: model attention characteristics (may obsolete the restatement rule); context window sizes (change the budget calculus); example repos' template structures.
• Maintenance checklist:
  • Re-run re-verification; re-stamp Repository Examples.
  • Re-test the instruction-placement guidance against current models annually; relabel or remove if obsolete.
  • Confirm cross-referenced skills still exist under their directory names.