agentsclimarketplace

Advanced prompting techniques

Skill AlexYedi/alex-agents-skills/skills/advanced-prompting-techniques

Apply advanced prompting techniques beyond basic instructions: Chain-of-Thought, Self-Consistency, Tree-of-Thoughts, Constrained Sampling, and modular prompt engineering. Use when basic prompts hit a quality ceiling, when reasoning tasks need step-by-step thinking, when output format must be guaranteed, or when designing prompts as composable modules. Triggers: "improve my prompt reasoning", "Chain-of-Thought for X", "force JSON output", "Tree-of-Thoughts", "self-consistency check", "modular prompt design". Produces concrete prompting strategy with technique selection and prompt template.From its SKILL.md

Install
npx -y skills add AlexYedi/alex-agents-skills --skill advanced-prompting-techniques

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

  • no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
  • 1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

10.6 KB, as published. Nobody here has run it

Advanced Prompting Techniques

You apply prompting beyond basic instructions to improve reasoning, output reliability, and quality. Most teams stop at "give the model an instruction" and miss the techniques that 5-10x performance on hard tasks.


When to Use This Skill

  • Basic prompts hit a quality ceiling on reasoning-heavy tasks
  • Output format must be guaranteed (JSON, structured data)
  • Need to reduce randomness in repeated runs
  • Designing prompts as composable modules across an application
  • Selecting between Chain-of-Thought variants

The Modular Prompt Framework

Treat every prompt as composed of these components. Tune each independently.

ComponentWhat it doesExample
PersonaRole the model should adopt"You are a senior incident responder."
InstructionThe specific task"Triage this alert and recommend next steps."
ContextBackground needed for the taskRecent on-call notes, system topology
FormatStructure of the output"Return JSON with fields: severity, root_cause_hypothesis, next_steps[]"
AudienceWho reads the output"Output for an engineer who was just paged at 3am"
ToneVoice characteristics"Direct, no preamble, action-oriented"
DataInputs the task operates onThe alert text, logs, metrics
Examples (few-shot)Concrete demonstrations2-3 input/output pairs

Why Modular?

When a prompt fails, you can fix the broken component without rewriting the whole thing. You can also reuse strong components across prompts.


Chain-of-Thought (CoT)

Force the model to reason explicitly before answering. Massively improves performance on multi-step reasoning.

Variants

VariantWhen to useCost
Zero-Shot CoT"Let's think step by step" appended to promptFree
Few-Shot CoTProvide reasoning examples in the promptHigher token cost, better quality
Auto-CoTGenerate reasoning examples programmaticallyOne-time setup

Default: Zero-shot CoT for any reasoning task. Add few-shot examples if accuracy is still insufficient.


Self-Consistency

Sample the same prompt N times with temperature > 0, then majority-vote the final answer.

WhenWhy
Tasks with discrete answers (math, classification, code generation)Reduces variance from sampling randomness
Stakes are high enough to justify N× cost3-5 samples is typical sweet spot

Key tradeoff: N× cost for variance reduction. Use sparingly on critical decisions.


Tree-of-Thoughts (ToT)

Explore multiple reasoning paths, evaluate them, and prune low-probability branches.

VariantMechanismUse when
Full ToTIterative model calls; tree expansion + evaluation + pruningComplex multi-step problems where wrong moves are costly
Zero-Shot ToTSingle prompt simulating multi-expert conversationLighter alternative; cheaper but less rigorous

Caveat: ToT is expensive. Reserve for hard problems where CoT + Self-Consistency are insufficient.


Constrained Sampling

Force output adherence to a specific format by restricting valid tokens at generation time.

MethodHow it works
Logit filteringMask invalid tokens during sampling
Grammar constraintsUse a grammar (BNF, regex) to constrain output
JSON mode (OpenAI/Anthropic)API-level JSON enforcement
Outlines / GuidanceLibraries that wrap constrained sampling

When to use: Structured outputs that must validate (JSON, regex, enums). Don't rely on prompt instructions alone for structured outputs at scale — constrained sampling is the only reliable way.


Principles

  • Be explicit and unambiguous. "Score 1-5" is ambiguous. "Score 1-5 where 1=poor and 5=excellent, return only the integer" is testable.
  • Specify output format with markers. Use clear delimiters (XML tags, code fences, specific keys) so downstream parsing is reliable.
  • Provide context, not just instructions. Reference texts, examples, and constraints reduce hallucinations more than clever instructions.
  • Decompose complex tasks. Break large prompts into subtasks. Each subtask is easier to optimize independently.
  • Encourage thinking when answers are non-obvious. CoT is cheap; use it for any task where reasoning matters.
  • Treat prompts as code. Version them. Eval them. Refactor them. Don't ship "the prompt that worked yesterday."

Anti-Patterns to Avoid

Vague Instructions

Looks like: "Summarize this in a helpful way."

Why it fails: "Helpful" is undefined; output varies wildly across runs.

The fix: Specify length, format, audience, and tone explicitly. Provide an example output.

Output Format via Instruction Only

Looks like: "Return your answer as JSON" without constrained sampling.

Why it fails: Models occasionally violate format. Downstream parsers crash.

The fix: Use constrained sampling (JSON mode, grammar) for structured outputs. Instructions alone are insufficient at scale.

One Giant Prompt

Looks like: A 2000-token prompt trying to do task analysis, planning, execution, and output formatting in one call.

Why it fails: Hard to debug; each subtask gets less attention; output quality degrades.

The fix: Decompose. Separate prompts for analysis, planning, execution, formatting. Compose them programmatically.

Skipping CoT on Reasoning Tasks

Looks like: Asking a math or logic question with no "think step by step."

Why it fails: Model jumps to answer; reasoning is wrong; quality is poor.

The fix: Always add CoT for reasoning. Cost is low; quality gain is large.


Decision Rules

ConditionAction
Multi-step reasoning taskChain-of-Thought (zero-shot first)
High-stakes discrete answer (math, classification)Self-Consistency on top of CoT
Complex problem, wrong moves costlyTree-of-Thoughts
Output must be JSON / structuredConstrained sampling, NOT just instructions
Output format keeps driftingAdd explicit format markers + example output
Prompt is failing in unclear waysDecompose into smaller subtask prompts
Repeated runs vary too muchLower temperature OR Self-Consistency with majority vote

Worked Example: Triaging Production Incidents

Bad prompt: "Given this alert, decide what to do."

Good prompt (modular + CoT + constrained):

Persona: You are a senior incident responder.
Instruction: Analyze the alert and produce a triage decision.

Context:
- Service topology: <embedded JSON>
- Recent on-call notes: <embedded text>
- Current SLO budget: <number>

Format: Return JSON with these keys, in this order:
  reasoning: string (your step-by-step analysis)
  severity: "P0" | "P1" | "P2" | "P3"
  root_cause_hypothesis: string
  next_steps: string[] (max 3, ranked)
  confidence: number (0-1)

Process:
1. First, restate what you observe in the alert.
2. Cross-reference with topology and recent notes.
3. Consider what could cause this pattern.
4. Decide severity based on user impact.
5. Output the JSON.

Alert: <alert text>

What this does: Decomposes the task (steps 1-5), forces reasoning before answer, specifies format precisely, gives the model the relevant context.


Gotchas

  • CoT can hide errors. Models sometimes generate plausible-looking reasoning that's wrong. Validate critical reasoning steps independently.
  • Self-Consistency cost compounds. N samples × T tokens per sample. Budget accordingly.
  • Constrained sampling can hurt quality. Forcing JSON when the natural answer is prose can produce stilted outputs. Test before relying.
  • Few-shot examples can poison. Models pattern-match strongly to examples. If examples have errors, model copies the errors.
  • Prompt versions matter. Two prompts that look "the same" can score 10% apart. Always A/B test prompt changes against a fixed eval set.

Source: Hands-On Large Language Models by Jay Alammar and Maarten Grootendorst, prompt engineering chapters.


Extension — the in-context learning ladder & prompt structure

Source: Roberto Infante, AI Agents and Applications (Manning), Ch. 2. Paraphrased.

Climb this ladder only as far as the task forces you — each rung costs tokens and complexity:

RungWhat it isReach for it whenWatch out for
Zero-shotInstructions onlyThe task is common and the model knows itFails on novel/intricate reasoning
One-shotOne worked exampleYou need to pin the output shapeOne example rarely teaches a concept
Two-shotTwo examples (a contrast)One didn't generalizeMay still under-specify a rule
Few-shotSeveral examples, delimiter-structured fieldsThe model must induce a rule from patternsEvery example burns tokens; keep examples out of the template
Chain-of-ThoughtExamples/instructions spelling out reasoning stepsMulti-step/deterministic reasoning where order mattersModern reasoning models often self-decompose — re-test if you still need it
Tree of ThoughtExplore multiple branches, evaluate, backtrackStrategic tasks needing lookahead (planning, puzzles)Much more expensive; overkill for simple tasks
Thread of ThoughtSystematically parse noisy/huge context, filter distractionsChaotic contexts that derail the modelA focus aid, not a reasoning-depth booster

8-part prompt structure (checklist, not a form): Persona · Context · Instruction · Input · Steps · Tone · Output format · Examples. You rarely need all eight and needn't label them (models infer purpose). Two high-leverage habits: force the output format (JSON/CSV/table with named fields when output feeds code) and condition role + tone + length caps (models pad without them).

Durability note: a prompting weakness you carefully patch today often disappears when the base model improves — periodically re-test whether your scaffolding still earns its keep.

Keep looking

Skills are one crate of 608,666. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.