Prompt triage

Skill ayaqen/Agents-Scripts/skills/prompt-triage

Debug LLM prompt or agent misbehavior systematically: isolate inputs, form hypotheses, test one variable at a time.From its SKILL.md

Install

npx -y skills add ayaqen/Agents-Scripts --skill prompt-triage

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

2 stars2 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

2.6 KB, as published. Nobody here has run it

prompt-triage

Debugging an LLM-powered feature is still debugging: reproduce, hypothesize, isolate. The difference is that the "code" includes prompts, context assembly, and sampling — and the failure is often in what the model saw, not what you meant.

When to use

An LLM feature returns wrong, malformed, or inconsistent output.
An agent ignores instructions, loops, or uses tools incorrectly.
Output quality regressed after a prompt, model, or context-pipeline change.

Workflow

Capture the exact request. Log or reconstruct the final rendered prompt: system prompt, few-shots, retrieved context, tool schemas, and the sampled parameters (model id, temperature, max tokens). The bug is usually in this artifact, not in your template file.
Read what the model actually saw. Common findings: truncated context, retrieval garbage, contradictory instructions from two sources, malformed few-shot examples, tool schemas that don't match the instructions.
Classify the failure:
- Never worked → specification problem: the prompt underspecifies the behavior or asks for something the model can't verify.
- Works sometimes → distribution problem: rerun the same input N times; if output varies past tolerance, constrain (schema/structured output, lower temperature, tighter instructions) or verify downstream.
- Regressed → diff the rendered prompts (not the templates) between good and bad versions; bisect the change.
Change one variable per experiment — one instruction, one example, one parameter — and rerun the same fixed inputs. Prompt changes interact; batched edits make results unattributable.
Prefer structural fixes over incantations: schema-constrained output beats "please return valid JSON"; a validation-and-retry step beats a longer prompt; splitting a task the model conflates beats emphatic capitalization.
Pin the fix with an eval. Add the failing case (and near-misses) to a small regression set (see eval-design) so the next prompt edit can't silently reintroduce it.

Pitfalls

Editing the template while the bug lives in context assembly, retrieval, or truncation.
Judging a stochastic system from n=1 runs — both the bug report and your "fix" need multiple samples.
Stacking instruction patches until the prompt contradicts itself. Periodically rewrite clean instead of appending.
Blaming the model before checking the rendered prompt. It's almost always the rendered prompt.