agentsclimarketplace

Experiment idea planner

Skill StaryMoon/ai-researcher-skills/skills/experiment-idea-planner

Practical skill pack for AI research agents: paper briefing, experiment planning, rebuttal drafting, radar maintenance, and GitHub upkeep.

Install
npx -y skills add StaryMoon/ai-researcher-skills --skill experiment-idea-planner

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

1.5 KB, as published. Nobody here has run it

Experiment Idea Planner

Purpose

Turn a paper idea, codebase, or research hunch into a ranked experiment plan with concrete gates. The plan should help a researcher avoid wasting GPU time on weak branches.

When To Use

Use this skill when the user asks what to try next, how to improve a model, how to reproduce a paper, how to compare branches, or how to design a short research sprint.

Inputs

  • Current hypothesis or paper idea.
  • Local code path and current baseline.
  • Available compute budget.
  • Metric, dataset, or validation split.
  • Prior failed attempts if available.

Workflow

  1. Restate the target metric and baseline. If they are absent, ask for them or define a minimal proxy.
  2. Convert broad ideas into small experiment candidates.
  3. Rank candidates by expected value, cost, and interpretability.
  4. For each candidate, define implementation scope, config knobs, required files, and expected failure mode.
  5. Add a gate: minimal metric improvement, qualitative signal, runtime ceiling, or ablation sanity check.
  6. Add a stop rule for weak branches.
  7. Produce the first command or file-edit route when the codebase is available.

Output

  • Baseline
  • Ranked Experiment Queue
  • Quick Gates
  • Stop Rules
  • First Implementation Step
  • What Would Be Publishable

Quality Gates

  • Do not propose expensive full runs before a quick gate.
  • Keep comparisons on the same validation slice.
  • Avoid adding complexity without a measurable reason.
  • If the baseline is unverified, make baseline verification experiment 0.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.