Experiment design

Skill dongzhigang13305312738-art/paper-skills/experiment-design

Design experiment plans with progressive stages — initial implementation, baseline tuning, creative research, and ablation studies. Plan baselines, datasets, hyperparameter sweeps, and evaluation metrics. Use when planning experiments for a research paper.From its SKILL.md

Install

npx -y skills add dongzhigang13305312738-art/paper-skills --skill experiment-design

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

2.9 KB, 637 tokens by cl100k_base, as published. Nobody here has run it

Experiment Design

Design structured, progressive experiment plans for research papers.

Input

$0 — Research idea, plan, or method description

References

4-stage progressive experiment prompts: ~/.claude/skills/experiment-design/references/stage-prompts.md

Scripts

Generate experiment design

python ~/.claude/skills/experiment-design/scripts/design_experiments.py --plan research_plan.json --output experiment_design.json
python ~/.claude/skills/experiment-design/scripts/design_experiments.py --method "contrastive learning" --task classification --format markdown

Generates baselines, ablation matrix, hyperparameter grid, metric selection. Stdlib-only.

4-Stage Progressive Framework (from AI-Scientist-v2)

Stage 1: Initial Implementation

Focus on getting a basic working implementation
Use a simple dataset
Aim for basic functional correctness
Completion: at least one working (non-buggy) implementation

Stage 2: Baseline Tuning

Tune hyperparameters (learning rate, epochs, batch size)
Do NOT change model architecture
Test on at least TWO datasets
Completion: stable training curves, improvement over Stage 1

Stage 3: Creative Research

Explore novel improvements and insights
Be creative and think outside the box
Test on at least THREE datasets
Completion: demonstrated novel improvement

Stage 4: Ablation Studies

Systematic component analysis
Each ablation tests a different aspect
Use same datasets as Stage 3
Completion: all planned ablations done

Output Format

{
  "stages": [
    {
      "name": "initial_implementation",
      "goals": ["Basic working baseline", "Simple dataset"],
      "max_iterations": 5,
      "completion_criteria": "Working implementation with non-zero accuracy"
    }
  ],
  "baselines": ["Method A", "Method B"],
  "datasets": ["Dataset1", "Dataset2", "Dataset3"],
  "metrics": ["accuracy", "F1", "inference_time"],
  "ablation_components": ["component_A", "component_B"],
  "hyperparameter_grid": {
    "lr": [1e-4, 1e-3, 1e-2],
    "batch_size": [32, 64, 128]
  },
  "num_seeds": 3
}

Rules

Always start simple (Stage 1) before complex experiments
Each stage builds on the best result from the previous stage
Multi-seed evaluation for statistical significance
Document every experiment run in notes.txt
Generate figures for training curves and comparisons

Related Skills

Upstream: research-planning, idea-generation
Downstream: experiment-code, data-analysis
See also: paper-assembly

What ships with it: 2 files

11.8 KB alongside SKILL.md, 1 of them executable

references/

stage-prompts.md3.4 KB

scripts/

design_experiments.pyruns8.3 KB