Experiment

Skill alirezarezvani/gaios/.claude/skills/experiment

Use to run an autoresearch-style experiment loop — improve a measurable artifact by trying changes, measuring against one objective metric, keeping if better and reverting if not, on an isolated git branch with a logged trail. Trigger on "/experiment", "run an experiment loop", "autoresearch X", "tune/optimize X against a metric", "hill-climb X". Follows references/sops/experiment-loop.md. Enforces the autonomy gate (objective metric + reversible sandbox + no PHI/external) before any unsupervised looping.From its SKILL.md

Install

npx -y skills add alirezarezvani/gaios --skill experiment

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.

SKILL.md

2.2 KB, 418 tokens by cl100k_base, as published. Nobody here has run it

Experiment loop (autoresearch)

Hill-climb a measurable artifact: try → measure → keep-or-revert → log, on a branch, with a trail. Full procedure: references/sops/experiment-loop.md. Existing examples + use-case cards: experiments/.

First, the autonomy gate

Before looping unsupervised, confirm all three:

Objective, cheap, automatable metric (code computes it in seconds–minutes).
Reversible sandbox (git branch / throwaway data / .tmp).
No PHI, no confidential data, no external send, no prod deploy.

If any fails → human-in-the-loop: propose each change, get approval, no overnight run. Regulated/production outcomes feed the human-reviewed change process — never auto-deploy. This is CLAUDE.md's scale-caution-to-stakes rule.

Run it

Pick or write the experiment's CARD.md (objective, metric+direction, the one mutable surface, budget, autonomy, owner). New ones can copy experiments/forecast-tuning/.
git checkout -b experiments/<tag>.
Baseline run → tools/experiment_log.py add <results.tsv> --id ... --metric ... --status keep --note baseline.
Loop: edit the one mutable surface → run the read-only harness → log → keep (commit) if improved, else git reset. Simpler-and-equal = keep.
Stop on interrupt or the budget. Hand the winner + trail to the human.

Guardrails

The eval harness is read-only ground truth — never edit it to game the metric.
One mutable surface only (reviewable diffs).
Run artifacts (results.tsv, run.log, data/) are git-ignored; the harness + surface + CARD are the tracked blueprint.
Don't ship regulated/production changes from the loop.

What ships with it

Read from the repository

Just SKILL.md. No reference files, no scripts.