Auto research engineer
Use to autonomously optimize ONE asset against a single objective metric — change it, score it with a locked measuring stick, keep the winner / revert the loser, log each round, and repeat overnight until a target/plateau/wall-clock stop. Adapted from Karpathy's autoresearch loop.From its SKILL.md
npx -y skills add tpop78/memory-os --skill auto-research-engineerAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
SKILL.md
4.9 KB, as published. Nobody here has run it
Auto Research Engineer
You optimize ONE asset toward ONE number. You change the asset, score it, keep what wins,
revert what loses, log every round, and keep going — unattended — until a stop condition fires
or the human stops you. The three-file system lives in .memory/autoloop/<tag>/.
The three files
- INSTRUCTIONS.md — the goal, the metric (+direction +target), the asset path(s), the budget, the stop conditions, the rules. Human-locked: you never edit it.
- SCORING.sh — the locked measuring stick. Run it; read the single number it prints (the last numeric token on its last line). You may read and run it; you must NEVER edit it. No moving the goalposts — the definition of "better" is fixed.
- RESULTS.tsv — the ledger you append to. Columns:
round ref score delta cost_s status change.
Setup (only if not already scaffolded — normally /auto-research did this)
Scaffold with node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/autoloop-init.mjs <tag>, then help the
human fill INSTRUCTIONS.md and implement SCORING.sh. Run the FIT CHECK below before looping.
FIT CHECK (run before the first loop; be honest)
A target is only worth optimizing if all three MUST-HAVES hold:
- Objectively scored — a real number, not "make it look nicer".
- Fast feedback — minutes/hours, not weeks (no SEO-reindex / multi-month churn).
- Write access — you can actually change the asset (a file/API, not a published video). NICE-TO-HAVES (more = more powerful): high feedback volume; cheap to fail; a consistent, repeatable measuring stick. If a must-have fails, say so plainly and propose a better-shaped target instead of pretending.
Mode (set in INSTRUCTIONS)
- git-worktree — create a dedicated
autoresearch/<tag>worktree after the human confirms the run. Snapshot only the declared asset paths before each round; commit only those paths for a winner and restore only those paths frombest/for a loser. Never run repository-wide reset or clean. - snapshot — not in git: before each change, the current best lives in
.memory/autoloop/<tag>/best/; save each round torounds/NNN/, promote a winner tobest/, restorebest/to revert a loser.
The loop — repeat until a stop condition fires
- Integrity check. Re-hash INSTRUCTIONS.md + SCORING.sh; confirm YOU have not changed them. If the human changed them mid-run, reload and announce it.
- Baseline (round 0). Score the asset as-is (no change). Record it as the current best.
- One hypothesis, one change. Edit ONLY the declared asset path(s). Never touch any other file.
- Score. Run
SCORING.shwithin the per-round budget; read the single number. If it exceeds 2× the budget, kill it and treat the round as a crash. - Decide (direction from INSTRUCTIONS):
- better than best → keep: promote the round to
best/, then commit only declared assets in git-worktree mode. New best. - not better → revert: restore only declared asset paths from
best/. - crashed / non-numeric → if trivially fixable (typo, missing import) fix and retry once; else crash: revert and move on.
- better than best → keep: promote the round to
- Log one row to RESULTS.tsv (
status= baseline|keep|revert|crash). Append a one-line note to.memory/JOURNAL.mdand update.memory/STATE.md(Now= "auto-research <tag>, best=<score>"). - Check stop conditions from INSTRUCTIONS: target_score reached,
plateau_roundsconsecutive non-improvements, orwallclock_capelapsed. If any fired → STOP and summarize. Otherwise loop.
Autonomy & guardrails
- Do not ask "should I continue?" Once looping, run unattended until a stop condition or the human
interrupts. This intentionally overrides memory-os's
break_atcheckpoint nudge for the run. - Write only the declared asset path(s). Never edit INSTRUCTIONS.md or SCORING.sh.
- Stay isolated. Git runs happen only in the recorded dedicated worktree. Refuse to start if the worktree cannot be established or any declared path escapes it.
- Preserve unrelated work. Never use repository-wide reset, clean, restore, checkout, or broad staging. Snapshot and restore only the declared assets.
- Every loser reverts to a known-good state — never leave the asset broken.
- Bounded: the stop conditions guarantee the run ends; never loop truly forever.
On finishing
Summarize: rounds run, baseline → best, total improvement, and where the winning asset lives
(dedicated worktree branch HEAD or best/). Offer to write a short report and ask before removing
the worktree.