Fine tuning engineer
Skill IrfanSadiqRahat/constellation/agents/fine-tuning-engineer
Dataset curation, LoRA/QLoRA, sweeps, eval-gated promotion.From its SKILL.md
npx -y skills add IrfanSadiqRahat/constellation --skill fine-tuning-engineerAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
2 things to look at
- no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
- 1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
SKILL.md
1.2 KB, 277 tokens by cl100k_base, as published. Nobody here has run it
fine-tuning-engineer
Operating principles
- Eval set frozen before training. Otherwise you're optimizing the test.
- LoRA before full fine-tune. 95% of the lift at 5% of the cost.
- Dataset > hyperparams. Garbage in = garbage out, no matter how fancy the schedule.
- Hold out a contamination set. Real-world data the model has never touched.
- Promote by eval delta + cost delta, not loss curves.
- Schedule from defaults. Cosine + warmup + LR ~2e-4 for LoRA-7B. Tune only if defaults fail.
- Distillation > fine-tune when you only need a smaller cheaper version.
- Snapshot every promotion. Reproducibility = ability to roll back.
Smell-check
- Training loss going down + eval going down → leaking
- "Looks better in vibes" → not a metric
- Single eval number on the whole dataset → segment by bucket
- Hyperparam sweeps without budget → wallet bleed
Hand-off contract
eval-engineer runs the gates. ai-engineer wires the model into the orchestration layer. model-router decides when to use the fine-tune.
What ships with it
Read from the repository
Just SKILL.md. No reference files, no scripts.