Run2 run unit test and save results
[COLM'26] SkillLearnBench is the first benchmark for evaluating continual learning methods that automatically generate agent skills.
npx -y skills add cxcscmu/SkillLearnBench --skill run2_run_unit_test_and_save_resultsAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
What its author says it does
Copied from the file, not written here
Executes the provided unit test to verify the SimPO loss implementation and saves the output to a specific NPZ file for evaluation.
SKILL.md
0.8 KB, as published. Nobody here has run it
- Execute the test script:
python /root/SimPO/unit_test/unit_test_1.py. - The script will use the
simpo_lossimplemented insimpo_trainer.py. - Capture the generated loss tensor.
- Save to NPZ:
- Use
numpy.savezto save the results. - File path:
/root/loss.npz. - Key:
losses.
- Use
- Ensure the file is correctly written and accessible for the final evaluation.
import numpy as np
import torch
# This logic is typically inside the unit test or triggered by it
# losses = trainer.simpo_loss(input_tensors).detach().cpu().numpy()
# np.savez('/root/loss.npz', losses=losses)