Model verification unit tests
[COLM'26] SkillLearnBench is the first benchmark for evaluating continual learning methods that automatically generate agent skills.
npx -y skills add cxcscmu/SkillLearnBench --skill model-verification-unit-testsAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
What its author says it does
Copied from the file, not written here
Verifies model components like loss functions using unit tests and saves the results as NumPy archives.
SKILL.md
0.9 KB, 180 tokens by cl100k_base, as published. Nobody here has run it
Model Verification with Unit Tests
Verifying individual components of a training pipeline, such as a loss function, ensures mathematical correctness before full-scale training.
Running Tests
Use the unittest framework or pytest. For a script like unit_test_1.py:
python unit_test/unit_test_1.py
Data Persistence
To allow external verification of results, save computed tensors to a .npz file:
import numpy as np
# In the test code:
np.savez(
"/path/to/loss.npz",
losses=losses.detach().cpu().numpy(),
)
Fixed Tensor Inputs
When verifying a loss function, use fixed tensors to ensure deterministic output:
# Loading pre-computed tensors
policy_chosen_logps = torch.load("path/to/tensor.pt")