Run2 parallel orchestration
[COLM'26] SkillLearnBench is the first benchmark for evaluating continual learning methods that automatically generate agent skills.
npx -y skills add cxcscmu/SkillLearnBench --skill run2_parallel-orchestrationAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
What its author says it does
Copied from the file, not written here
Structured approach to grid search with nested loops and parallelization.
SKILL.md
1.2 KB, as published. Nobody here has run it
Parallel Orchestration for Hyperparameter Search
A structured way to iterate through multiple hyperparameters while optimizing for performance.
Strategy
- Loop through
shape_weight(most expensive since it changes the distance metric). - For each
shape_weight, precompute distance matrices for all images. - Parallelize the evaluation of
min_samplesandepsilonfor thatshape_weight.
from joblib import Parallel, delayed
def process_weight(w, citsci_groups, expert_groups, all_images, ms_range, eps_range):
# Precompute dist matrices
dist_matrices = {img: precompute_distances(citsci_groups[img], w)
for img in citsci_groups if img in expert_groups}
results = []
for ms in ms_range:
for eps in eps_range:
f1s, deltas = [], []
for img in all_images:
# ... evaluation logic ...
f1s.append(f1)
if not np.isnan(delta): deltas.append(delta)
results.append({'F1': np.mean(f1s), 'delta': np.mean(deltas), ...})
return results