Run3 glm metric validator

[COLM'26] SkillLearnBench is the first benchmark for evaluating continual learning methods that automatically generate agent skills.

Install

npx -y skills add cxcscmu/SkillLearnBench --skill run3_glm-metric-validator

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

What its author says it does

Copied from the file, not written here

Calculate specific RMSE metrics by merging simulation and observation data using exact datetime and rounded-depth matching.

1.3 KB, 277 tokens by cl100k_base, as published. Nobody here has run it

Data Preparation:
- Load field observations from field_temp_oxy.csv.
- Ensure observation datetime values are truncated or aligned to match the simulation's temporal resolution (e.g., setting minutes/seconds to zero if the simulation outputs daily/hourly values).
Merge Logic:
- Apply round().astype(int) to the depth values of both the simulation and observation datasets to ensure integer-based depth matching.
- Perform an inner join on the datetime and rounded_depth columns. No interpolation or nearest-neighbor matching is permitted for the final evaluation.
Metric Calculation:
- Overall RMSE: Calculate the Root Mean Square Error for all matched pairs.
- Annual Deep RMSE: Calculate RMSE for all matched pairs where rounded_depth $\ge 13$.
- Summer Deep RMSE: Calculate RMSE for matched pairs where rounded_depth $\ge 13$ and the month is June, July, August, or September (6-9).
Reporting:
- Save the results to /root/metrics.json using the keys: overall_rmse, annual_deep_rmse, summer_deep_rmse, overall_n_pairs, annual_deep_n_pairs, and summer_deep_n_pairs.