Run2 itinerary validation check
[COLM'26] SkillLearnBench is the first benchmark for evaluating continual learning methods that automatically generate agent skills.
npx -y skills add cxcscmu/SkillLearnBench --skill run2_itinerary-validation-checkAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
What its author says it does
Copied from the file, not written here
A final verification step to ensure the JSON output meets all structural and constraint requirements before delivery.
SKILL.md
1.1 KB, 233 tokens by cl100k_base, as published. Nobody here has run it
- Day Count Verification: Ensure the
planarray contains exactly 7 objects representing Day 1 to Day 7. - Attraction Field Validation:
- Iterate through every day object.
- Verify that the
attractionstring is not empty (""). - Verify that the
attractionstring is not a null-placeholder ("-"). - Confirm the trailing semicolon format.
- Constraint Check:
- Confirm
transportationnever mentions "flight" or "plane". - Confirm
accommodationis sourced from the database and is pet-friendly. - Confirm
data_sourceslists all files accessed (e.g., distance matrix, restaurant csv, etc.).
- Confirm
- City Consistency: Ensure that if the
current_cityis "from A to B", the restaurants and attractions for that day are located in A, B, or a city on the path between them. - Budget Check: Sum the estimated costs of restaurants and accommodations (if price data is available in the CSV) to ensure the total is within the $5,100 limit.