Data retrieval
[COLM'26] SkillLearnBench is the first benchmark for evaluating continual learning methods that automatically generate agent skills.From the repository description
npx -y skills add cxcscmu/SkillLearnBench --skill data-retrievalAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
SKILL.md
1.4 KB, 307 tokens by cl100k_base, as published. Nobody here has run it
name: data-retrieval description: How to search and extract information from large JSON data files. Use this skill whenever you need to process JSON files in /root/DATA to answer specific questions.
Data Retrieval from JSON
This skill provides strategies for efficiently extracting information from JSON data files.
Strategies
- Large File Handling: When dealing with large JSON files, avoid reading the entire file at once if possible. Use
grep_searchto find keywords and then read specific lines around the matches. - Key-Value Identification: Identify the core keys (e.g., "Market Research Report", "Competitors", "Team Members") before diving into the content.
- Data Filtering: Use the structure of the JSON to filter relevant sections (e.g., look for an array of "competitors" or "team_members").
- Multiple Files: When a query requires data from multiple files (e.g., mapping employee names to IDs), perform the search in parallel or sequential steps to link the information.
Patterns
- Question to Key Mapping:
- "Market Research Report" ->
market_research_reportor similar keys. - "Competitors" ->
competitors. - "Team Members" ->
team_membersoremployees. - "Insights/Strengths/Weaknesses" -> Look for these keywords within competitor descriptions or feedback arrays.
- "Demo URLs" -> Search for "http" or "url" within relevant sections.
- "Market Research Report" ->
What ships with it
Read from the repository
Just SKILL.md. No reference files, no scripts.