Run adk evals
Skill Open-Agent-Tools/General-OAT-Skills/skills/run-adk-evals
Runs Google Agent Development Kit (ADK) evaluations in sorted order with detailed results, failure summaries, and rate limiting between tests. Use when the user wants to run ADK evals, test agent performance, or validate agent behavior against evaluation datasets.From its SKILL.md
npx -y skills add Open-Agent-Tools/General-OAT-Skills --skill run-adk-evalsAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 3 stars3 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
SKILL.md
2.1 KB, as published. Nobody here has run it
Run all Google Agent Development Kit (ADK) evaluations in the specified folder in numerical then alphabetical order with detailed results and failure summaries.
Usage: Specify the folder containing the ADK agent and evaluations (e.g., test_agent/)
Example:
/run-adk-evals my_agent/
This will run all evaluations in my_agent/evals/ directory in sorted order.
Prerequisites:
- Ensure required environment variables are set (e.g., GOOGLE_API_KEY, MCP server URLs)
- Any required services must be running
- ADK dependencies installed (
pip install -r requirements.txt)
Process:
- Validate Environment: Check required environment variables and dependencies
- Discover Evaluations: Find all *_test.json files in the evals/ directory
- Sort Execution Order: Numerical then alphabetical order (e.g., test_config.json, then alphabetical *_test.json)
- Run Each Evaluation: Execute
adk evalwith--print-detailed-resultsfor each test - Summarize Results: Provide detailed summary of all results and failure reasons
ADK Eval Command Format: For each evaluation, run:
PYTHONPATH=.:$PYTHONPATH adk eval \
--config_file_path {agent_folder}/evals/test_config.json \
--print_detailed_results \
{agent_folder} \
{agent_folder}/evals/{test_file}.json
Rate Limiting: Add 10-second delays between tests to avoid API limits. Use sequential execution only.
Output Format:
- Clear section headers with timestamps
- Individual test results with pass/fail status
- Detailed failure analysis for any failed evaluations
- Overall summary with statistics
Error Handling: Rate limiting delays between tests, comprehensive error reporting for troubleshooting.