Run1 token usage tracking
[COLM'26] SkillLearnBench is the first benchmark for evaluating continual learning methods that automatically generate agent skills.
npx -y skills add cxcscmu/SkillLearnBench --skill run1_token-usage-trackingAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
What its author says it does
Copied from the file, not written here
Methods for calculating or retrieving the number of tokens consumed during an LLM inference task.
SKILL.md
0.8 KB, as published. Nobody here has run it
To meet the requirement of logging consumed tokens:
- API Metadata: If using an external API (like OpenAI or Anthropic), retrieve
usage.total_tokensfrom the response object. - Local Calculation: If using a local model or needing a manual count, use a library like
tiktokenor the model's native tokenizer.# Example for tiktoken import tiktoken encoding = tiktoken.get_encoding("cl100k_base") num_tokens = len(encoding.encode(text_input)) + len(encoding.encode(text_output)) - Validation: Always ensure the
tokensvalue in the final JSON is a numeric type (e.g.,123) and not a string ("123").