Trl setup

Skill cxcscmu/SkillLearnBench/skills/b1-one-shot-claude-sonnet-4-6/nlp-paper-reproduction/trl-setup

[COLM'26] SkillLearnBench is the first benchmark for evaluating continual learning methods that automatically generate agent skills.

Install

npx -y skills add cxcscmu/SkillLearnBench --skill trl-setup

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

What its author says it does

Copied from the file, not written here

Setup and installation of TRL (Transformer Reinforcement Learning) library with compatible torch/transformers versions. Use when setting up preference optimization training environments.

SKILL.md

1.2 KB, as published. Nobody here has run it

TRL Setup

Installation

pip install torch transformers trl accelerate peft datasets --break-system-packages

Compatibility Notes

TRL imports from trl.trainer.utils: DPODataCollatorWithPadding, pad_to_length, etc.
Some imports may change across TRL versions (e.g., trl_sanitze_kwargs_for_tagging vs trl_sanitize_kwargs_for_tagging)
If running in externally-managed Python (Debian), use --break-system-packages flag

Common Import Errors

trl_sanitze_kwargs_for_tagging not found → check trl version, may be renamed or removed
Missing CPOTrainer → upgrade trl: pip install --upgrade trl

Running Tests from Project Root

cd /root/SimPO && python -m pytest unit_test/unit_test_1.py -v
# or
cd /root/SimPO && python unit_test/unit_test_1.py

Key TRL Trainer Utilities

DPODataCollatorWithPadding: handles padding for preference pairs
pad_to_length: pads tensor to specified length
disable_dropout_in_model: disables dropout during eval