Trl setup

Skill cxcscmu/SkillLearnBench/skills/b1-one-shot-claude-sonnet-4-6/nlp-paper-reproduction/trl-setup

Setup and installation of TRL (Transformer Reinforcement Learning) library with compatible torch/transformers versions. Use when setting up preference optimization training environments.From its SKILL.md

Install

npx -y skills add cxcscmu/SkillLearnBench --skill trl-setup

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

SKILL.md

1.2 KB, 264 tokens by cl100k_base, as published. Nobody here has run it

TRL Setup

Installation

pip install torch transformers trl accelerate peft datasets --break-system-packages

Compatibility Notes

TRL imports from trl.trainer.utils: DPODataCollatorWithPadding, pad_to_length, etc.
Some imports may change across TRL versions (e.g., trl_sanitze_kwargs_for_tagging vs trl_sanitize_kwargs_for_tagging)
If running in externally-managed Python (Debian), use --break-system-packages flag

Common Import Errors

trl_sanitze_kwargs_for_tagging not found → check trl version, may be renamed or removed
Missing CPOTrainer → upgrade trl: pip install --upgrade trl

Running Tests from Project Root

cd /root/SimPO && python -m pytest unit_test/unit_test_1.py -v
# or
cd /root/SimPO && python unit_test/unit_test_1.py

Key TRL Trainer Utilities

DPODataCollatorWithPadding: handles padding for preference pairs
pad_to_length: pads tensor to specified length
disable_dropout_in_model: disables dropout during eval

What ships with it

Read from the repository

Just SKILL.md. No reference files, no scripts.