Run2 pytorch preference optimization

Skill cxcscmu/SkillLearnBench/skills/b2-self-feedback-claude-opus-4-6/nlp-paper-reproduction/run2_pytorch-preference-optimization

Environment setup and testing patterns for SimPO preference optimization with PyTorch.From its SKILL.md

Install

npx -y skills add cxcscmu/SkillLearnBench --skill run2_pytorch-preference-optimization

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

SKILL.md

1.2 KB, 379 tokens by cl100k_base, as published. Nobody here has run it

PyTorch Preference Optimization Environment

Required Environment

Python 3.10 (deadsnakes PPA on Ubuntu 24.04)
torch==2.2.2 (CPU or CUDA)
transformers==4.44.2
trl==0.9.6
accelerate==0.29.2
peft==0.7.1
datasets==2.18.0
numpy==1.26.4
rich (required by trl.trainer.utils)

Setup Steps

# Install Python 3.10
add-apt-repository -y ppa:deadsnakes/ppa
apt-get install -y python3.10 python3.10-venv python3.10-dev

# Create venv
python3.10 -m venv /root/simpo_env
source /root/simpo_env/bin/activate

# Install packages
pip install torch==2.2.2 --index-url https://download.pytorch.org/whl/cpu
pip install transformers==4.44.2 accelerate==0.29.2 datasets==2.18.0 \
    trl==0.9.6 peft==0.7.1 numpy==1.26.4 rich

Key Gotcha

trl==0.9.6 depends on rich but doesn't declare it as a dependency. Must install separately.

Running Tests

cd /root/SimPO
source /root/simpo_env/bin/activate
python -m unittest unit_test.unit_test_1

Logging Environment

(python -VV && echo "---" && python -m pip freeze) > /root/python_info.txt

What ships with it

Read from the repository

Just SKILL.md. No reference files, no scripts.