Dataset audit

Before working with any dataset, audit it. Do not assume you know what the data looks like, inspect it. Looking at data is the best way to boost model performance and the cheapest way to prevent failed jobs.

One call does the audit:

hf-dataset-inspect <repo_id> [--config C] [--split S] [--rows N]

It returns validity status, configs and splits, the schema, row counts, sample rows, and parquet info as one JSON object.

Check, in order:

1. The dataset loads at all (status). If it does not, tell the user; never substitute a different dataset silently.
2. Columns match the training method:
   • SFT: "messages", "text", or "prompt"/"completion"
   • DPO: "prompt", "chosen", "rejected"
   • GRPO: "prompt"
3. Row counts per split are sane for the plan (enough for training, an eval split exists when evaluation is expected).
4. Sample rows look right: read several end to end. Surface anything notable to the user: class imbalance, missing values, unexpected formats, duplicates, truncated text, license or gating surprises.

Private datasets and ones behind an access request need HF auth; if the inspect call reports an auth error, run /hf-doctor first.

Record the verified column names in your plan and assert them at the top of every training script. The audit is a prerequisite for /hf-jobs preflight; a job submitted against unverified columns is a wasted GPU hour.