agentsclimarketplace
Sign in

Paired dataset audit

Skill StaryMoon/paired-dataset-audit-skill/skills/paired-dataset-audit

Audit paired image datasets for missing counterparts, corrupt files, shape or channel mismatches, duplicate content, and suspicious samples, then produce a JSON report and visual contact sheet. Use before training restoration, translation, segmentation, super-resolution, or other paired vision models.From its SKILL.md

Install
npx -y skills add StaryMoon/paired-dataset-audit-skill --skill paired-dataset-audit

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

1.7 KB, 298 tokens by cl100k_base, as published. Nobody here has run it

Paired Dataset Audit

Check the dataset before spending GPU time. Keep the audit read-only: do not rename, delete, or resize source files unless the user separately approves a repair plan.

Workflow

  1. Identify the two dataset roots and confirm how filenames correspond.
  2. Run the default relative-stem matcher:
python3 scripts/audit_paired_dataset.py data/input data/target \
  --output output/dataset-audit
  1. Inspect report.md and pair-samples.png.
  2. Treat missing pairs, corrupt files, and shape mismatches as blocking by default.
  3. Review duplicate groups before calling them leakage; legitimate burst frames or repeated masks can share content.
  4. If repair is requested, create a separate repaired copy and retain audit.json as provenance.

Matching rules

The default key is the relative path without the final extension. scene01/rain/0001.png matches scene01/rain/0001.jpg. Use --flat-stem only when both roots have globally unique stems.

Quality rules

  • Never modify source directories during audit.
  • Verify decoded dimensions and modes, not only filenames.
  • Report duplicate hashes separately for each side.
  • Sample contact sheets deterministically so CI results remain stable.
  • Exit non-zero with --strict when blocking defects exist.

See references/repair-policy.md before applying fixes.

What ships with it: 3 files

8.1 KB alongside SKILL.md, 1 of them executable

agents/

references/

scripts/

Keep looking

Skills are one crate of 326,144. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.