agentsclimarketplace

Image benchmark report

Skill StaryMoon/image-benchmark-report-skill/skills/image-benchmark-report

Agent Skill for PSNR/SSIM image benchmarks with per-image scores and worst-case reports.

Install
npx -y skills add StaryMoon/image-benchmark-report-skill --skill image-benchmark-report

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

  • 23 days oldThe repository was created 23 days ago. New is not bad, but a brand new repository carrying a familiar-sounding name is the shape a typosquat arrives in, and there has been no time for anyone else to find a problem with it.
  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Evaluate paired prediction and ground-truth image folders with PSNR and SSIM, preserve per-image scores, and generate worst-case visual comparisons plus HTML, CSV, and JSON reports. Use for image restoration, super-resolution, denoising, dehazing, deraining, compression, or generation benchmark review.

SKILL.md

1.8 KB, as published. Nobody here has run it

Image Benchmark Report

Produce metrics and visual evidence together. A mean score without pair coverage, preprocessing settings, and worst examples is not a complete benchmark.

Workflow

  1. Confirm prediction and ground-truth roots and filename matching.
  2. Confirm color space, border crop, and whether image sizes must match exactly.
  3. Run:
python3 scripts/build_image_benchmark.py predictions ground-truth \
  --output output/benchmark \
  --crop-border 0 \
  --color-space rgb
  1. Check pair coverage in summary.json before quoting averages.
  2. Open report.html and inspect the worst-scoring samples. Look for registration errors, wrong color range, accidental resizing, or filename mismatches.
  3. Keep scores.csv with any table or paper number derived from the run.

Metric contract

  • Images are decoded to RGB and evaluated in [0, 1].
  • --color-space y evaluates the luma channel using the documented conversion.
  • SSIM uses skimage.metrics.structural_similarity with channel-aware defaults.
  • Different image sizes are blocking unless --resize-prediction is explicitly requested.

Quality rules

  • Never average unmatched or unreadable files into zero-valued scores.
  • Record every resize and crop operation.
  • Use --strict in CI to reject missing pairs and size mismatches.
  • Inspect worst cases even when the aggregate improves.

Read references/metric-notes.md before comparing against published values.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.