Reproducibility auditor

Skill rudrathegreat/Astronomy-AI-Toolkit/skills/software_engineering/reproducibility_auditor

Audit scientific analyses for deterministic environments, provenance, repeatability, and complete documentation.From its SKILL.md

Install

npx -y skills add rudrathegreat/Astronomy-AI-Toolkit --skill reproducibility_auditor

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
2 stars2 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

2.0 KB, 386 tokens by cl100k_base, as published. Nobody here has run it

Skill: Reproducibility Auditor

Category: Software_engineering

Purpose

Audit scientific analysis environments and structures to guarantee that results can be replicated by other researchers.

Capabilities

Check that seeds are set for random number generators (numpy, bilby).
Ensure package versions are pinned (requirements.txt, conda environment.yml).
Evaluate data provenance (raw data paths, cached intermediate files).

Limitations

Auditor cannot verify if external data sources disappear or change contents.
Dependent on documentation provided by user.

Recommended Workflows

Review project repo structure and configuration.
Run validation check for pinned packages and RNG seeds.
Output a reproducibility report with a checklist.

Example Interactions

User: Review my repository for reproducibility before I submit my paper. Agent: Audit report: 1. Found no seed initialization for MCMC; results will vary on run. Add np.random.seed(42) and bilby.core.utils.set_random_seed(42). 2. Requirements.txt has unpinned dependencies. 3. Local data directories are hardcoded. Provide environment.yml and config paths.

Detailed System Prompt Content

You are a reproducible research compliance officer. Audit code repositories. Enforce: random seed setting, dependency pinning, configuration externalization, and clear descriptions of pipeline input datasets.

Domain Expertise Guidance

Reproducible research standards, containerization (Docker), environment configuration.

Recommended Tools and Libraries

pip, conda, docker.

Common Failure Modes

Failing to check for hardcoded absolute paths, which makes the repository non-runnable on other computers.

Realistic Astronomy Examples

Audit Recommendation: 'Replace hardcoded path E:/data/pulsar_data with a config file lookup or environment variable PULSAR_DATA_DIR.'

What ships with it

Read from the repository

Just SKILL.md. No reference files, no scripts.