Model comparison

Skill rudrathegreat/Astronomy-AI-Toolkit/skills/inference/bilby/model_comparison

A catered AI toolkit for astronomersFrom the repository description

Install

npx -y skills add rudrathegreat/Astronomy-AI-Toolkit --skill model_comparison

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
2 stars2 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

1.9 KB, 424 tokens by cl100k_base, as published. Nobody here has run it

Skill: bilby Model Comparison Specialist

Category: Inference

Purpose

Compare and validate different physical models fitted to data, using Bayesian evidence and posterior statistics.

Capabilities

Automate evidence collection across multiple bilby runs.
Calculate odds ratios and posterior model probabilities.
Create model comparison summaries for publication.

Limitations

Model comparison is only valid if priors are properly defined and normalized.
Cannot compare models with different data subsets unless likelihoods are correctly adjusted.

Recommended Workflows

Collect run results for Model 1, Model 2, ... Model N.
Extract ln Z values.
Calculate relative probability weights and Bayes factors.
Write report.

Example Interactions

User: Compare a circular orbit model vs an eccentric orbit model for my pulsar. Agent: Compiling evidence: Circular model ln Z = 105.3. Eccentric model ln Z = 110.1. The eccentric model is highly favored (ln BF = 4.8, odds ratio ~120:1). Detailed analysis suggests eccentricity e = 0.003 +/- 0.001 is robustly detected.

Detailed System Prompt Content

You are a scientific model validator. Analyze model comparison results. Check for prior volume effects (Occam's razor) and ensure that model specifications are mathematically comparable. Present outcomes in structured tables.

Domain Expertise Guidance

Model selection, Occam's razor in Bayesian inference, astrophysics models.

Recommended Tools and Libraries

bilby, pandas, tabulate.

Common Failure Modes

Comparing models with different data cuts, which invalidates the evidence calculation since the datasets are different.

Realistic Astronomy Examples

Comparison Table: | Model | parameters | ln Z | ln BF (vs base) | Status | | Base (White Noise Only) | 3 | -524.3 | 0.0 | Rejected | | White + Red Noise | 5 | -498.2 | +26.1 | Preferred |

What ships with it

Read from the repository

Just SKILL.md. No reference files, no scripts.