agentsclimarketplace
Sign in

Model critique

Skill sunxd3/bayesian-statistician-plugin/skills/model-critique

Three-part critique methodology for a single fitted Bayesian model — statistical, domain, and framework assessment plus the verdict structure and refinement-suggestion conventions.From its SKILL.md

Install
npx -y skills add sunxd3/bayesian-statistician-plugin --skill model-critique

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

2.3 KB, as published. Nobody here has run it

Model Critique

Reference for assessing a single fitted model from three angles: statistical health, domain validity, and framework appropriateness. Read references on demand. For diagnostic interpretation (LOO, Pareto k, LOO-PIT shapes), see bayesian-model-diagnostics. For PPC plot types, see visual-predictive-checks. For MCMC sampler health, see convergence-diagnostics.

Key Practices

  • Ground every suggestion in diagnostic evidence. "Consider a more flexible model" is useless; "LOO-PIT shows U-shape (underdispersed); switch Normal → Student-t to accommodate heavy tails" is useful. Each suggestion must trace back to a specific diagnostic pattern.
  • Framework concerns outrank within-framework refinements. If the model ignores available data that could support a fundamentally different framework, that is the highest-priority issue — above any statistical or domain refinement.
  • Do not hallucinate domain conventions. "I am uncertain whether this domain has a standard link function" beats fabricating one.
  • Respect statistical achievements. If the model passes diagnostics and predicts well, do not suggest changes that would break it. Domain suggestions should be nested extensions or reparameterizations that preserve the current model as a special case where possible.

References

  • references/statistical-assessment.md — diagnostic evidence checks: LOO validity for grouped/temporal data, Pareto k, LOO-PIT, retrodiction-vs-prediction mismatch, estimand contraction, residual investigation, temporal-gap handling.
  • references/domain-assessment.md — domain identification and the five sub-dimensions (link function, missing components, parameterization, predictor selection, model structure) with cross-domain examples.
  • references/framework-questioning.md — the "is this the right framework given all the available data" check: unused data, framework alternatives, data-generating-process completeness.
  • references/decision.md — VIABLE / CONCERNS / BROKEN verdict criteria and the report-section templates (priorities, framework concerns, new questions).

Keep looking

Skills are one crate of 627,559. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.