agentsclimarketplace
Sign in

Statistical consistency audit

Skill Nicowyn/paper-audit-skills/.agents/skills/statistical-consistency-audit

Recompute and validate reported statistics in papers and supplements. Use for table arithmetic, percentages and counts, weighted means, Bernoulli variance, signed-bias formulas, minimum sample-size thresholds, confidence intervals, standard errors, z statistics, p-values, rounding feasibility, and contradictions across text, tables, equations, and appendices.From its SKILL.md

Install
npx -y skills add Nicowyn/paper-audit-skills --skill statistical-consistency-audit

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

  • 23 days oldThe repository was created 23 days ago. New is not bad, but a brand new repository carrying a familiar-sounding name is the shape a typosquat arrives in, and there has been no time for anyone else to find a problem with it.
  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

2.8 KB, 513 tokens by cl100k_base, as published. Nobody here has run it

Statistical Consistency Audit

Convert a reported claim into explicit inputs, a formula, a rounding rule, and an independently computed output.

Workflow

  1. Verify the source visually. Record the PDF page and table, figure, or equation.
  2. Transcribe the reported inputs without silently correcting them.
  3. State direction and definitions explicitly. Keep signed quantities signed.
  4. Record precision. Treat a continuous value rounded to d decimals as an interval of half a unit in the last displayed place. Encode rounded continuous input precision with input_decimals; omit it for exact discrete counts, sample sizes, and category totals. Use tolerance: 0 for an exact reported count or identity. Use decimals for an output displayed after decimal rounding, or an explicit non-zero tolerance only when that comparison radius is substantively justified.
  5. Create a JSON case using references/check-spec.md.
  6. Run the deterministic checker:
python -m paper_audit check case.json
  1. Confirm high-impact failures with a second calculation or direct identity.
  2. Treat ROUNDING_FEASIBLE as unresolved. Classify a mismatch as verified only if feasible rounding intervals and documented adjustments cannot reconcile it.
  3. Report the smallest affected claim and then assess downstream impact.

Supported deterministic checks

  • difference
  • ratio
  • sum
  • weighted_mean
  • bernoulli_variance
  • signed_bias_variance
  • minimum_sample_size
  • confidence_interval
  • z_test

Read references/rounding-and-inference.md before using rounded values, p-values, clustered estimates, adjusted tests, or equivalence language.

Rules

  • A tool result is evidence of a numerical relationship, not evidence of author intent.
  • Do not mix coefficients, standard errors, p-values, or confidence intervals from different models or variance estimators.
  • Non-rejection is not equivalence. Require a prespecified margin and an equivalence or non-inferiority procedure.
  • If a reported bias is absolute but the cited identity uses a signed bias, convert the direction before substitution.
  • For integer thresholds under a strict inequality, verify both Nmin - 1 and Nmin; do not round the continuous root.
  • If OCR may have changed a sign, exponent, decimal separator, or glyph, retain the issue as unresolved until the rendered page is checked.

What ships with it: 12 files

242.9 KB alongside SKILL.md, 9 of them executable

agents/

references/

scripts/

Keep looking

Skills are one crate of 326,834. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.