Figure visual consistency audit

Figure Visual Consistency Audit

Treat a plot as a source artifact with geometry, not as transcribed prose. Preserve the distinction between visible glyphs, plotted data, analysis data, and the underlying observations.

Required workflow

1. Record the parent PDF or image path and SHA-256, page, figure and panel label, crop transform, rotation, dimensions, and render provenance. Never overwrite the supplied artifact. The case artifact_path must name an immutable PNG or SVG crop/export whose coordinate space is exactly 0..width_px by 0..height_px; do not point it at an uncropped parent PDF while supplying crop-relative coordinates. A parent-derived case cannot receive a substantive verdict unless the checker verifies the parent artifact and crop boundary; PDF parent page geometry is currently retained but deterministically indeterminate. A native synthetic or original vector source must use the schema's explicit native-source record instead of inventing a parent.
2. State the exact claim before digitizing: displayed slope direction, monotonicity, approximate straightness on a declared scale, a reported plotted value, or agreement with supplied plot data.
3. Use the strongest available route:
   • Regenerate the named figure from reviewed code and data with $reproducibility-audit.
   • Recover vector paths and text coordinates from a born-digital PDF.
   • Use supervised raster digitization when vector geometry is unavailable.
   • Use chart-to-table models only to propose candidates for human verification.
4. Read references/coordinate-recovery.md. Calibrate each axis with two separated endpoint ticks and at least one independent validation tick. Declare linear, log10, or natural-log scale and normal or reversed direction.
5. Record every selected point with stable series and point IDs, pixel center, pixel uncertainty, and clear, ambiguous, or overplotted visibility. Actual PNG media require render_dpi, complete render provenance, and non-zero uncertainty for every axis and point click, regardless of whether extraction_method says manual_digitization or raster_supervised.
6. Create a schema 1.1 case conforming to schemas/figure-check-case.schema.json. Start with review.status: unverified, review.method: not_reviewed, and any syntactically valid 64-character placeholder hash, then run:

python .agents/skills/figure-visual-consistency-audit/scripts/check_figure.py \
  figure-case.json --overlay figure-overlay.svg

Use a new overlay path. The command refuses to overwrite an existing file.

7. Copy details.review.computed_coordinate_manifest_sha256 from the result, inspect the transparent overlay against the source, and verify every glyph, series assignment, and uncertainty box. A passing artifact hash and calibration do not show that the auditor selected the correct glyphs or series.
8. Replace the review block with status: verified, method: source_and_overlay, the computed manifest hash, a reviewer identifier, and an RFC 3339 timestamp; rerun the checker. The manifest binds the complete claim (including target series, analysis scale, sampling unit, and thresholds), artifact/provenance record, axes, and every selected point. Any later claim or coordinate change invalidates the attestation and forces indeterminate. This is an attributable review declaration, not a digital signature.
9. Read references/trend-and-linearity.md. Route functional-form, uncertainty, dependence, weighting, and influence questions to $statistical-model-audit rather than treating visible geometry as a model test.
10. Compare figure labels, units, sample, transformations, and visible values with the caption, text, tables, supplement, plot data, and generating code. Use $table-data-consistency-audit for the support chain.
11. Read references/evidence-boundaries.md, classify the combined record with $research-integrity-audit, and retain the case JSON, CLI output, overlay, source hash, tool versions, and manual decisions.

Deterministic result meanings

• consistent: the media type, canvas, artifact hash, provenance, and reviewed coordinate manifest are verified, and unambiguous visible geometry supports the narrowly declared display claim under the supplied coordinate envelope and thresholds.
• inconsistent: the same verified chain contradicts that display claim or materially exceeds the declared linearity threshold.
• indeterminate: missing artifact or coordinate-review verification, insufficient distinct x positions, conservative measurement bounds, single-glyph influence, visibility, overplotting, or sampling-unit uncertainty prevents a verdict.
• Exit 0 includes consistent and indeterminate; exit 1 means inconsistent; exit 2 means the audit input or artifact chain is invalid.

The coordinate envelope represents digitization and calibration uncertainty. It is not a confidence interval, standard error, or model-based sampling uncertainty.

Output

Return:

• source identity and inspected scope;
• axis anchors, independent tick validation, scale, direction, and calibration output;
• recovered pixel and data-coordinate intervals by series and point;
• nominal OLS, Pearson, Spearman, Theil-Sen, per-point leave-one-out slopes with conservative coordinate/anchor intervals and driving-point IDs, quadratic, monotonicity, and coordinate-envelope linearity diagnostics actually computed;
• overlay path and manual review status;
• caption, table, data, code, and regenerated-figure comparisons;
• limitations, supported statement, evidence class, and evidence needed to resolve uncertainty.

Stop conditions

• Do not infer raw data from visible glyph count when points may overlap, be jittered, binned, sampled, weighted, or transparent.
• Do not call a relationship linear merely because a line looks straight; distinguish raw and axis-transformed scales.
• Do not infer causation from a figure or fitted line.
• Do not treat OCR, automated point detection, image similarity, or a chart-to-table model as an independent misconduct finding.
• Do not classify a plot-only discrepancy above unresolved_anomaly. A verified_reporting_error requires a deterministic contradiction with identified plot data, code, or a regenerated artifact on the same sample and transformation.
• Do not infer author intent.