Siggraph artifact evaluation

SIGGRAPH Artifact Evaluation (Replicability Stamp)

SIGGRAPH does not run the ACM Artifact Review and Badging scheme that SIGSOFT and systems venues use. The computer-graphics community's equivalent is a community-run replicability stamp, earned after acceptance and independent of the review: the Graphics Replicability Stamp Initiative (GRSI, replicabilitystamp.org) and the related Code Replicability in Computer Graphics (CRCG, replicability.graphics). The distinction matters: a stamp certifies that a volunteer rebuilt your code and reproduced your paper's results, and archives that code for the community. Facts below trace to resources/official-source-map.md; confirm the current GRSI process before you package.

What the stamp is (and is not)

	Graphics Replicability Stamp (GRSI)	ACM Artifact Badges (other venues)
Who runs it	Volunteer researchers from the graphics community	The venue's artifact evaluation committee
What it certifies	Your code replicates the paper's results	Available / Functional / Reusable / Reproduced
When	Post-acceptance, on the initiative's own schedule	Post-acceptance, on the venue's AE deadline
Archiving	Software Heritage snapshot + long-term ID (since 2023)	DOI-issuing archive
Scope	Independent of SIGGRAPH's review; optional recognition	Tied to the venue's AE track

The GRSI has recognized graphics code since 2016 and is supported across the field's venues (ACM TOG, IEEE TVCG, Wiley CGF, Elsevier C&G, CAD/CAGD). CRCG has, since July 2020, focused specifically on checking whether SIGGRAPH papers' results are replicable. Neither is required for publication — but the stamp is the credible, checkable signal of a runnable contribution in this community.

What a volunteer actually does

Assume a graphics-literate volunteer clones your repository on their own machine and tries to regenerate a headline result from your paper. They are checking replicability of results, not just that the code compiles:

Contribution type	What the volunteer reproduces	Common failure caught
A rendering technique	A converged image matching a paper figure, within tolerance	Non-deterministic output; missing scene assets
A geometry/mesh method	A processed mesh matching the reported statistics	Hard-coded absolute paths; unshipped input meshes
A simulation	A representative frame/sequence from the paper	Unseeded RNG; platform-specific solver drift
A learning-based method	A result from released weights, not retraining from scratch	Weights absent; inference needs undocumented data

Design so the first headline result reproduces from a documented command on a clean checkout.

Packaging plan for a replicable graphics artifact

[Build]        a documented, pinned build (CMake/conda/Docker) that compiles on a clean machine;
               name exact compiler/CUDA/driver versions where GPU code is involved
[Assets]       ship (or give a stable download for) the scenes, meshes, textures, and weights the
               results need -- code without inputs cannot replicate a figure
[Determinism]  fix seeds; document tolerance for floating-point/GPU non-determinism; state which
               results are bit-exact vs perceptually-equal
[Mapping]      a table: paper figure/table -> command -> expected output image/metric/frame
[Timings]      report the hardware and the wall-clock the paper claims, so timings are checkable
[License]      an OSI-approved license so the code can be shared and reused
[Archive]      a Software Heritage snapshot (GRSI archives accepted code there) + a Zenodo DOI

Determinism is the graphics-specific hard part

Reproducing an image is not reproducing a number. Plan for it:

• State the comparison metric and tolerance. Volunteers cannot judge "looks right" — give PSNR/SSIM/LPIPS or a diff threshold against a bundled reference image.
• Pin the stochastic pieces. Monte Carlo renderers, stochastic simulations, and GPU reductions drift; seed them and document the residual non-determinism.
• Bundle reference outputs. Include the exact images/frames the paper reports so a volunteer compares against ground truth, not a re-render of their own.
• Separate "replicate the result" from "retrain the model." For learning-based work, ship weights and an inference path; full retraining is rarely the replication target.

Calibration

• The stamp is post-acceptance and optional; it is decided by the initiative's volunteers on their timeline, not by the SIGGRAPH committee, and not on the camera-ready deadline.
• GRSI and CRCG are related but distinct efforts; check each initiative's current submission instructions and supported-venue list before submitting your code.
• A stamp strengthens a paper's reach but is never a substitute for in-paper evidence — the review already happened without it.

Output format

[Target] Graphics Replicability Stamp (GRSI) / CRCG check / both
[Headline result] <figure/table the volunteer will reproduce>
[Clean-machine build] compiles + runs from documented command? yes/no
[Assets] scenes/meshes/weights shipped or stably linked? yes/no
[Determinism] seeds fixed + tolerance + reference outputs bundled? yes/no
[Claim mapping] <figure -> command -> expected output present?>
[Archive] Software Heritage snapshot + DOI + OSI license? yes/no
[Fixes before submitting code] <ordered>