Fse supplementary

FSE Supplementary

Use this when assembling FSE supplementary material. The governing rule is simple and strict: the paper must be judgeable from the reviewed pages alone. The artifact and any appendix support the paper; they do not hold the argument. Reviewers read the artifact at their discretion, so anything the decision depends on lives in the body.

What goes where

Content	Body (within page budget)	Artifact / appendix
Research questions and their answers	Yes	—
Core technique or study design	Yes	Full parameter grids, extra config
The headline results and their threats	Yes	Full result tables, secondary plots
Statistical tests behind a claim	The result + test named	The computation notebook
Subject-system list	Summary + selection criterion	The complete list
Raw data, logs, scripts	—	Yes
Reproduction instructions	A pointer	The README and run scripts

If a reviewer would need to open the artifact to know whether to accept, the paper is mis-partitioned — move that evidence into the body.

The page-budget discipline

FSE's ACM-template budget (FSE 2027: ≤18 pages of text and figures, plus reference pages) is generous compared with a two-column conference limit, but it is finite and counts figures, tables, and any in-paper appendix. Consequences:

• You can usually keep a full threats-to-validity treatment and the primary result tables in the body — do so; they are decision-critical.
• Long protocol tables, per-subject breakdowns, and additional RQ variants go to the artifact with explicit forward references.
• Do not use the artifact to smuggle in a result that would not fit — an argument that only closes with material outside the reviewed pages reads as unreviewable.

Double-anonymous supplementary rules

[No identity]   no authors, affiliations, acknowledgements, grants, cluster paths, or repo owners
                anywhere in the archive or appendix
[Anonymized links] the Data Availability link points at an anonymizing host, not a personal repo
[Clean archive]  no .git history, .DS_Store, credentials, caches, or large irrelevant files
[Opens clean]    verify the archive unzips and the README orients a reader in one minute on a
                fresh machine

Appendix architecture (when the template allows in-paper appendices)

• Order appendix sections to mirror the RQ order; reviewers navigate by RQ, not page number.
• Keep each appendix section referenced from the body at least once — orphaned material is invisible under discretionary review.
• Confirm on the current call whether appendices count against the page budget or attach separately; this varies and changes the split.

Vignette: splitting a mixed-methods paper

A paper mining pull requests plus interviewing developers: the body keeps the RQs, the study design, the headline quantitative results with effect sizes, the key qualitative themes, and a threats subsection; the artifact holds the mining scripts with pinned SHAs, the extracted dataset, the full codebook with inter-rater agreement, the interview protocol, and the analysis notebooks. Nothing decision-critical lives only in the artifact, because artifact inspection is discretionary and the reviewers judge from the pages.

Output format

[Supplement status] ready / needs fixes / not ready
[Partition check] anything decision-critical outside the body? <none / move: what>
[Page budget] body within limit? appendices counted correctly? yes/no
[Anonymity] archive clean of identity + metadata? passed/issues
[Body dependency] <what a reviewer can decide without opening the artifact>