Recsys supplementary

RecSys Supplementary

Use this when deciding where each piece of supporting material goes. RecSys is unusual: it does not invite a separate anonymous supplement file the way OpenReview venues do. Instead the appendix counts inside the content-page budget, and executable or bulky material lives in the anonymous repository linked from the paper. That constraint drives every placement decision.

The three-bucket decision

Bucket	What belongs here	Constraint
Main body (8 / 4 pages)	The recommendation claim, method, headline offline table, the offline-online bridge, key ablation	Must stand alone; a reviewer should grasp the contribution without the appendix
Appendix (inside the budget)	Proof of any bound, dataset details, tuning grids, extra ablations, per-dataset breakdowns	Every page here is a page not spent on the body — include only what changes belief
Anonymous repository	Code, configs, checkpoints, logged propensities, raw run outputs, seeds	Reviewers may not open it; nothing decision-critical can live only here

The sharpest RecSys-specific edge: because the appendix eats your page budget, authors are tempted to push essential evidence (like the equal-budget tuning grid) into the repository — but that is exactly the evidence a skeptical reviewer wants to see stated, so summarize it in the appendix and keep the full grid in the repository.

What to keep in the body no matter what

• The method and its assumption (e.g., positivity for an off-policy estimator).
• The headline ranking table with tuned baselines and reported variance.
• The one ablation that isolates the mechanism.
• One sentence stating the split protocol and whether metrics are full-ranking.

What gets read, and how carefully

Item	Inspection likelihood	Practical implication
Headline offline table	High	Belongs in the body, not the appendix
Tuning grid summary	Medium-high	Summarize in the appendix; reviewers check for equal budget
Extra datasets / ablations	Medium	Reference each from the body or it is invisible
Code repository	Variable (reviewer discretion)	README must orient a reader in one minute
Raw run logs	Low	Repository only; never rely on them for a claim

Vignette: splitting an off-policy ranking paper

A submission proposing an exposure-corrected ranker: the body keeps the estimator, its positivity assumption, the tuned headline table, and the simulator bridge figure; the appendix holds the propensity-model details, the equal-budget tuning grid summary, and per-dataset breakdowns; the repository carries the seeded runner, checkpoints, and logged propensities. Nothing decision- critical lives only in the repository, because repository inspection is discretionary.

Cross-reference discipline

Body Table 1  --cites-->  App. B (tuning grid)  --mirrors-->  repo/configs/*.yaml
Body Fig. 1   --cites-->  App. C (simulator setup)  --mirrors-->  repo/sim/run.sh

Orphaned appendix material — a table no body sentence points to — wastes budget and is effectively invisible under reviewer discretion.

Output format

[Placement status] Ready / Needs fixes / Not ready
[Body-only items] <method / assumption / headline table / key ablation / split statement>
[Appendix items] <proofs / tuning-grid summary / extra datasets — inside the budget>
[Repository items] <code / configs / checkpoints / propensities / logs>
[Main-paper dependency] <what breaks if the appendix or repo is ignored>