Dac artifact evaluation

DAC Artifact Evaluation

Read the venue reality first. Unlike the software-engineering venues (FSE/ICSE/ISSTA) and the computer-architecture venues (MICRO/ISCA/HPCA), DAC has historically not operated a standing, badge-issuing artifact-evaluation track for research manuscripts; whether the current cycle adds one is 待核实 (resources/official-source-map.md). So do not build for an ACM badge that DAC does not award. Build instead for the two things that actually matter at DAC: making a skeptical reviewer trust your QoR numbers and making the community able to reuse and cite your work.

What an EDA artifact is for at DAC

Goal	What it buys you	How it is earned
Reviewer credibility	A QoR claim that reads as real, not cherry-picked	Standard benchmarks, disclosed flow, a runnable path
Community reuse	Citations and follow-on work built on your tool	Clean open-source release, good docs, a license
Contest/leaderboard standing	Recognition on ISPD/TAU/CAD-contest tasks	A tool that runs on the contest harness

There is no badge to chase, so the design target is a package a stranger can run and a reviewer would believe — not a checklist an evaluator scores.

Packaging plan for an EDA artifact

[Flow]        pin the toolchain: OpenROAD / ABC / Yosys / KLayout versions, or a Docker image;
              avoid "install these commercial tools yourself" as the only path
[Benchmarks]  ship or clearly reference the exact benchmark release (ISPD/EPFL/ISCAS/ITC/TAU/
              CircuitNet) with its version
[PDK/library] include the open PDK/library used (Nangate/ASAP7) or document the NDA-gated one
[README]      what it is; how to install; how to run one small demo in minutes; how to reproduce
              each headline QoR number; expected runtime and outputs
[Mapping]     an explicit table: paper claim -> script + benchmark -> expected QoR result
[Seeds]       fixed seeds and a note on nondeterminism for stochastic/RL flows
[License]     an OSI-approved license so others can build on it
[Archive]     deposit a versioned release in a DOI-issuing archive (Zenodo/Software Heritage) after
              acceptance for a stable citation

The clean-machine, bounded-time test

Even without formal evaluators, assume a reviewer or a future user gives your package a short, bounded try on a clean machine. Design the first ten minutes to succeed:

• A one-command small demo that runs a tiny circuit through your tool and prints a recognizable QoR number.
• No dependency on the authors' cluster, private PDK, or a live license server for the demo path.
• A clear separation between the fast demo and the slow full-benchmark reproduction (which may take hours on large designs — say so).

Anonymized review artifact vs public release

• At submission (double-blind): if you link the artifact for reviewers, anonymize it exactly like the paper — no author/lab names, no personal repos, no cluster paths, no commercial-flow fingerprints. Reviewers are not required to open it, so it can only strengthen a paper that already stands on six pages.
• After acceptance: replace anonymized placeholders with the public, licensed, DOI-archived release cited in the camera-ready; this is the version the community reuses.

Worked vignette: packaging a placement tool + benchmarks

A paper contributes an ML-guided placer evaluated on ISPD circuits. A credible, reusable package: a Docker image with the placer and an OpenROAD flow pre-built; a run_demo.sh that places one small ISPD circuit and reports wirelength/DRC in under a minute; a reproduce/ directory whose scripts regenerate each per-benchmark table with fixed seeds; a claim-to-script-to-benchmark mapping in the README; the exact ISPD release and an open PDK; the trained model checkpoint and the train/test design split; and an MIT/Apache license. State plainly which numbers are turnkey and which need the slow full-suite run on a large machine.

Calibration

• Do not assume a DAC artifact badge exists; verify the current cycle before promising one.
• If DAC or a co-located workshop does introduce an artifact/open-source track in a given year, re-read its specific rules — do not assume ACM SIGSOFT-style badge names transfer.
• The payoff is credibility and citations; budget the artifact accordingly, not as a graded deliverable.

Output format

[Artifact goal]   reviewer credibility / community reuse / contest harness
[Contents]        tool + flow versions / benchmarks + version / PDK / model + split / license
[Ten-minute test] does install + small demo succeed on a clean machine? yes/no
[Claim mapping]   claim -> script -> benchmark -> expected QoR present? yes/no
[Anon vs public]  anonymized review artifact / DOI-archived public release planned? yes/no
[Badge check]     does this cycle actually run an artifact track? verify -> yes/no/待核实