Sensys reproducibility

SenSys Reproducibility

A SenSys result is reproducible when someone on different hardware can understand — and, from your traces, re-obtain — your numbers. The threat is not messy code; it is provenance that evaporates when the deployment ends. Energy measured without its method, accuracy scored against forgotten ground truth, and firmware whose toolchain is unrecorded cannot be reproduced at any price once the testbed is torn down. Capture it while the nodes are still powered.

Capture it live, not later

Provenance	Capture while live	Why it cannot be reconstructed
Energy method	Instrument model, sampling rate, integration boundaries, sleep floor	A stored current number loses its method; "42 µA" of what phases?
Hardware	MCU/SoC part + clock, sensor config, radio + TX power, board revision, battery/harvester spec	A later "same board" is rarely bit-identical
Firmware	Sources + toolchain version + compiler flags, pinned	A rebuild on a new toolchain shifts timing and energy
Ground truth	Reference protocol + the reference's own error	Labels without their protocol are unfalsifiable
Deployment	Placement, environment, duration, node uptime/failures	Conditions are gone once the deployment ends
Harvest input	Energy-source trace (light/RF/vibration), buffer sizing	Behavior across brownouts is unreproducible without the input

The reproducibility that matters is cross-hardware

Reproducing your own result on your own testbed proves little. The SenSys bar is that a different lab can interpret a mismatch: when their number differs from yours, the provenance tells them why (different MCU clock, different harvest input) rather than leaving them to guess. Ship recorded traces so the analysis can be re-run even by someone who lacks your hardware — this is what turns a deployment result into a reproducible one and underpins the trace-replay path in sensys-artifact-evaluation.

Minimum reproducible bundle for a SenSys figure:
  traces/fig4_power.csv        # raw power samples + timestamps
  traces/fig4_sensor.csv       # the sensor stream behind the same figure
  analysis/reproduce_fig4.py   # regenerates Fig. 4 from the two traces
  ENERGY.md                    # instrument, rate, integration boundaries
  HARDWARE.md                  # part numbers, clocks, board rev, harvester spec
  firmware/ + toolchain.lock   # pinned build that produced the traces

Decide early what can ship

Some SenSys artifacts carry release constraints that a purely computational paper never faces:

• Sensor data of people or spaces may need consent, anonymization, or aggregation before it can be released — decide at collection time, because retroactive consent is usually impossible.
• Firmware and hardware designs may touch third-party IP or a vendor NDA; confirm what is releasable before promising an Available badge.
• Deployment locations can be sensitive (critical infrastructure, private property); scrub identifying detail from released traces.

Resolving these late forces a choice between a weak artifact and a broken promise. Resolve them in the experiment plan (sensys-experiments).

Reproducibility checklist

[ ] Energy method (instrument, rate, boundaries) recorded, not just the number.
[ ] Hardware provenance: parts, clocks, board revision, battery/harvester spec.
[ ] Firmware sources + pinned toolchain + compiler flags archived.
[ ] Ground-truth protocol and the reference's own error documented.
[ ] Deployment conditions + honest uptime/failure log captured while live.
[ ] Harvest-input traces + buffer sizing stored for batteryless results.
[ ] Recorded traces shipped so figures re-generate without your hardware.
[ ] Release constraints (consent, NDA, location) resolved at collection time.

Output format

[Live-capture] which provenance is captured vs. still at risk while the testbed runs
[Cross-HW]     can a different lab interpret a mismatch from what you shipped? Y/N
[Traces]       do shipped traces regenerate the headline figures without hardware? Y/N
[Release]      consent/NDA/location constraints resolved? open items
[Open]         the provenance whose loss would most damage reproducibility