Skip to Content
ResearchReproducibilityReproducibility

Reproducibility

Every 2026-onward paper lands with seeds, wheel SHAs, code hashes, and the exact experiment configs so a reviewer (or a future us) can rerun from scratch and get the same numbers back.

All runs are driven by Maneuver.Map’s /experiments endpoint. Each experiment writes an experiment.json with a provenance block that captures git state, installed-package hash, and hardware fingerprint. See the provenance module source.

Per-paper runbooks

  • ICCD — reproducing the CRDT + DTN million-agent coherence claims.
  • DMB + TF-ACO — reproducing the phase-transition sweeps at 500k agents.
  • HMA — reproducing the hierarchical market auction numbers for lunar construction.

The reproducibility contract

For each paper we commit to:

  1. Seeds. The seed tree is published: experiment-level, per- generation, per-candidate, per-repeat. Rerunning with the same leaves yields byte-identical final_swarm_metrics.
  2. Wheel SHAs. The exact Gossamer and Leviathan wheels that produced the numbers, pinned in the appendix.
  3. Configs. JSON bodies that were POSTed to /experiments. Copy and paste into a local Maneuver.Map to re-run.
  4. Provenance. git describe, pip freeze hash, CPU count, and CUDA visibility at run time.
  5. Artifacts. Expected experiment.json, final figures, and summary CSVs produced by the run, archived at a stable URL.

When a future paper refines a baseline, the old runbook keeps working; we version the configs rather than overwriting them.

Last updated on