Reproducibility and Data Availability
Arboria’s research tools are proprietary. That constrains what reproducibility can mean here, and we would rather state the constraint plainly than publish instructions nobody outside the lab can execute. This page is the honest version: what we pin, what each paper records, what we will share on request, and what becomes public.
What every run records
Experiments are driven through Maneuver.Map’s
experiment API. Each cell of each sweep writes an experiment.json
carrying a provenance block captured at run time:
Package versions. The resolved versions of the two libraries that determine behaviour — Gossamer (coordination primitives, tasks, metrics) and the Leviathan binding (physics) — plus NumPy, SciPy, pandas, and PyArrow.
An environment lockfile hash. A SHA256 over every installed
name==version pair in the worker environment. This is the fingerprint
that actually identifies a run. It is cheaper to record than a full
lockfile and changes the moment any dependency moves.
A hardware fingerprint. Platform, Python version, machine architecture, CPU count, and whether a CUDA device was visible — enough to attribute wall-clock numbers.
The full seed tree. Every stochastic call site draws from an explicit
generator seeded through a deterministic tree
(exp_seed → gen_seed → candidate_seed → repeat_seed). Two runs at the
same leaves produce identical metrics.
Git state, when a checkout exists. The provenance block has fields for
describe, commit, branch, and a dirty flag. Under our production
worker these are null: the engine runs in-process inside a prebuilt
container that carries no .git directory. We do not paper over this —
the lockfile hash and the pinned image digest are the reproducible
identity there, not a commit hash.
Note what is not in the block: we do not compute wheel SHA256 digests at run time. Where a paper pins one, it comes from the release artifact, not from provenance capture.
Canonical run identities
Each published result names the exact batch it derives from. Batches are immutable once written; a refined analysis gets a new batch rather than overwriting an old one.
| Result | Batch | Cells |
|---|---|---|
| Phase Diagram of Coordination Under Delay — main grid | dcc_p1-20260707T225301-299e08d9 | 900 |
| Delay Phase Diagram — Vicsek anchor | vicsek-20260707T171708-2084915b | 45 |
| Delay Phase Diagram — intrinsic-timescale scaling (§5.4) | dcc_p4a_fine-20260711T004604-b3aae702 | 520 |
| Anticipatory Coordination via Peer-State Prediction | dcc_p3-20260708T130306-5d8c150d | 200 |
| Freshness vs Throughput under Lossy Communication | dcc_p5-20260710T233935-7fdfc793 | 320 |
| The Coordination Cliff Scales with the Task Timescale — main grid | dcc_p4b-20260710T233740-c926ca7c | 720 |
| Task-Timescale Ratio — falsification control | dcc_p4b_control-20260710T195844-927907d4 | 80 |
| Energy-Aware Hierarchical Markets — uncontended control | hma_paper-20260710T214442-b5d0aae7 | 15 |
| Hierarchical Markets — capacity sweep (Table 1b) | hma_load-20260710T214629-1f5bed81 | 75 |
| Hierarchical Markets — hauler-routing control (see note) | hma_steering-20260712T003454-2282f97b | 30 |
| Hierarchical Markets — routing × capacity (see note) | hma_load_steering-20260712T003456-a1012f93 | 150 |
A note on the two HMA steering batches. They re-run the published configuration under
both hauler routings on the current model (Gossamer 0.7.0). They are recorded here because
they establish something the published batches cannot: the HMA contended-regime figures do
not reproduce across the 0.6.0 → 0.7.0 model change. The runs are individually
deterministic — five physically-identical replicas return identical output to the kilogram —
so this is a change in the modelled dynamics, not simulator noise, and it traces to a
floating-point tie-break in the micro→depot assignment that cascades over 8,000 steps. The
HMA paper’s Table 1b is therefore pinned to 0.6.0, and its magnitudes are scoped
accordingly; see its Appendix F. The DCC results are unaffected — every one of their figures
was regenerated from its committed batch and reproduces its published numbers.
Every figure in every paper is generated by maneuver-map/scripts/figures.py, which declares
in code the batch and metric each figure derives from. python scripts/figures.py --all --check
re-derives each figure’s series from GCS and diffs it against the committed sidecar, so a
figure cannot silently drift from the data behind it.
The original phase-diagram and anticipation grids ran against Gossamer 0.3.1
(wheel SHA256 b28a53354f1800a29bfe880ef5a2bcad0acfc29724d916cf263677e10a22bcfb)
and the Leviathan base image leviathan-base:py-0.2.1
(digest sha256:1ed753f5de8d9ea08b3325c1bc4c954ce6064fdc00e06c303acb1bae508f6b64).
The later grids — the intrinsic-timescale scaling (dcc_p4a_fine) and the
freshness-vs-throughput sweep (dcc_p5, which needs the rebuilt base carrying the
delivered-edge communication model) — ran against Gossamer 0.6.0; the scaling
grid’s shared cells reproduce the 0.3.1 phase-diagram numbers to ~10⁻¹⁰,
confirming the coordination seam is unchanged across that span. All ran on
Python 3.10.12 under Linux x86_64 (glibc 2.35). The grid definitions,
fixed parameters, and expected outcomes are stated in each paper’s
reproducibility appendix rather than duplicated here.
What we share
Configuration and results, on request. The experiment specifications,
per-cell experiment.json files, and aggregated summary tables behind any
published figure are available to reviewers, programme committees, and
collaborators. Write to
research@arborialabs.com naming the
batch.
Not the engine. Leviathan, Gossamer, and Maneuver.Map are proprietary and are not distributed. The toolchain documentation describes their interfaces, configuration surfaces, and the models they implement in enough detail to reimplement the experiment against another simulator; the Foundations and Techniques pages cover the underlying algorithms, which are drawn from the open literature.
What becomes public
The constellation benchmark now under construction is intended as a neutral, versioned, publicly runnable suite — scenarios, baselines, fault models, and a leaderboard harness that anyone can execute without access to our internal stack. That release is where external reproduction stops being conditional. Until it lands, treat the numbers in our papers the way you would treat any custom-simulator result: scoped to the fidelity of the simulator, anchored where possible against a known external result (the Vicsek order–disorder decay, in the case of the delay papers), and open to challenge.
Corrections
If a published number does not survive re-examination we will say so on the paper page and in updates, with the batch that superseded it. Silent edits to results are not something we do.