Commit Graph

5 Commits

Author SHA1 Message Date
Ludovic Marie a4355c7c9b Catch flaky harnesses: --repeat, and three runs in the gate
parity.py --repeat N runs each harness N times; its runs must agree, else it
is reported FLAKY and recorded as failing. Later rounds run on the machine
that ran the first, reusing its oracle results, and never two at once there.
With --flake-check REF (passed by gate.py, merge base with parity-base; by
supervise.py integrate, the integration branch before the merge), harnesses
whose script, oracle or probe is new or changed must pass three runs in a
row; a pass records its rounds so it is not repeated again. gate.py matches
shards of a known failure by the harness's name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 17:10:08 +02:00
Ludovic Marie a05f9b7b8c Run harnesses on the Mac mini by default
Every harness gave the same result on the Mac mini as on Linux, except
options and chunk_generation, where Rust's float functions use the platform
libm and round differently on macOS: those stay off the Mac. gate.py and
supervise.py no longer force 8 local harnesses, so parity.py keeps the
desktop to 3. The ThinkPad is listed (2 harnesses, nice 19) but not used by
default. Oracle runs get a 15-minute hang guard instead of 5, since one oracle
on a fully loaded machine can take more than 5 minutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 16:42:07 +02:00
Ludovic Marie e8f2ccd403 Fail the gate when the harness runner dies, and clear large harness outputs
tools/gate.py took a crashed tools/parity.py for a pass with no new
failures. The runner now records each completed run, and the gate fails
unless its own run finished. A missing or empty trace (the disk was full)
no longer crashes the runner.

After a harness passes, the runner deletes its large regenerated outputs
(64 MB or more), keeping the vanilla oracle outputs that reusing an oracle
run needs. Full suites left 2-4 GB files in every worktree and filled the
disk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 18:56:28 +02:00
Ludovic Marie abee1f165b Let the gate pass with harnesses that already fail on the integration branch
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 15:05:57 +02:00
Ludovic Marie 6396e8c161 Run the vanilla harnesses in parallel, only when their inputs change
tools/parity.py traces each harness (its Python, Java oracle sources and
the Rust sources of the probes it runs, from Cargo's dep-info) and reruns
it only when one of those, the vanilla artifacts or the toolchains change.
Harnesses run in parallel within the available memory, and their probes
build without LTO and with 16 codegen units into .local/parity-target: a
one-line change rebuilds every horizon-world probe in 3 seconds instead of
122. The full suite takes about 20 minutes instead of an hour.

tools/gate.py is the check a branch passes before integration, and
tools/supervise.py manages the per-area worktrees: status, new worktrees,
saving unfinished work, and integrating a branch only when the gate passes.

Harnesses find probes through lab.cargo_target() instead of a hard-coded
target/, the save round-trip harness builds the binaries it launches, and
the mob AI harness now fails on divergences like every other harness
(--report-only keeps the old behaviour). Debug builds keep line tables
only, so worktrees stay small.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 15:05:36 +02:00