Ludovic Marie
d061bdf2fb
A pass is keyed by the content of the harness's dependencies, so it is also valid in any worktree with the same files. Passes are published to parity-shared/ beside the shared vanilla data, and a worktree adopts one whose key matches its own files instead of running the harness: an agent's gate only runs what its own changes affect, instead of the whole suite from scratch. Every runner on the machine also shares PARITY_MACHINE_JOBS slots, so several agents' gates can't overload it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
51 lines
5.5 KiB
Markdown
51 lines
5.5 KiB
Markdown
# Parity workflow
|
|
|
|
How parity work is run, checked and merged. The harnesses themselves are described in [Rust rewrite status](rust-parity.md).
|
|
|
|
## Running the vanilla harnesses
|
|
|
|
`tools/parity.py` runs the `tools/test_*_parity.py` harnesses in parallel and reruns only what a change can affect:
|
|
|
|
```sh
|
|
python3 tools/parity.py # harnesses whose inputs changed, or that never passed
|
|
python3 tools/parity.py food light # these harnesses, whatever their state
|
|
python3 tools/parity.py --failed # the ones that failed last time
|
|
python3 tools/parity.py --all # everything, from scratch: the audit (about 20 minutes on ludodesktop)
|
|
python3 tools/parity.py --fresh food # don't reuse vanilla oracle results
|
|
python3 tools/parity.py --list # what would run
|
|
python3 tools/parity.py --status # last result of each harness
|
|
```
|
|
|
|
Each harness runs unmodified under a tracer that records the files it depended on: its Python modules, its Java oracle and the oracle sources javac pulled in, other repository files it read, and the Rust sources of the probes it ran (from Cargo's `.d` dep-info files). A pass stands until one of those files, the pinned vanilla artifacts, `Cargo.lock` or the Rust/Java toolchains change. State and logs are in `.local/parity-runner/`; each harness's own outputs stay in its `.local/<name>/` directory.
|
|
|
|
Vanilla oracle runs are reused as well. `tools/parity_shim.py` stands in for `java` and `javac`: it records each oracle run's outputs and the content of every file under the paths its arguments name, and replays the run while those are unchanged. After a Rust-only change a harness pays only for its Rust side (`block_update`: 53 s to 5 s). Files an oracle writes outside its named paths aren't tracked, so a harness that fails while reusing oracle results is rerun fresh, and stops reusing them if it then passes. `--all` and `--fresh` never reuse them.
|
|
|
|
Probes build into `.local/parity-target/` with release optimisation but no LTO and 16 codegen units. On ludodesktop that rebuilds every `horizon-world` probe in 3 seconds after a one-line change, against 122 seconds with the release profile. Rust gives these settings identical results (no fast-math; overflow checks as in release), and the full harness suite passes under them. Probes that harnesses build in the dev profile get `opt-level = 3` too, keeping their overflow checks and debug assertions. Harnesses must locate binaries through `cargo metadata` or `lab.cargo_target()`, never a hard-coded `target/`.
|
|
|
|
Passes are shared between worktrees. A harness's key depends only on the content of its dependencies, not on where they are, so every pass is also written to `parity-shared/` beside the shared vanilla data (`PARITY_SHARED` overrides it). A worktree whose files match those a harness passed with, in any worktree, counts that pass without running the harness: after merging the integration branch, an agent's gate only runs the harnesses its own changes affect.
|
|
|
|
Every runner on the machine shares `PARITY_MACHINE_JOBS` slots (default: half the CPU threads), so several agents' runs together can't overload it. Parallel harnesses are also admitted by memory: each harness's recorded peak is reserved, within three quarters of the memory available at the start. `-j` caps the count (default: a third of the CPU threads).
|
|
|
|
## The gate
|
|
|
|
`tools/gate.py` is what a branch must pass before it is integrated: the public-tree check, rustfmt, clippy with warnings denied, the Rust and Python tests, then the harnesses whose inputs changed. It stops at the first failure and writes `.local/gate.json`. `--known-failures a,b` lets harnesses that already fail on the integration branch keep failing. `--quick` checks formatting, lints and Python tests only; `--no-parity` skips the harnesses.
|
|
|
|
## Parallel work: worktrees and the integration branch
|
|
|
|
Parity work happens in one worktree and branch per area, next to an integration worktree that has the integration branch (`parity-base`, pushed as `t3/achieve-full-parity`) checked out. All worktrees share the vanilla data through `.local/` links to a `shared-local/` directory beside them.
|
|
|
|
```sh
|
|
python3 tools/supervise.py status # branches, progress, unpushed work, last gate
|
|
python3 tools/supervise.py new dispensers # new worktree and branch from the integration branch
|
|
python3 tools/supervise.py save # commit and push unfinished work before agents stop
|
|
python3 tools/supervise.py integrate parity/dispensers # merge, gate, push; undo the merge if the gate fails
|
|
```
|
|
|
|
An agent working an area merges the integration branch first, uses `tools/parity.py` on its harnesses while it works, runs `tools/gate.py` before its last commit and pushes its branch. Integration happens only through `supervise.py integrate`, which runs the gate with the harnesses already failing on the integration branch as known failures: a branch may leave those failing, or fix them, but must not make any other harness fail.
|
|
|
|
On ludodesktop, source `~/code/horizon-parity/env.sh` first: it sets the ALSA build path and the Cargo job cap. Debug builds carry line tables only (`[profile.dev]` in `Cargo.toml`), which keeps backtraces readable while keeping each worktree's build small; a fresh worktree builds `horizon-client` and its tests in under a minute.
|
|
|
|
## Reading vanilla
|
|
|
|
Besides the Vineflower sources in `.local/minecraft/<version>/sources/`, `tools/rea_vanilla.py` queries the shipped bytecode through REA (callers through lambdas, overloads, a second decompiler); see [decompilation status](decompilation-status.md#querying-the-shipped-bytecode-with-rea).
|