Orchestrator's review of #2859: a nightly can hold pull requests nobody
reviewed yet, and their build scripts run in `build`. game-build.yml gets a
`trusted` input, true only from release.yml:
- the Rust cache (GitHub-hosted Linux and Windows): its own key for an
untrusted build (`nightly-<name>`), never saved; a release restores only
what a release saved;
- macOS: an untrusted build's exporters and build jobs run on the runner
that already runs pull requests' parity jobs (macOS + horizon-game), never
the release runner (MAC_RUNNER) nor parity-main's;
- a trusted build doesn't read the machine's shared sccache, which pull
requests' jobs write (runner-env.sh: HORIZON_NO_SCCACHE).
tests/test_nightly.py pins all three.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ludovic's decision (2026-10-27, from kacpherek's proposal under #2779): the
Nightly is where contributors test pull requests before the gate.
kacpherek, rdbtCVS and levzzz5154 (.github/nightly-reviewers.txt) and
Ludovic pick them with `/nightly 2640 2642`, `add`, `remove`, `clear`,
`status` or `dry-run` in any issue or pull request.
- nightly-command.yml (issue_comment, main's copy): checks who asked, saves
the set on nightly-state, starts `nightly build`, answers "🤖 Nightly: …".
It never checks out or merges a pull request.
- nightly-build.yml: `pick` builds main + the set (newest heads and merged or
closed ones dropped at the scheduled run), pushes it as
nightly/build-<date>-<n>, records it in builds.jsonl; build, sign and
publish are unchanged; `answer` tells the thread how it ended.
- Notes say a Nightly can include pull requests not reviewed yet, picked by
trusted reviewers, and list each with its author and who picked it.
- docs/release-flow.md: who can pick, how, how Ludovic removes or pauses them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dry run publishes nothing, so it can't race a real release; in the shared game-release group a release queued
behind it replaced it as the pending run and GitHub cancelled it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nightly-build.yml builds, once a night, the newest union/<agent>-<name> branch on origin when it is newer than
main and holds something main lacks (else main), and publishes it as the `nightly` pre-release; the previous one
stays a week as nightly-<date> (tools/ci/nightly.py pick/rotate/prune). Launchers ignore it until a player turns
Nightly on (next PR).
The release builds move unchanged into game-build.yml, a reusable workflow both release.yml and the nightly call
from main's copy. The nightly builds with no secrets and the macOS keychain locked; only the sign job holds the
key, and it runs nothing but main's sign_release.sh over the archives, for the message "horizon release nightly".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The five mod_content tests that load the whole content root took minutes of
every `cargo test -p horizon-client --lib` on prepared data. They are now
#[ignore]d; pr_ready runs them when the diff touches what they load, and CI's
verify job runs them filtered with --ignored on its existing build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The maintainer asked that CI stop using his desktop (2026-10-26). playhorizon's
runner now carries horizon-game and the desktop's runner none of the job labels;
full-parity jobs go to playhorizon only, with a 6 h timeout on its 4 CPUs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A crate split reaches every harness (#2160: 352 harnesses, 71 min on a Mac
mini runner) and held the Mac mini's runners while other pull requests'
parity jobs queued for over an hour. With the full-parity label, the job
runs on the desktop's runner only (8 at a time) and waits for it while the
desktop is in use; its timeout is 300 min, since a job under way when
someone comes back goes on with 2 threads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A head tested alone keeps failing harnesses that main already fixed. Since 10-25 evening, 9 of 21 failed PR
parity jobs failed only on such fixes: server_knockback (heads without #2114, 7 times), mob_ai's oracle heap
(without #2270), save_roundtrip (without #2212). #2160 needed 4 parity runs this way.
The job now merges main's current tip into the head and selects harnesses against that main commit, so only
the pull request's own changes count. A branch that doesn't change the runner scripts or the selection gets
main's, which replaces the two steps that copied main's vanilla-data.sh and parity.py in by hand. A head that
conflicts with main is tested alone, with a warning. The job summary names the main commit, and the check is
still `parity / parity` on the head.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
verify's cargo runs never build launcher/ (its own workspace), so #2059 and the
registry ratings commit each passed alone and left main's launcher failing to
compile (E0061). The new launcher job runs its rustfmt, clippy -D warnings and lib
tests when a pull request changes the launcher, a crate it builds or the root
manifests, and on every push to main.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Twice today (runs 40301239508, 40311712302) mob_ai failed in parity-main, passed when bisect ran it again at
the same HEAD, and the run still ended red, so no game release could follow main for three hours. The
harness step now continues on error and the bisect step gives the verdict: 0 when every failed harness
passes again at HEAD, 1 when they fail at the base too or a merge is named.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GitHub started both 10-26 schedules over an hour late, so the hour check skipped both and no audit ran.
lan_integration takes ~2,400 s under load, past the 1,800 s per-harness limit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🤖 Agent: merged main in. The old head's parity failure was server_dimension_players, which passes on main's parity-main. tests/test_parity_reach.py OK.
ludovics-mac-mini-2 holds only the horizon-parity-main label; the release
runner no longer has it, since a release waited behind each new
parity-main run there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🤖 Agent: verify passed at 0c84eb307. Merged by hand: the PR parity job is queued behind the same runners, and this changes only how that job prepares its data. The next PR parity job on ludovics-mac-mini is the real test.
A PR head older than the fix in vanilla-data.sh left a real
.local/horizon-client without sounds.json on runner ludovics-mac-mini;
every later job there failed subtitle. Like the harness selection, the
job now runs main's script unless the branch changes it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The queued parity-main run waited 2 h behind PR parity jobs on the two
horizon-game runners, each merge replacing the pending one, and releases
need it green. The release runner gets a horizon-parity-main label; a
release only starts after parity-main passed, so the two never compete.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The reach parse cache lives in the shared parity folder, keyed by content,
so a new worktree starts warm (written atomically, other worktrees'
entries kept within bounds).
- A harness's reach search stops as soon as it reaches a changed file that
differs in more than its `mod` declarations; partial searches are not
memoized. Declaration diffs are memoized (one git show per file).
- A harness whose changed dependency is outside crates/ is picked without a
search (narrowing always keeps those).
- After PARITY_REACH_BUDGET seconds (default 180) the remaining harnesses
count every source their probes link.
- crate_of is cached; no pathlib in the hot paths.
Same 79 harnesses on the 19-file diff from contrib-gate-16; CPU 19.9 s cold /
16.6 s warm before, 13.1 s cold / 9.9 s warm after (Mac mini under load).
parity.yml: skip the prepare_output check for heads made before #1947 (no
OUTPUT_VERSION, no tool): the job runs main's workflow on the head's files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nix-build names a dev output's GC root <name>-dev, so the runner's PKG_CONFIG_PATH pointed at links
that didn't exist and every client build failed on libudev; it now uses the store path nix-build
prints. The package temperature spikes for a second under load, so the heat check takes the median
of 5 readings. Restarting the watcher no longer throttles a running job.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🤖 Agent: verified: OUTPUT_VERSION = 1 and output-versions.txt carry the digest measured on 10 CI preparations. tools/prepare_output.py check passes on v0.2.64's preparation and fails on v0.2.47's. This head's own parity job prepares with its preparer and checks the digest. Merging.
Disk: clear_large_outputs only looked at .local directories named after the
harness, so the 174 harnesses writing to rust-<name>-parity kept every dump
(86 GB in one runner checkout, 13 GB for state_provider alone). It now
clears those too, keeps vanilla outputs only up to 256 MB (larger ones cost
more disk than rerunning their oracle), and handles shard names.
collision, world and geometry dropped one mkdtemp run directory per run
(11 GB): each run now removes the earlier ones.
Time: harnesses inherited the runner's stdin, a pipe on GitHub's runner, and
parity_shim never reuses an oracle run whose stdin is a pipe, so oracle reuse
never worked on CI. Harnesses now get /dev/null.
CI: parity.yml falls back to the head's own selection when the names main's
`--list` printed don't match its summary count.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The clean checkout removes target/, and vanilla-data.sh builds horizon-prepare only when it
prepares afresh, so compile-exporters had no binary on a cached preparation. The bundle is
also compiled in the job instead of reused from .local/exporters, which vanilla-data.sh links
to preparations that pull requests' parity jobs write, and the job fails if anything but our
classes and the manifest would leave the runner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
parity-main runs the vanilla harnesses on every push to main, one run at a
time: merges that land during a run wait in one pending run, which checks
every merge since main's last green parity-main (tools/parity_main.py base,
then parity.py --changed). A failure is bisected over the batch's
first-parent merges, running only the failing harnesses, and the breaking
merge's pull request gets a comment.
Game releases ship only a commit both checks and parity-main passed on
(release_train.py; release.yml also starts when parity-main finishes; a
forced release doesn't wait). The per-PR parity job now runs only for a PR
that changes a harness, an oracle, a probe or the runner, or carries
run-parity.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The parity job checks out the PR head, so every branch made before #1964
still selected with the old per-crate rule. The step now runs main's
tools/parity.py and tools/parity_reach.py with --changed --list, restores the
head's files, and runs the named harnesses with the head's parity.py. A
branch that edits either file, or a failure of main's selection, falls back
to the head's own `--changed` run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Compile and validate original exporter bundles during release builds, cache development compilation, and reuse initialized JVMs and collision data during preparation. Keep legacy preparers compatible while current releases need only a Java runtime.
Co-Authored-By: Codex <noreply@openai.com>
runner_pause.py pauses on any seat0 session (locked too), games, GPU use or a pause file, and is
the only thing that starts the runner; a job under way is throttled, then the runner stops. The
desktop host in parity_hosts.json refuses work while paused and its workers stop on a pause. The
nightly audit stays on the Mac mini. Docs and a unit test of the watcher.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Launchers copy a player's prepared resources to a new release while its preparer declares the
same OUTPUT_VERSION (the launcher side is its own pull request). The number is bumped by hand;
CI keeps it honest: the parity job already prepares with every change to the preparer, and
tools/prepare_output.py check now fails when that preparation's digest is not the one
crates/horizon-prepare/output-versions.txt gives the current number, or when the file changed
other than by appending a version. The digest canonicalises what differs between two runs of
the same preparer (JSON key order, creative tab tag order, lambda class addresses, logs, the
sounds): 10 preparations by 5 preparer versions on the three Mac runners give one digest, and a Linux preparation by an older preparer gives the Mac runners' digest for it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GitHub's "Latest release" is the newest game release, so new players download the game's archive, which
does not start on its own (#1647). Making a launcher release the latest instead would break every
installed horizon-server, install.sh and install.ps1: they ask for releases/latest and expect the game's
archives, and a server that can't update can't fetch a fix either. So the game stays the latest and says
where to go:
- The notes open with "Want to play? Download the Horizon launcher, not the files below", linking
playhorizon.dev and the launcher releases. It is raw HTML on one line, which GitHub renders and the
launcher's notes view and one-line summary skip (pulldown-cmark HTML events), so launchers already
installed don't tell players to download themselves. The game footer that said the same at the bottom
goes. The site's Releases page leaves the block out.
- release.yml gives each archive a label ("Game files the launcher installs: Linux (...)", "Dedicated
server for Linux (...)"), which GitHub shows instead of the file name. Everything that downloads
matches the asset's name, which is unchanged.
Fixes#1647
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Its own workspace, as the telemetry server's, for its own Railway service.
GitHub sign-in with no scope (the user's token reads the login once and is
dropped), signed-cookie sessions, an origin check and a CSRF token, six
submissions per account and sixty in all per hour. A submission runs the
file's own checks (horizon_mods::index::inspect, the signature, the entry's
fields through registry::project), uploads it to the registry's `staging`
release as <sha256>.<extension>, writes mods/<id>.toml on a new branch with
the submitter as co-author, and opens the pull request. Only an entry's
owners publish updates; a key change is a pull request by hand. Shader packs
are refused until their format exists.
Tested end to end against a mock GitHub: sign-in, the signed sample mod,
what lands on GitHub, and what is refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Preserve Java parity oracles and comparison benchmark instrumentation. Move their remaining dependency locks under tools/benchmark-deps and remove legacy product packaging, fixtures, and CI checks.
Co-Authored-By: OpenAI Codex gpt-6.1-sol <noreply@openai.com>
One release per merge made about 40 releases in 31 hours, hard for players to follow (asked on Discord).
release-now still releases at once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
horizon-mods' runtime is no longer a default feature, and its sandbox test
requires runtime and pack instead of a dev-dependency on itself (that
dev-dependency turned the runtime on in every workspace build with examples,
the parity probe build included). The client gets a mods feature that turns
the runtime on; without it game/mods.rs keeps content mods and logs why code
mods don't start, and every hook does nothing.
Release builds (client and server) pass the feature, CI's clippy and tests
pass it (and pack, for the sandbox test), and pr_ready does where horizon-mods
is checked. horizon_mods::find moves to manifest.rs: content mods use it
without the runtime.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cargo test --workspace built and linked all ~275 examples, 59% of the
test build (review-plan section 1), though none of their tests run.
Test the libraries, binaries and integration tests, then the doc tests,
as tools/gate.py and tools/pr_ready.py already do. Clippy --all-targets
still type-checks every example, and the parity job builds and runs them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ludovic's decision (2026-10-23): no more Alpha and Stable. release.yml
publishes every game release as the release and GitHub's latest instead of
a pre-release that promote.yml made stable once a day. The checks that
gated a promotion run in the release itself: `decide` holds the train while
an issue labelled release-blocker is open (forced runs too), and the publish
job runs `release_train.py gate` right before publishing: the Mac mini's
smoke-test.json must be there and show a world was drawn, and no blocker may
be open. Otherwise nothing is published.
Notes cover everything since the last release that is not a pre-release, so
the first single-channel release lists all of it since v0.2.0. promote.yml,
tools/release_channels.py and its tests are deleted. The cadence is
unchanged (5 minutes of settling, 20 at most, release-now at once).
Docs and site: one channel in release-flow, versioning, security, the FAQ,
install, the home page and the Releases page (Game / Launcher filter); no
player-facing Alpha wording left.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🤖 Agent: merged after verify passed. telemetry.playhorizon.dev is live on Railway (service telemetry in horizon-site, volume /data): /healthz, /v1/live (a test ping counted) and /v1/benchmarks/summary (CORS for playhorizon.dev) answer. The PR's local end-to-end runs are in the description.
- DEFAULT_ENDPOINT is https://telemetry.playhorizon.dev. The heartbeat, the launcher's live count and the
outbox upload all go there; HORIZON_TELEMETRY_URL still names another server, and HORIZON_TELEMETRY_URL=off
sends nothing at all. CI workflows and the harnesses that start the game or the launcher set off. Consent
is unchanged: the four switches stay opt-in, only the anonymous ping is always on.
- GET /v1/benchmarks/summary: players' benchmark uploads grouped by GPU, CPU, render distance and game, groups
of 3+ installs only, medians per install then across installs, no install ID, day, OS version, driver,
memory or screen. Cached 10 minutes, CORS for the website. Same rules as website/scripts/benchmarks.mjs,
tested on the same fixtures; a deleted install leaves its groups at once.
- /v1/aggregate no longer lists the newest 50 raw benchmark reports (hardware, driver, day, one install each):
uploads are published only through the summary.
- Behind Railway's proxy the rate limit counts X-Real-IP, else the last X-Forwarded-For entry (the one the
proxy adds; a client could choose the first). Still in memory only, never stored or logged.
- crates/horizon-telemetry-server/railway.json: Dockerfile build from the repository root, health check
/healthz, one replica in europe-west4 (one SQLite file on the /data volume).
- Docs: docs/telemetry.md, the server README (the summary's shape, Railway setup), the privacy page (the ping
is on, where the data lives, how benchmark uploads are shown).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop the Horizon menu style (options_horizon.rs, front.rs, the live title
backdrop and its post grade, the launcher's menu-style setting, CLI and MCP
tool). Every screen is vanilla's. Horizon's own settings hang off vanilla's
Options the way mods add theirs: Horizon Settings... and Mods... take the
Telemetry/Credits slots, and Horizon Settings opens Graphics, Advanced
Graphics, Shader Packs, Controller Settings, Extras and Mods, each a vanilla
options list with tooltips. Play with friends moves behind vanilla's Friends
buttons: the pause menu's opens Play with Friends (share, copy, stop), the
title screen's opens Join a Friend, and Direct Connection takes a full code.
Old files load as before: horizon.txt's menuStyle line and
horizon-graphics.txt's level line are ignored.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The whole of Horizon is in beta, so the channel that installs every build as
soon as it is built is called Alpha; Stable keeps its name. The launcher's
Settings, the update-channel command, the MCP tool, the release notes footer
and the docs say Alpha. update_channel is saved as "alpha"; "nightly" and the
old beta_builds switch still read as Alpha.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The daily promotion moves to 20:00 UTC (21:00 in Paris in winter time):
PROMOTE_AT and promote.yml's cron (20:07, retries until 21:37), with the docs
that give the time. A launcher left open on Nightly looks for new releases
every 30 minutes; Stable keeps 4 hours. The launcher README's Settings
screenshot shows the Updates choice.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The launcher's "Beta builds" switch becomes an Updates choice of two channels,
stored as "update_channel": "stable" | "nightly" (an old "beta_builds": true
reads as nightly). The CLI (update-channel) and MCP (update_channel) show and
set it, and list each release's channel as stable or nightly.
Nightly ships constantly: the release train waits 5 minutes for a merge wave to
settle (20 at most) instead of 15 (an hour). Stable moves once a day:
promote.yml runs at 19:00 UTC (retries until 20:37) and promotes the newest beta
that had been out 6 hours by 19:00, so late runs and retries pick the same one.
The stalled-promotion issue waits 30 hours. Release notes, docs, the website's
install and FAQ pages say Stable and Nightly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🤖 Agent: merged after integration wave 1 on the desktop (main e730dc14 + #1108#1114#1131#1157#1158#1162 = ae482a72): fmt, clippy -D warnings, workspace tests (one port-race flake in horizon-server panel, passes on rerun), release build, tools/parity.py --changed origin/main 299/302 then the 3 rerun: save_roundtrip pass, world_reopen pass x2 (one early-count flake), dedicated_server = known flake also failing on main (fixed by #1183/#1184).
Ludovic Marie
rdbtCVS