Commit Graph

11 Commits

Author SHA1 Message Date
Ludovic Marie 886ecd9b20 Merge remote-tracking branch 'origin/main' into perf/reach-planning 2026-10-26 06:13:02 +01:00
Ludovic Marie 0c84eb3079 ci: PR parity jobs prepare vanilla data with main's script
A PR head older than the fix in vanilla-data.sh left a real
.local/horizon-client without sounds.json on runner ludovics-mac-mini;
every later job there failed subtitle. Like the harness selection, the
job now runs main's script unless the branch changes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 03:18:39 +01:00
Ludovic Marie 7b0192118e parity: choose harnesses faster (shared parse cache, early stop, time budget)
- The reach parse cache lives in the shared parity folder, keyed by content,
  so a new worktree starts warm (written atomically, other worktrees'
  entries kept within bounds).
- A harness's reach search stops as soon as it reaches a changed file that
  differs in more than its `mod` declarations; partial searches are not
  memoized. Declaration diffs are memoized (one git show per file).
- A harness whose changed dependency is outside crates/ is picked without a
  search (narrowing always keeps those).
- After PARITY_REACH_BUDGET seconds (default 180) the remaining harnesses
  count every source their probes link.
- crate_of is cached; no pathlib in the hot paths.

Same 79 harnesses on the 19-file diff from contrib-gate-16; CPU 19.9 s cold /
16.6 s warm before, 13.1 s cold / 9.9 s warm after (Mac mini under load).

parity.yml: skip the prepare_output check for heads made before #1947 (no
OUTPUT_VERSION, no tool): the job runs main's workflow on the head's files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 02:13:09 +01:00
Ludovic Marie e1a7892361 Merge pull request #1985 from ludovic111/perf/parity-disk
🤖 Agent: merged on verify. CI speed (Ludovic's focus tonight); measurements in the body and in horizon-parity/ci-speed/LOG.md.
2026-10-26 00:52:57 +01:00
Ludovic Marie 373d9c08c0 Merge pull request #1947 from ludovic111/feat/prepare-output-check
🤖 Agent: verified: OUTPUT_VERSION = 1 and output-versions.txt carry the digest measured on 10 CI preparations. tools/prepare_output.py check passes on v0.2.64's preparation and fails on v0.2.47's. This head's own parity job prepares with its preparer and checks the digest. Merging.
2026-10-26 00:47:05 +01:00
Ludovic Marie 5034e4557d parity: clear rust-<name>-parity outputs after a pass, reuse oracles on CI
Disk: clear_large_outputs only looked at .local directories named after the
harness, so the 174 harnesses writing to rust-<name>-parity kept every dump
(86 GB in one runner checkout, 13 GB for state_provider alone). It now
clears those too, keeps vanilla outputs only up to 256 MB (larger ones cost
more disk than rerunning their oracle), and handles shard names.
collision, world and geometry dropped one mkdtemp run directory per run
(11 GB): each run now removes the earlier ones.

Time: harnesses inherited the runner's stdin, a pipe on GitHub's runner, and
parity_shim never reuses an oracle run whose stdin is a pipe, so oracle reuse
never worked on CI. Harnesses now get /dev/null.

CI: parity.yml falls back to the head's own selection when the names main's
`--list` printed don't match its summary count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 00:44:22 +01:00
Ludovic Marie 836c1ee3b9 CI parity: pick harnesses with main's selection code, run them at the head
The parity job checks out the PR head, so every branch made before #1964
still selected with the old per-crate rule. The step now runs main's
tools/parity.py and tools/parity_reach.py with --changed --list, restores the
head's files, and runs the named harnesses with the head's parity.py. A
branch that edits either file, or a failure of main's selection, falls back
to the head's own `--changed` run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 00:27:32 +01:00
Ludovic Marie 116a7f7dca horizon-prepare: OUTPUT_VERSION, the number of what it prepares, checked by CI
Launchers copy a player's prepared resources to a new release while its preparer declares the
same OUTPUT_VERSION (the launcher side is its own pull request). The number is bumped by hand;
CI keeps it honest: the parity job already prepares with every change to the preparer, and
tools/prepare_output.py check now fails when that preparation's digest is not the one
crates/horizon-prepare/output-versions.txt gives the current number, or when the file changed
other than by appending a version. The digest canonicalises what differs between two runs of
the same preparer (JSON key order, creative tab tag order, lambda class addresses, logs, the
sounds): 10 preparations by 5 preparer versions on the three Mac runners give one digest, and a Linux preparation by an older preparer gives the Mac runners' digest for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 22:48:51 +01:00
Ludovic Marie be16401f90 Telemetry: send to telemetry.playhorizon.dev, a benchmarks summary, the Railway config
- DEFAULT_ENDPOINT is https://telemetry.playhorizon.dev. The heartbeat, the launcher's live count and the
  outbox upload all go there; HORIZON_TELEMETRY_URL still names another server, and HORIZON_TELEMETRY_URL=off
  sends nothing at all. CI workflows and the harnesses that start the game or the launcher set off. Consent
  is unchanged: the four switches stay opt-in, only the anonymous ping is always on.
- GET /v1/benchmarks/summary: players' benchmark uploads grouped by GPU, CPU, render distance and game, groups
  of 3+ installs only, medians per install then across installs, no install ID, day, OS version, driver,
  memory or screen. Cached 10 minutes, CORS for the website. Same rules as website/scripts/benchmarks.mjs,
  tested on the same fixtures; a deleted install leaves its groups at once.
- /v1/aggregate no longer lists the newest 50 raw benchmark reports (hardware, driver, day, one install each):
  uploads are published only through the summary.
- Behind Railway's proxy the rate limit counts X-Real-IP, else the last X-Forwarded-For entry (the one the
  proxy adds; a client could choose the first). Still in memory only, never stored or logged.
- crates/horizon-telemetry-server/railway.json: Dockerfile build from the repository root, health check
  /healthz, one replica in europe-west4 (one SQLite file on the /data volume).
- Docs: docs/telemetry.md, the server README (the summary's shape, Railway setup), the privacy page (the ping
  is on, where the data lives, how benchmark uploads are shown).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 21:46:32 +02:00
Ludovic Marie 9f385280ef Version facts in one generated file: crates/horizon-core/src/version.rs
tools/gen_version_consts.py writes what the pinned client jar's version.json
states (name, id, protocol, data version, pack formats, Java, stable, build
time) into horizon_core::version, and every Rust site reads it from there:
PROTOCOL_VERSION, GAME_VERSION, DATA_VERSION, the save's and the datafixer's
version name, /version, the crash report, the legacy ping, the dedicated
server's start line, the title screens, the default resource folder, the
resource pack format, and the launcher's and horizon-ctl's fallbacks. CI
checks the file against the pinned jar.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 12:22:20 +02:00
Ludovic Marie cef652d026 checks: verify always runs, so a skipped job can never satisfy main's required check
Labels started `checks` runs whose verify job was skipped, and GitHub counts a skipped required
check as passed: auto-merge merged #823 and #837 before the real verify finished. Labels no longer
start `checks` (the fork-only run-parity label has its own workflow), and verify has no condition.
The parity job moves into a reusable workflow both call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 13:58:27 +02:00