perf_bench ran every game in a 1280x720 window only. It now runs the
standard set at 1280x720 and 1920x1080 (--sizes picks others), each size
its own reports: <label>-rd<N>.json at 720p as before, <label>-1080p-rd<N>.json
at 1080p.
Horizon takes vanilla's --width and --height for its window's starting size
(1280x720 by default); perf_bench passes them to every game, and to Horizon
only off 720p, so builds before this one still run the 720p set.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without min_chunks (the old call) wait_loaded behaves as before, which tests/test_perf_bench.py's
old-builds case relies on.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Builds that report status.streaming are loaded once chunk counts are
stable and no stage (generation, decoration, light, meshing) has held work
for half a second. Meshing going quiet also waited through the remeshing
that follows a load (fluids settling, random ticks: 1500 falling to 60
meshes/s for ~150 s at RD16), which is simulation, not loading. Older
builds keep the meshing-rate rule.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each game runs --repeats times (3) per render distance and writes one v1
report per distance (benchmarks/perf/<label>-rd<N>.json): figures as means
with standard deviation and 95% CI, per-flight frame entries with the fixed
histogram, the hardware without the host name. frames() and summary() match
crates/horizon-bench value for value (tested on the same inputs).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tools/perf_bench.py runs Horizon, the vanilla 26.3 client and vanilla with
Sodium, Lithium, FerriteCore and ImmediatelyFast through the same steps:
fresh world (seed 1234, creative) per render distance, start to title, start
to in-world, then per scene teleport, chunk load, settle and a fixed 192-block
flight. Every frame time is kept and one function computes average FPS, 1% and
0.1% lows, p50/p95/p99/p99.9 and a frame-time histogram for every game; RSS is
sampled from outside the process for every game. One options.txt (vanilla's
keys, Fancy preset values with a Custom preset so the render distance holds)
is written for all of them.
Vanilla is driven by a dev-only Fabric mod (mods/horizon/src/bench) compiled
against the user's own client jar: it creates the world with
WorldOpenFlows.createFreshLevel, teleports with /tp and places the player on
the path every frame as horizon-ctl fly does. Nothing of the game is
redistributed and the performance mods are fetched from Modrinth with the
hashes pinned in packs/mods.lock.json.
Horizon side: `fly ... PITCH raw` also replies with every frame time, and
--offscreen drives frames from the event loop, so a window nobody sees (a
locked macOS session) still renders.
docs/benchmark.md describes what is measured, how fairness is kept, the
caveats and how to reproduce it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ludovic Marie