Benchmark: Horizon's server against vanilla 26.3 with mixed bots, as installed

The first published benchmark for the working version (docs/server.md
section 10, item 4). It ran the v2026.10.18.15 release, as install.sh
installs it, against vanilla 26.3 with 1, 10, 25 and 50 mixed bots. The
servers ran on the ThinkPad (quiet, under nice 19) and the bots on the
Mac mini over the tailnet. Vanilla's default game rules applied on both;
Horizon's server runs the locator bar now.

Both servers kept 20 TPS throughout. Horizon used 3.7 to 5.2 times less
memory. Its average tick was shorter with 1, 10 and 25 bots and 18 %
longer with 50, while it generated about twice the new terrain a second
(the exploring bots fly as fast as each server sends chunks). Its P99
was lower at every count.

benchmarks/server-mixed-2026-10-18/ holds the README, the report and the
machine's load; docs/server.md links it from sections 9 and 10.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ludovic Marie
2026-10-18 19:26:26 +02:00
parent f422cbb79a
commit 40972de77a
4 changed files with 789 additions and 1 deletions
@@ -0,0 +1,125 @@
# Horizon's server against vanilla 26.3: mixed bots (2026-10-18)
The first benchmark from docs/server.md section 10 (item 4): the same bots on Horizon's dedicated server, as
`install.sh` installs it today, and on vanilla 26.3's, one server at a time on the same quiet machine.
There were 1, 10, 25 and 50 bots, all with the `mixed` behaviour.
## Summary
- **Both servers held 20 TPS in every run.**
- **Memory:** Horizon's RSS was 3.7 to 5.2 times smaller: 0.65 GB against 3.4 GB with one player, and
1.0 GB against 4.0 GB with 50. Vanilla runs with the usual fixed heap (`-Xms4g -Xmx4g`), so its RSS
is mostly that heap.
- **Tick time (MSPT):**
- with 1, 10 and 25 players, Horizon's average tick was shorter (2.4 against 17.9 ms, 12.4 against 34.5,
24.0 against 31.6);
- with 50, it was longer (35.2 against 29.7 ms);
- Horizon's P99 was lower at every count, and so were its worst 5-second windows (241 ms at most,
against 424 to 2320 ms for vanilla).
- **The work was not equal, and the difference counts against Horizon.** In `mixed`, 20 % of the bots
fly straight out at 22 blocks a second, as fast as each server sends them chunks. Horizon kept up:
- with 10 or more players it sent 2.9 to 5.9 times as many chunks a second, and generated 1.9 to 2.9
times as much new terrain a second;
- its walking bots each received 3300 to 4000 chunks, against 640 to 1100 on vanilla once 10 or more
were online.
So at 50 players Horizon's tick is 18 % longer while it generates twice the new terrain. The
equal-rate explore pass of `../server-equal-work-2026-10-17/` is the fair comparison for exploring.
- **Start (fresh world, to `Done`):** 10 to 20 s for Horizon, 14.5 to 28.6 s for vanilla.
## Results
MSPT is milliseconds per tick from each server's own `/tick query` (50 is the limit for 20 TPS). P95 and
P99 are over the whole run. "Worst P99" is the highest P99 of a single 5-second query. RSS and CPU cover
the server's whole process tree (100 % is one thread; the machine has 4). "New chunks/s" counts chunks
no bot had before. "Pulled back" counts position packets that moved a bot back.
| Bots | Server | MSPT | P95 | P99 | Worst P99 | TPS | RSS (mean / max) | CPU | Start | Chunks/s | New chunks/s | Pulled back |
|---:|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| 1 | Horizon | **2.4** | 5.2 | 18.2 | 45.6 | 20 | **0.65 / 0.66 GB** | 25 % | 10.9 s | 9.9 | 0.3 | 0 |
| 1 | vanilla | 17.9 | 27.6 | 47.3 | 423.8 | 20 | 3.38 / 3.39 GB | 64 % | 14.8 s | 9.9 | 0.3 | 0 |
| 10 | Horizon | **12.4** | 36.0 | 65.9 | 117.1 | 20 | **0.94 / 1.01 GB** | 210 % | 10.2 s | 98.2 | 36.5 | 6267 |
| 10 | vanilla | 34.5 | 63.5 | 138.6 | 884.7 | 20 | 3.59 / 3.66 GB | 217 % | 14.5 s | 24.3 | 19.1 | 5500 |
| 25 | Horizon | **24.0** | 49.7 | 80.6 | 168.8 | 20 | **1.01 / 1.18 GB** | 192 % | 14.7 s | 178.2 | 31.9 | 5532 |
| 25 | vanilla | 31.6 | 70.0 | 222.7 | 2319.6 | 20 | 3.74 / 4.06 GB | 218 % | 28.6 s | 30.4 | 10.8 | 4524 |
| 50 | Horizon | 35.2 | 69.0 | 96.6 | 240.8 | 20 | **1.01 / 1.25 GB** | 218 % | 19.8 s | 273.2 | 34.6 | 8369 |
| 50 | vanilla | **29.7** | 62.3 | 137.4 | 489.3 | 20 | 4.00 / 4.60 GB | 294 % | 20.6 s | 94.4 | 17.8 | 12034 |
Chunks each bot received, by behaviour (mean):
| Bots | Server | walk | explore | idle |
|---:|---|---:|---:|---:|
| 10 | Horizon | 3992 | 7360 | 473 |
| 10 | vanilla | 680 | 2824 | 358 |
| 25 | Horizon | 3928 | 2475 | 473 |
| 25 | vanilla | 640 | 738 | 180 |
| 50 | Horizon | 3300 | 1192 | 460 |
| 50 | vanilla | 1105 | 629 | 220 |
No bot failed to join or was dropped, and no chunk was sent twice (`resent` 0), on either server. Vanilla
also logged "moved too quickly" 12, 324 and 2193 times at 10, 25 and 50 bots.
## Method
`tools/server_bench.py`, unchanged, runs each configuration on a fresh server:
- the same `server.properties` on both: seed 12345, view and simulation distance 10, offline, creative,
difficulty easy, `sync-chunk-writes=true`, white-list off;
- `gamerule respawn_radius 0`, and otherwise vanilla's default game rules on both. The locator bar is on
for both: Horizon's server has run it since v2026.10.18.13. The 2026-10-17 runs turned it off on vanilla
because Horizon had none then;
- the server waits for `Done`, then the bots join one a second, warm up for 60 seconds, and are measured
for 300 seconds, with `/tick query` every 5 seconds and RSS and CPU every second.
The bots are `horizon-bots --behaviour mixed`: 60 % walk, 20 % explore and 20 % idle, each choice
deterministic per bot. All are creative and fly 24 blocks above their spawn:
- `walk`: random points within 48 blocks at 5.6 blocks a second;
- `explore`: straight out at up to 22 blocks a second, as fast as chunks come;
- `idle`: hovers.
A bot doesn't move while the chunk it stands in hasn't arrived, and it acknowledges chunk batches as
vanilla's client does.
**Machine.** The ThinkPad: Intel Core i7-7500U (2 cores, 4 threads), 32 GB, NixOS (Linux 6.18). No other
session ran on it (`load.txt`: the fleet monitor shows 0 sessions throughout). The busy periods are the
servers themselves, which used 2 to 3 of its 4 threads from 10 bots up, so both servers were CPU-bound
there. The servers ran under `nice -n 19`. The desktop was meant to run this (24 threads) but didn't
wake. The bots ran on the Mac mini (M4 Pro) and reached the ThinkPad over the tailnet, so they took none
of the servers' CPU. The Mac mini was shared with other work (70 to 90 % busy), which only affects the
bots' timing.
**What ran.**
- Horizon: the published v2026.10.18.15 release, `horizon-client` (the engine) at 6c078b28, as
`install.sh` installs it. Its game files were prepared from the owner's own Minecraft by the release's
`horizon-prepare`. `horizon-server` (the supervisor, which takes no part in the tick) came from #898
(4ea2243e), because the release's copy can't find game files prepared without sounds (#893).
`horizon-bots` came from main.
- Vanilla: the pinned 26.3 server jar (the owner's own copy) on OpenJDK 25.0.4.1, `-Xms4g -Xmx4g`.
On Linux the release's engine needs `libasound.so.2` and `libudev.so.1`. The ThinkPad has them only in
the Nix store, so the run set `LD_LIBRARY_PATH` to them (see #898).
## Reproduce
curl -fsSL https://raw.githubusercontent.com/ludovic111/horizon/main/packaging/server/install.sh | sh
horizon-prepare --prepare --server-jar /path/to/server.jar # your own copy
cargo build --release -p horizon-bots # on the machine that runs the bots
# in a checkout whose .local/minecraft holds the pinned vanilla jars (tools/lab.py prepare):
HORIZON_BENCH_BIN=<a folder with horizon-server and horizon-client> \
python3 tools/server_bench.py --servers horizon,vanilla --bots 1,10,25,50 --behaviour mixed --nice 19 \
--bots-via 'ssh <bots machine>' --bots-bin /path/to/horizon-bots --connect <this machine's address> \
--out mixed.json
`mixed.json` is the report (`horizon-server-benchmark`: machine, versions, settings and every run), and
`load.txt` is the ThinkPad's load before and during the run.
## Next
- An equal-work `mixed` pass (`--explore-speed` at a rate both servers keep up with), then 100 bots.
- The same run on a bigger machine, so neither server is CPU-bound at 25 to 50 bots.
- Paper and Fabric + Lithium (docs/server.md section 9).
- At 50 players Horizon's tick is longer than vanilla's while it generates and sends far more. The
profile of the 2026-10-17 run (streaming and block entity ticks growing with players) is
perf-server's to take on.
+286
View File
@@ -0,0 +1,286 @@
ludothinkpad — OK
RAM 29.1 GB free of 31.2 GB · swap 0.3 GB/15.6 GB
CPU 4 threads, 2% busy (agents 0.0, rest 0.1 cores) · 67°C · load 2.1
· disk 295.5 GB free · battery 96%
Sessions, most important first:
(finished session) 0.0 GB leftovers: node 0.0 GB
Other machines (≤ 1 min old): desktop offline · macbook offline · macmini ok: 26.6 GB free, 14 threads 69% busy, 1 sessions
# During the run, once a minute (uptime, then ~/fleet/load fleet)
18:25:30
18:25:30 up 8 days 3:08, 0 users, load average: 2.13, 1.53, 1.29
ludothinkpad ok: 29.1 GB free, 4 threads 2% busy, 0 sessions
desktop offline
macmini busy: 27.6 GB free, 14 threads 87% busy, 1 sessions
18:26:34
18:26:34 up 8 days 3:09, 0 users, load average: 1.70, 1.56, 1.31
ludothinkpad ok: 28.5 GB free, 4 threads 23% busy, 0 sessions
desktop offline
macmini busy: 26.4 GB free, 14 threads 85% busy, 1 sessions
18:27:38
18:27:38 up 8 days 3:10, 0 users, load average: 1.10, 1.43, 1.28
ludothinkpad ok: 28.5 GB free, 4 threads 4% busy, 0 sessions
desktop offline
macmini ok: 27.7 GB free, 14 threads 80% busy, 1 sessions
18:28:42
18:28:42 up 8 days 3:11, 0 users, load average: 0.93, 1.29, 1.24
ludothinkpad ok: 28.5 GB free, 4 threads 22% busy, 0 sessions
desktop offline
macmini ok: 26.0 GB free, 14 threads 82% busy, 1 sessions
18:29:46
18:29:46 up 8 days 3:12, 0 users, load average: 0.62, 1.12, 1.18
ludothinkpad ok: 28.5 GB free, 4 threads 12% busy, 0 sessions
desktop offline
macmini ok: 25.7 GB free, 14 threads 81% busy, 1 sessions
18:30:50
18:30:51 up 8 days 3:13, 0 users, load average: 0.79, 1.05, 1.15
ludothinkpad ok: 28.5 GB free, 4 threads 7% busy, 0 sessions
desktop offline
macmini busy: 27.0 GB free, 14 threads 88% busy, 1 sessions
18:31:55
18:31:55 up 8 days 3:14, 0 users, load average: 0.67, 0.96, 1.11
ludothinkpad ok: 28.6 GB free, 4 threads 73% busy, 0 sessions
desktop offline
macmini ok: 27.3 GB free, 14 threads 79% busy, 1 sessions
18:32:59
18:32:59 up 8 days 3:15, 0 users, load average: 4.01, 1.91, 1.44
ludothinkpad ok: 25.9 GB free, 4 threads 24% busy, 0 sessions
desktop offline
macmini ok: 26.4 GB free, 14 threads 85% busy, 1 sessions
18:34:03
18:34:03 up 8 days 3:16, 0 users, load average: 2.24, 1.84, 1.44
ludothinkpad ok: 25.9 GB free, 4 threads 21% busy, 0 sessions
desktop offline
macmini busy: 29.2 GB free, 14 threads 86% busy, 1 sessions
18:35:07
18:35:07 up 8 days 3:17, 0 users, load average: 1.28, 1.63, 1.40
ludothinkpad ok: 25.8 GB free, 4 threads 21% busy, 0 sessions
desktop offline
macmini busy: 31.0 GB free, 14 threads 86% busy, 1 sessions
18:36:11
18:36:11 up 8 days 3:18, 0 users, load average: 0.79, 1.41, 1.33
ludothinkpad ok: 25.9 GB free, 4 threads 18% busy, 0 sessions
desktop offline
macmini busy: 26.3 GB free, 14 threads 93% busy, 1 sessions
18:37:15
18:37:15 up 8 days 3:20, 0 users, load average: 1.44, 1.47, 1.36
ludothinkpad ok: 25.9 GB free, 4 threads 23% busy, 0 sessions
desktop offline
macmini busy: 26.8 GB free, 14 threads 86% busy, 1 sessions
18:38:19
18:38:19 up 8 days 3:21, 0 users, load average: 1.37, 1.45, 1.36
ludothinkpad ok: 25.9 GB free, 4 threads 13% busy, 0 sessions
desktop offline
macmini busy: 26.7 GB free, 14 threads 86% busy, 1 sessions
18:39:24
18:39:24 up 8 days 3:22, 0 users, load average: 2.27, 1.66, 1.44
ludothinkpad ok: 28.3 GB free, 4 threads 62% busy, 0 sessions
desktop offline
macmini ok: 27.1 GB free, 14 threads 70% busy, 1 sessions
18:40:28
18:40:28 up 8 days 3:23, 0 users, load average: 1.95, 1.72, 1.48
ludothinkpad ok: 28.3 GB free, 4 threads 63% busy, 0 sessions
desktop offline
macmini ok: 26.7 GB free, 14 threads 84% busy, 1 sessions
18:41:32
18:41:32 up 8 days 3:24, 0 users, load average: 3.32, 2.22, 1.67
ludothinkpad ok: 28.2 GB free, 4 threads 61% busy, 0 sessions
desktop offline
macmini busy: 26.8 GB free, 14 threads 85% busy, 1 sessions
18:42:36
18:42:36 up 8 days 3:25, 0 users, load average: 3.13, 2.33, 1.75
ludothinkpad ok: 28.2 GB free, 4 threads 60% busy, 0 sessions
desktop offline
macmini busy: 27.1 GB free, 14 threads 91% busy, 1 sessions
18:43:40
18:43:40 up 8 days 3:26, 0 users, load average: 3.27, 2.48, 1.83
ludothinkpad busy: 28.1 GB free, 4 threads 90% busy, 0 sessions
desktop offline
macmini busy: 28.3 GB free, 14 threads 92% busy, 1 sessions
18:44:44
18:44:44 up 8 days 3:27, 0 users, load average: 2.14, 2.32, 1.82
ludothinkpad ok: 28.1 GB free, 4 threads 55% busy, 0 sessions
desktop offline
macmini busy: 30.4 GB free, 14 threads 88% busy, 1 sessions
18:45:48
18:45:48 up 8 days 3:28, 0 users, load average: 4.94, 3.06, 2.11
ludothinkpad busy: 25.9 GB free, 4 threads 86% busy, 0 sessions
desktop offline
macmini ok: 28.7 GB free, 14 threads 84% busy, 1 sessions
18:46:53
18:46:53 up 8 days 3:29, 0 users, load average: 6.55, 3.92, 2.47
ludothinkpad busy: 25.4 GB free, 4 threads 99% busy, 0 sessions
desktop offline
macmini ok: 29.3 GB free, 14 threads 81% busy, 1 sessions
18:47:57
18:47:57 up 8 days 3:30, 0 users, load average: 8.50, 4.96, 2.93
ludothinkpad busy: 24.0 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini busy: 27.9 GB free, 14 threads 90% busy, 1 sessions
18:49:01
18:49:01 up 8 days 3:31, 0 users, load average: 8.59, 5.68, 3.31
ludothinkpad busy: 22.7 GB free, 4 threads 99% busy, 0 sessions
desktop offline
macmini ok: 28.2 GB free, 14 threads 85% busy, 1 sessions
18:50:05
18:50:05 up 8 days 3:32, 0 users, load average: 7.81, 6.07, 3.61
ludothinkpad busy: 24.5 GB free, 4 threads 90% busy, 0 sessions
desktop offline
macmini busy: 28.6 GB free, 14 threads 96% busy, 1 sessions
18:51:09
18:51:09 up 8 days 3:33, 0 users, load average: 8.29, 6.50, 3.93
ludothinkpad busy: 24.3 GB free, 4 threads 92% busy, 0 sessions
desktop offline
macmini busy: 28.0 GB free, 14 threads 94% busy, 1 sessions
18:52:14
18:52:14 up 8 days 3:35, 0 users, load average: 5.58, 6.06, 3.95
ludothinkpad busy: 26.0 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini busy: 26.3 GB free, 14 threads 87% busy, 1 sessions
18:53:18
18:53:18 up 8 days 3:36, 0 users, load average: 4.53, 5.69, 3.97
ludothinkpad busy: 27.2 GB free, 4 threads 89% busy, 0 sessions
desktop offline
macmini busy: 26.8 GB free, 14 threads 87% busy, 1 sessions
18:54:22
18:54:22 up 8 days 3:37, 0 users, load average: 5.31, 5.64, 4.07
ludothinkpad busy: 26.3 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini busy: 29.3 GB free, 14 threads 91% busy, 1 sessions
18:55:26
18:55:26 up 8 days 3:38, 0 users, load average: 10.47, 6.99, 4.63
ludothinkpad busy: 26.1 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini busy: 26.7 GB free, 14 threads 90% busy, 1 sessions
18:56:30
18:56:30 up 8 days 3:39, 0 users, load average: 9.27, 7.45, 4.96
ludothinkpad busy: 26.2 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini busy: 28.5 GB free, 14 threads 92% busy, 1 sessions
18:57:35
18:57:35 up 8 days 3:40, 0 users, load average: 7.79, 7.39, 5.10
ludothinkpad busy: 26.1 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini ok: 27.1 GB free, 14 threads 80% busy, 1 sessions
18:58:39
18:58:39 up 8 days 3:41, 0 users, load average: 8.06, 7.55, 5.32
ludothinkpad busy: 26.4 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini busy: 31.8 GB free, 14 threads 95% busy, 1 sessions
18:59:43
18:59:43 up 8 days 3:42, 0 users, load average: 8.44, 7.65, 5.50
ludothinkpad busy: 26.2 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini busy: 31.4 GB free, 14 threads 94% busy, 1 sessions
19:00:47
19:00:47 up 8 days 3:43, 0 users, load average: 11.04, 8.66, 6.00
ludothinkpad busy: 24.9 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini busy: 31.8 GB free, 14 threads 88% busy, 1 sessions
19:01:51
19:01:51 up 8 days 3:44, 0 users, load average: 11.91, 9.37, 6.41
ludothinkpad busy: 24.4 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini ok: 32.9 GB free, 14 threads 77% busy, 1 sessions
19:02:56
19:02:56 up 8 days 3:45, 0 users, load average: 11.64, 9.85, 6.79
ludothinkpad ok: 25.5 GB free, 4 threads 70% busy, 0 sessions
desktop offline
macmini busy: 30.9 GB free, 14 threads 91% busy, 1 sessions
19:04:00
19:04:00 up 8 days 3:46, 0 users, load average: 11.42, 10.06, 7.06
ludothinkpad busy: 23.4 GB free, 4 threads 99% busy, 0 sessions
desktop offline
macmini busy: 28.4 GB free, 14 threads 94% busy, 1 sessions
19:05:04
19:05:04 up 8 days 3:47, 0 users, load average: 9.83, 9.96, 7.24
ludothinkpad busy: 23.2 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini ok: 31.4 GB free, 14 threads 80% busy, 1 sessions
19:06:09
19:06:09 up 8 days 3:48, 0 users, load average: 8.38, 9.68, 7.34
ludothinkpad ok: 22.7 GB free, 4 threads 17% busy, 0 sessions
desktop offline
macmini busy: 29.8 GB free, 14 threads 86% busy, 1 sessions
19:07:13
19:07:13 up 8 days 3:49, 0 users, load average: 4.51, 8.27, 7.00
ludothinkpad ok: 25.7 GB free, 4 threads 75% busy, 0 sessions
desktop offline
macmini busy: 26.5 GB free, 14 threads 89% busy, 1 sessions
19:08:17
19:08:17 up 8 days 3:51, 0 users, load average: 6.77, 8.13, 7.04
ludothinkpad busy: 25.8 GB free, 4 threads 95% busy, 0 sessions
desktop offline
macmini busy: 30.8 GB free, 14 threads 91% busy, 1 sessions
19:09:22
19:09:22 up 8 days 3:52, 0 users, load average: 8.27, 8.25, 7.14
ludothinkpad busy: 25.2 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini busy: 28.3 GB free, 14 threads 91% busy, 1 sessions
19:10:26
19:10:26 up 8 days 3:53, 0 users, load average: 8.88, 8.57, 7.34
ludothinkpad busy: 24.8 GB free, 4 threads 99% busy, 0 sessions
desktop offline
macmini ok: 32.4 GB free, 14 threads 72% busy, 1 sessions
19:11:30
19:11:30 up 8 days 3:54, 0 users, load average: 13.33, 10.05, 7.94
ludothinkpad busy: 24.7 GB free, 4 threads 97% busy, 0 sessions
desktop offline
macmini busy: 30.1 GB free, 14 threads 87% busy, 1 sessions
19:12:35
19:12:35 up 8 days 3:55, 0 users, load average: 10.92, 10.13, 8.12
ludothinkpad ok: 26.4 GB free, 4 threads 76% busy, 0 sessions
desktop offline
macmini busy: 29.3 GB free, 14 threads 86% busy, 1 sessions
19:13:39
19:13:39 up 8 days 3:56, 0 users, load average: 9.98, 9.89, 8.16
ludothinkpad busy: 25.8 GB free, 4 threads 90% busy, 0 sessions
desktop offline
macmini busy: 28.8 GB free, 14 threads 93% busy, 1 sessions
19:14:43
19:14:43 up 8 days 3:57, 0 users, load average: 8.95, 9.63, 8.19
ludothinkpad busy: 25.1 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini ok: 29.6 GB free, 14 threads 71% busy, 1 sessions
19:15:47
19:15:47 up 8 days 3:58, 0 users, load average: 12.02, 10.38, 8.55
ludothinkpad busy: 24.5 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini busy: 28.1 GB free, 14 threads 93% busy, 1 sessions
19:16:52
19:16:52 up 8 days 3:59, 0 users, load average: 13.82, 11.21, 8.95
ludothinkpad busy: 23.9 GB free, 4 threads 100% busy, 0 sessions
desktop offline
macmini ok: 28.4 GB free, 14 threads 75% busy, 1 sessions
19:17:56
19:17:56 up 8 days 4:00, 0 users, load average: 11.95, 11.20, 9.10
ludothinkpad busy: 25.0 GB free, 4 threads 99% busy, 0 sessions
desktop offline
macmini ok: 28.4 GB free, 14 threads 81% busy, 1 sessions
19:19:00
19:19:00 up 8 days 4:01, 0 users, load average: 11.46, 11.14, 9.22
ludothinkpad busy: 24.6 GB free, 4 threads 95% busy, 0 sessions
desktop offline
macmini ok: 26.4 GB free, 14 threads 83% busy, 1 sessions
19:20:04
19:20:04 up 8 days 4:02, 0 users, load average: 11.11, 10.97, 9.28
ludothinkpad busy: 24.6 GB free, 4 threads 93% busy, 0 sessions
desktop offline
macmini ok: 26.7 GB free, 14 threads 85% busy, 1 sessions
19:21:08
19:21:08 up 8 days 4:03, 0 users, load average: 9.81, 10.60, 9.27
ludothinkpad busy: 24.4 GB free, 4 threads 91% busy, 0 sessions
desktop offline
macmini ok: 27.8 GB free, 14 threads 75% busy, 1 sessions
19:22:13
19:22:13 up 8 days 4:04, 0 users, load average: 5.98, 9.51, 9.00
ludothinkpad ok: 23.8 GB free, 4 threads 18% busy, 0 sessions
desktop offline
macmini busy: 29.7 GB free, 14 threads 85% busy, 1 sessions
19:23:17
19:23:17 up 8 days 4:06, 0 users, load average: 4.01, 8.27, 8.60
ludothinkpad ok: 23.1 GB free, 4 threads 56% busy, 0 sessions
desktop offline
macmini ok: 28.2 GB free, 14 threads 79% busy, 1 sessions
@@ -0,0 +1,367 @@
{
"kind": "horizon-server-benchmark",
"date": "2026-10-18T17:23:30+00:00",
"machine": {
"system": "Linux",
"machine": "x86_64",
"cpus": 4,
"note": "ThinkPad: Intel Core i7-7500U (2 cores, 4 threads), 32 GB, NixOS, Linux 6.18; the bots on the Mac mini (M4 Pro) over the tailnet"
},
"commit": "4ea2243e4d1a44f910857bd759845db70e3690a8",
"java": [
"openjdk 25.0.4.1 2026-08-18"
],
"settings": {
"servers": "horizon,vanilla",
"bots": "1,10,25,50",
"behaviour": "mixed",
"join_every": 1.0,
"warmup": 60,
"seconds": 300,
"seed": 12345,
"explore_speed": 0,
"gamerule": "",
"heap": "4g",
"out": "mixed.json",
"nice": 19,
"bots_via": "ssh macmini",
"bots_bin": "horizon-bots (on the Mac mini)",
"connect": "<the ThinkPad's tailnet address>",
"properties": {
"level-seed": "12345",
"view-distance": "10",
"simulation-distance": "10",
"online-mode": "false",
"white-list": "false",
"gamemode": "creative",
"force-gamemode": "true",
"spawn-protection": "0",
"max-players": "200",
"difficulty": "easy",
"sync-chunk-writes": "true"
}
},
"runs": [
{
"server": "horizon",
"bots": 1,
"behaviour": "mixed",
"start_seconds": 10.92,
"mspt": {
"average": 2.38,
"p50": 1.97,
"p95": 5.17,
"p99": 18.2
},
"mspt_worst_p99": 45.6,
"tps": 20.0,
"rss_mb": {
"mean": 653.7,
"max": 659.2
},
"cpu_percent": 24.51,
"chunks_per_second": 9.9,
"unique_per_second": 0.3,
"forgotten_per_second": 9.93,
"waited_ticks_per_second": 0.0,
"chunks_total": 3951,
"chunks_per_bot": {
"walk": 3951.0
},
"unique_chunks_per_bot": {
"walk": 699.0
},
"resent": 0,
"pulled_back": 0,
"moved_too_quickly": 0,
"moved_wrongly": 0,
"bots_joined": 1,
"bots_failed": [],
"tick_queries": 58
},
{
"server": "vanilla",
"bots": 1,
"behaviour": "mixed",
"start_seconds": 14.75,
"mspt": {
"average": 17.9,
"p50": 18.39,
"p95": 27.58,
"p99": 47.28
},
"mspt_worst_p99": 423.8,
"tps": 20.0,
"rss_mb": {
"mean": 3382.06,
"max": 3394.1
},
"cpu_percent": 64.02,
"chunks_per_second": 9.92,
"unique_per_second": 0.29,
"forgotten_per_second": 9.95,
"waited_ticks_per_second": 0.0,
"chunks_total": 3846,
"chunks_per_bot": {
"walk": 3846.0
},
"unique_chunks_per_bot": {
"walk": 699.0
},
"resent": 0,
"pulled_back": 0,
"moved_too_quickly": 0,
"moved_wrongly": 0,
"bots_joined": 1,
"bots_failed": [],
"tick_queries": 58
},
{
"server": "horizon",
"bots": 10,
"behaviour": "mixed",
"start_seconds": 10.22,
"mspt": {
"average": 12.39,
"p50": 8.33,
"p95": 35.99,
"p99": 65.9
},
"mspt_worst_p99": 117.1,
"tps": 20.0,
"rss_mb": {
"mean": 935.85,
"max": 1008.6
},
"cpu_percent": 210.37,
"chunks_per_second": 98.22,
"unique_per_second": 36.49,
"forgotten_per_second": 98.25,
"waited_ticks_per_second": 0.0,
"chunks_total": 39618,
"chunks_per_bot": {
"walk": 3992.17,
"explore": 7359.5,
"idle": 473.0
},
"unique_chunks_per_bot": {
"walk": 696.83,
"explore": 6869.0,
"idle": 473.0
},
"resent": 0,
"pulled_back": 6267,
"moved_too_quickly": 0,
"moved_wrongly": 0,
"bots_joined": 10,
"bots_failed": [],
"tick_queries": 58
},
{
"server": "vanilla",
"bots": 10,
"behaviour": "mixed",
"start_seconds": 14.54,
"mspt": {
"average": 34.47,
"p50": 29.67,
"p95": 63.52,
"p99": 138.58
},
"mspt_worst_p99": 884.7,
"tps": 20.0,
"rss_mb": {
"mean": 3594.26,
"max": 3661.7
},
"cpu_percent": 217.13,
"chunks_per_second": 24.34,
"unique_per_second": 19.13,
"forgotten_per_second": 20.12,
"waited_ticks_per_second": 0.62,
"chunks_total": 10443,
"chunks_per_bot": {
"walk": 679.83,
"explore": 2824.0,
"idle": 358.0
},
"unique_chunks_per_bot": {
"walk": 389.17,
"explore": 2824.0,
"idle": 358.0
},
"resent": 0,
"pulled_back": 5500,
"moved_too_quickly": 12,
"moved_wrongly": 0,
"bots_joined": 10,
"bots_failed": [],
"tick_queries": 58
},
{
"server": "horizon",
"bots": 25,
"behaviour": "mixed",
"start_seconds": 14.73,
"mspt": {
"average": 24.0,
"p50": 21.07,
"p95": 49.72,
"p99": 80.57
},
"mspt_worst_p99": 168.8,
"tps": 20.0,
"rss_mb": {
"mean": 1013.07,
"max": 1179.4
},
"cpu_percent": 192.47,
"chunks_per_second": 178.23,
"unique_per_second": 31.85,
"forgotten_per_second": 177.93,
"waited_ticks_per_second": 60.59,
"chunks_total": 73657,
"chunks_per_bot": {
"walk": 3927.87,
"explore": 2474.8,
"idle": 473.0
},
"unique_chunks_per_bot": {
"walk": 691.73,
"explore": 2447.8,
"idle": 473.0
},
"resent": 0,
"pulled_back": 5532,
"moved_too_quickly": 0,
"moved_wrongly": 0,
"bots_joined": 25,
"bots_failed": [],
"tick_queries": 58
},
{
"server": "vanilla",
"bots": 25,
"behaviour": "mixed",
"start_seconds": 28.57,
"mspt": {
"average": 31.59,
"p50": 25.18,
"p95": 70.03,
"p99": 222.74
},
"mspt_worst_p99": 2319.6,
"tps": 20.0,
"rss_mb": {
"mean": 3742.51,
"max": 4055.0
},
"cpu_percent": 218.34,
"chunks_per_second": 30.37,
"unique_per_second": 10.84,
"forgotten_per_second": 29.44,
"waited_ticks_per_second": 50.91,
"chunks_total": 14193,
"chunks_per_bot": {
"walk": 640.2,
"explore": 738.0,
"idle": 180.0
},
"unique_chunks_per_bot": {
"walk": 211.33,
"explore": 737.6,
"idle": 180.0
},
"resent": 0,
"pulled_back": 4524,
"moved_too_quickly": 324,
"moved_wrongly": 0,
"bots_joined": 25,
"bots_failed": [],
"tick_queries": 56
},
{
"server": "horizon",
"bots": 50,
"behaviour": "mixed",
"start_seconds": 19.84,
"mspt": {
"average": 35.2,
"p50": 32.12,
"p95": 69.04,
"p99": 96.56
},
"mspt_worst_p99": 240.8,
"tps": 20.0,
"rss_mb": {
"mean": 1013.2,
"max": 1249.1
},
"cpu_percent": 217.68,
"chunks_per_second": 273.18,
"unique_per_second": 34.58,
"forgotten_per_second": 268.19,
"waited_ticks_per_second": 138.29,
"chunks_total": 115527,
"chunks_per_bot": {
"walk": 3300.1,
"explore": 1192.4,
"idle": 460.0
},
"unique_chunks_per_bot": {
"walk": 632.67,
"explore": 1186.9,
"idle": 460.0
},
"resent": 0,
"pulled_back": 8369,
"moved_too_quickly": 0,
"moved_wrongly": 0,
"bots_joined": 50,
"bots_failed": [],
"tick_queries": 57
},
{
"server": "vanilla",
"bots": 50,
"behaviour": "mixed",
"start_seconds": 20.58,
"mspt": {
"average": 29.74,
"p50": 23.6,
"p95": 62.25,
"p99": 137.41
},
"mspt_worst_p99": 489.3,
"tps": 20.0,
"rss_mb": {
"mean": 3995.71,
"max": 4597.8
},
"cpu_percent": 294.31,
"chunks_per_second": 94.36,
"unique_per_second": 17.76,
"forgotten_per_second": 94.69,
"waited_ticks_per_second": 101.48,
"chunks_total": 41637,
"chunks_per_bot": {
"walk": 1104.97,
"explore": 628.8,
"idle": 220.0
},
"unique_chunks_per_bot": {
"walk": 279.4,
"explore": 626.4,
"idle": 220.0
},
"resent": 0,
"pulled_back": 12034,
"moved_too_quickly": 2193,
"moved_wrongly": 0,
"bots_joined": 50,
"bots_failed": [],
"tick_queries": 57
}
],
"horizon_build": "v2026.10.18.15 release binaries (horizon-client, the engine, at 6c078b28); horizon-server from #898 (4ea2243e, the supervisor only)"
}
+11 -1
View File
@@ -480,6 +480,16 @@ reproducible by anyone, and show the same gameplay on both sides.
and every packet woke its connection's writer thread (#649). Together they took the 50-bot walking tick
from 281 ms to 31 ms. At 25 walking bots most of what remains is the engine's streaming and block entity
ticks, both of which grow with players (the README has the profile).
- **Mixed bots, as installed (2026-10-18,** `benchmarks/server-mixed-2026-10-18/`, the first published
run for the working version). It used the v2026.10.18.15 release as `install.sh` installs it, against
vanilla 26.3, with 1, 10, 25 and 50 `mixed` bots and vanilla's default game rules on both (Horizon has
the locator bar now). The servers ran on the ThinkPad and the bots on the Mac mini. Results:
- Both kept 20 TPS in every run.
- Horizon's RSS was 3.7 to 5.2 times smaller (1.0 GB against 4.0 GB at 50).
- Horizon's average tick was shorter with 1, 10 and 25 bots (2.4 against 17.9 ms, 12.4 against 34.5,
24.0 against 31.6) and longer with 50 (35.2 against 29.7). Its P99 was lower at every count.
- Horizon also did more work: it kept up with the exploring bots, so it generated 1.9 to 2.9 times
the new terrain a second.
- **Ownership.** perf-server-1 owns making the tick faster. This work only measures it, files what it finds,
and fixes startup, RAM and packaging.
@@ -498,7 +508,7 @@ vanilla.
archive in the release.
3. The web panel MVP: status, live TPS/MSPT/RAM/CPU, players, console, backups and login links.
4. `horizon-bots` and `tools/server_bench.py`, with a first run against vanilla (1/10/25/50 bots, `mixed`)
published in `benchmarks/`.
published in `benchmarks/`: done, `benchmarks/server-mixed-2026-10-18/` (section 9).
5. Small engine fixes found on the way: logging refused logins (vanilla's lines).
**Where it stands (2026-10-16):** items 1 to 4 are built and in review (section 11). What's left for the 23rd: