diff --git a/benchmarks/server-2026-10-17/README.md b/benchmarks/server-2026-10-17/README.md new file mode 100644 index 000000000..3399a6130 --- /dev/null +++ b/benchmarks/server-2026-10-17/README.md @@ -0,0 +1,93 @@ +# Dedicated server against vanilla 26.3, with bots (2026-10-17) + +`horizon-client --dedicated` (run by `horizon-server`) against the pinned vanilla 26.3 `server.jar`. Each +run uses `horizon-bots` and `tools/server_bench.py`, with 1, 10, 25, 50 and 100 bots, `walk` and then +`explore`. The two servers take turns at each bot count. + +**Read this first: the machine was overloaded.** The Mac mini was shared with other agents' builds and +harnesses. Its load average was 55 to 141 on 14 threads, and the CPU was 77–89 % busy before either server +started (`load.txt`, sampled every 30 s). The 1- and 10-bot rows compare the servers fairly enough. Above +that, both servers collapsed: vanilla's watchdog killed it after a 60-second tick in 5 of the 10 large runs. +Treat those rows as "it falls over on a starved machine", not as each server's capacity. + +## Setup + +- **Machine:** Mac mini M4 Pro (14 threads, 64 GB), macOS 27.0.1. +- **Horizon:** release build of `main` at `123987c4` (`horizon-server`, `horizon-client`, `horizon-bots`). + The `commit` field in the JSON is that build. `worktree_commit` is the checkout the script ran from. +- **Vanilla:** the pinned 26.3 `server.jar` on OpenJDK 25.0.4.1, with `-Xms4g -Xmx4g`. +- **Properties:** + - `level-seed=12345`, view and simulation distance 10, creative (forced); + - `online-mode=false`, `difficulty=easy`, `max-players=200`. +- **Each run:** a fresh server, bots joining one per second, a 60 s warm-up, then 180 s measured. +- **Command:** + ``` + HORIZON_BENCH_BIN= python3 tools/server_bench.py --servers horizon,vanilla \ + --bots 1,10,25,50,100 --behaviour walk|explore --warmup 60 --seconds 180 + ``` +- **Columns:** + - **TPS and MSPT** come from `/tick query`. + - **RSS** is the mean over the run. **CPU** is the process tree, as a percentage of one core. + - **Chunks/s** is what the bots received. + - **Load** is the machine's load average during that run. + +## Results + +### `walk`: bots walk around the spawn + +| Bots | Server | TPS | MSPT avg | MSPT p99 | RSS | CPU | Chunks/s | Load | +|---:|---|---:|---:|---:|---:|---:|---:|---:| +| 1 | Horizon | 20.0 | **4.9 ms** | 86 ms | **1.63 GB** | 18 % | 10.5 | 68 | +| 1 | vanilla | 20.0 | 13.4 ms | 84 ms | 3.47 GB | 26 % | 10.1 | 64 | +| 10 | Horizon | 20.0 | **11.5 ms** | 138 ms | **1.57 GB** | 38 % | 101.5 | 55 | +| 10 | vanilla | 20.0 | 21.2 ms | 112 ms | 3.46 GB | 49 % | 93.7 | 58 | +| 25 | Horizon | **0.9** | 1115 ms | 4513 ms | 1.61 GB | 45 % | 200.9 | 83 | +| 25 | vanilla | **20.0** | 38.8 ms | 250 ms | 3.58 GB | 69 % | 255.2 | 85 | +| 50 | Horizon | 0.46 | 2192 ms | 14169 ms | 1.71 GB | 24 % | 125.7 | 125 | +| 50 | vanilla | crashed | 60 s tick, watchdog | | | | | 141 | +| 100 | Horizon | 0.38 | 2638 ms | 17095 ms | 1.91 GB | 46 % | 222.0 | 109 | +| 100 | vanilla | crashed | 60 s tick, watchdog | | | | | 87 | + +### `explore`: bots fly outwards in straight lines + +| Bots | Server | TPS | MSPT avg | MSPT p99 | RSS | CPU | Chunks/s | Load | +|---:|---|---:|---:|---:|---:|---:|---:|---:| +| 1 | Horizon | 20.0 | 11.6 ms | 123 ms | **1.69 GB** | 120 % | 44.5 | 95 | +| 1 | vanilla | 20.0 | 15.9 ms | 110 ms | 3.52 GB | 92 % | 32.3 | 110 | +| 10 | Horizon | 20.0 | 8.2 ms | 48 ms | 2.03 GB | 275 % | **10.8** | 80 | +| 10 | vanilla | 20.0 | 30.8 ms | 282 ms | 4.62 GB | 169 % | 41.4 | 66 | +| 25 | Horizon | 20.0 | 14.6 ms | 81 ms | 1.94 GB | 231 % | **0.0** | 76 | +| 25 | vanilla | crashed | 60 s tick, watchdog | | | | | 84 | +| 50 | Horizon | 13.9 | 72.2 ms | 285 ms | 2.69 GB | 163 % | **0.0** | 83 | +| 50 | vanilla | crashed | 60 s tick, watchdog | | | | | 98 | +| 100 | Horizon | 4.2 | 239 ms | 581 ms | 1.84 GB | 106 % | **0.0** | 122 | +| 100 | vanilla | crashed | 60 s tick, watchdog | | | | | 140 | + +From 10 `explore` bots up, Horizon's ticks look short only because it does not do the work. Its bots +received few or no new chunks: generation and lighting fall behind fast fliers, and the hub sends only lit +chunks. Vanilla generated 4 times more terrain for 10 bots before it collapsed at 25. This build has no +movement checks (#586), so Horizon never pulls fast players back the way vanilla does. + +## What it shows + +- **1 to 10 players:** Horizon ticks in about half of vanilla's time with less than half its memory. That + holds in both behaviours. +- **25 walking players: Horizon collapses and vanilla doesn't.** A 20-second `sample` of the main thread + during a separate 25-bot `walk` run (MSPT 1464 ms) found the cause. 96 % of the server thread was inside + `Hub::tick` → `ChunkEncoder::encode_in` (`crates/horizon-client/src/lan/chunks.rs`), encoding chunk + packets for the bots: + - `Paletted::from_values` re-palettes every section from the unpacked values; + - `PalettedContainer::unpack_u16` is called, among other places, by `occupied()`, which unpacks the 24 + sections of each of the 9 neighbouring columns for every chunk sent; + - `light_data` runs twice per chunk (`encode` computes it, and `encode_in` overwrites it); + - `client_heightmaps` is computed again for each player. + + Every chunk is encoded again for every player who receives it. At about 200 chunks/s, that alone fills the + tick. +- **Memory:** Horizon's RSS stayed at 1.6–2.7 GB in every run, against 3.5–4.6 GB for vanilla with its + 4 GB heap. + +## Files + +- `walk.json`, `explore.json`: the `horizon-server-benchmark` reports. +- `load.txt`: the machine's load average and CPU use every 30 s, with the share taken by agents. diff --git a/benchmarks/server-2026-10-17/explore.json b/benchmarks/server-2026-10-17/explore.json new file mode 100644 index 000000000..9995ba738 --- /dev/null +++ b/benchmarks/server-2026-10-17/explore.json @@ -0,0 +1,212 @@ +{ + "kind": "horizon-server-benchmark", + "date": "2026-10-17T08:36:03+00:00", + "machine": { + "system": "Darwin", + "machine": "arm64", + "cpus": 14 + }, + "commit": "123987c4591b8209cc3e7df495482298803bbd9a", + "java": [ + "openjdk 25.0.4.1 2026-08-18 LTS" + ], + "settings": { + "servers": "horizon,vanilla", + "bots": "1,10,25,50,100", + "behaviour": "explore", + "join_every": 1.0, + "warmup": 60, + "seconds": 180, + "seed": 12345, + "heap": "4g", + "out": "explore.json", + "properties": { + "level-seed": "12345", + "view-distance": "10", + "simulation-distance": "10", + "online-mode": "false", + "white-list": "false", + "gamemode": "creative", + "force-gamemode": "true", + "spawn-protection": "0", + "max-players": "200", + "difficulty": "easy", + "sync-chunk-writes": "true" + } + }, + "runs": [ + { + "server": "horizon", + "bots": 1, + "start_seconds": 14.21, + "mspt": { + "average": 11.56, + "p50": 3.73, + "p95": 50.42, + "p99": 122.65 + }, + "mspt_worst_p99": 317.7, + "tps": 20.0, + "rss_mb": { + "mean": 1686.0, + "max": 1730.8 + }, + "cpu_percent": 119.52, + "chunks_per_second": 44.48, + "bots_joined": 1, + "bots_failed": [], + "tick_queries": 32 + }, + { + "server": "vanilla", + "bots": 1, + "start_seconds": 36.45, + "mspt": { + "average": 15.93, + "p50": 9.44, + "p95": 51.1, + "p99": 110.42 + }, + "mspt_worst_p99": 318.2, + "tps": 20.0, + "rss_mb": { + "mean": 3515.74, + "max": 3579.8 + }, + "cpu_percent": 92.34, + "chunks_per_second": 32.26, + "bots_joined": 1, + "bots_failed": [], + "tick_queries": 31 + }, + { + "server": "horizon", + "bots": 10, + "start_seconds": 26.41, + "mspt": { + "average": 8.23, + "p50": 5.34, + "p95": 25.13, + "p99": 47.67 + }, + "mspt_worst_p99": 127.9, + "tps": 20.0, + "rss_mb": { + "mean": 2029.74, + "max": 2151.8 + }, + "cpu_percent": 274.97, + "chunks_per_second": 10.82, + "bots_joined": 10, + "bots_failed": [], + "tick_queries": 33 + }, + { + "server": "vanilla", + "bots": 10, + "start_seconds": 20.36, + "mspt": { + "average": 30.76, + "p50": 10.81, + "p95": 121.47, + "p99": 281.86 + }, + "mspt_worst_p99": 753.1, + "tps": 20.0, + "rss_mb": { + "mean": 4616.8, + "max": 4873.9 + }, + "cpu_percent": 168.65, + "chunks_per_second": 41.36, + "bots_joined": 10, + "bots_failed": [], + "tick_queries": 32 + }, + { + "server": "horizon", + "bots": 25, + "start_seconds": 17.82, + "mspt": { + "average": 14.61, + "p50": 10.07, + "p95": 43.57, + "p99": 80.55 + }, + "mspt_worst_p99": 127.2, + "tps": 20.0, + "rss_mb": { + "mean": 1936.5, + "max": 1996.0 + }, + "cpu_percent": 230.76, + "chunks_per_second": 0.0, + "bots_joined": 25, + "bots_failed": [], + "tick_queries": 32 + }, + { + "server": "vanilla", + "bots": 25, + "start_seconds": 30.92, + "error": "[Errno 32] Broken pipe" + }, + { + "server": "horizon", + "bots": 50, + "start_seconds": 23.74, + "mspt": { + "average": 72.2, + "p50": 57.53, + "p95": 207.54, + "p99": 284.82 + }, + "mspt_worst_p99": 399.2, + "tps": 13.85, + "rss_mb": { + "mean": 2694.26, + "max": 2781.4 + }, + "cpu_percent": 162.62, + "chunks_per_second": 0.0, + "bots_joined": 50, + "bots_failed": [], + "tick_queries": 32 + }, + { + "server": "vanilla", + "bots": 50, + "start_seconds": 35.21, + "error": "[Errno 32] Broken pipe" + }, + { + "server": "horizon", + "bots": 100, + "start_seconds": 25.69, + "mspt": { + "average": 239.02, + "p50": 245.93, + "p95": 442.44, + "p99": 581.13 + }, + "mspt_worst_p99": 776.5, + "tps": 4.18, + "rss_mb": { + "mean": 1835.34, + "max": 1838.4 + }, + "cpu_percent": 106.19, + "chunks_per_second": 0.0, + "bots_joined": 100, + "bots_failed": [], + "tick_queries": 32 + }, + { + "server": "vanilla", + "bots": 100, + "start_seconds": 36.46, + "error": "[Errno 32] Broken pipe" + } + ], + "worktree_commit": "171fbeb8ac4877109fc95e93c5db1b0f2360652b" +} \ No newline at end of file diff --git a/benchmarks/server-2026-10-17/load.txt b/benchmarks/server-2026-10-17/load.txt new file mode 100644 index 000000000..aaeddf9f3 --- /dev/null +++ b/benchmarks/server-2026-10-17/load.txt @@ -0,0 +1,181 @@ +2026-10-17T07:02:41Z load 64.69 cpu 80% busy (agents 4.2, rest 3.9 cores) +2026-10-17T07:03:12Z load 66.17 cpu 93% busy (agents 3.5, rest 9.6 cores) +2026-10-17T07:03:43Z load 65.37 cpu 98% busy (agents 3.5, rest 7.1 cores) +2026-10-17T07:04:14Z load 61.95 cpu 79% busy (agents 5.4, rest 6.0 cores) +2026-10-17T07:04:44Z load 65.21 cpu 78% busy (agents 4.7, rest 6.1 cores) +2026-10-17T07:05:14Z load 77.01 cpu 86% busy (agents 3.0, rest 9.2 cores) +2026-10-17T07:05:44Z load 71.26 cpu 91% busy (agents 2.9, rest 9.7 cores) +2026-10-17T07:06:16Z load 62.39 cpu 80% busy (agents 3.9, rest 7.1 cores) +2026-10-17T07:06:47Z load 68.72 cpu 94% busy (agents 5.0, rest 8.2 cores) +2026-10-17T07:07:18Z load 74.78 cpu 94% busy (agents 5.6, rest 7.2 cores) +2026-10-17T07:07:50Z load 80.02 cpu 85% busy (agents 4.8, rest 8.5 cores) +2026-10-17T07:08:22Z load 78.84 cpu 88% busy (agents 5.8, rest 7.0 cores) +2026-10-17T07:08:52Z load 66.27 cpu 93% busy (agents 7.7, rest 5.6 cores) +2026-10-17T07:09:22Z load 60.90 cpu 93% busy (agents 9.9, rest 4.5 cores) +2026-10-17T07:09:52Z load 54.38 cpu 91% busy (agents 4.0, rest 8.5 cores) +2026-10-17T07:10:23Z load 51.89 cpu 72% busy (agents 4.8, rest 4.9 cores) +2026-10-17T07:10:53Z load 54.13 cpu 67% busy (agents 3.6, rest 4.9 cores) +2026-10-17T07:11:24Z load 64.71 cpu 80% busy (agents 7.6, rest 5.4 cores) +2026-10-17T07:11:55Z load 68.80 cpu 79% busy (agents 8.4, rest 3.1 cores) +2026-10-17T07:12:26Z load 63.15 cpu 65% busy (agents 4.1, rest 7.0 cores) +2026-10-17T07:12:57Z load 53.85 cpu 92% busy (agents 8.5, rest 4.3 cores) +2026-10-17T07:13:27Z load 48.25 cpu 85% busy (agents 5.2, rest 5.0 cores) +2026-10-17T07:13:57Z load 44.37 cpu 83% busy (agents 5.4, rest 7.7 cores) +2026-10-17T07:14:27Z load 55.72 cpu 84% busy (agents 6.2, rest 5.0 cores) +2026-10-17T07:14:58Z load 59.28 cpu 85% busy (agents 7.5, rest 4.2 cores) +2026-10-17T07:15:29Z load 56.33 cpu 75% busy (agents 7.8, rest 2.6 cores) +2026-10-17T07:15:59Z load 56.73 cpu 86% busy (agents 7.1, rest 3.9 cores) +2026-10-17T07:16:29Z load 55.69 cpu 87% busy (agents 8.9, rest 4.3 cores) +2026-10-17T07:17:00Z load 51.02 cpu 84% busy (agents 5.9, rest 6.2 cores) +2026-10-17T07:17:30Z load 52.09 cpu 87% busy (agents 8.7, rest 3.7 cores) +2026-10-17T07:18:00Z load 51.21 cpu 92% busy (agents 6.4, rest 6.4 cores) +2026-10-17T07:18:30Z load 54.91 cpu 66% busy (agents 2.7, rest 7.1 cores) +2026-10-17T07:19:01Z load 56.12 cpu 66% busy (agents 7.1, rest 3.1 cores) +2026-10-17T07:19:32Z load 59.15 cpu 79% busy (agents 6.3, rest 4.1 cores) +2026-10-17T07:20:02Z load 64.32 cpu 83% busy (agents 8.2, rest 3.5 cores) +2026-10-17T07:20:32Z load 66.42 cpu 76% busy (agents 7.1, rest 3.9 cores) +2026-10-17T07:21:02Z load 66.45 cpu 81% busy (agents 5.5, rest 6.1 cores) +2026-10-17T07:21:33Z load 70.55 cpu 82% busy (agents 7.0, rest 4.7 cores) +2026-10-17T07:22:03Z load 72.95 cpu 85% busy (agents 4.8, rest 6.6 cores) +2026-10-17T07:22:33Z load 75.48 cpu 95% busy (agents 4.8, rest 8.2 cores) +2026-10-17T07:23:04Z load 75.45 cpu 48% busy (agents 3.0, rest 5.5 cores) +2026-10-17T07:23:34Z load 75.99 cpu 82% busy (agents 3.6, rest 6.9 cores) +2026-10-17T07:24:06Z load 72.16 cpu 86% busy (agents 4.2, rest 7.2 cores) +2026-10-17T07:24:37Z load 85.91 cpu 68% busy (agents 2.3, rest 7.3 cores) +2026-10-17T07:25:09Z load 109.60 cpu 59% busy (agents 2.1, rest 5.9 cores) +2026-10-17T07:25:41Z load 100.52 cpu 92% busy (agents 4.6, rest 7.3 cores) +2026-10-17T07:26:12Z load 87.53 cpu 71% busy (agents 5.5, rest 4.2 cores) +2026-10-17T07:26:42Z load 80.26 cpu 91% busy (agents 8.6, rest 4.0 cores) +2026-10-17T07:27:12Z load 79.82 cpu 90% busy (agents 5.6, rest 6.0 cores) +2026-10-17T07:27:43Z load 81.69 cpu 83% busy (agents 6.5, rest 5.3 cores) +2026-10-17T07:28:13Z load 96.47 cpu 81% busy (agents 8.0, rest 5.4 cores) +2026-10-17T07:28:44Z load 92.13 cpu 87% busy (agents 3.4, rest 9.7 cores) +2026-10-17T07:29:14Z load 75.93 cpu 80% busy (agents 4.3, rest 7.4 cores) +2026-10-17T07:29:44Z load 76.33 cpu 85% busy (agents 3.6, rest 8.0 cores) +2026-10-17T07:30:15Z load 77.18 cpu 69% busy (agents 2.0, rest 9.5 cores) +2026-10-17T07:30:46Z load 91.47 cpu 72% busy (agents 3.1, rest 9.0 cores) +2026-10-17T07:31:17Z load 97.16 cpu 75% busy (agents 2.5, rest 7.7 cores) +2026-10-17T07:31:48Z load 97.59 cpu 75% busy (agents 1.9, rest 9.1 cores) +2026-10-17T07:32:20Z load 91.54 cpu 83% busy (agents 1.2, rest 10.0 cores) +2026-10-17T07:32:51Z load 90.94 cpu 61% busy (agents 2.5, rest 7.8 cores) +2026-10-17T07:33:22Z load 100.43 cpu 81% busy (agents 2.1, rest 8.8 cores) +2026-10-17T07:33:53Z load 123.45 cpu 76% busy (agents 3.9, rest 7.3 cores) +2026-10-17T07:34:24Z load 139.49 cpu 85% busy (agents 3.3, rest 9.0 cores) +2026-10-17T07:34:56Z load 157.36 cpu 91% busy (agents 3.5, rest 9.5 cores) +2026-10-17T07:35:29Z load 161.48 cpu 82% busy (agents 4.5, rest 7.0 cores) +2026-10-17T07:36:02Z load 143.38 cpu 77% busy (agents 5.9, rest 5.4 cores) +2026-10-17T07:36:32Z load 135.93 cpu 74% busy (agents 5.3, rest 5.6 cores) +2026-10-17T07:37:03Z load 134.20 cpu 78% busy (agents 6.1, rest 5.4 cores) +2026-10-17T07:37:34Z load 131.11 cpu 80% busy (agents 3.4, rest 6.7 cores) +2026-10-17T07:38:05Z load 135.55 cpu 73% busy (agents 3.4, rest 7.4 cores) +2026-10-17T07:38:36Z load 139.04 cpu 77% busy (agents 2.8, rest 8.1 cores) +2026-10-17T07:39:07Z load 146.25 cpu 84% busy (agents 2.5, rest 8.6 cores) +2026-10-17T07:39:38Z load 153.62 cpu 86% busy (agents 4.8, rest 7.3 cores) +2026-10-17T07:40:09Z load 143.04 cpu 71% busy (agents 5.5, rest 5.8 cores) +2026-10-17T07:40:40Z load 140.93 cpu 81% busy (agents 4.5, rest 6.8 cores) +2026-10-17T07:41:10Z load 135.26 cpu 87% busy (agents 3.9, rest 6.7 cores) +2026-10-17T07:41:42Z load 130.29 cpu 79% busy (agents 2.5, rest 7.9 cores) +2026-10-17T07:42:12Z load 122.54 cpu 85% busy (agents 2.4, rest 8.3 cores) +2026-10-17T07:42:43Z load 129.43 cpu 82% busy (agents 3.5, rest 9.0 cores) +2026-10-17T07:43:15Z load 124.15 cpu 81% busy (agents 3.1, rest 8.4 cores) +2026-10-17T07:43:46Z load 112.18 cpu 84% busy (agents 3.0, rest 8.3 cores) +2026-10-17T07:44:17Z load 104.49 cpu 83% busy (agents 4.4, rest 8.2 cores) +2026-10-17T07:44:51Z load 101.11 cpu 77% busy (agents 3.8, rest 8.6 cores) +2026-10-17T07:45:22Z load 88.91 cpu 86% busy (agents 5.3, rest 6.2 cores) +2026-10-17T07:45:52Z load 75.92 cpu 84% busy (agents 4.9, rest 7.3 cores) +2026-10-17T07:46:23Z load 71.83 cpu 89% busy (agents 3.9, rest 7.2 cores) +2026-10-17T07:46:54Z load 75.07 cpu 88% busy (agents 5.4, rest 7.7 cores) +2026-10-17T07:47:25Z load 78.04 cpu 86% busy (agents 2.7, rest 7.8 cores) +2026-10-17T07:47:56Z load 75.74 cpu 89% busy (agents 3.8, rest 7.7 cores) +2026-10-17T07:48:26Z load 79.47 cpu 85% busy (agents 3.0, rest 9.2 cores) +2026-10-17T07:48:57Z load 94.67 cpu 72% busy (agents 1.1, rest 8.1 cores) +2026-10-17T07:49:29Z load 108.05 cpu 87% busy (agents 4.5, rest 8.2 cores) +2026-10-17T07:50:00Z load 95.52 cpu 90% busy (agents 6.8, rest 6.6 cores) +2026-10-17T07:50:31Z load 86.78 cpu 93% busy (agents 3.5, rest 8.6 cores) +2026-10-17T07:51:01Z load 84.06 cpu 93% busy (agents 4.7, rest 8.2 cores) +2026-10-17T07:51:32Z load 82.29 cpu 94% busy (agents 6.2, rest 7.6 cores) +2026-10-17T07:52:02Z load 87.70 cpu 92% busy (agents 3.6, rest 7.7 cores) +2026-10-17T07:52:33Z load 87.61 cpu 94% busy (agents 2.4, rest 8.7 cores) +2026-10-17T07:53:05Z load 98.97 cpu 73% busy (agents 4.1, rest 7.2 cores) +2026-10-17T07:53:37Z load 102.02 cpu 89% busy (agents 5.6, rest 7.5 cores) +2026-10-17T07:54:09Z load 106.62 cpu 87% busy (agents 4.9, rest 7.6 cores) +2026-10-17T07:54:40Z load 119.40 cpu 71% busy (agents 4.0, rest 6.1 cores) +2026-10-17T07:55:12Z load 129.93 cpu 87% busy (agents 5.1, rest 6.4 cores) +2026-10-17T07:55:44Z load 130.60 cpu 89% busy (agents 5.3, rest 7.6 cores) +2026-10-17T07:56:15Z load 128.56 cpu 88% busy (agents 3.4, rest 9.0 cores) +2026-10-17T07:56:47Z load 119.80 cpu 86% busy (agents 4.5, rest 7.8 cores) +2026-10-17T07:57:19Z load 109.94 cpu 90% busy (agents 4.2, rest 8.4 cores) +2026-10-17T07:57:50Z load 105.59 cpu 89% busy (agents 3.7, rest 8.6 cores) +2026-10-17T07:58:21Z load 95.04 cpu 86% busy (agents 3.6, rest 9.2 cores) +2026-10-17T07:58:52Z load 82.20 cpu 90% busy (agents 4.2, rest 8.6 cores) +2026-10-17T07:59:24Z load 84.72 cpu 86% busy (agents 3.0, rest 8.4 cores) +2026-10-17T07:59:55Z load 92.43 cpu 91% busy (agents 3.5, rest 8.8 cores) +2026-10-17T08:00:26Z load 82.76 cpu 80% busy (agents 3.0, rest 8.6 cores) +2026-10-17T08:00:57Z load 83.25 cpu 89% busy (agents 2.5, rest 9.3 cores) +2026-10-17T08:01:28Z load 87.05 cpu 88% busy (agents 2.7, rest 9.3 cores) +2026-10-17T08:01:59Z load 85.58 cpu 90% busy (agents 2.8, rest 9.2 cores) +2026-10-17T08:02:30Z load 76.00 cpu 81% busy (agents 2.4, rest 8.4 cores) +2026-10-17T08:03:01Z load 68.58 cpu 83% busy (agents 4.0, rest 7.6 cores) +2026-10-17T08:03:32Z load 71.30 cpu 85% busy (agents 2.2, rest 9.0 cores) +2026-10-17T08:04:03Z load 70.07 cpu 84% busy (agents 1.9, rest 9.1 cores) +2026-10-17T08:04:33Z load 69.29 cpu 80% busy (agents 5.1, rest 7.3 cores) +2026-10-17T08:05:03Z load 63.54 cpu 75% busy (agents 3.2, rest 8.0 cores) +2026-10-17T08:05:34Z load 71.88 cpu 83% busy (agents 1.9, rest 9.5 cores) +2026-10-17T08:06:05Z load 68.99 cpu 87% busy (agents 3.0, rest 8.8 cores) +2026-10-17T08:06:36Z load 70.77 cpu 84% busy (agents 1.4, rest 10.1 cores) +2026-10-17T08:07:07Z load 64.76 cpu 87% busy (agents 3.0, rest 8.4 cores) +2026-10-17T08:07:37Z load 68.91 cpu 87% busy (agents 2.3, rest 9.8 cores) +2026-10-17T08:08:08Z load 60.76 cpu 78% busy (agents 2.3, rest 7.3 cores) +2026-10-17T08:08:39Z load 61.50 cpu 74% busy (agents 1.8, rest 8.9 cores) +2026-10-17T08:09:09Z load 61.58 cpu 77% busy (agents 3.4, rest 7.5 cores) +2026-10-17T08:09:40Z load 56.41 cpu 79% busy (agents 2.8, rest 8.3 cores) +2026-10-17T08:10:11Z load 66.65 cpu 84% busy (agents 3.2, rest 8.9 cores) +2026-10-17T08:10:42Z load 71.19 cpu 77% busy (agents 1.8, rest 8.4 cores) +2026-10-17T08:11:13Z load 81.45 cpu 88% busy (agents 1.8, rest 10.0 cores) +2026-10-17T08:11:45Z load 76.88 cpu 84% busy (agents 1.7, rest 9.6 cores) +2026-10-17T08:12:15Z load 78.99 cpu 85% busy (agents 4.2, rest 8.2 cores) +2026-10-17T08:12:47Z load 83.76 cpu 85% busy (agents 3.7, rest 8.2 cores) +2026-10-17T08:13:19Z load 82.81 cpu 92% busy (agents 3.0, rest 9.4 cores) +2026-10-17T08:13:51Z load 82.56 cpu 91% busy (agents 2.7, rest 9.2 cores) +2026-10-17T08:14:22Z load 82.13 cpu 86% busy (agents 3.4, rest 9.5 cores) +2026-10-17T08:14:53Z load 83.63 cpu 82% busy (agents 3.6, rest 8.8 cores) +2026-10-17T08:15:25Z load 85.15 cpu 85% busy (agents 3.1, rest 8.8 cores) +2026-10-17T08:15:56Z load 83.50 cpu 84% busy (agents 2.9, rest 9.5 cores) +2026-10-17T08:16:28Z load 89.84 cpu 90% busy (agents 1.5, rest 10.9 cores) +2026-10-17T08:17:00Z load 85.24 cpu 96% busy (agents 4.0, rest 9.6 cores) +2026-10-17T08:17:32Z load 79.73 cpu 93% busy (agents 3.6, rest 9.0 cores) +2026-10-17T08:18:03Z load 68.70 cpu 90% busy (agents 3.0, rest 9.7 cores) +2026-10-17T08:18:33Z load 67.27 cpu 89% busy (agents 2.6, rest 10.1 cores) +2026-10-17T08:19:05Z load 70.63 cpu 84% busy (agents 2.1, rest 10.3 cores) +2026-10-17T08:19:36Z load 80.50 cpu 88% busy (agents 2.3, rest 9.0 cores) +2026-10-17T08:20:08Z load 93.51 cpu 90% busy (agents 4.1, rest 8.8 cores) +2026-10-17T08:20:39Z load 96.52 cpu 83% busy (agents 2.7, rest 9.3 cores) +2026-10-17T08:21:10Z load 92.06 cpu 90% busy (agents 3.2, rest 9.4 cores) +2026-10-17T08:21:41Z load 92.15 cpu 88% busy (agents 3.6, rest 8.8 cores) +2026-10-17T08:22:12Z load 86.95 cpu 85% busy (agents 3.9, rest 8.0 cores) +2026-10-17T08:22:44Z load 89.42 cpu 91% busy (agents 5.0, rest 8.2 cores) +2026-10-17T08:23:15Z load 83.03 cpu 89% busy (agents 4.4, rest 7.8 cores) +2026-10-17T08:23:46Z load 76.29 cpu 78% busy (agents 2.6, rest 9.1 cores) +2026-10-17T08:24:17Z load 85.85 cpu 85% busy (agents 3.8, rest 8.4 cores) +2026-10-17T08:24:49Z load 91.09 cpu 85% busy (agents 3.0, rest 9.3 cores) +2026-10-17T08:25:20Z load 101.77 cpu 88% busy (agents 4.5, rest 8.1 cores) +2026-10-17T08:25:52Z load 111.52 cpu 90% busy (agents 3.4, rest 8.6 cores) +2026-10-17T08:26:24Z load 98.44 cpu 87% busy (agents 4.2, rest 8.2 cores) +2026-10-17T08:26:55Z load 92.66 cpu 89% busy (agents 3.9, rest 8.9 cores) +2026-10-17T08:27:26Z load 90.51 cpu 89% busy (agents 3.8, rest 8.6 cores) +2026-10-17T08:27:58Z load 94.91 cpu 84% busy (agents 4.2, rest 8.3 cores) +2026-10-17T08:28:29Z load 112.87 cpu 87% busy (agents 4.0, rest 8.3 cores) +2026-10-17T08:29:01Z load 119.21 cpu 88% busy (agents 3.2, rest 8.9 cores) +2026-10-17T08:29:33Z load 124.83 cpu 90% busy (agents 2.9, rest 9.1 cores) +2026-10-17T08:30:06Z load 135.92 cpu 83% busy (agents 3.6, rest 8.8 cores) +2026-10-17T08:30:39Z load 141.27 cpu 88% busy (agents 3.8, rest 8.7 cores) +2026-10-17T08:31:11Z load 138.41 cpu 87% busy (agents 2.5, rest 9.3 cores) +2026-10-17T08:31:44Z load 146.34 cpu 85% busy (agents 4.2, rest 8.3 cores) +2026-10-17T08:32:16Z load 143.47 cpu 90% busy (agents 4.7, rest 8.0 cores) +2026-10-17T08:32:48Z load 146.37 cpu 90% busy (agents 3.6, rest 8.7 cores) +2026-10-17T08:33:20Z load 138.78 cpu 90% busy (agents 2.0, rest 8.9 cores) +2026-10-17T08:33:51Z load 133.00 cpu 87% busy (agents 3.4, rest 9.1 cores) +2026-10-17T08:34:23Z load 140.84 cpu 89% busy (agents 3.1, rest 8.7 cores) +2026-10-17T08:34:55Z load 149.49 cpu 86% busy (agents 5.1, rest 7.8 cores) +2026-10-17T08:35:27Z load 144.90 cpu 87% busy (agents 4.5, rest 8.7 cores) +2026-10-17T08:35:58Z load 134.49 cpu 84% busy (agents 3.6, rest 9.3 cores) diff --git a/benchmarks/server-2026-10-17/walk.json b/benchmarks/server-2026-10-17/walk.json new file mode 100644 index 000000000..7198b70cf --- /dev/null +++ b/benchmarks/server-2026-10-17/walk.json @@ -0,0 +1,228 @@ +{ + "kind": "horizon-server-benchmark", + "date": "2026-10-17T07:50:18+00:00", + "machine": { + "system": "Darwin", + "machine": "arm64", + "cpus": 14 + }, + "commit": "123987c4591b8209cc3e7df495482298803bbd9a", + "java": [ + "openjdk 25.0.4.1 2026-08-18 LTS" + ], + "settings": { + "servers": "horizon,vanilla", + "bots": "1,10,25,50,100", + "behaviour": "walk", + "join_every": 1.0, + "warmup": 60, + "seconds": 180, + "seed": 12345, + "heap": "4g", + "out": "walk.json", + "properties": { + "level-seed": "12345", + "view-distance": "10", + "simulation-distance": "10", + "online-mode": "false", + "white-list": "false", + "gamemode": "creative", + "force-gamemode": "true", + "spawn-protection": "0", + "max-players": "200", + "difficulty": "easy", + "sync-chunk-writes": "true" + } + }, + "runs": [ + { + "server": "horizon", + "bots": 1, + "start_seconds": 22.91, + "mspt": { + "average": 4.93, + "p50": 1.72, + "p95": 23.33, + "p99": 86.25 + }, + "mspt_worst_p99": 475.7, + "tps": 20.0, + "rss_mb": { + "mean": 1628.87, + "max": 1633.1 + }, + "cpu_percent": 18.17, + "chunks_per_second": 10.48, + "bots_joined": 1, + "bots_failed": [], + "tick_queries": 33 + }, + { + "server": "vanilla", + "bots": 1, + "start_seconds": 33.89, + "mspt": { + "average": 13.41, + "p50": 9.5, + "p95": 41.09, + "p99": 84.06 + }, + "mspt_worst_p99": 208.2, + "tps": 20.0, + "rss_mb": { + "mean": 3469.62, + "max": 3486.7 + }, + "cpu_percent": 26.33, + "chunks_per_second": 10.06, + "bots_joined": 1, + "bots_failed": [], + "tick_queries": 33 + }, + { + "server": "horizon", + "bots": 10, + "start_seconds": 12.33, + "mspt": { + "average": 11.47, + "p50": 2.65, + "p95": 56.68, + "p99": 138.06 + }, + "mspt_worst_p99": 473.0, + "tps": 20.0, + "rss_mb": { + "mean": 1572.0, + "max": 1573.6 + }, + "cpu_percent": 37.64, + "chunks_per_second": 101.51, + "bots_joined": 10, + "bots_failed": [], + "tick_queries": 34 + }, + { + "server": "vanilla", + "bots": 10, + "start_seconds": 15.49, + "mspt": { + "average": 21.16, + "p50": 12.0, + "p95": 70.48, + "p99": 111.87 + }, + "mspt_worst_p99": 251.0, + "tps": 20.0, + "rss_mb": { + "mean": 3464.16, + "max": 3551.2 + }, + "cpu_percent": 49.3, + "chunks_per_second": 93.68, + "bots_joined": 10, + "bots_failed": [], + "tick_queries": 34 + }, + { + "server": "horizon", + "bots": 25, + "start_seconds": 20.8, + "mspt": { + "average": 1115.2, + "p50": 973.35, + "p95": 2720.07, + "p99": 4512.68 + }, + "mspt_worst_p99": 6442.0, + "tps": 0.9, + "rss_mb": { + "mean": 1604.66, + "max": 1616.6 + }, + "cpu_percent": 45.25, + "chunks_per_second": 200.94, + "bots_joined": 25, + "bots_failed": [], + "tick_queries": 33 + }, + { + "server": "vanilla", + "bots": 25, + "start_seconds": 29.8, + "mspt": { + "average": 38.8, + "p50": 15.39, + "p95": 147.7, + "p99": 250.31 + }, + "mspt_worst_p99": 605.5, + "tps": 20.0, + "rss_mb": { + "mean": 3581.43, + "max": 3633.0 + }, + "cpu_percent": 68.73, + "chunks_per_second": 255.15, + "bots_joined": 25, + "bots_failed": [], + "tick_queries": 33 + }, + { + "server": "horizon", + "bots": 50, + "start_seconds": 19.09, + "mspt": { + "average": 2192.19, + "p50": 365.02, + "p95": 11159.22, + "p99": 14168.57 + }, + "mspt_worst_p99": 14322.9, + "tps": 0.46, + "rss_mb": { + "mean": 1711.49, + "max": 1745.4 + }, + "cpu_percent": 23.9, + "chunks_per_second": 125.7, + "bots_joined": 50, + "bots_failed": [], + "tick_queries": 11 + }, + { + "server": "vanilla", + "bots": 50, + "start_seconds": 34.71, + "error": "[Errno 32] Broken pipe" + }, + { + "server": "horizon", + "bots": 100, + "start_seconds": 24.26, + "mspt": { + "average": 2638.4, + "p50": 407.98, + "p95": 12928.61, + "p99": 17095.1 + }, + "mspt_worst_p99": 17095.1, + "tps": 0.38, + "rss_mb": { + "mean": 1906.04, + "max": 1921.0 + }, + "cpu_percent": 45.97, + "chunks_per_second": 221.96, + "bots_joined": 100, + "bots_failed": [], + "tick_queries": 9 + }, + { + "server": "vanilla", + "bots": 100, + "start_seconds": 23.3, + "error": "[Errno 32] Broken pipe" + } + ], + "worktree_commit": "57ee531c639e1e899d63a786fd878ad9cb1c76e4" +} \ No newline at end of file