docs: record the completed ten-agent optimization sequence

This commit is contained in:
rdbtCVS
2026-10-22 17:06:09 +01:00
parent 753c071c37
commit c9c7cda779
+44
View File
@@ -2,6 +2,20 @@
Profiling baseline: `41055168afcd8687c7efe525545de896f17f4478` (2026-10-22). No production optimization is included in this profiling step. Measurements identify work to investigate; they do not establish a candidate speedup or complete Minecraft world parity.
## Completed sequence: ten agents
The ten sequential GPT-6.1 Sol/high agents are complete. Three production changes are retained in independent pull requests:
| Change | Measured complete-workload reduction | Pull request |
| --- | --- | --- |
| Saved list-palette loading (Round 2) | 10.257% normal saved load | [#1223](https://github.com/ludovic111/horizon/pull/1223) |
| Aquifer neighborhood reuse (Rounds 3 and 7) | Primary generation + decoration: 4.152% / 2.377% at one/four workers | [#1229](https://github.com/ludovic111/horizon/pull/1229) |
| Saved-light reconstruction (Round 8) | 6.179% normal / 10.981% flat saved load, incremental to palette loading | [#1231](https://github.com/ludovic111/horizon/pull/1231) |
These paired-median reductions have different workloads/baselines and must not be added together or treated as total game startup improvements. Retained changes passed their scoped pinned vanilla oracles, exact baseline/candidate snapshots, and repository quick self-checks. The final retained source passed 1,443 tests with serialization after an initial client-test crash. No universal world/gameplay parity claim is made.
Surface dispatch, section access and NBT feasibility stopped early. The state-cache key candidate was reverted below the default 5% complete-load gate. Aquifer work resumed under the user's explicit lower threshold, then passed full qualification. The final decompression attempt passed the main loading gate but was reverted because related valid-stream workloads regressed. All rejected production/dependency changes are restored; no eleventh agent was started. Further optimization awaits user feedback.
## Reproduction and artifacts
Machine: AMD Ryzen 5 5600X, 6 cores/12 logical CPUs, 32 GiB RAM, Linux 7.2.11-zen1-1-zen x86_64; Rust/Cargo 1.99.0, OpenJDK 25.0.4.1. CPU boost remained enabled, governor `powersave`; normal desktop background activity was not isolated. Builds, timed workloads and sampling runs were sequential. Release uses `opt-level=3`, thin LTO, one codegen unit; client benchmarks disable audio/controllers and need no window/GPU. Do not compare these numbers directly with earlier `fast` profile reports.
@@ -140,3 +154,33 @@ Evidence is retained under `/srv/bob-data/horizon-chunk-campaign-41055168/state-
One discarded process pair followed by two alternating measured pairs used `--rounds 5 --limit 256 --checksum`. Complete normal-load paired reductions were **3.257% and 2.412%** (median **2.835%**); baseline/candidate process-median medians were **49.725 / 48.316 ms** per 256 chunks. Native decode improved **5.917%** (20.266 / 19.067 ms); inflate/NBT varied by **-0.844%**. Compressed-record and block/biome/light checksums matched in every process. Those checksums are limited benchmark checks, not comprehensive semantic or vanilla parity evidence.
Decision: **early no-go** at the incremental 5% complete-load gate. Both production files were restored exactly to their pre-candidate contents; only this report remains. No flat/long qualification, comprehensive snapshot comparison, fresh parity suite, tests, commit or PR were pursued for this rejected candidate. The shared Cargo target contains the rejected executable; future work must use the preserved accepted binary or rebuild restored sources.
## Round 7: aquifer qualification — retained
The user explicitly accepted the aquifer pilot's smaller gain and later confirmed the repeated 3.3–3.5% observations were sufficient. The candidate was restored and qualified with five alternating process pairs per workload, startup/streaming/memory checks, fresh client/server aquifer/terrain/material oracles, and 36 exact pre/post-decoration snapshot comparisons. All 20 measured Overworld pairs improve; no material regression was established in the declared checks. The cache adds 312 bytes per transient aquifer object with no new allocation calls. Full results, limitations and reproduction are in [PR #1229](https://github.com/ludovic111/horizon/pull/1229).
## Round 8: saved-light loading — retained
Detect uniform saved nibble layers before allocation and decode remaining layers in byte pairs. Final-source complete loading improves by paired medians 6.179% normal and 10.981% flat, incremental to the accepted palette optimization. All ten measured pairs improve. All 691 full metadata snapshots and 50,331,648 fresh pinned Java light values match; the repository quick self-check passes with 1,443 tests. The separate native-decode loop measured slower despite unchanged source; its cause remains unisolated and is disclosed in [PR #1231](https://github.com/ludovic111/horizon/pull/1231).
## Round 9: NBT payload and modified-UTF fast paths — early no-go
Inspected the unchanged NBT reader at accepted HEAD `fb2be075` and its actual release assembly in the preserved accepted saved-light executable; captured pinned 26.3 integer-array/JDK modified-UTF bytecode for evidence. Checked the open-PR catalog on 2026-10-22; no NBT-reader optimization overlap was found. Two concrete candidates were bulk big-endian integer-array decoding after one checked slice, avoiding repeated per-element bounds/offset/push checks, and an ASCII widening path before the existing lossless modified-UTF decoder. The latter must retain raw NUL, accepted overlong sequences and unpaired UTF-16 surrogates; arrays must retain count/byte-limit/truncation errors and offsets.
Decision: **early no-go before implementation**. The valid original mixed-loader profile attributes 4.08% exclusive to all `Reader::payload` work and 2.96% exclusive to all `Reader::string` work; NBT runs in both complete-load and inflate/NBT loops. These are not complete-load shares or strict speedup ceilings. Array decoding affects only part of payload work, while ASCII widening retains string allocation and map construction. Neither narrow candidate offers a credible incremental 5% complete `SaveDir::load` reduction; combining them would still require unusually large savings across both functions. The accepted palette and saved-light improvements are already in HEAD and are not counted again. No production edits, builds, timing runs, tests, parity jobs, candidate, commit or PR were made; no new performance/correctness result is claimed.
Bounded evidence is outside Git at `/srv/bob-data/horizon-chunk-campaign-41055168/nbt-feasibility/`: source/accepted-binary hashes, assembly, pinned bytecode and the open-PR catalog. NBT source SHA256 remains `a32fd0ac8fe94ab1b23c000b51c3a81b28d40c518c5befb5272818792d1cf1ac`. The separate decompression lead for Round 10 is the existing profile's `inflate_fast_help_avx2` at 15.71% exclusive; no backend investigation was performed in this round.
## Round 10: whole-buffer zlib decompression — rejected
Inspected the actual region decoder (`crates/horizon-world/src/compression.rs`), its flate2/zlib-rs backend, current open PRs and the official libdeflate/libdeflater API and portability documentation. No overlapping decompression PR was found on 2026-10-22. Tested libdeflater 1.26.1 as a safe whole-buffer zlib fast path, with the original decoder retained for errors and output limits. Gzip, uncompressed records, LZ4 and all encoding paths were unchanged. The initial output buffer used compressed length ×16, clamped to 16–128 KiB and the caller's output limit. All 3,285 scanned real zlib records, including proto chunks, fit that policy.
The preserved accepted `fb2be075` saved-light executable is the baseline; source/dependency/resource/corpus hashes and release flags match the campaign protocol. After a discarded pair, the two-pair normal pilot improved complete loading by **11.908% and 11.941%**. The formatted candidate then completed five alternating process pairs per corpus after a discarded pair, with no concurrent builds/tests/profiles. Complete normal loading improved **11.079% (9.548–12.314%)**, from median 46.642 to 41.474 ms per 256 chunks; inflate/NBT improved 18.452%. Flat complete loading improved **2.568% (1.915–7.363%)**, 10.635 to 10.367 ms, supporting no 5% flat claim. All timed checksums match. Peak RSS ranges overlap (normal baseline/candidate medians 161,656/161,008 KiB; flat 160,860/160,188 KiB). The normal gain is incremental to accepted palette, aquifer and saved-light changes.
Partial correctness gates passed for that candidate: four codec tests, including differential valid/damaged/truncated/trailing/concatenated/dictionary/limit streams and all 65,536 zlib headers; all **691 comprehensive loaded-metadata snapshots** match the accepted source byte-for-byte. A fresh pinned Minecraft 26.3 `RegionFileVersion` oracle generated 62 valid fixtures covering all four codecs, large/high-ratio data and Java zlib levels 0–9 with all three strategies. Both Rust implementations decode these exactly; Horizon encoding bytes match the original, and pinned Minecraft decodes all 62 Horizon outputs exactly. These are focused loading/codec checks, not a complete world-parity claim.
Decision: **no-go because related valid-input workloads materially regress**. The initial bounded fast path adds a failed trial before legacy decode on high-ratio/large streams; isolated decode probes showed roughly 26–39% regressions for representative 60/64/128 KiB cases. Geometric output growth was also rejected after poor uniform-input results. A final bounded policy restricted the fast path to compressed records of 2–64 KiB, resolving those first cases, but 24 additional controlled-compressibility fixtures exposed **27–71% isolated decode regressions near 131 KiB**, about **26–27% at 256 KiB**, and about **9% on one 1 MiB case**. These microbenchmarks use the original and candidate decoder in the same release executable, a discarded pair and five alternating measured pairs; they are related-workload evidence, not complete-world timing claims. No complete-load gain is claimed for the final gated policy. Removing this tradeoff requires further design beyond this bounded attempt.
All production and Cargo changes were restored exactly, and owned temporary examples were removed. No final repository quick-check, additional parity work, commit or PR was pursued after rejection. The shared release target contains rejected candidate artifacts and must not be used as an accepted baseline. Earlier campaign documentation, including the uncommitted Round 9 report, is preserved. Evidence remains outside Git at `/srv/bob-data/horizon-chunk-campaign-41055168/decompression/`: authenticated sources and binaries for each policy, dependencies, build/test logs, paired runners/raw observations, RSS, fresh Java fixtures/bytecode, 691 snapshot outputs, controlled-compressibility probes and `decision.json`. Agent 10 ends this optimization sequence; further work awaits user feedback.