docs/modpacks.md: memory, NeoForge comparison, worldgen and Chipped status

The NBT tables stream, the fluid tables reuse the game's facts and the content
root is kept between starts (#753, #754, #756). Horizon with ATM10's 40 data-tier
mods against the same mods on NeoForge 21.1.259. Worldgen carry-over and what it
does not cover yet (#763, #764). Chipped's harness row with its self-drop tables
(#762). The overlay cache line said it was cached; it is now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ludovic Marie
2026-10-17 23:45:23 +02:00
parent 9b97ba09de
commit 90053d15aa
+94 -14
View File
@@ -236,7 +236,15 @@ How the game loads them, without touching vanilla's path:
so the same mod set gives the same ids. Saves store block names, so ids never reach the disk.
- Loaders that read only `data/minecraft/` (recipes, loot, tags) read every namespace, as
vanilla does. With no mods there is only `minecraft`, so nothing changes.
- The overlay is cached like the resource-pack overlay and rebuilt when a mod file changes.
- The overlay is built on the first start with a set of mods and kept for later starts. A stamp
(`horizon-mod-content.stamp`) hashes the game executable, the base tables and every mod file's
size and mtime. Adding, removing or editing a mod, or a new build of Horizon, rebuilds it. For
ATM10's 40 data-tier mods the build takes about a minute of the first start on a busy machine.
- **Worldgen.** Mods' structures, structure sets, template pools, processor lists, configured and
placed features join the `worldgen/*.json` bundles. Their structure templates go under
`worldgen/structure/<ns>/`. Their biome tags are merged into `worldgen/biome_tags.json`, minus
the biomes and tags Horizon does not have. Mod biomes, noise and density functions are not
carried yet.
- **Limits.** Chunk sections size their global palette to the registry: 16 bits for vanilla's
35,723 states (the same chunks, saves and packets as before), one more bit each time content
mods double it, up to 2^20 states (1,048,576). Past that the content root is refused with the
@@ -398,6 +406,40 @@ Mechanical, the same for every mod, and no agent:
carries those over.
5. Write `<id>.hmod`. For a data-tier mod, it is now ported.
Two cases the oracle fills in:
- **Loot registered in code.** Chipped ships 7 loot tables for about 7,000 blocks; the rest
drop themselves through Java code. When a declared block has no table in the jar and the
oracle shows vanilla's drop-self, carry writes that table in 26.3's shape. When only the
states with one property value drop it (a door's lower half), the entry gets vanilla's
`match_block` condition. Chipped: 6,843 tables added.
- **Worldgen carry-over.** Structure libraries give vanilla's types other names. Carry maps
these to vanilla's:
- `lithostitched:jigsaw`, MoogsStructureLib's and Repurposed Structures' generic jigsaws
become `minecraft:jigsaw`;
- their `advanced_random_spread` becomes `minecraft:random_spread`;
- their versioned, legacy and empty pool elements become vanilla's.
The rules these libraries add on top of vanilla's (no start in liquid, a minimum distance,
caps per piece) are lost, so a mapped structure can start where the mod would not. NeoForge's
convention biome tags (`#c:is_forest`, ...) come from the loader jar
(`<work>/loader/neoforge-<version>-universal.jar`, fetched dev-only and SHA-1 checked) and
are written into the carry. Each mod's `carry-report.json` lists what was mapped and the
types left as they are. The game skips the structures it cannot decode.
On ATM10's 40 data-tier mods:
- 1,061 files were mapped (MVS 413, MNS 210, MSS 157, MES 110, Repurposed Structures 104,
ChoiceTheorem's Overhauled Village 66), and 103 convention biome tags written.
- When Dungeons Arise, Structory and Towns and Towers use vanilla's types already.
- Not generated yet, because their types are Java code: YUNG's API (491 elements and
jigsaws: every "YUNG's Better" mod), Formations (179), the nether jigsaw variants of Moog's
and Repurposed (69), MO Structures (38), and Repurposed's mineshafts, mansions, monuments
and cities.
In a world with the 40 mods, seed 1234, `/locate structure mvs:azelea_house` finds one 27
blocks from spawn and it stands there as the template builds it; When Dungeons Arise's
bandit towers and Structory's chapel locate too.
### 5. Code: one agent per mod
For tiers logic, machines and deep, in dependency order, as many agents at once as the AI
@@ -473,12 +515,15 @@ user's jars are, and its facts and results stay in the work folder. The first re
| Glassential-renewed 3.4.7 | 105/114 | 5177/5177 | 5177/5177 | 4727/5177 | 6447/6447 | 210/210 | 105/105 | 143/143 | 106/115 |
| Factory Blocks 1.4.0 | 42/49 | 42/42 | 0/42 | 42/42 | 294/294 | 84/84 | 42/42 | 1/1 | 42/49 |
| Cloud Glass 1.1.0 | 16/16 | 16/16 | 16/16 | 0/16 | 112/112 | 32/32 | – | 31/31 | 16/16 |
| Chipped 4.0.2 | 6857/6981 | 46212/46443 | 46443/46443 | 46396/46443 | 149093/149093 | 13548/13714 | 11984/12378 | 6/13 | 6849/6973 |
What fails is what API 0.2 cannot express yet: shapes of their own (Japanese doors, hedges,
paths a sixteenth high, glass without collision), sound events of the mod's own
(`factory_blocks:metal_sound`), and blocks whose vanilla class has no vanilla block with the
same states (garage doors, double gates, engraved and facing paths, fans). Every drop, recipe
and tag of the declared blocks matches.
and tag of the declared blocks matches. Chipped's 124 missing blocks are its own classes (workbenches and
`DirectionBlock`), and its workbench recipes need those blocks. Its drops match since carry writes
the self-drop tables it registers in code (step 4).
### 7. Coverage report
@@ -520,9 +565,9 @@ files; `inventory`; `carry` and the harness on the eight mods we downloaded; `re
By tier, the 367 not yet: data 57 (carry-over and the harness, no agent: their jars were not
downloaded), logic 66 (API 0.3), machines 97 (0.4), deep 92 (0.5), client 35, unclassified 20.
Ported: Macaw's Trapdoors. Partial: Macaw's Doors, Fences and Walls, Paths and Pavings,
Glassential, Factory Blocks, Cloud Glass (see the harness table) and MVS, whose 784 worldgen
files are carried but not yet read by the game. Of the 8, 7 are MIT and one LGPL; nothing is
published.
Glassential, Factory Blocks, Cloud Glass, Chipped (see the harness table) and MVS, whose
structures now generate (see step 4). Of the 9, 7 are MIT, one LGPL and Chipped is under the
Terrarium License; nothing is published.
## Measuring "10x"
@@ -582,8 +627,9 @@ vanilla's 35,723; the per-state tables are the next thing to measure), while the
inside its 2 GB heap. Caveats, so none of this is a "10x" claim yet: other sessions and CI
runners kept the machine's load average between 30 and 56 on 14 threads, so the confidence
intervals are as wide as the means (`perf_bench.py --table`); Horizon's frames are paced near
the display's 144 Hz at 8 chunks; Horizon's title time includes building the content overlay on
a first start (about 5 s, cached afterwards); and the pack runs 1.21.1 against Horizon's 26.3.
the display's 144 Hz at 8 chunks; Horizon's title time includes building the content overlay
(about 5 s for six mods; it was rebuilt on every start then, and is kept for later starts now);
and the pack runs 1.21.1 against Horizon's 26.3.
A quiet rerun with more games is the next measurement.
**Where the RAM goes (2026-10-17).** RSS overstates it on macOS. `ps` counts pages the allocator
@@ -620,8 +666,44 @@ indexes ids by a hash checked against the states: -22 MB live for vanilla, -124
ATM10's, -30 MB and -140 MB of footprint. The startup peaks are what is left. Each step parses
its per-state tables whole: as JSON trees, or as NBT trees of 300-600 MB each for ATM10
(`block-properties.nbt`, the collision, support, occlusion and movement tables). The peak
reaches 3.9 GB during worldgen loading, and the footprint stays near the peak. Typed, streaming
readers for those tables are the next cut.
reaches 3.9 GB during worldgen loading, and the footprint stays near the peak.
The NBT tables now stream: the readers check each state's identity as it is read, and no
`states` tree is built (#753). That tree was about 240 MB for each table. With the carried
ATM10 mods (153,511 states), the peak of each step falls by about 210 MB:
| step | before MB | after MB |
|---|---:|---:|
| light | 911 | 699 |
| collision | 945 | 733 |
| motion | 1,427 | 1,215 |
| worldgen, uncached | 3,000 | 2,788 |
The fluid tables no longer decode the 211 MB `gameplay-facts.json` a second time (#754). The
content root is kept between starts (#756). What is left at startup:
- `facts.json` goes through a JSON tree, then an NBT tree, in both the registry and the assets
step;
- on a first start, the content-root build and `Behaviours`' decode of `gameplay-facts.json`
(cached afterwards in the behaviour snapshot).
**Against NeoForge, same 40 mods (2026-10-17).** The 40 data-tier mods carried from ATM10 (the
42 without Regions Unexplored and MO Structures, whose libraries are only on CurseForge),
against their jars on NeoForge 21.1.259 with `-Xmx2G`. `perf_bench.py --port-work`, RD 8, one
game each, load average 80-125 on 14 threads; `--warm` measures a later start:
| game | title / world s | RSS steady / peak MB | footprint steady / peak MB |
|---|---|---|---|
| Horizon before #756 (every start) | 81.1 / 102.8 | 4,008 / 4,164 | 2,368 / 2,554 |
| Horizon, first start | 105.2 / 128.8 | 3,795 / 3,993 | 2,306 / 2,560 |
| **Horizon, later starts** | **21.6 / 33.7** | **2,629 / 2,935** | **2,445 / 2,731** |
| NeoForge + 40 mods | 53.2 / 119.9 | 3,099 / 3,203 | 3,304 / 3,381 |
On later starts, Horizon uses 15 % less RSS and 26 % less footprint than NeoForge with the
same mods. It reaches the title 2.5x sooner and the world 3.6x sooner, and its flights ran at
93-233 FPS against NeoForge's 27-62. The first start is slower than NeoForge's to the title,
because it builds the content root once. One game per row on a loaded machine is not a
benchmark; a quiet rerun with repeats is still to do.
## The working version for 2026-10-23
@@ -648,11 +730,9 @@ In scope (each is a PR or a small stack):
6. **Measured:** load time, RAM and FPS of that pack on NeoForge 1.21.1 against Horizon, with
the full ATM10 on NeoForge as the reference row.
Out of scope for that date, and next in order: worldgen carry-over (the 30 or so structure and biome mods;
the first one looked at, Moog's Voyager Structures, declares all 130 structures and 116 structure
sets with MoogsStructureLib's own `generic_jigsaw_structure` and `advanced_random_spread` types and
NeoForge's `c:` biome tags, so its carry needs a mapping to vanilla's jigsaw and random spread
types plus the oracle's biome tags, not only bundle merging),
Out of scope for that date, and next in order: the rest of worldgen carry-over (mods'
structures, sets, pools and features generate now, see step 4; mod biomes, noise and density
functions, and the structure types written in Java such as YUNG's and Formations', do not yet),
behaviour callbacks and server events (0.3), block entities, menus, capabilities and net
(0.4), the first agent-ported logic-tier mod as the pilot of step 5, then machines. Porting all
of ATM10 is a campaign. Each step publishes its coverage number; we make no claim beyond what