Test equivalent arithmetic in the sampled terrain data lookup

This commit is contained in:
Ludovic Marie
2026-10-04 04:26:04 +02:00
parent 812ad831b1
commit c6ec0b5e14
19 changed files with 419 additions and 7 deletions
+6
View File
@@ -67,6 +67,8 @@ jobs:
run: python -c "from tools.build_mod import build_culling_empty_fixture; build_culling_empty_fixture()"
- name: Compile the private culling ray geometry fixture
run: python -c "from tools.build_mod import build_integration_fixture; build_integration_fixture('culling-rays', 'dev.horizonlab.dispatch.CullingRayIntegration', ['entityculling'], ['CullingRayAccess'])"
- name: Compile the private terrain position lookup fixture
run: python -c "from tools.build_mod import build_integration_fixture; build_integration_fixture('terrain-position', 'dev.horizonlab.dispatch.PositionIntegration')"
- name: Compile the private terrain read shutdown fixture
run: python -c "from tools.build_mod import build_integration_fixture; build_integration_fixture('terrain-read', 'dev.horizonlab.dispatch.TerrainReadIntegration')"
- name: Install headless graphics libraries for applied-hook checks
@@ -84,6 +86,9 @@ jobs:
- name: Compare applied culling ray geometry and provider queries
if: runner.os == 'Linux'
run: python tools/test_culling_rays.py
- name: Verify actual full-data position lookup equivalence
if: runner.os == 'Linux'
run: python tools/test_terrain_position.py
- name: Verify interrupted DH reads and preserved shared futures
if: runner.os == 'Linux'
run: python tools/test_terrain_read.py
@@ -103,3 +108,4 @@ jobs:
dist/culling-empty-integration.json
dist/terrain-read-integration.json
dist/culling-rays-integration.json
dist/terrain-position-integration.json
+7 -1
View File
@@ -6,7 +6,7 @@ Client-first performance research for Minecraft Java **26.3**, preserving vanill
## Play on your Mac
Download **horizon-mac-starter-0.15.0.zip** from [Releases](https://github.com/ludovic111/minecraft-optimization-lab/releases), extract it and open `Mac-Start.command`. It verifies and installs official universal Prism Launcher in `~/Applications` if needed, then opens the balanced pack import. Sign into Microsoft **inside Prism yourself**. This project needs no credentials.
Download **horizon-mac-starter-0.16.0.zip** from [Releases](https://github.com/ludovic111/minecraft-optimization-lab/releases), extract it and open `Mac-Start.command`. It verifies and installs official universal Prism Launcher in `~/Applications` if needed, then opens the balanced pack import. Sign into Microsoft **inside Prism yourself**. This project needs no credentials.
Use native **Java 25 ARM64**, maximum heap **6144 MiB**, minimum **512 MiB** on your 16 GB M2 Pro. Prism can download Java automatically. Start at 1280×800 or 1920×1200; full Retina resolution is a larger GPU workload. [START-HERE.txt](packs/START-HERE.txt) has details. The same `.mrpack` files import into Prism on Windows and Linux.
@@ -22,6 +22,7 @@ Use native **Java 25 ARM64**, maximum heap **6144 MiB**, minimum **512 MiB** on
| dispatch-retry (experimental) | 12 | 128 | 6144 MiB |
| tick-dedup (experimental) | 12 | 128 | 6144 MiB |
| culling-empty (experimental) | 12 | 128 | 6144 MiB |
| terrain-position (experimental) | 12 | 128 | 6144 MiB |
| culling-rays (experimental) | 12 | 128 | 6144 MiB |
| foreground-first (combined experiment) | 12 | 128 | 6144 MiB |
| foreground-vulkan (combined experiment) | 12 | 128 | 6144 MiB |
@@ -36,6 +37,11 @@ Distant LOD is simplified terrain. Nearby blocks retain full detail. Initial gen
The optional `queue-wait` pack yields during pending distant-terrain debounce retries; `adaptive-queue` combines it with the existing controllers. [Queue experiment and measurement limits](docs/debounce-queue.md).
Version 0.16 adds a separate [terrain position arithmetic experiment](docs/terrain-position.md)
for the sampled full-data lookup. It retains the pinned library’s unusual
negative-coordinate rounding and invalid-level errors. Balanced keeps it off;
whole-game and Mac performance still require measurement.
Version 0.15 adds an isolated [culling ray scratch experiment](docs/culling-rays.md)
that reuses two numeric vectors within each culling instance while retaining
upstream arithmetic and decisions. Balanced keeps it off; hardware gains remain
+57
View File
@@ -0,0 +1,57 @@
# Equivalent position arithmetic in the LOD data lookup
A live allocation/CPU investigation sampled Distant Horizons 3.3.4's relative
position conversion on render-loader threads. This is a numeric step within
`FullDataSourceV2.getDataPointAtBlockPos`; the first candidate is a Java hook at
that call, with no native transition, cache, new worker or retained per-source
field. The original library handles containment, column selection and datapoint
search. No writes, world generation, block mechanics or rendering budgets change.
Select `terrain-position` for an isolated comparison with `m2-balanced`.
Graphics, worker and other controller settings match. This hook is off in all
other profiles, including combined profiles, and requires exactly DH 3.3.4.
Disable `enabled` in `config/horizon-terrain-position.properties` and restart.
Reports and both comparison tools capture its actual applied flag.
The conversion retains the original numeric output, including its unusual
negative-coordinate bias. A plain coordinate shift failed a differential test
and was discarded; correcting that apparent rounding is outside this hook's
scope. The replacement computes truncation toward zero for a positive power-of-two
divisor using shifts and sign masks. For a negative coordinate, the publisher's
bias is applied before truncation, and the final relative coordinate retains its
original remainder result. The zero-shift bias is retained too.
The decoded coordinate is a signed 28-bit value. For shift values 0 through 30,
all adjusted negative values and their truncation correction fit a signed 32-bit
integer. Positive coordinates use the same zero correction. Java's masked shift
count is retained for larger valid level differences. A sign-bit divisor (masked
shift 31) and invalid output levels delegate to the original method, preserving
its result or exception class/message. The hook is numeric and has no mutable
shared state.
The actual publisher differential suite checks 1,525,313 cases, including
1,124,986 invalid-level exceptions. Cases include powers-of-two boundaries,
positive/negative coordinates, integer extrema passed through the original
encoding, all output bytes across selected input levels, reserved position bits
and a million randomized position/level combinations. The observed negative
counterexample is retained explicitly.
Private probes rejected a numeric cache, which provided no useful benefit and
was slower with varying inputs. A version with coordinate-sign branches also
became slower with a broad input set and was discarded. A subsequent version
using sign masks matched the randomized results and reduced caller CPU time in
the local synthetic loop across its tested input-set sizes. These are mechanism
observations, not whole-game FPS, retained RAM, GPU cost or Mac performance.
The applied Fabric fixture matched 491,520 public lookups across 30 generated
data owners and six detail levels, with 327,049 populated-column results. Output
and source-data fingerprints matched, and both modes verified whether the
redirect was actually installed.
Timing remains sensitive to warm-up. The first two-million-lookup observation
was about 120 ms caller CPU upstream versus 225 ms with the hook. Subsequent
warmed six-million-lookup sets observed 250/261 ms and, after longer warm-up
and compilation waits, 331/260 ms. The later set favors the hook, but these
are fixed-order single forks with three sequential observations, not independent
repetitions or a statistical speed estimate. Compiler-diagnostic observations
are kept separate. Live captures and independent platform checks are required
before recommending it.
@@ -97,6 +97,7 @@ public final class HorizonClient implements ClientModInitializer {
context.put("wakeable_idle_wait", dev.horizonlab.mixin.HorizonMixinPlugin.debounceWaitEnabled());
context.put("dispatch_retry_backoff", dev.horizonlab.mixin.HorizonMixinPlugin.dispatchRetryEnabled());
context.put("tick_membership_dedup", dev.horizonlab.mixin.HorizonMixinPlugin.tickDedupEnabled());
context.put("terrain_position_arithmetic", dev.horizonlab.mixin.HorizonMixinPlugin.terrainPositionEnabled());
context.put("culling_ray_scratch", dev.horizonlab.mixin.HorizonMixinPlugin.cullingRayScratchEnabled());
context.put("empty_culling_whitelist", dev.horizonlab.mixin.HorizonMixinPlugin.cullingEmptyEnabled());
context.put("interruptible_terrain_read", dev.horizonlab.mixin.HorizonMixinPlugin.terrainReadWaitEnabled());
@@ -0,0 +1,30 @@
package dev.horizonlab.geometry;
import com.seibel.distanthorizons.core.pos.DhSectionPos;
/** Numeric-only conversion; preserve the pinned library's truncation and negative-coordinate bias. */
public final class RelativeTerrainPosition {
private RelativeTerrainPosition() {}
public static long resolve(long position, byte outputLevel) {
int delta = outputLevel - DhSectionPos.getDetailLevel(position);
int shift = delta & 31;
// Invalid levels keep the publisher exception; a sign-bit divisor keeps its original arithmetic.
if (delta < 0 || shift == 31) {
return DhSectionPos.getDhSectionRelativePositionForDetailLevel(position, outputLevel);
}
int divisorMask = (1 << shift) - 1;
int negativeBias = divisorMask | ((divisorMask - 1) >>> 31);
return DhSectionPos.encode(outputLevel,
coordinate(DhSectionPos.getX(position), shift, divisorMask, negativeBias),
coordinate(DhSectionPos.getZ(position), shift, divisorMask, negativeBias));
}
private static int coordinate(int value, int shift, int divisorMask, int negativeBias) {
int negative = value >> 31;
int adjusted = value + (negative & negativeBias);
// Adding the divisor mask only for a negative adjusted value gives truncation toward zero.
int truncated = (adjusted + ((adjusted >> 31) & divisorMask)) >> shift;
return (truncated + negative) & 63;
}
}
@@ -0,0 +1,16 @@
package dev.horizonlab.mixin;
import dev.horizonlab.geometry.RelativeTerrainPosition;
import org.spongepowered.asm.mixin.Mixin;
import org.spongepowered.asm.mixin.injection.At;
import org.spongepowered.asm.mixin.injection.Redirect;
/** Replace only the numeric conversion in the sampled full-data lookup. */
@Mixin(targets = "com.seibel.distanthorizons.core.dataObjects.fullData.sources.FullDataSourceV2", remap = false)
public abstract class DhRelativePositionMixin {
@Redirect(method = "getDataPointAtBlockPos", at = @At(value = "INVOKE",
target = "Lcom/seibel/distanthorizons/core/pos/DhSectionPos;getDhSectionRelativePositionForDetailLevel(JB)J"))
private long horizon$relativePosition(long position, byte outputLevel) {
return RelativeTerrainPosition.resolve(position, outputLevel);
}
}
@@ -12,6 +12,7 @@ import org.spongepowered.asm.mixin.extensibility.IMixinInfo;
/** Internal hooks are opt-in and limited to the inspected, pinned DH version. */
public final class HorizonMixinPlugin implements IMixinConfigPlugin {
private static boolean terrainPosition;
private static boolean workerPacing;
private static boolean debounceWait;
private static boolean dispatchRetry;
@@ -130,8 +131,24 @@ public final class HorizonMixinPlugin implements IMixinConfigPlugin {
} catch (Exception error) {
LoggerFactory.getLogger("horizonlab").warn("Culling ray scratch vectors disabled: invalid configuration", error);
}
try {
Properties properties = new Properties();
var positionFile = loader.getConfigDir().resolve("horizon-terrain-position.properties");
if (Files.exists(positionFile)) {
try (var reader = Files.newBufferedReader(positionFile)) { properties.load(reader); }
}
boolean requested = Boolean.parseBoolean(properties.getProperty("enabled", "false"));
boolean supported = loader.getModContainer("distanthorizons")
.map(mod -> mod.getMetadata().getVersion().getFriendlyString().equals("3.3.4")).orElse(false);
terrainPosition = requested && supported;
if (requested) LoggerFactory.getLogger("horizonlab").info(
"Relative terrain position arithmetic: {} (requires pinned DH 3.3.4)", terrainPosition ? "enabled" : "unavailable");
} catch (Exception error) {
LoggerFactory.getLogger("horizonlab").warn("Terrain position arithmetic disabled: invalid configuration", error);
}
}
public static boolean terrainPositionEnabled() { return terrainPosition; }
public static boolean workerPacingEnabled() { return workerPacing; }
public static boolean debounceWaitEnabled() { return debounceWait; }
public static boolean dispatchRetryEnabled() { return dispatchRetry; }
@@ -147,6 +164,7 @@ public final class HorizonMixinPlugin implements IMixinConfigPlugin {
if (mixinClassName.endsWith("CullingEmptyWhitelistMixin")) return cullingEmpty;
if (mixinClassName.endsWith("CullingRayScratchMixin")) return cullingRayScratch;
if (mixinClassName.endsWith("DhTerrainReadWaitMixin")) return terrainReadWait;
if (mixinClassName.endsWith("DhRelativePositionMixin")) return terrainPosition;
return true;
}
@Override public String getRefMapperConfig() { return null; }
@@ -1,7 +1,7 @@
{
"schemaVersion": 1,
"id": "horizonlab",
"version": "0.15.0",
"version": "0.16.0",
"name": "Horizon Lab",
"description": "Client frame pacing and server tick measurements for reproducible optimization experiments.",
"authors": [
@@ -14,7 +14,8 @@
"DhTickDedupMixin",
"CullingEmptyWhitelistMixin",
"DhTerrainReadWaitMixin",
"CullingRayScratchMixin"
"CullingRayScratchMixin",
"DhRelativePositionMixin"
],
"injectors": {
"defaultRequire": 1
@@ -0,0 +1,121 @@
package dev.horizonlab.dispatch;
import com.seibel.distanthorizons.core.dataObjects.fullData.sources.FullDataSourceV2;
import com.seibel.distanthorizons.core.pos.DhSectionPos;
import com.seibel.distanthorizons.core.util.FullDataPointUtil;
import dev.horizonlab.mixin.HorizonMixinPlugin;
import it.unimi.dsi.fastutil.longs.LongArrayList;
import java.lang.management.ManagementFactory;
import java.security.MessageDigest;
import java.util.HexFormat;
import java.util.Random;
import net.fabricmc.api.ClientModInitializer;
import net.fabricmc.fabric.api.client.event.lifecycle.v1.ClientLifecycleEvents;
/** Private applied-hook fixture, using generated column data and the actual public lookup. */
public final class PositionIntegration implements ClientModInitializer {
private static volatile long checksum;
@Override public void onInitializeClient() {
ClientLifecycleEvents.CLIENT_STARTED.register(client -> {
try { exercise(); }
catch (Exception failure) { throw new AssertionError("Terrain position fixture failed", failure); }
if (Boolean.getBoolean("horizon.fixture.exit")) System.exit(0);
});
}
private static FullDataSourceV2 source(byte level, int x, int z) throws Exception {
var owner = FullDataSourceV2.createEmpty(DhSectionPos.encode(level, x, z));
for (int cx = 0; cx < 64; cx++) for (int cz = 0; cz < 64; cz++) {
var column = new LongArrayList();
for (int layer = 2; layer >= 0; layer--) {
long point = FullDataPointUtil.encode(1 + ((cx * 7 + cz * 11 + layer) & 255), 32,
layer * 32, (byte) 15, (byte) (layer * 3));
if (FullDataPointUtil.getHeight(point) != 32 || FullDataPointUtil.getBottomY(point) != layer * 32)
throw new AssertionError("Generated publisher data has wrong height");
column.add(point);
}
owner.dataPoints[FullDataSourceV2.relativePosToIndex(cx, cz)] = column;
}
return owner;
}
private static long dataFingerprint(FullDataSourceV2 owner) {
long hash = owner.getPos();
for (var column : owner.dataPoints) {
hash = hash * 31 + column.size();
for (int index = 0; index < column.size(); index++) hash = hash * 31 + column.getLong(index);
}
return hash;
}
private static void exercise() throws Exception {
boolean enabled = HorizonMixinPlugin.terrainPositionEnabled();
boolean applied = java.util.Arrays.stream(FullDataSourceV2.class.getDeclaredMethods())
.anyMatch(method -> method.getName().contains("horizon$relativePosition"));
if (applied != enabled) throw new AssertionError("Lookup redirect does not match requested state");
var digest = MessageDigest.getInstance("SHA-256");
byte[] bytes = new byte[8];
var random = new Random(0x49a91);
int queries = 0, nonzero = 0, owners = 0;
long fingerprints = 0;
for (byte level : new byte[] {6, 7, 8, 10, 14, 20}) {
for (int[] origin : new int[][] {{-1, -1}, {-7, 9}, {0, 0}, {13, -21}, {11, -13}}) {
var owner = source(level, origin[0], origin[1]);
long before = dataFingerprint(owner);
int minX = DhSectionPos.getMinCornerBlockX(owner.getPos());
int minZ = DhSectionPos.getMinCornerBlockZ(owner.getPos());
int width = DhSectionPos.getDetailLevelWidthInBlocks(level);
for (int index = 0; index < 16384; index++) {
int x = minX + random.nextInt(width), z = minZ + random.nextInt(width);
if ((index & 1023) == 0) x = minX - 1;
if ((index & 1023) == 1) z = minZ + width;
int y = random.nextInt(144) - 80;
long value = owner.getDataPointAtBlockPos(x, y, z, -64);
if (value != 0) nonzero++;
for (int at = 0; at < 8; at++) bytes[at] = (byte) (value >>> (56 - 8 * at));
digest.update(bytes); queries++;
}
if (before != dataFingerprint(owner)) throw new AssertionError("Lookup mutated source data");
fingerprints = fingerprints * 31 + before; owners++;
}
}
if (nonzero < 10000) throw new AssertionError("Fixture did not exercise populated columns");
var owner = source((byte) 10, -13, -7);
int[] xs = new int[256], zs = new int[256];
int minX = DhSectionPos.getMinCornerBlockX(owner.getPos()), minZ = DhSectionPos.getMinCornerBlockZ(owner.getPos());
for (int index = 0; index < 256; index++) { xs[index] = minX + index * 3; zs[index] = minZ + (index * 37 & 1023); }
loop(owner, xs, zs, 10000000);
Thread.sleep(1000);
var cpu = ManagementFactory.getThreadMXBean();
var allocations = (com.sun.management.ThreadMXBean) cpu;
if (allocations.isThreadAllocatedMemorySupported() && !allocations.isThreadAllocatedMemoryEnabled())
allocations.setThreadAllocatedMemoryEnabled(true);
long tid = Thread.currentThread().threadId();
long wall = 0, cpuTime = 0, allocated = 0;
StringBuilder observations = new StringBuilder("[");
for (int repeat = 0; repeat < 3; repeat++) {
loop(owner, xs, zs, 4000000);
Thread.sleep(200);
long allocationStart = allocations.getThreadAllocatedBytes(tid), cpuStart = cpu.getCurrentThreadCpuTime(), start = System.nanoTime();
loop(owner, xs, zs, 2000000);
long observedWall = System.nanoTime() - start, observedCpu = cpu.getCurrentThreadCpuTime() - cpuStart;
long observedAllocation = allocations.getThreadAllocatedBytes(tid) - allocationStart;
wall += observedWall; cpuTime += observedCpu; allocated += observedAllocation;
if (repeat != 0) observations.append(',');
observations.append("{\"cpu_ns\":").append(observedCpu).append(",\"wall_ns\":").append(observedWall)
.append(",\"allocated_bytes\":").append(observedAllocation).append('}');
}
observations.append(']');
System.out.println("HORIZON_POSITION_RESULT {\"enabled\":" + enabled + ",\"applied_hook_checked\":true,\"owners\":" + owners
+ ",\"queries\":" + queries + ",\"nonzero\":" + nonzero + ",\"lookup_digest\":\"" + HexFormat.of().formatHex(digest.digest())
+ "\",\"data_fingerprint\":" + fingerprints + ",\"data_unchanged\":true,\"measured_checks\":6000000,\"checksum\":" + checksum
+ ",\"allocated_bytes\":" + allocated + ",\"cpu_ns\":" + cpuTime + ",\"wall_ns\":" + wall + ",\"observations\":" + observations + "}");
}
private static void loop(FullDataSourceV2 owner, int[] xs, int[] zs, int count) {
long sum = 0;
for (int index = 0; index < count; index++) sum += owner.getDataPointAtBlockPos(xs[index & 255], -48, zs[index & 255], -64);
checksum = sum;
}
}
@@ -0,0 +1,52 @@
package dev.horizonlab.geometry;
import com.seibel.distanthorizons.core.pos.DhSectionPos;
import java.util.ArrayList;
import java.util.Random;
/** Differential checks against the actual pinned publisher method, including its unusual rounding. */
public final class RelativeTerrainPositionTest {
private static int cases, failures;
private static void check(long position, byte level) {
cases++;
long expected;
try { expected = DhSectionPos.getDhSectionRelativePositionForDetailLevel(position, level); }
catch (IllegalArgumentException upstream) {
try { RelativeTerrainPosition.resolve(position, level); }
catch (IllegalArgumentException patched) {
if (!upstream.getMessage().equals(patched.getMessage()) || upstream.getClass() != patched.getClass())
throw new AssertionError("Publisher exception changed");
failures++;
return;
}
throw new AssertionError("Invalid level stopped throwing");
}
long actual = RelativeTerrainPosition.resolve(position, level);
if (expected != actual) throw new AssertionError("Relative position changed: " + position + "/" + level);
}
public static void main(String[] args) {
var values = new ArrayList<Integer>();
values.add(Integer.MIN_VALUE); values.add(Integer.MAX_VALUE); values.add(0);
for (int bit = 0; bit <= 27; bit++) {
int boundary = 1 << bit;
for (int offset = -1; offset <= 1; offset++) {
values.add(boundary + offset); values.add(-boundary + offset);
}
}
for (int input : new int[] {0, 1, 6, 30, 31, 32, 63, 64, 95, 96, 126, 127}) {
for (int output = -128; output <= 127; output++) {
for (int index = 0; index < values.size(); index++) {
int x = values.get(index), z = values.get((index * 37 + 13) % values.size());
check(DhSectionPos.encode((byte) input, x, z), (byte) output);
}
}
}
// A direct shift changes this original negative-coordinate result; preserve it exactly.
check(-7336704456267971543L, (byte) 55);
Random random = new Random(0x6f9d1413);
for (int index = 0; index < 1000000; index++) check(random.nextLong(), (byte) random.nextInt(256));
System.out.println("Actual publisher position differential checks passed: cases=" + cases + " exceptions=" + failures);
}
}
+4
View File
@@ -66,6 +66,10 @@ PROFILES
dynamic entity/block-entity whitelists. Registered callbacks stay upstream.
Disable in config/horizon-culling-empty.properties (enabled=false), then restart.
Compare against m2-balanced; no whole-game or Mac FPS gain is measured yet.
- terrain-position: same balanced settings, with equivalent position arithmetic
in the full-data lookup. Original rounding and fallback errors are retained.
Disable in config/horizon-terrain-position.properties (enabled=false), then
restart. Compare against m2-balanced; no game or Mac FPS gain is established.
- culling-rays: same balanced settings, with two per-instance scratch vectors for
temporary visibility-ray calculations. Upstream arithmetic and decisions stay
in place. Disable in config/horizon-culling-rays.properties (enabled=false),
+1 -1
View File
@@ -3,7 +3,7 @@
"minecraft": "26.3",
"java_major": 25,
"fabric_loader": "0.19.5",
"pack_version": "0.15.0",
"pack_version": "0.16.0",
"options_data_version": 5023,
"projects": {
"sodium": {
+21
View File
@@ -111,6 +111,18 @@ class PackTests(unittest.TestCase):
"config/horizon-culling-empty.properties", "config/horizon-terrain-read-wait.properties"]:
self.assertEqual(baseline[filename], candidate[filename])
def test_position_experiment_preserves_balanced_settings(self):
baseline = build_pack.overrides("m2-balanced", b"fixture")
candidate = build_pack.overrides("terrain-position", b"fixture")
for name in build_pack.PROFILES:
self.assertEqual(build_pack.overrides(name, b"fixture")["config/horizon-terrain-position.properties"],
b"enabled=true\n" if name == "terrain-position" else b"enabled=false\n")
for filename in ["options.txt", "config/DistantHorizons.toml", "config/horizon-adaptive.properties",
"config/horizon-worker-pacing.properties", "config/horizon-queue-wait.properties",
"config/horizon-dispatch-retry.properties", "config/horizon-tick-dedup.properties",
"config/horizon-culling-empty.properties", "config/horizon-culling-rays.properties", "config/horizon-terrain-read-wait.properties"]:
self.assertEqual(baseline[filename], candidate[filename])
def test_terrain_read_shutdown_fix_only_requested_with_distant_terrain(self):
for name, profile in build_pack.PROFILES.items():
files = build_pack.overrides(name, b"fixture")
@@ -188,6 +200,15 @@ class ComparisonTests(unittest.TestCase):
with self.assertRaises(ValueError):
compare_benchmarks.compare(self.report(), other)
def test_position_hook_change_is_an_explicit_experiment(self):
baseline = self.report()
baseline["terrain_position_arithmetic"] = False
candidate = copy.deepcopy(baseline)
candidate["terrain_position_arithmetic"] = True
result = compare_benchmarks.compare(baseline, candidate)
self.assertEqual(result["mismatches"], [])
self.assertTrue(any("terrain_position_arithmetic differs" in warning for warning in result["warnings"]))
def test_excluded_frames_disclose_that_process_counters_cover_inactive_time(self):
baseline = self.report()
candidate = copy.deepcopy(baseline)
+2 -1
View File
@@ -66,7 +66,7 @@ function compareReports(a,b){
else if(JSON.stringify(a[key])!==JSON.stringify(b[key]))warnings.push('JVM '+key+' differs. Treat it as an explicit experiment variable.');
}
if(a.kind==='frames'){
for(const key of ['adaptive_budget_enabled','adaptive_target_fps','precise_worker_pacing','debounce_queue_wait','wakeable_idle_wait','dispatch_retry_backoff','tick_membership_dedup','empty_culling_whitelist','culling_ray_scratch','interruptible_terrain_read','dh_thread_count','dh_runtime_ratio']){
for(const key of ['adaptive_budget_enabled','adaptive_target_fps','precise_worker_pacing','debounce_queue_wait','wakeable_idle_wait','dispatch_retry_backoff','tick_membership_dedup','empty_culling_whitelist','culling_ray_scratch','terrain_position_arithmetic','interruptible_terrain_read','dh_thread_count','dh_runtime_ratio']){
if(key in a&&key in b&&JSON.stringify(a[key])!==JSON.stringify(b[key]))warnings.push('Experiment '+key+' differs.');
}
for(const [label,r] of [['Baseline',a],['Candidate',b]]){
@@ -126,6 +126,7 @@ function renderDetails(side){
fields.push(['Rejected dispatch retry', r.dispatch_retry_backoff===true?'on':r.dispatch_retry_backoff===false?'off':'not captured']);
fields.push(['Wakeable queue idle', r.wakeable_idle_wait===true?'on':r.wakeable_idle_wait===false?'off':'not captured']);
if(r.kind==='frames')fields.push(['Tick membership deduplication',r.tick_membership_dedup===true?'on':r.tick_membership_dedup===false?'off':'not captured']);
if(r.kind==='frames')fields.push(['Terrain position arithmetic',r.terrain_position_arithmetic===true?'on':r.terrain_position_arithmetic===false?'off':'not captured']);
if(r.kind==='frames')fields.push(['Culling ray scratch',r.culling_ray_scratch===true?'on':r.culling_ray_scratch===false?'off':'not captured']);
if(r.kind==='frames')fields.push(['Empty culling whitelist',r.empty_culling_whitelist===true?'on':r.empty_culling_whitelist===false?'off':'not captured']);
if(r.kind==='frames')fields.push(['Interruptible terrain read',r.interruptible_terrain_read===true?'on':r.interruptible_terrain_read===false?'off':'not captured']);
+3
View File
@@ -96,6 +96,7 @@ def build():
"-d", str(adapter_tests),
str(SOURCE / "test/java/dev/horizonlab/client/DistantTerrainBudgetTest.java"),
str(SOURCE / "test/java/dev/horizonlab/client/DistantTerrainSettingsTest.java"),
str(SOURCE / "test/java/dev/horizonlab/geometry/RelativeTerrainPositionTest.java"),
str(SOURCE / "test/java/dev/horizonlab/metrics/ResidentMemoryTest.java")], check=True)
subprocess.run(["java", "-cp", os.pathsep.join([str(adapter_tests), test_classpath]),
"dev.horizonlab.client.DistantTerrainBudgetTest"], check=True)
@@ -103,6 +104,8 @@ def build():
"dev.horizonlab.client.DistantTerrainSettingsTest"], check=True)
subprocess.run(["java", "--enable-native-access=ALL-UNNAMED", "-cp", os.pathsep.join([str(adapter_tests), test_classpath]),
"dev.horizonlab.metrics.ResidentMemoryTest"], check=True)
subprocess.run(["java", "-cp", os.pathsep.join([str(adapter_tests), test_classpath]),
"dev.horizonlab.geometry.RelativeTerrainPositionTest"], check=True)
output = ROOT / "dist" / f"horizon-lab-{lock['pack_version']}.jar"
output.parent.mkdir(exist_ok=True)
entries = {path.relative_to(classes).as_posix(): path.read_bytes() for path in classes.rglob("*.class")}
+4 -1
View File
@@ -26,6 +26,7 @@ PROFILES = {
"dispatch-retry": {"title": "Rejected dispatch retry experiment", "mods": CORE + ["distanthorizons"], "render": 12, "lod": 128, "heap": 6144, "dispatch_retry": True},
"tick-dedup": {"title": "Terrain tick membership experiment", "mods": CORE + ["distanthorizons"], "render": 12, "lod": 128, "heap": 6144, "tick_dedup": True},
"culling-empty": {"title": "Empty culling whitelist experiment", "mods": CORE + ["distanthorizons"], "render": 12, "lod": 128, "heap": 6144, "culling_empty": True},
"terrain-position": {"title": "Relative terrain position experiment", "mods": CORE + ["distanthorizons"], "render": 12, "lod": 128, "heap": 6144, "terrain_position": True},
"culling-rays": {"title": "Culling ray scratch experiment", "mods": CORE + ["distanthorizons"], "render": 12, "lod": 128, "heap": 6144, "culling_rays": True},
"foreground-first": {"title": "Foreground-first combined experiment", "mods": CORE + ["distanthorizons"], "render": 12, "lod": 128, "heap": 6144, "adaptive": True, "pacing": True, "queue_wait": True, "dispatch_retry": True, "tick_dedup": True, "target_fps": 120},
"foreground-vulkan": {"title": "Foreground-first Vulkan experiment", "mods": CORE + ["distanthorizons"], "render": 12, "lod": 128, "heap": 6144, "adaptive": True, "pacing": True, "queue_wait": True, "dispatch_retry": True, "tick_dedup": True, "target_fps": 120, "backend": "vulkan"},
@@ -82,7 +83,7 @@ def overrides(profile_name, mod_bytes):
files = {f"mods/horizon-lab-{read_lock()['pack_version']}.jar": mod_bytes,
"config/horizon-profile.txt": (profile_name + "\n").encode(),
"options.txt": options(profile).encode()}
experiment = any(profile.get(key, False) for key in ["adaptive", "pacing", "queue_wait", "dispatch_retry", "tick_dedup", "culling_empty", "culling_rays"])
experiment = any(profile.get(key, False) for key in ["adaptive", "pacing", "queue_wait", "dispatch_retry", "tick_dedup", "culling_empty", "culling_rays", "terrain_position"])
config = ROOT / "packs/config" / f"{'m2-balanced' if experiment else profile_name}.toml"
files["config/horizon-adaptive.properties"] = (
f"enabled={str(profile.get('adaptive', False)).lower()}\ntargetFps={profile.get('target_fps', 60)}\n").encode()
@@ -98,6 +99,8 @@ def overrides(profile_name, mod_bytes):
f"enabled={str(profile.get('culling_empty', False)).lower()}\n").encode()
files["config/horizon-culling-rays.properties"] = (
f"enabled={str(profile.get('culling_rays', False)).lower()}\n").encode()
files["config/horizon-terrain-position.properties"] = (
f"enabled={str(profile.get('terrain_position', False)).lower()}\n").encode()
files["config/horizon-terrain-read-wait.properties"] = (
f"enabled={str(bool(profile['lod'])).lower()}\n").encode()
if profile["lod"]:
+1 -1
View File
@@ -28,7 +28,7 @@ def compare(baseline, candidate):
warnings = ["World, route, warm-up, shader state and LOD cache must also match; these are not verified by this tool."]
if baseline["kind"] == "frames":
for field in ["adaptive_budget_enabled", "adaptive_target_fps", "precise_worker_pacing", "debounce_queue_wait",
"wakeable_idle_wait", "dispatch_retry_backoff", "tick_membership_dedup", "empty_culling_whitelist", "culling_ray_scratch", "interruptible_terrain_read", "dh_thread_count", "dh_runtime_ratio"]:
"wakeable_idle_wait", "dispatch_retry_backoff", "tick_membership_dedup", "empty_culling_whitelist", "culling_ray_scratch", "terrain_position_arithmetic", "interruptible_terrain_read", "dh_thread_count", "dh_runtime_ratio"]:
if field in baseline and field in candidate and baseline[field] != candidate[field]:
warnings.append(f"Experiment {field} differs: {baseline[field]!r} vs {candidate[field]!r}.")
for name, report in [("baseline", baseline), ("candidate", candidate)]:
+72
View File
@@ -0,0 +1,72 @@
#!/usr/bin/env python3
"""Compare the actual full-data lookup with original and equivalent position arithmetic."""
import argparse
import hashlib
import json
import os
import platform
import shutil
import subprocess
import sys
try:
from . import build_mod, build_pack
except ImportError:
import build_mod
import build_pack
try:
from .test_culling_rays import run_fixture
except ImportError:
from test_culling_rays import run_fixture
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--native-display", action="store_true")
args = parser.parse_args()
root = build_pack.ROOT
fixture = build_mod.build_integration_fixture("terrain-position", "dev.horizonlab.dispatch.PositionIntegration")
mod = root / "dist" / f"horizon-lab-{build_pack.read_lock()['pack_version']}.jar"
original_sha256 = hashlib.sha256(mod.read_bytes()).hexdigest()
results = []
for enabled in [False, True]:
instance = root / ".local/runtime" / ("terrain-position-fixture-patched" if enabled else "terrain-position-fixture-upstream")
build_pack.prepare_instance("m2-balanced", instance, mod)
if hashlib.sha256((instance / "mods" / mod.name).read_bytes()).hexdigest() != original_sha256:
raise AssertionError("Fixture runtime mod does not match the intended build")
shutil.copy2(fixture, instance / "mods/terrain-position-fixture.jar")
(instance / "config/horizon-terrain-position.properties").write_text(f"enabled={str(enabled).lower()}\n")
command = [sys.executable, str(root / "tools/runtime.py"), "demo", "--instance", str(instance), "--heap", "3G", "--compact-headers"]
if platform.system() == "Linux" and not args.native_display:
command = ["xvfb-run", "-a", *command]
environment = os.environ.copy()
environment["_JAVA_OPTIONS"] = "-Dhorizon.fixture.exit=true"
run = run_fixture(command, instance, environment)
marker = "HORIZON_POSITION_RESULT "
reports = [json.loads(line.split(marker, 1)[1]) for line in run.stdout.splitlines() if marker in line]
if run.returncode or len(reports) != 1:
raise RuntimeError(f"Terrain position fixture failed; inspect {instance / 'fixture.log'}")
report = reports[0]
if report["enabled"] != enabled or report["owners"] != 30 or report["queries"] != 491520 or not report["data_unchanged"] or not report["applied_hook_checked"]:
raise AssertionError("Applied position fixture did not complete its lookup checks")
results.append(report)
print(json.dumps(report), flush=True)
for key in ["lookup_digest", "data_fingerprint", "nonzero", "checksum"]:
if results[0][key] != results[1][key]:
raise AssertionError(f"Position redirect changed upstream lookup: {key}")
output = root / "dist/terrain-position-integration.json"
output.write_text(json.dumps({"schema": 1, "platform": platform.platform(), "architecture": platform.machine(),
"minecraft": "26.3", "distant_horizons": "3.3.4", "original_mod_sha256": original_sha256, "results": results,
"notes": ["Actual public full-data lookup inside Fabric with generated numeric column data.",
"Thirty owners span positive/negative origins and six detail levels; output digests and source fingerprints match.",
"The actual redirect method is checked in each mode; source column data remains unchanged.",
"Three sequential warmed caller-loop observations are not independent forks; JIT compilation can vary.",
"Not whole-game FPS/RAM, rendered-pixel equivalence or a Mac GPU benchmark."]}, indent=2) + "\n")
print(f"Applied terrain position checks passed: {output}")
if __name__ == "__main__":
main()