Merge pull request #11 from ludovic111/priority-dispatch-backoff

Bound rejected terrain dispatch retries without retaining the dispatcher lock
This commit is contained in:
Ludovic Marie
2026-10-04 00:44:03 +02:00
committed by GitHub
20 changed files with 479 additions and 8 deletions
+10
View File
@@ -1,6 +1,7 @@
name: checks
on:
push:
branches: [main]
pull_request:
permissions:
contents: read
@@ -56,6 +57,14 @@ jobs:
run: python -c "from tools.build_mod import benchmark_headers; benchmark_headers()"
- name: Test pack integrity and comparisons
run: python -m unittest discover -s tests -v
- name: Compile the private dispatch fixture
run: python -c "from tools.build_mod import build_dispatch_fixture; build_dispatch_fixture()"
- name: Install headless graphics libraries for applied-hook checks
if: runner.os == 'Linux'
run: sudo apt-get update && sudo apt-get install -y xvfb libgl1 libegl1 libvulkan1 mesa-vulkan-drivers libasound2t64 libxkbcommon0
- name: Exercise the applied dispatcher hooks and upstream rejection path
if: runner.os == 'Linux'
run: python tools/test_dispatch.py
- name: Upload original mod and pack manifests only
uses: actions/upload-artifact@v4
with:
@@ -67,3 +76,4 @@ jobs:
dist/debounce-queue-benchmark.txt
dist/idle-queue-benchmark.txt
dist/object-header-benchmark.txt
dist/dispatch-integration.json
+7 -2
View File
@@ -6,7 +6,7 @@ Client-first performance research for Minecraft Java **26.3**, preserving vanill
## Play on your Mac
Download **horizon-mac-starter-0.9.0.zip** from [Releases](https://github.com/ludovic111/minecraft-optimization-lab/releases), extract it and open `Mac-Start.command`. It verifies and installs official universal Prism Launcher in `~/Applications` if needed, then opens the balanced pack import. Sign into Microsoft **inside Prism yourself**. This project needs no credentials.
Download **horizon-mac-starter-0.10.0.zip** from [Releases](https://github.com/ludovic111/minecraft-optimization-lab/releases), extract it and open `Mac-Start.command`. It verifies and installs official universal Prism Launcher in `~/Applications` if needed, then opens the balanced pack import. Sign into Microsoft **inside Prism yourself**. This project needs no credentials.
Use native **Java 25 ARM64**, maximum heap **6144 MiB**, minimum **512 MiB** on your 16 GB M2 Pro. Prism can download Java automatically. Start at 1280×800 or 1920×1200; full Retina resolution is a larger GPU workload. [START-HERE.txt](packs/START-HERE.txt) has details. The same `.mrpack` files import into Prism on Windows and Linux.
@@ -19,6 +19,7 @@ Use native **Java 25 ARM64**, maximum heap **6144 MiB**, minimum **512 MiB** on
| adaptive-pacing (experiment) | 12 | 128 | 6144 MiB |
| queue-wait (experimental) | 12 | 128 | 6144 MiB |
| adaptive-queue (experimental) | 12 | 128 | 6144 MiB |
| dispatch-retry (experimental) | 12 | 128 | 6144 MiB |
| low-memory | 8 | none | 3072 MiB |
| far-distance | 12 | 256 | 8192 MiB, with sufficient physical RAM |
| cinematic | 12 | 128 | 6144 MiB |
@@ -30,6 +31,10 @@ Distant LOD is simplified terrain. Nearby blocks retain full detail. Initial gen
The optional `queue-wait` pack yields during pending distant-terrain debounce retries; `adaptive-queue` combines it with the existing controllers. [Queue experiment and measurement limits](docs/debounce-queue.md).
Version 0.10 adds an isolated [rejected-dispatch retry experiment](docs/dispatch-retry.md).
It bounds retry passes when a DH worker rejects queued work, preserving original
requeue handling and allowing other executors to progress. Balanced keeps it off.
Version 0.9 also makes the queue's idle wait wakeable for incoming work, preserving
its debounce rules and bounded cache-maintenance cadence. It remains part of the
optional queue profiles, with separate wake-latency measurements.
@@ -77,7 +82,7 @@ python3 tools/build_mod.py
python3 tools/build_release.py
```
Gitignored `dist/` contains eleven packs, our original instrumentation JAR, client/server starter ZIPs and SHA256SUMS. [mods.lock.json](packs/mods.lock.json) pins mod versions, exact dependency closure, SHA-1/SHA-512 and source/license; [fabric.lock.json](packs/fabric.lock.json) pins loader/library SHA-256. Packs use the [Modrinth format](https://support.modrinth.com/en/articles/8802351-modrinth-modpack-format-mrpack). They embed only original project code; third-party mods, game and assets download from their publishers.
Gitignored `dist/` contains twelve packs, our original instrumentation JAR, client/server starter ZIPs and SHA256SUMS. [mods.lock.json](packs/mods.lock.json) pins mod versions, exact dependency closure, SHA-1/SHA-512 and source/license; [fabric.lock.json](packs/fabric.lock.json) pins loader/library SHA-256. Packs use the [Modrinth format](https://support.modrinth.com/en/articles/8802351-modrinth-modpack-format-mrpack). They embed only original project code; third-party mods, game and assets download from their publishers.
The server starter uses Lithium/FerriteCore and `/horizonserver bench <label> <seconds>`. [Server instructions](docs/server-quickstart.txt) cover setup and a baseline. Files are prepared before you accept Minecraft's EULA locally. A 50-active-player workload remains to be measured.
@@ -0,0 +1,34 @@
{
"schema": 1,
"platform": "Linux-6.17.0-1022-azure-x86_64-with-glibc2.39",
"architecture": "x86_64",
"minecraft": "26.3",
"distant_horizons": "3.3.4",
"results": [
{
"enabled": false,
"submission_allocated_bytes": 216987192,
"submission_cpu_ns": 372949442,
"submission_wall_ns": 401548875,
"rejections": 63790,
"tasks_completed": 2,
"occupied_slots": 0
},
{
"enabled": true,
"submission_allocated_bytes": 103384,
"submission_cpu_ns": 941367,
"submission_wall_ns": 959453,
"rejections": 6,
"tasks_completed": 2,
"occupied_slots": 0
}
],
"notes": [
"Synthetic full physical worker queue; both modes use the actual publisher dispatcher inside Fabric.",
"Allocation and CPU measure the submission caller only; retry scheduler activity is excluded.",
"Upstream keeps the caller blocked for the requested 200 ms observation window; patched returns earlier.",
"Both modes verify exact task counts, recovered occupied slots and fixture worker shutdown.",
"Not a game FPS, terrain completion, whole-game RAM, or hardware compatibility benchmark."
]
}
+70
View File
@@ -0,0 +1,70 @@
# Rejected dispatch retry experiment
A local game recording showed allocation samples repeatedly entering DH's
`ThreadPoolExecutor.AbortPolicy`, exception construction and diagnostic strings.
The direct allocation counters showed substantial allocation on render-loader
workers. This was a scheduler retry path, rather than evidence of useful terrain
generation. Inspecting the pinned DH 3.3.4 dispatcher confirmed that rejection
requeues the same tracked task and immediately tries it again while retaining
the dispatcher lock. A busy or unavailable physical pool can make that loop
allocate and prevent other executors from progressing.
The optional `dispatch-retry` profile uses balanced graphics, mods, distances,
worker count and heap guidance. It enables only the new dispatcher hook; adaptive
budgeting, precise worker pacing and queue waiting stay disabled. Other profiles
explicitly disable the new hook, so existing isolated comparisons remain useful.
It requires exact DH 3.3.4. Set `enabled=false` in
`config/horizon-dispatch-retry.properties` and restart to disable it.
After one rejection for an executor in a dispatch pass, the hook ends that
executor's inner submission loop. Other executors can consume available logical
slots in the same pass. The original catch still requeues the original task or
clears it during shutdown. The original lock, priority ordering, occupied-slot
increments/decrements, exception handling and scheduled-future cancellation
remain in use. A rejected pass requests a later retry after 1 ms; a pass with
only paused executors retains DH's original 1000 ms interval. Shutdown prevents
new retry scheduling. OS timers and lock contention can delay the requested retry.
This bounds repeated rejection within each pass. It still constructs an exception
for a rejected submission, and frequent callers can cause additional passes.
It is not a complete replacement scheduler or proof of faster terrain completion.
No simulation rule, server tick rate, render distance or voxel content is changed.
## Applied-hook integration test
Build with `python tools/build_mod.py`, then run `python tools/test_dispatch.py`.
On Linux the latter uses Xvfb; `--native-display` uses an existing display.
The tool builds a private fixture mod under `.local/mod-build`, runs two isolated
Fabric clients with the dispatcher hook disabled and enabled, and exits before
opening a world. It uses the local demo launcher and requires no account.
Official game libraries/assets and publisher mods remain private downloads.
The fixture never enters a release pack or production JAR.
The fixture creates an actual publisher dispatcher with two executors. It replaces
one executor's physical pool with a one-worker, one-slot pool and deliberately
fills it with blocking work outside the dispatcher. This reproduces rejection
while logical slots remain free. Upstream retains the caller in the retry loop
during a 200 ms observation window and stalls independent queued work. With the
hook, the caller returns and the other executor progresses before releasing the
blocked physical worker. Both runs then require each tracked task to run exactly
once, empty logical queues, zero occupied slots and physical-worker shutdown.
The sanitized output reports the submission caller's wall time, CPU time and
allocated bytes, plus total observed physical-pool rejections. Retry-scheduler
CPU/allocation is excluded from the caller counters. The observation window and
earlier patched return produce different caller durations; these measurements
describe the rejected-submission path, not normal task throughput. Linux CI runs
the applied hooks; Windows and ARM Mac CI compile the fixture and production mod.
In [Linux CI run 37158922187](https://github.com/ludovic111/minecraft-optimization-lab/actions/runs/37158922187),
[the raw fixture result](../benchmarks/dispatch-retry-2026-10-04/linux-applied-hooks.json)
reported 216,987,192 caller allocation bytes and 63,790 rejections upstream,
versus 103,384 caller bytes and 6 rejections with the hook. Caller wall times were
401.549 ms and 0.959 ms respectively. These are single, unwarmed failure-path
observations, including initialization/JIT effects, rather than normal throughput
or a fixed-duration whole-process comparison. Both modes completed two tasks.
An initial patched Linux game also loaded, recorded 120 seconds and shut down
normally. Its subsequent JFR capture avoided the earlier allocation storm.
The captures had different cache histories; they do not establish an FPS or
whole-game memory improvement, and this is not an M2 graphics benchmark.
+1 -1
View File
@@ -73,7 +73,7 @@ continued updating, and worlds/databases/workers closed cleanly. The actual
publisher queue microbenchmark passes identity/priority/count checks on Linux,
Windows and Apple Silicon macOS; its raw results and timing overshoot are documented
in [the queue experiment](debounce-queue.md). These are separate from game-FPS,
block-update-latency or RAM-reduction claims. All eleven pack overrides retain
block-update-latency or RAM-reduction claims. All twelve pack overrides retain
simulation distance 12 and explicitly set the intended experimental flags.
The 0.6.0 resident-memory sampler ran in the actual adaptive-pacing demo world.
@@ -0,0 +1,14 @@
package dev.horizonlab.budget;
/** Per-picker state, used only while its existing dispatch lock is held. */
public final class DispatchRetry {
private boolean rejectedExecutor;
private boolean rejectedPass;
public void beginPass() { rejectedExecutor = false; rejectedPass = false; }
public void beginExecutor() { rejectedExecutor = false; }
public void rejected() { rejectedExecutor = true; rejectedPass = true; }
public int visibleSlots(int occupied) { return rejectedExecutor ? Integer.MAX_VALUE : occupied; }
public boolean needsRetry(boolean original, boolean shutdown) { return !shutdown && (original || rejectedPass); }
public long retryMilliseconds(long original) { return rejectedPass ? 1 : original; }
}
@@ -94,6 +94,7 @@ public final class HorizonClient implements ClientModInitializer {
context.put("precise_worker_pacing", dev.horizonlab.mixin.HorizonMixinPlugin.workerPacingEnabled());
context.put("debounce_queue_wait", dev.horizonlab.mixin.HorizonMixinPlugin.debounceWaitEnabled());
context.put("wakeable_idle_wait", dev.horizonlab.mixin.HorizonMixinPlugin.debounceWaitEnabled());
context.put("dispatch_retry_backoff", dev.horizonlab.mixin.HorizonMixinPlugin.dispatchRetryEnabled());
context.put("adaptive_target_fps", targetFps);
context.put("render_distance", client.options.renderDistance().get());
context.put("simulation_distance", client.options.simulationDistance().get());
@@ -0,0 +1,76 @@
package dev.horizonlab.mixin;
import com.seibel.distanthorizons.core.util.threading.PriorityTaskPicker;
import dev.horizonlab.budget.DispatchRetry;
import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicInteger;
import org.spongepowered.asm.mixin.Final;
import org.spongepowered.asm.mixin.Mixin;
import org.spongepowered.asm.mixin.Pseudo;
import org.spongepowered.asm.mixin.Shadow;
import org.spongepowered.asm.mixin.Unique;
import org.spongepowered.asm.mixin.injection.At;
import org.spongepowered.asm.mixin.injection.Constant;
import org.spongepowered.asm.mixin.injection.Inject;
import org.spongepowered.asm.mixin.injection.ModifyConstant;
import org.spongepowered.asm.mixin.injection.ModifyVariable;
import org.spongepowered.asm.mixin.injection.Redirect;
import org.spongepowered.asm.mixin.injection.callback.CallbackInfo;
@Pseudo
@Mixin(targets = "com.seibel.distanthorizons.core.util.threading.PriorityTaskPicker", remap = false)
public abstract class DhPriorityDispatchMixin {
@Shadow @Final private AtomicInteger occupiedThreadsRef;
@Shadow @Final private AtomicBoolean isShutDownRef;
@Unique private final DispatchRetry horizonDispatch = new DispatchRetry();
// This site is after the original lock acquisition, including the failed-tryLock early return.
@Inject(method = "startNextTask", at = @At(value = "INVOKE",
target = "Lcom/seibel/distanthorizons/core/util/threading/PriorityTaskPicker;getExecutorIteratorSortedByShortestTotalRunTime()Ljava/util/Iterator;"),
require = 1, allow = 1, remap = false)
private void horizonBeginDispatchPass(boolean blocking, CallbackInfo callback) {
horizonDispatch.beginPass();
}
@Redirect(method = "startNextTask", at = @At(value = "INVOKE",
target = "Lcom/seibel/distanthorizons/core/util/threading/PriorityTaskPicker$Executor;canRun()Z"),
require = 1, allow = 1, remap = false)
private boolean horizonBeginExecutor(PriorityTaskPicker.Executor executor) {
horizonDispatch.beginExecutor();
return executor.canRun();
}
@Redirect(method = "startNextTask", at = @At(value = "INVOKE",
target = "Lcom/seibel/distanthorizons/core/util/threading/PriorityTaskPicker$Executor;runTask(Ljava/lang/Runnable;)V"),
require = 1, allow = 1, remap = false)
private void horizonMarkRejected(PriorityTaskPicker.Executor executor, Runnable task) {
try {
executor.runTask(task);
} catch (RejectedExecutionException rejected) {
horizonDispatch.rejected();
throw rejected; // Original catch requeues the same task or clears it during shutdown.
}
}
@Redirect(method = "startNextTask", at = @At(value = "INVOKE",
target = "Ljava/util/concurrent/atomic/AtomicInteger;get()I"), require = 1, allow = 1, remap = false)
private int horizonLeaveRejectedExecutor(AtomicInteger counter) {
int occupied = counter.get();
return counter == occupiedThreadsRef
? horizonDispatch.visibleSlots(occupied) : occupied;
}
// Inspected DH 3.3.4 local slot 2 is the existing "executor paused" retry flag.
@ModifyVariable(method = "startNextTask", at = @At("LOAD"), index = 2,
require = 1, allow = 1, remap = false)
private boolean horizonRetryRejectedPass(boolean original) {
return horizonDispatch.needsRetry(original, isShutDownRef.get());
}
@ModifyConstant(method = "startNextTask", constant = @Constant(longValue = 1000L),
require = 1, allow = 1, remap = false)
private long horizonRetryDelay(long original) {
return horizonDispatch.retryMilliseconds(original);
}
}
@@ -14,6 +14,7 @@ import org.spongepowered.asm.mixin.extensibility.IMixinInfo;
public final class HorizonMixinPlugin implements IMixinConfigPlugin {
private static boolean workerPacing;
private static boolean debounceWait;
private static boolean dispatchRetry;
@Override public void onLoad(String mixinPackage) {
var loader = FabricLoader.getInstance();
@@ -47,13 +48,30 @@ public final class HorizonMixinPlugin implements IMixinConfigPlugin {
} catch (Exception error) {
LoggerFactory.getLogger("horizonlab").warn("Debounce queue wait disabled: invalid configuration", error);
}
try {
Properties properties = new Properties();
var dispatchFile = loader.getConfigDir().resolve("horizon-dispatch-retry.properties");
if (Files.exists(dispatchFile)) {
try (var reader = Files.newBufferedReader(dispatchFile)) { properties.load(reader); }
}
boolean requested = Boolean.parseBoolean(properties.getProperty("enabled", "false"));
boolean supported = loader.getModContainer("distanthorizons")
.map(mod -> mod.getMetadata().getVersion().getFriendlyString().equals("3.3.4")).orElse(false);
dispatchRetry = requested && supported;
if (requested) LoggerFactory.getLogger("horizonlab").info(
"DH rejected dispatch retry: {} (requires pinned DH 3.3.4)", dispatchRetry ? "enabled" : "unavailable");
} catch (Exception error) {
LoggerFactory.getLogger("horizonlab").warn("Dispatch retry disabled: invalid configuration", error);
}
}
public static boolean workerPacingEnabled() { return workerPacing; }
public static boolean debounceWaitEnabled() { return debounceWait; }
public static boolean dispatchRetryEnabled() { return dispatchRetry; }
@Override public boolean shouldApplyMixin(String targetClassName, String mixinClassName) {
if (mixinClassName.endsWith("DhWorkerPacingMixin")) return workerPacing;
if (mixinClassName.endsWith("DhDebounceWaitMixin")) return debounceWait;
if (mixinClassName.endsWith("DhPriorityDispatchMixin")) return dispatchRetry;
return true;
}
@Override public String getRefMapperConfig() { return null; }
@@ -1,7 +1,7 @@
{
"schemaVersion": 1,
"id": "horizonlab",
"version": "0.9.0",
"version": "0.10.0",
"name": "Horizon Lab",
"description": "Client frame pacing and server tick measurements for reproducible optimization experiments.",
"authors": [
@@ -4,6 +4,6 @@
"plugin": "dev.horizonlab.mixin.HorizonMixinPlugin",
"compatibilityLevel": "JAVA_25",
"mixins": ["ServerTickMixin"],
"client": ["FrameMixin", "DhWorkerPacingMixin", "DhDebounceWaitMixin"],
"client": ["FrameMixin", "DhWorkerPacingMixin", "DhDebounceWaitMixin", "DhPriorityDispatchMixin"],
"injectors": {"defaultRequire": 1}
}
@@ -0,0 +1,23 @@
package dev.horizonlab.budget;
public final class DispatchRetryTest {
private static void check(boolean condition) { if (!condition) throw new AssertionError(); }
public static void main(String[] args) {
DispatchRetry policy = new DispatchRetry();
policy.beginPass();
policy.beginExecutor();
check(policy.visibleSlots(0) == 0 && policy.visibleSlots(3) == 3);
check(!policy.needsRetry(false, false) && policy.needsRetry(true, false));
check(policy.retryMilliseconds(1000) == 1000);
policy.rejected();
check(policy.visibleSlots(0) == Integer.MAX_VALUE && policy.needsRetry(false, false));
check(policy.retryMilliseconds(1000) == 1);
policy.beginExecutor();
check(policy.visibleSlots(1) == 1); // Another executor can consume a real free slot.
check(policy.needsRetry(false, false)); // Still retry the earlier rejected executor.
check(!policy.needsRetry(false, true) && !policy.needsRetry(true, true));
policy.beginPass();
check(!policy.needsRetry(false, false) && policy.retryMilliseconds(1000) == 1000);
System.out.println("Dispatch retry policy tests passed.");
}
}
@@ -0,0 +1,108 @@
package dev.horizonlab.dispatch;
import com.seibel.distanthorizons.core.util.threading.PriorityTaskPicker;
import com.seibel.distanthorizons.core.util.threading.RateLimitedThreadPoolExecutor;
import dev.horizonlab.mixin.HorizonMixinPlugin;
import java.lang.reflect.Field;
import java.lang.management.ManagementFactory;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
import net.fabricmc.api.ClientModInitializer;
import net.fabricmc.fabric.api.client.event.lifecycle.v1.ClientLifecycleEvents;
/** Private fixture mod, excluded from the production JAR. Run inside Fabric to test applied hooks. */
public final class DispatchIntegration implements ClientModInitializer {
@Override public void onInitializeClient() {
ClientLifecycleEvents.CLIENT_STARTED.register(client -> {
try { exercise(HorizonMixinPlugin.dispatchRetryEnabled()); }
catch (Exception failure) { throw new AssertionError("Dispatch integration failed", failure); }
if (Boolean.getBoolean("horizon.fixture.exit")) System.exit(0);
});
}
private static void exercise(boolean enabled) throws Exception {
PriorityTaskPicker picker = new PriorityTaskPicker();
var blocked = picker.createExecutor("Horizon fixture blocked", 1);
var independent = picker.createExecutor("Horizon fixture independent", 2);
Field poolField = blocked.getClass().getDeclaredField("threadPoolExecutor");
poolField.setAccessible(true);
((RateLimitedThreadPoolExecutor) poolField.get(blocked)).shutdownNow();
var physical = new RateLimitedThreadPoolExecutor(1, runnable -> {
Thread thread = new Thread(runnable, "Horizon fixture physical");
thread.setDaemon(true);
return thread;
}, new ArrayBlockingQueue<Runnable>(1));
AtomicInteger rejections = new AtomicInteger();
var abort = new ThreadPoolExecutor.AbortPolicy();
physical.setRejectedExecutionHandler((task, pool) -> {
rejections.incrementAndGet();
abort.rejectedExecution(task, pool);
});
poolField.set(blocked, physical);
CountDownLatch entered = new CountDownLatch(1), release = new CountDownLatch(1);
CountDownLatch submitted = new CountDownLatch(1), completed = new CountDownLatch(2);
AtomicInteger firstRuns = new AtomicInteger(), secondRuns = new AtomicInteger();
AtomicLong callerBytes = new AtomicLong(), callerCpu = new AtomicLong(), callerWall = new AtomicLong();
AtomicReference<Throwable> callerFailure = new AtomicReference<>();
var threads = (com.sun.management.ThreadMXBean) ManagementFactory.getThreadMXBean();
threads.setThreadAllocatedMemoryEnabled(true);
threads.setThreadCpuTimeEnabled(true);
try {
// Physical work is deliberately outside the picker: reproduce a busy executor with free logical slots.
physical.execute(() -> {
entered.countDown();
try { release.await(10, TimeUnit.SECONDS); }
catch (InterruptedException interrupted) { Thread.currentThread().interrupt(); }
});
if (!entered.await(5, TimeUnit.SECONDS)) throw new AssertionError("Physical worker did not start");
physical.execute(() -> {}); // Fill its only physical queue slot.
Thread caller = new Thread(() -> {
long id = Thread.currentThread().threadId();
long bytes = threads.getThreadAllocatedBytes(id), cpu = threads.getCurrentThreadCpuTime(), wall = System.nanoTime();
try { blocked.execute(() -> { firstRuns.incrementAndGet(); completed.countDown(); }); }
catch (Throwable failure) { callerFailure.set(failure); }
finally {
callerWall.set(System.nanoTime() - wall);
callerBytes.set(threads.getThreadAllocatedBytes(id) - bytes);
callerCpu.set(threads.getCurrentThreadCpuTime() - cpu);
submitted.countDown();
}
}, "Horizon fixture caller");
caller.setDaemon(true);
caller.start();
boolean returned = submitted.await(enabled ? 2000 : 200, TimeUnit.MILLISECONDS);
if (returned != enabled) throw new AssertionError("Unexpected rejection lock behavior: hook=" + enabled);
independent.execute(() -> { secondRuns.incrementAndGet(); completed.countDown(); });
long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(5);
if (enabled) {
while (secondRuns.get() == 0 && System.nanoTime() < deadline) Thread.sleep(1);
if (secondRuns.get() != 1 || firstRuns.get() != 0) throw new AssertionError("Blocked executor prevented independent work");
} else if (secondRuns.get() != 0) throw new AssertionError("Upstream did not reproduce dispatcher blocking");
release.countDown();
if (!completed.await(5, TimeUnit.SECONDS)) throw new AssertionError("Requeued task was lost");
caller.join(5000);
if (caller.isAlive() || callerFailure.get() != null) throw new AssertionError("Submission failed", callerFailure.get());
Field occupiedField = PriorityTaskPicker.class.getDeclaredField("occupiedThreadsRef");
occupiedField.setAccessible(true);
AtomicInteger occupied = (AtomicInteger) occupiedField.get(picker);
deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(5);
while (occupied.get() != 0 && System.nanoTime() < deadline) Thread.sleep(1);
if (occupied.get() != 0 || firstRuns.get() != 1 || secondRuns.get() != 1
|| blocked.getQueueSize() != 0 || independent.getQueueSize() != 0)
throw new AssertionError("Task count, queue, or occupied slots corrupted");
System.out.println("HORIZON_DISPATCH_RESULT {\"enabled\":" + enabled
+ ",\"submission_allocated_bytes\":" + callerBytes.get()
+ ",\"submission_cpu_ns\":" + callerCpu.get() + ",\"submission_wall_ns\":" + callerWall.get()
+ ",\"rejections\":" + rejections.get() + ",\"tasks_completed\":2,\"occupied_slots\":0}");
} finally {
release.countDown();
picker.shutdownNow();
if (!physical.awaitTermination(5, TimeUnit.SECONDS)) throw new AssertionError("Fixture worker did not shut down");
}
}
}
+7 -1
View File
@@ -9,7 +9,8 @@ YOUR MAC
2. Sign into your Microsoft account in Prism yourself. No credentials are
shared with this project. You need an owned Minecraft Java license.
3. Enable Prism automatic Java downloads, using Java 25 ARM64 on Apple Silicon.
Instance > Edit > Settings > Java: maximum memory 6144 MiB, minimum 512 MiB.
Instance > Edit > Settings > Java: check the Memory override box, then set
maximum memory 6144 MiB and minimum 512 MiB.
If auto Java fails, install a native ARM64 Java 25 JDK and select it there.
4. Launch. Start with a new test world or a COPY of a world from Minecraft 26.3.
This release is 26.3; do not load an older-version world for a fair comparison.
@@ -52,6 +53,11 @@ PROFILES
For an isolated comparison use queue-wait versus m2-balanced. For the combined
controls use adaptive-queue versus adaptive-pacing. Disable the queue hook in
config/horizon-queue-wait.properties (enabled=false), then restart.
- dispatch-retry: same balanced settings, with bounded retries when DH rejects
a task submission. The original task is requeued; other executors can progress.
This is a separate experiment: adaptive, worker-pacing and queue-wait are off.
Set enabled=false in config/horizon-dispatch-retry.properties and restart
to disable it. A synthetic applied-hook test checks exact task completion.
- low-memory: 8 full-detail chunks, no LOD. Set maximum memory 3072 MiB.
For a matched comparison set both instances to the same render distance.
- far-distance: 12 full-detail plus 256 LOD chunks. Start with 8192 MiB ONLY on
+1 -1
View File
@@ -3,7 +3,7 @@
"minecraft": "26.3",
"java_major": 25,
"fabric_loader": "0.19.5",
"pack_version": "0.9.0",
"pack_version": "0.10.0",
"options_data_version": 5023,
"projects": {
"sodium": {
+12
View File
@@ -50,6 +50,18 @@ class PackTests(unittest.TestCase):
with self.assertRaisesRegex(ValueError, "Missing pinned dependency"):
build_pack.validate_selection(build_pack.read_lock(), ["iris"])
def test_dispatch_experiment_does_not_enable_other_controllers(self):
for name, profile in build_pack.PROFILES.items():
files = build_pack.overrides(name, b"original fixture")
dispatch = files["config/horizon-dispatch-retry.properties"].decode()
self.assertEqual(dispatch, "enabled=true\n" if name == "dispatch-retry" else "enabled=false\n")
if name == "dispatch-retry":
for setting in ["worker-pacing", "queue-wait"]:
self.assertEqual(files[f"config/horizon-{setting}.properties"], b"enabled=false\n")
self.assertTrue(files["config/horizon-adaptive.properties"].startswith(b"enabled=false\n"))
self.assertEqual(files["config/DistantHorizons.toml"],
build_pack.overrides("m2-balanced", b"original fixture")["config/DistantHorizons.toml"])
def test_wrong_game_or_download_path_is_rejected(self):
for field, value in [("game_versions", ["1.21"]), ("filename", "../bad.jar"), ("url", "http://unsafe.invalid")]:
lock = copy.deepcopy(build_pack.read_lock())
+1
View File
@@ -100,6 +100,7 @@ function renderDetails(side){
const r=entry.report,fields=[['Profile',r.profile||r.label||'Unlabeled'],['Recording',format(r.elapsed_seconds)+' s · '+format(r.stats.count)+' samples'],['Platform',[r.os,r.architecture].filter(Boolean).join(' ')||'Not captured']];
if(r.kind==='frames')fields.push(['Graphics',[r.gpu,r.graphics_backend].filter(Boolean).join(' · ')||'Not captured'],['Window',number(r.framebuffer_width)?r.framebuffer_width+' × '+r.framebuffer_height:'Not captured'],['Experiments','Adaptive: '+(r.adaptive_budget_enabled?'on':'off')+' · precise pacing: '+(r.precise_worker_pacing?'on':'off')+' · queue wait: '+(r.debounce_queue_wait?'on':'off')]);
fields.push(['JVM', 'Compact headers: '+(r.compact_object_headers===true?'on':r.compact_object_headers===false?'off':'not captured')+' · GC: '+(Array.isArray(r.garbage_collectors)?r.garbage_collectors.join(', '):'not captured')]);
fields.push(['Rejected dispatch retry', r.dispatch_retry_backoff===true?'on':r.dispatch_retry_backoff===false?'off':'not captured']);
fields.push(['Wakeable queue idle', r.wakeable_idle_wait===true?'on':r.wakeable_idle_wait===false?'off':'not captured']);
for(const [label,value]of fields)target.append(text('dt',label),text('dd',value));
}
+26
View File
@@ -47,6 +47,10 @@ def test_metrics():
SOURCE / "test/java/dev/horizonlab/budget/WakeableIdleWaitTest.java"]
subprocess.run(["javac", "--release", "25", "-d", str(destination), *map(str, idle_sources)], check=True)
subprocess.run(["java", "-cp", str(destination), "dev.horizonlab.budget.WakeableIdleWaitTest"], check=True)
dispatch_sources = [SOURCE / "main/java/dev/horizonlab/budget/DispatchRetry.java",
SOURCE / "test/java/dev/horizonlab/budget/DispatchRetryTest.java"]
subprocess.run(["javac", "--release", "25", "-d", str(destination), *map(str, dispatch_sources)], check=True)
subprocess.run(["java", "-cp", str(destination), "dev.horizonlab.budget.DispatchRetryTest"], check=True)
def build():
@@ -109,6 +113,28 @@ def build():
return output
def build_dispatch_fixture():
"""Create a private integration-test mod; never include it in release packs."""
arguments = [json.loads(line) for line in (BUILD / "javac.args").read_text().splitlines()]
classpath = arguments[arguments.index("-classpath") + 1] + os.pathsep + str(BUILD / "classes")
classes = BUILD / "dispatch-fixture"
if classes.exists():
shutil.rmtree(classes)
classes.mkdir(parents=True)
subprocess.run(["javac", "--release", "25", "-proc:none", "-classpath", classpath,
"-d", str(classes), str(SOURCE / "test/java/dev/horizonlab/dispatch/DispatchIntegration.java")], check=True)
output = BUILD / "dispatch-fixture.jar"
with zipfile.ZipFile(output, "w", compression=zipfile.ZIP_DEFLATED) as archive:
for path in sorted(classes.rglob("*.class")):
archive.write(path, path.relative_to(classes).as_posix())
archive.writestr("fabric.mod.json", json.dumps({"schemaVersion": 1, "id": "horizon_dispatch_fixture",
"version": "1.0.0", "environment": "client",
"entrypoints": {"client": ["dev.horizonlab.dispatch.DispatchIntegration"]},
"depends": {"horizonlab": "*", "distanthorizons": "3.3.4"}}))
print(f"Private dispatch integration fixture: {output}")
return output
def benchmark_workers():
arguments = [json.loads(line) for line in (BUILD / "javac.args").read_text().splitlines()]
classpath = os.pathsep.join([str(BUILD / "classes"), arguments[arguments.index("-classpath") + 1]])
+4 -1
View File
@@ -23,6 +23,7 @@ PROFILES = {
"adaptive-pacing": {"title": "Adaptive precise pacing experiment", "mods": CORE + ["distanthorizons"], "render": 12, "lod": 128, "heap": 6144, "adaptive": True, "pacing": True},
"queue-wait": {"title": "Debounce queue wait experiment", "mods": CORE + ["distanthorizons"], "render": 12, "lod": 128, "heap": 6144, "queue_wait": True},
"adaptive-queue": {"title": "Adaptive pacing and queue wait experiment", "mods": CORE + ["distanthorizons"], "render": 12, "lod": 128, "heap": 6144, "adaptive": True, "pacing": True, "queue_wait": True},
"dispatch-retry": {"title": "Rejected dispatch retry experiment", "mods": CORE + ["distanthorizons"], "render": 12, "lod": 128, "heap": 6144, "dispatch_retry": True},
"low-memory": {"title": "Low memory", "mods": CORE, "render": 8, "lod": 0, "heap": 3072},
"far-distance": {"title": "Far distance", "mods": CORE + ["distanthorizons"], "render": 12, "lod": 256, "heap": 8192},
"cinematic": {"title": "Cinematic (optional)", "mods": CORE + ["distanthorizons", "iris", "complementary-reimagined"], "render": 12, "lod": 128, "heap": 6144},
@@ -76,7 +77,7 @@ def overrides(profile_name, mod_bytes):
files = {f"mods/horizon-lab-{read_lock()['pack_version']}.jar": mod_bytes,
"config/horizon-profile.txt": (profile_name + "\n").encode(),
"options.txt": options(profile).encode()}
experiment = any(profile.get(key, False) for key in ["adaptive", "pacing", "queue_wait"])
experiment = any(profile.get(key, False) for key in ["adaptive", "pacing", "queue_wait", "dispatch_retry"])
config = ROOT / "packs/config" / f"{'m2-balanced' if experiment else profile_name}.toml"
files["config/horizon-adaptive.properties"] = (
f"enabled={str(profile.get('adaptive', False)).lower()}\ntargetFps=60\n").encode()
@@ -84,6 +85,8 @@ def overrides(profile_name, mod_bytes):
f"enabled={str(profile.get('pacing', False)).lower()}\n").encode()
files["config/horizon-queue-wait.properties"] = (
f"enabled={str(profile.get('queue_wait', False)).lower()}\n").encode()
files["config/horizon-dispatch-retry.properties"] = (
f"enabled={str(profile.get('dispatch_retry', False)).lower()}\n").encode()
if profile["lod"]:
if not config.exists():
raise FileNotFoundError(f"Missing verified DH profile config: {config}")
+64
View File
@@ -0,0 +1,64 @@
#!/usr/bin/env python3
"""Exercise applied DH dispatch hooks in isolated Fabric clients; no world or account needed."""
import argparse
import json
import os
from pathlib import Path
import platform
import shutil
import subprocess
import sys
try:
from . import build_mod, build_pack
except ImportError:
import build_mod
import build_pack
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--native-display", action="store_true", help="Use the existing display instead of Linux Xvfb")
args = parser.parse_args()
root = build_pack.ROOT
fixture = build_mod.build_dispatch_fixture()
version = build_pack.read_lock()["pack_version"]
mod = root / "dist" / f"horizon-lab-{version}.jar"
results = []
for enabled in [False, True]:
instance = root / ".local/runtime" / ("dispatch-fixture-patched" if enabled else "dispatch-fixture-upstream")
build_pack.prepare_instance("dispatch-retry", instance, mod)
shutil.copy2(fixture, instance / "mods/dispatch-fixture.jar")
(instance / "config/horizon-dispatch-retry.properties").write_text(f"enabled={str(enabled).lower()}\n")
command = [sys.executable, str(root / "tools/runtime.py"), "demo", "--instance", str(instance),
"--heap", "3G", "--compact-headers"]
if platform.system() == "Linux" and not args.native_display:
command = ["xvfb-run", "-a", *command]
environment = os.environ.copy()
environment["_JAVA_OPTIONS"] = "-Dhorizon.fixture.exit=true"
run = subprocess.run(command, cwd=root, env=environment, text=True, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, timeout=240)
(instance / "fixture.log").write_text(run.stdout)
marker = "HORIZON_DISPATCH_RESULT "
reports = [json.loads(line.split(marker, 1)[1]) for line in run.stdout.splitlines() if marker in line]
if run.returncode or len(reports) != 1:
raise RuntimeError(f"Dispatch fixture failed; inspect {instance / 'fixture.log'}")
report = reports[0]
if report["enabled"] != enabled or report["tasks_completed"] != 2 or report["occupied_slots"] != 0:
raise AssertionError("Fixture outcome did not match the selected hook")
results.append(report)
print(json.dumps(report), flush=True)
output = root / "dist/dispatch-integration.json"
output.write_text(json.dumps({"schema": 1, "platform": platform.platform(), "architecture": platform.machine(),
"minecraft": "26.3", "distant_horizons": "3.3.4", "results": results,
"notes": ["Synthetic full physical worker queue; both modes use the actual publisher dispatcher inside Fabric.",
"Allocation and CPU measure the submission caller only; retry scheduler activity is excluded.",
"Upstream keeps the caller blocked for the requested 200 ms observation window; patched returns earlier.",
"Both modes verify exact task counts, recovered occupied slots and fixture worker shutdown.",
"Not a game FPS, terrain completion, whole-game RAM, or hardware compatibility benchmark."]}, indent=2) + "\n")
print(f"Applied-hook integration checks passed: {output}")
if __name__ == "__main__":
main()