horizon-airdef/docs/validation/server-thread-migration.md

# Server-thread integration coverage migration: validation ledger

Tested on 2026-10-24, Linux / NVIDIA Vulkan, with Java 25 and the pinned 26.3
resources and jars. Base: `e423e570c`. Player integration uses real Horizon
clients under Xvfb and real control sockets. The migrated game drivers use fresh
temporary game, launcher and XDG data/config homes; telemetry, audio and launcher
self-update are disabled. No player data or prepared resources are committed.
An interrupted historical-world generator initially inherited the harness
environment instead of those per-game homes. It wrote only to ignored `.local`
fixture folders; it was stopped, and subsequent validation uses the already
prepared historical worlds. That setup mistake is not counted as a passing run.

## Before editing

Open PR searches for server-thread, integration, LAN and friends work found no
overlapping migration. The full original LAN suite was started with
`--server-thread` against a build of the unchanged base before repository edits.

| Run | Hosting | Result |
| --- | --- | --- |
| Original full LAN attempt | Server thread | 58 checks reported: 39 passed, 18 failed, deflection skipped; the runner then aborted at `light` with an uncaught `TypeError`. |
| Original remaining 45 checks | Server thread | 31 passed, 14 failed, no skips. Used the unchanged loaded driver and original executable. |
| Friends, including `--baseline` | Server thread | Plain LAN, direct invitation, local relay and ended-invite rejection passed. |

Together the two LAN invocations attempted all 103 original checks. The initial
`sign` case left its editor open, breaking the later death-screen assertion.
`predict` suspended the whole host process; resuming it interrupted a socket
read (`Interrupted system call`), disconnected the guest and caused later
session failures. The remaining run also exposed sign placement outside reach
and cascading dead-player failures. These are recorded failures, not passes.

## Infrastructure changes validated during migration

- Close the sign editor after the original placement check. Give the two-player
  sign case a reachable arena and wait for placement data/feedback before typing.
  Observe the submitted text on both clients within the existing assertion budget.
  The revised sign case passed three consecutive server-thread reruns.
- Release held input, close screens and respawn dead players before independent
  cases. Keep every case's death, menu and held-action assertions inside the case.
- Replace process signals with a server barrier for prediction and an intentional
  window-thread stall for frame-recovery tests. Prediction still tests immediate
  client prediction, authoritative agreement, iron-door rejection and exactly-once
  sound; it also asserts that the server clock did not advance at the barrier.
- Route mob/projectile/hurt-log inspection to the owning server. Deflection runs
  in the default suite rather than reporting an in-frame-only skip as a pass.
- Record skipped checks explicitly as uncovered, and record ordinary case errors
  without aborting the rest of the suite. Write mode and results to `results.json`.
- Serialize the two CI integration entry points on the shared GPU. An earlier
  concurrent comparison failed to start a graphics device because of GPU memory
  exhaustion; it was rerun serially, without changing a gameplay timeout.

## Post-migration runs

The first complete server-thread migration run, before the case-cleanup fixes,
ran all 103 original cases: 88 passed and 15 failed. Failures were `deflect`,
`light`, `swings`, `cracks`, `effects`, `portals`, `flight`, `boats`, `minecarts`,
`frames`, `hands`, `riding`, `mounts`, `contacts` and `absorption`. Many followed
the host's deflection damage/death. They are not used as evidence of a clean suite.

The next complete default server-thread run executed all 107 cases (the original
103 plus four separate host/guest sensor and dispenser input cases): **97 passed,
10 failed, zero skipped**. Failures were `commands`, `deflect`, `swings`, `flight`,
`boats`, `minecarts`, `keepinv`, `fishing`, `sculk_input_host` and
`sculk_input_guest`. Lighting passed this run; earlier lighting failures are not
claimed as a persistent blocker.

A subsequent input-timing correction runs the existing double-jump key/wait
sequence through one `horizon-ctl script` process, preserving the seven-tick
flight window. A targeted server-thread rerun passed flight for both players,
swings, boats and minecarts. Command completion still failed: the observable
suggestion list did not appear within the existing two-tick budget. The in-frame
rerun passed flight and failed completion in the same way. Passing isolated
swings/vehicle cases does not clear the unexplained failures in the full sequence.
The full-run inventory retention and fishing failures also remain unresolved.

Friends passed plain LAN, direct invitation, local relay and ended-invite checks
both after making server-thread hosting the default and with explicit `--in-frame`.

Targeted in-frame comparisons passed chest, sign placement, prediction, clock
recovery, lighting and deflection. The revised two-player sign case and both
new dispenser input cases passed too. Command completion failed twice: Tab after
`/gamem` left `/gamem` unchanged. The host's sculk input passed; the guest's failed.

Targeted server-thread checks passed prediction and both new dispenser input
cases. Dispensing checks authoritative inventory consumption and exactly one
matching item on each client. Both sensor cases confirmed movement on the client
and server, but the sensor stayed inactive. Deflection still failed the original
no-fireball-damage assertion after allowing the click through a normal client
input tick. An experimental frozen-world synchronization approach also failed
and is not part of the final implementation.

## Deterministic rule and source checks

`python3 tools/parity.py command_exec dispenser sculk --timeout 1500 --jobs 2`:

| Direct fixture | Result |
| --- | --- |
| Command execution | 781/781 command/callback records match the vanilla oracle. |
| Dispenser and related rule calls | 6000 cases match; no difference groups. |
| Sculk | 435 scenarios match every compared tick. |

All three harnesses passed. Their seeded rule calls and controlled ticks remain
direct; they do not claim client/server boundary coverage. The existing
`net_harness.rs` remains a real dedicated-server client fixture.

The first quick self-check passed 289 Python tests, rustfmt and Clippy. Rust tests
reported 1081 passed, one failed and 26 ignored: the shared prepared resource
folder lacked `registries.json`. The pinned local datagen report was copied into
a private ignored resource copy; the failed
`mod_content::tests::content_root_loads_with_every_table` test then passed.
The focused environment/parity-runner/self-check Python tests passed (38 tests).

The full `python3 tools/pr_ready.py --base origin/main` completed: 1205 Rust tests
and 291 Python tests passed, as did rustfmt, Clippy and documentation checks.
Its parity phase failed after the historical-world setup interruption described
above, so the overall self-check is **not green**. A subsequent `parity.py --changed origin/main --timeout 1500` selected 160
harnesses, 151 requiring runs. Three completed and passed (`firework_tooltip`,
`held_drop`, `interaction_input`). Both integration wrappers exceeded that
invocation's 25-minute per-harness deadline: friends was waiting for the shared
GPU lock, and LAN was still executing cases. Automatic retries were stopped and
the detached game processes cleaned up. The remaining affected harnesses did not
complete. This is incomplete validation, not a passing corrected parity gate;
compilation is not a substitute for the gameplay failures above.

The final source self-check, `python3 tools/pr_ready.py --quick`, passed 1205 Rust
tests, 291 Python tests, rustfmt, Clippy and documentation checks. After correcting
the integration build arguments to name each binary (so traced selection includes
`game.rs` and `game/integrated.rs`), the named client/control/relay build and 38
focused Python tests passed. Dependency extraction from the real Cargo dep-info
confirmed both game files are selected.

## Limitations recorded before the follow-up fixes

These were the remaining blockers when this ledger was first written; the
completion runs below record their status after the follow-up fixes.

- Server-player footsteps do not reach sculk sensors. `tick_entity_events` in
  `game/sculk.rs` emits player motion events from `Game.player`; the integrated
  server's players live in its guest hub. The in-frame host passes and its guest
  fails too, separating this shared server-player gap from the deterministic rule
  oracle. A fix needs per-player event identity/state rather than weakening the test.
- The server-thread host takes fireball damage in deflection; the equivalent
  in-frame check passes. Client-visible and authoritative projectile state diverge
  during the synchronization investigation. No gameplay fix is included.
- Command completion fails in both modes as described above; it is not reported
  as a passing fallback comparison.
- Full-sequence swings, vehicle, inventory-retention and fishing failures need
  further fixture/state investigation. Swings and both vehicles passed isolated
  reruns; that evidence does not establish a gameplay defect or a full-suite pass.

No failing case is skipped, expectation relaxed, or existing timeout increased to
hide these results. At that stage the PR stayed draft with these coverage blockers.

## Combined validation before the gameplay split (2026-10-25)

Merged `main` at `bd591f985` and resolved the overlapping motion, projectile and
test-driver changes. The shared projectile inertia helper retains main's custom
acceleration power, with a regression test. Integration environments now call
`lab.game_env`; the friends entry point delegates its actual launches to that
isolated driver. The dedicated-server process keeps one environment argument,
and the control-tool connection test uses a scratch home rather than reading
the player's No AI setting.

The first complete run after that merge executed **119 cases: 114 passed,
five failed, zero skipped**. Failures were `swings`, `flight`, `boats`,
`minecarts` and `keepinv`. All the additional regression cases from main passed.
Command completion, deflection, fishing and both sensor/dispenser input roles
also passed in that full sequence.

The failures exposed fixture setup, input cadence and an off-hand clearing bug:

- Aim at the hoe target's top face, observe that target, then send a normal
  mouse click. Keep the exact one-swing/farmland assertion. A subsequent full
  repeat passed flight and both vehicle checks but still failed the hoe case;
  it was stopped after identifying the root cause rather than counted as green.
  The earlier explosion case leaves an off-hand shield, and `/clear` omitted
  that slot. Vanilla correctly blocks hoe use with that shield. Follow
  `Inventory.clearOrCountMatchingItems`, `getContainerSize`, `getItem` and its
  equipment-slot mapping: clear off-hand items and visit armor feet to head.
  New oracle cases cover off-hand counting, partial/full removal and limited
  armor-slot ordering. The explosion-then-swings sequence passes after this fix.
- Keep the original double-jump key sequence and seven-tick flight rule; run
  this tick-sensitive case at four ticks per second so IPC scheduling on a busy
  machine cannot stretch the input past that window. Restore 20 TPS afterward.
- Deliver the original ten vehicle punches and two-tick waits over one control
  connection. Separate process launches had stretched the gaps to ten ticks,
  letting the vehicle's damage decay before the next hit.
- Protect inactive inventory-test players after the low-health totem case,
  assert that the active player is alive before the intended death, and provide
  a full floor under scattered death drops. A targeted retry with the old small
  platform lost a helmet over its edge; both players' exact drop/retention
  assertions passed with the larger floor.

Targeted server-thread reruns passed all five corrected cases, plus totems,
fishing and both sensor input roles. Friends passed plain LAN, direct invites,
the local relay and ended-invite rejection. The six focused parity harnesses
passed: `command_exec`, `dispenser`, `sculk`, `dedicated_server`,
`server_entities` and `server_player_damage`.

The normal-runner full repeat also exposed an XP observation race: the guest
had already collected all 21 points before the two-tick in-flight snapshot
(`seen=0`, with the correct XP gain and no remaining orbs). Freeze spawning,
step two world ticks, observe the tracked orbs, then unfreeze for the original
pickup assertions. The targeted check passed with three visible orbs and
exactly 21 points; the full runner restarted with that correction.

That repeat passed the former XP, swings, flight, boat and minecart failures,
but its level-ticking subcase found a wandering pig on a neighboring block at
Y=66 instead of the landing floor at Y=65. The armor stand landed correctly.
This subcase measures falling and simultaneous level ticking. Each level now
gets a full clear landing/walking area, retaining the pigs' normal AI. A
hazard-then-levels rerun confirmed that both pigs and armor stands land at the
expected heights with distinct entity IDs across levels.

The broader parity invocation completed **116/117 runs passed**. The remaining
`tab_list` comparison matched 998/1000 lists and all 400 skin images. Both
list mismatches contained `Ⅻ`: the oracle's bitmap-only font setup used the
missing glyph instead of vanilla's Unicode fallback. Load the full default font
reference chain from the prepared assets with vanilla's provider definitions
and default option filters. Prefetch the fake skin lookups together before the
unchanged one-second observation window, retaining deliberately pending cases.
A fresh rerun passed **1000/1000 lists, 3510 frames and 400/400 skin images**,
with zero oracle errors. Fresh secure-chat and dimension-player parity reruns
passed after the command fix as well. The extended command oracle matched all
**843 command/callback records**.

Rechecking the shared font helper passed eight of nine additional harnesses;
`pack_screen` matched 2778/3000 cases with its old bitmap-only oracle. Its
newline/control glyph metrics also require the Unicode fallback (the newline
advance is 9 with the full font, 6 with the bitmap-only setup). Give this oracle
the same full default font and prepared asset root. A fresh rerun matched
**3000/3000 pack screens**, with zero oracle errors and unchanged comparisons.

The source self-check passed **1237 Rust tests and 270 Python tests**, rustfmt,
Clippy and doc tests. The private resource copy contains both pinned reports
and a regular `facts.json`, as required by the resource-isolation regression.
Prepared historical test worlds were copied into the ignored fixture cache
where available; they are not player saves or committed resources.

The final in-frame comparison passed **14/14 cases, zero skipped**: commands,
deflection, both sensor roles, flight, swings, boats, minecarts, chest, prediction,
signs, both dispenser roles and inventory retention. A separate in-frame XP
comparison passed with three visible orbs and exactly 21 points. Friends and
the dedicated-server harness passed through the normal parity runner.

The final normal-runner server-thread LAN repeat passed **119/119 cases,
zero skipped**, in 1940 seconds. This full sequence includes all earlier failure
cases, both sensor/dispenser roles and the additional regressions from main.
The final full `python3 tools/pr_ready.py` gate passed against merge base
`bd591f985`: **1237 Rust tests, 270 Python tests**, rustfmt, Clippy, public-tree
checks and documentation tests. Its parity stage selected **118 affected
harnesses**; 117 had current content-matched passing records and the remaining
harness passed during the gate. No failed or skipped harness remained.
The gate reported the existing scope warning (35 files, 584 lines of code);
the combined migration exceeded the suggested 500-line size. At that point
the combined PR was marked ready for review. The split below supersedes that
readiness status and the earlier description block.


## Gameplay fixes split into independent PRs (2026-10-25)

At the user's request, the three gameplay fixes were removed from this PR's
net diff and opened independently against `main`:

- [#1903](https://github.com/ludovic111/horizon/pull/1903): `/clear` off-hand
  handling and vanilla inventory-slot order, including the added command-oracle
  cases. The independent branch matches **843/843 command/callback records**.
- [#1904](https://github.com/ludovic111/horizon/pull/1904): connected-client
  fireball motion and authoritative corrections, including the acceleration
  regression and a focused `remote_fireball` integration harness. The new
  harness fails on unchanged main in both hosting modes and passes with only
  that gameplay fix applied.
- [#1905](https://github.com/ludovic111/horizon/pull/1905): per-player sculk
  emission and history, including a focused `server_player_sculk` integration
  harness. Without the fix, both server-thread players and the in-frame guest
  fail while the in-frame host passes. With only this fix, **4/4 cases** pass,
  along with **435/435 vanilla scenarios matching on every tick**.

Each independent branch passes the quick source self-check using
`lab.game_env()` scratch homes (287 Python tests; 1241 Rust tests for clear
and sculk, 1242 for fireballs; formatting, Clippy and doc tests). Their broader
parity selections are listed for CI, rather than claimed as standalone full
parity passes.

The migration was merged with main at `8177a6f45` before removing the fixes.
Its test migration, fixture synchronization, isolated environments, control
inspection and font-oracle corrections remain. The **119/119 full LAN pass**
and full affected-parity readiness result above belong to the combined
pre-extraction snapshot `cc9410b6`; they do not establish that the migration
alone passes after the gameplay code has been removed. This PR stays draft
until the three dependencies land and its full integration suite is rerun.

The post-split migration quick source gate passes **1241 Rust tests and 290
Python tests**, formatting, Clippy, public-tree and doc checks. It lists **70
affected parity harnesses** for CI. Full post-split integration validation is
pending the three gameplay dependencies; no replacement full-suite pass is
claimed here.