From 94b5d2dbc4dc448335cec1fc43c3e7bd30959745 Mon Sep 17 00:00:00 2001 From: Ludovic Marie <136255825+ludovic111@users.noreply.github.com> Date: Thu, 15 Oct 2026 18:26:10 +0200 Subject: [PATCH 1/2] tools/ci: the daily version-port agent (detect, port, harnesses, report, publish dry run), not installed Co-Authored-By: Claude Opus 5.5 --- docs/version-ports.md | 49 ++++++++++++++-- tools/ci/dev.horizon.version-port.plist | 21 +++++++ tools/ci/version-port-daily.sh | 78 +++++++++++++++++++++++++ 3 files changed, 144 insertions(+), 4 deletions(-) create mode 100644 tools/ci/dev.horizon.version-port.plist create mode 100755 tools/ci/version-port-daily.sh diff --git a/docs/version-ports.md b/docs/version-ports.md index 5271128f2..a303c401b 100644 --- a/docs/version-ports.md +++ b/docs/version-ports.md @@ -97,9 +97,11 @@ latest release (and, with `--snapshots`, the latest snapshot) with the pin and w `versions/detect-state.json`, and prints what is new; exit status 0 means "nothing new", 10 "new release", 11 "new snapshot". It needs only the network, so it can run anywhere: -- **Mac mini, launchd (chosen):** a `~/fleet` launchd agent runs `version_port.py detect --then port` daily at - 14:00 UTC (Mojang publishes on weekdays around 10:00-14:00 UTC), in a dedicated worktree of `origin/main`, - at `~/fleet/load prio low`. The whole port runs where the vanilla data, Java and the harnesses are. +- **Mac mini, launchd (chosen):** the agent `dev.horizon.version-port` (`tools/ci/dev.horizon.version-port.plist`) + runs `tools/ci/version-port-daily.sh` daily at 16:00 Zurich time (14:00 UTC in summer, 15:00 in winter; Mojang + publishes on weekdays around 10:00-14:00 UTC), in a dedicated worktree of `origin/main` + (`~/code/horizon-parity/port-main`), at `~/fleet/load prio low`, nice 10. The whole port runs where the vanilla + data, Java and the harnesses are. See "Turning on the daily agent" below. - **GitHub Actions:** a scheduled job could run `detect` alone and open a "Minecraft X is out" issue; the Mac mini would still have to do the rest, so it adds a hop for nothing. Kept as the fallback if the Mac mini is down for long. @@ -281,6 +283,45 @@ title. Its `research-lock.json` makes `horizon-prepare` ask for NEW, so players official launcher; the launcher already reads each release's game version from its `research-lock.json`, so players who haven't can keep the previous Horizon release. Supporting two Minecraft versions in one build is out of scope. +## Turning on the daily agent + +Not installed: it runs Mojang downloads, Cargo, Java and every harness on the Mac mini unattended, and its +`publish` dry run prepares issues under a new `version-port` label, so both need Ludovic's approval. What it does +on each run (`tools/ci/version-port-daily.sh`): + +1. update its worktree of `origin/main` (`~/code/horizon-parity/port-main`, made on the first run, `.local/` linking + the shared data as a parity worktree does); +2. `version_port.py detect --dry-run` (`--snapshots` with `HORIZON_PORT_SNAPSHOTS=1`): nothing new ends the run; +3. for a new release: `port NEW` (prepare both, diff, regen into `ports/NEW` on the local branch `port/NEW`, report), + `tools/parity.py` on main as the baseline (only what never passed or changed reruns), `harnesses NEW` against + that baseline, `report OLD NEW`, and `publish OLD NEW --check-github` (the dry run: the log lists the issues it + would file). A snapshot is only prepared, diffed and reported; +4. `detect` again, so the version is remembered only once its port went through (a failed run retries the next day), + and a note on Kira card 28 (`HORIZON_PORT_KIRA_CARD`) saying the report is ready. + +It never runs `publish --yes`, never pushes and never changes `shared-local/`. `HORIZON_PORT_VERSION=V` ports V at +once (by hand, or to replay a version); `HORIZON_PORT_REF` checks out another ref than `origin/main`. + +To turn it on, on the Mac mini (as Ludovic, once approved): + +```sh +# 1. The label publish files issues under (once): +gh label create version-port --repo ludovic111/horizon --color 5319e7 \ + --description "Work to port Horizon to a new Minecraft release (tools/version_port.py publish)" +# 2. The script, as a copy from main (it updates its own worktree, so it never runs from a checkout): +curl -fsSL https://raw.githubusercontent.com/ludovic111/horizon/main/tools/ci/version-port-daily.sh \ + -o ~/code/horizon-parity/version-port-daily.sh +# 3. The agent: +curl -fsSL https://raw.githubusercontent.com/ludovic111/horizon/main/tools/ci/dev.horizon.version-port.plist \ + | sed "s|@HOME@|$HOME|g" > ~/Library/LaunchAgents/dev.horizon.version-port.plist +launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/dev.horizon.version-port.plist +# A run now, to see it work (log: ~/Library/Logs/horizon-version-port.log): +launchctl kickstart gui/$(id -u)/dev.horizon.version-port +``` + +To turn it off: `launchctl bootout gui/$(id -u)/dev.horizon.version-port` and delete the plist. When the report is +in, publishing is still by hand: `python3 tools/version_port.py publish OLD NEW` (dry run), then `--yes`. + ## What stays manual, and how small it gets Automatic: detecting, downloading, verifying, the data generator, every diff, the class-to-module map, regenerating @@ -390,4 +431,4 @@ python3 tools/vanilla_map.py versions/26.3/classes.json net.minecraft.world.food | `regen` (`tools/port_regen.py`): candidate worktree, its `.local/`, the generators' classes decompiled alone, every generator, `regen.json` in the report | done | | Candidate harness run (`tools/port_harness.py`, `harnesses`): horizon-prepare with sounds, every harness, pass / oracle / fail, newly failing against a baseline, in the report | done | | `publish` (`tools/port_publish.py`): dry run by default, issues, tracking issue, Kira card | done (never run with `--yes`) | -| launchd agent | later | +| Daily launchd agent (`tools/ci/version-port-daily.sh`, `tools/ci/dev.horizon.version-port.plist`) | written, not installed: needs Ludovic's approval | diff --git a/tools/ci/dev.horizon.version-port.plist b/tools/ci/dev.horizon.version-port.plist new file mode 100644 index 000000000..333749aaa --- /dev/null +++ b/tools/ci/dev.horizon.version-port.plist @@ -0,0 +1,21 @@ + + + + + Labeldev.horizon.version-port + ProgramArguments + /bin/bash@HOME@/code/horizon-parity/version-port-daily.sh + + StartCalendarIntervalHour16Minute0 + LowPriorityIO + Nice10 + StandardOutPath@HOME@/Library/Logs/horizon-version-port.log + StandardErrorPath@HOME@/Library/Logs/horizon-version-port.log + EnvironmentVariables + HOME@HOME@ + PATH@HOME@/.local/bin:@HOME@/.cargo/bin:/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin + HORIZON_PORT_KIRA_CARD28 + + diff --git a/tools/ci/version-port-daily.sh b/tools/ci/version-port-daily.sh new file mode 100755 index 000000000..d669c78ec --- /dev/null +++ b/tools/ci/version-port-daily.sh @@ -0,0 +1,78 @@ +#!/bin/bash +# Daily on the Mac mini, from launchd (tools/ci/dev.horizon.version-port.plist; docs/version-ports.md, "Detect"): +# when Mojang releases a Minecraft version Horizon doesn't pin, port it as far as the report: prepare, diff, +# regen (ports/NEW on the local branch port/NEW), every harness against it, the report, and publish's dry run. +# It never publishes (version_port.py publish --yes stays a person's call) and never pushes. +# +# It runs from a copy taken from main, never from a checkout (it updates its own worktree of origin/main): +# curl -fsSL https://raw.githubusercontent.com/ludovic111/horizon/main/tools/ci/version-port-daily.sh \ +# -o ~/code/horizon-parity/version-port-daily.sh +# +# HORIZON_REPO the clone to make the worktree from (default ~/code/horizon) +# HORIZON_PORT_MAIN the worktree of origin/main it works in (default ~/code/horizon-parity/port-main) +# HORIZON_PORT_REF what that worktree checks out (default origin/main) +# HORIZON_PORT_VERSION=V port release V now, without asking Mojang what is new (by hand, or to replay one) +# HORIZON_PORT_SNAPSHOTS=1 snapshots too: prepared, diffed and reported only (no regen, no harnesses) +# HORIZON_PORT_KIRA_CARD=N post a note on Kira card N when a port report is ready +set -euo pipefail +repo=${HORIZON_REPO:-$HOME/code/horizon} +root=${HORIZON_PORT_MAIN:-$HOME/code/horizon-parity/port-main} +ref=${HORIZON_PORT_REF:-origin/main} +shared=$HOME/code/horizon-parity/shared-local +log() { echo "$(date -u +%FT%TZ) $*"; } + +if [ ! -e "$root/.git" ]; then + git -C "$repo" fetch -q origin + git -C "$repo" worktree add -q --detach "$root" "$ref" +fi +cd "$root" +git fetch -q origin +git checkout -q --detach "$ref" +if [ ! -d .local ]; then + mkdir .local + for d in minecraft horizon-client tools datagen; do ln -s "$shared/$d" ".local/$d"; done +fi +# shellcheck disable=SC1091 +source "$HOME/code/horizon-parity/env.sh" +"$HOME/fleet/load" prio low --for 12h >/dev/null 2>&1 || true + +# --dry-run: what is new is remembered (versions/detect-state.json) only once its port went through, so a port +# that fails is tried again the next day. +if [ -n "${HORIZON_PORT_VERSION:-}" ]; then + kind=release new=$HORIZON_PORT_VERSION +else + set +e + python3 tools/version_port.py detect --dry-run ${HORIZON_PORT_SNAPSHOTS:+--snapshots} > "$root/.local/detect.txt" + found=$? + set -e + cat "$root/.local/detect.txt" + case $found in + 0) exit 0 ;; + 10) kind=release ;; + 11) kind=snapshot ;; + *) log "detect failed ($found)"; exit "$found" ;; + esac + new=$(sed -n "s/^New $kind: \([^ ]*\) .*/\1/p" "$root/.local/detect.txt" | head -1) +fi +old=$(python3 -c 'import json; print(json.load(open("research-lock.json"))["minecraft"]["version"])') +log "Minecraft $new ($kind) is out; Horizon pins $old" + +if [ "$kind" = snapshot ]; then + python3 tools/version_port.py port "$new" --no-regen +else + python3 tools/version_port.py port "$new" # prepare both, diff, regen, report + # The baseline: every harness on main against the pinned version (what passed before is not rerun). + python3 tools/parity.py > "$root/.local/baseline.log" 2>&1 || true + python3 tools/version_port.py harnesses "$new" --old "$old" --baseline .local/parity-runner/state.json + python3 tools/version_port.py report "$old" "$new" --lines 30 + python3 tools/version_port.py publish "$old" "$new" --check-github --show 0 +fi +[ -n "${HORIZON_PORT_VERSION:-}" ] || python3 tools/version_port.py detect ${HORIZON_PORT_SNAPSHOTS:+--snapshots} > /dev/null || true +report=${HORIZON_VERSIONS:-$HOME/code/horizon-parity/versions}/diff-$old-$new/report.md +log "Port report: $report" +if [ -n "${HORIZON_PORT_KIRA_CARD:-}" ]; then + text="Minecraft $new ($kind) is out. Port report ready on the Mac mini ($report); publish's dry run is in the log." + node "$HOME/code/horizon-parity/tools/kira.mjs" card_note \ + "$(python3 -c 'import json,sys; print(json.dumps({"ref": "card " + sys.argv[1], "text": sys.argv[2]}))' \ + "$HORIZON_PORT_KIRA_CARD" "$text")" > /dev/null || true +fi From d8eafd7dc0b5c6befcc58b02e9819c7a08b2cd43 Mon Sep 17 00:00:00 2001 From: Ludovic Marie <136255825+ludovic111@users.noreply.github.com> Date: Thu, 15 Oct 2026 20:36:14 +0200 Subject: [PATCH 2/2] version-port-daily.sh: no fleet-load priority change (it lowered the calling session); launchd's Nice and LowPriorityIO do it Co-Authored-By: Claude Opus 5.5 --- docs/version-ports.md | 3 ++- tools/ci/version-port-daily.sh | 1 - 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/version-ports.md b/docs/version-ports.md index a303c401b..623e3c082 100644 --- a/docs/version-ports.md +++ b/docs/version-ports.md @@ -100,7 +100,8 @@ latest release (and, with `--snapshots`, the latest snapshot) with the pin and w - **Mac mini, launchd (chosen):** the agent `dev.horizon.version-port` (`tools/ci/dev.horizon.version-port.plist`) runs `tools/ci/version-port-daily.sh` daily at 16:00 Zurich time (14:00 UTC in summer, 15:00 in winter; Mojang publishes on weekdays around 10:00-14:00 UTC), in a dedicated worktree of `origin/main` - (`~/code/horizon-parity/port-main`), at `~/fleet/load prio low`, nice 10. The whole port runs where the vanilla + (`~/code/horizon-parity/port-main`), at nice 10 with low-priority I/O (the plist's `Nice` and + `LowPriorityIO`; `~/fleet/load prio low` would lower the calling agent session instead). The whole port runs where the vanilla data, Java and the harnesses are. See "Turning on the daily agent" below. - **GitHub Actions:** a scheduled job could run `detect` alone and open a "Minecraft X is out" issue; the Mac mini would still have to do the rest, so it adds a hop for nothing. Kept as the fallback if the Mac mini is down for diff --git a/tools/ci/version-port-daily.sh b/tools/ci/version-port-daily.sh index d669c78ec..18a094d95 100755 --- a/tools/ci/version-port-daily.sh +++ b/tools/ci/version-port-daily.sh @@ -34,7 +34,6 @@ if [ ! -d .local ]; then fi # shellcheck disable=SC1091 source "$HOME/code/horizon-parity/env.sh" -"$HOME/fleet/load" prio low --for 12h >/dev/null 2>&1 || true # --dry-run: what is new is remembered (versions/detect-state.json) only once its port went through, so a port # that fails is tried again the next day.