Merge pull request #1805 from kacpherek/pr/pregen

🤖 Agent: thank you kacpherek! /pregen is in, off by default: a server turns it on with horizon-pregen=true in server.properties (server.toml [properties]), single player and LAN with --pregen. With it off, the command tree and /help match vanilla. Tested on a dedicated server through horizon-server (on: 1,369 chunks generated and saved), single player on the server thread and in-frame, and 11 harnesses including command_exec.
This commit is contained in:
Ludovic Marie
2026-10-26 01:54:20 +01:00
committed by GitHub
15 changed files with 778 additions and 14 deletions
@@ -592,3 +592,21 @@ fn mod_commands_join_the_tree_for_every_player() {
// The game's own tree is left as it was.
assert!(vanilla::game_dispatcher().find("where").is_none());
}
#[test]
fn horizon_commands_are_off_unless_the_server_turns_them_on() {
// By default the game runs vanilla's tree: /help lists no /pregen.
assert!(vanilla::game_dispatcher().find("pregen").is_none());
for dev in [false, true] {
let vanilla_tree = vanilla::game_tree(dev, false);
assert!(vanilla_tree.find("pregen").is_none());
let d = vanilla::game_tree(dev, true);
let parse = d.parse("pregen start 100 circle", &reg());
assert_eq!(parse.executable().unwrap().1, "pregen.start");
// Vanilla's commands are the same with or without it.
assert_eq!(
d.nodes[0].children.len(),
vanilla_tree.nodes[0].children.len() + 1
);
}
}
+62 -5
View File
@@ -538,12 +538,69 @@ fn apply_vanilla_order(d: &mut Dispatcher) {
}
/// The tree the game runs: a release integrated server's, or with
/// `HORIZON_DEV_COMMANDS` set the development one
/// (SharedConstants.DEBUG_DEV_COMMANDS).
/// (SharedConstants.DEBUG_DEV_COMMANDS), plus Horizon's own commands when
/// the server turned them on (`set_horizon_commands`).
pub fn game_dispatcher() -> &'static Dispatcher {
if std::env::var_os("HORIZON_DEV_COMMANDS").is_some() {
dispatcher()
} else {
release_dispatcher()
game_tree(
std::env::var_os("HORIZON_DEV_COMMANDS").is_some(),
HORIZON_COMMANDS.load(std::sync::atomic::Ordering::Relaxed),
)
}
/// Whether this process's server has Horizon's own commands (`/pregen`). Off
/// by default, so `/help` and the command tree are vanilla's; a dedicated
/// server turns them on with `horizon-pregen=true` in server.properties, the
/// game with `--pregen` (docs/pregeneration.md).
static HORIZON_COMMANDS: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
/// Turns Horizon's own commands on or off for the trees `game_dispatcher`
/// returns from now on.
pub fn set_horizon_commands(on: bool) {
HORIZON_COMMANDS.store(on, std::sync::atomic::Ordering::Relaxed);
}
/// The server.properties key that turns `/pregen` on (Horizon's own; vanilla
/// ignores keys it doesn't know).
pub const PREGEN_PROPERTY: &str = "horizon-pregen";
pub(crate) fn game_tree(dev: bool, horizon: bool) -> &'static Dispatcher {
static HORIZON: [std::sync::OnceLock<Dispatcher>; 2] =
[const { std::sync::OnceLock::new() }; 2];
match (dev, horizon) {
(true, false) => dispatcher(),
(false, false) => release_dispatcher(),
(dev, true) => HORIZON[usize::from(dev)].get_or_init(|| {
let mut d = build_with(dev);
horizon_commands(&mut d, 0);
d
}),
}
}
/// Commands vanilla doesn't have, after its own: `/pregen`
/// (docs/pregeneration.md).
fn horizon_commands(d: &mut Dispatcher, root: usize) {
let pregen = d.literal(root, "pregen");
d.requires(pregen, GAMEMASTERS);
let start = d.literal(pregen, "start");
let radius = d.argument(
start,
"radius",
ArgType::Integer {
min: 0,
max: 30_000_000,
},
);
d.executes(radius, "pregen.start");
for shape in ["square", "circle"] {
let shape = d.literal(radius, shape);
d.executes(shape, "pregen.start");
let center = d.argument(shape, "center", ArgType::ColumnPos);
d.executes(center, "pregen.start");
}
for (word, id) in [
("pause", "pregen.pause"),
("continue", "pregen.continue"),
("cancel", "pregen.cancel"),
("status", "pregen.status"),
] {
let w = d.literal(pregen, word);
d.executes(w, id);
}
}
pub fn dispatcher() -> &'static Dispatcher {
+4 -1
View File
@@ -361,7 +361,7 @@ impl Options {
match arg.as_str() {
"--help" | "-h" => {
println!(
"Horizon — Minecraft Java Edition {} in Rust\n\n --resources PATH Prepared resources (horizon-prepare)\n --render-distance N Chunks, 2..64 (default 12)\n --world NAME World folder under saves/ (default \"New World\")\n --seed N World seed for new worlds (default 0)\n --width N, --height N Window size (default 1280 x 720)\n --creative Creative mode (default survival)\n --test-terrain Simple test terrain instead of vanilla generation\n --frames N Exit after N frames (testing)\n --world-frames N Exit after N frames drawn in the world (testing)\n --smoke-report PATH Write a JSON report on exit\n --benchmark SCENARIO Run a benchmark (standard, village, forest, ocean, mountain), write its report to benchmarks/ and quit\n --control Accept horizon-ctl commands on a local socket\n --control-socket PATH Control socket path (default $XDG_RUNTIME_DIR/horizon-control.sock)\n --server HOST[:PORT] Join a multiplayer server (offline mode) at start\n --name NAME Player name for multiplayer (default: generated once, kept in horizon-player.txt)\n --session-file PATH Signed-in account from the launcher (read once, then deleted)\n --dev-mod PATH A Horizon mod (.hmod or folder) to develop: always on, reloaded when rebuilt\n --in-frame Run single player's world in the frame, not on its server thread\n\nControls: mouse look, WASD move, Space jump, Left Shift sneak, Left Ctrl sprint,\nleft click break, right click place/use, 1-9/scroll hotbar, E inventory,\nQ drop, F3 debug, Esc pause.",
"Horizon — Minecraft Java Edition {} in Rust\n\n --resources PATH Prepared resources (horizon-prepare)\n --render-distance N Chunks, 2..64 (default 12)\n --world NAME World folder under saves/ (default \"New World\")\n --seed N World seed for new worlds (default 0)\n --width N, --height N Window size (default 1280 x 720)\n --creative Creative mode (default survival)\n --test-terrain Simple test terrain instead of vanilla generation\n --frames N Exit after N frames (testing)\n --world-frames N Exit after N frames drawn in the world (testing)\n --smoke-report PATH Write a JSON report on exit\n --benchmark SCENARIO Run a benchmark (standard, village, forest, ocean, mountain), write its report to benchmarks/ and quit\n --control Accept horizon-ctl commands on a local socket\n --control-socket PATH Control socket path (default $XDG_RUNTIME_DIR/horizon-control.sock)\n --server HOST[:PORT] Join a multiplayer server (offline mode) at start\n --name NAME Player name for multiplayer (default: generated once, kept in horizon-player.txt)\n --session-file PATH Signed-in account from the launcher (read once, then deleted)\n --dev-mod PATH A Horizon mod (.hmod or folder) to develop: always on, reloaded when rebuilt\n --in-frame Run single player's world in the frame, not on its server thread\n --pregen Add Horizon's /pregen command (docs/pregeneration.md)\n\nControls: mouse look, WASD move, Space jump, Left Shift sneak, Left Ctrl sprint,\nleft click break, right click place/use, 1-9/scroll hotbar, E inventory,\nQ drop, F3 debug, Esc pause.",
horizon_core::VANILLA_VERSION
);
return Ok(None);
@@ -378,6 +378,8 @@ impl Options {
"--control" => o.control = Some(horizon_client::control::default_path()),
"--server-thread" => o.server_thread = true,
"--in-frame" | "--no-server-thread" => o.server_thread = false,
// Horizon's own commands (/pregen), off by default as on vanilla.
"--pregen" => horizon_client::command::vanilla::set_horizon_commands(true),
_ => {
let value = args
.next()
@@ -10694,6 +10696,7 @@ impl Game<'_> {
// The loading screen's chunk map fills ring by ring, as vanilla's.
self.streaming.loading =
self.options.execution_role.is_client() && self.loading_reason().is_some();
self.tick_pregen();
let (uploads, unloaded) =
horizon_client::spans::time("streaming", || self.streaming.update(&mut self.world));
if let Some(lod) = &mut self.lod {
@@ -10,6 +10,7 @@ mod items;
mod misc;
mod place;
mod players;
pub(super) mod pregen;
mod raid;
mod terrain_cmds;
mod text_commands;
@@ -758,6 +759,7 @@ impl Game<'_> {
"ride.mount" | "ride.dismount" => self.cmd_ride(id, ctx),
"spreadplayers" => self.cmd_spreadplayers(ctx),
id if id.starts_with("forceload.") => self.cmd_forceload(id, ctx),
id if id.starts_with("pregen.") => self.cmd_pregen(id, ctx),
"fillbiome" => self.cmd_fillbiome(ctx),
id if id.starts_with("worldborder.") => self.cmd_worldborder(id, ctx),
id if id.starts_with("loot.") => self.cmd_loot(id, ctx),
@@ -57,6 +57,8 @@ pub(in crate::game) struct CommandExtras {
/// The `CommandResultCallback` results of the outermost source, when a
/// harness records them.
pub results: Option<Vec<(bool, i32)>>,
/// The running `/pregen` task.
pub pregen: Option<super::pregen::PregenTask>,
}
use horizon_core::version::{BUILD_TIME, DATA_VERSION, STABLE, VANILLA_VERSION, VERSION_ID};
@@ -0,0 +1,276 @@
//! `/pregen`, Horizon's own command: generate, light and save every chunk of a
//! square or a circle ahead of the players (docs/pregeneration.md). The task
//! lives on the server and hands its band to the level's streaming, whose
//! workers do the work; chunks behind the band unload and are saved as any
//! chunk the players leave.
use super::{CommandError, Context, Dim, Game, R};
use horizon_client::{
lan::data::component_nbt,
pregen::{Pregen, Shape},
text::{self, Text},
};
use horizon_protocol::packets::play::{self, Clientbound as C};
use std::time::{Duration, Instant};
/// How often the progress is reported.
const REPORT_EVERY: Duration = Duration::from_secs(5);
/// How long the band may stay put before the task looks for what holds it.
const STALL: Duration = Duration::from_secs(30);
pub(in crate::game) struct PregenTask {
pregen: Pregen,
dim: Dim,
paused: bool,
/// Time spent running, up to the last pause, and when it last resumed.
ran: Duration,
resumed: Instant,
reported: Instant,
/// When the band last moved, to notice a task that stopped advancing.
moved: Instant,
/// The player who started it, told the progress too (a guest's connection).
owner: Option<u32>,
}
impl PregenTask {
fn running_for(&self) -> Duration {
if self.paused {
self.ran
} else {
self.ran + self.resumed.elapsed()
}
}
fn progress(&self) -> Text {
let p = &self.pregen;
let seconds = self.running_for().as_secs_f64();
let rate = p.done as f64 / seconds.max(1e-3);
let left = p.total - p.done;
let eta = if rate > 0.0 {
duration(left as f64 / rate)
} else {
"unknown".into()
};
Text::literal(format!(
"Pregenerating {}: {} of {} chunks ({:.1}%), {rate:.1} chunks/s, {} left",
self.dim.id(),
p.done,
p.total,
100.0 * p.done as f64 / p.total.max(1) as f64,
if p.done == p.total {
"nothing".into()
} else {
eta
},
))
}
}
fn failure(message: &str) -> CommandError {
CommandError::new(Text::literal(message))
}
/// `1h 02m 03s`.
fn duration(seconds: f64) -> String {
let s = seconds.round() as u64;
match (s / 3600, s / 60 % 60, s % 60) {
(0, 0, s) => format!("{s}s"),
(0, m, s) => format!("{m}m {s:02}s"),
(h, m, s) => format!("{h}h {m:02}m {s:02}s"),
}
}
impl Game<'_> {
pub(super) fn cmd_pregen(&mut self, id: &str, ctx: &Context) -> R<i32> {
match id {
"pregen.start" => {
if self.cmds.pregen.is_some() {
return Err(failure(
"Pregeneration is already running: /pregen cancel stops it",
));
}
let radius = Self::int(ctx, "radius").unwrap_or(0);
let shape = if ctx.literals.iter().any(|l| l == "circle") {
Shape::Circle
} else {
Shape::Square
};
let [x, z] = if ctx.get("center").is_some() {
self.column_arg(ctx, "center")?
} else {
let p = self.source_position();
[p[0].floor() as i32, p[2].floor() as i32]
};
let pregen = Pregen::new((x >> 4, z >> 4), radius >> 4, shape);
let now = Instant::now();
let total = pregen.total;
self.feedback(Text::literal(format!(
"Pregenerating {total} chunks of {}: a {} of radius {radius} around {x} {z}",
self.dim.id(),
match shape {
Shape::Square => "square",
Shape::Circle => "circle",
},
)));
self.cmds.pregen = Some(PregenTask {
pregen,
dim: self.dim,
paused: false,
ran: Duration::ZERO,
resumed: now,
reported: now,
moved: now,
owner: self.lan_acting,
});
Ok(total as i32)
}
"pregen.pause" | "pregen.continue" => {
let pause = id == "pregen.pause";
let task = self
.cmds
.pregen
.as_mut()
.ok_or_else(|| failure("No pregeneration is running"))?;
if task.paused != pause {
task.paused = pause;
if pause {
task.ran += task.resumed.elapsed();
} else {
task.resumed = Instant::now();
}
}
let (progress, dim) = (task.progress(), task.dim);
if pause && self.dim == dim {
self.streaming.set_band(None);
}
self.feedback(
Text::literal(if pause { "Paused. " } else { "Continuing. " }).append(progress),
);
Ok(1)
}
"pregen.cancel" => {
let task = self
.cmds
.pregen
.take()
.ok_or_else(|| failure("No pregeneration is running"))?;
if self.dim == task.dim {
self.streaming.set_band(None);
}
self.feedback(Text::literal("Cancelled. ").append(task.progress()));
Ok(task.pregen.done as i32)
}
_ => {
let task = self
.cmds
.pregen
.as_ref()
.ok_or_else(|| failure("No pregeneration is running"))?;
let progress = task.progress();
let done = task.pregen.done as i32;
self.feedback(progress);
Ok(done)
}
}
}
/// Move the task's band past the rows that are lit and hand it to the
/// level's streaming, and report the progress now and then. Runs before
/// each streaming update of the level in place.
pub(in crate::game) fn tick_pregen(&mut self) {
let Some(task) = self.cmds.pregen.as_mut() else {
return;
};
if task.dim != self.dim || task.paused {
return;
}
// A row is done once its chunks are lit and post-processed
// (`postprocess_chunks`, on the tick), as vanilla's are by the time
// they are FULL: then they may unload and be saved.
let (streaming, chunks) = (&mut self.streaming, &self.world.chunks);
if task.pregen.advance(|x, z| {
streaming.is_lit(x, z)
&& chunks
.get(&(x, z))
.is_some_and(|c| c.postprocess.is_empty())
}) {
task.moved = Instant::now();
}
streaming.set_band(task.pregen.band());
if task.moved.elapsed() >= STALL {
// Nothing advanced for a while: say what the first row waits
// for, and queue light again for chunks that lost their job.
task.moved = Instant::now();
let row = task.pregen.first_row();
let waiting = row
.iter()
.filter(|k| chunks.get(k).is_some_and(|c| !c.postprocess.is_empty()))
.count();
let [missing, lighting, unlit] = streaming.unstick_light(&mut self.world, &row);
log::warn!(
"Horizon: pregeneration waits on row {:?}: {missing} not loaded, {lighting} being lit, \
{unlit} unlit with no light job (queued again), {waiting} not post-processed; {:?}",
row.first(),
streaming.stages()
);
}
let finished = task.pregen.finished();
if finished || task.reported.elapsed() >= REPORT_EVERY {
task.reported = Instant::now();
let (owner, mut message) = (task.owner, task.progress());
if finished {
message = Text::literal("Done. ")
.append(message)
.with_color(text::GREEN);
self.cmds.pregen = None;
}
self.pregen_report(owner, message);
}
}
/// Whether the level in place has a running task, which the server pumps
/// between its ticks (`pump_pregen`).
pub(in crate::game) fn pregen_runs_here(&self) -> bool {
self.cmds
.pregen
.as_ref()
.is_some_and(|t| t.dim == self.dim && !t.paused)
}
/// The level a running task needs ticked, whoever is in it.
pub(in crate::game) fn pregen_level(&self) -> Option<Dim> {
self.cmds.pregen.as_ref().map(|t| t.dim)
}
/// Keep the workers fed until `until`: collect what they finished and
/// hand them the next jobs as soon as each one is done, rather than once
/// a tick.
pub(in crate::game) fn pump_pregen(&mut self, until: Instant) {
while self.pregen_runs_here() {
let now = Instant::now();
if now >= until {
break;
}
self.streaming
.wait((until - now).min(Duration::from_millis(5)));
self.update_world();
}
}
/// The console and the player who started the task get the progress.
fn pregen_report(&mut self, owner: Option<u32>, message: Text) {
if self.options.dedicated.is_some() {
super::super::dedicated::log("INFO", &message.plain());
} else {
log::info!("{}", message.plain());
}
if let Some(host) = &self.lan {
if let Some(g) = owner.and_then(|c| host.hub.guest(c)) {
g.send(&C::SystemChat(play::SystemChat {
content: component_nbt(&message.component()),
overlay: false,
}));
}
} else if self.options.dedicated.is_none() {
self.system_message(message);
}
}
}
@@ -30,7 +30,7 @@ fn ticks_to_seconds(ticks: i64) -> String {
impl Game<'_> {
// ---- /forceload ----------------------------------------------------------
fn column_arg(&self, ctx: &Context, name: &str) -> R<[i32; 2]> {
pub(super) fn column_arg(&self, ctx: &Context, name: &str) -> R<[i32; 2]> {
let c = self
.coords(ctx, name)
.ok_or_else(|| err("argument.pos2d.incomplete", &[]))?;
+11 -1
View File
@@ -155,6 +155,14 @@ pub(super) fn preflight(options: &mut Options) -> Result<bool> {
for w in &props.warnings {
log("WARN", w);
}
// Horizon's own commands are the owner's choice, off as on vanilla.
if props
.raw
.get(horizon_client::command::vanilla::PREGEN_PROPERTY)
.is_some_and(|v| v.trim().eq_ignore_ascii_case("true"))
{
horizon_client::command::vanilla::set_horizon_commands(true);
}
props.store(&settings)?;
let eula_file = PathBuf::from("eula.txt");
let agreed = match eula::agreed(&eula_file) {
@@ -667,7 +675,9 @@ impl Game<'_> {
self.poll_server_control(&mut state);
let players = self.lan.as_ref().map_or(0, |h| h.hub.guests.len());
if pause_after > 0 {
if players == 0 {
// A `/pregen` task keeps the server busy as a player would:
// paused, nothing would post-process its chunks.
if players == 0 && self.pregen_level().is_none() {
state.empty_ticks += 1;
} else {
state.empty_ticks = 0;
+10 -3
View File
@@ -365,7 +365,12 @@ impl Game<'_> {
}
*next += self.clock.tick_duration();
if *next > now {
std::thread::sleep(*next - now);
// A `/pregen` task keeps the workers fed until the next tick.
self.pump_pregen(*next);
let now = Instant::now();
if *next > now {
std::thread::sleep(*next - now);
}
} else if now - *next > Duration::from_secs(2) {
*next = now;
}
@@ -373,7 +378,7 @@ impl Game<'_> {
/// The levels this server tick runs, in `MinecraftServer.levels` order
/// (the Overworld, the Nether, the End): every level with a player in
/// it, or the Overworld while nobody is online.
/// it or a `/pregen` task, or the Overworld while nobody is online.
pub(super) fn levels_to_tick(&self) -> Vec<Dim> {
let guests = self
.lan
@@ -381,7 +386,9 @@ impl Game<'_> {
.map_or(&[][..], |h| h.hub.guests.as_slice());
let mut dims: Vec<Dim> = [Dim::Overworld, Dim::Nether, Dim::End]
.into_iter()
.filter(|d| guests.iter().any(|g| g.dimension == d.id()))
.filter(|d| {
guests.iter().any(|g| g.dimension == d.id()) || self.pregen_level() == Some(*d)
})
.collect();
if dims.is_empty() {
dims.push(Dim::Overworld);
+1
View File
@@ -146,6 +146,7 @@ pub mod player_sync;
pub mod poi_sections;
pub mod portal_sections;
pub mod pose;
pub mod pregen;
pub mod projectiles;
pub mod raid;
pub mod random_sequences;
+191
View File
@@ -0,0 +1,191 @@
//! Pregeneration (`/pregen`): every chunk of a square or a circle generated,
//! lit and saved ahead of the players. The area is cut into strips of columns
//! and each strip is swept north to south by a [`Band`] a few rows deep:
//! streaming generates the band's chunks and keeps them until they are lit,
//! then the band moves on and the rows behind it unload and are saved, so
//! memory stays bounded whatever the radius.
use crate::streaming::Band;
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Shape {
Square,
Circle,
}
/// The widest strip, in chunks. Strips are swept one after another; the few
/// columns along the edge between two are generated with both.
const STRIP_WIDTH: i32 = 256;
/// How many chunks a band holds at least: enough to keep every worker busy
/// while the rows before it finish.
const BAND_CHUNKS: i32 = 1024;
pub struct Pregen {
/// The centre chunk and the radius in chunks.
pub center: (i32, i32),
pub radius: i32,
pub shape: Shape,
/// The strips' columns, west to east.
strips: Vec<(i32, i32)>,
strip: usize,
/// The band's first row: every row of the strip before it is done.
row: i32,
/// Chunks of the area lit so far, and in all.
pub done: u64,
pub total: u64,
}
impl Pregen {
pub fn new(center: (i32, i32), radius: i32, shape: Shape) -> Self {
let radius = radius.max(0);
let (x0, width) = (center.0 - radius, 2 * radius + 1);
let count = (width + STRIP_WIDTH - 1) / STRIP_WIDTH;
// The columns `i * width / count` on, in i64: the product overflows
// for a radius of a few hundred thousand chunks.
let edge = |i: i32| x0 + (i64::from(width) * i64::from(i) / i64::from(count)) as i32;
let strips = (0..count).map(|i| (edge(i), edge(i + 1) - 1)).collect();
let mut pregen = Self {
center,
radius,
shape,
strips,
strip: 0,
row: 0,
done: 0,
total: 0,
};
pregen.total = (center.1 - radius..=center.1 + radius)
.map(|z| pregen.row_chunks(x0, x0 + width - 1, z))
.sum();
pregen.row = pregen.rows().map_or(0, |r| r.0);
pregen
}
/// How far from the centre column the area reaches at `d` chunks from
/// the centre row (or, the same, from the centre column), if it does.
fn half_width(&self, d: i32) -> Option<i32> {
let (r, d) = (i64::from(self.radius), i64::from(d));
match self.shape {
Shape::Square => (d.abs() <= r).then_some(self.radius),
Shape::Circle => (d * d <= r * r).then(|| (r * r - d * d).isqrt() as i32),
}
}
fn contains(&self, x: i32, z: i32) -> bool {
self.half_width(z - self.center.1)
.is_some_and(|h| (x - self.center.0).abs() <= h)
}
/// The area's chunks in columns `x0..=x1` of row `z`.
fn row_chunks(&self, x0: i32, x1: i32, z: i32) -> u64 {
self.half_width(z - self.center.1).map_or(0, |h| {
let (lo, hi) = (x0.max(self.center.0 - h), x1.min(self.center.0 + h));
(i64::from(hi) - i64::from(lo) + 1).max(0) as u64
})
}
/// The first and last rows of the current strip that hold chunks of the
/// area.
fn rows(&self) -> Option<(i32, i32)> {
let &(x0, x1) = self.strips.get(self.strip)?;
let nearest = (x0 - self.center.0).max(self.center.0 - x1).max(0);
let h = self.half_width(nearest)?;
Some((self.center.1 - h, self.center.1 + h))
}
/// The band streaming should hold now; none once every strip is done.
pub fn band(&self) -> Option<Band> {
let &(x0, x1) = self.strips.get(self.strip)?;
let (_, last) = self.rows()?;
let rows = (BAND_CHUNKS + (x1 - x0)) / (x1 - x0 + 1);
Some(Band {
x0,
x1,
z0: self.row,
z1: (self.row + rows - 1).min(last),
circle: (self.shape == Shape::Circle).then_some((self.center, self.radius)),
})
}
/// Move past the band's first rows once all their chunks are `lit`.
/// Returns whether the band moved.
pub fn advance(&mut self, lit: impl Fn(i32, i32) -> bool) -> bool {
let mut moved = false;
while let (Some(&(x0, x1)), Some((_, last))) = (self.strips.get(self.strip), self.rows()) {
let z = self.row;
if !(x0..=x1).all(|x| !self.contains(x, z) || lit(x, z)) {
break;
}
self.done += self.row_chunks(x0, x1, z);
self.row += 1;
if self.row > last {
self.strip += 1;
self.row = self.rows().map_or(self.row, |r| r.0);
}
moved = true;
}
moved
}
/// The chunks of the band's first row, which it waits for.
pub fn first_row(&self) -> Vec<(i32, i32)> {
let Some(&(x0, x1)) = self.strips.get(self.strip) else {
return Vec::new();
};
(x0..=x1)
.filter(|&x| self.contains(x, self.row))
.map(|x| (x, self.row))
.collect()
}
pub fn finished(&self) -> bool {
self.strip >= self.strips.len()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn sweeps_every_chunk_of_the_area_once() {
for shape in [Shape::Square, Shape::Circle] {
for (center, radius) in [((0, 0), 0), ((3, -7), 5), ((-40, 12), 130), ((0, 0), 300)] {
let mut p = Pregen::new(center, radius, shape);
let r = i64::from(radius);
let area = (-radius..=radius)
.flat_map(|dz| (-radius..=radius).map(move |dx| (dx, dz)))
.filter(|&(dx, dz)| {
shape == Shape::Square
|| i64::from(dx).pow(2) + i64::from(dz).pow(2) <= r * r
})
.count() as u64;
assert_eq!(p.total, area, "{shape:?} {radius}");
let mut seen = std::collections::HashSet::new();
while !p.finished() {
let band = p.band().unwrap();
assert!(band.z0 <= band.z1);
let row = p.first_row();
for &(x, z) in &row {
assert!(seen.insert((x, z)), "{x} {z} twice");
}
assert!(
p.advance(|x, z| z == band.z0 && row.contains(&(x, z))) || row.is_empty()
);
}
assert_eq!(
(p.done, seen.len() as u64),
(area, area),
"{shape:?} {radius}"
);
}
}
}
#[test]
fn a_huge_area_is_planned_at_once() {
let started = std::time::Instant::now();
let p = Pregen::new((0, 0), 30_000_000 >> 4, Shape::Circle);
assert!(p.total > 11_000_000_000_000);
assert!(started.elapsed() < std::time::Duration::from_secs(5));
}
}
+149 -3
View File
@@ -278,6 +278,11 @@ pub struct Streaming {
cover: Option<ExtraCover>,
/// Mesh the sections for drawing (false on a server thread).
pub mesh: bool,
/// The pregeneration band (`set_band`).
band: Option<Band>,
/// The columns and the next row the band's decoration ranks were given
/// for (`set_band`).
band_ranked: Option<(i32, i32, i32)>,
}
/// Snapshot only after a scan found no missing candidates. Public range fields
/// and `World::chunks` can change without going through Streaming, so compare
@@ -289,6 +294,7 @@ struct GenerationScan {
radius: i32,
extra: Vec<((i32, i32), i32)>,
forced: HashSet<(i32, i32)>,
band: Option<Band>,
chunk_revision: u64,
}
impl GenerationScan {
@@ -298,6 +304,7 @@ impl GenerationScan {
&& self.radius == ranges.radius
&& self.extra == ranges.extra
&& self.forced == *ranges.forced
&& self.band.as_ref() == ranges.band
&& self.chunk_revision == world.chunks.membership_revision()
}
fn remember(&mut self, ranges: Ranges<'_>, world: &World) {
@@ -307,6 +314,7 @@ impl GenerationScan {
self.extra.clear();
self.extra.extend_from_slice(ranges.extra);
self.forced.clone_from(ranges.forced);
self.band = ranges.band.copied();
self.chunk_revision = world.chunks.membership_revision();
}
}
@@ -348,6 +356,10 @@ impl GenerationScan {
let r = radius + margin.max(reach + 1);
runs.extend((center.1 - r..=center.1 + r).map(|z| (z, center.0 - r, center.0 + r)));
}
if let Some(band) = input.ranges.band {
let r = margin.max(reach + 1);
runs.extend((band.z0 - r..=band.z1 + r).map(|z| (z, band.x0 - r, band.x1 + r)));
}
runs.sort_unstable();
let mut runs = runs.into_iter().peekable();
while let Some((z, x0, mut x1)) = runs.next() {
@@ -392,6 +404,7 @@ impl GenerationScan {
extra: input.ranges.extra,
forced: input.ranges.forced,
cover: input.ranges.cover,
band: input.ranges.band,
};
self.remember(ranges, input.world);
} else {
@@ -564,6 +577,8 @@ impl Streaming {
extra: Vec::new(),
cover: None,
mesh: true,
band: None,
band_ranked: None,
}
}
/// Shares generation resources, but gives distant work its own worker state.
@@ -916,6 +931,7 @@ impl Streaming {
p.is_some()
&& i64::from(x - center.0).pow(2) + i64::from(z - center.1).pow(2) > r4 * r4
&& !ranges.in_extra(x, z, 4)
&& !ranges.in_band(x, z, 4)
&& !ranges
.forced
.iter()
@@ -985,6 +1001,7 @@ impl Streaming {
extra: &self.extra,
forced: &self.forced,
cover: ExtraCover::matching(&self.cover, &self.extra),
band: self.band.as_ref(),
};
let (protos, generating) = (&self.protos, &self.generating);
self.decoration_order.retain(|x, z| {
@@ -1113,6 +1130,7 @@ impl Streaming {
extra: &self.extra,
forced: &self.forced,
cover: ExtraCover::matching(&self.cover, &self.extra),
band: self.band.as_ref(),
};
self.waiting.retain(|&(x, sy, z)| {
let back = ranges.in_view(x, z);
@@ -1189,6 +1207,7 @@ impl Streaming {
extra: &self.extra,
forced: &self.forced,
cover: ExtraCover::matching(&self.cover, &self.extra),
band: self.band.as_ref(),
},
world,
generating: &self.generating,
@@ -1285,6 +1304,7 @@ impl Streaming {
extra: &self.extra,
forced: &self.forced,
cover: ExtraCover::matching(&self.cover, &self.extra),
band: self.band.as_ref(),
}
}
fn near_finished(&self, x: i32, z: i32, reach: i32) -> bool {
@@ -1354,6 +1374,66 @@ impl Streaming {
fn in_range(&self, x: i32, z: i32, margin: i32) -> bool {
self.ranges().in_range(x, z, margin)
}
pub fn band(&self) -> Option<Band> {
self.band
}
/// Generate, light and keep the chunks of `band` (none: stop), and let
/// go of what earlier bands held as it falls out of range. The chunks a
/// band reaches are ranked for decoration row by row, as they first come
/// within reach, so that a task decorates in the same order however its
/// work is timed and shared among the workers.
pub fn set_band(&mut self, band: Option<Band>) {
if self.band == band {
return;
}
self.band = band;
self.generation_scan.exhausted = false;
let Some(band) = band else {
return;
};
// Decoration reaches two chunks past the band and generation three.
const REACH: i32 = 4;
let (x0, x1) = (band.x0 - REACH, band.x1 + REACH);
let first = match self.band_ranked {
Some((rx0, rx1, next)) if (rx0, rx1) == (x0, x1) => next,
_ => band.z0 - REACH,
};
let end = band.z1 + REACH;
for z in first..=end {
for x in x0..=x1 {
self.decoration_order.rank_or_append(x, z);
}
}
self.band_ranked = Some((x0, x1, end.max(first - 1) + 1));
}
/// What holds up the columns `keys`, for a task that stopped advancing:
/// how many are not loaded, loaded but unlit with a light job in flight,
/// and unlit with none (those whose neighbours are all loaded are queued
/// for light again).
pub fn unstick_light(&mut self, world: &mut World, keys: &[(i32, i32)]) -> [usize; 3] {
let mut counts = [0; 3];
for &key in keys {
if !world.chunks.contains_key(&key) {
counts[0] += 1;
} else if self.lit.contains(&key) {
} else if self.lighting.contains_key(&key) {
counts[1] += 1;
} else {
counts[2] += 1;
if Self::region(world, key.0, key.1).is_some() {
world.dirty_light.insert(key);
}
}
}
counts
}
/// Wait up to `timeout` for a worker to finish a job; the next `update`
/// takes it. A server pumps generation with this between its ticks.
pub fn wait(&mut self, timeout: std::time::Duration) {
if let Ok(done) = self.results.recv_timeout(timeout) {
self.deferred.push(done);
}
}
}
/// `ChunkTrackingView.isInViewDistance`: whether the column (x, z) is drawn
/// at this render distance around `center`, the camera's column
@@ -1373,6 +1453,53 @@ struct Ranges<'a> {
/// `extra` as a grid, when one was made for it: `in_extra` without a
/// scan of every area.
cover: Option<&'a ExtraCover>,
band: Option<&'a Band>,
}
/// The chunks a pregeneration task (`/pregen`) wants lit now: rows `z0..=z1`
/// of the columns `x0..=x1`, those within `circle` (a centre and a radius in
/// chunks) when there is one. Like an extra area, it finishes its chunks and
/// the ring around them, so that its own chunks can be lit, and generation
/// reaches past that. A task moves it down a strip of columns a row at a
/// time, so each chunk is generated once and only the rows in flight stay
/// loaded.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Band {
pub x0: i32,
pub x1: i32,
pub z0: i32,
pub z1: i32,
pub circle: Option<((i32, i32), i32)>,
}
impl Band {
/// Whether (x, z) lies within `margin` (Chebyshev) of one of the band's
/// chunks: the square of side `2 * margin + 1` around it meets the
/// rectangle at a point inside the circle.
pub fn within(&self, x: i32, z: i32, margin: i32) -> bool {
let (x0, x1) = (
x.saturating_sub(margin).max(self.x0),
x.saturating_add(margin).min(self.x1),
);
let (z0, z1) = (
z.saturating_sub(margin).max(self.z0),
z.saturating_add(margin).min(self.z1),
);
if x0 > x1 || z0 > z1 {
return false;
}
self.circle.is_none_or(|((cx, cz), r)| {
let dx = i64::from(cx.clamp(x0, x1)) - i64::from(cx);
let dz = i64::from(cz.clamp(z0, z1)) - i64::from(cz);
dx * dx + dz * dz <= i64::from(r) * i64::from(r)
})
}
/// The band's chunks go after the players' (lower runs first), row by
/// row, as the task wants them lit.
fn priority(&self, x: i32, z: i32) -> i64 {
const AFTER_PLAYERS: i64 = 1 << 40;
AFTER_PLAYERS
+ (i64::from(z - self.z0 + 8).max(0) << 20)
+ i64::from(x - self.x0 + 8).max(0)
}
}
/// For every column near the extra areas, how far it lies outside the
/// nearest one's square (Chebyshev distance less its radius; 0 inside). A server has an area per player, and streaming asks of every
@@ -1468,10 +1595,14 @@ impl ExtraCover {
impl Ranges<'_> {
/// `distance`, or `rings` while the level loading screen is up.
fn priority(&self, x: i32, z: i32, loading: bool) -> i64 {
if loading {
let own = if loading {
self.rings(x, z)
} else {
self.distance(x, z)
};
match self.band {
Some(band) if band.within(x, z, 4) => own.min(band.priority(x, z)),
_ => own,
}
}
/// Ring by ring: the Chebyshev distance to the nearest centre first, then
@@ -1532,10 +1663,18 @@ impl Ranges<'_> {
let (dx, dz) = (gap(x - self.center.0), gap(z - self.center.1));
let r = i64::from(self.radius + 1);
// The extra areas are squares: their finished square grown by `reach`.
dx * dx + dz * dz <= r * r || self.in_extra(x, z, 1 + reach)
dx * dx + dz * dz <= r * r
|| self.in_extra(x, z, 1 + reach)
|| self.in_band(x, z, 1 + reach)
}
fn in_range(&self, x: i32, z: i32, margin: i32) -> bool {
self.in_main_range(x, z, margin) || self.in_extra(x, z, margin)
self.in_main_range(x, z, margin)
|| self.in_extra(x, z, margin)
|| self.in_band(x, z, margin)
}
/// Within `margin` of the pregeneration band.
fn in_band(&self, x: i32, z: i32, margin: i32) -> bool {
self.band.is_some_and(|b| b.within(x, z, margin))
}
/// Within `margin` of an extra area's square (Chebyshev distance, as
/// vanilla's ticket levels spread).
@@ -1659,6 +1798,7 @@ mod tests {
extra: &self.extra,
forced: &self.forced,
cover: None,
band: None,
},
world: &self.world,
generating: &self.generating,
@@ -1972,6 +2112,7 @@ mod tests {
extra: &[],
forced: &forced,
cover: None,
band: None,
};
let square = |x: i32, z: i32, n: i32| {
(-n..=n).flat_map(move |dz| (-n..=n).map(move |dx| (x + dx, z + dz)))
@@ -2027,6 +2168,7 @@ mod tests {
extra: &[],
forced: &forced,
cover: None,
band: None,
};
let at = |dx: i32, dz: i32| (2 + dx, -1 + dz);
let key = |(x, z): (i32, i32), loading| r.priority(x, z, loading);
@@ -2053,6 +2195,7 @@ mod tests {
extra: &[((radius * 2 + 9, 3), 4)],
forced: &forced,
cover: None,
band: None,
};
let span = radius * 3 + 20;
for x in -span..=span {
@@ -2093,6 +2236,7 @@ mod tests {
extra: &extra,
forced: &forced,
cover: None,
band: None,
};
let level = |x: i32, z: i32| 31 + ((x - 40).abs().max((z + 9).abs()) - view).max(0);
for x in 40 - view - 8..=40 + view + 8 {
@@ -2166,6 +2310,7 @@ mod cover_tests {
extra: &extra,
forced: &forced,
cover,
band: None,
};
let (scan, grid) = (ranges(None), ranges(Some(&cover)));
let span = [4, 30, 120][case % 3] + 40;
@@ -2224,6 +2369,7 @@ mod cover_tests {
extra: &extra,
forced: &forced,
cover,
band: None,
};
// The columns streaming asks about: everything kept, and the
// generation scan's ring past it.
+1
View File
@@ -35,6 +35,7 @@ To play, use [the launcher](../launcher/README.md); the project site is [playhor
## Architecture: world generation
- [World generation](world-generation.md): the pipeline from noise to features.
- [Pregeneration](pregeneration.md): `/pregen`, generating, lighting and saving an area ahead of the players.
- [Noise generation](noise-generation.md), [compiled density functions](density-functions.md), [structure density](structure-density.md), [terrain filling](terrain-filling.md).
- Biomes: [lookup](biome-lookup.md), [temperatures](biome-temperatures.md).
- Surfaces: [material surfaces](material-surfaces.md), [material conditions](material-conditions.md), [rule libraries](material-rule-libraries.md), [vertical anchors](vertical-anchors.md), [value providers](value-providers.md).
+3
View File
@@ -218,6 +218,9 @@ horizon-server rollback back to the release before
While the server runs, `op`, `whitelist` and `cmd` go to the game's console, so they act exactly as vanilla's
console commands do. While it's stopped, they edit vanilla's files directly. Any console command works through
`cmd`: `horizon-server cmd ban Griefer`, `horizon-server cmd say Restarting in 5 minutes`.
To generate the world around spawn ahead of the players, Horizon has its own `pregen` command, off until
`server.toml` has `horizon-pregen = true` under `[properties]`: `horizon-server cmd pregen start 2500`
([pregeneration](pregeneration.md)).
`config set` changes `server.toml` without opening it: `horizon-server config set world.difficulty=hard
server.max_players=10`. Each key is a section and a name from the file (`properties.spawn-protection=0` for any
+47
View File
@@ -0,0 +1,47 @@
# Pregeneration: `/pregen`
`/pregen` generates every chunk of a square or a circle ahead of the players, lights it and saves it, as
Chunky does on a Java server. It is Horizon's own command (vanilla has none), available in single player
with cheats on and to operators (level 2) on a server, from the chat or the server console
(`horizon-server cmd pregen start 2500`).
It is off until you turn it on, so `/help` and the command tree stay vanilla's:
- a server: `horizon-pregen = true` under `[properties]` in `server.toml` (or `horizon-pregen=true` in a
vanilla folder's `server.properties`), then restart it;
- single player and LAN: start the game with `--pregen`.
```text
/pregen start <radius> [square|circle] [<x> <z>] radius in blocks; the centre defaults to where you
stand (the world spawn from the console)
/pregen status chunks done, chunks per second, time left
/pregen pause | continue stop scheduling new chunks, and go on
/pregen cancel stop; what is done stays saved
```
The area is the chunks within `radius / 16` chunks of the centre chunk: a square of side
`2 * (radius / 16) + 1` (radius 2500 is 313 x 313 = 97,969 chunks), or the chunks whose distance from the
centre chunk is at most `radius / 16`. It runs in the dimension the command was run in; that level keeps
ticking while the task runs, whoever is in it. The progress goes to the server log every five seconds
and to the player who started it, and once more when it is done. A task is not kept across a restart:
start it again and the chunks already saved load instead of generating.
## How it works
- `horizon_client::pregen::Pregen` cuts the area into strips at most 256 chunks wide and sweeps each
north to south with a band of rows (`streaming::Band`) holding about a thousand chunks.
- Streaming treats the band like a player's area: it generates its chunks and the rings around them that
decoration and light need, and lights its own chunks. Once every chunk of the band's first row is lit
and post-processed (`LevelChunk.postProcessGeneration`, which vanilla runs before a chunk is FULL), the
band moves down a row; the rows that fall out of its reach unload and are saved like any chunk the
players leave, so memory holds only the rows in flight.
- The chunks a band reaches are ranked for decoration row by row as they come within its reach
(`Streaming::set_band`). Decoration is order dependent where neighbours' writes overlap
(`DecorationOrder`), and this makes a task decorate in the same order however its jobs are timed and
shared among the workers: the same seed gives the same chunks.
- Between server ticks the server waits for the workers' results and hands out the next jobs at once
(`Streaming::wait`, `pump_pregen`), instead of once a tick: a chunk passes through five jobs
(generate, decorate, finish, light, save), each needing its neighbours' earlier ones.
The chunks of the band's outer ring are finished (FULL) but not lit, as at the edge of what a player has
loaded; they light when their neighbours load later.