Horizon shader packs: horizon-shaders.toml (schema 1) and the .hshader package
A new crate, horizon-shader-api, reads and checks Horizon's own WGSL shader packs (docs/api/shaders.md), so the game, the launcher and the registry judge a pack the same way. This first part is the format: the manifest (id, name, version, the api line, shared files, the sky, material and post stages, options typed for WGSL override constants, a GPU time budget) and the package (a folder or a .hshader zip, built reproducibly so its SHA-256 depends only on its files). The stages' WGSL is checked in the next parts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -45,6 +45,7 @@ Game workspace (root `Cargo.toml`); `launcher/` and `crates/horizon-telemetry-se
|
||||
| `horizon-friends`, `horizon-relay` | Play with Friends: invite codes, encrypted tunnel, the relay ([docs/play-with-friends.md](docs/play-with-friends.md)) |
|
||||
| `horizon-mods`, `horizon-mod-sdk`, `cargo-horizon-mod` | Sandboxed WebAssembly mods: runtime, SDK + WIT, `cargo horizon-mod` ([docs/mods.md](docs/mods.md)) |
|
||||
| `horizon-port-pack` | Porting Java modpacks ([docs/modpacks.md](docs/modpacks.md)) |
|
||||
| `horizon-shader-api` | Horizon's own WGSL shader packs ([docs/api/shaders.md](docs/api/shaders.md)): `horizon-shaders.toml`, the `.hshader` package; shared by the game, the launcher and the registry |
|
||||
| `horizon-shaders` | Shader packs (the `shaders/` folder format), rebuilt clean-room from a written spec of the format; no wgpu |
|
||||
| `horizon-theme` | Design tokens and fonts for the launcher, the game's own menus and the website |
|
||||
| `horizon-telemetry`, `horizon-telemetry-server`, `horizon-bench` | Opt-in data collection, its ingest server, the benchmark report format |
|
||||
|
||||
Generated
+9
@@ -1437,6 +1437,15 @@ dependencies = [
|
||||
"ureq",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "horizon-shader-api"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"serde",
|
||||
"toml",
|
||||
"zip",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "horizon-shaders"
|
||||
version = "0.1.0"
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
[workspace]
|
||||
exclude = ["launcher", "crates/horizon-telemetry-server", "crates/horizon-mod-registry", "third_party/wgpu-hal"]
|
||||
members = ["crates/horizon-core", "crates/horizon-dualsense", "crates/horizon-threads", "crates/horizon-world", "crates/horizon-client", "crates/horizon-server-files", "crates/horizon-server", "crates/horizon-bots", "crates/horizon-assets", "crates/horizon-protocol", "crates/horizon-theme", "crates/horizon-prepare", "crates/horizon-telemetry", "crates/horizon-bench", "crates/horizon-friends", "crates/horizon-relay", "crates/horizon-mod-sdk", "crates/horizon-mods", "crates/horizon-mods/tests/guest", "crates/cargo-horizon-mod", "crates/horizon-port-pack", "crates/horizon-shaders", "crates/horizon-release-sign", "crates/horizon-mod-sdk/examples/coords-hud", "xtask"]
|
||||
members = ["crates/horizon-core", "crates/horizon-dualsense", "crates/horizon-threads", "crates/horizon-world", "crates/horizon-client", "crates/horizon-server-files", "crates/horizon-server", "crates/horizon-bots", "crates/horizon-assets", "crates/horizon-protocol", "crates/horizon-theme", "crates/horizon-prepare", "crates/horizon-telemetry", "crates/horizon-bench", "crates/horizon-friends", "crates/horizon-relay", "crates/horizon-mod-sdk", "crates/horizon-mods", "crates/horizon-mods/tests/guest", "crates/cargo-horizon-mod", "crates/horizon-port-pack", "crates/horizon-shaders", "crates/horizon-shader-api", "crates/horizon-release-sign", "crates/horizon-mod-sdk/examples/coords-hud", "xtask"]
|
||||
resolver = "2"
|
||||
|
||||
[workspace.package]
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
[package]
|
||||
name = "horizon-shader-api"
|
||||
version = "0.1.0"
|
||||
edition.workspace = true
|
||||
license.workspace = true
|
||||
rust-version = "1.85"
|
||||
description = "Horizon's own shader packs (WGSL, docs/api/shaders.md): horizon-shaders.toml, the .hshader package"
|
||||
|
||||
[dependencies]
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
toml = "1.1.6"
|
||||
zip = { version = "2", default-features = false, features = ["deflate"] }
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
@@ -0,0 +1,10 @@
|
||||
//! Horizon's own shader packs (docs/api/shaders.md): WGSL written against
|
||||
//! the renderer's stages, described by `horizon-shaders.toml` and shipped as
|
||||
//! a folder or a `.hshader`. This crate reads and checks packs; the game, the
|
||||
//! launcher and the registry all use it, so a pack is judged the same way
|
||||
//! everywhere.
|
||||
pub mod manifest;
|
||||
pub mod package;
|
||||
|
||||
pub use manifest::{API, MANIFEST, Manifest, OptionKind, PackOption, SCHEMA, Stage};
|
||||
pub use package::{EXTENSION, Pack};
|
||||
@@ -0,0 +1,365 @@
|
||||
//! `horizon-shaders.toml`, a shader pack's manifest (schema 1, docs/shaders.md).
|
||||
use serde::Deserialize;
|
||||
|
||||
/// The manifest's file name at the root of a pack.
|
||||
pub const MANIFEST: &str = "horizon-shaders.toml";
|
||||
/// The manifest schema this build reads.
|
||||
pub const SCHEMA: u32 = 1;
|
||||
/// The shader API line this build serves: the stages, their signatures and
|
||||
/// the inputs a pack's WGSL may read.
|
||||
pub const API: &str = "0.1";
|
||||
/// Full-screen passes a pack may chain.
|
||||
pub const MAX_POST_PASSES: usize = 8;
|
||||
/// Options a pack may declare.
|
||||
pub const MAX_OPTIONS: usize = 64;
|
||||
/// The most GPU time a pack may declare for itself, in milliseconds.
|
||||
pub const MAX_GPU_MS: f32 = 16.0;
|
||||
|
||||
/// A checked manifest.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct Manifest {
|
||||
/// `[a-z0-9_]{1,64}`: the pack's settings key and registry id.
|
||||
pub id: String,
|
||||
pub name: String,
|
||||
pub version: String,
|
||||
/// The API line the pack is written for.
|
||||
pub api: String,
|
||||
pub description: String,
|
||||
pub authors: Vec<String>,
|
||||
pub license: Option<String>,
|
||||
/// WGSL files put before every stage's own file, in order.
|
||||
pub shared: Vec<String>,
|
||||
pub sky: Option<Stage>,
|
||||
pub material: Option<Stage>,
|
||||
/// Full-screen passes, in the order they run.
|
||||
pub post: Vec<Stage>,
|
||||
pub options: Vec<PackOption>,
|
||||
pub budget: Budget,
|
||||
}
|
||||
|
||||
/// One stage: the WGSL file that defines its function.
|
||||
#[derive(Clone, Debug, Deserialize, PartialEq)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct Stage {
|
||||
pub file: String,
|
||||
/// Shown in the GPU time overlay; the file name when absent.
|
||||
#[serde(default)]
|
||||
pub name: Option<String>,
|
||||
}
|
||||
|
||||
/// An option the player can change: a WGSL `override` constant of the
|
||||
/// same name, set when the pipelines are built.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct PackOption {
|
||||
/// The `override`'s name in the pack's WGSL.
|
||||
pub id: String,
|
||||
pub name: String,
|
||||
pub description: String,
|
||||
pub kind: OptionKind,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub enum OptionKind {
|
||||
/// `override <id>: bool`.
|
||||
Bool { default: bool },
|
||||
/// `override <id>: f32`; the slider moves by `step`.
|
||||
Float {
|
||||
default: f32,
|
||||
min: f32,
|
||||
max: f32,
|
||||
step: f32,
|
||||
},
|
||||
/// `override <id>: i32`.
|
||||
Int { default: i32, min: i32, max: i32 },
|
||||
/// `override <id>: u32`, the index of the chosen value.
|
||||
Choice { values: Vec<String>, default: u32 },
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default, PartialEq)]
|
||||
pub struct Budget {
|
||||
/// The GPU time the pack aims to add to a frame, in milliseconds; the
|
||||
/// overlay shows the pack over budget past it.
|
||||
pub gpu_ms: Option<f32>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
struct Raw {
|
||||
schema: u32,
|
||||
id: String,
|
||||
name: String,
|
||||
version: String,
|
||||
api: String,
|
||||
#[serde(default)]
|
||||
description: String,
|
||||
#[serde(default)]
|
||||
authors: Vec<String>,
|
||||
license: Option<String>,
|
||||
#[serde(default)]
|
||||
shared: Vec<String>,
|
||||
sky: Option<Stage>,
|
||||
material: Option<Stage>,
|
||||
#[serde(default)]
|
||||
post: Vec<Stage>,
|
||||
#[serde(default)]
|
||||
options: Vec<RawOption>,
|
||||
#[serde(default)]
|
||||
budget: RawBudget,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
struct RawOption {
|
||||
id: String,
|
||||
name: String,
|
||||
#[serde(default)]
|
||||
description: String,
|
||||
#[serde(rename = "type")]
|
||||
kind: String,
|
||||
default: toml::Value,
|
||||
min: Option<toml::Value>,
|
||||
max: Option<toml::Value>,
|
||||
step: Option<f64>,
|
||||
values: Option<Vec<String>>,
|
||||
}
|
||||
|
||||
#[derive(Default, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
struct RawBudget {
|
||||
gpu_ms: Option<f64>,
|
||||
}
|
||||
|
||||
impl Manifest {
|
||||
pub fn parse(text: &str) -> Result<Manifest, String> {
|
||||
let raw: Raw = toml::from_str(text).map_err(|e| format!("{MANIFEST}: {e}"))?;
|
||||
let err = |e: String| format!("{MANIFEST}: {e}");
|
||||
if raw.schema != SCHEMA {
|
||||
return Err(err(format!(
|
||||
"schema {}; this Horizon reads {SCHEMA}",
|
||||
raw.schema
|
||||
)));
|
||||
}
|
||||
if raw.api != API {
|
||||
return Err(err(format!(
|
||||
"written for shader API {}; this Horizon serves {API}",
|
||||
raw.api
|
||||
)));
|
||||
}
|
||||
let id_ok = (1..=64).contains(&raw.id.len())
|
||||
&& raw
|
||||
.id
|
||||
.bytes()
|
||||
.all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'_');
|
||||
if !id_ok {
|
||||
return Err(err(format!("id {:?} is not [a-z0-9_]{{1,64}}", raw.id)));
|
||||
}
|
||||
if raw.name.trim().is_empty() || raw.version.trim().is_empty() {
|
||||
return Err(err("name and version can't be empty".into()));
|
||||
}
|
||||
if raw.sky.is_none() && raw.material.is_none() && raw.post.is_empty() {
|
||||
return Err(err(
|
||||
"no stage: give it a [sky], a [material] or a [[post]]".into()
|
||||
));
|
||||
}
|
||||
if raw.post.len() > MAX_POST_PASSES {
|
||||
return Err(err(format!(
|
||||
"{} post passes; a pack may chain {MAX_POST_PASSES}",
|
||||
raw.post.len()
|
||||
)));
|
||||
}
|
||||
let stages = raw.sky.iter().chain(&raw.material).chain(&raw.post);
|
||||
for file in raw.shared.iter().chain(stages.map(|s| &s.file)) {
|
||||
if !crate::package::plain_path(file) || !file.ends_with(".wgsl") {
|
||||
return Err(err(format!("{file:?} is not a plain path to a .wgsl file")));
|
||||
}
|
||||
}
|
||||
if raw.options.len() > MAX_OPTIONS {
|
||||
return Err(err(format!(
|
||||
"{} options; a pack may declare {MAX_OPTIONS}",
|
||||
raw.options.len()
|
||||
)));
|
||||
}
|
||||
let mut options = Vec::with_capacity(raw.options.len());
|
||||
for (i, o) in raw.options.into_iter().enumerate() {
|
||||
let id = o.id.clone();
|
||||
let option = o.check().map_err(|e| err(format!("option {id:?}: {e}")))?;
|
||||
if options[..i].iter().any(|p: &PackOption| p.id == option.id) {
|
||||
return Err(err(format!("option {id:?} is declared twice")));
|
||||
}
|
||||
options.push(option);
|
||||
}
|
||||
let gpu_ms = match raw.budget.gpu_ms {
|
||||
Some(ms) if !(ms > 0.0 && ms <= f64::from(MAX_GPU_MS)) => {
|
||||
return Err(err(format!(
|
||||
"budget.gpu_ms {ms} is not in (0, {MAX_GPU_MS}]"
|
||||
)));
|
||||
}
|
||||
ms => ms.map(|ms| ms as f32),
|
||||
};
|
||||
Ok(Manifest {
|
||||
id: raw.id,
|
||||
name: raw.name,
|
||||
version: raw.version,
|
||||
api: raw.api,
|
||||
description: raw.description,
|
||||
authors: raw.authors,
|
||||
license: raw.license,
|
||||
shared: raw.shared,
|
||||
sky: raw.sky,
|
||||
material: raw.material,
|
||||
post: raw.post,
|
||||
options,
|
||||
budget: Budget { gpu_ms },
|
||||
})
|
||||
}
|
||||
|
||||
/// Every WGSL file the manifest names, shared files first, each once.
|
||||
pub fn files(&self) -> Vec<&str> {
|
||||
let mut files: Vec<&str> = Vec::new();
|
||||
let stages = self.sky.iter().chain(&self.material).chain(&self.post);
|
||||
for file in self.shared.iter().chain(stages.map(|s| &s.file)) {
|
||||
if !files.contains(&file.as_str()) {
|
||||
files.push(file);
|
||||
}
|
||||
}
|
||||
files
|
||||
}
|
||||
}
|
||||
|
||||
/// A WGSL identifier the renderer leaves to packs: ASCII, not starting
|
||||
/// with a digit, `_` or the renderer's own `horizon_` prefix.
|
||||
pub fn pack_identifier(name: &str) -> bool {
|
||||
let mut bytes = name.bytes();
|
||||
bytes.next().is_some_and(|b| b.is_ascii_alphabetic())
|
||||
&& bytes.all(|b| b.is_ascii_alphanumeric() || b == b'_')
|
||||
&& name.len() <= 64
|
||||
&& !name.starts_with("horizon_")
|
||||
}
|
||||
|
||||
impl RawOption {
|
||||
fn check(self) -> Result<PackOption, String> {
|
||||
if !pack_identifier(&self.id) {
|
||||
return Err(
|
||||
"the id must be a WGSL identifier: a letter, then letters, digits or _, up to 64, not starting with horizon_".into(),
|
||||
);
|
||||
}
|
||||
let int = |v: &toml::Value| {
|
||||
v.as_integer()
|
||||
.and_then(|i| i32::try_from(i).ok())
|
||||
.ok_or_else(|| format!("{v} is not a 32-bit integer"))
|
||||
};
|
||||
let float = |v: &toml::Value| {
|
||||
v.as_float()
|
||||
.or_else(|| v.as_integer().map(|i| i as f64))
|
||||
.filter(|f| f.is_finite())
|
||||
.map(|f| f as f32)
|
||||
.ok_or_else(|| format!("{v} is not a number"))
|
||||
};
|
||||
let need = |v: Option<toml::Value>, key: &str| v.ok_or_else(|| format!("needs {key}"));
|
||||
let unused = |present: bool, key: &str| {
|
||||
if present {
|
||||
Err(format!("{key} doesn't apply to type = {:?}", self.kind))
|
||||
} else {
|
||||
Ok(())
|
||||
}
|
||||
};
|
||||
let kind = match self.kind.as_str() {
|
||||
"bool" => {
|
||||
unused(self.min.is_some() || self.max.is_some(), "min/max")?;
|
||||
unused(self.step.is_some(), "step")?;
|
||||
unused(self.values.is_some(), "values")?;
|
||||
let default = self
|
||||
.default
|
||||
.as_bool()
|
||||
.ok_or("default must be true or false")?;
|
||||
OptionKind::Bool { default }
|
||||
}
|
||||
"float" => {
|
||||
unused(self.values.is_some(), "values")?;
|
||||
let (min, max) = (
|
||||
float(&need(self.min, "min")?)?,
|
||||
float(&need(self.max, "max")?)?,
|
||||
);
|
||||
let default = float(&self.default)?;
|
||||
let step = self.step.map_or((max - min) / 100.0, |s| s as f32);
|
||||
if !(min < max && (min..=max).contains(&default)) {
|
||||
return Err(format!("needs min < max and default in [{min}, {max}]"));
|
||||
}
|
||||
if !(step > 0.0 && step <= max - min) {
|
||||
return Err(format!("step {step} is not in (0, max - min]"));
|
||||
}
|
||||
OptionKind::Float {
|
||||
default,
|
||||
min,
|
||||
max,
|
||||
step,
|
||||
}
|
||||
}
|
||||
"int" => {
|
||||
unused(self.step.is_some(), "step")?;
|
||||
unused(self.values.is_some(), "values")?;
|
||||
let (min, max) = (int(&need(self.min, "min")?)?, int(&need(self.max, "max")?)?);
|
||||
let default = int(&self.default)?;
|
||||
if !(min < max && (min..=max).contains(&default)) {
|
||||
return Err(format!("needs min < max and default in [{min}, {max}]"));
|
||||
}
|
||||
OptionKind::Int { default, min, max }
|
||||
}
|
||||
"choice" => {
|
||||
unused(self.min.is_some() || self.max.is_some(), "min/max")?;
|
||||
unused(self.step.is_some(), "step")?;
|
||||
let values = self.values.ok_or("needs values")?;
|
||||
if !(2..=32).contains(&values.len()) {
|
||||
return Err("needs 2 to 32 values".into());
|
||||
}
|
||||
if values
|
||||
.iter()
|
||||
.enumerate()
|
||||
.any(|(i, v)| v.trim().is_empty() || values[..i].contains(v))
|
||||
{
|
||||
return Err("values must be distinct and not empty".into());
|
||||
}
|
||||
let default = self
|
||||
.default
|
||||
.as_str()
|
||||
.and_then(|d| values.iter().position(|v| v == d))
|
||||
.ok_or("default must be one of the values")?;
|
||||
OptionKind::Choice {
|
||||
values,
|
||||
default: default as u32,
|
||||
}
|
||||
}
|
||||
other => {
|
||||
return Err(format!("type {other:?} is not bool, float, int or choice"));
|
||||
}
|
||||
};
|
||||
Ok(PackOption {
|
||||
id: self.id,
|
||||
name: self.name,
|
||||
description: self.description,
|
||||
kind,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
impl OptionKind {
|
||||
/// The WGSL type of the option's `override`.
|
||||
pub fn wgsl_type(&self) -> &'static str {
|
||||
match self {
|
||||
OptionKind::Bool { .. } => "bool",
|
||||
OptionKind::Float { .. } => "f32",
|
||||
OptionKind::Int { .. } => "i32",
|
||||
OptionKind::Choice { .. } => "u32",
|
||||
}
|
||||
}
|
||||
|
||||
/// The default as the pipeline constant wgpu takes.
|
||||
pub fn default_constant(&self) -> f64 {
|
||||
match *self {
|
||||
OptionKind::Bool { default } => f64::from(u8::from(default)),
|
||||
OptionKind::Float { default, .. } => f64::from(default),
|
||||
OptionKind::Int { default, .. } => f64::from(default),
|
||||
OptionKind::Choice { default, .. } => f64::from(default),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,175 @@
|
||||
//! A shader pack on disk: a folder holding `horizon-shaders.toml`, or the
|
||||
//! same files zipped as a `.hshader` (what the registry and the launcher
|
||||
//! carry). Both read into one [`Pack`].
|
||||
use crate::manifest::{MANIFEST, Manifest};
|
||||
use std::{
|
||||
collections::BTreeMap,
|
||||
io::{Read, Write},
|
||||
path::Path,
|
||||
};
|
||||
|
||||
/// A packaged pack's extension.
|
||||
pub const EXTENSION: &str = "hshader";
|
||||
/// Bytes per WGSL file, per other file, and in all.
|
||||
const MAX_WGSL: u64 = 256 << 10;
|
||||
const MAX_OTHER: u64 = 1 << 20;
|
||||
const MAX_TOTAL: u64 = 8 << 20;
|
||||
/// Files in a `.hshader`.
|
||||
const MAX_ENTRIES: usize = 512;
|
||||
/// Files a pack may hold beside its manifest and WGSL.
|
||||
const EXTRAS: [&str; 5] = [
|
||||
"README.md",
|
||||
"LICENSE",
|
||||
"LICENSE.md",
|
||||
"LICENSE.txt",
|
||||
"icon.png",
|
||||
];
|
||||
|
||||
/// A pack read from a folder or a `.hshader`: its manifest and every file,
|
||||
/// by path from the pack's root (`/`-separated).
|
||||
#[derive(Debug)]
|
||||
pub struct Pack {
|
||||
pub manifest: Manifest,
|
||||
pub files: BTreeMap<String, Vec<u8>>,
|
||||
}
|
||||
|
||||
/// A relative path made of plain parts: no root, `..`, `\`, empty or
|
||||
/// hidden parts.
|
||||
pub fn plain_path(path: &str) -> bool {
|
||||
!path.is_empty()
|
||||
&& path.split('/').all(|part| {
|
||||
!part.is_empty()
|
||||
&& !part.starts_with('.')
|
||||
&& part
|
||||
.bytes()
|
||||
.all(|b| b.is_ascii_alphanumeric() || b"._-".contains(&b))
|
||||
})
|
||||
}
|
||||
|
||||
impl Pack {
|
||||
/// Reads `path`: a `.hshader` file, or a folder (its hidden files and
|
||||
/// folders, such as `.git`, are skipped).
|
||||
pub fn read(path: &Path) -> Result<Pack, String> {
|
||||
let mut files = BTreeMap::new();
|
||||
if path.is_dir() {
|
||||
read_dir(path, "", &mut files)?;
|
||||
} else {
|
||||
let file = std::fs::File::open(path).map_err(|e| format!("{}: {e}", path.display()))?;
|
||||
let mut zip = zip::ZipArchive::new(file)
|
||||
.map_err(|e| format!("{}: not a .{EXTENSION} (zip): {e}", path.display()))?;
|
||||
if zip.len() > MAX_ENTRIES {
|
||||
return Err(format!("over {MAX_ENTRIES} files in the package"));
|
||||
}
|
||||
for i in 0..zip.len() {
|
||||
let mut entry = zip.by_index(i).map_err(|e| e.to_string())?;
|
||||
if entry.is_dir() {
|
||||
continue;
|
||||
}
|
||||
let name = entry.name().to_owned();
|
||||
let mut bytes = Vec::new();
|
||||
(&mut entry)
|
||||
.take(MAX_OTHER.max(MAX_WGSL) + 1)
|
||||
.read_to_end(&mut bytes)
|
||||
.map_err(|e| format!("{name}: {e}"))?;
|
||||
if files.insert(name.clone(), bytes).is_some() {
|
||||
return Err(format!("{name} is in the package twice"));
|
||||
}
|
||||
}
|
||||
}
|
||||
Pack::from_files(files)
|
||||
}
|
||||
|
||||
/// Checks a pack's files: a manifest, the WGSL it names (as UTF-8),
|
||||
/// nothing it doesn't use, and sizes within bounds.
|
||||
pub fn from_files(files: BTreeMap<String, Vec<u8>>) -> Result<Pack, String> {
|
||||
let text = files
|
||||
.get(MANIFEST)
|
||||
.ok_or_else(|| format!("{MANIFEST}: missing at the pack's root"))?;
|
||||
let manifest = Manifest::parse(
|
||||
std::str::from_utf8(text).map_err(|_| format!("{MANIFEST}: not UTF-8"))?,
|
||||
)?;
|
||||
let used = manifest.files();
|
||||
let mut total = 0;
|
||||
for (name, bytes) in &files {
|
||||
if !plain_path(name) {
|
||||
return Err(format!("{name:?} is not a plain path"));
|
||||
}
|
||||
let size = bytes.len() as u64;
|
||||
total += size;
|
||||
let max = if name.ends_with(".wgsl") {
|
||||
if !used.contains(&name.as_str()) {
|
||||
return Err(format!("{name} is not used by any stage or in shared"));
|
||||
}
|
||||
if std::str::from_utf8(bytes).is_err() {
|
||||
return Err(format!("{name}: not UTF-8"));
|
||||
}
|
||||
MAX_WGSL
|
||||
} else if name == MANIFEST || EXTRAS.contains(&name.as_str()) {
|
||||
MAX_OTHER
|
||||
} else {
|
||||
return Err(format!(
|
||||
"{name}: a pack holds only {MANIFEST}, .wgsl files and {}",
|
||||
EXTRAS.join(", ")
|
||||
));
|
||||
};
|
||||
if size > max {
|
||||
return Err(format!("{name} is over {} KiB", max >> 10));
|
||||
}
|
||||
}
|
||||
if total > MAX_TOTAL {
|
||||
return Err(format!("the pack is over {} MiB", MAX_TOTAL >> 20));
|
||||
}
|
||||
if let Some(missing) = used.iter().find(|f| !files.contains_key(**f)) {
|
||||
return Err(format!("{missing}: named in {MANIFEST} but missing"));
|
||||
}
|
||||
Ok(Pack { manifest, files })
|
||||
}
|
||||
|
||||
/// A WGSL file's text (every file the manifest names is checked UTF-8).
|
||||
pub fn source(&self, file: &str) -> Option<&str> {
|
||||
self.files
|
||||
.get(file)
|
||||
.and_then(|b| std::str::from_utf8(b).ok())
|
||||
}
|
||||
|
||||
/// The `.hshader` bytes: the files in path order, with fixed times and
|
||||
/// permissions, so the same pack always gives the same bytes (and the
|
||||
/// same SHA-256).
|
||||
pub fn package(&self) -> Result<Vec<u8>, String> {
|
||||
let mut zip = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
|
||||
let options = zip::write::SimpleFileOptions::default()
|
||||
.compression_method(zip::CompressionMethod::Deflated)
|
||||
.last_modified_time(zip::DateTime::default())
|
||||
.unix_permissions(0o644);
|
||||
let err = |e: zip::result::ZipError| e.to_string();
|
||||
for (name, bytes) in &self.files {
|
||||
zip.start_file(name.as_str(), options).map_err(err)?;
|
||||
zip.write_all(bytes).map_err(|e| e.to_string())?;
|
||||
}
|
||||
Ok(zip.finish().map_err(err)?.into_inner())
|
||||
}
|
||||
}
|
||||
|
||||
fn read_dir(dir: &Path, prefix: &str, files: &mut BTreeMap<String, Vec<u8>>) -> Result<(), String> {
|
||||
let entries = std::fs::read_dir(dir).map_err(|e| format!("{}: {e}", dir.display()))?;
|
||||
for entry in entries {
|
||||
let entry = entry.map_err(|e| format!("{}: {e}", dir.display()))?;
|
||||
let name = entry.file_name().to_string_lossy().into_owned();
|
||||
if name.starts_with('.') {
|
||||
continue;
|
||||
}
|
||||
let path = entry.path();
|
||||
let rel = format!("{prefix}{name}");
|
||||
if path.is_dir() {
|
||||
read_dir(&path, &format!("{rel}/"), files)?;
|
||||
} else {
|
||||
let size = entry.metadata().map_err(|e| format!("{rel}: {e}"))?.len();
|
||||
if size > MAX_TOTAL {
|
||||
return Err(format!("{rel} is over {} MiB", MAX_TOTAL >> 20));
|
||||
}
|
||||
let bytes = std::fs::read(&path).map_err(|e| format!("{rel}: {e}"))?;
|
||||
files.insert(rel, bytes);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
The schema 1 sample read by `tests/schema.rs`. Its WGSL is checked once the stages exist.
|
||||
@@ -0,0 +1,3 @@
|
||||
fn luminance(c: vec3<f32>) -> f32 {
|
||||
return dot(c, vec3<f32>(0.2126, 0.7152, 0.0722));
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
# Schema 1 of horizon-shaders.toml with every key. Readers of later schemas
|
||||
# keep reading this file (docs/api/stability.md, "Data formats").
|
||||
schema = 1
|
||||
id = "sample_pack"
|
||||
name = "Sample Pack"
|
||||
version = "1.0.0"
|
||||
api = "0.1"
|
||||
description = "Every key of schema 1."
|
||||
authors = ["Horizon"]
|
||||
license = "MIT"
|
||||
shared = ["common.wgsl"]
|
||||
|
||||
[sky]
|
||||
file = "sky.wgsl"
|
||||
|
||||
[material]
|
||||
file = "material.wgsl"
|
||||
name = "Terrain"
|
||||
|
||||
[[post]]
|
||||
file = "post/grade.wgsl"
|
||||
name = "Colour grade"
|
||||
|
||||
[[post]]
|
||||
file = "post/vignette.wgsl"
|
||||
|
||||
[[options]]
|
||||
id = "saturation"
|
||||
name = "Saturation"
|
||||
description = "How colourful the picture is."
|
||||
type = "float"
|
||||
default = 1.1
|
||||
min = 0.0
|
||||
max = 2.0
|
||||
step = 0.05
|
||||
|
||||
[[options]]
|
||||
id = "vignette"
|
||||
name = "Vignette"
|
||||
type = "bool"
|
||||
default = true
|
||||
|
||||
[[options]]
|
||||
id = "samples"
|
||||
name = "Samples"
|
||||
type = "int"
|
||||
default = 4
|
||||
min = 1
|
||||
max = 16
|
||||
|
||||
[[options]]
|
||||
id = "tonemap"
|
||||
name = "Tone mapping"
|
||||
type = "choice"
|
||||
values = ["None", "Soft", "Filmic"]
|
||||
default = "Soft"
|
||||
|
||||
[budget]
|
||||
gpu_ms = 1.5
|
||||
@@ -0,0 +1,3 @@
|
||||
fn material(input: MaterialInput) -> Surface {
|
||||
return input.surface;
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
override saturation: f32 = 1.1;
|
||||
override tonemap: u32 = 1u;
|
||||
|
||||
fn post(input: PostInput) -> vec4<f32> {
|
||||
let c = input.color.rgb;
|
||||
let graded = mix(vec3<f32>(luminance(c)), c, saturation);
|
||||
return vec4<f32>(select(graded, graded / (graded + 0.5) * 1.5, tonemap > 0u), input.color.a);
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
override vignette: bool = true;
|
||||
override samples: i32 = 4;
|
||||
|
||||
fn post(input: PostInput) -> vec4<f32> {
|
||||
let d = distance(input.uv, vec2<f32>(0.5));
|
||||
let dark = select(1.0, 1.0 - smoothstep(0.4, 0.9, d) * 0.5, vignette);
|
||||
return vec4<f32>(input.color.rgb * dark, input.color.a);
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
fn sky(input: SkyInput) -> vec4<f32> {
|
||||
return input.color;
|
||||
}
|
||||
@@ -0,0 +1,238 @@
|
||||
//! Schema 1 of `horizon-shaders.toml` and the `.hshader` package.
|
||||
use horizon_shader_api::{MANIFEST, Manifest, OptionKind, Pack};
|
||||
use std::{collections::BTreeMap, path::Path};
|
||||
|
||||
fn sample_dir() -> &'static Path {
|
||||
Path::new(concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/tests/samples/schema-1"
|
||||
))
|
||||
}
|
||||
|
||||
fn sample_text() -> String {
|
||||
std::fs::read_to_string(sample_dir().join(MANIFEST)).unwrap()
|
||||
}
|
||||
|
||||
fn refused(text: &str) -> String {
|
||||
Manifest::parse(text).expect_err("should be refused")
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reads_every_key_of_schema_1() {
|
||||
let m = Manifest::parse(&sample_text()).unwrap();
|
||||
assert_eq!(
|
||||
(
|
||||
m.id.as_str(),
|
||||
m.name.as_str(),
|
||||
m.version.as_str(),
|
||||
m.api.as_str()
|
||||
),
|
||||
("sample_pack", "Sample Pack", "1.0.0", "0.1")
|
||||
);
|
||||
assert_eq!(m.authors, ["Horizon"]);
|
||||
assert_eq!(m.license.as_deref(), Some("MIT"));
|
||||
assert_eq!(m.sky.as_ref().unwrap().file, "sky.wgsl");
|
||||
assert_eq!(
|
||||
m.material.as_ref().unwrap().name.as_deref(),
|
||||
Some("Terrain")
|
||||
);
|
||||
let post: Vec<_> = m.post.iter().map(|s| s.file.as_str()).collect();
|
||||
assert_eq!(post, ["post/grade.wgsl", "post/vignette.wgsl"]);
|
||||
assert_eq!(
|
||||
m.files(),
|
||||
[
|
||||
"common.wgsl",
|
||||
"sky.wgsl",
|
||||
"material.wgsl",
|
||||
"post/grade.wgsl",
|
||||
"post/vignette.wgsl"
|
||||
]
|
||||
);
|
||||
let kinds: Vec<_> = m.options.iter().map(|o| (o.id.as_str(), &o.kind)).collect();
|
||||
assert_eq!(
|
||||
kinds,
|
||||
[
|
||||
(
|
||||
"saturation",
|
||||
&OptionKind::Float {
|
||||
default: 1.1,
|
||||
min: 0.0,
|
||||
max: 2.0,
|
||||
step: 0.05
|
||||
}
|
||||
),
|
||||
("vignette", &OptionKind::Bool { default: true }),
|
||||
(
|
||||
"samples",
|
||||
&OptionKind::Int {
|
||||
default: 4,
|
||||
min: 1,
|
||||
max: 16
|
||||
}
|
||||
),
|
||||
(
|
||||
"tonemap",
|
||||
&OptionKind::Choice {
|
||||
values: vec!["None".into(), "Soft".into(), "Filmic".into()],
|
||||
default: 1
|
||||
}
|
||||
),
|
||||
]
|
||||
);
|
||||
let types: Vec<_> = m.options.iter().map(|o| o.kind.wgsl_type()).collect();
|
||||
assert_eq!(types, ["f32", "bool", "i32", "u32"]);
|
||||
let constants: Vec<_> = m
|
||||
.options
|
||||
.iter()
|
||||
.map(|o| o.kind.default_constant())
|
||||
.collect();
|
||||
assert_eq!(constants, [f64::from(1.1f32), 1.0, 4.0, 1.0]);
|
||||
assert_eq!(m.budget.gpu_ms, Some(1.5));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_smallest_manifest() {
|
||||
let m = Manifest::parse(
|
||||
"schema = 1\nid = \"p\"\nname = \"P\"\nversion = \"1\"\napi = \"0.1\"\n[[post]]\nfile = \"a.wgsl\"\n",
|
||||
)
|
||||
.unwrap();
|
||||
assert!(m.sky.is_none() && m.options.is_empty() && m.budget.gpu_ms.is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn refuses_what_this_build_does_not_read() {
|
||||
let text = sample_text();
|
||||
let cases = [
|
||||
(text.replace("schema = 1", "schema = 2"), "schema 2"),
|
||||
(
|
||||
text.replace("api = \"0.1\"", "api = \"0.2\""),
|
||||
"shader API 0.2",
|
||||
),
|
||||
(
|
||||
text.replace("\"sample_pack\"", "\"Sample\""),
|
||||
"is not [a-z0-9_]",
|
||||
),
|
||||
(
|
||||
text.replace("license = \"MIT\"", "licence = \"MIT\""),
|
||||
"unknown field",
|
||||
),
|
||||
(
|
||||
text.replace("gpu_ms = 1.5", "gpu_ms = 40.0"),
|
||||
"budget.gpu_ms",
|
||||
),
|
||||
(
|
||||
text.replace("\"common.wgsl\"", "\"../common.wgsl\""),
|
||||
"not a plain path",
|
||||
),
|
||||
(
|
||||
text.replace("\"sky.wgsl\"", "\"sky.glsl\""),
|
||||
"not a plain path to a .wgsl",
|
||||
),
|
||||
(
|
||||
text.replace("id = \"samples\"", "id = \"vignette\""),
|
||||
"declared twice",
|
||||
),
|
||||
(
|
||||
text.replace("id = \"samples\"", "id = \"horizon_samples\""),
|
||||
"WGSL identifier",
|
||||
),
|
||||
(
|
||||
text.replace("default = \"Soft\"", "default = \"Hard\""),
|
||||
"one of the values",
|
||||
),
|
||||
(
|
||||
text.replace("default = 1.1", "default = 3.0"),
|
||||
"default in [0, 2]",
|
||||
),
|
||||
(
|
||||
text.replace("default = 4", "default = 4.5"),
|
||||
"not a 32-bit integer",
|
||||
),
|
||||
(
|
||||
text.replace("type = \"bool\"", "type = \"color\""),
|
||||
"is not bool",
|
||||
),
|
||||
(
|
||||
text.replace("default = true", "default = true\nmin = 0"),
|
||||
"doesn't apply",
|
||||
),
|
||||
];
|
||||
for (text, needle) in cases {
|
||||
let e = refused(&text);
|
||||
assert!(e.contains(needle), "{needle:?} not in {e:?}");
|
||||
assert!(e.starts_with(MANIFEST), "{e}");
|
||||
}
|
||||
let no_stage = "schema = 1\nid = \"p\"\nname = \"P\"\nversion = \"1\"\napi = \"0.1\"\n";
|
||||
assert!(refused(no_stage).contains("no stage"));
|
||||
let many = (0..9).map(|i| format!("[[post]]\nfile = \"p{i}.wgsl\"\n"));
|
||||
assert!(refused(&format!("{no_stage}{}", many.collect::<String>())).contains("9 post passes"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reads_the_sample_folder_and_its_package_alike() {
|
||||
let folder = Pack::read(sample_dir()).unwrap();
|
||||
assert_eq!(folder.files.len(), 7);
|
||||
assert!(
|
||||
folder
|
||||
.source("post/grade.wgsl")
|
||||
.unwrap()
|
||||
.contains("fn post")
|
||||
);
|
||||
let bytes = folder.package().unwrap();
|
||||
assert_eq!(
|
||||
bytes,
|
||||
folder.package().unwrap(),
|
||||
"packaging is reproducible"
|
||||
);
|
||||
let dir = std::env::temp_dir().join(format!("horizon-shader-api-{}", std::process::id()));
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
let path = dir.join("sample_pack.hshader");
|
||||
std::fs::write(&path, &bytes).unwrap();
|
||||
let packaged = Pack::read(&path).unwrap();
|
||||
std::fs::remove_dir_all(&dir).unwrap();
|
||||
assert_eq!(packaged.files, folder.files);
|
||||
assert_eq!(packaged.manifest, folder.manifest);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn refuses_files_a_pack_does_not_use() {
|
||||
let base = || -> BTreeMap<String, Vec<u8>> { Pack::read(sample_dir()).unwrap().files };
|
||||
let mut files = base();
|
||||
files.insert("old.wgsl".into(), b"fn f() {}".to_vec());
|
||||
assert!(
|
||||
Pack::from_files(files)
|
||||
.unwrap_err()
|
||||
.contains("old.wgsl is not used")
|
||||
);
|
||||
let mut files = base();
|
||||
files.insert("run.sh".into(), Vec::new());
|
||||
assert!(
|
||||
Pack::from_files(files)
|
||||
.unwrap_err()
|
||||
.contains("run.sh: a pack holds only")
|
||||
);
|
||||
let mut files = base();
|
||||
files.remove("sky.wgsl");
|
||||
assert!(
|
||||
Pack::from_files(files)
|
||||
.unwrap_err()
|
||||
.contains("sky.wgsl: named in")
|
||||
);
|
||||
let mut files = base();
|
||||
files.insert("sky.wgsl".into(), vec![0xff, 0xfe]);
|
||||
assert!(Pack::from_files(files).unwrap_err().contains("not UTF-8"));
|
||||
let mut files = base();
|
||||
files.insert("post/grade.wgsl".into(), vec![b' '; 300 << 10]);
|
||||
assert!(
|
||||
Pack::from_files(files)
|
||||
.unwrap_err()
|
||||
.contains("over 256 KiB")
|
||||
);
|
||||
let mut files = base();
|
||||
files.remove(MANIFEST);
|
||||
assert!(
|
||||
Pack::from_files(files)
|
||||
.unwrap_err()
|
||||
.contains("missing at the pack's root")
|
||||
);
|
||||
}
|
||||
+45
-1
@@ -34,6 +34,49 @@ how the world is fed to the GPU.
|
||||
benchmark ([benchmark-report.md](../benchmark-report.md)) records the pack and its cost, so "fast" stays measured.
|
||||
- **Off by default.** Without a pack the renderer is vanilla's look, and no pack code runs.
|
||||
|
||||
## The pack format (schema 1)
|
||||
|
||||
A pack is a folder, or the same files zipped as a `.hshader`, with `horizon-shaders.toml` at its root
|
||||
(crate `horizon-shader-api`; the sample with every key is
|
||||
`crates/horizon-shader-api/tests/samples/schema-1/`):
|
||||
|
||||
```toml
|
||||
schema = 1 # the manifest's format; a reader refuses schemas newer than it knows
|
||||
id = "soft_light" # [a-z0-9_]{1,64}: the settings key and the registry id
|
||||
name = "Soft Light"
|
||||
version = "1.0.0"
|
||||
api = "0.1" # the shader API line the WGSL is written for
|
||||
description = "..." # optional, like authors and license
|
||||
shared = ["common.wgsl"] # optional: WGSL put before every stage's file, in order
|
||||
|
||||
[sky] # each stage is optional, but a pack has at least one
|
||||
file = "sky.wgsl"
|
||||
|
||||
[material]
|
||||
file = "material.wgsl"
|
||||
|
||||
[[post]] # full-screen passes, in the order they run (8 at most)
|
||||
file = "post/grade.wgsl"
|
||||
name = "Colour grade" # optional: the GPU time overlay's label
|
||||
|
||||
[[options]] # 64 at most; `id` is the WGSL `override` it sets
|
||||
id = "saturation"
|
||||
name = "Saturation"
|
||||
type = "float" # bool, float (min, max, step), int (min, max), choice (values)
|
||||
default = 1.1
|
||||
min = 0.0
|
||||
max = 2.0
|
||||
step = 0.05
|
||||
|
||||
[budget]
|
||||
gpu_ms = 1.5 # optional: the GPU time the pack aims to add (0 to 16 ms)
|
||||
```
|
||||
|
||||
An option's `override` has the type of its kind: `bool`, `f32`, `i32`, or `u32` for a choice (the index of the
|
||||
value). The pack holds only the manifest, the `.wgsl` files it names (each used, UTF-8, 256 KiB at most),
|
||||
`README.md`, `LICENSE` (`.md`, `.txt`) and `icon.png`, under plain paths, 8 MiB in all. A `.hshader` is built
|
||||
the same way every time (files in path order, fixed dates), so a pack's SHA-256 depends only on its files.
|
||||
|
||||
## Guide, reference and examples
|
||||
|
||||
None yet. When the first stage lands: a guide here, a generated reference (`shaders-reference.md`) and an example pack
|
||||
@@ -46,9 +89,10 @@ pack exist ([stability.md](stability.md)).
|
||||
|
||||
## Not there yet
|
||||
|
||||
Everything on this page. The order: the renderer hooks shared with the format loader, then the material and post
|
||||
The stages and everything that runs a pack (the format above is read and checked). The order: the renderer hooks shared with the format loader, then the material and post
|
||||
stages with options and hot reload, then the overlay, lighting, sky and shadow stages.
|
||||
|
||||
## Changelog
|
||||
|
||||
- 2026-10-25: the pack format, schema 1: `horizon-shaders.toml` and the `.hshader` package.
|
||||
- 2026-10-24: the earlier shader pack code was removed; this API is designed from scratch.
|
||||
|
||||
Reference in New Issue
Block a user