Opt-in data collection (1/5): consent, payloads, outbox and a gated uploader
horizon-telemetry, shared by the launcher, the game and the ingest server: - consent.json in the game folder: four switches, all off until the player answers, a consent version that re-asks when what is collected changes, the answer's time, and a random install ID made at the first yes. - Payloads: perf summaries (percentiles + 14 frame-time buckets from a fixed-size Timings recorder), crash and bug reports, and benchmark reports carried as docs/benchmark-report.md defines them; size caps per kind. - A scrubber for user folders, user names, e-mail and IP addresses, UUIDs and caller-known private terms. - The outbox: only allowed kinds get in, daily caps, 200 items / 2 MiB. - zstd batches with bounded decoding; the uploader re-reads the consent and drops withdrawn items before connecting, backs off and caps batches a day. - tests/consent_gating.rs: a local mock server sees zero connections with the consent missing, outdated or withdrawn. No server is set (DEFAULT_ENDPOINT is None): nothing is sent yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Generated
+27
@@ -955,6 +955,17 @@ dependencies = [
|
||||
"sha1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "horizon-telemetry"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"getrandom 0.3.4",
|
||||
"ruzstd",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"ureq",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "horizon-theme"
|
||||
version = "0.1.0"
|
||||
@@ -2266,6 +2277,15 @@ version = "1.0.23"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f"
|
||||
|
||||
[[package]]
|
||||
name = "ruzstd"
|
||||
version = "0.9.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "69e4ce4a039f42bf25920b95667e4d789e98749ab11bbb8d2c2936f788421bb5"
|
||||
dependencies = [
|
||||
"twox-hash",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "same-file"
|
||||
version = "1.0.6"
|
||||
@@ -2313,6 +2333,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
|
||||
dependencies = [
|
||||
"serde_core",
|
||||
"serde_derive",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -2758,6 +2779,12 @@ version = "0.25.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d2df906b07856748fa3f6e0ad0cbaa047052d4a7dd609e231c4f72cee8c36f31"
|
||||
|
||||
[[package]]
|
||||
name = "twox-hash"
|
||||
version = "2.1.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "86a801b3cea342a06d468c8710662aa29e5e05e4f5c0d62f00bbb7f2ad7941c2"
|
||||
|
||||
[[package]]
|
||||
name = "typenum"
|
||||
version = "1.20.1"
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
[workspace]
|
||||
exclude = ["launcher"]
|
||||
members = ["crates/horizon-core", "crates/horizon-world", "crates/voxel-kernels", "crates/horizon-client", "crates/horizon-assets", "crates/horizon-protocol", "crates/horizon-theme", "crates/horizon-prepare", "xtask"]
|
||||
members = ["crates/horizon-core", "crates/horizon-world", "crates/voxel-kernels", "crates/horizon-client", "crates/horizon-assets", "crates/horizon-protocol", "crates/horizon-theme", "crates/horizon-prepare", "crates/horizon-telemetry", "xtask"]
|
||||
resolver = "2"
|
||||
|
||||
[workspace.package]
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
[package]
|
||||
name = "horizon-telemetry"
|
||||
version = "0.1.0"
|
||||
edition.workspace = true
|
||||
license.workspace = true
|
||||
rust-version = "1.90"
|
||||
description = "Opt-in data collection: the player's consent, the payloads, the local outbox and its upload"
|
||||
|
||||
[dependencies]
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
# zstd in pure Rust: batches are compressed before they leave and checked when they arrive.
|
||||
ruzstd = "0.9"
|
||||
# The random install ID and outbox file names.
|
||||
getrandom = "0.3"
|
||||
# Sending, for the launcher; the ingest server builds without it.
|
||||
ureq = { version = "2", default-features = false, features = ["tls"], optional = true }
|
||||
|
||||
[features]
|
||||
default = ["upload"]
|
||||
upload = ["dep:ureq"]
|
||||
|
||||
[[test]]
|
||||
name = "consent_gating"
|
||||
required-features = ["upload"]
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
@@ -0,0 +1,198 @@
|
||||
//! What goes over the wire: an install's items as one JSON object, zstd-compressed
|
||||
//! (`Content-Encoding: zstd`). The server decodes with [`decode`], which bounds what it inflates.
|
||||
use std::io::Read;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::Item;
|
||||
use crate::consent::is_install_id;
|
||||
|
||||
/// A batch on the wire, compressed.
|
||||
pub const MAX_COMPRESSED: usize = 256 * 1024;
|
||||
/// A batch's JSON: more than the largest item, so any item fits on its own.
|
||||
pub const MAX_JSON: usize = 1024 * 1024;
|
||||
pub const MAX_ITEMS: usize = 64;
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
pub struct Batch {
|
||||
pub schema: u32,
|
||||
pub install_id: String,
|
||||
pub items: Vec<Item>,
|
||||
}
|
||||
|
||||
impl Batch {
|
||||
pub fn new(install_id: &str, items: Vec<Item>) -> Self {
|
||||
Self {
|
||||
schema: crate::SCHEMA,
|
||||
install_id: install_id.to_owned(),
|
||||
items,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn json(&self) -> Vec<u8> {
|
||||
serde_json::to_vec(self).unwrap_or_default()
|
||||
}
|
||||
}
|
||||
|
||||
pub fn encode(batch: &Batch) -> Vec<u8> {
|
||||
ruzstd::encoding::compress_to_vec(
|
||||
batch.json().as_slice(),
|
||||
ruzstd::encoding::CompressionLevel::Fastest,
|
||||
)
|
||||
}
|
||||
|
||||
/// Inflates and checks a batch: its size before and after, its schema and install ID, and
|
||||
/// every item.
|
||||
pub fn decode(bytes: &[u8]) -> Result<Batch, String> {
|
||||
if bytes.len() > MAX_COMPRESSED {
|
||||
return Err(format!("over {MAX_COMPRESSED} bytes"));
|
||||
}
|
||||
let decoder = ruzstd::decoding::StreamingDecoder::new(bytes).map_err(|e| e.to_string())?;
|
||||
let mut json = Vec::new();
|
||||
decoder
|
||||
.take(MAX_JSON as u64 + 1)
|
||||
.read_to_end(&mut json)
|
||||
.map_err(|e| e.to_string())?;
|
||||
if json.len() > MAX_JSON {
|
||||
return Err(format!("over {MAX_JSON} bytes inflated"));
|
||||
}
|
||||
let batch: Batch = serde_json::from_slice(&json).map_err(|e| e.to_string())?;
|
||||
if batch.schema == 0 || batch.schema > crate::SCHEMA {
|
||||
return Err(format!("unknown schema {}", batch.schema));
|
||||
}
|
||||
if !is_install_id(&batch.install_id) {
|
||||
return Err("not an install ID".into());
|
||||
}
|
||||
if batch.items.is_empty() || batch.items.len() > MAX_ITEMS {
|
||||
return Err(format!("1 to {MAX_ITEMS} items"));
|
||||
}
|
||||
batch.items.iter().try_for_each(Item::validate)?;
|
||||
Ok(batch)
|
||||
}
|
||||
|
||||
/// Splits items into batches of at most [`MAX_ITEMS`] and [`MAX_JSON`] bytes, in order.
|
||||
pub fn split<T>(items: Vec<(T, Item)>) -> Vec<Vec<(T, Item)>> {
|
||||
let mut batches: Vec<Vec<(T, Item)>> = Vec::new();
|
||||
let mut size = 0;
|
||||
for (tag, item) in items {
|
||||
let len = serde_json::to_vec(&item).map_or(0, |j| j.len()) + 1;
|
||||
let full = batches
|
||||
.last()
|
||||
.is_none_or(|b| b.len() >= MAX_ITEMS || size + len > MAX_JSON - 1024);
|
||||
if full {
|
||||
batches.push(Vec::new());
|
||||
size = 0;
|
||||
}
|
||||
size += len;
|
||||
if let Some(batch) = batches.last_mut() {
|
||||
batch.push((tag, item));
|
||||
}
|
||||
}
|
||||
batches
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::payload::{Crash, Payload, Perf, Source, Timings};
|
||||
|
||||
const ID: &str = "0123456789abcdef0123456789abcdef";
|
||||
|
||||
fn perf() -> Item {
|
||||
let mut t = Timings::new();
|
||||
(0..50_000).for_each(|i| t.record(6.0 + (i % 97) as f32 / 10.0));
|
||||
Item::new(
|
||||
Source::Game,
|
||||
"v0.4.2",
|
||||
Payload::Perf(Perf {
|
||||
frames: t.summary(),
|
||||
seconds: 600,
|
||||
..Perf::default()
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_batch_round_trips_and_compresses() {
|
||||
let batch = Batch::new(ID, vec![perf(); 12]);
|
||||
let bytes = encode(&batch);
|
||||
assert_eq!(decode(&bytes).unwrap(), batch);
|
||||
let json = batch.json().len();
|
||||
assert!(bytes.len() * 4 < json, "{} of {json} bytes", bytes.len());
|
||||
assert!(
|
||||
bytes.len() < 2048,
|
||||
"a day's sessions: {} bytes",
|
||||
bytes.len()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bad_batches_are_refused() {
|
||||
assert!(decode(b"not zstd").is_err());
|
||||
assert!(
|
||||
decode(&vec![0; MAX_COMPRESSED + 1])
|
||||
.unwrap_err()
|
||||
.contains("over")
|
||||
);
|
||||
let mut batch = Batch::new("someone", vec![perf()]);
|
||||
assert!(decode(&encode(&batch)).unwrap_err().contains("install ID"));
|
||||
batch.install_id = ID.into();
|
||||
batch.schema = crate::SCHEMA + 1;
|
||||
assert!(decode(&encode(&batch)).unwrap_err().contains("schema"));
|
||||
batch.schema = crate::SCHEMA;
|
||||
batch.items.clear();
|
||||
assert!(decode(&encode(&batch)).is_err());
|
||||
let huge = Item::new(
|
||||
Source::Game,
|
||||
"v1",
|
||||
Payload::Crash(Crash {
|
||||
stack: "x".repeat(40_000),
|
||||
..Crash::default()
|
||||
}),
|
||||
);
|
||||
assert!(
|
||||
decode(&encode(&Batch::new(ID, vec![huge])))
|
||||
.unwrap_err()
|
||||
.contains("over its")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_zip_bomb_stops_at_the_limit() {
|
||||
// Compresses to a few kilobytes, inflates past the limit.
|
||||
let json = format!(
|
||||
r#"{{"schema":1,"install_id":"{ID}","items":[],"pad":"{}"}}"#,
|
||||
" ".repeat(MAX_JSON * 4)
|
||||
);
|
||||
let bytes = ruzstd::encoding::compress_to_vec(
|
||||
json.as_bytes(),
|
||||
ruzstd::encoding::CompressionLevel::Fastest,
|
||||
);
|
||||
assert!(bytes.len() < MAX_COMPRESSED);
|
||||
assert!(decode(&bytes).unwrap_err().contains("inflated"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn items_split_into_batches_by_count_and_size() {
|
||||
let items: Vec<(usize, Item)> = (0..150).map(|i| (i, perf())).collect();
|
||||
let batches = split(items);
|
||||
assert_eq!(
|
||||
batches.iter().map(Vec::len).collect::<Vec<_>>(),
|
||||
[64, 64, 22]
|
||||
);
|
||||
assert_eq!(batches[2].last().unwrap().0, 149);
|
||||
let big = || {
|
||||
Item::new(
|
||||
Source::Launcher,
|
||||
"0.1.5",
|
||||
Payload::Benchmark(serde_json::json!({"pad": "x".repeat(120_000)})),
|
||||
)
|
||||
};
|
||||
let batches = split((0..20).map(|i| (i, big())).collect());
|
||||
assert!(batches.len() >= 3);
|
||||
for batch in batches {
|
||||
let json = Batch::new(ID, batch.into_iter().map(|b| b.1).collect()).json();
|
||||
assert!(json.len() <= MAX_JSON);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,191 @@
|
||||
//! The player's answer, kept in `telemetry/consent.json` in the game folder: one switch per
|
||||
//! kind of data, all off until they turn one on, with the consent version and when they answered.
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::{CONSENT_VERSION, Kind};
|
||||
|
||||
/// What the player allows, kind by kind.
|
||||
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
pub struct Choices {
|
||||
/// Performance summaries: frame-time percentiles and buckets, tick times, memory, startup.
|
||||
pub performance: bool,
|
||||
/// Crash reports, without paths, names or addresses.
|
||||
pub crashes: bool,
|
||||
/// Bug reports the player writes and sends themselves.
|
||||
pub bugs: bool,
|
||||
/// Benchmark reports from the launcher and the game.
|
||||
pub benchmarks: bool,
|
||||
}
|
||||
|
||||
impl Choices {
|
||||
pub const NONE: Self = Self {
|
||||
performance: false,
|
||||
crashes: false,
|
||||
bugs: false,
|
||||
benchmarks: false,
|
||||
};
|
||||
pub const ALL: Self = Self {
|
||||
performance: true,
|
||||
crashes: true,
|
||||
bugs: true,
|
||||
benchmarks: true,
|
||||
};
|
||||
|
||||
pub fn allows(self, kind: Kind) -> bool {
|
||||
match kind {
|
||||
Kind::Perf => self.performance,
|
||||
Kind::Crash => self.crashes,
|
||||
Kind::Bug => self.bugs,
|
||||
Kind::Benchmark => self.benchmarks,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn set(&mut self, kind: Kind, on: bool) {
|
||||
match kind {
|
||||
Kind::Perf => self.performance = on,
|
||||
Kind::Crash => self.crashes = on,
|
||||
Kind::Bug => self.bugs = on,
|
||||
Kind::Benchmark => self.benchmarks = on,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn any(self) -> bool {
|
||||
Kind::ALL.into_iter().any(|k| self.allows(k))
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
pub struct Consent {
|
||||
/// The [`CONSENT_VERSION`] the player answered; 0 if they never did.
|
||||
pub version: u32,
|
||||
/// When they answered, in seconds since 1970 (UTC).
|
||||
pub answered: u64,
|
||||
pub choices: Choices,
|
||||
/// Random, made at the first yes and replaced on reset; it ties a player's items together
|
||||
/// so they can be deleted, and nothing else. Empty until then.
|
||||
#[serde(skip_serializing_if = "String::is_empty")]
|
||||
pub install_id: String,
|
||||
}
|
||||
|
||||
impl Consent {
|
||||
pub fn path(home: &Path) -> PathBuf {
|
||||
crate::dir(home).join("consent.json")
|
||||
}
|
||||
|
||||
/// The saved answer; a missing or unreadable file is "never asked", which allows nothing.
|
||||
pub fn load(home: &Path) -> Self {
|
||||
std::fs::read(Self::path(home))
|
||||
.ok()
|
||||
.and_then(|b| serde_json::from_slice(&b).ok())
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
pub fn save(&self, home: &Path) -> std::io::Result<()> {
|
||||
let json = serde_json::to_vec_pretty(self).map_err(std::io::Error::other)?;
|
||||
crate::write_atomic(&Self::path(home), &json)
|
||||
}
|
||||
|
||||
/// Whether the launcher should ask: never answered, or what is collected has changed since.
|
||||
pub fn needs_asking(&self) -> bool {
|
||||
self.version < CONSENT_VERSION
|
||||
}
|
||||
|
||||
/// Whether items of `kind` may be kept and sent. Never before an answer to the current
|
||||
/// question (a newer launcher's answer covers an older game's).
|
||||
pub fn allows(&self, kind: Kind) -> bool {
|
||||
!self.needs_asking() && !self.install_id.is_empty() && self.choices.allows(kind)
|
||||
}
|
||||
|
||||
/// Records the player's answer now. The install ID is made at the first yes and kept when
|
||||
/// they turn everything off, so "Delete my data" can still name it.
|
||||
pub fn answer(&mut self, choices: Choices) {
|
||||
self.version = CONSENT_VERSION;
|
||||
self.answered = crate::now_secs();
|
||||
self.choices = choices;
|
||||
if choices.any() && self.install_id.is_empty() {
|
||||
self.install_id = new_install_id();
|
||||
}
|
||||
}
|
||||
|
||||
/// A new install ID (after "Delete my data"): nothing sent later links to what came before.
|
||||
pub fn reset_install_id(&mut self) {
|
||||
self.install_id = if self.choices.any() {
|
||||
new_install_id()
|
||||
} else {
|
||||
String::new()
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/// 128 random bits in hex.
|
||||
pub fn new_install_id() -> String {
|
||||
crate::random_hex(16)
|
||||
}
|
||||
|
||||
/// Whether `id` looks like one of ours: 32 lowercase hex digits.
|
||||
pub fn is_install_id(id: &str) -> bool {
|
||||
id.len() == 32 && id.bytes().all(|b| matches!(b, b'0'..=b'9' | b'a'..=b'f'))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn nothing_is_allowed_before_the_player_answers() {
|
||||
let home = crate::test_home::new("consent-default");
|
||||
let consent = Consent::load(&home);
|
||||
assert!(consent.needs_asking());
|
||||
assert!(Kind::ALL.iter().all(|&k| !consent.allows(k)));
|
||||
// A broken file is the same as none.
|
||||
std::fs::create_dir_all(crate::dir(&home)).unwrap();
|
||||
std::fs::write(Consent::path(&home), b"{not json").unwrap();
|
||||
assert_eq!(Consent::load(&home), Consent::default());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_answer_allows_only_what_was_chosen_and_is_saved() {
|
||||
let home = crate::test_home::new("consent-answer");
|
||||
let mut consent = Consent::load(&home);
|
||||
consent.answer(Choices {
|
||||
crashes: true,
|
||||
..Choices::NONE
|
||||
});
|
||||
consent.save(&home).unwrap();
|
||||
let back = Consent::load(&home);
|
||||
assert_eq!(back, consent);
|
||||
assert!(!back.needs_asking());
|
||||
assert!(back.allows(Kind::Crash));
|
||||
assert!(!back.allows(Kind::Perf) && !back.allows(Kind::Bug));
|
||||
assert!(!back.allows(Kind::Benchmark));
|
||||
assert!(is_install_id(&back.install_id));
|
||||
assert!(back.answered > 1_700_000_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_older_answer_allows_nothing_until_asked_again() {
|
||||
let mut consent = Consent::default();
|
||||
consent.answer(Choices::ALL);
|
||||
consent.version = CONSENT_VERSION - 1;
|
||||
assert!(consent.needs_asking());
|
||||
assert!(Kind::ALL.iter().all(|&k| !consent.allows(k)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_id_is_made_for_a_no_and_a_reset_changes_it() {
|
||||
let mut consent = Consent::default();
|
||||
consent.answer(Choices::NONE);
|
||||
assert!(consent.install_id.is_empty());
|
||||
consent.answer(Choices::ALL);
|
||||
let first = consent.install_id.clone();
|
||||
consent.answer(Choices::NONE);
|
||||
assert_eq!(consent.install_id, first, "kept so the data can be deleted");
|
||||
consent.answer(Choices::ALL);
|
||||
consent.reset_install_id();
|
||||
assert!(is_install_id(&consent.install_id) && consent.install_id != first);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
//! Opt-in data collection, shared by the launcher, the game and the ingest server.
|
||||
//!
|
||||
//! Nothing is collected or sent until the player says yes, kind by kind ([`Consent`]): the
|
||||
//! game and the launcher only [`outbox::push`] items whose kind is allowed, and the launcher's
|
||||
//! [`upload::flush`] drops anything the player has since turned off before it opens a
|
||||
//! connection. Items are small summaries (percentiles and histogram buckets, never raw streams),
|
||||
//! dated to the day, with no account, world, server or address in them ([`scrub`]); they leave
|
||||
//! in zstd batches ([`batch`]). `docs/telemetry.md` lists every field.
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
pub mod batch;
|
||||
pub mod consent;
|
||||
pub mod outbox;
|
||||
pub mod payload;
|
||||
pub mod scrub;
|
||||
pub mod system;
|
||||
#[cfg(feature = "upload")]
|
||||
pub mod upload;
|
||||
|
||||
pub use consent::{Choices, Consent};
|
||||
pub use payload::{Item, Kind, Payload, Source};
|
||||
|
||||
/// The payload format. New optional fields keep it; a field that changes meaning or goes
|
||||
/// bumps it, and the server keeps reading the older ones.
|
||||
pub const SCHEMA: u32 = 1;
|
||||
|
||||
/// What players agree to. Bumped when what is collected changes: the launcher then asks again,
|
||||
/// and until the player answers nothing new is kept or sent.
|
||||
pub const CONSENT_VERSION: u32 = 1;
|
||||
|
||||
/// The ingest server, or none while there isn't one: then nothing is ever sent and the outbox
|
||||
/// stays within its caps. `HORIZON_TELEMETRY_URL` overrides it (tests, a local server).
|
||||
pub const DEFAULT_ENDPOINT: Option<&str> = None;
|
||||
|
||||
/// Where items are sent, if anywhere.
|
||||
pub fn endpoint() -> Option<String> {
|
||||
match std::env::var("HORIZON_TELEMETRY_URL") {
|
||||
Ok(url) if !url.trim().is_empty() => Some(url.trim().trim_end_matches('/').to_owned()),
|
||||
_ => DEFAULT_ENDPOINT.map(str::to_owned),
|
||||
}
|
||||
}
|
||||
|
||||
/// `telemetry/` in the game folder: the consent, the outbox and the upload state, read by the
|
||||
/// launcher and the game alike.
|
||||
pub fn dir(home: &Path) -> PathBuf {
|
||||
home.join("telemetry")
|
||||
}
|
||||
|
||||
/// Days since 1970-01-01 (UTC): the only time an item carries.
|
||||
pub fn today() -> u32 {
|
||||
(now_secs() / 86_400) as u32
|
||||
}
|
||||
|
||||
pub(crate) fn now_secs() -> u64 {
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.map_or(0, |d| d.as_secs())
|
||||
}
|
||||
|
||||
/// `bytes` random bytes in lowercase hex.
|
||||
pub(crate) fn random_hex(bytes: usize) -> String {
|
||||
let mut buf = vec![0u8; bytes];
|
||||
// Without the OS's randomness the time still makes names that don't collide in practice.
|
||||
if getrandom::fill(&mut buf).is_err() {
|
||||
let nanos = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.map_or(0, |d| d.as_nanos());
|
||||
for (i, b) in buf.iter_mut().enumerate() {
|
||||
*b = (nanos >> (8 * (i % 16))) as u8;
|
||||
}
|
||||
}
|
||||
buf.iter().map(|b| format!("{b:02x}")).collect()
|
||||
}
|
||||
|
||||
/// Writes `bytes` through a temporary file, so a reader never sees half of it.
|
||||
pub(crate) fn write_atomic(path: &Path, bytes: &[u8]) -> std::io::Result<()> {
|
||||
if let Some(parent) = path.parent() {
|
||||
std::fs::create_dir_all(parent)?;
|
||||
}
|
||||
let tmp = path.with_extension(format!("tmp-{}", random_hex(4)));
|
||||
std::fs::write(&tmp, bytes)?;
|
||||
std::fs::rename(&tmp, path)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
pub(crate) mod test_home {
|
||||
use std::path::PathBuf;
|
||||
|
||||
/// An empty folder of its own under the system's temporary folder.
|
||||
pub fn new(name: &str) -> PathBuf {
|
||||
let dir =
|
||||
std::env::temp_dir().join(format!("horizon-telemetry-{name}-{}", super::random_hex(6)));
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
dir
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,236 @@
|
||||
//! Items waiting to be sent: one small JSON file each in `telemetry/outbox/`, written by the game
|
||||
//! and the launcher, sent and removed by the launcher's [`crate::upload::flush`]. Only kinds the
|
||||
//! player allows get in, each within its daily cap, and the folder never holds more than
|
||||
//! [`MAX_FILES`] items or [`MAX_BYTES`].
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::{Consent, Item, Kind};
|
||||
|
||||
pub const MAX_FILES: usize = 200;
|
||||
pub const MAX_BYTES: u64 = 2 * 1024 * 1024;
|
||||
|
||||
pub fn dir(home: &Path) -> PathBuf {
|
||||
crate::dir(home).join("outbox")
|
||||
}
|
||||
|
||||
/// How many items of each kind were kept today, so a crash loop doesn't fill the outbox.
|
||||
#[derive(Default, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
struct Kept {
|
||||
day: u32,
|
||||
counts: [u32; 4],
|
||||
}
|
||||
|
||||
fn kept_path(home: &Path) -> PathBuf {
|
||||
crate::dir(home).join("kept.json")
|
||||
}
|
||||
|
||||
/// Keeps `item` to be sent later if the player allows its kind and today's cap for it isn't
|
||||
/// reached; returns whether it was kept. An invalid item (too large…) is an error.
|
||||
pub fn push(home: &Path, item: &Item) -> std::io::Result<bool> {
|
||||
let kind = item.kind();
|
||||
if !Consent::load(home).allows(kind) {
|
||||
return Ok(false);
|
||||
}
|
||||
item.validate().map_err(std::io::Error::other)?;
|
||||
let today = crate::today();
|
||||
let mut kept: Kept = std::fs::read(kept_path(home))
|
||||
.ok()
|
||||
.and_then(|b| serde_json::from_slice(&b).ok())
|
||||
.filter(|k: &Kept| k.day == today)
|
||||
.unwrap_or(Kept {
|
||||
day: today,
|
||||
..Kept::default()
|
||||
});
|
||||
let slot = Kind::ALL.iter().position(|&k| k == kind).unwrap_or(0);
|
||||
if kept.counts[slot] >= kind.daily_cap() {
|
||||
return Ok(false);
|
||||
}
|
||||
let json = serde_json::to_vec(item).map_err(std::io::Error::other)?;
|
||||
let dir = dir(home);
|
||||
std::fs::create_dir_all(&dir)?;
|
||||
make_room(&dir, json.len() as u64)?;
|
||||
// Names sort by day, so the oldest go first when there is no room.
|
||||
let name = format!(
|
||||
"{:06}-{}-{}.json",
|
||||
item.day,
|
||||
kind.name(),
|
||||
crate::random_hex(6)
|
||||
);
|
||||
crate::write_atomic(&dir.join(name), &json)?;
|
||||
kept.counts[slot] += 1;
|
||||
let kept = serde_json::to_vec(&kept).map_err(std::io::Error::other)?;
|
||||
crate::write_atomic(&kept_path(home), &kept)?;
|
||||
Ok(true)
|
||||
}
|
||||
|
||||
fn files(dir: &Path) -> Vec<(PathBuf, u64)> {
|
||||
let mut files: Vec<(PathBuf, u64)> = std::fs::read_dir(dir)
|
||||
.into_iter()
|
||||
.flatten()
|
||||
.flatten()
|
||||
.filter(|e| e.path().extension().is_some_and(|x| x == "json"))
|
||||
.map(|e| (e.path(), e.metadata().map_or(0, |m| m.len())))
|
||||
.collect();
|
||||
files.sort();
|
||||
files
|
||||
}
|
||||
|
||||
/// Removes the oldest items until one more of `incoming` bytes fits.
|
||||
fn make_room(dir: &Path, incoming: u64) -> std::io::Result<()> {
|
||||
let files = files(dir);
|
||||
let mut count = files.len();
|
||||
let mut bytes: u64 = files.iter().map(|f| f.1).sum();
|
||||
for (path, size) in files {
|
||||
if count < MAX_FILES && bytes + incoming <= MAX_BYTES {
|
||||
break;
|
||||
}
|
||||
std::fs::remove_file(path)?;
|
||||
count -= 1;
|
||||
bytes -= size;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The items waiting, oldest first, with their files. Unreadable files are removed.
|
||||
pub fn pending(home: &Path) -> Vec<(PathBuf, Item)> {
|
||||
files(&dir(home))
|
||||
.into_iter()
|
||||
.filter_map(|(path, _)| {
|
||||
let item = std::fs::read(&path)
|
||||
.ok()
|
||||
.and_then(|b| serde_json::from_slice::<Item>(&b).ok());
|
||||
if item.is_none() {
|
||||
let _ = std::fs::remove_file(&path);
|
||||
}
|
||||
item.map(|i| (path, i))
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// The waiting items the player still allows, oldest first; the others are deleted, never sent.
|
||||
pub fn retain_allowed(home: &Path) -> Vec<(PathBuf, Item)> {
|
||||
let consent = Consent::load(home);
|
||||
pending(home)
|
||||
.into_iter()
|
||||
.filter(|(path, item)| {
|
||||
let allowed = consent.allows(item.kind());
|
||||
if !allowed {
|
||||
let _ = std::fs::remove_file(path);
|
||||
}
|
||||
allowed
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Removes every waiting item ("Delete my data", or everything turned off).
|
||||
pub fn clear(home: &Path) -> std::io::Result<()> {
|
||||
match std::fs::remove_dir_all(dir(home)) {
|
||||
Err(e) if e.kind() != std::io::ErrorKind::NotFound => Err(e),
|
||||
_ => Ok(()),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::payload::{Bug, Crash, Payload, Source};
|
||||
use crate::{Choices, test_home};
|
||||
|
||||
fn allow(home: &Path, choices: Choices) {
|
||||
let mut consent = Consent::load(home);
|
||||
consent.answer(choices);
|
||||
consent.save(home).unwrap();
|
||||
}
|
||||
|
||||
fn crash() -> Item {
|
||||
Item::new(Source::Game, "v1", Payload::Crash(Crash::default()))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn nothing_is_kept_without_consent_for_its_kind() {
|
||||
let home = test_home::new("outbox-consent");
|
||||
assert!(!push(&home, &crash()).unwrap());
|
||||
allow(
|
||||
&home,
|
||||
Choices {
|
||||
bugs: true,
|
||||
..Choices::NONE
|
||||
},
|
||||
);
|
||||
assert!(!push(&home, &crash()).unwrap());
|
||||
assert!(pending(&home).is_empty());
|
||||
assert!(!dir(&home).exists());
|
||||
allow(
|
||||
&home,
|
||||
Choices {
|
||||
crashes: true,
|
||||
..Choices::NONE
|
||||
},
|
||||
);
|
||||
assert!(push(&home, &crash()).unwrap());
|
||||
assert_eq!(pending(&home).len(), 1);
|
||||
assert_eq!(pending(&home)[0].1, crash());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn each_kind_has_a_daily_cap() {
|
||||
let home = test_home::new("outbox-cap");
|
||||
allow(&home, Choices::ALL);
|
||||
let kept = (0..20).filter(|_| push(&home, &crash()).unwrap()).count();
|
||||
assert_eq!(kept as u32, Kind::Crash.daily_cap());
|
||||
// Another kind still gets in.
|
||||
let bug = Item::new(
|
||||
Source::Launcher,
|
||||
"0.1.5",
|
||||
Payload::Bug(Bug {
|
||||
text: "x".into(),
|
||||
..Bug::default()
|
||||
}),
|
||||
);
|
||||
assert!(push(&home, &bug).unwrap());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_oldest_items_make_room() {
|
||||
let home = test_home::new("outbox-room");
|
||||
allow(&home, Choices::ALL);
|
||||
let dir = dir(&home);
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
let old = serde_json::to_vec(&crash()).unwrap();
|
||||
for i in 0..MAX_FILES {
|
||||
std::fs::write(dir.join(format!("000001-crash-{i:04}.json")), &old).unwrap();
|
||||
}
|
||||
assert!(push(&home, &crash()).unwrap());
|
||||
let files = files(&dir);
|
||||
assert_eq!(files.len(), MAX_FILES);
|
||||
assert!(!dir.join("000001-crash-0000.json").exists());
|
||||
assert!(
|
||||
files
|
||||
.last()
|
||||
.unwrap()
|
||||
.0
|
||||
.file_name()
|
||||
.unwrap()
|
||||
.to_str()
|
||||
.unwrap()
|
||||
.starts_with(&format!("{:06}", crate::today()))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn invalid_items_and_broken_files_are_refused() {
|
||||
let home = test_home::new("outbox-invalid");
|
||||
allow(&home, Choices::ALL);
|
||||
let empty_bug = Item::new(Source::Game, "v1", Payload::Bug(Bug::default()));
|
||||
assert!(push(&home, &empty_bug).is_err());
|
||||
std::fs::create_dir_all(dir(&home)).unwrap();
|
||||
std::fs::write(dir(&home).join("000001-bug-x.json"), b"{").unwrap();
|
||||
assert!(pending(&home).is_empty());
|
||||
assert!(files(&dir(&home)).is_empty());
|
||||
clear(&home).unwrap();
|
||||
clear(&home).unwrap();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,421 @@
|
||||
//! What an item holds. Summaries only: a session's frame times become a few percentiles and
|
||||
//! [`BUCKET_EDGES_MS`] counts; benchmark reports are the documented report
|
||||
//! (`docs/benchmark-report.md`) as it is.
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// The four kinds of data a player can allow, one switch each.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum Kind {
|
||||
Perf,
|
||||
Crash,
|
||||
Bug,
|
||||
Benchmark,
|
||||
}
|
||||
|
||||
impl Kind {
|
||||
pub const ALL: [Kind; 4] = [Kind::Perf, Kind::Crash, Kind::Bug, Kind::Benchmark];
|
||||
|
||||
pub fn name(self) -> &'static str {
|
||||
match self {
|
||||
Kind::Perf => "perf",
|
||||
Kind::Crash => "crash",
|
||||
Kind::Bug => "bug",
|
||||
Kind::Benchmark => "benchmark",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn from_name(name: &str) -> Option<Kind> {
|
||||
Kind::ALL.into_iter().find(|k| k.name() == name)
|
||||
}
|
||||
|
||||
/// The largest an item of this kind may be, as JSON.
|
||||
pub fn max_bytes(self) -> usize {
|
||||
match self {
|
||||
Kind::Perf => 4 * 1024,
|
||||
Kind::Crash => 32 * 1024,
|
||||
Kind::Bug => 80 * 1024,
|
||||
Kind::Benchmark => 128 * 1024,
|
||||
}
|
||||
}
|
||||
|
||||
/// How many items of this kind one install keeps a day; the rest are dropped.
|
||||
pub fn daily_cap(self) -> u32 {
|
||||
match self {
|
||||
Kind::Perf => 12,
|
||||
Kind::Crash => 5,
|
||||
Kind::Bug => 5,
|
||||
Kind::Benchmark => 10,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// What made the item.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum Source {
|
||||
Launcher,
|
||||
Game,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
pub struct Item {
|
||||
/// Days since 1970 (UTC): no finer time is kept.
|
||||
pub day: u32,
|
||||
pub source: Source,
|
||||
/// The Horizon release (`v0.4.2`), or the launcher's version.
|
||||
pub version: String,
|
||||
#[serde(flatten)]
|
||||
pub payload: Payload,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
#[serde(tag = "kind", content = "data", rename_all = "snake_case")]
|
||||
pub enum Payload {
|
||||
Perf(Perf),
|
||||
Crash(Crash),
|
||||
Bug(Bug),
|
||||
/// A benchmark report as `docs/benchmark-report.md` defines it, with its own schema field.
|
||||
Benchmark(serde_json::Value),
|
||||
}
|
||||
|
||||
impl Item {
|
||||
pub fn new(source: Source, version: impl Into<String>, payload: Payload) -> Self {
|
||||
Self {
|
||||
day: crate::today(),
|
||||
source,
|
||||
version: version.into(),
|
||||
payload,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn kind(&self) -> Kind {
|
||||
match self.payload {
|
||||
Payload::Perf(_) => Kind::Perf,
|
||||
Payload::Crash(_) => Kind::Crash,
|
||||
Payload::Bug(_) => Kind::Bug,
|
||||
Payload::Benchmark(_) => Kind::Benchmark,
|
||||
}
|
||||
}
|
||||
|
||||
/// Checks what the client and the server both refuse: an item over its kind's size, an
|
||||
/// overlong text, a benchmark report that is not an object.
|
||||
pub fn validate(&self) -> Result<(), String> {
|
||||
let size = serde_json::to_vec(self).map_or(usize::MAX, |j| j.len());
|
||||
let kind = self.kind();
|
||||
if size > kind.max_bytes() {
|
||||
return Err(format!(
|
||||
"a {} item of {size} bytes is over its {} bytes",
|
||||
kind.name(),
|
||||
kind.max_bytes()
|
||||
));
|
||||
}
|
||||
if self.version.len() > 64 {
|
||||
return Err("the version is too long".into());
|
||||
}
|
||||
match &self.payload {
|
||||
Payload::Bug(bug) if bug.text.trim().is_empty() => {
|
||||
Err("a bug report needs text".into())
|
||||
}
|
||||
Payload::Bug(bug) if bug.text.chars().count() > Bug::MAX_TEXT => Err(format!(
|
||||
"a bug report is at most {} characters",
|
||||
Bug::MAX_TEXT
|
||||
)),
|
||||
Payload::Benchmark(report) if !report.is_object() => {
|
||||
Err("a benchmark report is a JSON object".into())
|
||||
}
|
||||
_ => Ok(()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The machine, as coarse as is useful: no serial numbers, names or exact memory.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
pub struct System {
|
||||
/// `linux`, `macos` or `windows`.
|
||||
pub os: String,
|
||||
/// The distribution or system version (`macOS 26.0`, `Fedora Linux 43`).
|
||||
pub os_version: String,
|
||||
pub arch: String,
|
||||
pub cpu: String,
|
||||
pub threads: u16,
|
||||
/// Installed memory, rounded to the GiB.
|
||||
pub ram_gb: u16,
|
||||
pub gpu: String,
|
||||
pub driver: String,
|
||||
/// `Metal`, `Vulkan`, `Dx12`…
|
||||
pub backend: String,
|
||||
}
|
||||
|
||||
/// The settings that change performance.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
pub struct Settings {
|
||||
pub render_distance: u8,
|
||||
pub simulation_distance: u8,
|
||||
/// 0: unlimited.
|
||||
pub max_fps: u16,
|
||||
pub vsync: bool,
|
||||
pub fullscreen: bool,
|
||||
/// `fast`, `fancy` or `fabulous`.
|
||||
pub graphics: String,
|
||||
}
|
||||
|
||||
/// One play session's performance.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
pub struct Perf {
|
||||
pub system: System,
|
||||
pub settings: Settings,
|
||||
/// Seconds of play measured (menus and loading screens excluded).
|
||||
pub seconds: u32,
|
||||
pub frames: Summary,
|
||||
/// The integrated server's milliseconds per tick, in single player.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub ticks: Option<Summary>,
|
||||
/// The game's peak resident memory.
|
||||
pub peak_ram_mb: u32,
|
||||
/// From start to the title screen.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub to_title_ms: Option<u32>,
|
||||
/// From choosing a world to playing in it, the first time this session.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub to_world_ms: Option<u32>,
|
||||
}
|
||||
|
||||
/// A crash, scrubbed: the error and its stack, the thread, and the details that say what the
|
||||
/// game was doing (graphics adapter, settings), never its log, world or server.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
pub struct Crash {
|
||||
pub system: System,
|
||||
pub error: String,
|
||||
/// One frame a line, `function (file:line:col)`.
|
||||
pub stack: String,
|
||||
pub thread: String,
|
||||
pub details: Vec<(String, String)>,
|
||||
}
|
||||
|
||||
/// A bug report the player wrote, with the log only if they chose to add it after reading it.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
pub struct Bug {
|
||||
pub system: System,
|
||||
pub text: String,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub log: Option<String>,
|
||||
}
|
||||
|
||||
impl Bug {
|
||||
pub const MAX_TEXT: usize = 4000;
|
||||
}
|
||||
|
||||
/// Durations at a glance: percentiles and how many fell in each of [`BUCKET_EDGES_MS`].
|
||||
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
pub struct Summary {
|
||||
pub count: u64,
|
||||
pub p50_ms: f32,
|
||||
pub p95_ms: f32,
|
||||
pub p99_ms: f32,
|
||||
pub max_ms: f32,
|
||||
/// One count a bucket: under 4 ms, 4–6 ms, … 100–250 ms, 250 ms and more.
|
||||
pub buckets: Vec<u32>,
|
||||
}
|
||||
|
||||
/// The upper edges of [`Summary::buckets`] in milliseconds, around the usual refresh rates
|
||||
/// (240, 165, 144, 120, 60, 30 Hz…); the last bucket has no edge.
|
||||
pub const BUCKET_EDGES_MS: [f32; 13] = [
|
||||
4.0, 6.0, 8.0, 10.0, 12.0, 14.0, 16.7, 20.0, 25.0, 33.3, 50.0, 100.0, 250.0,
|
||||
];
|
||||
|
||||
/// Records durations in tenths of a millisecond up to [`Timings::FINE_MS`] (beyond, they count
|
||||
/// at the top) for percentiles finer than the buckets; 20 KB whatever the session's length.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Timings {
|
||||
tenths: Vec<u32>,
|
||||
count: u64,
|
||||
max_ms: f32,
|
||||
}
|
||||
|
||||
impl Default for Timings {
|
||||
fn default() -> Self {
|
||||
Self::new()
|
||||
}
|
||||
}
|
||||
|
||||
impl Timings {
|
||||
pub const FINE_MS: usize = 500;
|
||||
|
||||
pub fn new() -> Self {
|
||||
Self {
|
||||
tenths: vec![0; Self::FINE_MS * 10 + 1],
|
||||
count: 0,
|
||||
max_ms: 0.0,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn record(&mut self, ms: f32) {
|
||||
if !ms.is_finite() || ms < 0.0 {
|
||||
return;
|
||||
}
|
||||
let bin = ((ms * 10.0) as usize).min(self.tenths.len() - 1);
|
||||
self.tenths[bin] = self.tenths[bin].saturating_add(1);
|
||||
self.count += 1;
|
||||
self.max_ms = self.max_ms.max(ms);
|
||||
}
|
||||
|
||||
pub fn count(&self) -> u64 {
|
||||
self.count
|
||||
}
|
||||
|
||||
/// The duration under which a fraction `p` of them fell, to the tenth of a millisecond.
|
||||
fn percentile(&self, p: f64) -> f32 {
|
||||
let rank = ((self.count as f64 * p).ceil() as u64).max(1);
|
||||
let mut seen = 0u64;
|
||||
for (bin, &n) in self.tenths.iter().enumerate() {
|
||||
seen += u64::from(n);
|
||||
if seen >= rank {
|
||||
return if bin + 1 == self.tenths.len() {
|
||||
self.max_ms
|
||||
} else {
|
||||
((bin + 1) as f32 / 10.0).min(self.max_ms)
|
||||
};
|
||||
}
|
||||
}
|
||||
self.max_ms
|
||||
}
|
||||
|
||||
pub fn summary(&self) -> Summary {
|
||||
let mut buckets = vec![0u32; BUCKET_EDGES_MS.len() + 1];
|
||||
for (bin, &n) in self.tenths.iter().enumerate() {
|
||||
let ms = bin as f32 / 10.0;
|
||||
let bucket = BUCKET_EDGES_MS
|
||||
.iter()
|
||||
.position(|&edge| ms < edge - 0.01)
|
||||
.unwrap_or(BUCKET_EDGES_MS.len());
|
||||
buckets[bucket] = buckets[bucket].saturating_add(n);
|
||||
}
|
||||
let round = |ms: f32| (ms * 10.0).round() / 10.0;
|
||||
Summary {
|
||||
count: self.count,
|
||||
p50_ms: round(self.percentile(0.50)),
|
||||
p95_ms: round(self.percentile(0.95)),
|
||||
p99_ms: round(self.percentile(0.99)),
|
||||
max_ms: round(self.max_ms),
|
||||
buckets,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn items_are_tagged_by_kind_and_read_back() {
|
||||
let item = Item::new(
|
||||
Source::Game,
|
||||
"v0.4.2",
|
||||
Payload::Bug(Bug {
|
||||
text: "Water flickers".into(),
|
||||
..Bug::default()
|
||||
}),
|
||||
);
|
||||
let json = serde_json::to_value(&item).unwrap();
|
||||
assert_eq!(json["kind"], "bug");
|
||||
assert_eq!(json["source"], "game");
|
||||
assert_eq!(json["data"]["text"], "Water flickers");
|
||||
assert_eq!(serde_json::from_value::<Item>(json).unwrap(), item);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_benchmark_report_is_carried_as_it_is() {
|
||||
let report = serde_json::json!({"schema": "horizon-benchmark-report/1", "scenes": []});
|
||||
let item = Item::new(
|
||||
Source::Launcher,
|
||||
"0.1.5",
|
||||
Payload::Benchmark(report.clone()),
|
||||
);
|
||||
let json = serde_json::to_value(&item).unwrap();
|
||||
assert_eq!(json["data"], report);
|
||||
assert!(item.validate().is_ok());
|
||||
let not_object = Item::new(Source::Launcher, "0.1.5", Payload::Benchmark(1.into()));
|
||||
assert!(not_object.validate().is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn oversized_items_are_refused() {
|
||||
let bug = |text: String| {
|
||||
Item::new(
|
||||
Source::Game,
|
||||
"v1",
|
||||
Payload::Bug(Bug {
|
||||
text,
|
||||
..Bug::default()
|
||||
}),
|
||||
)
|
||||
};
|
||||
assert!(bug("x".repeat(Bug::MAX_TEXT)).validate().is_ok());
|
||||
assert!(bug("x".repeat(Bug::MAX_TEXT + 1)).validate().is_err());
|
||||
assert!(bug(" ".into()).validate().is_err());
|
||||
let huge = Item::new(
|
||||
Source::Game,
|
||||
"v1",
|
||||
Payload::Crash(Crash {
|
||||
stack: "frame\n".repeat(10_000),
|
||||
..Crash::default()
|
||||
}),
|
||||
);
|
||||
assert!(huge.validate().unwrap_err().contains("over its"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn timings_give_percentiles_and_buckets() {
|
||||
let mut t = Timings::new();
|
||||
// 90 frames at 16.6 ms (60 Hz), 9 at 30 ms, one 400 ms hitch.
|
||||
(0..90).for_each(|_| t.record(16.6));
|
||||
(0..9).for_each(|_| t.record(30.0));
|
||||
t.record(400.0);
|
||||
t.record(f32::NAN);
|
||||
let s = t.summary();
|
||||
assert_eq!(s.count, 100);
|
||||
assert_eq!(s.p50_ms, 16.7);
|
||||
assert_eq!(s.p95_ms, 30.1);
|
||||
assert_eq!(s.p99_ms, 30.1);
|
||||
assert_eq!(s.max_ms, 400.0);
|
||||
assert_eq!(s.buckets.len(), 14);
|
||||
assert_eq!(s.buckets[6], 90, "14–16.7 ms");
|
||||
assert_eq!(s.buckets[9], 9, "25–33.3 ms");
|
||||
assert_eq!(s.buckets[13], 1, "250 ms and more");
|
||||
assert_eq!(s.buckets.iter().sum::<u32>(), 100);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hitches_beyond_the_fine_range_keep_their_maximum() {
|
||||
let mut t = Timings::new();
|
||||
t.record(2_000.0);
|
||||
let s = t.summary();
|
||||
assert_eq!((s.p50_ms, s.max_ms), (2_000.0, 2_000.0));
|
||||
assert_eq!(s.buckets[13], 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_session_summary_is_small() {
|
||||
let mut t = Timings::new();
|
||||
(0..200_000).for_each(|i| t.record(5.0 + (i % 300) as f32 / 10.0));
|
||||
let perf = Item::new(
|
||||
Source::Game,
|
||||
"v0.4.2",
|
||||
Payload::Perf(Perf {
|
||||
frames: t.summary(),
|
||||
ticks: Some(t.summary()),
|
||||
..Perf::default()
|
||||
}),
|
||||
);
|
||||
let size = serde_json::to_vec(&perf).unwrap().len();
|
||||
assert!(size < 1024, "{size} bytes");
|
||||
assert!(perf.validate().is_ok());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,354 @@
|
||||
//! Takes out of text what could say who or where a player is, before it is kept to be sent:
|
||||
//! the user folder's name in paths, the user name, e-mail and IP addresses, UUIDs (accounts),
|
||||
//! and the terms the caller knows are private (world names, server addresses, account names).
|
||||
|
||||
/// What replaces each kind of match.
|
||||
pub const USER: &str = "<user>";
|
||||
pub const EMAIL: &str = "<email>";
|
||||
pub const IP: &str = "<ip>";
|
||||
pub const UUID: &str = "<uuid>";
|
||||
pub const PRIVATE: &str = "<private>";
|
||||
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct Scrubber {
|
||||
/// Exact terms to hide, longest first.
|
||||
terms: Vec<String>,
|
||||
user: Option<String>,
|
||||
}
|
||||
|
||||
impl Scrubber {
|
||||
/// A scrubber that also knows this machine's user name.
|
||||
pub fn new() -> Self {
|
||||
let user = ["USER", "USERNAME", "LOGNAME"]
|
||||
.iter()
|
||||
.find_map(|k| std::env::var(k).ok().filter(|u| u.trim().len() >= 3));
|
||||
Self {
|
||||
terms: Vec::new(),
|
||||
user,
|
||||
}
|
||||
}
|
||||
|
||||
/// Without the machine's user name (tests).
|
||||
pub fn bare() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
pub fn with_user(mut self, user: impl Into<String>) -> Self {
|
||||
self.user = Some(user.into()).filter(|u| u.trim().len() >= 3);
|
||||
self
|
||||
}
|
||||
|
||||
/// Hides `term` wherever it appears. Terms under 3 characters are ignored: they would
|
||||
/// blank out ordinary words.
|
||||
pub fn term(mut self, term: impl Into<String>) -> Self {
|
||||
let term = term.into();
|
||||
if term.trim().len() >= 3 && !self.terms.contains(&term) {
|
||||
self.terms.push(term);
|
||||
self.terms.sort_by_key(|t| std::cmp::Reverse(t.len()));
|
||||
}
|
||||
self
|
||||
}
|
||||
|
||||
pub fn scrub(&self, text: &str) -> String {
|
||||
let mut out = text.to_owned();
|
||||
for term in &self.terms {
|
||||
out = out.replace(term.as_str(), PRIVATE);
|
||||
}
|
||||
out = user_folders(&out);
|
||||
out = replace_tokens(&out, email_at);
|
||||
out = replace_tokens(&out, ipv4_at);
|
||||
out = replace_tokens(&out, ipv6_at);
|
||||
out = replace_tokens(&out, uuid_at);
|
||||
if let Some(user) = &self.user {
|
||||
out = replace_word(&out, user, USER);
|
||||
}
|
||||
out
|
||||
}
|
||||
}
|
||||
|
||||
/// `/Users/ana/…`, `/home/ana/…`, `C:\Users\Ana Silva\…` (also JSON-escaped): the folder after
|
||||
/// `Users` or `home` becomes [`USER`], the rest of the path stays. Windows names have spaces, so
|
||||
/// the name runs to the next separator, quote, bracket or line end.
|
||||
fn user_folders(text: &str) -> String {
|
||||
let mut out = String::with_capacity(text.len());
|
||||
let mut rest = text;
|
||||
'scan: while !rest.is_empty() {
|
||||
for parent in ["Users", "home"] {
|
||||
for sep in ["/", "\\\\", "\\"] {
|
||||
let head = format!("{sep}{parent}{sep}");
|
||||
if let Some(after) = rest.strip_prefix(head.as_str()) {
|
||||
let name_len = after
|
||||
.find(|c: char| {
|
||||
matches!(
|
||||
c,
|
||||
'/' | '\\' | '"' | '\'' | '\n' | '\r' | ')' | ']' | ',' | ';'
|
||||
)
|
||||
})
|
||||
.unwrap_or(after.len());
|
||||
out.push_str(&head);
|
||||
if name_len > 0 {
|
||||
out.push_str(USER);
|
||||
}
|
||||
rest = &after[name_len..];
|
||||
continue 'scan;
|
||||
}
|
||||
}
|
||||
}
|
||||
let c = rest.chars().next().unwrap_or_default();
|
||||
out.push(c);
|
||||
rest = &rest[c.len_utf8()..];
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Replaces every match of `matcher` (which returns the length of a match starting at a byte
|
||||
/// offset, if one does) with its label.
|
||||
fn replace_tokens(
|
||||
text: &str,
|
||||
matcher: fn(&[u8], usize) -> Option<(usize, &'static str)>,
|
||||
) -> String {
|
||||
let bytes = text.as_bytes();
|
||||
let mut out = String::with_capacity(text.len());
|
||||
let mut i = 0;
|
||||
let mut copied = 0;
|
||||
while i < bytes.len() {
|
||||
if text.is_char_boundary(i)
|
||||
&& let Some((len, label)) = matcher(bytes, i)
|
||||
{
|
||||
out.push_str(&text[copied..i]);
|
||||
out.push_str(label);
|
||||
i += len;
|
||||
copied = i;
|
||||
} else {
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
out.push_str(&text[copied..]);
|
||||
out
|
||||
}
|
||||
|
||||
fn word_byte(b: u8) -> bool {
|
||||
b.is_ascii_alphanumeric() || b == b'_'
|
||||
}
|
||||
|
||||
/// Whether a token may start at `i`: not in the middle of a word, number or address.
|
||||
fn starts_token(bytes: &[u8], i: usize) -> bool {
|
||||
i == 0 || !(word_byte(bytes[i - 1]) || matches!(bytes[i - 1], b'.' | b':' | b'-' | b'@'))
|
||||
}
|
||||
|
||||
fn ends_token(bytes: &[u8], end: usize) -> bool {
|
||||
end == bytes.len() || !word_byte(bytes[end])
|
||||
}
|
||||
|
||||
fn email_at(bytes: &[u8], i: usize) -> Option<(usize, &'static str)> {
|
||||
let local = |b: u8| word_byte(b) || matches!(b, b'.' | b'%' | b'+' | b'-');
|
||||
if !starts_token(bytes, i) || !local(bytes[i]) {
|
||||
return None;
|
||||
}
|
||||
let at = i + bytes[i..].iter().position(|&b| !local(b))?;
|
||||
if bytes.get(at) != Some(&b'@') {
|
||||
return None;
|
||||
}
|
||||
let domain = |b: u8| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-');
|
||||
let end = at + 1 + bytes[at + 1..].iter().take_while(|&&b| domain(b)).count();
|
||||
let host = &bytes[at + 1..end];
|
||||
let dot = host.iter().rposition(|&b| b == b'.')?;
|
||||
(dot > 0 && host.len() - dot > 2).then_some((end - i, EMAIL))
|
||||
}
|
||||
|
||||
fn ipv4_at(bytes: &[u8], i: usize) -> Option<(usize, &'static str)> {
|
||||
if !starts_token(bytes, i) {
|
||||
return None;
|
||||
}
|
||||
let mut end = i;
|
||||
for part in 0..4 {
|
||||
if part > 0 {
|
||||
if bytes.get(end) != Some(&b'.') {
|
||||
return None;
|
||||
}
|
||||
end += 1;
|
||||
}
|
||||
let digits = bytes[end..]
|
||||
.iter()
|
||||
.take_while(|b| b.is_ascii_digit())
|
||||
.count();
|
||||
if digits == 0 || digits > 3 {
|
||||
return None;
|
||||
}
|
||||
let n: u32 = std::str::from_utf8(&bytes[end..end + digits])
|
||||
.ok()?
|
||||
.parse()
|
||||
.ok()?;
|
||||
if n > 255 {
|
||||
return None;
|
||||
}
|
||||
end += digits;
|
||||
}
|
||||
// A fifth part makes it a version number (`10.0.19045.1`), not an address.
|
||||
if bytes.get(end) == Some(&b'.') && bytes.get(end + 1).is_some_and(u8::is_ascii_digit) {
|
||||
return None;
|
||||
}
|
||||
// A port goes with it.
|
||||
if bytes.get(end) == Some(&b':') {
|
||||
let digits = bytes[end + 1..]
|
||||
.iter()
|
||||
.take_while(|b| b.is_ascii_digit())
|
||||
.count();
|
||||
if (1..=5).contains(&digits) {
|
||||
end += 1 + digits;
|
||||
}
|
||||
}
|
||||
ends_token(bytes, end).then_some((end - i, IP))
|
||||
}
|
||||
|
||||
/// Eight hex groups with colons, or fewer around a `::` (at least three groups, so a Rust path
|
||||
/// such as `a::b` is left alone).
|
||||
fn ipv6_at(bytes: &[u8], i: usize) -> Option<(usize, &'static str)> {
|
||||
if !starts_token(bytes, i) {
|
||||
return None;
|
||||
}
|
||||
let len = bytes[i..]
|
||||
.iter()
|
||||
.take_while(|b| b.is_ascii_hexdigit() || **b == b':')
|
||||
.count();
|
||||
let token = std::str::from_utf8(&bytes[i..i + len]).ok()?;
|
||||
let token = token.trim_end_matches(':');
|
||||
let groups: Vec<&str> = token.split(':').collect();
|
||||
let filled = groups.iter().filter(|g| !g.is_empty()).count();
|
||||
let ok = groups.iter().all(|g| g.len() <= 4)
|
||||
&& (filled == 8 || (token.contains("::") && filled >= 3))
|
||||
&& groups.iter().any(|g| g.bytes().any(|b| b.is_ascii_digit()));
|
||||
(ok && ends_token(bytes, i + token.len())).then_some((token.len(), IP))
|
||||
}
|
||||
|
||||
/// `0f8c…-…` (8-4-4-4-12) or the same 32 hex digits without dashes, as accounts are named.
|
||||
fn uuid_at(bytes: &[u8], i: usize) -> Option<(usize, &'static str)> {
|
||||
if !starts_token(bytes, i) {
|
||||
return None;
|
||||
}
|
||||
let hex = |range: std::ops::Range<usize>| {
|
||||
bytes
|
||||
.get(range)
|
||||
.is_some_and(|s| s.iter().all(u8::is_ascii_hexdigit))
|
||||
};
|
||||
let dashed = [8usize, 13, 18, 23]
|
||||
.iter()
|
||||
.all(|&d| bytes.get(i + d) == Some(&b'-'))
|
||||
&& [(0, 8), (9, 13), (14, 18), (19, 23), (24, 36)]
|
||||
.iter()
|
||||
.all(|&(a, b)| hex(i + a..i + b));
|
||||
if dashed && ends_token(bytes, i + 36) {
|
||||
return Some((36, UUID));
|
||||
}
|
||||
(hex(i..i + 32) && ends_token(bytes, i + 32)).then_some((32, UUID))
|
||||
}
|
||||
|
||||
/// Whole-word occurrences of `word`, in any case.
|
||||
fn replace_word(text: &str, word: &str, label: &str) -> String {
|
||||
let lower = text.to_lowercase();
|
||||
let needle = word.to_lowercase();
|
||||
// Lowercasing can change byte lengths outside ASCII: then match exactly instead.
|
||||
let (hay, needle) = if lower.len() == text.len() && needle.len() == word.len() {
|
||||
(lower.as_str(), needle.as_str())
|
||||
} else {
|
||||
(text, word)
|
||||
};
|
||||
let bytes = text.as_bytes();
|
||||
let mut out = String::with_capacity(text.len());
|
||||
let mut copied = 0;
|
||||
let mut from = 0;
|
||||
while let Some(found) = hay[from..].find(needle) {
|
||||
let start = from + found;
|
||||
let end = start + needle.len();
|
||||
let alone = (start == 0 || !word_byte(bytes[start - 1])) && ends_token(bytes, end);
|
||||
if alone {
|
||||
out.push_str(&text[copied..start]);
|
||||
out.push_str(label);
|
||||
copied = end;
|
||||
}
|
||||
from = end;
|
||||
}
|
||||
out.push_str(&text[copied..]);
|
||||
out
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn user_folders_lose_their_name() {
|
||||
let s = Scrubber::bare();
|
||||
assert_eq!(
|
||||
s.scrub("at /Users/ana.b/code/horizon/src/main.rs:10"),
|
||||
"at /Users/<user>/code/horizon/src/main.rs:10"
|
||||
);
|
||||
assert_eq!(
|
||||
s.scrub("/home/ana/.local/share/horizon"),
|
||||
"/home/<user>/.local/share/horizon"
|
||||
);
|
||||
assert_eq!(
|
||||
s.scrub("C:\\Users\\Ana Silva\\x\nnext"),
|
||||
"C:\\Users\\<user>\\x\nnext"
|
||||
);
|
||||
assert_eq!(s.scrub(r"C:\Users\ana\AppData"), r"C:\Users\<user>\AppData");
|
||||
assert_eq!(
|
||||
s.scrub(r#"{"path":"C:\\Users\\ana\\AppData"}"#),
|
||||
r#"{"path":"C:\\Users\\<user>\\AppData"}"#
|
||||
);
|
||||
assert_eq!(s.scrub("/usr/home/"), "/usr/home/");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_user_name_goes_wherever_it_stands_alone() {
|
||||
let s = Scrubber::bare().with_user("ludo");
|
||||
assert_eq!(
|
||||
s.scrub("Ludo's machine, user ludo"),
|
||||
"<user>'s machine, user <user>"
|
||||
);
|
||||
assert_eq!(s.scrub("ludovic"), "ludovic");
|
||||
assert_eq!(Scrubber::bare().with_user("al").scrub("al"), "al");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn addresses_and_accounts_are_hidden() {
|
||||
let s = Scrubber::bare();
|
||||
assert_eq!(s.scrub("joined 192.168.1.20:25565 ok"), "joined <ip> ok");
|
||||
assert_eq!(s.scrub("from 10.0.0.1."), "from <ip>.");
|
||||
assert_eq!(s.scrub("[2a01:e0a:1f2::5]"), "[<ip>]");
|
||||
assert_eq!(s.scrub("mail ana.b+mc@example.org now"), "mail <email> now");
|
||||
assert_eq!(
|
||||
s.scrub(
|
||||
"uuid 069a79f4-44e9-4726-a5be-fca90e38aaf5 and 069a79f444e94726a5befca90e38aaf5"
|
||||
),
|
||||
"uuid <uuid> and <uuid>"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn code_and_versions_are_left_alone() {
|
||||
let s = Scrubber::bare();
|
||||
for text in [
|
||||
"std::panicking::begin_panic (src/game.rs:12:5)",
|
||||
"a::b::c",
|
||||
"driver 31.0.15.3623, Windows 10.0.19045.1",
|
||||
"wgpu 30.0.1",
|
||||
"commit 9fceb02d0ae598e95dc970b74767f19372d61af8",
|
||||
"user@host",
|
||||
] {
|
||||
assert_eq!(s.scrub(text), text);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn known_private_terms_go_first() {
|
||||
let s = Scrubber::bare()
|
||||
.term("My World")
|
||||
.term("play.example.net")
|
||||
.term("ab");
|
||||
assert_eq!(
|
||||
s.scrub("Saving My World; joined play.example.net:25565, ab"),
|
||||
"Saving <private>; joined <private>:25565, ab"
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
//! The machine as an item describes it ([`System`]): what the OS says, read the same way on
|
||||
//! every start. The game adds the GPU, its driver and the graphics backend.
|
||||
use crate::payload::System;
|
||||
|
||||
fn command_output(program: &str, args: &[&str]) -> Option<String> {
|
||||
let out = std::process::Command::new(program)
|
||||
.args(args)
|
||||
.output()
|
||||
.ok()?;
|
||||
let text = String::from_utf8_lossy(&out.stdout).trim().to_owned();
|
||||
(out.status.success() && !text.is_empty()).then_some(text)
|
||||
}
|
||||
|
||||
fn read(path: &str) -> Option<String> {
|
||||
std::fs::read_to_string(path).ok()
|
||||
}
|
||||
|
||||
/// The value after `key` in a `key: value` or `key=value` listing.
|
||||
fn field(text: &str, key: &str) -> Option<String> {
|
||||
text.lines().find_map(|l| {
|
||||
let rest = l.strip_prefix(key)?;
|
||||
let value = rest.trim_start().strip_prefix([':', '='])?;
|
||||
Some(value.trim().trim_matches('"').to_owned())
|
||||
})
|
||||
}
|
||||
|
||||
fn os_version() -> Option<String> {
|
||||
if cfg!(target_os = "linux") {
|
||||
read("/etc/os-release").and_then(|t| field(&t, "PRETTY_NAME"))
|
||||
} else if cfg!(target_os = "macos") {
|
||||
command_output("sw_vers", &["-productVersion"]).map(|v| format!("macOS {v}"))
|
||||
} else if cfg!(windows) {
|
||||
command_output("cmd", &["/C", "ver"])
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
fn cpu() -> Option<String> {
|
||||
if cfg!(target_os = "linux") {
|
||||
read("/proc/cpuinfo").and_then(|t| field(&t, "model name"))
|
||||
} else if cfg!(target_os = "macos") {
|
||||
command_output("sysctl", &["-n", "machdep.cpu.brand_string"])
|
||||
} else {
|
||||
std::env::var("PROCESSOR_IDENTIFIER").ok()
|
||||
}
|
||||
}
|
||||
|
||||
fn ram_bytes() -> Option<u64> {
|
||||
if cfg!(target_os = "linux") {
|
||||
let kib = read("/proc/meminfo").and_then(|t| field(&t, "MemTotal"))?;
|
||||
kib.split_whitespace()
|
||||
.next()?
|
||||
.parse::<u64>()
|
||||
.ok()
|
||||
.map(|k| k * 1024)
|
||||
} else if cfg!(target_os = "macos") {
|
||||
command_output("sysctl", &["-n", "hw.memsize"])?
|
||||
.parse()
|
||||
.ok()
|
||||
} else if cfg!(windows) {
|
||||
command_output(
|
||||
"powershell",
|
||||
&[
|
||||
"-NoProfile",
|
||||
"-Command",
|
||||
"(Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory",
|
||||
],
|
||||
)?
|
||||
.parse()
|
||||
.ok()
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
impl System {
|
||||
/// This machine, without the GPU.
|
||||
pub fn detect() -> Self {
|
||||
const GIB: f64 = (1u64 << 30) as f64;
|
||||
Self {
|
||||
os: std::env::consts::OS.to_owned(),
|
||||
os_version: os_version().unwrap_or_default(),
|
||||
arch: std::env::consts::ARCH.to_owned(),
|
||||
cpu: cpu().unwrap_or_default(),
|
||||
threads: std::thread::available_parallelism().map_or(0, |n| n.get() as u16),
|
||||
ram_gb: ram_bytes().map_or(0, |b| (b as f64 / GIB).round() as u16),
|
||||
..Self::default()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn fields_are_read_from_listings() {
|
||||
let os_release =
|
||||
"NAME=\"Fedora Linux\"\nPRETTY_NAME=\"Fedora Linux 43 (Workstation Edition)\"\n";
|
||||
assert_eq!(
|
||||
field(os_release, "PRETTY_NAME").as_deref(),
|
||||
Some("Fedora Linux 43 (Workstation Edition)")
|
||||
);
|
||||
let cpuinfo = "processor\t: 0\nmodel name\t: AMD Ryzen 9 7950X 16-Core Processor\n";
|
||||
assert_eq!(
|
||||
field(cpuinfo, "model name").as_deref(),
|
||||
Some("AMD Ryzen 9 7950X 16-Core Processor")
|
||||
);
|
||||
assert_eq!(
|
||||
field("MemTotal: 32788148 kB", "MemTotal").as_deref(),
|
||||
Some("32788148 kB")
|
||||
);
|
||||
assert_eq!(field("NAME=x", "PRETTY_NAME"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn this_machine_is_described() {
|
||||
let system = System::detect();
|
||||
assert_eq!(system.os, std::env::consts::OS);
|
||||
assert!(system.threads > 0);
|
||||
assert!(system.gpu.is_empty(), "the game fills the GPU in");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,141 @@
|
||||
//! Sends the outbox, from the launcher at start and exit. Before anything else it reads the
|
||||
//! consent again and deletes what the player no longer allows, so with everything off it
|
||||
//! returns without opening a connection. Failures back off (a minute, doubling, at most a day)
|
||||
//! and at most [`MAX_BATCHES_A_DAY`] batches leave a day.
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::time::Duration;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::batch::{self, Batch};
|
||||
use crate::{Consent, outbox};
|
||||
|
||||
pub const MAX_BATCHES_A_DAY: u32 = 8;
|
||||
const TIMEOUT: Duration = Duration::from_secs(10);
|
||||
|
||||
#[derive(Debug, PartialEq, Eq)]
|
||||
pub enum Flush {
|
||||
/// Nothing to send, or nothing allowed: no connection was made.
|
||||
Nothing,
|
||||
/// Waiting after a failure, or for tomorrow's allowance.
|
||||
Later,
|
||||
Sent {
|
||||
items: usize,
|
||||
batches: usize,
|
||||
},
|
||||
/// The server could not be reached or failed; tried again later.
|
||||
Failed(String),
|
||||
}
|
||||
|
||||
/// The uploader's own state, beside the consent.
|
||||
#[derive(Default, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
struct State {
|
||||
day: u32,
|
||||
batches: u32,
|
||||
failures: u32,
|
||||
retry_at: u64,
|
||||
}
|
||||
|
||||
fn state_path(home: &Path) -> PathBuf {
|
||||
crate::dir(home).join("upload.json")
|
||||
}
|
||||
|
||||
fn load(home: &Path) -> State {
|
||||
std::fs::read(state_path(home))
|
||||
.ok()
|
||||
.and_then(|b| serde_json::from_slice(&b).ok())
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
fn save(home: &Path, state: &State) {
|
||||
if let Ok(json) = serde_json::to_vec(state) {
|
||||
let _ = crate::write_atomic(&state_path(home), &json);
|
||||
}
|
||||
}
|
||||
|
||||
fn agent() -> ureq::Agent {
|
||||
ureq::AgentBuilder::new().timeout(TIMEOUT).build()
|
||||
}
|
||||
|
||||
/// Sends what the player allows to `endpoint` (`https://host`, without a trailing slash).
|
||||
pub fn flush(home: &Path, endpoint: &str) -> Flush {
|
||||
let consent = Consent::load(home);
|
||||
// What was turned off since it was kept never leaves.
|
||||
let allowed = outbox::retain_allowed(home);
|
||||
if allowed.is_empty() {
|
||||
return Flush::Nothing;
|
||||
}
|
||||
let mut state = load(home);
|
||||
let today = crate::today();
|
||||
if state.day != today {
|
||||
state.day = today;
|
||||
state.batches = 0;
|
||||
}
|
||||
if crate::now_secs() < state.retry_at || state.batches >= MAX_BATCHES_A_DAY {
|
||||
return Flush::Later;
|
||||
}
|
||||
let agent = agent();
|
||||
let url = format!("{endpoint}/v1/batch");
|
||||
let (mut items, mut batches) = (0, 0);
|
||||
for group in batch::split(allowed) {
|
||||
if state.batches >= MAX_BATCHES_A_DAY {
|
||||
break;
|
||||
}
|
||||
let (paths, list): (Vec<PathBuf>, Vec<crate::Item>) = group.into_iter().unzip();
|
||||
let body = batch::encode(&Batch::new(&consent.install_id, list));
|
||||
let sent = agent
|
||||
.post(&url)
|
||||
.set("Content-Type", "application/json")
|
||||
.set("Content-Encoding", "zstd")
|
||||
.send_bytes(&body);
|
||||
match sent {
|
||||
Ok(_) => {
|
||||
state.batches += 1;
|
||||
state.failures = 0;
|
||||
items += paths.len();
|
||||
batches += 1;
|
||||
paths.iter().for_each(|p| drop(std::fs::remove_file(p)));
|
||||
}
|
||||
// Refused for what it is (not for load): sending it again would not help.
|
||||
Err(ureq::Error::Status(code, _)) if (400..500).contains(&code) && code != 429 => {
|
||||
state.batches += 1;
|
||||
paths.iter().for_each(|p| drop(std::fs::remove_file(p)));
|
||||
}
|
||||
Err(e) => {
|
||||
state.failures = (state.failures + 1).min(16);
|
||||
state.retry_at = crate::now_secs() + (60u64 << (state.failures - 1)).min(86_400);
|
||||
save(home, &state);
|
||||
return Flush::Failed(e.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
save(home, &state);
|
||||
if batches == 0 {
|
||||
Flush::Later
|
||||
} else {
|
||||
Flush::Sent { items, batches }
|
||||
}
|
||||
}
|
||||
|
||||
/// Asks the server to delete everything sent under `install_id`.
|
||||
pub fn delete_remote(endpoint: &str, install_id: &str) -> Result<(), String> {
|
||||
if !crate::consent::is_install_id(install_id) {
|
||||
return Err("no install ID: nothing was sent".into());
|
||||
}
|
||||
agent()
|
||||
.delete(&format!("{endpoint}/v1/installs/{install_id}"))
|
||||
.call()
|
||||
.map(|_| ())
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// The batch the next [`flush`] would send, as readable JSON ("View what would be sent").
|
||||
pub fn preview(home: &Path) -> String {
|
||||
let consent = Consent::load(home);
|
||||
let items = outbox::retain_allowed(home)
|
||||
.into_iter()
|
||||
.map(|(_, item)| item)
|
||||
.collect();
|
||||
serde_json::to_string_pretty(&Batch::new(&consent.install_id, items)).unwrap_or_default()
|
||||
}
|
||||
@@ -0,0 +1,231 @@
|
||||
//! Nothing leaves the machine without consent: a local mock server counts every connection the
|
||||
//! uploader opens, and with the player's consent missing, outdated or withdrawn there are none.
|
||||
use std::io::{BufRead, BufReader, Read, Write};
|
||||
use std::net::TcpListener;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use horizon_telemetry::batch;
|
||||
use horizon_telemetry::payload::{Bug, Crash, Payload, Perf, Source};
|
||||
use horizon_telemetry::upload::{self, Flush};
|
||||
use horizon_telemetry::{Choices, Consent, Item, Kind, outbox};
|
||||
|
||||
/// What the mock server saw: one entry a connection, `METHOD path` and the body.
|
||||
type Seen = Arc<Mutex<Vec<(String, Vec<u8>)>>>;
|
||||
|
||||
/// An HTTP server on a free local port answering every request with `status`.
|
||||
fn mock_server(status: u16) -> (String, Seen) {
|
||||
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
|
||||
let url = format!("http://{}", listener.local_addr().unwrap());
|
||||
let seen: Seen = Arc::default();
|
||||
let log = seen.clone();
|
||||
std::thread::spawn(move || {
|
||||
for stream in listener.incoming() {
|
||||
let Ok(mut stream) = stream else { continue };
|
||||
let mut reader = BufReader::new(stream.try_clone().unwrap());
|
||||
let mut request = String::new();
|
||||
reader.read_line(&mut request).unwrap();
|
||||
let mut length = 0;
|
||||
loop {
|
||||
let mut line = String::new();
|
||||
reader.read_line(&mut line).unwrap();
|
||||
if line.trim().is_empty() {
|
||||
break;
|
||||
}
|
||||
if let Some((k, v)) = line.split_once(':')
|
||||
&& k.eq_ignore_ascii_case("content-length")
|
||||
{
|
||||
length = v.trim().parse().unwrap();
|
||||
}
|
||||
}
|
||||
let mut body = vec![0; length];
|
||||
reader.read_exact(&mut body).unwrap();
|
||||
let line = request
|
||||
.split_whitespace()
|
||||
.take(2)
|
||||
.collect::<Vec<_>>()
|
||||
.join(" ");
|
||||
log.lock().unwrap().push((line, body));
|
||||
let _ = write!(
|
||||
stream,
|
||||
"HTTP/1.1 {status} X\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"
|
||||
);
|
||||
}
|
||||
});
|
||||
(url, seen)
|
||||
}
|
||||
|
||||
fn home(name: &str) -> PathBuf {
|
||||
let mut bytes = [0u8; 6];
|
||||
getrandom::fill(&mut bytes).unwrap();
|
||||
let suffix: String = bytes.iter().map(|b| format!("{b:02x}")).collect();
|
||||
let dir = std::env::temp_dir().join(format!("horizon-telemetry-gating-{name}-{suffix}"));
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
dir
|
||||
}
|
||||
|
||||
fn answer(home: &Path, choices: Choices) -> Consent {
|
||||
let mut consent = Consent::load(home);
|
||||
consent.answer(choices);
|
||||
consent.save(home).unwrap();
|
||||
consent
|
||||
}
|
||||
|
||||
fn crash() -> Item {
|
||||
Item::new(
|
||||
Source::Game,
|
||||
"v0.4.2",
|
||||
Payload::Crash(Crash {
|
||||
error: "panicked at src/game.rs:1:1".into(),
|
||||
..Crash::default()
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
fn perf() -> Item {
|
||||
Item::new(Source::Game, "v0.4.2", Payload::Perf(Perf::default()))
|
||||
}
|
||||
|
||||
/// Puts an item straight into the outbox, as a game from before a withdrawal left it.
|
||||
fn leftover(home: &Path, item: &Item, name: &str) {
|
||||
let dir = outbox::dir(home);
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
std::fs::write(dir.join(name), serde_json::to_vec(item).unwrap()).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn never_asked_sends_nothing() {
|
||||
let (url, seen) = mock_server(204);
|
||||
let home = home("never");
|
||||
for item in [crash(), perf()] {
|
||||
assert!(!outbox::push(&home, &item).unwrap());
|
||||
}
|
||||
leftover(&home, &crash(), "000001-crash-a.json");
|
||||
assert_eq!(upload::flush(&home, &url), Flush::Nothing);
|
||||
assert!(seen.lock().unwrap().is_empty());
|
||||
assert!(
|
||||
outbox::pending(&home).is_empty(),
|
||||
"the leftover is deleted, not kept"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn everything_off_sends_nothing_and_deletes_what_waited() {
|
||||
let (url, seen) = mock_server(204);
|
||||
let home = home("withdrawn");
|
||||
answer(&home, Choices::ALL);
|
||||
assert!(outbox::push(&home, &crash()).unwrap());
|
||||
assert!(outbox::push(&home, &perf()).unwrap());
|
||||
answer(&home, Choices::NONE);
|
||||
assert_eq!(upload::flush(&home, &url), Flush::Nothing);
|
||||
assert!(seen.lock().unwrap().is_empty());
|
||||
assert!(outbox::pending(&home).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_outdated_consent_sends_nothing() {
|
||||
let (url, seen) = mock_server(204);
|
||||
let home = home("outdated");
|
||||
let mut consent = answer(&home, Choices::ALL);
|
||||
assert!(outbox::push(&home, &crash()).unwrap());
|
||||
consent.version = 0;
|
||||
consent.save(&home).unwrap();
|
||||
assert_eq!(upload::flush(&home, &url), Flush::Nothing);
|
||||
assert!(seen.lock().unwrap().is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn only_the_allowed_kinds_leave_in_one_compressed_batch() {
|
||||
let (url, seen) = mock_server(204);
|
||||
let home = home("partial");
|
||||
let consent = answer(&home, Choices::ALL);
|
||||
for item in [crash(), perf(), crash()] {
|
||||
assert!(outbox::push(&home, &item).unwrap());
|
||||
}
|
||||
answer(
|
||||
&home,
|
||||
Choices {
|
||||
crashes: true,
|
||||
..Choices::NONE
|
||||
},
|
||||
);
|
||||
assert_eq!(
|
||||
upload::flush(&home, &url),
|
||||
Flush::Sent {
|
||||
items: 2,
|
||||
batches: 1
|
||||
}
|
||||
);
|
||||
let seen = seen.lock().unwrap();
|
||||
assert_eq!(seen.len(), 1);
|
||||
assert_eq!(seen[0].0, "POST /v1/batch");
|
||||
let batch = batch::decode(&seen[0].1).unwrap();
|
||||
assert_eq!(batch.install_id, consent.install_id);
|
||||
assert!(batch.items.iter().all(|i| i.kind() == Kind::Crash));
|
||||
assert_eq!(batch.items.len(), 2);
|
||||
assert!(outbox::pending(&home).is_empty());
|
||||
drop(seen);
|
||||
assert_eq!(upload::flush(&home, &url), Flush::Nothing);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_failing_server_is_retried_later_and_keeps_the_items() {
|
||||
let (url, seen) = mock_server(503);
|
||||
let home = home("failing");
|
||||
answer(&home, Choices::ALL);
|
||||
assert!(outbox::push(&home, &crash()).unwrap());
|
||||
assert!(matches!(upload::flush(&home, &url), Flush::Failed(_)));
|
||||
assert_eq!(upload::flush(&home, &url), Flush::Later);
|
||||
assert_eq!(
|
||||
seen.lock().unwrap().len(),
|
||||
1,
|
||||
"no second try inside the back-off"
|
||||
);
|
||||
assert_eq!(outbox::pending(&home).len(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_refused_batch_is_dropped() {
|
||||
let (url, seen) = mock_server(400);
|
||||
let home = home("refused");
|
||||
answer(&home, Choices::ALL);
|
||||
assert!(outbox::push(&home, &crash()).unwrap());
|
||||
assert_eq!(upload::flush(&home, &url), Flush::Later);
|
||||
assert_eq!(seen.lock().unwrap().len(), 1);
|
||||
assert!(outbox::pending(&home).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_preview_is_what_would_be_sent_and_deletion_names_the_install() {
|
||||
let (url, seen) = mock_server(204);
|
||||
let home = home("preview");
|
||||
let consent = answer(
|
||||
&home,
|
||||
Choices {
|
||||
bugs: true,
|
||||
..Choices::NONE
|
||||
},
|
||||
);
|
||||
let bug = Item::new(
|
||||
Source::Launcher,
|
||||
"0.1.5",
|
||||
Payload::Bug(Bug {
|
||||
text: "The sky is green".into(),
|
||||
..Bug::default()
|
||||
}),
|
||||
);
|
||||
assert!(outbox::push(&home, &bug).unwrap());
|
||||
leftover(&home, &crash(), "000001-crash-a.json");
|
||||
let preview: serde_json::Value = serde_json::from_str(&upload::preview(&home)).unwrap();
|
||||
assert_eq!(preview["items"].as_array().unwrap().len(), 1);
|
||||
assert_eq!(preview["items"][0]["data"]["text"], "The sky is green");
|
||||
assert!(seen.lock().unwrap().is_empty(), "previewing sends nothing");
|
||||
upload::delete_remote(&url, &consent.install_id).unwrap();
|
||||
assert!(upload::delete_remote(&url, "").is_err());
|
||||
let seen = seen.lock().unwrap();
|
||||
assert_eq!(seen.len(), 1);
|
||||
assert_eq!(
|
||||
seen[0].0,
|
||||
format!("DELETE /v1/installs/{}", consent.install_id)
|
||||
);
|
||||
}
|
||||
@@ -9,6 +9,7 @@ To play, use [the launcher](../launcher/README.md); the project site is [playhor
|
||||
- [Graphics](graphics.md): the Simple and Advanced graphics settings, the graphics benchmark and what it measured.
|
||||
- [Extras](extras.md): optional zoom, minimap, FPS and keystroke overlays and auto-reconnect, all off by default.
|
||||
- [Launcher](../launcher/README.md): installs, updates and starts builds; its CLI and MCP server; releasing it.
|
||||
- [Opt-in data collection](telemetry.md): the consent, exactly what is sent, how it is stored and sent.
|
||||
- Menus: the title, world list, create world, multiplayer, options and pause screens share the launcher's design tokens and fonts (`crates/horizon-theme`), drawn by `ui_theme` over a live frozen backdrop (`game/backdrop.rs`, blurred and graded by `post.rs`). Options → Interface → Menu style switches to vanilla's screens (Classic, kept in `horizon.txt`; `HORIZON_MENU_STYLE=horizon|classic` overrides it, and scripted runs default to Classic). [Title](media/menu-title.png) ([webm](media/menu-title.webm)), [worlds](media/menu-worlds.png), [options](media/menu-options.png), [pause](media/menu-pause.png).
|
||||
|
||||
## Parity
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
# Opt-in data collection
|
||||
|
||||
Horizon collects nothing unless the player says yes. The launcher asks once, with four separate switches that are all off until the player turns one on, and the same switches stay in Settings. The game and the launcher obey the same answer. The code is in `crates/horizon-telemetry`.
|
||||
|
||||
## Consent
|
||||
|
||||
`telemetry/consent.json` in the game folder (`HORIZON_HOME`, else `horizon` in the platform data folder):
|
||||
|
||||
```json
|
||||
{ "version": 1, "answered": 1792060800, "choices": { "performance": true, "crashes": true, "bugs": false, "benchmarks": false }, "install_id": "3f2c…" }
|
||||
```
|
||||
|
||||
- `version` is the consent version the player answered (`CONSENT_VERSION`). It goes up when what is collected changes. The launcher then asks again, and until the player answers nothing new is kept or sent.
|
||||
- `answered` is when they answered, in seconds since 1970 (UTC).
|
||||
- `install_id` is 128 random bits, made the first time the player says yes to anything. It ties their items together so they can be deleted, and nothing else. "Delete my data" asks the server to delete everything under it, then replaces it.
|
||||
|
||||
Nothing leaves the machine without consent:
|
||||
|
||||
- `outbox::push` keeps an item only if its kind is allowed.
|
||||
- `upload::flush` reads the consent again before sending and deletes any waiting item whose kind the player has turned off since, so with everything off it never opens a connection.
|
||||
|
||||
`tests/consent_gating.rs` checks this against a local mock server that counts connections.
|
||||
|
||||
## What is sent
|
||||
|
||||
Every item has `day` (days since 1970, UTC: no finer time), `source` (`launcher` or `game`), `version` (the release), `kind` and `data`.
|
||||
|
||||
| Kind | Switch | `data` | Largest | A day |
|
||||
|---|---|---|---|---|
|
||||
| `perf` | performance | `system`, `settings` (render and simulation distance, max FPS, vsync, fullscreen, graphics mode), `seconds` played, `frames` and `ticks` summaries, `peak_ram_mb`, `to_title_ms`, `to_world_ms` | 4 KiB | 12 |
|
||||
| `crash` | crash reports | `system`, `error`, `stack` (one frame a line), `thread`, the graphics and settings details, all scrubbed. No log, no world, no server | 32 KiB | 5 |
|
||||
| `bug` | bug reports | `system`, the player's `text` (up to 4000 characters), and the `log` only if they add it after reading it | 80 KiB | 5 |
|
||||
| `benchmark` | benchmark uploads | the benchmark report as `docs/benchmark-report.md` defines it, as it is, with its own schema field | 128 KiB | 10 |
|
||||
|
||||
`system` is the OS and its version, the architecture, the CPU model and thread count, installed memory rounded to the GiB, and the GPU, driver and graphics backend.
|
||||
|
||||
A summary (`frames`, `ticks`) is the count, the 50th, 95th and 99th percentile and the maximum in milliseconds (to the tenth), and 14 bucket counts: under 4 ms, 4–6, 6–8, 8–10, 10–12, 12–14, 14–16.7, 16.7–20, 20–25, 25–33.3, 33.3–50, 50–100, 100–250, and 250 ms and more. Raw frame times never leave: `Timings` records them in tenths of a millisecond (20 KB however long the session) and only its summary is kept. A perf item is under 1 KiB.
|
||||
|
||||
Never sent: account names or UUIDs, world names, server or LAN addresses, chat, file paths with the user's name, IP addresses. `scrub::Scrubber` replaces the user folder in paths (`/Users/<user>/…`, `C:\Users\<user>\…`), the user name, e-mail and IP addresses and UUIDs. It also replaces the terms the caller passes (the world's name, the server's address, account names). The server does not store the sender's IP address.
|
||||
|
||||
## Storage and sending
|
||||
|
||||
- `telemetry/outbox/` holds one JSON file per item, `<day>-<kind>-<random>.json`. It never holds more than 200 items or 2 MiB, and the oldest go first. `kept.json` counts today's items per kind against the caps above.
|
||||
- The launcher sends the outbox when it starts and when it closes. Items go as one JSON batch (`{"schema":1,"install_id":…,"items":[…]}`, up to 64 items and 1 MiB), compressed with zstd: `POST /v1/batch` with `Content-Encoding: zstd`. A day's perf summaries are under 2 KB on the wire.
|
||||
- After a failure the launcher waits a minute, doubling up to a day, and sends at most 8 batches a day. A batch the server refuses with a 4xx (not 429) is dropped.
|
||||
- `HORIZON_TELEMETRY_URL` sets the server. There is none by default: until there is, nothing is sent and the outbox stays within its caps.
|
||||
- `SCHEMA` versions the format. New optional fields keep it. A field that changes meaning or goes bumps it, and the server keeps reading the older ones.
|
||||
|
||||
## Server
|
||||
|
||||
- `POST /v1/batch`
|
||||
- `DELETE /v1/installs/<install_id>`: deletes everything sent under that ID.
|
||||
- `GET /v1/aggregate`: the anonymous totals the website's results page reads.
|
||||
|
||||
Retention, rate limits and storage are described with the server.
|
||||
Reference in New Issue
Block a user