Files
horizon-airdef/docs/telemetry.md
T
Ludovic Marie 43138159b2 Opt-in data collection (1/5): consent, payloads, outbox and a gated uploader
horizon-telemetry, shared by the launcher, the game and the ingest server:
- consent.json in the game folder: four switches, all off until the player
  answers, a consent version that re-asks when what is collected changes, the
  answer's time, and a random install ID made at the first yes.
- Payloads: perf summaries (percentiles + 14 frame-time buckets from a
  fixed-size Timings recorder), crash and bug reports, and benchmark reports
  carried as docs/benchmark-report.md defines them; size caps per kind.
- A scrubber for user folders, user names, e-mail and IP addresses, UUIDs and
  caller-known private terms.
- The outbox: only allowed kinds get in, daily caps, 200 items / 2 MiB.
- zstd batches with bounded decoding; the uploader re-reads the consent and
  drops withdrawn items before connecting, backs off and caps batches a day.
- tests/consent_gating.rs: a local mock server sees zero connections with the
  consent missing, outdated or withdrawn.

No server is set (DEFAULT_ENDPOINT is None): nothing is sent yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 02:21:13 +02:00

56 lines
4.6 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Opt-in data collection
Horizon collects nothing unless the player says yes. The launcher asks once, with four separate switches that are all off until the player turns one on, and the same switches stay in Settings. The game and the launcher obey the same answer. The code is in `crates/horizon-telemetry`.
## Consent
`telemetry/consent.json` in the game folder (`HORIZON_HOME`, else `horizon` in the platform data folder):
```json
{ "version": 1, "answered": 1792060800, "choices": { "performance": true, "crashes": true, "bugs": false, "benchmarks": false }, "install_id": "3f2c…" }
```
- `version` is the consent version the player answered (`CONSENT_VERSION`). It goes up when what is collected changes. The launcher then asks again, and until the player answers nothing new is kept or sent.
- `answered` is when they answered, in seconds since 1970 (UTC).
- `install_id` is 128 random bits, made the first time the player says yes to anything. It ties their items together so they can be deleted, and nothing else. "Delete my data" asks the server to delete everything under it, then replaces it.
Nothing leaves the machine without consent:
- `outbox::push` keeps an item only if its kind is allowed.
- `upload::flush` reads the consent again before sending and deletes any waiting item whose kind the player has turned off since, so with everything off it never opens a connection.
`tests/consent_gating.rs` checks this against a local mock server that counts connections.
## What is sent
Every item has `day` (days since 1970, UTC: no finer time), `source` (`launcher` or `game`), `version` (the release), `kind` and `data`.
| Kind | Switch | `data` | Largest | A day |
|---|---|---|---|---|
| `perf` | performance | `system`, `settings` (render and simulation distance, max FPS, vsync, fullscreen, graphics mode), `seconds` played, `frames` and `ticks` summaries, `peak_ram_mb`, `to_title_ms`, `to_world_ms` | 4 KiB | 12 |
| `crash` | crash reports | `system`, `error`, `stack` (one frame a line), `thread`, the graphics and settings details, all scrubbed. No log, no world, no server | 32 KiB | 5 |
| `bug` | bug reports | `system`, the player's `text` (up to 4000 characters), and the `log` only if they add it after reading it | 80 KiB | 5 |
| `benchmark` | benchmark uploads | the benchmark report as `docs/benchmark-report.md` defines it, as it is, with its own schema field | 128 KiB | 10 |
`system` is the OS and its version, the architecture, the CPU model and thread count, installed memory rounded to the GiB, and the GPU, driver and graphics backend.
A summary (`frames`, `ticks`) is the count, the 50th, 95th and 99th percentile and the maximum in milliseconds (to the tenth), and 14 bucket counts: under 4 ms, 4–6, 6–8, 8–10, 10–12, 12–14, 14–16.7, 16.7–20, 20–25, 25–33.3, 33.3–50, 50–100, 100–250, and 250 ms and more. Raw frame times never leave: `Timings` records them in tenths of a millisecond (20 KB however long the session) and only its summary is kept. A perf item is under 1 KiB.
Never sent: account names or UUIDs, world names, server or LAN addresses, chat, file paths with the user's name, IP addresses. `scrub::Scrubber` replaces the user folder in paths (`/Users/<user>/…`, `C:\Users\<user>\…`), the user name, e-mail and IP addresses and UUIDs. It also replaces the terms the caller passes (the world's name, the server's address, account names). The server does not store the sender's IP address.
## Storage and sending
- `telemetry/outbox/` holds one JSON file per item, `<day>-<kind>-<random>.json`. It never holds more than 200 items or 2 MiB, and the oldest go first. `kept.json` counts today's items per kind against the caps above.
- The launcher sends the outbox when it starts and when it closes. Items go as one JSON batch (`{"schema":1,"install_id":…,"items":[…]}`, up to 64 items and 1 MiB), compressed with zstd: `POST /v1/batch` with `Content-Encoding: zstd`. A day's perf summaries are under 2 KB on the wire.
- After a failure the launcher waits a minute, doubling up to a day, and sends at most 8 batches a day. A batch the server refuses with a 4xx (not 429) is dropped.
- `HORIZON_TELEMETRY_URL` sets the server. There is none by default: until there is, nothing is sent and the outbox stays within its caps.
- `SCHEMA` versions the format. New optional fields keep it. A field that changes meaning or goes bumps it, and the server keeps reading the older ones.
## Server
- `POST /v1/batch`
- `DELETE /v1/installs/<install_id>`: deletes everything sent under that ID.
- `GET /v1/aggregate`: the anonymous totals the website's results page reads.
Retention, rate limits and storage are described with the server.