Merge pull request #1815 from ludovic111/feat/roadmap

🤖 Agent: merged after verify passed (hub: website tests 30/30, roadmap tests + sync check, telemetry server 35 tests; local end-to-end run by roadmap-1).
This commit is contained in:
Ludovic Marie
2026-10-25 16:17:14 +01:00
committed by GitHub
7 changed files with 1655 additions and 2 deletions
+5
View File
@@ -16,6 +16,7 @@
<a href="#download"><b>Download</b></a> ·
<a href="https://playhorizon.dev/docs/"><b>Documentation</b></a> ·
<a href="https://github.com/ludovic111/horizon/releases"><b>Release notes</b></a> ·
<a href="ROADMAP.md"><b>Roadmap</b></a> ·
<a href="#project-status"><b>Project status</b></a> ·
<a href="#contributing"><b>Contributing</b></a>
</p>
@@ -109,6 +110,10 @@ Everything prepared lands in the gitignored `.local/`. It is derived from the ga
The client starts on the title screen. Command-line options (`--world`, `--seed`, `--creative`, `--render-distance`, `--dedicated`), controls, scripting through `horizon-ctl` and the architecture are described in [the native client guide](docs/native-client.md). The rest of the developer documentation is indexed in [docs/README.md](docs/README.md), including [the parity workflow](docs/parity-workflow.md).
## Roadmap
What is being done now, what comes next and where you can help, in plain words and in technical detail: [ROADMAP.md](ROADMAP.md). [playhorizon.dev/hub](https://playhorizon.dev/hub) shows it with the work board, the journal of what shipped and how to help.
## Contributing
Issues and pull requests are welcome: see [CONTRIBUTING.md](CONTRIBUTING.md) for how we work (vanilla first, small pull requests, labels), or [the contributing guide](https://playhorizon.dev/docs/contributing) on the site. Questions and ideas: [Discord](https://discord.gg/HcTeMZxe5E). Before opening a pull request, run the self-check and paste the block it prints:
+341
View File
@@ -0,0 +1,341 @@
# Horizon roadmap
<!-- Generated by tools/roadmap.py from roadmap/roadmap.toml: edit that file, then run `python3 tools/roadmap.py build`. -->
What is being done on Horizon, what comes next and where you can help. There are no dates: things are Now, Next or Later. Each item is said in plain words, with the technical picture folded under it. The same roadmap, with a Simple / Technical switch, the board, the journal and how to help, is on [playhorizon.dev/hub](https://playhorizon.dev/hub). Pull requests carry the `roadmap:<id>` label of their item.
## At a glance
Horizon is Minecraft: Java Edition 26.3 rewritten from scratch in Rust, so it runs faster on the same computer. Single player, LAN, servers, resource packs and Horizon's own mods work today, on your own copy of the game.
**Being done now**
- **Steady frames at long render distances**: Making Horizon faster than vanilla on every computer, with smooth camera movement even at 32 chunks, by spreading the work better over the processor's cores.
- **Modpacks' world generation, natively**: Making the world-generation mods of popular modpacks (new structures, YUNG's dungeons and mineshafts, packs built on Lithostitched) generate their worlds in Horizon, natively and fast.
- **Playing on servers: everything shows as it should**: When you play on a server, other players, mobs, items and menus look and behave as in vanilla: skins, poses, item names, inventories and server networks.
- **Single player as a server: the last differences**: Single player now runs a small server in the background, as vanilla does. We are fixing what still behaves differently there; every fix also works for LAN and server players.
- **Horizon's mod registry**: One place for mods, modpacks and resource packs made for Horizon: browse them on the website and install them from the launcher in one click.
**Next**: Microsoft sign-in (coming soon); CurseForge in the mod browser (coming soon); Fresh benchmarks against vanilla; Pistons push players and mobs; Horizon on Intel Macs.
**How to help**: Play it and tell us what differs from vanilla (Discord or GitHub), run the benchmark on your computer, vote for what you want next, or pick one of the Help wanted items.
**Developers**: Read CONTRIBUTING.md, pick a Help wanted item or a To do card, check vanilla's code for the behaviour, and paste the block `python3 tools/pr_ready.py` prints in your pull request.
## Now
### Performance
#### Steady frames at long render distances
Making Horizon faster than vanilla on every computer, with smooth camera movement even at 32 chunks, by spreading the work better over the processor's cores.
<details><summary>Technical</summary>
Frame pacing at render distance 32 ([#678](https://github.com/ludovic111/horizon/issues/678)): long frames come from work competing with the render thread. In review: world generation workers build free structure starts instead of queueing them ([#1760](https://github.com/ludovic111/horizon/issues/1760)), the worker pool leaves one core as vanilla does ([#1778](https://github.com/ludovic111/horizon/issues/1778)), and the terrain vertex arena is read as storage where flat draws run ([#1742](https://github.com/ludovic111/horizon/issues/1742)). Still open: far terrain at 48 to 64 chunks ([#718](https://github.com/ludovic111/horizon/issues/718)) and CPU-bound fields of block entities ([#717](https://github.com/ludovic111/horizon/issues/717)). Every change shows before and after numbers from `tools/perf_bench.py` against vanilla 26.3 ([docs/benchmark.md](docs/benchmark.md)).
</details>
#### Faster block lookups
Speeding up the code that reads and writes the blocks of the world, which almost everything in the game goes through: generation, lighting, meshing and ticking.
<details><summary>Technical</summary>
`PalettedContainer` and `PackedStorage` find an entry's word by multiply and shift instead of an integer division ([#1790](https://github.com/ludovic111/horizon/issues/1790), [#1794](https://github.com/ludovic111/horizon/issues/1794)); meshing looks up baked models by index, not by name ([#1747](https://github.com/ludovic111/horizon/issues/1747)). Each change keeps the paletted storage harnesses identical and shows its microbenchmark ([docs/performance/paletted-container-access-results.md](docs/performance/paletted-container-access-results.md)).
</details>
### Modpacks
#### Modpacks' world generation, natively
Making the world-generation mods of popular modpacks (new structures, YUNG's dungeons and mineshafts, packs built on Lithostitched) generate their worlds in Horizon, natively and fast.
<details><summary>Technical</summary>
`horizon-port-pack` carries a pack's data-driven content over; structure mods that need code get a native port: YUNG's worldgen as written ([#1746](https://github.com/ludovic111/horizon/issues/1746)), Lithostitched's limited and guaranteed elements with pool and processor modifiers ([#1777](https://github.com/ludovic111/horizon/issues/1777)), its processors as the game's own types ([#1765](https://github.com/ludovic111/horizon/issues/1765)). Proved by the structure blocks and layout harnesses ([#1753](https://github.com/ludovic111/horizon/issues/1753), [#1757](https://github.com/ludovic111/horizon/issues/1757)). Mods whose Java code does more than world generation don't run ([docs/modpacks.md](docs/modpacks.md)).
</details>
### Servers and multiplayer
#### Playing on servers: everything shows as it should
When you play on a server, other players, mobs, items and menus look and behave as in vanilla: skins, poses, item names, inventories and server networks.
<details><summary>Technical</summary>
Client-of-a-server fixes in `game/remote*.rs` and `horizon-protocol`: mobs' aggressive and item-use flags ([#1802](https://github.com/ludovic111/horizon/issues/1802), [#1798](https://github.com/ludovic111/horizon/issues/1798)), ghost items after a remote menu ([#1795](https://github.com/ludovic111/horizon/issues/1795)), skins ([#1652](https://github.com/ludovic111/horizon/issues/1652)), cookies and transfer packets for proxy networks ([#1650](https://github.com/ludovic111/horizon/issues/1650)), firework boosts ([#1671](https://github.com/ludovic111/horizon/issues/1671)), living-entity attributes and simulation distance ([#1731](https://github.com/ludovic111/horizon/issues/1731)), styled item names ([#1702](https://github.com/ludovic111/horizon/issues/1702)), primed TNT between updates ([#1710](https://github.com/ludovic111/horizon/issues/1710)).
</details>
#### Server dialogs and login screens
Servers that show dialogs, such as the register and login screens of AuthMe, will work in Horizon.
<details><summary>Technical</summary>
26.3's dialog packets and model ([#1720](https://github.com/ludovic111/horizon/issues/1720)), the dialog screen with its buttons and actions ([#1734](https://github.com/ludovic111/horizon/issues/1734)), then text and boolean inputs ([#1735](https://github.com/ludovic111/horizon/issues/1735)).
</details>
### Mods and API
#### Horizon's mod registry
One place for mods, modpacks and resource packs made for Horizon: browse them on the website and install them from the launcher in one click.
<details><summary>Technical</summary>
Merged: the registry index (schema 1, `.hpack`, checks before install, [#1634](https://github.com/ludovic111/horizon/issues/1634)), `cargo horizon-mod check` ([#1642](https://github.com/ludovic111/horizon/issues/1642)) and the submission service (`crates/horizon-mod-registry`, [#1659](https://github.com/ludovic111/horizon/issues/1659)). In review: the website's /mods ([#1596](https://github.com/ludovic111/horizon/issues/1596)) and /mods/submit ([#1635](https://github.com/ludovic111/horizon/issues/1635)), and in the launcher the registry as a source ([#1636](https://github.com/ludovic111/horizon/issues/1636)), install, enable and takedowns ([#1637](https://github.com/ludovic111/horizon/issues/1637)), the Mods page ([#1638](https://github.com/ludovic111/horizon/issues/1638)) and modpacks as their own profiles ([#1640](https://github.com/ludovic111/horizon/issues/1640)).
More: [docs/mods.md](docs/mods.md)
</details>
### Gameplay parity
#### Single player as a server: the last differences
Single player now runs a small server in the background, as vanilla does. We are fixing what still behaves differently there; every fix also works for LAN and server players.
<details><summary>Technical</summary>
Since [#1514](https://github.com/ludovic111/horizon/issues/1514) single player runs on the server thread (`game/integrated.rs`) and the window joins it as a client; `--in-frame` keeps the old path. The open `[server-thread]` issues list the gaps. In review: a tool that changes a block ends the click ([#1797](https://github.com/ludovic111/horizon/issues/1797)), potions keep their potion over the connection ([#1796](https://github.com/ludovic111/horizon/issues/1796)), mobs' aggression and arm swings ([#1802](https://github.com/ludovic111/horizon/issues/1802)). [docs/server-levels.md](docs/server-levels.md).
</details>
#### Play in your language
Item names, container titles and server messages follow the language you pick, as in vanilla. Today parts of the game stay in English.
<details><summary>Technical</summary>
Translatable components resolved through the selected language for items, containers and server text ([#1806](https://github.com/ludovic111/horizon/issues/1806)), on top of the bundled languages the prepared resources carry ([#1594](https://github.com/ludovic111/horizon/issues/1594)). Left: the client's own screens ([#1672](https://github.com/ludovic111/horizon/issues/1672)) and the main menu with zh_cn ([#1774](https://github.com/ludovic111/horizon/issues/1774)).
</details>
### Rendering
#### Shader packs, written again from scratch
Shader packs were removed and are being rebuilt from scratch. They come back, off by default, once the best-known packs draw properly.
<details><summary>Technical</summary>
A clean-room rewrite in `horizon-shaderpack`, from the packs' public format documentation only: reading packs, the preprocessor and `.properties` files are merged ([#1584](https://github.com/ludovic111/horizon/issues/1584), [#1585](https://github.com/ludovic111/horizon/issues/1585), [#1586](https://github.com/ludovic111/horizon/issues/1586)); translating programs to WGSL through naga, buffers, uniforms and the terrain, entity, sky, weather and hand stages are in review ([#1698](https://github.com/ludovic111/horizon/issues/1698) to [#1801](https://github.com/ludovic111/horizon/issues/1801)).
More: [#1561](https://github.com/ludovic111/horizon/issues/1561) · [#1771](https://github.com/ludovic111/horizon/issues/1771) · [#1786](https://github.com/ludovic111/horizon/issues/1786)
</details>
### Horizon extras
#### Discord Rich Presence
Your Discord profile can show that you are playing Horizon, with a switch to turn it off.
<details><summary>Technical</summary>
A Discord IPC client on its own thread ([#1787](https://github.com/ludovic111/horizon/issues/1787)) and the presence with its setting ([#1788](https://github.com/ludovic111/horizon/issues/1788)). Merged: it ships with the next release.
</details>
### Code and tools
#### A leaner codebase that builds faster
Cleaning up Horizon's code: removing what nothing uses and making it quicker to build, so fixes and new features land faster.
<details><summary>Technical</summary>
A review of the whole tree: dead code and stubs removed ([#1432](https://github.com/ludovic111/horizon/issues/1432), [#1658](https://github.com/ludovic111/horizon/issues/1658)), one probe binary per crate ([#1453](https://github.com/ludovic111/horizon/issues/1453), [#1454](https://github.com/ludovic111/horizon/issues/1454)), shared harness helpers ([#1448](https://github.com/ludovic111/horizon/issues/1448)), clippy clean on the whole tree ([#1707](https://github.com/ludovic111/horizon/issues/1707)). [docs/architecture.md](docs/architecture.md) maps each feature to its files and harnesses.
</details>
## Next
### Performance
#### Fresh benchmarks against vanilla
New public numbers comparing Horizon with vanilla on many kinds of computers, measured the same way every time, on the website's Benchmarks page.
<details><summary>Technical</summary>
Redo the comparison after the current performance work ([#1073](https://github.com/ludovic111/horizon/issues/1073)); reliability, coverage and presentation ([#1346](https://github.com/ludovic111/horizon/issues/1346)); a scene must count as loaded only once its first chunk arrives ([#1749](https://github.com/ludovic111/horizon/issues/1749)). Players' reports come in through `--benchmark` and `horizon-launcher data benchmark` ([docs/benchmark.md](docs/benchmark.md)).
</details>
### Modpacks
#### CurseForge in the mod browser (coming soon)
Coming soon: search and download CurseForge mods and modpacks from the launcher, as you can with Modrinth today.
<details><summary>Technical</summary>
The mod browser lists CurseForge as coming soon ([docs/mod-browser.md](docs/mod-browser.md)): it needs CurseForge API access. Meanwhile packs import from a CurseForge zip or instance folder, and jars only CurseForge hosts come from the player's own instance ([docs/modpacks.md](docs/modpacks.md)).
</details>
### Gameplay parity
#### Pistons push players and mobs
Pistons and slime blocks will push players, mobs and items as in vanilla. Today moving blocks pass through them.
<details><summary>Technical</summary>
Port `PistonMovingBlockEntity.moveCollidedEntities` and `moveStuckEntities` (honey), the slime bounce and `Entity.limitPistonMovement` for `MoverType.PISTON`, in single player and for a server's players. Issues [#1496](https://github.com/ludovic111/horizon/issues/1496), [#1609](https://github.com/ludovic111/horizon/issues/1609), [#1608](https://github.com/ludovic111/horizon/issues/1608), [#1704](https://github.com/ludovic111/horizon/issues/1704), [#1758](https://github.com/ludovic111/horizon/issues/1758).
</details>
### Launcher
#### Microsoft sign-in (coming soon)
Coming soon: sign in with your Microsoft account to play on online-mode servers. The code is ready; it waits for Microsoft's approval.
<details><summary>Technical</summary>
Device-code sign-in in the launcher and online-mode login in the client ([docs/microsoft-account.md](docs/microsoft-account.md)). It is turned on by setting the app's client id once Minecraft API access is approved; until then the launcher shows "Coming soon".
More: [#1270](https://github.com/ludovic111/horizon/issues/1270)
</details>
#### Horizon on Intel Macs
Builds of the game and the launcher for Macs with an Intel processor.
<details><summary>Technical</summary>
x86_64 game and launcher builds, picked per processor ([#1171](https://github.com/ludovic111/horizon/issues/1171)). Before it ships it needs a run on a real Intel Mac: Rosetta on Apple Silicon only proves the CPU code. Requests: [#1161](https://github.com/ludovic111/horizon/issues/1161), [#1792](https://github.com/ludovic111/horizon/issues/1792).
</details>
## Later
### Gameplay parity
#### Following new Minecraft versions
When Mojang releases a new version of the game, Horizon follows it, and every system is checked against the new version again.
<details><summary>Technical</summary>
The port pipeline detects a release, diffs its classes against 26.3 and writes a port report ([docs/version-ports.md](docs/version-ports.md)); the harnesses then run against the new version.
</details>
### Horizon extras
#### An optional AI teammate
An AI teammate that plays beside you and that you can talk to. It is off unless you turn it on, and the No AI switch hides it completely.
<details><summary>Technical</summary>
Draft pull requests: the companion ([#1068](https://github.com/ludovic111/horizon/issues/1068)), its safety modes and their evaluation ([#1069](https://github.com/ludovic111/horizon/issues/1069)), and a route that finishes the game ([#1070](https://github.com/ludovic111/horizon/issues/1070)). Local models first; it ships only once the safety evaluation passes.
</details>
## Help wanted
### Performance
#### Benchmark your computer
Run Horizon's built-in benchmark and share the report: every new computer makes the performance work more accurate. No coding needed.
<details><summary>Technical</summary>
`horizon-launcher play --arg=--benchmark --arg=standard` writes a JSON report; `horizon-launcher data benchmark <report>` shows what would be sent and uploads it ([website/docs/performance.md](website/docs/performance.md)). Results from three installs with the same hardware appear on the Benchmarks page. Example: [#1711](https://github.com/ludovic111/horizon/issues/1711).
</details>
### Gameplay parity
#### End city banners keep their pattern
Banners in End cities show as plain magenta instead of with their pattern.
<details><summary>Technical</summary>
Compare the banner block entities the End city templates place with vanilla's (`EndCityPieces`, the templates' block entity NBT, `BannerBlockEntity`'s patterns), in `crates/horizon-world/src/worldgen/structure/`. Issue [#1438](https://github.com/ludovic111/horizon/issues/1438); possibly related: [#526](https://github.com/ludovic111/horizon/issues/526).
</details>
#### Enchanted loot in End cities
Armour and tools in End city chests should come enchanted, as in vanilla.
<details><summary>Technical</summary>
`loot.rs` implements `enchant_with_levels`: find why the End city chest table's rolls come out bare, and compare them with vanilla's for a few seeds. Issues [#1437](https://github.com/ludovic111/horizon/issues/1437), [#1660](https://github.com/ludovic111/horizon/issues/1660).
</details>
#### Cooldowns drawn on items
After you throw an ender pearl, its slot should show the cooldown sweeping away, as in vanilla.
<details><summary>Technical</summary>
Vanilla draws `ItemCooldowns`' percent as a white overlay over the slot (`GuiGraphicsExtractor.itemDecorations` and `itemCooldown`) in the hotbar and in inventories. Issue [#1763](https://github.com/ludovic111/horizon/issues/1763).
</details>
#### Chat settings and team prefixes
The Hidden and Commands Only chat settings should hide chat, and team prefixes should appear before names in chat.
<details><summary>Technical</summary>
Vanilla checks `ChatVisiblity` before it delivers chat and system messages ([#1376](https://github.com/ludovic111/horizon/issues/1376)), and chat uses the team-formatted display name ([#1378](https://github.com/ludovic111/horizon/issues/1378)). Both issues have exact steps to reproduce.
</details>
#### A pointing hand over buttons
The mouse pointer should turn into a pointing hand over menu buttons, as in vanilla 26.3.
<details><summary>Technical</summary>
26.3's `AbstractWidget` asks for a cursor from `CursorTypes` when it is hovered; winit's `CursorIcon::Pointer` gives the same shape. Issue [#1347](https://github.com/ludovic111/horizon/issues/1347).
</details>
#### Ctrl+Backspace and scroll bars in menus
Ctrl+Backspace should delete a whole word in the creative search, and every list's scroll bar should be draggable.
<details><summary>Technical</summary>
`EditBox` deletes by word with Ctrl ([#1764](https://github.com/ludovic111/horizon/issues/1764)); the Game Rules and Statistics lists' scroll bars can't be dragged ([#1782](https://github.com/ludovic111/horizon/issues/1782), after [#978](https://github.com/ludovic111/horizon/issues/978)).
</details>
#### Chickens flap while they fall
Chickens should flap their wings while they fall, not stand still in the air.
<details><summary>Technical</summary>
Vanilla's `Chicken.aiStep` updates `flap` and `flapSpeed` when the chicken isn't on the ground, and the model's wings follow them. Issue [#1706](https://github.com/ludovic111/horizon/issues/1706).
</details>
## Recently shipped
- **Server resource packs** (v0.2.48): Servers can send you their resource pack: Horizon asks, downloads it and applies it, like vanilla. [#1562](https://github.com/ludovic111/horizon/issues/1562) · [#1569](https://github.com/ludovic111/horizon/issues/1569) · [#1570](https://github.com/ludovic111/horizon/issues/1570)
- **Single player on its own server thread** (v0.2.39): Single player runs on a server thread, as vanilla does, so the world keeps ticking smoothly while the window draws. [#1514](https://github.com/ludovic111/horizon/issues/1514) · [docs/server-levels.md](docs/server-levels.md)
- **GPU occlusion culling on Macs** (v0.2.17): On Macs the GPU now skips terrain hidden behind hills and caves: 8 to 48% more frames per second at 32 chunks on an M4 Pro. [#1434](https://github.com/ludovic111/horizon/issues/1434)
- **Animated textures on the GPU** (v0.2.5): Water, lava and other animated blocks are now animated by the graphics card: 1.7 to 2.4 times better 1% lows on Intel graphics. [#1239](https://github.com/ludovic111/horizon/issues/1239)
- **Far Detail for integrated graphics** (v0.2.5): An optional Reduced setting for far terrain, suggested on integrated graphics, that saves 2 to 4 ms of GPU time per frame. [#1199](https://github.com/ludovic111/horizon/issues/1199) · [#1029](https://github.com/ludovic111/horizon/issues/1029)
- **The launcher's mod browser** (launcher@0.2.0): Search Modrinth for mods, modpacks and resource packs from the launcher; it says before installing what works in Horizon. [#1154](https://github.com/ludovic111/horizon/issues/1154) · [#1242](https://github.com/ludovic111/horizon/issues/1242) · [#1243](https://github.com/ludovic111/horizon/issues/1243) · [#1246](https://github.com/ludovic111/horizon/issues/1246) · [docs/mod-browser.md](docs/mod-browser.md)
- **A blocky, Minecraft-like launcher** (launcher@0.2.0): The launcher got a new blocky look in Horizon's purple, in dark and light. [#1269](https://github.com/ludovic111/horizon/issues/1269) · [docs/design-system.md](docs/design-system.md)
- **Dynamic lights** (v2026.10.22): An optional extra: torches you hold, drop or set on fire light up what is around them. [#1162](https://github.com/ludovic111/horizon/issues/1162)
- **Host a server from the launcher** (launcher-v0.15.0): The launcher's Servers page creates, starts, backs up and updates a Horizon server with buttons, and invites friends. [#1026](https://github.com/ludovic111/horizon/issues/1026) · [docs/hosting.md](docs/hosting.md)
+57
View File
@@ -0,0 +1,57 @@
# The roadmap
`roadmap.toml` is the one source of the roadmap. `tools/roadmap.py build` turns it into
[ROADMAP.md](../ROADMAP.md); the website's hub (playhorizon.dev/hub) and Kira read the same items. The work
board stays the task-level view (every issue and pull request); the roadmap is the overview, and each pull
request carries the `roadmap:<id>` label of the item it moves.
## An item
```toml
[[item]]
id = "server-dialogs" # lowercase words joined by dashes; never renamed (votes and links use it)
title = "Server dialogs and login screens"
status = "now" # now | next | later | help-wanted | shipped
area = "servers" # one of the [[area]] ids, listed in the order of priority
focus = true # optional: one of the 3 to 5 Now items in the overview
ongoing = true # optional: a workstream that goes on after its pull requests merge
plain = """Servers that show dialogs ... will work in Horizon.""" # players' words, at most 240 characters
dev = """26.3's dialog packets and model (#1720), ...""" # crates, approach, what is left
links = ["#1720", "#1734", "docs/hosting.md"] # #N, a repository path, or https://
shipped_in = "v0.2.58" # shipped items only: a game tag, launcher@x.y.z, or "website"
post = "..." # optional: the announcement when it ships, if `plain` doesn't read as one
announce = true # optional: when it ships, Kira also posts it on X (`post`, else `plain`)
```
- **No dates, ever**: Now, Next, Later. The check rejects month names, years, quarters and "next week".
"Coming soon" is the only phrase for something waiting on someone else (Microsoft sign-in, CurseForge).
- **Only what is decided.** Ideas stay in issues and in the suggestion queue until they are.
- **Only what is true in a release** for shipped items: a merged pull request isn't shipped until a release has it.
- `plain` is for players: what they will be able to do, not how. `dev` is for contributors: the code, the
vanilla classes, the pull requests, what is left. Both stay public: no machines, accounts or private notes.
- Help wanted items are good first contributions: a clear vanilla behaviour, an issue with steps, a place to
start in `dev`.
## Keeping it current
Whoever moves the work (the orchestrator on each heartbeat, or anyone in their pull request):
1. `python3 tools/roadmap.py sync`: every item whose linked pull requests all went out in a release, whose
linked issues are all closed and that isn't `ongoing` becomes `shipped`, with that release in
`shipped_in`, on top of the shipped items. It rewrites `roadmap.toml` and ROADMAP.md; open a pull request
with the change (`skip-release`).
2. `python3 tools/roadmap.py check`: the file's rules, then GitHub's state of every link. It fails when a
link is not an issue or pull request, when a Now item has no open pull request and none merged in the last
14 days (link the new work, or move it to Next or Later), and when every link of a Help wanted item is
closed. It also says which items `sync` will ship once a release has them.
3. When work starts, opens a pull request or is decided: add or move the item, add its links, then
`python3 tools/roadmap.py build`. Keep the overview's focus to the 3 to 5 biggest Now items.
4. `python3 tools/roadmap.py labels` (after the change merges): creates a `roadmap:<id>` label for each new
item, updates their colour (by status) and description, and adds the label to the open pull requests an item
links. Label a new pull request for an item by hand or by linking it and running `labels` again. Labels of
removed items are listed, never deleted.
5. Suggestions from the website and Discord are read and triaged by the maintainers. A good one becomes an
item in a normal pull request; none is published automatically.
CI runs `tests/test_roadmap.py` (in the Python tests), which fails when ROADMAP.md is not what `build` writes
or when the file breaks a rule; `python3 tools/roadmap.py check --offline` runs the same rules without GitHub.
+482
View File
@@ -0,0 +1,482 @@
# Horizon's roadmap: the one source of truth for ROADMAP.md, playhorizon.dev/roadmap and Kira's answers.
# Edit this file, then run `python3 tools/roadmap.py build`. How to keep it current: roadmap/README.md.
#
# No dates, ever: an item is Now, Next or Later. Each item says the same thing twice: `plain` for players
# (one or two sentences, at most 240 characters, so it fits a post when it ships) and `dev` for contributors
# (crates, approach, what is left). Links are "#123" (an issue or a pull request), a path in the repository,
# or an https URL. Only what is decided goes here; ideas wait in GitHub issues and the suggestion queue.
# `focus = true` puts a Now item in the overview (3 to 5 of them). `ongoing = true` marks a workstream that goes on
# after its linked pull requests merge: `sync` never ships it, so link its new pull requests as they open.
[overview]
today_plain = """Horizon is Minecraft: Java Edition 26.3 rewritten from scratch in Rust, so it runs faster on the \
same computer. Single player, LAN, servers, resource packs and Horizon's own mods work today, on your own copy of \
the game."""
today_dev = """A native Rust and wgpu reimplementation of Java 26.3: the same saves, rules and terrain, checked \
against the real game by more than 300 differential harnesses (a Java oracle and a Rust probe on the same inputs). \
Single player runs on its own server thread, like vanilla's integrated server."""
help_plain = """Play it and tell us what differs from vanilla (Discord or GitHub), run the benchmark on your \
computer, vote for what you want next, or pick one of the Help wanted items."""
help_dev = """Read CONTRIBUTING.md, pick a Help wanted item or a To do card, check vanilla's code for the \
behaviour, and paste the block `python3 tools/pr_ready.py` prints in your pull request."""
# Areas, in the project's order of priority: performance first, then modpacks and servers.
[[area]]
id = "performance"
name = "Performance"
[[area]]
id = "modpacks"
name = "Modpacks"
[[area]]
id = "servers"
name = "Servers and multiplayer"
[[area]]
id = "mods"
name = "Mods and API"
[[area]]
id = "gameplay"
name = "Gameplay parity"
[[area]]
id = "rendering"
name = "Rendering"
[[area]]
id = "launcher"
name = "Launcher"
[[area]]
id = "extras"
name = "Horizon extras"
[[area]]
id = "project"
name = "Code and tools"
# ---------------------------------------------------------------------------------------------------- Now
[[item]]
id = "smooth-frames"
title = "Steady frames at long render distances"
status = "now"
area = "performance"
focus = true
plain = """Making Horizon faster than vanilla on every computer, with smooth camera movement even at 32 chunks, \
by spreading the work better over the processor's cores."""
dev = """Frame pacing at render distance 32 (#678): long frames come from work competing with the render thread. \
In review: world generation workers build free structure starts instead of queueing them (#1760), the worker pool \
leaves one core as vanilla does (#1778), and the terrain vertex arena is read as storage where flat draws run \
(#1742). Still open: far terrain at 48 to 64 chunks (#718) and CPU-bound fields of block entities (#717). Every \
change shows before and after numbers from `tools/perf_bench.py` against vanilla 26.3 (docs/benchmark.md)."""
links = ["#678", "#1760", "#1778", "#1742", "#718", "#717", "docs/benchmark.md"]
[[item]]
id = "block-storage"
title = "Faster block lookups"
status = "now"
area = "performance"
plain = """Speeding up the code that reads and writes the blocks of the world, which almost everything in the game \
goes through: generation, lighting, meshing and ticking."""
dev = """`PalettedContainer` and `PackedStorage` find an entry's word by multiply and shift instead of an integer \
division (#1790, #1794); meshing looks up baked models by index, not by name (#1747). Each change keeps the paletted \
storage harnesses identical and shows its microbenchmark (docs/performance/paletted-container-access-results.md)."""
links = ["#1790", "#1794", "#1747", "docs/performance/paletted-container-access-results.md"]
[[item]]
id = "modpack-worldgen"
title = "Modpacks' world generation, natively"
status = "now"
area = "modpacks"
announce = true
focus = true
plain = """Making the world-generation mods of popular modpacks (new structures, YUNG's dungeons and mineshafts, \
packs built on Lithostitched) generate their worlds in Horizon, natively and fast."""
dev = """`horizon-port-pack` carries a pack's data-driven content over; structure mods that need code get a native \
port: YUNG's worldgen as written (#1746), Lithostitched's limited and guaranteed elements with pool and processor \
modifiers (#1777), its processors as the game's own types (#1765). Proved by the structure blocks and layout \
harnesses (#1753, #1757). Mods whose Java code does more than world generation don't run (docs/modpacks.md)."""
links = ["#1746", "#1777", "#1765", "#1753", "#1757", "docs/modpacks.md"]
[[item]]
id = "server-clients"
title = "Playing on servers: everything shows as it should"
status = "now"
ongoing = true
area = "servers"
focus = true
plain = """When you play on a server, other players, mobs, items and menus look and behave as in vanilla: skins, \
poses, item names, inventories and server networks."""
dev = """Client-of-a-server fixes in `game/remote*.rs` and `horizon-protocol`: mobs' aggressive and item-use flags \
(#1802, #1798), ghost items after a remote menu (#1795), skins (#1652), cookies and transfer packets for proxy \
networks (#1650), firework boosts (#1671), living-entity attributes and simulation distance (#1731), styled item \
names (#1702), primed TNT between updates (#1710)."""
links = ["#1802", "#1798", "#1795", "#1652", "#1650", "#1671", "#1731", "#1702", "#1710"]
[[item]]
id = "server-thread"
title = "Single player as a server: the last differences"
status = "now"
ongoing = true
area = "gameplay"
focus = true
plain = """Single player now runs a small server in the background, as vanilla does. We are fixing what still \
behaves differently there; every fix also works for LAN and server players."""
dev = """Since #1514 single player runs on the server thread (`game/integrated.rs`) and the window joins it as a \
client; `--in-frame` keeps the old path. The open `[server-thread]` issues list the gaps. In review: a tool that \
changes a block ends the click (#1797), potions keep their potion over the connection (#1796), mobs' aggression \
and arm swings (#1802). docs/server-levels.md."""
links = ["#1514", "#1797", "#1796", "#1802", "docs/server-levels.md"]
[[item]]
id = "mod-registry"
title = "Horizon's mod registry"
status = "now"
area = "mods"
announce = true
focus = true
plain = """One place for mods, modpacks and resource packs made for Horizon: browse them on the website and \
install them from the launcher in one click."""
dev = """Merged: the registry index (schema 1, `.hpack`, checks before install, #1634), `cargo horizon-mod check` \
(#1642) and the submission service (`crates/horizon-mod-registry`, #1659). In review: the website's /mods (#1596) \
and /mods/submit (#1635), and in the launcher the registry as a source (#1636), install, enable and takedowns \
(#1637), the Mods page (#1638) and modpacks as their own profiles (#1640)."""
links = ["#1634", "#1642", "#1659", "#1596", "#1635", "#1636", "#1637", "#1638", "#1640", "docs/mods.md"]
[[item]]
id = "server-dialogs"
title = "Server dialogs and login screens"
status = "now"
area = "servers"
plain = """Servers that show dialogs, such as the register and login screens of AuthMe, will work in Horizon."""
dev = """26.3's dialog packets and model (#1720), the dialog screen with its buttons and actions (#1734), then text \
and boolean inputs (#1735)."""
links = ["#1720", "#1734", "#1735"]
[[item]]
id = "languages"
title = "Play in your language"
status = "now"
area = "gameplay"
plain = """Item names, container titles and server messages follow the language you pick, as in vanilla. Today \
parts of the game stay in English."""
dev = """Translatable components resolved through the selected language for items, containers and server text \
(#1806), on top of the bundled languages the prepared resources carry (#1594). Left: the client's own screens \
(#1672) and the main menu with zh_cn (#1774)."""
links = ["#1806", "#1594", "#1672", "#1774"]
[[item]]
id = "shader-packs"
title = "Shader packs, written again from scratch"
status = "now"
ongoing = true
area = "rendering"
plain = """Shader packs were removed and are being rebuilt from scratch. They come back, off by default, once the \
best-known packs draw properly."""
dev = """A clean-room rewrite in `horizon-shaderpack`, from the packs' public format documentation only: reading \
packs, the preprocessor and `.properties` files are merged (#1584, #1585, #1586); translating programs to WGSL \
through naga, buffers, uniforms and the terrain, entity, sky, weather and hand stages are in review (#1698 to \
#1801)."""
links = ["#1561", "#1584", "#1586", "#1698", "#1771", "#1786", "#1801"]
[[item]]
id = "discord-presence"
title = "Discord Rich Presence"
status = "now"
area = "extras"
announce = true
plain = """Your Discord profile can show that you are playing Horizon, with a switch to turn it off."""
dev = """A Discord IPC client on its own thread (#1787) and the presence with its setting (#1788). Merged: it ships \
with the next release."""
links = ["#1787", "#1788"]
[[item]]
id = "lean-code"
title = "A leaner codebase that builds faster"
status = "now"
ongoing = true
area = "project"
plain = """Cleaning up Horizon's code: removing what nothing uses and making it quicker to build, so fixes and new \
features land faster."""
dev = """A review of the whole tree: dead code and stubs removed (#1432, #1658), one probe binary per crate (#1453, \
#1454), shared harness helpers (#1448), clippy clean on the whole tree (#1707). docs/architecture.md maps each \
feature to its files and harnesses."""
links = ["#1432", "#1658", "#1453", "#1454", "#1448", "#1707", "docs/architecture.md"]
# ---------------------------------------------------------------------------------------------------- Next
[[item]]
id = "microsoft-sign-in"
title = "Microsoft sign-in (coming soon)"
status = "next"
area = "launcher"
announce = true
plain = """Coming soon: sign in with your Microsoft account to play on online-mode servers. The code is ready; it \
waits for Microsoft's approval."""
dev = """Device-code sign-in in the launcher and online-mode login in the client (docs/microsoft-account.md). It is \
turned on by setting the app's client id once Minecraft API access is approved; until then the launcher shows \
"Coming soon"."""
links = ["docs/microsoft-account.md", "#1270"]
[[item]]
id = "curseforge"
title = "CurseForge in the mod browser (coming soon)"
status = "next"
area = "modpacks"
announce = true
plain = """Coming soon: search and download CurseForge mods and modpacks from the launcher, as you can with \
Modrinth today."""
dev = """The mod browser lists CurseForge as coming soon (docs/mod-browser.md): it needs CurseForge API access. \
Meanwhile packs import from a CurseForge zip or instance folder, and jars only CurseForge hosts come from the \
player's own instance (docs/modpacks.md)."""
links = ["docs/mod-browser.md", "docs/modpacks.md"]
[[item]]
id = "benchmarks"
title = "Fresh benchmarks against vanilla"
status = "next"
area = "performance"
plain = """New public numbers comparing Horizon with vanilla on many kinds of computers, measured the same way \
every time, on the website's Benchmarks page."""
dev = """Redo the comparison after the current performance work (#1073); reliability, coverage and presentation \
(#1346); a scene must count as loaded only once its first chunk arrives (#1749). Players' reports come in through \
`--benchmark` and `horizon-launcher data benchmark` (docs/benchmark.md)."""
links = ["#1073", "#1346", "#1749", "docs/benchmark.md"]
[[item]]
id = "pistons-push"
title = "Pistons push players and mobs"
status = "next"
area = "gameplay"
plain = """Pistons and slime blocks will push players, mobs and items as in vanilla. Today moving blocks pass \
through them."""
dev = """Port `PistonMovingBlockEntity.moveCollidedEntities` and `moveStuckEntities` (honey), the slime bounce and \
`Entity.limitPistonMovement` for `MoverType.PISTON`, in single player and for a server's players. Issues #1496, \
#1609, #1608, #1704, #1758."""
links = ["#1496", "#1609", "#1608", "#1704", "#1758"]
[[item]]
id = "intel-mac"
title = "Horizon on Intel Macs"
status = "next"
area = "launcher"
plain = """Builds of the game and the launcher for Macs with an Intel processor."""
dev = """x86_64 game and launcher builds, picked per processor (#1171). Before it ships it needs a run on a real \
Intel Mac: Rosetta on Apple Silicon only proves the CPU code. Requests: #1161, #1792."""
links = ["#1171", "#1161", "#1792"]
# ---------------------------------------------------------------------------------------------------- Later
[[item]]
id = "ai-companion"
title = "An optional AI teammate"
status = "later"
area = "extras"
plain = """An AI teammate that plays beside you and that you can talk to. It is off unless you turn it on, and the \
No AI switch hides it completely."""
dev = """Draft pull requests: the companion (#1068), its safety modes and their evaluation (#1069), and a route \
that finishes the game (#1070). Local models first; it ships only once the safety evaluation passes."""
links = ["#1068", "#1069", "#1070"]
[[item]]
id = "new-versions"
title = "Following new Minecraft versions"
status = "later"
area = "gameplay"
plain = """When Mojang releases a new version of the game, Horizon follows it, and every system is checked against \
the new version again."""
dev = """The port pipeline detects a release, diffs its classes against 26.3 and writes a port report \
(docs/version-ports.md); the harnesses then run against the new version."""
links = ["docs/version-ports.md"]
# ---------------------------------------------------------------------------------------------------- Help wanted
[[item]]
id = "benchmark-your-pc"
title = "Benchmark your computer"
status = "help-wanted"
area = "performance"
plain = """Run Horizon's built-in benchmark and share the report: every new computer makes the performance work \
more accurate. No coding needed."""
dev = """`horizon-launcher play --arg=--benchmark --arg=standard` writes a JSON report; `horizon-launcher data \
benchmark <report>` shows what would be sent and uploads it (website/docs/performance.md). Results from three \
installs with the same hardware appear on the Benchmarks page. Example: #1711."""
links = ["website/docs/performance.md", "#1711"]
[[item]]
id = "end-city-banners"
title = "End city banners keep their pattern"
status = "help-wanted"
area = "gameplay"
plain = """Banners in End cities show as plain magenta instead of with their pattern."""
dev = """Compare the banner block entities the End city templates place with vanilla's (`EndCityPieces`, the \
templates' block entity NBT, `BannerBlockEntity`'s patterns), in `crates/horizon-world/src/worldgen/structure/`. \
Issue #1438; possibly related: #526."""
links = ["#1438", "#526"]
[[item]]
id = "end-city-loot"
title = "Enchanted loot in End cities"
status = "help-wanted"
area = "gameplay"
plain = """Armour and tools in End city chests should come enchanted, as in vanilla."""
dev = """`loot.rs` implements `enchant_with_levels`: find why the End city chest table's rolls come out bare, and \
compare them with vanilla's for a few seeds. Issues #1437, #1660."""
links = ["#1437", "#1660"]
[[item]]
id = "item-cooldowns"
title = "Cooldowns drawn on items"
status = "help-wanted"
area = "gameplay"
plain = """After you throw an ender pearl, its slot should show the cooldown sweeping away, as in vanilla."""
dev = """Vanilla draws `ItemCooldowns`' percent as a white overlay over the slot \
(`GuiGraphicsExtractor.itemDecorations` and `itemCooldown`) in the hotbar and in inventories. Issue #1763."""
links = ["#1763"]
[[item]]
id = "chat-settings"
title = "Chat settings and team prefixes"
status = "help-wanted"
area = "gameplay"
plain = """The Hidden and Commands Only chat settings should hide chat, and team prefixes should appear before \
names in chat."""
dev = """Vanilla checks `ChatVisiblity` before it delivers chat and system messages (#1376), and chat uses the \
team-formatted display name (#1378). Both issues have exact steps to reproduce."""
links = ["#1376", "#1378"]
[[item]]
id = "pointer-hand"
title = "A pointing hand over buttons"
status = "help-wanted"
area = "gameplay"
plain = """The mouse pointer should turn into a pointing hand over menu buttons, as in vanilla 26.3."""
dev = """26.3's `AbstractWidget` asks for a cursor from `CursorTypes` when it is hovered; winit's \
`CursorIcon::Pointer` gives the same shape. Issue #1347."""
links = ["#1347"]
[[item]]
id = "text-field-keys"
title = "Ctrl+Backspace and scroll bars in menus"
status = "help-wanted"
area = "gameplay"
plain = """Ctrl+Backspace should delete a whole word in the creative search, and every list's scroll bar should be \
draggable."""
dev = """`EditBox` deletes by word with Ctrl (#1764); the Game Rules and Statistics lists' scroll bars can't be \
dragged (#1782, after #978)."""
links = ["#1764", "#1782"]
[[item]]
id = "chicken-flap"
title = "Chickens flap while they fall"
status = "help-wanted"
area = "gameplay"
plain = """Chickens should flap their wings while they fall, not stand still in the air."""
dev = """Vanilla's `Chicken.aiStep` updates `flap` and `flapSpeed` when the chicken isn't on the ground, and the \
model's wings follow them. Issue #1706."""
links = ["#1706"]
# ---------------------------------------------------------------------------------------------------- Shipped
# Newest first: `tools/roadmap.py sync` adds each newly shipped item on top.
[[item]]
id = "server-resource-packs"
title = "Server resource packs"
status = "shipped"
shipped_in = "v0.2.48"
area = "servers"
plain = """Servers can send you their resource pack: Horizon asks, downloads it and applies it, like vanilla."""
dev = """The dedicated server offers server.properties' pack (#1562); the client's pack manager, downloads, prompt \
and Server Resource Packs setting (#1569, #1570)."""
links = ["#1562", "#1569", "#1570"]
[[item]]
id = "server-thread-default"
title = "Single player on its own server thread"
status = "shipped"
shipped_in = "v0.2.39"
area = "gameplay"
plain = """Single player runs on a server thread, as vanilla does, so the world keeps ticking smoothly while the \
window draws."""
dev = """`game/integrated.rs` runs a second `Game` with one level per dimension; the window joins it through \
memory like a LAN guest (#1514)."""
links = ["#1514", "docs/server-levels.md"]
[[item]]
id = "metal-occlusion"
title = "GPU occlusion culling on Macs"
status = "shipped"
shipped_in = "v0.2.17"
area = "performance"
plain = """On Macs the GPU now skips terrain hidden behind hills and caves: 8 to 48% more frames per second at 32 \
chunks on an M4 Pro."""
dev = """GPU occlusion culling on Metal (#1434)."""
links = ["#1434"]
[[item]]
id = "gpu-animated-textures"
title = "Animated textures on the GPU"
status = "shipped"
shipped_in = "v0.2.5"
area = "performance"
plain = """Water, lava and other animated blocks are now animated by the graphics card: 1.7 to 2.4 times better 1% \
lows on Intel graphics."""
dev = """Frames uploaded once, no per-tick CPU blend or copy (#1239)."""
links = ["#1239"]
[[item]]
id = "far-detail"
title = "Far Detail for integrated graphics"
status = "shipped"
shipped_in = "v0.2.5"
area = "performance"
plain = """An optional Reduced setting for far terrain, suggested on integrated graphics, that saves 2 to 4 ms of GPU \
time per frame."""
dev = """Opt-in reduced far terrain (#1199), on far-terrain chunk summaries built on the workers (#1029)."""
links = ["#1199", "#1029"]
[[item]]
id = "mod-browser"
title = "The launcher's mod browser"
status = "shipped"
shipped_in = "launcher@0.2.0"
area = "modpacks"
plain = """Search Modrinth for mods, modpacks and resource packs from the launcher; it says before installing what \
works in Horizon."""
dev = """The Mods page (#1154), Java mods checked before install (#1242), project pages, filters and galleries \
(#1243), modpacks ported in the same step (#1246). docs/mod-browser.md."""
links = ["#1154", "#1242", "#1243", "#1246", "docs/mod-browser.md"]
[[item]]
id = "launcher-look"
title = "A blocky, Minecraft-like launcher"
status = "shipped"
shipped_in = "launcher@0.2.0"
area = "launcher"
plain = """The launcher got a new blocky look in Horizon's purple, in dark and light."""
dev = """The Blocks look (#1269), from `horizon-theme` (docs/design-system.md)."""
links = ["#1269", "docs/design-system.md"]
[[item]]
id = "dynamic-lights"
title = "Dynamic lights"
status = "shipped"
shipped_in = "v2026.10.22"
area = "extras"
plain = """An optional extra: torches you hold, drop or set on fire light up what is around them."""
dev = """Held, dropped and burning light sources (#1162), in Horizon Extras, off by default."""
links = ["#1162"]
[[item]]
id = "servers-page"
title = "Host a server from the launcher"
status = "shipped"
shipped_in = "launcher-v0.15.0"
area = "servers"
plain = """The launcher's Servers page creates, starts, backs up and updates a Horizon server with buttons, and \
invites friends."""
dev = """The launcher's Servers page (#1026) over `horizon-server` (docs/hosting.md)."""
links = ["#1026", "docs/hosting.md"]
+211
View File
@@ -0,0 +1,211 @@
import copy
import datetime
import tomllib
import unittest
from unittest import mock
from tools import roadmap
SAMPLE = '''[overview]
today_plain = "a"
today_dev = "b"
help_plain = "c"
help_dev = "d"
[[area]]
id = "performance"
name = "Performance"
# ---- Now
[[item]]
id = "fast"
title = "Fast"
status = "now"
area = "performance"
focus = true
plain = "Faster."
dev = """Multiply, \\
not divide (#10)."""
links = ["#10"]
[[item]]
id = "faster"
title = "Faster"
status = "now"
area = "performance"
focus = true
plain = "Even faster."
dev = "More (#11)."
links = ["#11", "#12"]
[[item]]
id = "fastest"
title = "Fastest"
status = "now"
area = "performance"
focus = true
ongoing = true
plain = "The fastest."
dev = "Most (#13)."
links = ["#13"]
# ---- Shipped
[[item]]
id = "old"
title = "Old"
status = "shipped"
shipped_in = "v0.2.1"
area = "performance"
plain = "Done."
dev = "Done (#1)."
links = ["#1"]
'''
def pull(state, merged_at=None, paths=("crates/horizon-client/src/game.rs",)):
return {"kind": "PullRequest", "state": state, "merged_at": merged_at, "merge_commit": "abc" if merged_at else None,
"paths": list(paths)}
def issue(state):
return {"kind": "Issue", "state": state, "merged_at": None, "merge_commit": None, "paths": []}
class RepositoryRoadmap(unittest.TestCase):
def test_the_roadmap_keeps_its_rules(self):
self.assertEqual(roadmap.validate(roadmap.load()), [])
def test_roadmap_md_is_generated_from_the_toml(self):
self.assertEqual(roadmap.OUTPUT.read_text(), roadmap.render(roadmap.load()),
"ROADMAP.md is out of date: run `python3 tools/roadmap.py build`")
class Rules(unittest.TestCase):
def setUp(self):
self.data = tomllib.loads(SAMPLE)
def problems(self, **changes):
data = copy.deepcopy(self.data)
data["item"][0].update(changes)
return roadmap.validate(data)
def test_the_sample_is_valid(self):
self.assertEqual(roadmap.validate(self.data), [])
def test_unknown_status_area_and_key(self):
self.assertTrue(any("status" in p for p in self.problems(status="soon")))
self.assertTrue(any("area" in p for p in self.problems(area="nowhere")))
self.assertTrue(any("unknown key" in p for p in self.problems(eta="soon")))
def test_plain_fits_a_post(self):
self.assertTrue(any("at most 240" in p for p in self.problems(plain="x" * 241)))
self.assertTrue(any("post" in p for p in self.problems(post="x" * 241)))
def test_shipped_needs_its_release(self):
self.assertTrue(any("shipped_in" in p for p in self.problems(status="shipped", focus=False)))
self.assertTrue(any("only shipped" in p for p in self.problems(shipped_in="v0.2.3")))
self.assertTrue(any("not a release" in p for p in self.problems(status="shipped", focus=False,
shipped_in="soon")))
def test_links_are_issues_urls_or_repository_paths(self):
self.assertEqual(self.problems(links=["#4", "https://playhorizon.dev", "docs/benchmark.md"]), [])
self.assertTrue(any("link" in p for p in self.problems(links=["docs/nothing-here.md"])))
self.assertTrue(any("link" in p for p in self.problems(links=["http://example.com"])))
def test_flags_are_true_or_false(self):
self.assertEqual(self.problems(announce=True), [])
self.assertTrue(any("announce is true or false" in p for p in self.problems(announce="yes")))
def test_three_to_five_items_in_focus(self):
self.assertTrue(any("focus" in p for p in self.problems(focus=False)))
def test_no_dates(self):
for text in ("Ready in March.", "Before 2027.", "In Q3.", "Next week.", "In 3 weeks.", "By the end of it."):
self.assertTrue(roadmap.dated(text), text)
for text in ("Coming soon.", "Pull request #2026.", "Java 26.3.", "Mobs march on.", "It may land."):
self.assertIsNone(roadmap.dated(text), text)
self.assertTrue(any("date" in p for p in self.problems(plain="Done in October.")))
class Render(unittest.TestCase):
def test_sections_links_and_the_overview(self):
text = roadmap.render(tomllib.loads(SAMPLE))
self.assertIn("## Now", text)
self.assertIn("## Recently shipped", text)
self.assertNotIn("## Later", text)
self.assertIn("- **Fast**: Faster.", text)
self.assertIn("Multiply, not divide ([#10](https://github.com/ludovic111/horizon/issues/10)).", text)
# A link the technical text already gives isn't repeated; the others are listed after it.
self.assertIn("More: [#12](https://github.com/ludovic111/horizon/issues/12)", text)
self.assertIn("- **Old** (v0.2.1): Done.", text)
class GitHubState(unittest.TestCase):
now = datetime.datetime(2030, 1, 20, tzinfo=datetime.timezone.utc)
def test_a_now_item_without_live_pull_requests_is_stale(self):
data = tomllib.loads(SAMPLE)
states = {1: pull("MERGED", "2029-12-01T00:00:00Z"), 10: pull("OPEN"),
11: pull("MERGED", "2029-11-01T00:00:00Z"), 12: issue("OPEN"),
13: pull("MERGED", "2030-01-19T00:00:00Z")}
problems, notes = roadmap.online_problems(data, states, self.now)
self.assertEqual(len(problems), 1)
self.assertIn("faster: stale", problems[0])
self.assertTrue(any("fastest: ongoing" in note for note in notes))
def test_missing_numbers_are_reported(self):
problems, _ = roadmap.online_problems(tomllib.loads(SAMPLE), {}, self.now)
self.assertTrue(any("#10 is neither" in p for p in problems))
class Labels(unittest.TestCase):
def test_creates_updates_and_adds_labels(self):
data = tomllib.loads(SAMPLE)
existing = {"roadmap:fast": {"color": "5B3FD6", "description": "Roadmap, now: Fast"},
"roadmap:faster": {"color": "000000", "description": "old"},
"roadmap:gone": {"color": "000000", "description": ""}}
states = {10: dict(pull("OPEN"), labels=["roadmap:fast"]), 11: dict(pull("OPEN"), labels=[]),
13: dict(pull("MERGED", "2030-01-01T00:00:00Z"), labels=[])}
changes, stale = roadmap.label_changes(data, existing, states)
self.assertEqual([(action, target) for action, target, _ in changes],
[("update", "roadmap:faster"), ("add", 11), ("create", "roadmap:fastest"),
("create", "roadmap:old")])
self.assertEqual(changes[0][2]["color"], "5b3fd6")
self.assertEqual(stale, ["roadmap:gone"])
class Sync(unittest.TestCase):
def test_family_of_a_change(self):
self.assertEqual(roadmap.family(["launcher/src/main.rs"]), "launcher")
self.assertEqual(roadmap.family(["website/src/main.js"]), "website")
self.assertEqual(roadmap.family(["launcher/src/main.rs", "crates/x/src/lib.rs"]), "game")
def test_merged_and_released_items_move_to_the_top_of_shipped(self):
data = tomllib.loads(SAMPLE)
states = {10: pull("MERGED", "2030-01-02T00:00:00Z"), 11: pull("MERGED", "2030-01-03T00:00:00Z"),
12: issue("OPEN"), 13: pull("MERGED", "2030-01-03T00:00:00Z")}
with mock.patch.object(roadmap, "first_release", return_value="v0.2.9") as release:
text, log = roadmap.sync(data, SAMPLE, states)
release.assert_called_once_with("abc", "game")
# "faster" keeps an open issue and "fastest" is ongoing: only "fast" ships.
self.assertEqual(log, ["fast: shipped in v0.2.9"])
new = tomllib.loads(text)
self.assertEqual([item["id"] for item in new["item"]], ["faster", "fastest", "fast", "old"])
fast = new["item"][2]
self.assertEqual((fast["status"], fast["shipped_in"]), ("shipped", "v0.2.9"))
self.assertNotIn("focus", fast)
self.assertEqual(fast["dev"], "Multiply, not divide (#10).")
self.assertIn("# ---- Shipped\n\n[[item]]\nid = \"fast\"", text)
def test_merged_but_unreleased_items_wait(self):
states = {10: pull("MERGED", "2030-01-02T00:00:00Z")}
with mock.patch.object(roadmap, "first_release", return_value=None):
text, log = roadmap.sync(tomllib.loads(SAMPLE), SAMPLE, states)
self.assertEqual(text, SAMPLE)
self.assertEqual(log, ["fast: merged, waiting for a release"])
if __name__ == "__main__":
unittest.main()
+4 -2
View File
@@ -108,7 +108,7 @@ def touched_crates(paths, packages):
def area(path):
"""The part of the project a path belongs to; None for what goes with any change (docs, tests, harnesses)."""
if is_doc(path) or path.startswith(("tests/", "mods/", "docs/")) or path == "research-lock.json" or \
if is_doc(path) or path.startswith(("tests/", "mods/", "docs/", "roadmap/")) or path == "research-lock.json" or \
(path.startswith("tools/") and not path.startswith("tools/ci/")):
return None
if path.startswith(("launcher/", "crates/horizon-prepare/")):
@@ -288,7 +288,9 @@ def plan(args, base, changed, crates_of, libraries, dependencies):
"""The checks for these changed files, in the order they run."""
rust = any(workspace_rust(path) for path in changed)
launcher = any(launcher_rust(path) for path in changed)
python = any(path.endswith(".py") or path.startswith(("tests/", "tools/")) for path in changed)
# The Python tests also hold ROADMAP.md to roadmap/roadmap.toml (tests/test_roadmap.py).
python = any(path.endswith(".py") or path.startswith(("tests/", "tools/", "roadmap/")) or path == "ROADMAP.md"
for path in changed)
crates = touched_crates(changed, crates_of) if rust or TOKENS in changed else []
# An unrecognised Rust path must not silently skip validation.
if rust and crates == []:
+555
View File
@@ -0,0 +1,555 @@
#!/usr/bin/env python3
"""Horizon's roadmap: roadmap/roadmap.toml is the source, ROADMAP.md is generated from it.
python3 tools/roadmap.py build write ROADMAP.md
python3 tools/roadmap.py build --check fail when ROADMAP.md is out of date (CI)
python3 tools/roadmap.py check --offline the file's own rules (CI)
python3 tools/roadmap.py check the same, then GitHub: links resolve, no stale Now item
python3 tools/roadmap.py sync [--dry-run] items whose pull requests all merged become shipped,
with the release they went out in
python3 tools/roadmap.py labels [--dry-run] a `roadmap:<id>` label per item on GitHub, and on the open
pull requests the item links
The online commands read GitHub with GITHUB_TOKEN, GH_TOKEN or `gh auth token`. Maintenance: roadmap/README.md.
"""
import argparse
import datetime
import json
import os
import re
import subprocess
import sys
import tomllib
import urllib.parse
import urllib.request
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
SOURCE = ROOT / "roadmap" / "roadmap.toml"
OUTPUT = ROOT / "ROADMAP.md"
REPO = "ludovic111/horizon"
SITE = "https://playhorizon.dev/hub"
STATUSES = {
"now": "Now",
"next": "Next",
"later": "Later",
"help-wanted": "Help wanted",
"shipped": "Recently shipped",
}
PLAIN_MAX = 240 # one post's worth, so a shipped item can be announced as written
TITLE_MAX = 70
FOCUS_RANGE = (3, 5)
STALE_DAYS = 14 # a Now item needs an open pull request, or one merged this recently
LABEL_PREFIX = "roadmap:"
# GitHub label colours by status, from the design system's palette.
LABEL_COLOURS = {"now": "5b3fd6", "next": "9a86ff", "later": "c5c7d1", "help-wanted": "1d7a48", "shipped": "646875"}
ITEM_KEYS = {"id", "title", "status", "area", "plain", "dev", "links", "focus", "ongoing", "shipped_in", "post",
"announce"}
ISSUE = re.compile(r"#(\d+)$")
ID = re.compile(r"[a-z0-9]+(-[a-z0-9]+)*$")
RELEASE = re.compile(r"(v\d[\w.]*|launcher@\d[\w.]*|launcher-v\d[\w.]*|website)$")
MONTHS = ("January|February|March|April|June|July|August|September|October|November|December")
# Kira's rule: no dates and no promises of when. "Coming soon" is the one allowed phrase.
DATE = re.compile(
rf"\b(?:{MONTHS})\b|\bMay \d|(?<![#\w.])20\d\d(?!\w|\.\d)|\bQ[1-4]\b|\btomorrow\b|\btonight\b"
r"|\b(?:next|this) (?:week|month|year)\b|\bin \d+ (?:days?|weeks?|months?)\b|\bby the end of\b",
re.IGNORECASE)
MONTH_CASE = re.compile(rf"\b(?:{MONTHS})\b")
def load(path=SOURCE):
with open(path, "rb") as source:
return tomllib.load(source)
def link_url(link):
"""Where a link points: "#12" is an issue or a pull request (GitHub redirects one to the other)."""
match = ISSUE.match(link)
if match:
return f"https://github.com/{REPO}/issues/{match[1]}"
return link
def dated(text):
"""The first date or deadline in the text, or None. A month counts only capitalised ("march" is a verb)."""
for match in DATE.finditer(text):
if match[0].lower() not in MONTHS.lower().split("|") or MONTH_CASE.fullmatch(match[0]):
return match[0]
return None
def validate(data, root=ROOT):
"""The file's own rules, without the network. Returns a list of problems."""
problems = []
overview = data.get("overview", {})
for key in ("today_plain", "today_dev", "help_plain", "help_dev"):
if not str(overview.get(key, "")).strip():
problems.append(f"overview: {key} is missing")
areas = [area.get("id") for area in data.get("area", [])]
if len(set(areas)) != len(areas):
problems.append("areas: an id appears twice")
seen = set()
items = data.get("item", [])
for index, item in enumerate(items):
where = f"item {item.get('id') or index + 1}"
for key in sorted(set(item) - ITEM_KEYS):
problems.append(f"{where}: unknown key {key!r}")
for key in ("id", "title", "status", "area", "plain", "dev"):
if not str(item.get(key, "")).strip():
problems.append(f"{where}: {key} is missing")
ident = item.get("id", "")
if ident and not ID.match(ident):
problems.append(f"{where}: ids are lowercase words joined by dashes")
if ident in seen:
problems.append(f"{where}: the id appears twice")
seen.add(ident)
status = item.get("status")
if status not in STATUSES:
problems.append(f"{where}: status {status!r} is not one of {', '.join(STATUSES)}")
if item.get("area") not in areas:
problems.append(f"{where}: area {item.get('area')!r} is not declared in an [[area]]")
if len(item.get("title", "")) > TITLE_MAX:
problems.append(f"{where}: the title is longer than {TITLE_MAX} characters")
for key in ("plain", "post"):
if len(item.get(key, "")) > PLAIN_MAX:
problems.append(f"{where}: {key} is {len(item[key])} characters, at most {PLAIN_MAX}")
for key in ("title", "plain", "dev", "post"):
found = dated(item.get(key, ""))
if found:
problems.append(f"{where}: {key} names a date or a deadline ({found!r}); the roadmap has none")
shipped_in = item.get("shipped_in")
if status == "shipped":
if not shipped_in:
problems.append(f"{where}: a shipped item needs shipped_in (a release tag, or \"website\")")
elif not RELEASE.match(shipped_in):
problems.append(f"{where}: shipped_in {shipped_in!r} is not a release tag or \"website\"")
elif shipped_in:
problems.append(f"{where}: only shipped items have shipped_in")
for key in ("focus", "ongoing", "announce"):
if key in item and not isinstance(item[key], bool):
problems.append(f"{where}: {key} is true or false")
if item.get("ongoing") and status not in ("now", "next", "later"):
problems.append(f"{where}: only Now, Next and Later items are ongoing")
if item.get("focus") and status != "now":
problems.append(f"{where}: only Now items can be in focus")
links = item.get("links", [])
if not isinstance(links, list):
problems.append(f"{where}: links is a list")
links = []
if not links and status != "later":
problems.append(f"{where}: needs at least one link (an issue, a pull request or a doc)")
for link in links:
if ISSUE.match(link) or link.startswith("https://"):
continue
if link.startswith(("/", "http:")) or ".." in link or not (root / link).exists():
problems.append(f"{where}: link {link!r} is neither #N, https:// nor a path in the repository")
focus = sum(1 for item in items if item.get("focus"))
low, high = FOCUS_RANGE
if not low <= focus <= high:
problems.append(f"focus: {focus} items are in focus, the overview wants {low} to {high}")
return problems
def areas_in_order(data):
return [(area["id"], area["name"]) for area in data.get("area", [])]
def by_area(data, status):
"""The items of a status, grouped by area in the areas' order, each group in the file's order."""
items = [item for item in data.get("item", []) if item.get("status") == status]
return [(name, [item for item in items if item["area"] == key])
for key, name in areas_in_order(data) if any(item["area"] == key for item in items)]
def linkify(text, links=()):
"""Markdown links for the text's #N, and for the item's own paths and URLs where the text names them."""
text = re.sub(r"(?<![\w\[/])#(\d+)\b", lambda m: f"[#{m[1]}]({link_url('#' + m[1])})", text)
for link in links:
if not ISSUE.match(link):
text = re.sub(rf"(?<![\w/\[])(?<!\]\(){re.escape(link)}(?![\w/])", f"[{link}]({link})", text, count=1)
return text
def link_list(links):
return " · ".join(f"[{link}]({link_url(link)})" for link in links)
def render(data):
"""ROADMAP.md, from the parsed TOML."""
overview = data["overview"]
items = data.get("item", [])
out = [
"# Horizon roadmap",
"",
"<!-- Generated by tools/roadmap.py from roadmap/roadmap.toml: edit that file, then run"
" `python3 tools/roadmap.py build`. -->",
"",
"What is being done on Horizon, what comes next and where you can help. There are no dates: things are"
" Now, Next or Later. Each item is said in plain words, with the technical picture folded under it."
f" The same roadmap, with a Simple / Technical switch, the board, the journal and how to help, is on"
f" [playhorizon.dev/hub]({SITE}). Pull requests carry the `roadmap:<id>` label of their item.",
"",
"## At a glance",
"",
overview["today_plain"].strip(),
"",
"**Being done now**",
"",
]
out += [f"- **{item['title']}**: {item['plain'].strip()}" for item in items if item.get("focus")]
nexts = [item["title"] for item in items if item.get("status") == "next"]
if nexts:
out += ["", "**Next**: " + "; ".join(nexts) + "."]
out += ["", f"**How to help**: {overview['help_plain'].strip()}", "",
f"**Developers**: {overview['help_dev'].strip()}"]
for status, heading in STATUSES.items():
groups = by_area(data, status)
if not groups:
continue
out += ["", f"## {heading}"]
if status == "shipped":
# Newest first: the file's order, where `sync` puts each newly shipped item on top.
out += [""] + [f"- **{item['title']}** ({item['shipped_in']}): {item['plain'].strip()} "
f"{link_list(item['links'])}" for item in items if item.get("status") == status]
continue
for area, group in groups:
out += ["", f"### {area}"]
for item in group:
out += ["", f"#### {item['title']}", "", item["plain"].strip(), "",
"<details><summary>Technical</summary>", "", linkify(item["dev"].strip(), item.get("links", []))]
more = [link for link in item.get("links", []) if not re.search(rf"{re.escape(link)}\b", item["dev"])]
if more:
out += ["", "More: " + link_list(more)]
out += ["", "</details>"]
return "\n".join(out) + "\n"
# --- GitHub ---------------------------------------------------------------------------------------------------
def token():
for name in ("GITHUB_TOKEN", "GH_TOKEN"):
if os.environ.get(name):
return os.environ[name]
try:
return subprocess.run(["gh", "auth", "token"], capture_output=True, text=True, check=True).stdout.strip()
except (OSError, subprocess.CalledProcessError):
sys.exit("roadmap: reading GitHub needs GITHUB_TOKEN, GH_TOKEN or `gh auth login` (or use --offline)")
def numbers_in(data):
return sorted({int(ISSUE.match(link)[1]) for item in data.get("item", [])
for link in item.get("links", []) if ISSUE.match(link)})
def github_states(numbers, auth):
"""{number: {"kind": "Issue" | "PullRequest", "state", "merged_at", "merge_commit", "paths"}} for those
that exist, in one GraphQL request per hundred numbers."""
owner, name = REPO.split("/")
states = {}
for start in range(0, len(numbers), 100):
fields = "\n".join(
f"n{n}: issueOrPullRequest(number: {n}) {{ __typename "
"... on Issue { state } "
"... on PullRequest { state mergedAt mergeCommit { oid } files(first: 100) { nodes { path } } "
"labels(first: 50) { nodes { name } } } }"
for n in numbers[start:start + 100])
query = f'query {{ repository(owner: "{owner}", name: "{name}") {{ {fields} }} }}'
request = urllib.request.Request(
"https://api.github.com/graphql", data=json.dumps({"query": query}).encode(),
headers={"Authorization": f"bearer {auth}", "User-Agent": "horizon-roadmap"})
with urllib.request.urlopen(request, timeout=30) as response:
answer = json.load(response)
for key, node in (answer.get("data", {}).get("repository") or {}).items():
if node:
states[int(key[1:])] = {
"kind": node["__typename"],
"state": node["state"],
"merged_at": node.get("mergedAt"),
"merge_commit": (node.get("mergeCommit") or {}).get("oid"),
"paths": [file["path"] for file in ((node.get("files") or {}).get("nodes") or [])],
"labels": [label["name"] for label in ((node.get("labels") or {}).get("nodes") or [])],
}
return states
def pull_requests(item, states):
return [states[n] for n in (int(ISSUE.match(link)[1]) for link in item.get("links", []) if ISSUE.match(link))
if n in states and states[n]["kind"] == "PullRequest"]
def open_issues(item, states):
return [n for n in (int(ISSUE.match(link)[1]) for link in item.get("links", []) if ISSUE.match(link))
if n in states and states[n]["kind"] == "Issue" and states[n]["state"] == "OPEN"]
def online_problems(data, states, now):
"""What only GitHub can tell: links that don't exist, Now items nobody works on, Help wanted items already
done, items `sync` would ship."""
problems, notes = [], []
recent = now - datetime.timedelta(days=STALE_DAYS)
for item in data.get("item", []):
where = f"item {item['id']}"
for link in item.get("links", []):
match = ISSUE.match(link)
if match and int(match[1]) not in states:
problems.append(f"{where}: {link} is neither an issue nor a pull request of {REPO}")
pulls = pull_requests(item, states)
status = item.get("status")
if status == "now":
alive = [p for p in pulls if p["state"] == "OPEN" or
(p["merged_at"] and datetime.datetime.fromisoformat(p["merged_at"]) >= recent)]
if not alive:
problems.append(f"{where}: stale: Now, but no open pull request and none merged in the last "
f"{STALE_DAYS} days (link the work, or move it to Next or Later)")
if status == "help-wanted":
issues = [states[int(ISSUE.match(link)[1])] for link in item.get("links", [])
if ISSUE.match(link) and int(ISSUE.match(link)[1]) in states]
if issues and all(i["state"] != "OPEN" for i in issues):
problems.append(f"{where}: every linked issue and pull request is closed: ship or drop it")
if status != "shipped" and pulls and all(p["state"] == "MERGED" for p in pulls):
if item.get("ongoing"):
notes.append(f"{where}: ongoing, and all its pull requests merged: link the new ones")
elif not open_issues(item, states):
notes.append(f"{where}: all its pull requests merged: `python3 tools/roadmap.py sync` ships it "
"once a release has them")
return problems, notes
# --- sync -----------------------------------------------------------------------------------------------------
def family(paths):
"""Which releases carry a change: the launcher's, the website (deployed from main) or the game's."""
if paths and all(path.startswith("launcher/") for path in paths):
return "launcher"
if paths and all(path.startswith("website/") for path in paths):
return "website"
return "game"
def first_release(commit, kind):
"""The oldest release tag of this kind that contains the commit, or None while none does."""
if kind == "website":
return "website"
tags = subprocess.run(["git", "tag", "--contains", commit, "--sort=creatordate"], cwd=ROOT,
capture_output=True, text=True).stdout.split()
wanted = (lambda tag: tag.startswith(("launcher@", "launcher-v"))) if kind == "launcher" else \
(lambda tag: re.match(r"v\d", tag))
return next((tag for tag in tags if wanted(tag)), None)
def release_of(pulls):
"""The release that shipped the last of these merged pull requests, or None."""
kinds = {family(p["paths"]) for p in pulls}
kind = "game" if "game" in kinds or len(kinds) > 1 else kinds.pop()
latest = max(pulls, key=lambda p: p["merged_at"])
return first_release(latest["merge_commit"], kind) if latest["merge_commit"] else None
def item_span(lines, ident):
"""[start, end) of the item's lines: its [[item]] header to its last key, without the comments that follow."""
starts = [i for i, line in enumerate(lines) if line.strip() == "[[item]]"]
for position, start in enumerate(starts):
end = starts[position + 1] if position + 1 < len(starts) else len(lines)
if not any(re.match(rf'id\s*=\s*"{re.escape(ident)}"\s*$', line) for line in lines[start:end]):
continue
while end > start and (not lines[end - 1].strip() or re.match(r"#(\s|-|$)", lines[end - 1])):
end -= 1
return start, end
raise KeyError(ident)
def mark_shipped(text, ident, release):
"""The TOML with this item shipped in `release`, moved to the top of the shipped items (newest first)."""
lines = text.split("\n")
start, end = item_span(lines, ident)
block = [line for line in lines[start:end] if not line.startswith("shipped_in")]
block = [f'status = "shipped"\nshipped_in = "{release}"' if re.match(r"status\s*=", line) else line
for line in block]
block = [line for line in block if not re.match(r"focus\s*=", line)]
rest = lines[:start] + lines[end:]
# Drop the blank line the block leaves behind.
if start < len(rest) and start > 0 and not rest[start - 1].strip() and not rest[start].strip():
del rest[start]
shipped = [i for i, line in enumerate(rest) if re.match(r'status\s*=\s*"shipped"', line)]
if shipped:
header = max(i for i in range(shipped[0] + 1) if rest[i].strip() == "[[item]]")
rest[header:header] = block + [""]
else:
rest += [""] + block
return "\n".join(rest)
def sync(data, text, states):
"""Ship every item whose linked pull requests all went out in a release and whose linked issues are all
closed, unless it is ongoing. Returns the new text and what happened."""
log = []
for item in data.get("item", []):
if item.get("status") == "shipped":
continue
pulls = pull_requests(item, states)
# An ongoing item, or one with an open issue, still has work left whatever its pull requests say.
if item.get("ongoing") or open_issues(item, states):
continue
if not pulls or any(p["state"] != "MERGED" for p in pulls):
continue
release = release_of(pulls)
if release is None:
log.append(f"{item['id']}: merged, waiting for a release")
continue
text = mark_shipped(text, item["id"], release)
log.append(f"{item['id']}: shipped in {release}")
return text, log
# --- labels ---------------------------------------------------------------------------------------------------
def label_of(item):
"""The GitHub label of an item: its name, colour and description (GitHub allows 100 characters)."""
description = f"Roadmap, {STATUSES[item['status']].lower()}: {item['title']}"
return {"name": LABEL_PREFIX + item["id"], "color": LABEL_COLOURS[item["status"]], "description": description[:100]}
def label_changes(data, existing, states):
"""What `labels` does: [(action, label or number, payload)] for labels to create or update, and open pull
requests linked from an item that lack its label. existing: {name: {"color", "description"}}; states as
github_states, plus each pull request's "labels"."""
changes = []
for item in data.get("item", []):
label = label_of(item)
have = existing.get(label["name"])
if have is None:
changes.append(("create", label["name"], label))
elif (have.get("color", "").lower(), have.get("description") or "") != (label["color"], label["description"]):
changes.append(("update", label["name"], label))
for link in item.get("links", []):
match = ISSUE.match(link)
state = states.get(int(match[1])) if match else None
if state and state["kind"] == "PullRequest" and state["state"] == "OPEN" and \
label["name"] not in state.get("labels", []):
changes.append(("add", int(match[1]), label["name"]))
stale = sorted(set(existing) - {LABEL_PREFIX + item["id"] for item in data.get("item", [])})
return changes, [name for name in stale if name.startswith(LABEL_PREFIX)]
def github_rest(method, path, auth, body=None):
request = urllib.request.Request(
f"https://api.github.com{path}", method=method, data=None if body is None else json.dumps(body).encode(),
headers={"Authorization": f"bearer {auth}", "Accept": "application/vnd.github+json",
"User-Agent": "horizon-roadmap"})
with urllib.request.urlopen(request, timeout=30) as response:
return json.load(response) if response.status != 204 else None
def existing_labels(auth):
labels, page = {}, 1
while True:
batch = github_rest("GET", f"/repos/{REPO}/labels?per_page=100&page={page}", auth)
labels.update({label["name"]: label for label in batch})
if len(batch) < 100:
return labels
page += 1
# --- commands -------------------------------------------------------------------------------------------------
def report(problems):
for problem in problems:
print(f"roadmap: {problem}", file=sys.stderr)
return 1 if problems else 0
def command_build(args):
data = load()
problems = validate(data)
if problems:
return report(problems)
markdown = render(data)
if args.check:
if not OUTPUT.exists() or OUTPUT.read_text() != markdown:
return report(["ROADMAP.md is out of date: run `python3 tools/roadmap.py build`"])
return 0
OUTPUT.write_text(markdown)
print(f"roadmap: wrote {OUTPUT.relative_to(ROOT)}")
return 0
def command_check(args):
data = load()
problems = validate(data)
if not problems and not args.offline:
states = github_states(numbers_in(data), token())
online, notes = online_problems(data, states, datetime.datetime.now(datetime.timezone.utc))
problems += online
for note in notes:
print(f"roadmap: {note}")
if not problems:
print(f"roadmap: {len(data.get('item', []))} items, all good")
return report(problems)
def command_sync(args):
data = load()
problems = validate(data)
if problems:
return report(problems)
subprocess.run(["git", "fetch", "--quiet", "--tags", "origin"], cwd=ROOT, check=False)
text, log = sync(data, SOURCE.read_text(), github_states(numbers_in(data), token()))
for line in log:
print(f"roadmap: {line}")
if args.dry_run or text == SOURCE.read_text():
return 0
new = tomllib.loads(text)
if len(new.get("item", [])) != len(data.get("item", [])) or validate(new):
return report(["sync produced an invalid file; nothing written"] + validate(new))
SOURCE.write_text(text)
OUTPUT.write_text(render(new))
print("roadmap: wrote roadmap/roadmap.toml and ROADMAP.md")
return 0
def command_labels(args):
data = load()
problems = validate(data)
if problems:
return report(problems)
auth = token()
changes, stale = label_changes(data, existing_labels(auth), github_states(numbers_in(data), auth))
for action, target, payload in changes:
print(f"roadmap: {action} {target}" + (f" ({payload})" if action == "add" else ""))
if args.dry_run:
continue
if action == "create":
github_rest("POST", f"/repos/{REPO}/labels", auth, payload)
elif action == "update":
github_rest("PATCH", f"/repos/{REPO}/labels/{urllib.parse.quote(target, safe='')}", auth,
{"color": payload["color"], "description": payload["description"]})
else:
github_rest("POST", f"/repos/{REPO}/issues/{target}/labels", auth, {"labels": [payload]})
for name in stale:
print(f"roadmap: {name} names no item any more: rename or delete it by hand (it stays on its pull requests)")
if not changes:
print("roadmap: labels up to date")
return 0
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
commands = parser.add_subparsers(dest="command", required=True)
build = commands.add_parser("build", help="write ROADMAP.md")
build.add_argument("--check", action="store_true", help="fail when ROADMAP.md is out of date")
check = commands.add_parser("check", help="the roadmap's rules, and GitHub's state of its links")
check.add_argument("--offline", action="store_true", help="skip GitHub")
sync_command = commands.add_parser("sync", help="ship the items whose pull requests all merged")
sync_command.add_argument("--dry-run", action="store_true", help="say what would change, write nothing")
labels = commands.add_parser("labels", help="a roadmap:<id> label per item, on its linked open pull requests")
labels.add_argument("--dry-run", action="store_true", help="say what would change, change nothing")
args = parser.parse_args(argv)
return {"build": command_build, "check": command_check, "sync": command_sync,
"labels": command_labels}[args.command](args)
if __name__ == "__main__":
sys.exit(main())