Merge pull request #2584 from ludovic111/perf/parity-manifest-aware
🤖 Agent: crate splits' workspace-only manifest changes no longer select every harness (354 to 209 on #2160 and #2215); other PRs select as before.
This commit is contained in:
@@ -64,7 +64,7 @@ A harness can declare, at the top level of its script:
|
||||
|
||||
### Which harnesses a change affects
|
||||
|
||||
`--changed` and `--changed-files` narrow each harness's Rust sources to those its probes can reach (`tools/parity_reach.py`), so a change to one module of `horizon-client` no longer reruns every harness whose probe links the crate. The narrowing works on items, not files: from the probe modules a harness ran (or the binary it built), a free item (fn, type, const, static) is reached when a reached item names it in its own module, through a path or an import of a module that is named, or through a glob import; an inherent impl's methods count one by one once their type is reached and their name is mentioned; a trait impl counts once its trait and its type are reached (std's traits and types always are); a macro counts once invoked. Names match by spelling, so the result only over-approximates. A reached file's parent modules count through their `mod` declarations of it, compared with the merge base; Rust files a change adds take part, a dependency that was deleted counts as changed, code under `#[cfg(test)]` never counts, and a harness whose probe can't be found keeps every source. Parses are cached by content in `reach-cache.json` of the shared parity folder (beside `passes/`), so a new worktree starts warm; a harness's search stops as soon as it reaches a changed file, and after `PARITY_REACH_BUDGET` seconds (default 180) the remaining harnesses count every source their probes link, so choosing never holds a slow machine for long. `--by-crate` turns the narrowing off. On the last 20 pull requests merged on 2026-10-25, the harnesses picked went from 2255 to 1061 (2384 to 1592 harness-minutes; game-binary harnesses, which link all of `game/`, set the floor), and every harness a CI parity run failed in the week before was still picked; every source file a sample of probes executed (LLVM `-C instrument-coverage`) is among those kept. Building the graph takes 15 s cold, under a second warm, and up to a minute and a half on a change that reaches many harnesses.
|
||||
`--changed` and `--changed-files` narrow each harness's Rust sources to those its probes can reach (`tools/parity_reach.py`), so a change to one module of `horizon-client` no longer reruns every harness whose probe links the crate. The narrowing works on items, not files: from the probe modules a harness ran (or the binary it built), a free item (fn, type, const, static) is reached when a reached item names it in its own module, through a path or an import of a module that is named, or through a glob import; an inherent impl's methods count one by one once their type is reached and their name is mentioned; a trait impl counts once its trait and its type are reached (std's traits and types always are); a macro counts once invoked. Names match by spelling, so the result only over-approximates. A reached file's parent modules count through their `mod` declarations of it, compared with the merge base; Rust files a change adds take part, a dependency that was deleted counts as changed, code under `#[cfg(test)]` never counts, and a harness whose probe can't be found keeps every source. Parses are cached by content in `reach-cache.json` of the shared parity folder (beside `passes/`), so a new worktree starts warm; a harness's search stops as soon as it reaches a changed file, and after `PARITY_REACH_BUDGET` seconds (default 180) the remaining harnesses count every source their probes link, so choosing never holds a slow machine for long. `--by-crate` turns the narrowing off. With `--changed`, `Cargo.lock` and `Cargo.toml` changes that only add, remove or move workspace crates (a crate split: no registry or git package, version, feature, dependency or build setting changed across them, compared with the merge base) don't count: the moved code counts through its sources. A workspace crate's feature that such a change defines or turns on is allowed only where every source naming it changed too (the new crate's own files). #2160's split went from 352 harnesses to 123 this way (its own changes to the runner still select them all). On the last 20 pull requests merged on 2026-10-25, the harnesses picked went from 2255 to 1061 (2384 to 1592 harness-minutes; game-binary harnesses, which link all of `game/`, set the floor), and every harness a CI parity run failed in the week before was still picked; every source file a sample of probes executed (LLVM `-C instrument-coverage`) is among those kept. Building the graph takes 15 s cold, under a second warm, and up to a minute and a half on a change that reaches many harnesses.
|
||||
|
||||
## The gate
|
||||
|
||||
|
||||
@@ -401,3 +401,65 @@ class KnownFlakyTests(unittest.TestCase):
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
|
||||
class WorkspaceManifestTests(unittest.TestCase):
|
||||
"""A crate split's Cargo.toml and Cargo.lock changes don't select every harness; a dependency change still does."""
|
||||
CLIENT = '[package]\nname = "horizon-client"\n\n[dependencies]\nserde = "1"\nwgpu = { version = "27", features = ["x"] }\n'
|
||||
LOCK = ('version = 4\n\n[[package]]\nname = "horizon-client"\nversion = "0.1.0"\ndependencies = [\n "serde",\n]\n\n'
|
||||
'[[package]]\nname = "serde"\nversion = "1.0.1"\nsource = "registry+https://github.com/rust-lang/crates.io-index"\n')
|
||||
|
||||
def verdict(self, base, head):
|
||||
"""base, head: {path: text or None}; the manifests workspace_only_manifests leaves out."""
|
||||
with tempfile.TemporaryDirectory() as temp, \
|
||||
mock.patch.object(parity, "ROOT", Path(temp)), \
|
||||
mock.patch.object(parity, "git_show", lambda _base, path: base.get(path)):
|
||||
for path, text in head.items():
|
||||
if text is not None:
|
||||
Path(temp, path).parent.mkdir(parents=True, exist_ok=True)
|
||||
Path(temp, path).write_text(text)
|
||||
return parity.workspace_only_manifests(set(head) | set(base), "base")
|
||||
|
||||
def test_a_crate_split_is_left_out(self):
|
||||
command = '[package]\nname = "horizon-command"\n\n[dependencies]\nserde = "1"\nhorizon-core = { path = "../horizon-core" }\n'
|
||||
client = self.CLIENT + 'horizon-command = { path = "../horizon-command" }\n'
|
||||
lock = self.LOCK.replace('[[package]]\nname = "serde"', '[[package]]\nname = "horizon-command"\nversion = "0.1.0"\n\n'
|
||||
'[[package]]\nname = "serde"')
|
||||
base = {"Cargo.toml": '[workspace]\nmembers = ["crates/horizon-client"]\nresolver = "2"\n',
|
||||
"crates/horizon-client/Cargo.toml": self.CLIENT, "Cargo.lock": self.LOCK}
|
||||
head = {"Cargo.toml": '[workspace]\nmembers = ["crates/horizon-client", "crates/horizon-command"]\nresolver = "2"\n',
|
||||
"crates/horizon-client/Cargo.toml": client, "crates/horizon-command/Cargo.toml": command, "Cargo.lock": lock}
|
||||
self.assertEqual(self.verdict(base, head), set(head))
|
||||
|
||||
def test_a_dependency_change_still_counts(self):
|
||||
base = {"crates/horizon-client/Cargo.toml": self.CLIENT, "Cargo.lock": self.LOCK}
|
||||
bumped = {"crates/horizon-client/Cargo.toml": self.CLIENT, "Cargo.lock": self.LOCK.replace("1.0.1", "1.0.2")}
|
||||
self.assertEqual(self.verdict(base, bumped), set())
|
||||
feature = {"crates/horizon-client/Cargo.toml": self.CLIENT.replace('["x"]', '["x", "y"]')}
|
||||
self.assertEqual(self.verdict({"crates/horizon-client/Cargo.toml": self.CLIENT}, feature), set())
|
||||
resolver = {"Cargo.toml": '[workspace]\nmembers = ["a"]\nresolver = "3"\n'}
|
||||
self.assertEqual(self.verdict({"Cargo.toml": '[workspace]\nmembers = ["a"]\nresolver = "2"\n'}, resolver), set())
|
||||
|
||||
def test_a_feature_of_a_workspace_crate_counts_through_its_sources(self):
|
||||
items = '[package]\nname = "horizon-items"\n\n[features]\ntest-support = []\n'
|
||||
client = self.CLIENT + '\n[dev-dependencies]\nhorizon-items = { path = "../horizon-items", features = ["test-support"] }\n'
|
||||
head = {"crates/horizon-items/Cargo.toml": items, "crates/horizon-client/Cargo.toml": client,
|
||||
"crates/horizon-items/src/lib.rs": '#[cfg(feature = "test-support")]\npub fn for_tests() {}\n'}
|
||||
base = {"crates/horizon-client/Cargo.toml": self.CLIENT}
|
||||
# The new crate's files are all changed, the code the feature gates with them.
|
||||
self.assertEqual(self.verdict(base, head), {"crates/horizon-items/Cargo.toml", "crates/horizon-client/Cargo.toml"})
|
||||
# Turning on a feature of a crate whose gated code didn't change selects as before.
|
||||
world = '[package]\nname = "horizon-world"\n\n[features]\nextra = ["more"]\nmore = []\n'
|
||||
with tempfile.TemporaryDirectory() as temp, mock.patch.object(parity, "ROOT", Path(temp)):
|
||||
Path(temp, "crates/horizon-world/src").mkdir(parents=True)
|
||||
Path(temp, "crates/horizon-world/Cargo.toml").write_text(world)
|
||||
Path(temp, "crates/horizon-world/src/lib.rs").write_text('#[cfg(feature = "more")]\nfn x() {}\n')
|
||||
self.assertFalse(parity.feature_sources_changed("horizon-world", "extra", {"crates/horizon-client/Cargo.toml"}))
|
||||
self.assertTrue(parity.feature_sources_changed("horizon-world", "extra", {"crates/horizon-world/src/lib.rs"}))
|
||||
self.assertFalse(parity.feature_sources_changed("horizon-nothing", "extra", set()))
|
||||
# A feature that reaches an external package counts.
|
||||
optional = {"crates/horizon-client/Cargo.toml": self.CLIENT + '\n[features]\nfast = ["wgpu/x"]\n'}
|
||||
self.assertEqual(self.verdict({"crates/horizon-client/Cargo.toml": self.CLIENT}, optional), set())
|
||||
|
||||
def test_without_a_base_everything_counts(self):
|
||||
self.assertEqual(parity.workspace_only_manifests({"Cargo.lock"}, None), set())
|
||||
|
||||
+100
-1
@@ -1246,6 +1246,101 @@ def trace_main(output, script, arguments):
|
||||
|
||||
# Files every harness's result depends on (part of environment_key): a change to one affects them all.
|
||||
SHARED_INPUTS = {"research-lock.json", "Cargo.lock"}
|
||||
# Manifest sections that say what a crate is and where its targets are, not what it builds with.
|
||||
MANIFEST_SECTION_KEPT = re.compile(r"\[\[?(?:package|lib|bin|example|test|bench|lints|badges)\b[^\]]*\]\]?$")
|
||||
WORKSPACE_CRATE = r"(horizon[\w-]*|cargo-horizon-mod|xtask)"
|
||||
# A workspace crate named as a dependency (by path, or from the workspace's own table) without choosing its features,
|
||||
# or listed as a member.
|
||||
WORKSPACE_LINE = re.compile(WORKSPACE_CRATE + r'\s*=\s*(?:\{(?![^}]*\bfeatures\b)[^}]*\b(?:path|workspace)\s*=[^}]*\}|true)'
|
||||
r'|' + WORKSPACE_CRATE + r'\.workspace\s*=\s*true|"crates/[^"]+",?')
|
||||
# A workspace crate named as a dependency with some of its features turned on.
|
||||
WORKSPACE_FEATURES = re.compile(WORKSPACE_CRATE + r'\s*=\s*\{(?=[^}]*\b(?:path|workspace)\s*=)(?![^}]*default-features)'
|
||||
r'[^}]*\bfeatures\s*=\s*\[([^\]]*)\][^}]*\}')
|
||||
|
||||
|
||||
def git_show(base, path):
|
||||
"""path's content at commit base, or None where it didn't exist."""
|
||||
result = subprocess.run(["git", "show", f"{base}:{path}"], cwd=ROOT, capture_output=True, text=True)
|
||||
return result.stdout if result.returncode == 0 else None
|
||||
|
||||
|
||||
def manifest_lines(text):
|
||||
"""A Cargo.toml's lines that decide what is built: (section, line) pairs, without its package's identity and
|
||||
targets, comments, or workspace crates (whose code counts through its own sources)."""
|
||||
lines, section = set(), ""
|
||||
name = re.search(r'^\[package\][^\[]*?^name\s*=\s*"([^"]+)"', text or "", re.M | re.S)
|
||||
for raw in (text or "").splitlines():
|
||||
line = re.sub(r"\s+", " ", raw.split("#", 1)[0]).strip()
|
||||
if not line:
|
||||
continue
|
||||
if line.startswith("["):
|
||||
# A crate's features are its own: the same line in two crates means two things.
|
||||
section = f"[features] of {name.group(1) if name else '?'}" if line == "[features]" else line
|
||||
continue
|
||||
if MANIFEST_SECTION_KEPT.match(section) or WORKSPACE_LINE.fullmatch(line) or \
|
||||
section == "[workspace]" and (line.startswith("members =") or line.startswith('"') or line == "]"):
|
||||
continue
|
||||
lines.add((section, line))
|
||||
return lines
|
||||
|
||||
|
||||
def lock_packages(text):
|
||||
"""Cargo.lock's packages from a registry or git (with their versions and checksums); workspace crates' entries
|
||||
only list what their manifests say."""
|
||||
return {block.strip() for block in (text or "").split("[[package]]") if "\nsource = " in block}
|
||||
|
||||
|
||||
def workspace_only_manifests(changed, base):
|
||||
"""The changed Cargo.lock and Cargo.toml files whose changes only add, remove or move workspace crates (a crate
|
||||
split): no external package, version, feature, or build setting changed across them. Their crates' code counts
|
||||
through its own sources, so these files don't make every harness linking the crates rerun."""
|
||||
manifests = sorted(path for path in changed if path == "Cargo.lock" or path.endswith("Cargo.toml"))
|
||||
if not manifests or base is None:
|
||||
return set()
|
||||
if "Cargo.lock" in manifests and \
|
||||
lock_packages(git_show(base, "Cargo.lock")) != lock_packages((ROOT / "Cargo.lock").read_text(encoding="utf-8")):
|
||||
return set()
|
||||
tomls = [path for path in manifests if path != "Cargo.lock"]
|
||||
before = set().union(*(manifest_lines(git_show(base, path)) for path in tomls)) if tomls else set()
|
||||
after = set().union(*(manifest_lines((ROOT / path).read_text(encoding="utf-8") if (ROOT / path).exists() else None)
|
||||
for path in tomls)) if tomls else set()
|
||||
# Workspace crates' features that a changed line defines or turns on: they change what compiles of that crate's
|
||||
# sources, which is fine only where every source naming them changed too (a split: the crate's files are new).
|
||||
gated = set()
|
||||
for section, line in before ^ after:
|
||||
wanted = WORKSPACE_FEATURES.fullmatch(line)
|
||||
if wanted:
|
||||
gated |= {(wanted.group(1), feature) for feature in re.findall(r'"([^"]+)"', wanted.group(2))}
|
||||
elif section.startswith("[features] of ") and re.fullmatch(r'[\w-]+ = \[(?: ?"[\w-]+",?)*\s*\]', line):
|
||||
gated.add((section[len("[features] of "):], line.split(" ", 1)[0]))
|
||||
else:
|
||||
return set() # an external package, a feature reaching one ("dep:", "crate/feature"), a build setting
|
||||
return set(manifests) if all(feature_sources_changed(crate, feature, changed) for crate, feature in gated) else set()
|
||||
|
||||
|
||||
def feature_sources_changed(crate, feature, changed):
|
||||
"""Whether every Rust source of workspace crate `crate` that names `feature`, or a feature it turns on, is in
|
||||
changed (False where the crate isn't found)."""
|
||||
listed = subprocess.run(["git", "ls-files", "*Cargo.toml"], cwd=ROOT, capture_output=True, text=True)
|
||||
manifests = [ROOT / path for path in listed.stdout.split()] if listed.returncode == 0 else list(ROOT.rglob("Cargo.toml"))
|
||||
for manifest in manifests:
|
||||
text = manifest.read_text(encoding="utf-8") if manifest.exists() else ""
|
||||
if re.search(r'^\[package\][^\[]*?^name\s*=\s*"' + re.escape(crate) + '"', text, re.M | re.S):
|
||||
break
|
||||
else:
|
||||
return False
|
||||
table = re.search(r"^\[features\]\n(.*?)(?=^\[|\Z)", text, re.M | re.S)
|
||||
implies = {name: re.findall(r'"([^"]+)"', value) for name, value in
|
||||
re.findall(r'^([\w-]+)\s*=\s*\[([^\]]*)\]', table.group(1) if table else "", re.M)}
|
||||
names, todo = set(), [feature]
|
||||
while todo:
|
||||
name = todo.pop()
|
||||
if name not in names:
|
||||
names.add(name)
|
||||
todo += [implied for implied in implies.get(name, []) if "/" not in implied and ":" not in implied]
|
||||
named = re.compile(r'feature\s*=\s*"(?:' + "|".join(map(re.escape, sorted(names))) + r')"')
|
||||
return all(path.relative_to(ROOT).as_posix() in changed for path in manifest.parent.rglob("*.rs")
|
||||
if named.search(path.read_text(encoding="utf-8", errors="replace")))
|
||||
|
||||
|
||||
def changed_files(ref):
|
||||
@@ -1449,7 +1544,11 @@ def main():
|
||||
if args.changed:
|
||||
merge_base = subprocess.run(["git", "merge-base", "HEAD", args.changed], cwd=ROOT, capture_output=True, text=True)
|
||||
base = merge_base.stdout.strip() or None
|
||||
affected = affected_harnesses(found, records, changed, reach=not args.by_crate, base=base)
|
||||
moved = workspace_only_manifests(changed, base) if not args.by_crate else set()
|
||||
if moved:
|
||||
print(f"{', '.join(sorted(moved))}: only workspace crates added, removed or moved; their sources count "
|
||||
"instead", flush=True)
|
||||
affected = affected_harnesses(found, records, changed - moved, reach=not args.by_crate, base=base)
|
||||
chosen = [name for name in found if name in affected and stale(name, state, hasher, environment)]
|
||||
print(f"{len(changed)} files changed{since}: {len(affected)} harnesses "
|
||||
f"depend on them, {len(chosen)} without a pass for their current inputs.", flush=True)
|
||||
|
||||
Reference in New Issue
Block a user