perf: batch the CODEOWNERS query so gv <paths> is 4x faster - #124
Draft
perryqh wants to merge 1 commit into
Draft
Conversation
perryqh
force-pushed
the
perf/o1-batch-codeowners-query
branch
from
August 20, 2026 21:20
03eff63 to
ee9064b
Compare
perryqh
force-pushed
the
perf/o1-batch-codeowners-query
branch
from
August 20, 2026 21:32
ee9064b to
ba60023
Compare
validate_files called team_for_file_from_codeowners once per path. That helper wraps the path in a one-element slice and hands it to the batch query, which reloads the config and re-reads and re-parses the entire CODEOWNERS file every time — parse_codeowners_entries is not memoized, unlike teams_by_github_team_name right beside it. Against an 18k-line CODEOWNERS that cost ~9.5ms per path, linearly: a 2000-file changeset spent 22s, of which ~20s was re-parsing the same file 2000 times. Now the paths are filtered once and handed to the batch query in a single call, which already parallelizes internally. Behavior notes: - Unowned files are still reported using the caller's original path string, and still in input order, so absolute paths render as before. - IO error granularity changes: the loop could attribute a failure to one path, while a batched read succeeds or fails for the whole set. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
perryqh
force-pushed
the
perf/o1-batch-codeowners-query
branch
from
August 20, 2026 21:35
ba60023 to
e9dedbf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
generate-and-validate <paths>on a 1000-file changeset goes from 14.6s to 3.6s (4.1x) on a 130k-file monorepo.What was wrong
validate_fileslooped over paths callingteam_for_file_from_codeowners. That helper wraps the path in a one-element slice and hands it to the batch query — which reloads the config and re-reads and re-parses the entire 17,981-line CODEOWNERS every time.parse_codeowners_entriesis not memoized, unliketeams_by_github_team_namedirectly beside it.Cost: ~9.5ms per path, linear. On a 1000-path changeset that is ~10s spent re-parsing the same file 1000 times.
The batch function already accepts a slice and already parallelizes across it with
par_iter. It was simply being called wrong.Why this targets
gv <paths>and notvalidate <paths>An earlier version of this work also bypassed the project build, which made
validate <paths>~1500x faster. That was closed (#123) becausevalidate <paths>is not equivalent tovalidate— it resolves ownership by reading the CODEOWNERS file, so it validates a derived artifact against itself and cannot detect a stale CODEOWNERS, an annotation naming a nonexistent team, or a file owned two ways. Making that path faster is not a win.gv <paths>is correct: it regenerates before validating. So this PR deliberately keeps the win that lands on the correct command and drops the one that only helped the incorrect one.This change does not extend or entrench the
validate <paths>path — it only stops the shared inner loop doing redundant work.Measured
Corpus: 130,934 tracked files, 91,206 owned, 17,981-line CODEOWNERS. macOS/aarch64, 11 cpus.
Interleaved A/B, 6 rounds, min-of-N.
validate_allis included as a control: this change should not affect it, and it doesn't — which is what makes the other two rows trustworthy.gv <1000 paths>gv <100 paths>validate_all(control)Interleaving matters here: a first attempt measured each branch in its own block and the machine drifted ~35% mid-run, which produced three convincing but entirely fake wins. Running every variant once per round makes drift shared rather than attributed, and the flat control confirms it worked.
The remaining 3.6s is the project build, which
generategenuinely needs — this change removes the per-file term, not the fixed cost.Correctness
unowned_globs, absolute paths, and paths absent from CODEOWNERS.gvstdout, stderr and exit code all byte-identical, and the generated CODEOWNERS (17,981 lines) is byte-identical.validate <paths>output also unchanged, so this is not a behavior change to that path — just a faster one.validate_filestests and the absolute-path case.Unowned files are still reported using the caller's original path string and in input order, so absolute paths render exactly as before. The query is keyed by project-relative path, so the original string is carried alongside rather than recomputed.
One behavior change
IO error granularity. The old loop could attribute a failure to one specific path (
path/x.rb: <error>). A batched read succeeds or fails for the whole set, so the error is no longer per-path. If per-path attribution matters to a caller, the batch signature needs to be fallible per item — worth a decision before merge.Follow-up
The parity gap in
validate <paths>is unfixed and is the more valuable piece of work: resolve ownership through the mappers for the supplied paths rather than reading CODEOWNERS. That would makevalidate <paths>trustworthy, and only then is bypassing the project build for it worth doing.🤖 Generated with Claude Code