Skip to content

Compile inflector regexes once instead of per call - #54

Merged
perryqh merged 1 commit into
mainfrom
perf/oncelock-regexes
Aug 20, 2026
Merged

Compile inflector regexes once instead of per call#54
perryqh merged 1 commit into
mainfrom
perf/oncelock-regexes

Conversation

@perryqh

@perryqh perryqh commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Best return per line of any change I've measured on pks check: ~40 lines, and it cuts total CPU time by 45%.

Note

Stacked on #53 (which is itself stacked on #52). The diff above shows only this change — one file. GitHub retargets the base automatically as each parent merges.

The problem

camelize and to_class_case run once per file while the Zeitwerk constant map is built — roughly 50k calls per invocation on a large codebase. Each call was compiling between two and seven regexes from scratch:

if class_name.contains("Statu") {
    let re = Regex::new("Statuse$").unwrap();
    class_name = re.replace_all(&class_name, "Status").to_string();
    let re = Regex::new("Statu$").unwrap();
    ...
}

CLASS_CASE_TO_SINGULAR.into_iter().for_each(|(plural, singular)| {
    if class_name.contains(plural) {
        let re = Regex::new(plural).unwrap();   // compiled inside a loop
        ...
    }
});

Regex::new constructs a DFA. Doing that per filename, for every file in the app, on every run, is not a marginal cost.

The change

Every pattern becomes a LazyLock<Regex>, compiled once per process. That's the whole change — one file, no API or behavior change.

Measured

On a 51,513-file application:

before after
user CPU time 13.6 s 7.4 s −45%
"infer constants from filename" phase 0.665 s 0.047 s −93%
wall clock (quiet machine) 5.15 s 4.53 s −12.1%
wall clock (machine under load) 8.02 s 7.58 s −5.5%

Two honest notes on those numbers:

  • Wall clock improves less than CPU time, because this phase is parallelized — the wasted work was partly hidden behind other cores. The 45% CPU reduction is the invariant; it reproduced identically under both conditions.
  • The wall-clock benefit therefore depends on available parallelism. I measured −12.1% on an idle machine and −5.5% while a VM and an editor were competing for cores. Both are real; CI machines are usually closer to the second. I've quoted the range rather than the flattering number.

One thing to know about the diff

This preserves an existing oddity rather than fixing it:

let re = Regex::new("Statuss").unwrap();
re.replace_all(&class_name, "Status").to_string();   // result discarded

That result has always been thrown away, so statuss is left untouched — and test_to_class_case pins that behavior with a ("statuss", false, "Statuss") case. I kept it verbatim with a comment explaining why. It looks like a bug, but fixing it changes inflection output and belongs in its own PR, not one whose only job is to stop recompiling regexes.

Two other files have the same compile-per-call pattern (file_utils.rs:108, rails_utils.rs:19). I left them alone: once this landed, the phase was down to 0.047 s and there was nothing left to win. Not worth the churn.

Verification

  • cargo test — 257 passing
  • cargo clippy --all-targets --all-features — clean
  • cargo fmt --all -- --check — clean
  • Behavior is unchanged by construction (same patterns, same order, same call sites), and the inflector's own test suite covers the singularization and acronym cases.

Heads-up: a pre-existing flaky test

While verifying, I hit gitignore_test::test_respect_gitignore_can_be_disabled failing on roughly 1 run in 3. It is not caused by this change — it reproduces at the same rate on the base branch, and passes 6/6 when run in isolation.

Cause: common::teardown() globs tests/fixtures/*/tmp/cache/packwerk and deletes the cache for every fixture, not just the one the calling test used. gitignore_test.rs calls it from 7 different tests, which Rust runs in parallel threads within the same binary — so one test deletes another's cache mid-run.

Worth fixing separately (scope teardown to the fixture the test actually touched). Flagging it here because it will make CI intermittently red regardless of this PR.

🤖 Generated with Claude Code

@perryqh
perryqh requested a review from a team as a code owner August 19, 2026 22:15
@github-project-automation github-project-automation Bot moved this to Triage in Modularity Aug 19, 2026
@perryqh
perryqh force-pushed the perf/oncelock-regexes branch from d278f48 to a6ab8fd Compare August 19, 2026 22:28

@dduugg dduugg left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. I checked this specifically for the ways a "just hoist the regexes" refactor can quietly change output, and it's clean on all of them.

Verified

  • Pattern strings and evaluation order are identical. SINGULARIZE is built with CLASS_CASE_TO_SINGULAR.map(...), which preserves the const array's order (Censuse, Leafe, Lefe, Daum), and .iter().for_each iterates that same order — so cascading replacements behave as before.
  • The guard rewrite is equivalent: class_name.contains(re.as_str()) vs the old contains(plural). Regex::as_str() returns the original pattern text, and all four table patterns are plain literals with no regex metacharacters, so re.as_str() == plural exactly. The outer contains("Statu") guard is untouched.
  • No escaping hazard — every pattern here is a static literal, and construction is still Regex::new, just hoisted.
  • The discarded "Statuss" result is still discarded, so statuss remains untouched and the ("statuss", false, "Statuss") case still pins it. Agreed that fixing it belongs in its own PR — I'd rather see that tradeoff stated explicitly, as you did, than silently "improved" inside a perf change.
  • The unwrap() timing shift (per-call -> first-use) is inert: every pattern is a static, known-valid literal, so there's no input that could make one fail to compile, in a rayon worker or anywhere else.
  • LazyLock is the right primitive — stable since 1.80, and the repo has no once_cell/lazy_static dependency or existing convention to match.
  • Your call on file_utils.rs:108 and rails_utils.rs:19 holds up: rails_utils.rs:19 (get_acronyms_from_disk) is called once per run at zeitwerk/mod.rs:120, and file_utils.rs:108 runs once per .erb file — neither is per-file-hot the way to_class_case/camelize are. Leaving them is right.

Worth knowing: this doesn't actually depend on #53. It touches only inflector_shim.rs; #53 touches Cargo.toml, dev/*, main.rs, packs.rs, checker.rs. Zero file overlap and no symbol dependency — cherry-picked straight onto main, cargo build --lib succeeds (LazyLock needs 1.80, main already pins 1.92.0). So if #53 stalls in review, this can be retargeted to #52 or main directly rather than waiting. #53's LTO change is what makes your numbers reproducible, but that's a measurement convenience, not a code dependency.

LazyLock::new(|| Regex::new("Statu$").unwrap());
static STATUSS: LazyLock<Regex> =
LazyLock::new(|| Regex::new("Statuss").unwrap());
static SINGULARIZE: LazyLock<[(Regex, &str); CLASS_CASE_TO_SINGULAR.len()]> =

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking test-coverage note, since this table is the part where order matters most.

test_to_class_case covers "lefe" but not "leafe", and has no case where a single input triggers two replacements in sequence (e.g. something matching both a Statu* pattern and a singular-table entry). Those cascading paths are exactly what would break if the iteration order of CLASS_CASE_TO_SINGULAR ever changed — and this diff is what makes that order an explicit property of a static rather than an incidental property of a loop.

Pre-existing gap and not this PR's job to close: the refactor doesn't touch transformation logic, only where the regexes get compiled. Just a good candidate for a follow-up case or two while the reasoning is fresh.

@perryqh
perryqh force-pushed the perf/oncelock-regexes branch from a6ab8fd to 88e790f Compare August 20, 2026 21:45
@perryqh
perryqh force-pushed the perf/oncelock-regexes branch from 88e790f to 47b2383 Compare August 20, 2026 21:51
Base automatically changed from perf/measure-setup to main August 20, 2026 22:06
`camelize` and `to_class_case` run once per file while the Zeitwerk constant map
is built -- roughly 50k calls per invocation on a large codebase -- and each call
was compiling between two and seven regexes from scratch. Building a regex
constructs a DFA, so this was not a marginal cost.

Hoisted every pattern into a `LazyLock<Regex>` compiled once per process.

MEASURED on a 51,513-file app:
  wall clock                            5.127s -> 4.656s  (-9.2%)
  user cpu time                        13.561s -> 7.370s  (-45.7%)
  "infer constants from filename" phase  0.689s -> <0.12s

Wall clock improves less than CPU time because this phase is parallelized across
cores, so the wasted work was partly hidden. It was still burning nearly half the
process's total CPU.

Behavior is deliberately unchanged, including one oddity: the "Statuss"
replacement has always had its result discarded, so "statuss" is left alone. That
is preserved verbatim with a comment, and the existing test that pins it still
passes. It looks like a bug, but fixing it belongs in its own change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@perryqh
perryqh force-pushed the perf/oncelock-regexes branch from 47b2383 to 3bef8d6 Compare August 20, 2026 22:06
@perryqh
perryqh merged commit 54014ec into main Aug 20, 2026
15 checks passed
@perryqh
perryqh deleted the perf/oncelock-regexes branch August 20, 2026 22:09
@github-project-automation github-project-automation Bot moved this from Triage to Done in Modularity Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants