[Host/Memory] Bound repeated executable compilation - #1167
Draft
brandonpayton wants to merge 4 commits into
Draft
Conversation
Repeated spawn and exec of the same VFS executable compiled another WebAssembly.Module every time. Short-lived Homebrew commands could therefore leave many compiled-code allocations awaiting engine collection after their process memory had retired. Key one bounded, cross-host LRU by the complete prepared byte snapshot length and SHA-256. This lets aliases share compilation while a write or replacement always selects new code. Coalesce concurrent misses, compile oversized modules uncached, and preserve the existing malformed-initial-image trap path. Cover content aliases, same-length replacement, concurrent reuse, eviction, failed compilation, and a real Node VFS replacement.
Precompiling initial executables in the kernel Worker must not change how a malformed initial module fails. A CompileError still belongs to the process-Worker loader path, while exec and posix_spawn reject bad modules during preflight. Launch a header-valid but truncated module through the real Node host. Verify that it creates the process, exits through the loader-error path, and reports the existing host diagnostic.
Keep the new cache and its focused unit tests consistent with the host TypeScript formatting contract.
Document the measured working set behind the default limits. Clarify that source size is a portable proxy, not native-code size. Lock both defaults with a focused policy test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Stock Brew starts many short-lived Ruby and utility processes while it
taps a repository. Kandelo retires each process address space, but
repeated launches still create heavy host-runtime work.
Tracing that path found one avoidable cost: every
spawnandexeccompiled the executable's WebAssembly bytes again, even when the same
Ruby binary had been compiled moments earlier.
This change bounds and reuses that compiled code. It is an independent
host-runtime improvement, but Chromium evidence now shows that it is
not the complete fix for the Brew renderer crash.
What changed
kernel-worker paths.
aliases share code while changed bytes cannot run stale code.
The cache retains compiled modules, not process address spaces. Every
process still receives a fresh
WebAssembly.Memory, and pthreads stillshare only their owning process's memory.
Evidence
A matched Node stock-Brew measurement used the same exact public
fixture with and without this cache:
The maximum-RSS change is 17.91 percent. The user-CPU change is
37.58 percent. These are bounded Node results, not broad performance
claims.
Focused cache, rootfs-export, and spawn-parity validation passed 22 of
22 tests. Host type checking and projection checks also passed.
Chromium result
Draft diagnostic PR #1168 ran the exact public core and canary
lifecycle on fresh Chromium runners. Both reached the same bounded
cache state:
Both renderers still crashed during the common first-party
brew tapphase, with no cgroup OOM event. The cache works, but another source of
short-lived process pressure remains.
The sealed fixture uses the older fork-heavy Ruby bottle. It does not
contain #1166's direct-
posix_spawnpath. Per-launch hashing of freshexecutable snapshots also remains under investigation.
Keep this pull request in draft until the revised Ruby lifecycle and
the browser cache cost are separated. Do not use its Node result alone
to claim that Chromium is fixed.
Scope
This changes the shared host runtime for Node and browser. It does not
change the kernel, POSIX behavior, guest ABI, package contents,
bottles, or VFS format.