feat(js): add js.spawn async task DSL - #76
Draft
nazarhussain wants to merge 1 commit into
Draft
Conversation
Worker-thread async previously meant hand-rolling napi.AsyncWork plus Deferred at every call site, and DSL values were unusable in the completion callback: js.env() panics there because only the sync wrappers establish the thread-local env context. js.spawn takes a comptime duck-typed task (compute/resolve/deinit, with optional errorMessage/reject) and returns a Promise that settles on the JS thread. Its completion callback sets the DSL env context, so resolve can return DSL types, and composes with OwnedTypedArray to hand results to JS without copying. compute deliberately does NOT get the env context — napi calls are illegal on the worker thread, so a panic there is the guard rail. Lifted from lodestar-z bindings/napi/async_task.zig, which was written to be upstreamed and is deleted once this ships.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
zapi has no DSL for worker-thread async.
js.Promiseis synchronous-only — itsDeferredhandle isn't preserved across the JS boundary — so async work means hand-rollingnapi.AsyncWork+napi.Deferredat every call site. lodestar-z carries ~100 lines of identical raw plumbing per binding (blst.zigasyncAggregateWithRandomness,shuffle.zig).There was also a hard blocker for doing this cleanly:
js.env()panics inside an async completion callback, because only the synchronous wrappers (wrap_function,wrap_class,export_module) establish the thread-local env context. That's why DSL constructors andTypedArray.fromExternalabort if used there.What this adds
js.spawn(Task, task, resource_name)runstask.compute()on the libuv worker pool and returns a JS Promise that settles on the JS thread. A task is any struct with:compute(*Task) !void— worker thread; must not touch napiresolve(*Task, napi.Env) !T— JS thread;Tmay be a DSL type (js.Number), an owned typed array (transferred without copying),napi.Value, orvoiddeinit(*Task) void— safe afterresolvetransferred ownershiperrorMessage(anyerror) [:0]const u8orreject(*Task, napi.Env, anyerror) !napi.Valuefor rejection controlThe completion callback wraps
context.setEnv/restoreEnv, which is what makes DSL types usable inresolve. It composes withOwnedTypedArray(#68) so results transfer to JS with no copy.Design decisions
The env context is established in
completeonly, notexecute. The handoff note suggested wrapping both, butcurrent_envisthreadlocaland napi calls are illegal on the libuv worker thread — setting it there would invite exactly the misuse the panic is meant to catch.computepanicking onjs.env()is the guard rail.The context lives in
src/js/async_task.zig, notsrc/async_work.zig. Putting it in the raw N-API layer would makenapidepend onjs/context.zig, inverting the layering. Rawnapi.AsyncWorkusers still get no implicit DSL context, which is correct — the DSL path isjs.spawn.resolve's return is converted directly rather than throughwrap_function.convertReturn, whose failure path throws a JS exception. In a completion callback we want a rejected promise, not a pending exception. Returning a DSL class instance is not supported yet (it needs an addon identity); the compile error names the supported types.Testing
8 new vitest cases written first and watched fail (
js.spawnmissing → compile error). Coverage: DSL value built in the completion callback, realPromiseinstance, concurrent tasks, owned typed array transfer, empty transfer, task-supplied rejection message, default@errorNamerejection,Errorinstance.The
setEnvline is verified load-bearing — removing it and re-running reproduces exactly the predicted failure:Stress run of 15,000 tasks (5,000 transfers + 5,000 rejections + 5,000 resolutions): no crash, RSS delta 5.1 MB.
Full suite:
zig build test:zapi, 141 vitest tests across 6 example addons,zig fmt --checkand biome clean.Follow-up
Once released, lodestar-z deletes
bindings/napi/async_task.zig(this was lifted from it, written to be upstreamed) and migratesshuffle.zigShuffleTaskandblst.zigasyncAggregateWithRandomness, guarded by its existing 32+ vitest cases.Worth deciding separately: #72 (double free in external typed array creation) was closed unmerged, and its described hazard in
fromExternal's catch-all is still present. It doesn't block this PR —OwnedTypedArray.intoValueis the path used here — but it's adjacent.🤖 Generated with Claude Code