QuickJS engine: baseline-snapshot startup optimization (~25× faster VM boot) - #3342
Merged
Conversation
…-safe requeue, stable PRNG seed
… getWorkflowQueueName for conflict requeue, Buffer-free asset decoding, function replacers for payload injection, maxEventsLimit guard
…les; harden step-listing e2e assertions against eventually-consistent reads
…l column can lag terminal status)
… assertions (analytics listing can omit attempt entirely)
…ed (encp) hook payloads open Main's sealed-box work (#3096) makes cross-deployment resumeHook() seal hook payloads to the target run's published X25519 public key. The shared start() path publishes that key regardless of engine, so QuickJS runs receive sealed payloads too — but the QuickJS entrypoint resolved only the bare symmetric key via importKey(), which cannot open encp envelopes. The first sealed hook payload wedged the run right after hook_received, timing out every hook/webhook e2e on Vercel prod (node:vm legs were fine — the node engine resolves the full capability via memoizeEncryptionKey). Resolve deriveRunPayloadKeys() in the entrypoint instead and widen the runtime's key types from CryptoKey to DecryptionKey. Writes stay symmetric (encrypt() with RunPayloadKeys takes the encr path). Regression test seals a payload exactly as resumeHook does and round-trips it through the VM.
…import, VM-leak guard, telemetry namespace, eval-string escaping - Deterministic crypto.getRandomValues/randomUUID in the VM bootstrap, drawing from the seeded Math.random (identical sequences to the node engine's vm/index.ts implementations); all crypto.subtle methods throw with step-function guidance. process.env exposed as a frozen copy, matching node. - Intl: throwing constructors (no ICU in QuickJS), and toLocale*-family methods (incl. localeCompare) throw when given an explicit locale so cross-engine divergence is loud instead of silently writing different values into the event log. No-argument forms keep working. - runtime.ts lazy-imports the QuickJS entrypoint at dispatch, keeping the ~1.3MB embedded WASM assets out of node-engine deployments. - runQuickJSWorkflow wraps the per-run phase so an exceptional exit disposes the VM instead of leaking it in a reused compute instance; corrected the misleading fail-loud comment (run_failed, not retry); warn when the event drain loop exhausts its iteration bound. - Telemetry attributes renamed quickjs.* → workflow.vm.* to stay in the file's workflow.* namespace. - Eval-string correlation-id interpolation uses JSON.stringify instead of quote-only escaping. - common-vm.test.ts pins the reducer/reviver superset invariant against common.ts so the duplicated sets can't silently drift. - Docs enumerate the remaining global-surface differences (subtle.digest, Intl, WebAssembly, Atomics); quickjs-entrypoint documents the known precondition-guard gap.
…tion + resumeId dedup) #1834 made resumeHook() fall back to enqueueing the run with a hookInput payload when the direct hook_received write fails transiently, with the runtime materializing the missing event on delivery. Only the node:vm path implemented it — the QuickJS dispatch returned before the node block, so the resilient payload was silently dropped and the new e2e timed out on every quickjs leg. - runtime.ts threads hookInput into runWorkflowWithQuickJS; the entrypoint materializes the missing hook_received after loading the event log (resumeId-keyed dedup, occurredAt from the resumeId ULID, local eventData substitution for lazy/ref responses, EntityConflict / HookNotFound handling) — mirroring the node block. - processEvents drops duplicate hook_received rows sharing a resumeId (first-in-log wins), matching the node engine's EventsConsumer dedup; the seen-set lives in the VM heap so it is deterministic per replay. Verified against the dev server with WORKFLOW_VM=quickjs: the resilient resume e2e passes and the materialization is observable in the logs; all 27 hook e2e tests green.
… WASM module caching
…loop event ceiling - Inline steps now claim via a lazy step_started carrying the input (step_created deferred, atomic create-claim in the world), with ownerMessageId stamped and authoritativeAttempt=1 — a concurrent invocation racing on the same fresh step loses with EntityConflictError and skips instead of both bare-starting the step and double-running the body. This also removes the stepsCreatedByUs set, whose 'created by us' invariant didn't survive the swallowed create-race conflict; redelivery backstops now key on hasCreatedEvent. - dispatchPendingOps' createdAttributeEvent/createdGetConflictHook signals are consumed again: when the loop exits suspended without ever reading back a self-written attr_set / getConflict hook_created (eventually-consistent listing lag), the entrypoint requeues immediately instead of parking the run awaiting_external with its unblocking event already written. - The server-supplied event ceiling is re-checked at the top of every continuation-loop turn (seenEventIds.size), so a single invocation fanning out inline can no longer grow the log arbitrarily past the operator's limit. The quickjs dispatch in runtime.ts converts MaxEventsExceededError into run_failed / MAX_EVENTS_EXCEEDED — the guard's throw previously nacked forever, parking runaway runs in 'running'. - Documented the deliberate decision that the platform function timeout is the only bound on inline chaining (budget parked per batch), matching the node engine.
(Re-applied onto the review-fixed base; original commits da27230 + 9814ed9 squashed.) Replace the in-VM serde bundle with a host-side codec (runtime/quickjs-serde.ts) built on quickjs-wasi 3.3's introspection primitives and devalue 5.9's pluggable stringify/parse operations — mirroring the node:vm engine's architecture. Review fixes incorporated: - reducer/reviver key sets are pinned against codec-devalue-vm's workflow mode by exhaustiveness tests (exact order for reducers — first match wins), so the handle-space codec can't silently drift from the shared value-space sets. - the devalue entry in minimumReleaseAgeExclude is removed: the exact version is pinned via the workspace catalog + lockfile, so the cooldown waiver was unnecessary (verified with both frozen and regular installs). - eval-string interpolation inherits the JSON.stringify(cid) hardening from the base branch.
…hreads Merge resolution — main's #3048 finals carried into the inline-loop architecture: - namespace + run-origin nextTraceCarrier threaded through runWorkflowWithQuickJS into every publish (step handoffs, hook_conflict requeue, wait continuations, immediate requeues) - suspended-exit requeues converted to FRESH messages (never { timeoutSeconds } visibility-redelivery of the current message — the hookInput redelivery trap fixed on #3048); exit wait sweep enqueues the continuation for the soonest unscheduled wait directly - entrypoint-side hookInput materialization dropped in favor of main's engine-agnostic prologue re-ensure in runtime.ts (with #3230's (runId, resumeId) claim protocol); dispatch stays inside the replay loop's try so engine failures classify into run_failed - interrupt handler keeps the perf branch's per-burst mutable budget, with main's configurable getReplayTimeoutMs() as the ceiling Review fixes (PR #3049 threads): - CRITICAL overflow wedge: overflow steps are handed to the queue in the same turn their step_created is written, BEFORE the event feed — the feed always observes those writes and continued the loop, so the old handoff was unreachable on the only turn that classified the steps as fresh (the cause of promiseRaceStressTestWorkflow hanging in the quickjs CI legs) - backstop gating: the deliveryAttempt > 1 gate (common case on worlds that advance attempts on routine redeliveries) is replaced with the node engine's ownership decision table — lease-active steps owned by another message arm a DELAYED backstop for the lease remainder under an epoch-scoped key; owner redeliveries and expired/unstamped steps dispatch immediately under the bare-correlationId key. Ownership is derived host-side from observed step_started/step_retrying events - ack-without-requeue: inline step terminals the feed has not surfaced raise the requeue signal, so the loop never acks with durably written terminals and nothing scheduled to consume them - idempotency keys bucketed by purpose (dispatch / backstop:<epoch> / retry:<n>) so worlds that retire used keys cannot swallow a later publish for the same step - live-feed terminal buffering: step/wait/attr terminals arriving before this VM constructs the corresponding resolver are buffered (__terminalBuffer, mirroring __hookPayloadBuffer) and settle the promise at construction — the single-scan continuation path previously dropped them and the await never settled Validated: core 1888 passed, full e2e 136/136 under WORKFLOW_VM=quickjs (nextjs-turbopack dev, world-local).
Carries the #3049 merge (and through it main/#3048) into the host-serde engine. The merge was textually clean but needed one semantic adaptation: the live-feed terminal buffer added on the perf branch (__terminalBuffer / __registerResolver) originally buffered raw bytes and deserialized them in-guest via Symbol.for('workflow-deserialize') — a global this branch retires along with the VM serde bundle. The buffer now stores host-deserialized VM values instead: the no-resolver branches of step_completed / step_failed run the same serde.deserialize() path as their resolver branches and buffer the resulting value ('resolve_value' / 'reject_value'), so draining at promise construction only forwards it. Validated: core 1932 passed; hook (26/26), promiseRace and fail e2e green under WORKFLOW_VM=quickjs.
Resolution: host-serde adaptations kept for all serde-threading and terminal-buffer conflicts (value-kind buffering via host deserialize); main's sleepWinsRace wait-continuation fix taken in the entrypoint scheduling sweep.
… pass-scoped handle disposal, byte-cache lifecycle - NUL (U+0000) safety across the WASM boundary: handle.toString() routes through JS_ToCString and silently truncates at the first NUL, and the C-string key APIs mangle NUL-bearing property keys (drop or collide). guestString() detects truncation by comparing against the handle's true guest length and recovers via in-VM JSON.stringify escaping; shapeOf verifies its fast host-string key list against a guest Object.keys count (+ duplicate check) and re-extracts through key handles on mismatch; get/hasOwn route NUL-bearing keys through length-aware guest string handles. All string funnels (primitives, symbol descriptions, error fields via chained/own reads, Headers entries, RegExp source/flags, URL href) go through guestString. Regression-tested down to the truncate-vs-collide enumeration shapes; fixes nullByteWorkflow on the quickjs e2e legs. - RetryableError's absent/invalid retryAfter fallback now reads the GUEST clock (the deterministic replay clock at the WASI layer) via a captured Date.now instead of the host wall clock — the in-VM reducer was replay-stable by construction and the host port silently lost that. - Pass-scoped handle disposal: serialize/deserialize sweep every intermediate handle their pass creates (call/invoke results, descriptor reads, dups, parse-op constructions), closing the ~one-leaked-handle-per-value-node growth across long-lived inline sessions. Implemented with module-owned tracking rather than vm.withScope: the library scope also captures the handles the host-callback trampoline wraps around C-owned argv pointers, and disposing those (Map/Set/Headers forEach visitors run mid-pass) double-frees guest values — observed as WASM memory corruption. identities is cleared per pass so freed-pointer reuse cannot alias entries across passes. - Byte-cache lifecycle: terminal drain now shares the per-VM cache with the suspension path (re-serializing an op at drain could re-invoke getters and produce different bytes for what the log treats as one value), and entries for settled ops — which neither collection filter can match again — are evicted, bounding the cache by the live pending set.
… accounting, loud unregistered-callback failures 3.3.1 ships the three fixes this branch surfaced upstream: - Borrowed host-callback handles (vercel-labs/quickjs-wasi#31): the trampoline's this/argv handles are scope-exempt, making vm.withScope safe around host callbacks. The serde's module-owned pass-disposal apparatus (passDisposal/track/runWithPassDisposal and ~18 track() wraps) is replaced by withScope in serialize/deserialize — simpler, and strictly more complete: every handle constructed during the pass is swept, not just the ones our creation funnels saw. Bench parity confirmed (within ~10% on the 50k-node extreme case, unchanged elsewhere; still 2.6-100x over the in-VM codec). - Real memoryLimit accounting (vercel-labs/quickjs-wasi#33): the engine's 256 MB VM ceiling now actually bounds retained guest allocations (usable-size was 0 on wasm32-wasi before, so the limit never accumulated). - Unregistered host callbacks throw (vercel-labs/quickjs-wasi#34): guest calls into missing callbacks fail loud instead of silently returning undefined — protection this engine wants for snapshot-restore re-registration bugs. Also merges origin/main (undici 7.29.0).
Evaluating the workflow bundle dominates VM startup (~74ms of a ~77ms boot for the 1.3MB e2e bundle) and full event replay pays it on EVERY invocation — a large share of the quickjs engine's TTFS gap vs node:vm, where V8 compiles the same script in single-digit ms. The bundle is identical across all runs of a deployment, so the engine now hydrates one VM per function instance (bootstrap + bundle eval), snapshots its memory, and starts every invocation with QuickJS.restore (~3ms) instead of re-evaluating. Measured on the real generated e2e flow bundle (154 workflows), boot to first suspension: fresh 79.4ms -> restored 3.2ms (24.8x). First invocation pays hydrate+restore (85.8ms, ~= one fresh boot); every subsequent invocation — including every replay wake — gets the discount. Determinism: replay requires module-scope user code to observe the run-seeded PRNG and deterministic clock, and a restored heap carries whatever module scope computed at hydrate time. The hydrate therefore runs with draw-counting placeholder host fns and a read-counting clock; a bundle that consumed either is marked ineligible and every invocation falls back to fresh evaluation (node:vm-parity semantics preserved exactly). When the gate passes, restore is byte-equivalent to fresh eval: the per-run host fns (random / __generateNanoid / __generateUlid) re-register by NAME on the restored VM before the workflow body runs, so the seeded draw sequence — and every correlationId — is identical. Pinned by a parity test that feeds Math.random() into a step input and byte-compares the serialized ops across fresh, first-restore and cached-restore invocations. Cache: per function instance, keyed on the bundle string (reference-stable in generated flow routes), promise-deduped for concurrent first invocations, capped at 4 entries; hydrate rejections evict for retry while eval failures cache as ineligible (the fresh path re-evaluates and surfaces the real, source-mapped error). Kill switch: WORKFLOW_QUICKJS_BASELINE_SNAPSHOT=0.
🦋 Changeset detectedLatest commit: 5848f8a The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
…the snapshot The intrinsics-replacement gate was structurally losing: its own post-eval probe executed guest-reachable code (CAPTURE_INTRINSICS calls Object.getOwnPropertyDescriptor / Object.getPrototypeOf), those dependencies were not in the identity signature, and a module-scope stateful wrapper around them both evaded detection AND had its side effects baked into the snapshot — fresh returned 0 from the reviewer's counter repro while restore returned the probe's call count. Replace detection with prevention: ALL guest-touching serde initialization (intrinsics capture, branded samples, well-known symbol lookups) is bundled into one CAPTURE_ROOT expression evaluated in the baseline VM BEFORE the bundle — the same capture-before-user-code ordering the fresh path has always had. The container handle's box lives in the snapshot's linear memory, its raw pointer rides the BaselineEntry, and every restored VM re-adopts it (adoptSerdeRoot) — serde init then performs only plain-data property reads and C-level classId reads: NO guest code executes after user code has run, on either path. Consequences: - the identity-signature gate and its expression-created skip-list are deleted (nothing to detect — module-scope intrinsic patching is now HARMLESS on the snapshot path, not merely detectable) - polyfill bundles become ELIGIBLE for the optimization and serialize through pristine intrinsics identically on both paths (test flipped from gating to byte-equality) - process.env injection converted from guest-source eval to handle-based installProcessEnv (captured Object.freeze + vm.hostToHandle): the old evalCode ran JSON.parse post-eval on the restore path only, the same observable-divergence class - the reviewer's stateful-wrapper repro is a regression test: the counter must be zero and identical across fresh and restored invocations
3.4.0 ships lossless string transport (vercel-labs/quickjs-wasi#35 — found by this PR's review cycle), so the SDK-side detection and escape machinery is deleted wholesale: - guestString (length + U+FFFD detection, in-VM escape fallback) — plain toString() is lossless now - the escapeString / hasOwnCall / jsonStringify / objectKeys captured intrinsics - keyNeedsHandleLookup and the handle-keyed get/hasOwn routing — the library routes inexpressible keys itself - shapeOf's guest-key verification pass (count/duplicate/U+FFFD scans) — enumeration is lossless Net ~130 lines and four captured intrinsics removed; the serde now uses the plain quickjs-wasi surface everywhere. Test honesty fix that 3.4.0 forced: the earlier lone-surrogate round-trip tests passed only via mutual corruption — the pre-3.4.0 lossy host→guest transport corrupted the guest comparison literals identically to the wire. With an honest transport they exposed that the WIRE itself (devalue emits lone surrogates raw; the wire is UTF-8) degrades lone surrogates to U+FFFD — in the node engine's reference codec exactly as here, verified. Bug-compatible parity is the load-bearing property (event logs replay across engines), so those tests now assert byte parity with the reference codec plus guest-observed equality with the reference codec's own round trip; NULs are devalue-escaped and asserted to survive exactly. Wire-level surrogate preservation is a product-wide devalue/UTF-8 question, tracked separately from this engine.
…s-baseline-snapshot; adopt official snapshot-portable handles Conflict resolution: objectFreeze (this branch's installProcessEnv dependency) is kept; the deleted escapeString/hasOwnCall intrinsics stay deleted (3.4.0's lossless transport made them redundant). adoptSerdeRoot now uses quickjs-wasi 3.4.0's official exportHandle/importHandle API (vercel-labs/quickjs-wasi#36) instead of constructing JSValueHandle from a raw box pointer: the token is minted by exportHandle at hydrate (with VM-ownership/liveness/borrowed validation), and importHandle duplicates the value per restored VM — independently owned handles, malformed-token validation, and no reliance on internal box-offset knowledge.
karthikscale3
approved these changes
Aug 6, 2026
karthikscale3
left a comment
karthikscale3
left a comment
Contributor
There was a problem hiding this comment.
ai review: Re-checked the latest head, pre-eval serde capture and snapshot-handle adoption, fresh-vs-restore regression coverage, and current CI. No remaining code-review blockers found; the Windows failure is an unrelated HMR timeout.
…shot # Conflicts: # packages/core/src/runtime/quickjs-runtime.ts # packages/core/src/runtime/quickjs-serde.ts
Contributor
📊 Workflow Benchmarkscommit Backend:
📈 STSO distribution vs main (inline / queue-hop histograms)1020 steps (inline) Cumulative STSO time: main 204348ms → this run 126314ms (Δ -78034ms, -38%) ℹ️ Metric definitions & methodologyThe collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: Best/P75/P90/P99 deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost 🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000 All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor ( Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the |
Contributor
|
No backport to This is a startup performance optimization (snapshot/restore of a hydrated QuickJS VM) plus new configuration surface ( To override, re-run the Backport to stable workflow manually via |
This was referenced Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #3263 (uses the host-serde engine's per-VM intrinsics capture, which re-runs cleanly against a restored heap).
Problem
Evaluating the workflow bundle dominates QuickJS VM startup — ~74ms of a ~77ms boot for the real generated e2e flow bundle (1.3MB, 154 workflows) — and full event replay pays it on every invocation, not just the first. This is a large share of the engine's TTFS gap vs node:vm, where V8 compiles the same script in single-digit milliseconds.
Approach
The bundle is identical across all runs of a deployment. The engine now hydrates one VM per function instance (bootstrap + bundle eval), snapshots its memory (quickjs-wasi
snapshot()/restore()), and starts every invocation by restoring the snapshot instead of re-evaluating:Every replay wake gets the discount, not just run start.
Determinism
Replay requires module-scope user code to observe the run-seeded PRNG and the deterministic clock; a restored heap carries whatever module scope computed at hydrate time. Two safeguards:
random/__generateNanoid/__generateUlidcallbacks re-register by NAME on the restored VM before the workflow body runs (quickjs-wasi restore semantics), so the seeded draw sequence — and every correlationId — is byte-identical to fresh eval. Pinned by a parity test that feedsMath.random()into a step input and byte-compares serialized pending ops across fresh, first-restore, and cached-restore invocations.Mechanics
WORKFLOW_QUICKJS_BASELINE_SNAPSHOT=0Testing
WORKFLOW_VM=quickjswith the snapshot path liveFollow-ups