An experimental closure-threading "JIT" for chidori-js

Status: RETIRED — implemented, measured, removed. The closure-threading backend described below was built, landed on by default with the full conformance/replay/differential gate green, measured (§5), and then removed on the same branch. The measurements are the reason: 1.01–1.11× on the micro-benchmarks — mostly under the environment's noise floor — in exchange for one boxed closure per bytecode op of memory, a first-activation compile cost, and a permanent second dispatch path that every future engine change would have to keep semantically identical to step. Meanwhile the cache work in docs/resume-performance.md (transpile ~3.9 ms/source, proto + regexp caches) delivered strictly larger wins for near-zero complexity. This is the same call quickjs-ng made when it removed its inline caches ("sometimes faster, sometimes slower, always more memory hungry"). The document is kept as the record of the experiment — the design, the determinism argument, and the numbers — so the next person tempted by a dispatch-side JIT can start from data. The implementation remains in this branch's git history (Add experimental closure-threading JIT for chidori-js).

This was a deliberate experiment that ran against the recommendation in docs/interpreter-optimization.md — §2 and §11.5 there explain why a JIT is the wrong tool for this engine; the experiment confirmed it empirically.

Current thinking on a JIT lives in docs/js-performance-roadmap.md §4 (2026-07): what a forbid(unsafe_code)-compatible tier could and couldn't be, why the determinism objection is narrower than §2 above states, and the product trigger under which a Cranelift baseline tier would become rational.

1. TL;DR

chidori-js runs JavaScript by lowering an oxc AST to stack-machine bytecode (compiler.rs) and interpreting it with a switch-dispatch loop (exec.rs::run_frame → step). A conventional native-code JIT is rejected in interpreter-optimization.md for three standing reasons: it needs unsafe executable memory (or a heavyweight backend like Cranelift), it fights the deterministic-replay contract with compile/deopt timing nondeterminism, and a measured agent-replay benchmark shows JS execution is well under 1 % of an agent's live wall-clock, so native codegen's end-to-end payoff is negligible.

All three reasons still hold. So this branch does not build a native JIT. It builds a closure-threading execution backend — the most a "JIT" can be here while keeping every load-bearing invariant intact:

• Zero unsafe. It is ordinary safe Rust: a Vec of boxed closures, one per bytecode op. No executable memory, no transmuted function pointers.
• No new dependencies. std only.
• Byte-identical deterministic replay. The JIT is a pure performance side effect. A runtime toggle (Vm::jit_enabled) selects backend, and the test suite asserts the two backends produce identical output, errors, and host-call journals (tests/jit.rs, plus the whole existing suite running JIT-on by default — including tests/replay.rs record→replay byte-identity).

It compiles each function once, on first activation, into a closure thread cached on its FuncProto. The hot ops (loads, cell/local access, arithmetic, comparisons, branches) run inline with operands pre-decoded into the closure; the long tail (calls, property access, generators, async, classes, super, with, exceptions, iteration) delegates to the unchanged step, so it can never drift from the reference interpreter.

2. Why "closure-threading," and why it's the honest ceiling here

interpreter-optimization.md §4 lays out the dispatch-optimization family and the Rust constraint: classic direct/token threading needs computed goto, subroutine threading needs guaranteed tail calls, and stable, unsafe-free Rust has neither. That's why that doc pursues fewer/cheaper dispatches (fusion, loop cleanup) rather than literal threading.

Closure-threading is the one member of that family that is expressible in safe Rust. Instead of an array of opcodes dispatched through one central match, the program becomes an array of closures, each capturing its decoded operands. Dispatch is a call through a per-ip function pointer rather than a re-entry of the giant match. It does not need computed goto or TCO because the driver loop still owns control flow (each closure returns a Ctl, the loop advances ip) — so the native stack does not grow per op.

This is genuinely "compiling to native code": the closures are compiled by rustc ahead of time, and the per-function "compile" step selects and parameterizes them. It is not speculative native codegen (no type guards, no deopt) — and that's the point: speculation + deopt is exactly the JIT machinery interpreter-optimization.md §2 calls out as the determinism-breaking, bug-prone part. Skipping it is what keeps replay byte-identical.

What it can and cannot win:

• Removes: per-op operand decoding (operands are pre-captured), and, for the specialized ops, the trip through the central match and the construction of a throwaway Result<Ctl, Value> payload. Some per-op work is also hoisted to compile time — e.g. a cell's stable_cells membership is resolved once at compile instead of a Vec::contains on every InitCell (12 % of executed ops in the Phase-0 survey).
• Does not remove: the operand-stack traffic itself (this is still a stack VM, not a register VM — that's interpreter-optimization.md Phase 4), nor the cost of the heavy ops, which still flow through step.

So this is the interpreter-dispatch ceiling reachable without a new instruction encoding, unsafe, or speculation. It overlaps with — and composes on top of — the fusion work already landed (Phase 2): fused superinstructions like CmpBranchFalse and LoadCellConst are themselves specialized in the thread.

3. Design

All code is in crates/chidori-js/src/jit.rs (+ small seams elsewhere).

3.1 The compiled form

type OpFn    = Box<dyn Fn(&mut Vm, &mut Frame) -> Result<Ctl, Value>>;
struct JitThread { ops: Vec<OpFn>, specialized: u32, fallback: u32 }

JitThread.ops is index-parallel to proto.code: exactly one closure per bytecode op, at the same index. Because indices are preserved, every jump target (absolute code offset) carries over unchanged — no remapping, no second encoding, no interaction with the fusion pass's offset remap.

Each closure returns the same Result<Ctl, Value> the switch interpreter's step returns for that op. That is the key to a minimal, low-risk integration: the surrounding run_frame driver is untouched and handles a closure's result with byte-identical control flow whether it came from a closure or from step.

3.2 Specialized vs. delegated ops

lower(proto, op) returns either a specialized closure or a fallback. The specialized set is the hot core the Phase-0 survey identified (interpreter-optimization.md §11.2): constants/literals, LoadArg, locals (LoadLocal/StoreLocal), cells (LoadCell/StoreCell/InitCell/ InitCellTdz/LoadCellConst, with the identical TDZ check), upvalues, stack manipulation (Pop/Dup/Swap/Rot3), all arithmetic/bitwise/unary ops, all eight comparisons, the branches (Jump/JumpIf*/CmpBranch{False,True}/peek jumps), and typeof.

Every specialized arm is a transcription of the corresponding step arm that reuses the identical helper — op_add, bin_arith, less_than, loose_equals, strict_equals, unary_arith, to_boolean, to_number, to_numeric, const_val. Reusing the same helper means coercion order, thrown error type/site, and ±0/NaN/BigInt/valueOf ordering are identical by construction, not by careful re-implementation.

Everything else — calls, new, property get/set, object/array literals, generators, await/yield, with, private elements, super, exceptions, the iterator protocol, modules, eval — lowers to a single fallback closure that clones the op and calls vm.step(frame, &op). The long tail therefore is the reference interpreter; it cannot diverge.

3.3 Caching & lifetime

The thread is cached on the proto:

struct FuncProto { …, jit: JitCache /* RefCell<Option<Rc<JitThread>>> */ }

JitCache::get_or_compile compiles on first activation and clones out the Rc, releasing the borrow before returning — so direct recursion (e.g. fib calling fib) never double-borrows the cell. Each proto (top-level and nested) owns an independent cache and compiles lazily on its own first call. The cache is a pure side effect: never serialized, never observed by the journal, rebuilt from scratch on every fresh Vm. (A FuncProto is immutable and shared by Rc; the cache is its only interior-mutable field.)

3.4 Integration

run_frame is the single chokepoint for all execution — ordinary calls, generator/async resume, module evaluation, promise reactions. The only change to it: after cloning the per-frame proto Rc, fetch the proto's thread (when jit_enabled) and source each op's result from it:

let stepped = match &jit {
    Some(thread) => (thread.ops[ip])(self, &mut frame),
    None         => self.step(&mut frame, &proto.code[ip]),
};
match stepped { /* … unchanged Ctl handling … */ }

Everything else in run_frame — the op budget, the amortized interrupt poll, the pending_throw/pending_return resume handling, the module-capture hook, the suspend variants, frame recycling — is byte-identical. Because the integration is at the single dispatch point that the whole engine funnels through, every caller (sync calls, generators, async, modules) gets the JIT for free, and the suspend/resume machinery is unaffected: a frame's representation (ip, stack, locals, cells) is unchanged, so a frame suspended under the JIT resumes identically (under either backend).

4. Determinism & the toggle (why "on by default" is safe)

The determinism contract (interpreter-optimization.md §6) requires that an optimization change timing and internal representation only — never results, error identity, enumeration order, microtask/promise ordering, or host-call sequencing. This JIT satisfies it the same way the fusion pass does, and proves it the same way:

• Vm::jit_enabled selects the backend at runtime. It is the differential oracle: the same program run both ways must produce identical observable behavior and an identical journal.
• tests/jit.rs::jit_matches_interpreter runs a broad corpus (numeric loops, every operator, coercion + toPrimitive, BigInt, closures/upvalues, TDZ, objects/arrays/Map/Set, strings/templates, try/catch/finally, switch, labeled break/continue, optional chaining, for-of/for-in, generators, destructuring, classes + private fields + super, mid-expression throws, async/await + microtask ordering, arguments) through both backends and asserts the (threw, console, error) triples are byte-identical.
• The entire existing suite runs JIT-on by default and is green, including tests/replay.rs (record→replay byte-identity, suspend→persist→resume), tests/async_gen.rs, tests/gc.rs, and the DOM suites.

Because the unspecialized long tail literally calls step, and the specialized ops reuse step's own helpers, the surface where divergence could even occur is small and directly differential-tested.

5. Results

Measurement caveat (unchanged from interpreter-optimization.md §7.6): the cloud dev/CI container cannot reliably resolve wall-clock deltas below ~10–15 %; re-running the same binary swings several percent. So the load-bearing result here is the deterministic dispatch proxy, with wall-clock reported only as indicative and min-of-N.

5.1 Deterministic dispatch proxy

cargo run -q --release --example jit_stats -p chidori-js reports, per workload, how many ops across the whole proto tree are specialized vs. delegated. specialized is the count of central-match dispatches removed per pass over the code — reproducible and environment-independent.

Measured here (whole proto tree, as-emitted; cargo run --release --example jit_stats):

~70–80 % of every workload's ops are specialized (run inline, off the central match). The data-movement + arithmetic + compare/branch core that the Phase-0 survey put at the bulk of executed ops (interpreter-optimization.md §11.2: ~40 % data movement alone) is specialized; the remaining fallback ops are the heavy, rarely-hot operations (calls, property access, allocation) where dispatch cost is already dwarfed by the op's own work.

5.2 Indicative wall-clock (min-of-N, JIT on vs. off)

Measured here (jit_stats, min-of-N, ms/run; criterion view: cargo bench -p chidori-js -- jit_vs_interp):

The JIT is faster on every workload, but most deltas (1–6 %) sit at or below this environment's ~10–15 % noise floor and cannot be claimed as real here. The one that clears it most is arith_loop (1.11×) — the most dispatch-bound workload (80 % specialized, a tight arithmetic loop with almost no per-op work to hide dispatch behind), which is exactly where removing dispatch should show up. fib_recursive barely moves (1.01×) because its time is dominated by call/frame setup (heavy ops on the fallback path), not dispatch. These numbers are directionally consistent with closure-threading theory; resolving the small wins quantitatively needs a quiet, frequency-pinned machine (§7.6).

5.3 Honest read

Per the agent-replay measurement (interpreter-optimization.md §11.5), JS execution is <1 % of an agent's live wall-clock, so this changes live agent latency negligibly — exactly as that doc predicts for any JS-execution optimization. Where a dispatch win actually lands is the zero-host replay/test path (~97 % interpreter) and compute-heavy steps: faster replay and CI, and faster tight numeric loops. That, not live latency, is the honest justification — the same conclusion the no-JIT analysis reached, now with a JIT in hand to measure rather than assume.

6. Limitations & possible next steps

• Still a stack VM. The closures push/pop the operand stack exactly as the bytecode dictates; the ~40 %-data-movement traffic is dispatched more cheaply but not eliminated. Eliminating it needs a register encoding (interpreter-optimization.md Phase 4) — orthogonal to this and composable.
• Boxed-closure indirection. Each op is a Box<dyn Fn>: one heap allocation per op at compile time and an indirect call per dispatch. The indirect call is the same shape of branch as the central match, so part of the predicted win is decode/operand savings rather than branch-prediction wins; this is why the proxy (dispatches removed) is the honest headline, not a branch-mispredict claim.
• No basic-block fusion (yet). The biggest closure-threading win — compiling a straight-line basic block into one closure that keeps intermediates in locals and bypasses the operand stack — is not done here, because doing it while preserving exact mid-block throw/coercion semantics is delicate. It is the natural follow-up if the proxy/timing justify it.
• Compile cost & memory. Compiling allocates one boxed closure per op on first activation. For run-once code this is pure overhead; it pays off only across repeated activations (loops, recursion, repeated calls, replay). A call-count threshold (compile only after N activations) would avoid the cold-code cost — a standard "tiering" refinement left out here for simplicity.

7. Where things live

• crates/chidori-js/src/jit.rs — the closure-threading compiler, the per-op lower, the JitThread/JitCache types.
• crates/chidori-js/src/exec.rs — run_frame dispatch integration; step, Ctl, const_val, bin_arith exposed pub(crate) for reuse.
• crates/chidori-js/src/bytecode.rs — FuncProto::jit cache field.
• crates/chidori-js/src/vm.rs — Vm::jit_enabled toggle.
• crates/chidori-js/tests/jit.rs — differential (toggle-equivalence) + structural tests.
• crates/chidori-js/benches/execution.rs — jit_vs_interp criterion group.
• crates/chidori-js/examples/jit_stats.rs — deterministic dispatch proxy + indicative timing.

8. References

• docs/interpreter-optimization.md — the no-JIT analysis, the Phase-0 op-frequency survey, the agent-replay <1 % result, and the ~10–15 % noise-floor caveat that frame this experiment.
• docs/replay.md — the deterministic-replay durability model the toggle-equivalence test protects.
• docs/conformance.md — the Test262 gate.