Commit graph

140 commits

Author SHA1 Message Date
Stan Lo
2b16f27a35
ZJIT: Fix ObjToString rewrite (#14196)
ZJIT: Fix ObjToString rewrite

Currently, the rewrite for `ObjToString` always replaces it with a
`SendWithoutBlock(to_s)` instruction when the receiver is not a
string literal. This is incorrect because it calls `to_s` on the
receiver even if it's already a string.

This change fixes it by:
- Avoiding the `SendWithoutBlock(to_s)` rewrite
- Implement codegen for `ObjToString`
2025-08-13 13:03:26 -07:00
Max Bernstein
ad12db4b3d ZJIT: Only validate HIR in debug mode 2025-08-13 13:52:41 -04:00
Stan Lo
df7d9812cc
ZJIT: Prepare non-leaf calls for SetGlobal (#14197)
Some checks failed
Windows / Windows 11-arm/Visual C++ (btest test-basic test-tool) (push) Waiting to run
Windows / Windows 2022/Visual C++ 2022 (check) (push) Waiting to run
Windows / Windows 2025/Visual C++ 2022 (check) (push) Waiting to run
Windows / Windows 2025/Visual C++ 2022 (test-bundled-gems) (push) Waiting to run
Windows / result (push) Blocked by required conditions
Ubuntu on WSL / wsl (push) Waiting to run
Annocheck / test-annocheck (push) Failing after 59s
Check Dependencies / Dependency checks (push) Failing after 56s
Misc / Miscellaneous checks (push) Failing after 1m4s
CodeQL / Analyze (push) Failing after 56s
Compilations / omnibus compilations, trigger (push) Failing after 52s
Update default gems list / Update default gems list (push) Has been skipped
parse.y / make (check) (push) Failing after 56s
parse.y / make (test-bundled-gems) (push) Failing after 56s
parse.y / make (test-bundler-parallel) (push) Failing after 57s
WebAssembly / make (map[debugflags: name:O2 optflags:-O2 wasmoptflags:-O2]) (push) Failing after 1m1s
Compilations / omnibus compilations, #5 (push) Has been skipped
Compilations / omnibus compilations, #6 (push) Has been skipped
Compilations / omnibus compilations, #8 (push) Has been skipped
Compilations / omnibus compilations, #10 (push) Has been skipped
Compilations / omnibus compilations, #11 (push) Has been skipped
BASERUBY Check / BASERUBY (push) Failing after 52s
Compilations / omnibus compilations, #1 (push) Has been skipped
Compilations / omnibus compilations, #2 (push) Has been skipped
Compilations / omnibus compilations, #3 (push) Has been skipped
Compilations / omnibus compilations, #4 (push) Has been skipped
Compilations / omnibus compilations, #7 (push) Has been skipped
Compilations / omnibus compilations, #9 (push) Has been skipped
Compilations / omnibus compilations, #12 (push) Has been skipped
Compilations / omnibus compilations, result (push) Successful in 1m14s
When trace_var is used, setting a global variable can cause exceptions
to be raised. We need to prepare for that.
2025-08-12 17:39:46 -07:00
Takashi Kokubun
231407c251
ZJIT: Avoid compiling failed ISEQs repeatedly (#14195) 2025-08-12 13:40:42 -07:00
Max Bernstein
998be6b3a4
ZJIT: Add flag to disable the HIR optimizer (#14181)
Also add a check in the bisect script that can assign blame to the HIR
optimizer.
2025-08-12 13:00:22 -04:00
Stan Lo
e29d333454
ZJIT: Implement concatstrings insn (#14154)
Co-authored-by: Alexander Momchilov <alexander.momchilov@shopify.com>
2025-08-11 15:07:26 -07:00
Takashi Kokubun
319550527f
ZJIT: Add compile/profile/GC/invalidation time stats (#14158)
Co-authored-by: Stan Lo <stan001212@gmail.com>
2025-08-11 13:21:45 -07:00
Takashi Kokubun
eb931a09c5
ZJIT: Fix "memory operand with non-register base" (#14153) 2025-08-08 11:24:39 -07:00
Takashi Kokubun
4fef87588a
ZJIT: Remove the need for unwrap() on with_num_bits() (#14144)
* ZJIT: Remove the need for unwrap() on with_num_bits()

* Fix arm64 tests

* Track the caller of with_num_bits

Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>

---------

Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
2025-08-07 16:56:27 -07:00
Stan Lo
2edc944702
ZJIT: Implement defined? codegen for non-yield calls (#14101) 2025-08-07 15:41:05 -07:00
Stan Lo
d25eb1eb5c
ZJIT: Optimize class guards by directly reading klass field (#14136)
Replace `rb_yarv_class_of` call with:
- a constant check for special constants (nil, fixnums, symbols, etc)
- a check for false
- direct memory read at offset 8 for regular heap objects for the class check
2025-08-07 15:38:02 -07:00
Takashi Kokubun
96c9e1e93a
ZJIT: Remove GC offsets overwritten by invalidation (#14102)
ZJIT: Remove GC offsts overwritten by invalidation
2025-08-07 15:30:02 -07:00
Max Bernstein
a260bbc550
ZJIT: Set PC before StringCopy (#14141)
ZJIT: Set PC before StringCopy

This function allocates.
2025-08-07 18:28:10 +00:00
Stan Lo
4a70f946a7
ZJIT: Implement SingleRactorMode invalidation (#14121)
* ZJIT: Implement SingleRactorMode invalidation

* ZJIT: Add macro for compiling jumps

* ZJIT: Fix typo in comment

* YJIT: Fix typo in comment

* ZJIT: Avoid using unexported types in zjit.h

`enum ruby_vminsn_type` is declared in `insns.inc` and is not exported.
Using it in `zjit.h` would cause build errors when the file including it
doesn't include `insns.inc`.
2025-08-06 13:51:41 -07:00
Takashi Kokubun
53b0462841
ZJIT: Add helpers to prepare for C calls (#14100) 2025-08-04 14:44:51 -07:00
Stan Lo
85510fc2ff ZJIT: Support annotating builtin functions
This allows us to annotate builtin functions with their return type.
2025-08-01 17:44:39 -07:00
Takashi Kokubun
543f8dcad3
ZJIT: Add the ISEQ name to Block asm comments (#14070) 2025-07-31 14:28:08 -07:00
Takashi Kokubun
12306c0c6f
ZJIT: Stub JIT-to-JIT calls (#14052) 2025-07-31 12:57:59 -07:00
Max Bernstein
0f7ee8e7a4 ZJIT: Get rid of CallInfo 2025-07-30 10:36:15 -07:00
Max Bernstein
7b10dbd55f ZJIT: Remove catch-all case to make it clearer what's unimplemented 2025-07-30 10:36:15 -07:00
Max Bernstein
ade4558602
ZJIT: Catch more failed recursive compilations (#14042)
Untangle the logic a bit and specifically:

* catch `gen_entry` failures
* don't set `start_ptr` until all recursive calls succeed

Co-authored-by: Alan Wu <alanwu@ruby-lang.org>
2025-07-29 16:36:25 -04:00
Max Bernstein
039f4139f8
ZJIT: Create delta debugging script to narrow JIT failures (#14041)
Add support for `--zjit-allowed-iseqs=SomeFile` and
`--zjit-log-compiled-iseqs=SomeFile` so we can restrict and inspect
which ISEQs get compiled.

Then add `jit_bisect.rb` which we can run to try and narrow a failing
script. For example:

    plum% ../tool/zjit_bisect.rb ../build-dev/miniruby "test.rb"
    I, [2025-07-29T12:41:18.657177 #96899]  INFO -- : Starting with JIT list of 4 items.
    I, [2025-07-29T12:41:18.657229 #96899]  INFO -- : Verifying items
    I, [2025-07-29T12:41:18.726213 #96899]  INFO -- : step fixed[0] and items[4]
    I, [2025-07-29T12:41:18.726246 #96899]  INFO -- : 4 candidates
    I, [2025-07-29T12:41:18.797212 #96899]  INFO -- : 2 candidates
    Reduced JIT list:
    bar@test.rb:8
    plum%

We start with 4 compiled functions and shrink to just one.
2025-07-29 19:43:14 +00:00
Takashi Kokubun
b22eb0e468
ZJIT: Add --zjit-stats (#14034) 2025-07-29 10:00:15 -07:00
Stan Lo
a0d0b84bad
ZJIT: Support invalidating constant patch points (#13998) 2025-07-28 14:48:41 -07:00
Max Bernstein
3f22434e1a ZJIT: Fix land race 2025-07-28 15:36:20 -04:00
Stan Lo
043489abc2
ZJIT: Inline guard type checks for some built-in types (#14017)
This implements similar fast-path guard type checks as YJIT.
2025-07-28 15:32:32 -04:00
Alan Wu
ff428b4dd0 ZJIT: Keep a frame pointer and use it for memory params
Previously, ZJIT miscompiled the following because of native SP
interference.

    def a(n1,n2,n3,n4,n5,n6,n7,n8) = [n8]
    a(0,0,0,0,0,0,0, :ok)

Commented problematic disassembly:

    ; call rb_ary_new_capa
    mov x0, #1
    mov x16, #0x1278
    movk x16, #0x4bc, lsl #16
    movk x16, #1, lsl #32
    blr x16
    ; call rb_ary_push
    mov x1, x0
    str x1, [sp, #-0x10]! ; c_push() from alloc_regs()
    mov x0, x1            ; arg0, the array
    ldur x1, [sp]         ; meant to be arg1=n8, but sp just moved!
    mov x16, #0x3968
    movk x16, #0x4bc, lsl #16
    movk x16, #1, lsl #32
    blr x16

Since the frame pointer stays constant in the body of the function,
static offsets based on it don't run the risk of being invalidated by SP
movements.

Pass the registers to preserve through Insn::FrameSetup. This allows ARM
to use STP and waste no gaps between EC, SP, and CFP.

x86 now preserves and restores RBP since we use it as the frame pointer.
Since all arches now have a frame pointer, remove offset based SP
movement in the epilogue and restore registers using the frame pointer.
2025-07-28 15:30:50 -04:00
Alan Wu
41149a96ef ZJIT: Fix clobbering register for self in gen_entry_params()
Previously, for 8+ params we wound up clobbering the self param when
putting the last param in memory in the JIT entry point:

    # ZJIT entry point: a@../test.rb:5
    <snip>
    ldur x0, [x19, #0x18]
    # set method params: 8
    ldur x1, [x21, #-0x58]
    ldur x2, [x21, #-0x50]
    ldur x3, [x21, #-0x48]
    ldur x4, [x21, #-0x40]
    ldur x5, [x21, #-0x38]
    ldur x11, [x21, #-0x30]
    ldur x12, [x21, #-0x28]
    ldur x0, [x21, #-0x20]
    stur x0, [sp, #-0x20]
    bl #0x11e17018c

Doing the memcpys for parameters in memory first avoids this clobbering.

    # set method params: 8
    ldur x0, [x21, #-0x20]
    stur x0, [sp, #-0x20]
    ldur x12, [x21, #-0x28]
    ldur x11, [x21, #-0x30]
    ldur x5, [x21, #-0x38]
    ldur x4, [x21, #-0x40]
    ldur x3, [x21, #-0x48]
    ldur x2, [x21, #-0x50]
    ldur x1, [x21, #-0x58]
    ldur x0, [x19, #0x18]
2025-07-23 13:29:03 -04:00
Alan Wu
33363030e1 ZJIT: Use rb_vm_env_write() for hir::Insn::SetLocal
We weren't firing write barriers before when writing to imemo/env
objects. Wbcheck caught this with test/ruby/test_refinement.rb:

    ruby -v: ruby 3.5.0dev (2025-07-22T17:05:58Z wbcheck 2569a80954) +ZJIT dev +PRISM +GC[wbcheck] [x86_64-linux]
    WBCHECK ERROR: Missed write barrier detected!
      Parent object: 0x558de9f4e6e0 (wb_protected: true)
        rb_obj_info_dump: 0x0000558de9f4e6e0 T_IMEMO/<env>
      Reference counts - snapshot: 3, writebarrier: 0, current: 4, missed: 1
      Missing reference to: 0x558decf37c30
        rb_obj_info_dump: 0x0000558decf37c30 method/UnboundMethod method

    WBCHECK SUMMARY: Found 1 objects with missed write barriers (1 total violations)
2025-07-22 18:04:28 -04:00
Alan Wu
e77eee96a3 ZJIT: Load return value before frame teardown
Or else the following returns garbage since it loads after
moving SP. Prior bad disassembly:

    def a(n1,n2,n3,n4,n5,n6,n7,n8) = n8
    a(1,1,1,1,1,1,1,0)

    # Block: bb0(v0, v1, v2, v3, v4, v5, v6, v7, v8)
    stp x29, x30, [sp, #-0x10]!
    mov x29, sp
    # bump C stack pointer
    sub sp, sp, #0x10
    # Insn: v10 Return v8
    # pop stack frame
    adds x19, x19, #0x38
    stur x19, [x20, #0x10]
    # restore C stack pointer
    add sp, sp, #0x10
    mov sp, x29
    ldp x29, x30, [sp], #0x10
    ldur x0, [sp]
    ret
2025-07-21 23:09:42 -04:00
Stan Lo
8df61bfc92
ZJIT: Support invalidating on method redefinition (#13875)
ZJIT: Support invalidating method redefinition

This commit adds support for the MethodRedefined invariant to be invalidated
when a method is redefined.

Changes:
- Added CME pointer to the MethodRedefined invariant in HIR
- Updated all places where MethodRedefined invariants are created to
    include the CME pointer
- Added handling for MethodRedefined invariants in gen_patch_point to
    call track_cme_assumption, which registers the patch point for
    invalidation when rb_zjit_cme_invalidate is called

This ensures that when a method is redefined, all JIT code that
depends on that method will be properly invalidated.
2025-07-18 15:36:51 +00:00
Max Bernstein
30b1368829
ZJIT: Create perf map files for profilers (#13941)
This lets us ZJIT compiled functions show up in the profiles of, say,
perf, or samply.

Fix https://github.com/Shopify/ruby/issues/634
2025-07-17 22:36:44 +00:00
Takashi Kokubun
04d43e1870
ZJIT: Give up JIT-to-JIT calls for 6+ args (#13939) 2025-07-17 12:22:26 -07:00
John Hawthorn
cb33f22f5b ZJIT: Precise GC writebarriers
This issues writebarriers for objects added via gc_offsets or by
profiling. This may be slower than writebarrier_remember, but we would
like it to be more debuggable.

Co-authored-by: Max Bernstein <ruby@bernsteinbear.com>
Co-authored-by: Stan Lo <stan001212@gmail.com>
2025-07-17 11:50:13 -07:00
Takashi Kokubun
39b844e064 Tweak the comment on mark_all_executable() a little [ci skip] 2025-07-17 11:42:07 -07:00
Takashi Kokubun
ff77473acb
ZJIT: Mark the code region executable on partial failures (#13937) 2025-07-17 11:39:03 -07:00
Takashi Kokubun
af1ad78bff Use a const block
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
2025-07-16 09:50:25 -07:00
Takashi Kokubun
8668e4dd07 ZJIT: Restore SP on side-exit chains 2025-07-16 09:50:25 -07:00
Takashi Kokubun
2250a66aa8 ZJIT: Fix SP alignment on JIT entry for x86_64 2025-07-16 09:50:25 -07:00
Alan Wu
b2ef33b3c5 ZJIT: Redo JIT function native stack frame layout
Previously, gen_param() access slots at `SP-x` for `x≥0` after subtracting from
SP, so it was accessing slots from above the top of the stack. Also, the
slots gen_entry_params() wrote to at entry point did not correspond to
the slots access inside the JIT function.

Redo the stack frame layout so that inside the function slots are at
`SP+x`. Write to those slots in the entry point by anticipating the size
of the frame.

Fixes test_spilled_method_args().
2025-07-15 14:47:32 -04:00
Alan Wu
50e2d58af8 ZJIT: Ban asm.load_into(Mem, ..) and avoid it in gen_entry_params()
Now that params can be in memory, this particular load_into() was
panicking with "Invalid operands for LDUR" with
test_spilled_method_args() on ARM.
Since it's documented to be for register destinations let's validate it.
2025-07-15 14:47:32 -04:00
Alan Wu
3922a14a22 ZJIT: Make lir::Opnd::const_ptr take any pointer to save on casts 2025-07-14 22:11:24 -04:00
Alan Wu
7a7f128d0d ZJIT: Add a ccall macro that also adds an LIR comment
This DRYs up the `asm_comment!` + `asm.ccall` combo, and makes ccalls
have a comment by default.
2025-07-14 22:11:24 -04:00
Takashi Kokubun
e288a86692
ZJIT: Restore SP register after JIT-to-JIT call (#13882)
Co-authored-by: Alan Wu <alansi.xingwu@shopify.com>
Co-authored-by: Stan Lo <stan.lo@shopify.com>
2025-07-14 12:22:13 -07:00
Takashi Kokubun
3ec46aad37
ZJIT: Mark objects baked in JIT code (#13862) 2025-07-11 13:24:08 -07:00
Takashi Kokubun
b760afe2b7
ZJIT: Improve asm comments for side exits (#13853)
* ZJIT: Improve asm comments for side exits

* Use GuardType(Type) and GuardBitEquals(VALUE)
2025-07-11 09:49:25 -07:00
Takashi Kokubun
b1828cbbfe
ZJIT: Implement patch points on BOP redefinition (#13850)
Co-authored-by: Max Bernstein <max@bernsteinbear.com>
2025-07-10 13:40:40 -07:00
Takashi Kokubun
9ab80a7455
ZJIT: Avoid optimizing locals on eval (#13840)
* ZJIT: Avoid optimizing locals on eval

* Maintain the local state for eval
2025-07-10 12:08:09 -07:00
Takashi Kokubun
f5085c70f2
ZJIT: Mark profiled objects when marking ISEQ (#13784) 2025-07-09 16:03:23 -07:00
Stan Lo
e2a81c738c ZJIT: Optimize opt_and and opt_or instructions for Fixnum 2025-07-09 17:50:41 -04:00