Commit graph

805 commits

Author SHA1 Message Date
nagachika
d78e1a4133 merge revision(s) f65a6c090c: [Backport #20921]
Fix use-after-free in constant cache

	[Bug #20921]

	When we create a cache entry for a constant, the following sequence of
	events could happen:

	- vm_track_constant_cache is called to insert a constant cache.
	- In vm_track_constant_cache, we first look up the ST table for the ID
	  of the constant. Assume the ST table exists because another iseq also
	  holds a cache entry for this ID.
	- We then insert into this ST table with the iseq_inline_constant_cache.
	- However, while inserting into this ST table, it allocates memory, which
	  could trigger a GC. Assume that it does trigger a GC.
	- The GC frees the one and only other iseq that holds a cache entry for
	  this ID.
	- In remove_from_constant_cache, it will appear that the ST table is now
	  empty because there are no more iseq with cache entries for this ID, so
	  we free the ST table.
	- We complete GC and continue our st_insert. However, this ST table has
	  been freed so we now have a use-after-free.

	This issue is very hard to reproduce, because it requires that the GC runs
	at a very specific time. However, we can make it show up by applying this
	patch which runs GC right before the st_insert to mimic the st_insert
	triggering a GC:

	    diff --git a/vm_insnhelper.c b/vm_insnhelper.c
	    index 3cb23f06f0..a93998136a 100644
	    --- a/vm_insnhelper.c
	    +++ b/vm_insnhelper.c
	    @@ -6338,6 +6338,10 @@ vm_track_constant_cache(ID id, void *ic)
	            rb_id_table_insert(const_cache, id, (VALUE)ics);
	        }

	    +    if (id == rb_intern("MyConstant")) rb_gc();
	    +
	        st_insert(ics, (st_data_t) ic, (st_data_t) Qtrue);
	    }

	And if we run this script:

	    Object.const_set("MyConstant", "Hello!")

	    my_proc = eval("-> { MyConstant }")
	    my_proc.call

	    my_proc = eval("-> { MyConstant }")
	    my_proc.call

	We can see that ASAN outputs a use-after-free error:

	    ==36540==ERROR: AddressSanitizer: heap-use-after-free on address 0x606000049528 at pc 0x000102f3ceac bp 0x00016d607a70 sp 0x00016d607a68
	    READ of size 8 at 0x606000049528 thread T0
	        #0 0x102f3cea8 in do_hash st.c:321
	        #1 0x102f3ddd0 in rb_st_insert st.c:1132
	        #2 0x103140700 in vm_track_constant_cache vm_insnhelper.c:6345
	        #3 0x1030b91d8 in vm_ic_track_const_chain vm_insnhelper.c:6356
	        #4 0x1030b8cf8 in rb_vm_opt_getconstant_path vm_insnhelper.c:6424
	        #5 0x1030bc1e0 in vm_exec_core insns.def:263
	        #6 0x1030b55fc in rb_vm_exec vm.c:2585
	        #7 0x1030fe0ac in rb_iseq_eval_main vm.c:2851
	        #8 0x102a82588 in rb_ec_exec_node eval.c:281
	        #9 0x102a81fe0 in ruby_run_node eval.c:319
	        #10 0x1027f3db4 in rb_main main.c:43
	        #11 0x1027f3bd4 in main main.c:68
	        #12 0x183900270  (<unknown module>)

	    0x606000049528 is located 8 bytes inside of 56-byte region [0x606000049520,0x606000049558)
	    freed by thread T0 here:
	        #0 0x104174d40 in free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x54d40)
	        #1 0x102ada89c in rb_gc_impl_free default.c:8183
	        #2 0x102ada7dc in ruby_sized_xfree gc.c:4507
	        #3 0x102ac4d34 in ruby_xfree gc.c:4518
	        #4 0x102f3cb34 in rb_st_free_table st.c:663
	        #5 0x102bd52d8 in remove_from_constant_cache iseq.c:119
	        #6 0x102bbe2cc in iseq_clear_ic_references iseq.c:153
	        #7 0x102bbd2a0 in rb_iseq_free iseq.c:166
	        #8 0x102b32ed0 in rb_imemo_free imemo.c:564
	        #9 0x102ac4b44 in rb_gc_obj_free gc.c:1407
	        #10 0x102af4290 in gc_sweep_plane default.c:3546
	        #11 0x102af3bdc in gc_sweep_page default.c:3634
	        #12 0x102aeb140 in gc_sweep_step default.c:3906
	        #13 0x102aeadf0 in gc_sweep_rest default.c:3978
	        #14 0x102ae4714 in gc_sweep default.c:4155
	        #15 0x102af8474 in gc_start default.c:6484
	        #16 0x102afbe30 in garbage_collect default.c:6363
	        #17 0x102ad37f0 in rb_gc_impl_start default.c:6816
	        #18 0x102ad3634 in rb_gc gc.c:3624
	        #19 0x1031406ec in vm_track_constant_cache vm_insnhelper.c:6342
	        #20 0x1030b91d8 in vm_ic_track_const_chain vm_insnhelper.c:6356
	        #21 0x1030b8cf8 in rb_vm_opt_getconstant_path vm_insnhelper.c:6424
	        #22 0x1030bc1e0 in vm_exec_core insns.def:263
	        #23 0x1030b55fc in rb_vm_exec vm.c:2585
	        #24 0x1030fe0ac in rb_iseq_eval_main vm.c:2851
	        #25 0x102a82588 in rb_ec_exec_node eval.c:281
	        #26 0x102a81fe0 in ruby_run_node eval.c:319
	        #27 0x1027f3db4 in rb_main main.c:43
	        #28 0x1027f3bd4 in main main.c:68
	        #29 0x183900270  (<unknown module>)

	    previously allocated by thread T0 here:
	        #0 0x104174c04 in malloc+0x94 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x54c04)
	        #1 0x102ada0ec in rb_gc_impl_malloc default.c:8198
	        #2 0x102acee44 in ruby_xmalloc gc.c:4438
	        #3 0x102f3c85c in rb_st_init_table_with_size st.c:571
	        #4 0x102f3c900 in rb_st_init_table st.c:600
	        #5 0x102f3c920 in rb_st_init_numtable st.c:608
	        #6 0x103140698 in vm_track_constant_cache vm_insnhelper.c:6337
	        #7 0x1030b91d8 in vm_ic_track_const_chain vm_insnhelper.c:6356
	        #8 0x1030b8cf8 in rb_vm_opt_getconstant_path vm_insnhelper.c:6424
	        #9 0x1030bc1e0 in vm_exec_core insns.def:263
	        #10 0x1030b55fc in rb_vm_exec vm.c:2585
	        #11 0x1030fe0ac in rb_iseq_eval_main vm.c:2851
	        #12 0x102a82588 in rb_ec_exec_node eval.c:281
	        #13 0x102a81fe0 in ruby_run_node eval.c:319
	        #14 0x1027f3db4 in rb_main main.c:43
	        #15 0x1027f3bd4 in main main.c:68
	        #16 0x183900270  (<unknown module>)

	This commit fixes this bug by adding a inserting_constant_cache_id field
	to the VM, which stores the ID that is currently being inserted and, in
	remove_from_constant_cache, we don't free the ST table for ID equal to
	this one.

	Co-Authored-By: Alan Wu <alanwu@ruby-lang.org>
2024-12-15 15:26:55 +09:00
nagachika
baa5bc16c1 merge revision(s) adaff1fc49: [Backport #19592]
[Bug #19592] Fix ext/Setup support

	After [1], using ext/Setup to link some, but not all extensions failed
	during linking. I did not know about this option, and had assumed that
	only `--with-static-linked-ext` builds can include statically linked
	extensions.

	Include the support code for statically linked extensions in all
	configurations like before [1]. Initialize the table lazily to minimize
	footprint on builds that have no statically linked extensions.

	[1]: 790cf4b6d0 "Fix autoload status of
	         statically linked extensions"
	---
	 load.c    | 28 +++++++++++++++++-----------
	 vm.c      |  3 ---
	 vm_core.h |  6 ------
	 3 files changed, 17 insertions(+), 20 deletions(-)
2024-04-07 14:41:29 +09:00
nagachika
5baf94f913 merge revision(s) 1f115f141d:
Speed up rebuilding the loaded feature index

	Rebuilding the loaded feature index slowed down with the bug fix
	for #17885 in 79a4484a07.  The
	slowdown was extreme if realpath emulation was used, but even when
	not emulated, it could be about 10x slower.

	This adds loaded_features_realpath_map to rb_vm_struct. This is a
	hidden hash mapping loaded feature paths to realpaths. When
	rebuilding the loaded feature index, look at this hash to get
	cached realpath values, and skip calling rb_check_realpath if a
	cached value is found.

	Fixes [Bug #19246]
	---
	 load.c    | 27 +++++++++++++++++++++++----
	 vm.c      |  2 ++
	 vm_core.h |  1 +
	 3 files changed, 26 insertions(+), 4 deletions(-)
2023-11-18 12:06:27 +09:00
nagachika
e777064e4b Revert "Re-apply "Ruby 3.2 - Speed up rebuilding the loaded feature index and realpath cache (#8023)" (#8252)"
This reverts commit 788b03d5ba.
2023-08-24 16:06:32 +09:00
MSP-Greg
788b03d5ba
Re-apply "Ruby 3.2 - Speed up rebuilding the loaded feature index and realpath cache (#8023)" (#8252)
* Re-apply "Ruby 3.2 - Speed up rebuilding the loaded feature index and realpath cache (#8023)"

* [CI] mingw.yml - remove IBM437 encoding for test-all, use cmd shell for test & test-all

* Skip failing test on mingw with readline.so

Co-authored-by: nagachika <nagachika@ruby-lang.org>

---------

Co-authored-by: nagachika <nagachika@ruby-lang.org>
2023-08-21 23:33:47 +00:00
nagachika
1c76244698 Revert "Ruby 3.2 - Speed up rebuilding the loaded feature index and realpath cache (#8023)"
This reverts commit 8346d1630b.
It seems break CI on MinGW.
1544745598
2023-07-29 17:58:42 +09:00
Kerem Kat
8346d1630b
Ruby 3.2 - Speed up rebuilding the loaded feature index and realpath cache (#8023)
* Speed up rebuilding the loaded feature index

Rebuilding the loaded feature index slowed down with the bug fix
for #17885 in 79a4484a07.  The
slowdown was extreme if realpath emulation was used, but even when
not emulated, it could be about 10x slower.

This adds loaded_features_realpath_map to rb_vm_struct. This is a
hidden hash mapping loaded feature paths to realpaths. When
rebuilding the loaded feature index, look at this hash to get
cached realpath values, and skip calling rb_check_realpath if a
cached value is found.

Fixes [Bug #19246]

* Add a realpath cache to reduce number of syscalls.

Number of lstat and stat syscalls for each 'require'd file is doubled,
because rb_realpath_internal is called from two places with the same
arguments in require_internal; once for checking the realpaths cache,
and once in load_iseq_eval when iseq is not found.

Introduce rb_realpath_internal_cached function to reuse the realpath_map
cache which memoizes rb_realpath_internal function, leading to less
syscalls and increased startup performance depending on the cost of the syscalls
in a particular environment.

---------

Co-authored-by: Jeremy Evans <code@jeremyevans.net>
2023-07-29 05:27:11 +00:00
NARUSE, Yui
0090cb82b0 merge revision(s) df6b72b8ff: [Backport #19348]
Avoid checking interrupt when loading iseq
	MIME-Version: 1.0
	Content-Type: text/plain; charset=UTF-8
	Content-Transfer-Encoding: 8bit

	The interrupt check will unintentionally release the VM lock when loading an iseq.
	And this will cause issues with the `debug` gem's
	[`ObjectSpace.each_iseq` method](0fcfc28aca/ext/debug/iseq_collector.c (L61-L67)),
	which wraps iseqs with a wrapper and exposes their internal states when they're actually not ready to be used.

	And when that happens, errors like this would occur and kill the `debug` gem's thread:

	```
	 DEBUGGER: ReaderThreadError: uninitialized InstructionSequence
	┃ DEBUGGER: Disconnected.
	┃ ["/opt/rubies/ruby-3.2.0/lib/ruby/gems/3.2.0/gems/debug-1.7.1/lib/debug/breakpoint.rb:247:in `absolute_path'",
	┃  "/opt/rubies/ruby-3.2.0/lib/ruby/gems/3.2.0/gems/debug-1.7.1/lib/debug/breakpoint.rb:247:in `block in iterate_iseq'",
	┃  "/opt/rubies/ruby-3.2.0/lib/ruby/gems/3.2.0/gems/debug-1.7.1/lib/debug/breakpoint.rb:246:in `each_iseq'",
	...
	```

	A way to reproduce the issue is to satisfy these conditions at the same time:

	1. `debug` gem calling `ObjectSpace.each_iseq` (e.g. [activating a `LineBreakpoint`](0fcfc28aca/lib/debug/breakpoint.rb (L246))).
	2. A large amount of iseq being loaded from another thread (possibly through the `bootsnap` gem).
	3. 1 and 2 iterating through the same iseq(s) at the same time.

	Because this issue requires external dependencies and a rather complicated timing setup to reproduce, I wasn't able to write a test case for it.
	But here's some pseudo code to help reproduce it:

	```rb
	require "debug/session"

	Thread.new do
	  100.times do
	    ObjectSpace.each_iseq do |iseq|
	      iseq.absolute_path
	    end
	  end
	end

	sleep 0.1

	load_a_bunch_of_iseq
	possibly_through_bootsnap
	```

	[Bug #19348]

	Co-authored-by: Peter Zhu <peter@peterzhu.ca>
	---
	 compile.c       |  2 +-
	 vm_core.h       |  1 +
	 vm_insnhelper.c | 11 +++++++++++
	 3 files changed, 13 insertions(+), 1 deletion(-)
2023-01-25 10:23:38 +09:00
Takashi Kokubun
f25e76fddd
MJIT: Improve comments for JIT fields [ci skip] 2022-12-08 23:48:30 -08:00
Takashi Kokubun
8893913ae6
MJIT: Clarify jit_unit is only for MJIT 2022-12-08 23:43:09 -08:00
Jemma Issroff
40a9964b89 Set max_iv_count (used for object shapes) based on inline caches
With this change, we're storing the iv name on an inline cache on
setinstancevariable instructions. This allows us to check the inline
cache to count instance variables set in initialize and give us an
estimate of iv capacity for an object.

For the purpose of estimating the number of instance variables required
for an object, we're assuming that all initialize methods will call
`super`.

This change allows us to estimate the number of instance variables
required without disassembling instruction sequences.

Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
2022-12-06 13:43:42 -08:00
Daniel Colson
c43951e60e Move BOP macros to separate file
This commit moves ruby_basic_operators and the unredefined macros out of
vm_core.h and into basic_operators.h so that we can use them more
broadly in places where we currently use a method look up via
`rb_method_basic_definition_p` (e.g. object.c, numeric.c, complex.c,
enum.c, but also in internal/compar.h after introducing BOP_CMP and
elsewhere if we introduce more BOPs)

The most controversial part of this change is probably moving
redefined_flag out of rb_vm_t. [vm_opt_method_def_table and
vm_opt_mid_table](9da2a5204f/vm.c)
are not part of rb_vm_t either, and I think this fits well with those.
But more significantly it seems to result in one fewer instruction. For
example:

Before:

```
(lldb) disassemble -n vm_opt_str_freeze
miniruby`vm_exec_core:
miniruby[0x10028233e] <+14558>: movq   0x11a86b(%rip), %rax      ; ruby_current_vm_ptr
miniruby[0x100282345] <+14565>: testb  $0x4, 0x242c(%rax)
```

After:

```
(lldb) disassemble -n vm_opt_str_freeze
ruby`vm_exec_core:
ruby[0x100280ebe] <+14510>: testb  $0x4, 0x120147(%rip)      ; ruby_vm_redefined_flag + 43
```

Co-authored-by: John Hawthorn <jhawthorn@github.com>
2022-12-06 12:37:23 -08:00
John Hawthorn
def258e775 Improve packing of iseq_constant_body struct
By moving the two bools into a packing gap above the mark_bits
pointer/union we can save 8 bytes in the struct and avoid an extra cache
line (328 bytes vs 320 bytes).

Co-authored-by: Adam Hess <HParker@github.com>
2022-12-01 16:31:54 -08:00
Samuel Williams
0436f1e15a
Introduce Fiber#storage for inheritable fiber-scoped variables. (#6612) 2022-12-01 23:00:33 +13:00
Alan Wu
790cf4b6d0 Fix autoload status of statically linked extensions
Previously, for statically-linked extensions, we used
`vm->loading_table` to delay calling the init function until the
extensions are required. This caused the extensions to look like they
are in the middle of being loaded even before they're required.
(`rb_feature_p()` returned true with a loading path output.) Combined
with autoload, queries like `defined?(CONST)` and `Module#autoload?`
were confused by this and returned nil incorrectly. RubyGems uses
`defined?` to detect if OpenSSL is available and failed when OpenSSL was
available in builds using `--with-static-linked-ext`.

Use a dedicated table for the init functions instead of adding them to
the loading table. This lets us remove some logic from non-EXTSTATIC
builds.

[Bug #19115]
2022-11-25 16:21:40 -05:00
Jemma Issroff
9c5e3671eb
Increment max_iv_count on class based on number of set_iv in initialize (#6788)
We can loosely predict the number of ivar sets on a class based on the
number of iv set instructions in the initialize method. This should give
us a more accurate estimate to use for initial size pool allocation,
which should in turn give us more cache hits.
2022-11-22 15:28:14 -05:00
Jemma Issroff
c726c48a3d Remove numiv from RObject
Since object shapes store the capacity of an object, we no longer
need the numiv field on RObjects. This gives us one extra slot which
we can use to give embedded objects one more instance variable (for a
total of 3 ivs). This commit removes the concept of numiv from RObject.
2022-11-10 10:11:34 -05:00
Jemma Issroff
5246f4027e Transition shape when object's capacity changes
This commit adds a `capacity` field to shapes, and adds shape
transitions whenever an object's capacity changes. Objects which are
allocated out of a bigger size pool will also make a transition from the
root shape to the shape with the correct capacity for their size pool
when they are allocated.

This commit will allow us to remove numiv from objects completely, and
will also mean we can guarantee that if two objects share shapes, their
IVs are in the same positions (an embedded and extended object cannot
share shapes). This will enable us to implement ivar sets in YJIT using
object shapes.

Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
2022-11-10 10:11:34 -05:00
Koichi Sasada
e35c528d72 push dummy frame for loading process
This patch pushes dummy frames when loading code for the
profiling purpose.

The following methods push a dummy frame:
* `Kernel#require`
* `Kernel#load`
* `RubyVM::InstructionSequence.compile_file`
* `RubyVM::InstructionSequence.load_from_binary`

https://bugs.ruby-lang.org/issues/18559
2022-10-20 17:38:28 +09:00
Sergey Fedorov
567725ed30
Fix and improve coroutines for Darwin (macOS) ppc/ppc64. (#5975) 2022-10-19 23:49:45 +13:00
Samuel Williams
ced1d17280
Improvements to IO::Buffer implementation and documentation. (#6525) 2022-10-12 12:59:05 +13:00
Jemma Issroff
913979bede
Make inline cache reads / writes atomic with object shapes
Prior to this commit, we were reading and writing ivar index and
shape ID in inline caches in two separate instructions when
getting and setting ivars. This meant there was a race condition
with ractors and these caches where one ractor could change
a value in the cache while another was still reading from it.

This commit instead reads and writes shape ID and ivar index to
inline caches atomically so there is no longer a race condition.

Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
2022-10-11 08:40:56 -07:00
Jemma Issroff
ad63b668e2
Revert "Revert "This commit implements the Object Shapes technique in CRuby.""
This reverts commit 9a6803c90b.
2022-10-11 08:40:56 -07:00
Aaron Patterson
9a6803c90b
Revert "This commit implements the Object Shapes technique in CRuby."
This reverts commit 68bc9e2e97d12f80df0d113e284864e225f771c2.
2022-09-30 16:01:50 -07:00
Jemma Issroff
d594a5a8bd
This commit implements the Object Shapes technique in CRuby.
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects.  Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness").  Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree.  Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.

For example:

```ruby
class Foo
  def initialize
    # Starts with shape id 0
    @a = 1 # transitions to shape id 1
    @b = 1 # transitions to shape id 2
  end
end

class Bar
  def initialize
    # Starts with shape id 0
    @a = 1 # transitions to shape id 1
    @b = 1 # transitions to shape id 2
  end
end

foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```

Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.

This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.

This commit also adds some methods for debugging shapes on objects.  See
`RubyVM::Shape` for more details.

For more context on Object Shapes, see [Feature: #18776]

Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
2022-09-28 08:26:21 -07:00
Aaron Patterson
06abfa5be6
Revert this until we can figure out WB issues or remove shapes from GC
Revert "* expand tabs. [ci skip]"

This reverts commit 830b5b5c35.

Revert "This commit implements the Object Shapes technique in CRuby."

This reverts commit 9ddfd2ca00.
2022-09-26 16:10:11 -07:00
Jemma Issroff
9ddfd2ca00 This commit implements the Object Shapes technique in CRuby.
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects.  Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness").  Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree.  Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.

For example:

```ruby
class Foo
  def initialize
    # Starts with shape id 0
    @a = 1 # transitions to shape id 1
    @b = 1 # transitions to shape id 2
  end
end

class Bar
  def initialize
    # Starts with shape id 0
    @a = 1 # transitions to shape id 1
    @b = 1 # transitions to shape id 2
  end
end

foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```

Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.

This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.

This commit also adds some methods for debugging shapes on objects.  See
`RubyVM::Shape` for more details.

For more context on Object Shapes, see [Feature: #18776]

Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
2022-09-26 09:21:30 -07:00
Samuel Williams
22af2e9084 Rework vm_core to use int first_lineno struct member. 2022-09-26 00:41:16 +13:00
Samuel Williams
75cf29f60d Rework first_lineno to be int. 2022-09-26 00:41:16 +13:00
John Hawthorn
679ef34586 New constant caching insn: opt_getconstant_path
Previously YARV bytecode implemented constant caching by having a pair
of instructions, opt_getinlinecache and opt_setinlinecache, wrapping a
series of getconstant calls (with putobject providing supporting
arguments).

This commit replaces that pattern with a new instruction,
opt_getconstant_path, handling both getting/setting the inline cache and
fetching the constant on a cache miss.

This is implemented by storing the full constant path as a
null-terminated array of IDs inside of the IC structure. idNULL is used
to signal an absolute constant reference.

    $ ./miniruby --dump=insns -e '::Foo::Bar::Baz'
    == disasm: #<ISeq:<main>@-e:1 (1,0)-(1,13)> (catch: FALSE)
    0000 opt_getconstant_path                   <ic:0 ::Foo::Bar::Baz>      (   1)[Li]
    0002 leave

The motivation for this is that we had increasingly found the need to
disassemble the instructions between the opt_getinlinecache and
opt_setinlinecache in order to determine the constant we are fetching,
or otherwise store metadata.

This disassembly was done:
* In opt_setinlinecache, to register the IC against the constant names
  it is using for granular invalidation.
* In rb_iseq_free, to unregister the IC from the invalidation table.
* In YJIT to find the position of a opt_getinlinecache instruction to
  invalidate it when the cache is populated
* In YJIT to register the constant names being used for invalidation.

With this change we no longe need disassemly for these (in fact
rb_iseq_each is now unused), as the list of constant names being
referenced is held in the IC. This should also make it possible to make
more optimizations in the future.

This may also reduce the size of iseqs, as previously each segment
required 32 bytes (on 64-bit platforms) for each constant segment. This
implementation only stores one ID per-segment.

There should be no significant performance change between this and the
previous implementation. Previously opt_getinlinecache was a "leaf"
instruction, but it included a jump (almost always to a separate cache
line). Now opt_getconstant_path is a non-leaf (it may
raise/autoload/call const_missing) but it does not jump. These seem to
even out.
2022-09-01 15:20:49 -07:00
Takashi Kokubun
d6f21b308b
Convert catch_except_t to stdbool
catch_excep_t is a field that exists for MJIT. In the process of
rewriting MJIT in Ruby, I added API to convert 1/0 of _Bool to
true/false, and it seemed confusing and hard to maintain if you
don't use _Bool for *_p fields.
2022-08-25 23:00:19 -07:00
Takashi Kokubun
485019c2bd
Rename mjit_exec to jit_exec (#6262)
* Rename mjit_exec to jit_exec

* Rename mjit_exec_slowpath to mjit_check_iseq

* Remove mjit_exec references from comments
2022-08-19 23:57:17 -07:00
Nobuyoshi Nakada
ee864beb7c
Simplify around USE_YJIT macro (#6240)
* Simplify around `USE_YJIT` macro

- Use `USE_YJIT` macro only instead of `YJIT_BUILD`.
- An intermediate macro `YJIT_SUPPORTED_P` is no longer used.

* Bail out if YJIT is enabled on unsupported platforms
2022-08-15 13:05:12 -04:00
Yusuke Endoh
8f7e188822 Add "rb_" prefixes to toplevel enum definitions
... as per ko1's request.
2022-07-22 23:10:24 +09:00
Yusuke Endoh
e763b1118b Move enum definitions out of struct definition 2022-07-22 23:10:24 +09:00
Takashi Kokubun
5b21e94beb Expand tabs [ci skip]
[Misc #18891]
2022-07-21 09:42:04 -07:00
Jemma Issroff
85ea46730d Separate TS_IVC and TS_ICVARC in is_entries buffers
This allows us to treat cvar caches differently than ivar caches.
2022-07-18 14:06:30 -07:00
John Hawthorn
b8247a1669 Correct comment explaining env flags [ci skip]
We use 4 values for env flags now, which also shifted over the frame
flags by one bit.
2022-07-14 09:59:34 -07:00
Aaron Patterson
8d63a04703 Flatten bitmap when there is only one element
We can avoid allocating a bitmap when the number of elements in the iseq
is fewer than the size of an iseq_bits_t
2022-06-23 16:52:00 -07:00
Aaron Patterson
1ccdb1a251 Update vm_core.h
Co-authored-by: Tomás Coêlho <36938811+tomascco@users.noreply.github.com>
2022-06-23 14:01:46 -07:00
Aaron Patterson
e23540e566 Speed up ISeq by marking via bitmaps and IC rearranging
This commit adds a bitfield to the iseq body that stores offsets inside
the iseq buffer that contain values we need to mark.  We can use this
bitfield to mark objects instead of disassembling the instructions.

This commit also groups inline storage entries and adds a counter for
each entry.  This allows us to iterate and mark each entry without
disassembling instructions

Since we have a bitfield and grouped inline caches, we can mark all
VALUE objects associated with instructions without actually
disassembling the instructions at mark time.

[Feature #18875] [ruby-core:109042]
2022-06-23 14:01:46 -07:00
Koichi Sasada
08cee2bf80 altstack is native thread's attr
Move th->altstack to th->nt->altstack.
2022-05-24 17:50:49 +09:00
Koichi Sasada
f3235ac095 add rb_th_serial()
`rb_th_serial(th)` returns th's serial for debug print purpose.
2022-05-24 10:06:51 +09:00
Koichi Sasada
d9984f39d3 remove NON_SCALAR_THREAD_ID support
`NON_SCALAR_THREAD_ID` shows `pthread_t` is non-scalar (non-pointer)
and only s390x is known platform. However, the supporting code is
very complex and it is only used for deubg print information.

So this patch removes the support of `NON_SCALAR_THREAD_ID`
and make the code simple.
2022-05-24 10:06:51 +09:00
Koichi Sasada
eab99b1d4b rb_thread_t::serial for debug
`rb_thread_t::serial` is auto-incremented serial number for
threads and it can overflow, it means the serial is not a ID
for each thread, it is only for debug print.

`RUBY_DEBUG_LOG` shows this information.

Also skip EC related information if EC is NULL. This patch
enable to use `RUBY_DEBUG_LOG` without setup EC.
2022-05-20 17:37:46 +09:00
Samuel Williams
f626998c4f
Delete autoload data from global features after autoload has completed. (#5910)
* Update naming of critical section assertions macros.

* Improved locking for autoload.
2022-05-17 00:50:02 +12:00
Samuel Williams
32de6097b2
Fix various autoload race conditions. (#5898)
* Add RUBY_VM_CRITICAL_SECTION for detecting unexpected context switch.

* Prevent race between GC mark and autoload setup.

* Protect race on autoload state.

* Avoid potential race condition when allocating `autoload_featuremap`.

* Add NEWS entry for autoload fixes.
2022-05-15 16:07:12 +12:00
Alan Wu
f90549cd38 Rust YJIT
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.

The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.

Because we want to be careful, YJIT is guarded behind a configure
option:

```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```

By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.

The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.

The CI tests have been updated and do not take any more resources than
before.

The development history of the Rust port is available at the following
commit for interested parties:
1fd9573d8b

Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.

[issue]: https://bugs.ruby-lang.org/issues/18481

Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Noah Gibbs <the.codefolio.guy@gmail.com>
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
2022-04-27 11:00:22 -04:00
Koichi Sasada
03d21a4fb0 introduce struct rb_native_thread
`rb_thread_t` contained `native_thread_data_t` to represent
thread implementation dependent data. This patch separates
them and rename it `rb_native_thread` and point it from
`rb_thraed_t`.

Now, 1 Ruby thread (`rb_thread_t`) has 1 native thread (`rb_native_thread`).
2022-04-23 03:08:27 +09:00
Koichi Sasada
1c4fc0241d rename thread internal naming
Now GVL is not process *Global* so this patch try to use
another words.

* `rb_global_vm_lock_t` -> `struct rb_thread_sched`
  * `gvl->owner` -> `sched->running`
  * `gvl->waitq` -> `sched->readyq`
* `rb_gvl_init` -> `rb_thread_sched_init`
* `gvl_destroy` -> `rb_thread_sched_destroy`
* `gvl_acquire` -> `thread_sched_to_running` # waiting -> ready -> running
* `gvl_release` -> `thread_sched_to_waiting` # running -> waiting
* `gvl_yield`   -> `thread_sched_yield`
* `GVL_UNLOCK_BEGIN` -> `THREAD_BLOCKING_BEGIN`
* `GVL_UNLOCK_END` -> `THREAD_BLOCKING_END`

* removed
  * `rb_ractor_gvl`
  * `rb_vm_gvl_destroy` (not used)

There are GVL functions such as `rb_thread_call_without_gvl()` yet
but I don't have good name to replace them. Maybe GVL stands for
"Greate Valuable Lock" or something like that.
2022-04-22 07:54:09 +09:00