Push a real iseq in rb_vm_push_frame_fname()
Previously, vm_make_env_each() (used during proc
creation and for the debug inspector C API) picked up the
non-GC-allocated iseq that rb_vm_push_frame_fname() creates,
which led to a SEGV when the GC tried to mark the non GC object.
Put a real iseq imemo instead. Speed should be about the same since
the old code also did a imemo allocation and a malloc allocation.
Real iseq allows ironing out the special-casing of dummy frames in
rb_execution_context_mark() and rb_execution_context_update(). A check
is added to RubyVM::ISeq#eval, though, to stop attempts to run dummy
iseqs.
[Bug #21180]
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
[Bug #21144] Win32: Use Windows time zone ID if TZ is not set
If the TZ environment variable is not set, the time zone names
retrieved from the system are localized for UI display and may vary
across editions and language packs for the same time zone.
Use the time zone IDs that are invariant across environments instead.
[Bug #21144] Win32: Convert the time zone name to the current locale
The Windows time zone IDs provided by Microsoft as of 24H1 are ASCII
only all, but the API itself is not impossible to set non-ASCII key
name. Prefer the current locale encoding for now until we move to
UTF-8 including environment variables and command line arguments.
Fix a race condition with interned strings sweeping.
[Bug #21172]
This fixes a rare CI failure.
The timeline of the race condition is:
- A `"foo" oid=1` string is interned.
- `"foo" oid=1` is no longer referenced and will be swept in the future.
- Another `"foo" oid=2` string is interned.
- `register_fstring` finds `"foo" oid=1`, but since it is about to be swept,
removes it from `fstring_table` and insert `"foo" oid=2` instead.
- `"foo" oid=1` is swept, since it has the `RSTRING_FSTR` flag,
a `st_delete` is issued in `fstring_table` which removes `"foo" oid=2`.
I don't know how to reproduce this bug consistently in a single test
case.
[Bug #21092] Fallback variables after execonf has done
When reading from a dummy makefile, the global variables initialized
in `init_mkmf` may not be overridden.
rb_feature_p: skip `get_expanded_load_path` for absolute paths
Ref: https://github.com/fxn/zeitwerk/pull/308
```ruby
require 'benchmark'
$LOAD_PATH << 'relative-path'
autoload :FOO, '/tmp/foo.rb'
puts Benchmark.realtime {
500_000.times do
Object.autoload?(:FOO)
end
}
```
The above script takes 2.5 seconds on `master`, and only
50ms on this branch.
When we're looking for a feature with an absolute path, we don't
need to call the expensive `get_expanded_load_path`.
[DOC] Tweak "Timezone Objects"
- Make method descriptions plain pragraphs in each method.
- Make "Argument" and "Returns" note-lists.
[Bug #20951] [DOC] About UTC offset calculation after `utc_to_local`
[Bug #20951] [DOC] About UTC offset calculation after `utc_to_local`
[Bug #21170]
st_table reserves -1 as a special hash value to indicate that an entry
has been deleted. So that that's a valid value to be returned from the
hash function, do_hash replaces -1 with 0 so that it is not mistaken for
the sentinel.
Previously, when upgrading an AR table to an ST table,
rb_st_add_direct_with_hash was used which did not perform the same
conversion, this could lead to a hash in a broken state where one if its
entries which was supposed to exist being marked as a tombstone.
The hash could then become further corrupted when the ST table required
resizing as the falsely tombstoned entry would be skipped but it would
be counted in num entries, leading to an uninitialized entry at index
15.
In most cases this will be really rare, unless using a very poorly
implemented custom hash function.
This also adds two debug assertions, one that st_add_direct_with_hash
does not receive the reserved hash value, and a second in
rebuild_table_with, which ensures that after we rebuild/compact a table
it contains the expected number of elements.
Co-authored-by: Alan Wu <alanwu@ruby-lang.org>
Evident with the crash reported in [Bug #20997], the C replacement
codegen functions aren't authored to handle block arguments (nor
should they because the extra code from the complexity defeats
optimization). Filter sites with VM_CALL_ARGS_BLOCKARG.
Co-Authored-By: Alan Wu <alanwu@ruby-lang.org>
Avoid opt_aset_with optimization inside multiple assignment
Previously, since the opt_aset_with optimization was introduced,
use of the opt_aset_with optimization inside multiple assignment
would result in a segfault or incorrect instructions.
Fixes [Bug #21012]
Co-authored-by: Nobuyoshi Nakada <nobu.nakada@gmail.com>
Fix use-after-free in ep in Proc#dup for ifunc procs
[Bug #20950]
ifunc proc has the ep allocated in the cfunc_proc_t which is the data of
the TypedData object. If an ifunc proc is duplicated, the ep points to
the ep of the source object. If the source object is freed, then the ep
of the duplicated object now points to a freed memory region. If we try
to use the ep we could crash.
For example, the following script crashes:
p = { a: 1 }.to_proc
100.times do
p = p.dup
GC.start
p.call
rescue ArgumentError
end
This commit changes ifunc proc to also duplicate the ep when it is duplicated.
Fix use-after-free in constant cache
[Bug #20921]
When we create a cache entry for a constant, the following sequence of
events could happen:
- vm_track_constant_cache is called to insert a constant cache.
- In vm_track_constant_cache, we first look up the ST table for the ID
of the constant. Assume the ST table exists because another iseq also
holds a cache entry for this ID.
- We then insert into this ST table with the iseq_inline_constant_cache.
- However, while inserting into this ST table, it allocates memory, which
could trigger a GC. Assume that it does trigger a GC.
- The GC frees the one and only other iseq that holds a cache entry for
this ID.
- In remove_from_constant_cache, it will appear that the ST table is now
empty because there are no more iseq with cache entries for this ID, so
we free the ST table.
- We complete GC and continue our st_insert. However, this ST table has
been freed so we now have a use-after-free.
This issue is very hard to reproduce, because it requires that the GC runs
at a very specific time. However, we can make it show up by applying this
patch which runs GC right before the st_insert to mimic the st_insert
triggering a GC:
diff --git a/vm_insnhelper.c b/vm_insnhelper.c
index 3cb23f06f0..a93998136a 100644
--- a/vm_insnhelper.c
+++ b/vm_insnhelper.c
@@ -6338,6 +6338,10 @@ vm_track_constant_cache(ID id, void *ic)
rb_id_table_insert(const_cache, id, (VALUE)ics);
}
+ if (id == rb_intern("MyConstant")) rb_gc();
+
st_insert(ics, (st_data_t) ic, (st_data_t) Qtrue);
}
And if we run this script:
Object.const_set("MyConstant", "Hello!")
my_proc = eval("-> { MyConstant }")
my_proc.call
my_proc = eval("-> { MyConstant }")
my_proc.call
We can see that ASAN outputs a use-after-free error:
==36540==ERROR: AddressSanitizer: heap-use-after-free on address 0x606000049528 at pc 0x000102f3ceac bp 0x00016d607a70 sp 0x00016d607a68
READ of size 8 at 0x606000049528 thread T0
#0 0x102f3cea8 in do_hash st.c:321
#1 0x102f3ddd0 in rb_st_insert st.c:1132
#2 0x103140700 in vm_track_constant_cache vm_insnhelper.c:6345
#3 0x1030b91d8 in vm_ic_track_const_chain vm_insnhelper.c:6356
#4 0x1030b8cf8 in rb_vm_opt_getconstant_path vm_insnhelper.c:6424
#5 0x1030bc1e0 in vm_exec_core insns.def:263
#6 0x1030b55fc in rb_vm_exec vm.c:2585
#7 0x1030fe0ac in rb_iseq_eval_main vm.c:2851
#8 0x102a82588 in rb_ec_exec_node eval.c:281
#9 0x102a81fe0 in ruby_run_node eval.c:319
#10 0x1027f3db4 in rb_main main.c:43
#11 0x1027f3bd4 in main main.c:68
#12 0x183900270 (<unknown module>)
0x606000049528 is located 8 bytes inside of 56-byte region [0x606000049520,0x606000049558)
freed by thread T0 here:
#0 0x104174d40 in free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x54d40)
#1 0x102ada89c in rb_gc_impl_free default.c:8183
#2 0x102ada7dc in ruby_sized_xfree gc.c:4507
#3 0x102ac4d34 in ruby_xfree gc.c:4518
#4 0x102f3cb34 in rb_st_free_table st.c:663
#5 0x102bd52d8 in remove_from_constant_cache iseq.c:119
#6 0x102bbe2cc in iseq_clear_ic_references iseq.c:153
#7 0x102bbd2a0 in rb_iseq_free iseq.c:166
#8 0x102b32ed0 in rb_imemo_free imemo.c:564
#9 0x102ac4b44 in rb_gc_obj_free gc.c:1407
#10 0x102af4290 in gc_sweep_plane default.c:3546
#11 0x102af3bdc in gc_sweep_page default.c:3634
#12 0x102aeb140 in gc_sweep_step default.c:3906
#13 0x102aeadf0 in gc_sweep_rest default.c:3978
#14 0x102ae4714 in gc_sweep default.c:4155
#15 0x102af8474 in gc_start default.c:6484
#16 0x102afbe30 in garbage_collect default.c:6363
#17 0x102ad37f0 in rb_gc_impl_start default.c:6816
#18 0x102ad3634 in rb_gc gc.c:3624
#19 0x1031406ec in vm_track_constant_cache vm_insnhelper.c:6342
#20 0x1030b91d8 in vm_ic_track_const_chain vm_insnhelper.c:6356
#21 0x1030b8cf8 in rb_vm_opt_getconstant_path vm_insnhelper.c:6424
#22 0x1030bc1e0 in vm_exec_core insns.def:263
#23 0x1030b55fc in rb_vm_exec vm.c:2585
#24 0x1030fe0ac in rb_iseq_eval_main vm.c:2851
#25 0x102a82588 in rb_ec_exec_node eval.c:281
#26 0x102a81fe0 in ruby_run_node eval.c:319
#27 0x1027f3db4 in rb_main main.c:43
#28 0x1027f3bd4 in main main.c:68
#29 0x183900270 (<unknown module>)
previously allocated by thread T0 here:
#0 0x104174c04 in malloc+0x94 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x54c04)
#1 0x102ada0ec in rb_gc_impl_malloc default.c:8198
#2 0x102acee44 in ruby_xmalloc gc.c:4438
#3 0x102f3c85c in rb_st_init_table_with_size st.c:571
#4 0x102f3c900 in rb_st_init_table st.c:600
#5 0x102f3c920 in rb_st_init_numtable st.c:608
#6 0x103140698 in vm_track_constant_cache vm_insnhelper.c:6337
#7 0x1030b91d8 in vm_ic_track_const_chain vm_insnhelper.c:6356
#8 0x1030b8cf8 in rb_vm_opt_getconstant_path vm_insnhelper.c:6424
#9 0x1030bc1e0 in vm_exec_core insns.def:263
#10 0x1030b55fc in rb_vm_exec vm.c:2585
#11 0x1030fe0ac in rb_iseq_eval_main vm.c:2851
#12 0x102a82588 in rb_ec_exec_node eval.c:281
#13 0x102a81fe0 in ruby_run_node eval.c:319
#14 0x1027f3db4 in rb_main main.c:43
#15 0x1027f3bd4 in main main.c:68
#16 0x183900270 (<unknown module>)
This commit fixes this bug by adding a inserting_constant_cache_id field
to the VM, which stores the ID that is currently being inserted and, in
remove_from_constant_cache, we don't free the ST table for ID equal to
this one.
Co-Authored-By: Alan Wu <alanwu@ruby-lang.org>
[Bug #20915] Fix SEGV with `TracePoint#parameters` and aliased C method
The following snippet results with a SEGV:
```ruby
C = Class.new do
alias_method :new_to_s, :to_s
end
TracePoint.new(:c_call, &:parameters).enable { C.new.new_to_s }
```
at MRI 3.3.6 and ruby 3.4.0dev
The root cause of the issue lies in the `rb_tracearg_parameters` function
within the `RUBY_EVENT_C_RETURN` branch. Specifically, when the invoked
method is an alias for a C function,
`rb_method_entry_without_refinements(..., trace_arg->called_id, ...)`
may return NULL. In that case we can fallback to `trace_arg->id`.
Fix a bug in rb_include_module that stops nested inclusion into module subclasses
This bug was present since the code was originally added by me
in 3556a834a2.
Fixes [Bug #20871]
[Bug #20868] Fix Method#hash to not change after compaction
The hash value of a Method must remain constant after a compaction, otherwise
it may not work as the key in a hash table.
For example:
def a; end
# Need this method here because otherwise the iseq may be on the C stack
# which would get pinned and not move during compaction
def get_hash
method(:a).hash
end
puts get_hash # => 2993401401091578131
GC.verify_compaction_references(expand_heap: true, toward: :empty)
puts get_hash # => -2162775864511574135
Mingw crt-git 12.0.0.r369.g0d4221712-1 now prohibits "command line
contains characters that are not supported in the active code page".
0d42217123/
Already Ruby builds `argv` in `rb_w32_sysinit`, instead of mswin- or
mingw-made `argv`. Just bypass the conversion in mingw crt.
