[Bug #19896]
fix memory leak in vm_method
This introduces a unified reference_count to clarify who is referencing a method.
This also allows us to treat the refinement method as the def owner since it counts itself as a reference
Co-authored-by: Peter Zhu <peter@peterzhu.ca>
---
gc.c | 4 +-
method.h | 6 +--
rjit_c.rb | 6 +--
test/ruby/test_module.rb | 4 +-
vm_insnhelper.c | 2 +-
vm_method.c | 105 +++++++++++++++++++----------------------------
6 files changed, 54 insertions(+), 73 deletions(-)
[Bug #19778] Add `-I` options for opt-dir to `$INCFLAGS`
These options have been separated from `$CFLAGS` already in the
other places.
---
configure.ac | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
Fix premature string collection during append
Previously, the following crashed due to use-after-free
with AArch64 Alpine Linux 3.18.3 (aarch64-linux-musl):
```ruby
str = 'a' * (32*1024*1024)
p({z: str})
```
32 MiB is the default for `GC_MALLOC_LIMIT_MAX`, and the crash
could be dodged by setting `RUBY_GC_MALLOC_LIMIT_MAX` to large values.
Under a debugger, one can see the `str2` of rb_str_buf_append()
getting prematurely collected while str_buf_cat4() allocates capacity.
Add GC guards so the buffer of `str2` lives across the GC run
initiated in str_buf_cat4().
[Bug #19792]
---
string.c | 2 ++
1 file changed, 2 insertions(+)
Fix memory leak in complemented method entries
[Bug #19894]
When a copy of a complemented method entry is created, there are two
issues:
1. IMEMO_FL_USER3 is not copied, so the complemented status is not
copied over.
2. In rb_method_entry_clone we increment both alias_count and
complemented_count. However, when we free the method entry in
rb_method_definition_release, we only decrement one of the two
counters, resulting in the rb_method_definition_t being leaked.
Co-authored-by: Adam Hess <adamhess1991@gmail.com>
---
method.h | 5 +++--
test/ruby/test_module.rb | 29 +++++++++++++++++++++++++++++
vm_method.c | 8 +++++---
3 files changed, 37 insertions(+), 5 deletions(-)
Fix missing write barrier in iseq instruction list
[Bug #19880]
There's a missing write barrier for operands in the iseq instruction
list, which can cause crashes.
It can be reproduced when Ruby is compiled with `-DRUBY_DEBUG_ENV=1`.
Using the following command:
```
RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR=0 RUBY_DEBUG=gc_stress ruby -w --disable=gems -Itool/lib -W0 test.rb
```
The following script crashes:
```
require "test/unit"
```
Fix unused_mut Rust warnings
Rust version 1.71.0 and up issue these warnings. On GitHub CI, the
warnings were previously seen in -DYJIT_FORCE_ENABLE runs.
If two threads are running, with one calling waitpid(-1), and another
calling waitpid($some_pid), and then $some_other_pid exits, we would
expect the waitpid(-1) call to retrieve that exit status; however, it
cannot actually do so until $some_pid _also_ exits.
This patch fixes the issue by unconditionally checking for pending
process group waits on SIGCHLD, and then allowing pending pid-only waits
to "steal" the notification.
[Fixes#19387]
* Re-apply "Ruby 3.2 - Speed up rebuilding the loaded feature index and realpath cache (#8023)"
* [CI] mingw.yml - remove IBM437 encoding for test-all, use cmd shell for test & test-all
* Skip failing test on mingw with readline.so
Co-authored-by: nagachika <nagachika@ruby-lang.org>
---------
Co-authored-by: nagachika <nagachika@ruby-lang.org>
Fix memory leak for incomplete lambdas
[Bug #19836]
The parser does not free the chain of `struct vtable`, which causes
memory leaks.
The following script reproduces this issue:
```
10.times do
100_000.times do
Ripper.parse("-> {")
end
puts `ps -o rss= -p #{$$}`
end
```
---
parse.y | 24 ++++++++++++++----------
test/ripper/test_ripper.rb | 7 +++++++
2 files changed, 21 insertions(+), 10 deletions(-)
Fix memory leak in parser for incomplete tokens
[Bug #19835]
The parser does not free the `tbl` of the `struct vtable` when there are
leftover `lvtbl` in the parser. This causes a memory leak.
The following script reproduces this issue:
```
10.times do
100_000.times do
Ripper.parse("class Foo")
end
puts `ps -o rss= -p #{$$}`
end
```
---
parse.y | 42 ++++++++++++++++++++++++++++--------------
test/ripper/test_ripper.rb | 7 +++++++
2 files changed, 35 insertions(+), 14 deletions(-)
Skip running brew upgrade (#8189)
This has been unstable:
1571398859
and I'm not sure if we need that in the first place, assuming the
OS image itself is maintained by GitHub.
---
.github/actions/setup/macos/action.yml | 1 -
1 file changed, 1 deletion(-)
Fix crash in NoMethodError for dummy frames
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
[Bug #19793]
Dummy frames are created at the top level when requiring another file.
While requiring a file, it will try to convert using encodings. Some of
these encodings will not respond to to_str. If method_missing is
redefined on Object, then it will call method_missing and attempt raise
an error. However, the iseq is invalid as it's a dummy frame so it will
write an invalid iseq to the created NoMethodError.
The following script crashes:
```
GC.stress = true
class Object
public :method_missing
end
File.write("/tmp/empty.rb", "")
require "/tmp/empty.rb"
```
With the following backtrace:
```
frame #0: 0x00000001000fa8b8 miniruby`RVALUE_MARKED(obj=4308637824) at gc.c:1638:12
frame #1: 0x00000001000fb440 miniruby`RVALUE_BLACK_P(obj=4308637824) at gc.c:1763:12
frame #2: 0x00000001000facdc miniruby`gc_writebarrier_incremental(a=4308637824, b=4308332208, objspace=0x000000010180b000) at gc.c:8822:9
frame #3: 0x00000001000faad8 miniruby`rb_gc_writebarrier(a=4308637824, b=4308332208) at gc.c:8864:17
frame #4: 0x000000010016aff0 miniruby`rb_obj_written(a=4308637824, oldv=36, b=4308332208, filename="../iseq.c", line=1279) at gc.h:804:9
frame #5: 0x0000000100162a60 miniruby`rb_obj_write(a=4308637824, slot=0x0000000100d09888, b=4308332208, filename="../iseq.c", line=1279) at gc.h:837:5
frame #6: 0x0000000100165b0c miniruby`iseqw_new(iseq=0x0000000100d09880) at iseq.c:1279:9
frame #7: 0x0000000100165a64 miniruby`rb_iseqw_new(iseq=0x0000000100d09880) at iseq.c:1289:12
frame #8: 0x00000001000d8324 miniruby`name_err_init_attr(exc=4309777920, recv=4304780496, method=827660) at error.c:1830:35
frame #9: 0x00000001000d1b80 miniruby`name_err_init(exc=4309777920, mesg=4308332496, recv=4304780496, method=827660) at error.c:1869:12
frame #10: 0x00000001000d1bd4 miniruby`rb_nomethod_err_new(mesg=4308332496, recv=4304780496, method=827660, args=4308332448, priv=0) at error.c:1957:5
frame #11: 0x000000010039049c miniruby`rb_make_no_method_exception(exc=4304914512, format=4308332496, obj=4304780496, argc=1, argv=0x000000016fdfab00, priv=0) at vm_eval.c:959:16
frame #12: 0x00000001003b3274 miniruby`raise_method_missing(ec=0x0000000100b06f40, argc=1, argv=0x000000016fdfab00, obj=4304780496, last_call_status=MISSING_NOENTRY) at vm_eval.c:999:15
frame #13: 0x00000001003945d4 miniruby`rb_method_missing(argc=1, argv=0x000000016fdfab00, obj=4304780496) at vm_eval.c:944:5
...
frame #23: 0x000000010038f5e4 miniruby`rb_vm_call_kw(ec=0x0000000100b06f40, recv=4304780496, id=2865, argc=1, argv=0x000000016fdfab00, me=0x0000000100cbfcf0, kw_splat=0) at vm_eval.c:326:12
frame #24: 0x00000001003c18e4 miniruby`call_method_entry(ec=0x0000000100b06f40, defined_class=4304927952, obj=4304780496, id=2865, cme=0x0000000100cbfcf0, argc=1, argv=0x000000016fdfab00, kw_splat=0) at vm_method.c:2720:20
frame #25: 0x00000001003c440c miniruby`check_funcall_exec(v=6171896792) at vm_eval.c:589:12
frame #26: 0x00000001000dec00 miniruby`rb_vrescue2(b_proc=(miniruby`check_funcall_exec at vm_eval.c:587), data1=6171896792, r_proc=(miniruby`check_funcall_failed at vm_eval.c:596), data2=6171896792, args="Pȗ") at eval.c:919:18
frame #27: 0x00000001000deab0 miniruby`rb_rescue2(b_proc=(miniruby`check_funcall_exec at vm_eval.c:587), data1=6171896792, r_proc=(miniruby`check_funcall_failed at vm_eval.c:596), data2=6171896792) at eval.c:900:17
frame #28: 0x000000010039008c miniruby`check_funcall_missing(ec=0x0000000100b06f40, klass=4304923536, recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000, respond=-1, def=36, kw_splat=0) at vm_eval.c:666:15
frame #29: 0x000000010038fa60 miniruby`rb_check_funcall_default_kw(recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000, def=36, kw_splat=0) at vm_eval.c:703:21
frame #30: 0x000000010038fb04 miniruby`rb_check_funcall(recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000) at vm_eval.c:685:12
frame #31: 0x00000001001c469c miniruby`convert_type_with_id(val=4304780496, tname="String", method=3233, raise=0, index=-1) at object.c:3061:15
frame #32: 0x00000001001c4a4c miniruby`rb_check_convert_type_with_id(val=4304780496, type=5, tname="String", method=3233) at object.c:3153:9
frame #33: 0x00000001002d59f8 miniruby`rb_check_string_type(str=4304780496) at string.c:2571:11
frame #34: 0x000000010014b7b0 miniruby`io_encoding_set(fptr=0x0000000100d09ca0, v1=4304780496, v2=4, opt=4) at io.c:11655:19
frame #35: 0x0000000100139a58 miniruby`rb_io_set_encoding(argc=1, argv=0x000000016fdfb450, io=4308334032) at io.c:13497:5
frame #36: 0x00000001003c0004 miniruby`ractor_safe_call_cfunc_m1(recv=4308334032, argc=1, argv=0x000000016fdfb450, func=(miniruby`rb_io_set_encoding at io.c:13487)) at vm_insnhelper.c:3271:12
...
frame #43: 0x0000000100390b08 miniruby`rb_funcall(recv=4308334032, mid=16593, n=1) at vm_eval.c:1137:12
frame #44: 0x00000001002a43d8 miniruby`load_file_internal(argp_v=6171899936) at ruby.c:2500:5
...
```
---
error.c | 4 +++-
test/ruby/test_require.rb | 15 +++++++++++++++
2 files changed, 18 insertions(+), 1 deletion(-)
addr2line.c: fix `DW_FORM_ref_addr` parsing for DWARF 2 (#8146)
addr2line.c: fix DW_FORM_ref_addr parsing for DWARF 2
This fixes a crash when retrieving backtrace info with YJIT enabled on
macOS with Rust 1.71.0. Since Rust 1.71.0, the DWARF info generated by
the Rust compiler uses DW_FORM_ref_addr instead of DW_FORM_ref4 for
pointers to other DIEs.
DW_FORM_ref_addr representation in DWARF 2 is different from DWARF 3+,
so we need to handle it separately.
This patch fixes the parsing of DW_FORM_ref_addr for DWARF 2, which is
the default DWARF version Rustc uses on macOS.
See the DWARF 2.0.0 spec, section 7.5.4 Attribute Encodings
https://dwarfstd.org/doc/dwarf-2.0.0.pdfhttps://bugs.ruby-lang.org/issues/19789
---
addr2line.c | 40 ++++++++++++++++++++++++++--------------
1 file changed, 26 insertions(+), 14 deletions(-)
* Speed up rebuilding the loaded feature index
Rebuilding the loaded feature index slowed down with the bug fix
for #17885 in 79a4484a07. The
slowdown was extreme if realpath emulation was used, but even when
not emulated, it could be about 10x slower.
This adds loaded_features_realpath_map to rb_vm_struct. This is a
hidden hash mapping loaded feature paths to realpaths. When
rebuilding the loaded feature index, look at this hash to get
cached realpath values, and skip calling rb_check_realpath if a
cached value is found.
Fixes [Bug #19246]
* Add a realpath cache to reduce number of syscalls.
Number of lstat and stat syscalls for each 'require'd file is doubled,
because rb_realpath_internal is called from two places with the same
arguments in require_internal; once for checking the realpaths cache,
and once in load_iseq_eval when iseq is not found.
Introduce rb_realpath_internal_cached function to reuse the realpath_map
cache which memoizes rb_realpath_internal function, leading to less
syscalls and increased startup performance depending on the cost of the syscalls
in a particular environment.
---------
Co-authored-by: Jeremy Evans <code@jeremyevans.net>
Fix autosplat conditions to handle ruby2_keywords case
Autosplat should not occur if there are two arguments but second
argument is an array containing a ruby2_keywords splat. Only
autosplat if a single argument to be yielded to the block, and there
is no splatted flagged keyword hash passed.
Fixes [Bug #19759]
---
test/ruby/test_proc.rb | 26 ++++++++++++++++++++++++++
vm_args.c | 3 ++-
2 files changed, 28 insertions(+), 1 deletion(-)
Use tools appropriate with CC
To get rid of mysterious errors such as:
```
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: error: libruby.3.3-static.a(/): The end of the file was unexpectedly encountered
```
and
```
ld: warning: ignoring file ../../libruby.3.3-static.a, building for macOS-x86_64 but attempting to link with file built for macOS-x86_64
Undefined symbols for architecture x86_64:
"_rb_rational_num", referenced from:
```
---
configure.ac | 6 ++++++
1 file changed, 6 insertions(+)
Use an st table for "too complex" objects
st tables will maintain insertion order so we can marshal dump / load
objects with instance variables in the same order they were set on that
particular instance
[ruby-core:112926] [Bug #19535]
Co-Authored-By: Jemma Issroff <jemmaissroff@gmail.com>
---
gc.c | 10 ++++------
include/ruby/st.h | 2 ++
object.c | 2 +-
ractor.c | 43 ++++++++++++++++++++++---------------------
shape.h | 6 +++---
st.c | 6 ++++++
test/ruby/test_shapes.rb | 21 +++++++++++++++++++++
variable.c | 28 ++++++++++++++--------------
vm_insnhelper.c | 2 +-
9 files changed, 74 insertions(+), 46 deletions(-)
Fix `PLATFORM_GET_INC`
On platforms where unaligned word access is not allowed, and if
`sizeof(val)` and `sizeof(type)` differ:
- `val` > `type`, `val` will be a garbage.
- `val` < `type`, outside `val` will be clobbered.
---
regexec.c | 2 +-
regint.h | 9 ++++++++-
2 files changed, 9 insertions(+), 2 deletions(-)
ObjectSpace::WeakMap: clean inverse reference when an entry is
re-assigned
[Bug #19531]
```ruby
wmap[1] = "A"
wmap[1] = "B"
```
In the example above, we need to remove the `"A" => 1` inverse reference
so that when `"A"` is GCed the `1` key isn't deleted.
---
test/ruby/test_weakmap.rb | 17 +++++++++
weakmap.c | 91 ++++++++++++++++++++++++++++++++++++++---------
2 files changed, 91 insertions(+), 17 deletions(-)
Handle unterminated unicode escapes in regexps
This fixes an infinite loop possible after ec3542229b.
For \u{} escapes in regexps, skip validation in the parser, and rely on the regexp
code to handle validation. This is necessary so that invalid unicode escapes in
comments in extended regexps are allowed.
Fixes [Bug #19750]
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
---
parse.y | 97 ++++++++++++++++++++++++++++++++-----------------
test/ruby/test_parse.rb | 16 ++++++++
2 files changed, 79 insertions(+), 34 deletions(-)
