[Bug #19896]
fix memory leak in vm_method
This introduces a unified reference_count to clarify who is referencing a method.
This also allows us to treat the refinement method as the def owner since it counts itself as a reference
Co-authored-by: Peter Zhu <peter@peterzhu.ca>
---
gc.c | 4 +-
method.h | 6 +--
rjit_c.rb | 6 +--
test/ruby/test_module.rb | 4 +-
vm_insnhelper.c | 2 +-
vm_method.c | 105 +++++++++++++++++++----------------------------
6 files changed, 54 insertions(+), 73 deletions(-)
Fix memory leak in complemented method entries
[Bug #19894]
When a copy of a complemented method entry is created, there are two
issues:
1. IMEMO_FL_USER3 is not copied, so the complemented status is not
copied over.
2. In rb_method_entry_clone we increment both alias_count and
complemented_count. However, when we free the method entry in
rb_method_definition_release, we only decrement one of the two
counters, resulting in the rb_method_definition_t being leaked.
Co-authored-by: Adam Hess <adamhess1991@gmail.com>
---
method.h | 5 +++--
test/ruby/test_module.rb | 29 +++++++++++++++++++++++++++++
vm_method.c | 8 +++++---
3 files changed, 37 insertions(+), 5 deletions(-)
If two threads are running, with one calling waitpid(-1), and another
calling waitpid($some_pid), and then $some_other_pid exits, we would
expect the waitpid(-1) call to retrieve that exit status; however, it
cannot actually do so until $some_pid _also_ exits.
This patch fixes the issue by unconditionally checking for pending
process group waits on SIGCHLD, and then allowing pending pid-only waits
to "steal" the notification.
[Fixes#19387]
* Re-apply "Ruby 3.2 - Speed up rebuilding the loaded feature index and realpath cache (#8023)"
* [CI] mingw.yml - remove IBM437 encoding for test-all, use cmd shell for test & test-all
* Skip failing test on mingw with readline.so
Co-authored-by: nagachika <nagachika@ruby-lang.org>
---------
Co-authored-by: nagachika <nagachika@ruby-lang.org>
Fix memory leak for incomplete lambdas
[Bug #19836]
The parser does not free the chain of `struct vtable`, which causes
memory leaks.
The following script reproduces this issue:
```
10.times do
100_000.times do
Ripper.parse("-> {")
end
puts `ps -o rss= -p #{$$}`
end
```
---
parse.y | 24 ++++++++++++++----------
test/ripper/test_ripper.rb | 7 +++++++
2 files changed, 21 insertions(+), 10 deletions(-)
Fix memory leak in parser for incomplete tokens
[Bug #19835]
The parser does not free the `tbl` of the `struct vtable` when there are
leftover `lvtbl` in the parser. This causes a memory leak.
The following script reproduces this issue:
```
10.times do
100_000.times do
Ripper.parse("class Foo")
end
puts `ps -o rss= -p #{$$}`
end
```
---
parse.y | 42 ++++++++++++++++++++++++++++--------------
test/ripper/test_ripper.rb | 7 +++++++
2 files changed, 35 insertions(+), 14 deletions(-)
Fix crash in NoMethodError for dummy frames
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
[Bug #19793]
Dummy frames are created at the top level when requiring another file.
While requiring a file, it will try to convert using encodings. Some of
these encodings will not respond to to_str. If method_missing is
redefined on Object, then it will call method_missing and attempt raise
an error. However, the iseq is invalid as it's a dummy frame so it will
write an invalid iseq to the created NoMethodError.
The following script crashes:
```
GC.stress = true
class Object
public :method_missing
end
File.write("/tmp/empty.rb", "")
require "/tmp/empty.rb"
```
With the following backtrace:
```
frame #0: 0x00000001000fa8b8 miniruby`RVALUE_MARKED(obj=4308637824) at gc.c:1638:12
frame #1: 0x00000001000fb440 miniruby`RVALUE_BLACK_P(obj=4308637824) at gc.c:1763:12
frame #2: 0x00000001000facdc miniruby`gc_writebarrier_incremental(a=4308637824, b=4308332208, objspace=0x000000010180b000) at gc.c:8822:9
frame #3: 0x00000001000faad8 miniruby`rb_gc_writebarrier(a=4308637824, b=4308332208) at gc.c:8864:17
frame #4: 0x000000010016aff0 miniruby`rb_obj_written(a=4308637824, oldv=36, b=4308332208, filename="../iseq.c", line=1279) at gc.h:804:9
frame #5: 0x0000000100162a60 miniruby`rb_obj_write(a=4308637824, slot=0x0000000100d09888, b=4308332208, filename="../iseq.c", line=1279) at gc.h:837:5
frame #6: 0x0000000100165b0c miniruby`iseqw_new(iseq=0x0000000100d09880) at iseq.c:1279:9
frame #7: 0x0000000100165a64 miniruby`rb_iseqw_new(iseq=0x0000000100d09880) at iseq.c:1289:12
frame #8: 0x00000001000d8324 miniruby`name_err_init_attr(exc=4309777920, recv=4304780496, method=827660) at error.c:1830:35
frame #9: 0x00000001000d1b80 miniruby`name_err_init(exc=4309777920, mesg=4308332496, recv=4304780496, method=827660) at error.c:1869:12
frame #10: 0x00000001000d1bd4 miniruby`rb_nomethod_err_new(mesg=4308332496, recv=4304780496, method=827660, args=4308332448, priv=0) at error.c:1957:5
frame #11: 0x000000010039049c miniruby`rb_make_no_method_exception(exc=4304914512, format=4308332496, obj=4304780496, argc=1, argv=0x000000016fdfab00, priv=0) at vm_eval.c:959:16
frame #12: 0x00000001003b3274 miniruby`raise_method_missing(ec=0x0000000100b06f40, argc=1, argv=0x000000016fdfab00, obj=4304780496, last_call_status=MISSING_NOENTRY) at vm_eval.c:999:15
frame #13: 0x00000001003945d4 miniruby`rb_method_missing(argc=1, argv=0x000000016fdfab00, obj=4304780496) at vm_eval.c:944:5
...
frame #23: 0x000000010038f5e4 miniruby`rb_vm_call_kw(ec=0x0000000100b06f40, recv=4304780496, id=2865, argc=1, argv=0x000000016fdfab00, me=0x0000000100cbfcf0, kw_splat=0) at vm_eval.c:326:12
frame #24: 0x00000001003c18e4 miniruby`call_method_entry(ec=0x0000000100b06f40, defined_class=4304927952, obj=4304780496, id=2865, cme=0x0000000100cbfcf0, argc=1, argv=0x000000016fdfab00, kw_splat=0) at vm_method.c:2720:20
frame #25: 0x00000001003c440c miniruby`check_funcall_exec(v=6171896792) at vm_eval.c:589:12
frame #26: 0x00000001000dec00 miniruby`rb_vrescue2(b_proc=(miniruby`check_funcall_exec at vm_eval.c:587), data1=6171896792, r_proc=(miniruby`check_funcall_failed at vm_eval.c:596), data2=6171896792, args="Pȗ") at eval.c:919:18
frame #27: 0x00000001000deab0 miniruby`rb_rescue2(b_proc=(miniruby`check_funcall_exec at vm_eval.c:587), data1=6171896792, r_proc=(miniruby`check_funcall_failed at vm_eval.c:596), data2=6171896792) at eval.c:900:17
frame #28: 0x000000010039008c miniruby`check_funcall_missing(ec=0x0000000100b06f40, klass=4304923536, recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000, respond=-1, def=36, kw_splat=0) at vm_eval.c:666:15
frame #29: 0x000000010038fa60 miniruby`rb_check_funcall_default_kw(recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000, def=36, kw_splat=0) at vm_eval.c:703:21
frame #30: 0x000000010038fb04 miniruby`rb_check_funcall(recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000) at vm_eval.c:685:12
frame #31: 0x00000001001c469c miniruby`convert_type_with_id(val=4304780496, tname="String", method=3233, raise=0, index=-1) at object.c:3061:15
frame #32: 0x00000001001c4a4c miniruby`rb_check_convert_type_with_id(val=4304780496, type=5, tname="String", method=3233) at object.c:3153:9
frame #33: 0x00000001002d59f8 miniruby`rb_check_string_type(str=4304780496) at string.c:2571:11
frame #34: 0x000000010014b7b0 miniruby`io_encoding_set(fptr=0x0000000100d09ca0, v1=4304780496, v2=4, opt=4) at io.c:11655:19
frame #35: 0x0000000100139a58 miniruby`rb_io_set_encoding(argc=1, argv=0x000000016fdfb450, io=4308334032) at io.c:13497:5
frame #36: 0x00000001003c0004 miniruby`ractor_safe_call_cfunc_m1(recv=4308334032, argc=1, argv=0x000000016fdfb450, func=(miniruby`rb_io_set_encoding at io.c:13487)) at vm_insnhelper.c:3271:12
...
frame #43: 0x0000000100390b08 miniruby`rb_funcall(recv=4308334032, mid=16593, n=1) at vm_eval.c:1137:12
frame #44: 0x00000001002a43d8 miniruby`load_file_internal(argp_v=6171899936) at ruby.c:2500:5
...
```
---
error.c | 4 +++-
test/ruby/test_require.rb | 15 +++++++++++++++
2 files changed, 18 insertions(+), 1 deletion(-)
Fix autosplat conditions to handle ruby2_keywords case
Autosplat should not occur if there are two arguments but second
argument is an array containing a ruby2_keywords splat. Only
autosplat if a single argument to be yielded to the block, and there
is no splatted flagged keyword hash passed.
Fixes [Bug #19759]
---
test/ruby/test_proc.rb | 26 ++++++++++++++++++++++++++
vm_args.c | 3 ++-
2 files changed, 28 insertions(+), 1 deletion(-)
Use an st table for "too complex" objects
st tables will maintain insertion order so we can marshal dump / load
objects with instance variables in the same order they were set on that
particular instance
[ruby-core:112926] [Bug #19535]
Co-Authored-By: Jemma Issroff <jemmaissroff@gmail.com>
---
gc.c | 10 ++++------
include/ruby/st.h | 2 ++
object.c | 2 +-
ractor.c | 43 ++++++++++++++++++++++---------------------
shape.h | 6 +++---
st.c | 6 ++++++
test/ruby/test_shapes.rb | 21 +++++++++++++++++++++
variable.c | 28 ++++++++++++++--------------
vm_insnhelper.c | 2 +-
9 files changed, 74 insertions(+), 46 deletions(-)
ObjectSpace::WeakMap: clean inverse reference when an entry is
re-assigned
[Bug #19531]
```ruby
wmap[1] = "A"
wmap[1] = "B"
```
In the example above, we need to remove the `"A" => 1` inverse reference
so that when `"A"` is GCed the `1` key isn't deleted.
---
test/ruby/test_weakmap.rb | 17 +++++++++
weakmap.c | 91 ++++++++++++++++++++++++++++++++++++++---------
2 files changed, 91 insertions(+), 17 deletions(-)
Handle unterminated unicode escapes in regexps
This fixes an infinite loop possible after ec3542229b.
For \u{} escapes in regexps, skip validation in the parser, and rely on the regexp
code to handle validation. This is necessary so that invalid unicode escapes in
comments in extended regexps are allowed.
Fixes [Bug #19750]
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
---
parse.y | 97 ++++++++++++++++++++++++++++++++-----------------
test/ruby/test_parse.rb | 16 ++++++++
2 files changed, 79 insertions(+), 34 deletions(-)
hash.c: Fix hash_iter_lev_dec corrupting shape
[Bug #19589]
When decrementing `iter_lev` from `65` to `64` the flags would be
corrupted, causing the shape_id to be invalid.
---
hash.c | 12 +++++++++---
test/ruby/test_hash.rb | 11 +++++++++++
2 files changed, 20 insertions(+), 3 deletions(-)
Fix crash in Time on 32-bit systems
[Bug #19575]
struct vtm is packed causing it to have a size that is not aligned on
32-bit systems. When allocating it on the stack, it will have unaligned
addresses which means that the fields won't be marked by the GC when
scanning the stack (since the GC only marks aligned addresses). This can
cause crashes when the fields are heap allocated objects like Bignums.
This commit moves the flags in struct time_object into struct vtm for
space efficiency and removes the need for packing.
This is an example of a crash:
ruby(rb_print_backtrace+0xd) [0x56848945] ../src/vm_dump.c:785
ruby(rb_vm_bugreport) ../src/vm_dump.c:1101
ruby(rb_assert_failure+0x7a) [0x56671857] ../src/error.c:878
ruby(vm_search_cc+0x0) [0x56666e47] ../src/vm_method.c:1366
ruby(rb_vm_search_method_slowpath) ../src/vm_insnhelper.c:2090
ruby(callable_method_entry+0x5) [0x568232d3] ../src/vm_method.c:1406
ruby(rb_callable_method_entry) ../src/vm_method.c:1413
ruby(gccct_method_search_slowpath) ../src/vm_eval.c:427
ruby(gccct_method_search+0x20f) [0x568237ef] ../src/vm_eval.c:476
ruby(opt_equality_by_mid_slowpath+0x2c) [0x5682388c] ../src/vm_insnhelper.c:2338
ruby(rb_equal+0x37) [0x566fe577] ../src/object.c:133
ruby(rb_big_eq+0x34) [0x56876ee4] ../src/bignum.c:5554
ruby(rb_int_equal+0x14) [0x566f3ed4] ../src/numeric.c:4640
ruby(rb_int_equal) ../src/numeric.c:4634
ruby(vm_call0_cfunc_with_frame+0x6d) [0x568303c2] ../src/vm_eval.c:148
ruby(vm_call0_cfunc) ../src/vm_eval.c:162
ruby(vm_call0_body) ../src/vm_eval.c:208
ruby(rb_funcallv_scope+0xd1) [0x56833971] ../src/vm_eval.c:85
ruby(RB_TEST+0x0) [0x567e8488] ../src/time.c:78
ruby(eq) ../src/time.c:78
ruby(small_vtm_sub) ../src/time.c:1523
ruby(timelocalw+0x23b) [0x567f3e9b] ../src/time.c:1593
ruby(time_s_alloc+0x0) [0x567f536b] ../src/time.c:3698
ruby(time_new_timew) ../src/time.c:2694
ruby(time_s_mktime) ../src/time.c:3698
---
test/ruby/test_time.rb | 7 ++-----
time.c | 57 ++++++++++++++++++++++++--------------------------
timev.h | 7 +++++--
3 files changed, 34 insertions(+), 37 deletions(-)
Fix memory leak for iclass
[Bug #19550]
If !RCLASS_EXT_EMBEDDED (e.g. 32 bit systems) then the rb_classext_t is
allocated throug malloc so it must be freed.
The issue can be seen in the following script:
```
20.times do
100_000.times do
mod = Module.new
Class.new do
include mod
end
end
# Output the Resident Set Size (memory usage, in KB) of the current Ruby process
puts `ps -o rss= -p #{$$}`
end
```
Before this fix, the max RSS is 280MB, while after this change, it's
30MB.
---
gc.c | 2 +-
test/ruby/test_module.rb | 15 +++++++++++++++
2 files changed, 16 insertions(+), 1 deletion(-)
IO::Buffer#resize: Free internal buffer if new size is zero (#7569)
`#resize(0)` on an IO::Buffer with internal buffer allocated will
result in calling `realloc(data->base, 0)`. The behavior of `realloc`
with size = 0 is implementation-defined (glibc frees the object
and returns NULL, while BSDs return an inaccessible object). And
thus such usage is deprecated in standard C (upcoming C23 will make it
UB).
To avoid this problem, just `free`s the memory when the new size is zero.
---
io_buffer.c | 5 +++++
test/ruby/test_io_buffer.rb | 18 ++++++++++++++++++
2 files changed, 23 insertions(+)
Fix crash when allocating classes with newobj hook
We need to zero out the whole slot when running the newobj hook for a
newly allocated class because the slot could be filled with garbage,
which would cause a crash if a GC runs inside of the newobj hook.
For example, the following script crashes:
```
require "objspace"
GC.stress = true
ObjectSpace.trace_object_allocations {
100.times do
Class.new
end
}
```
[Bug #19482]
---
gc.c | 8 +++++++-
test/objspace/test_objspace.rb | 7 +++++++
2 files changed, 14 insertions(+), 1 deletion(-)
YJIT: Generate Block::entry_exit with block entry PC
Previously, when Block::entry_exit is requested from any instruction
that is not the first one in the block, we generated the exit with an
incorrect PC. We should always be using the PC for the entry of the
block for Block::entry_exit.
It was a simple typo. The bug was [introduced][1] while we were
refactoring to use the current backend. Later, we had a chance to spot
this issue while [preparing][2] to enable unused variable warnings, but
didn't spot the issue.
Fixes [Bug #19463]
[1]: 27fcab995e
[2]: 31461c7e0e
---
test/ruby/test_yjit.rb | 41 +++++++++++++++++++++++++++++++++++++++++
yjit/src/codegen.rs | 4 ++--
2 files changed, 43 insertions(+), 2 deletions(-)
The class variable cache that was added in
https://github.com/ruby/ruby/pull/4544 changed the behavior of class
variables on cloned classes. As reported when a class is cloned AND a
class variable was set, and the class variable was read from the
original class, reading a class variable from the cloned class would
return the value from the original class.
This was happening because the IC (inline cache) is stored on the ISEQ
which is shared between the original and cloned class, therefore they
share the cache too.
To fix this we are now storing the `cref` in the cache so that we can
check if it's equal to the current `cref`. If it's different we don't
want to read from the cache. If it's the same we do. Cloned classes
don't share the same cref with their original class.
This will need to be backported to 3.1 in addition to 3.2 since the bug
exists in both versions.
We also added a marking function which was missing.
Fixes [Bug #19379]
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
When a class with a class variable is cloned we need to also copy the
cvar cache table from the original table to the clone. I found this bug
while working on fixing [Bug #19379]. While this does not fix that bug
directly it is still a required change to fix another bug revealed by
the fix in https://github.com/ruby/ruby/pull/7265
This needs to be backported to 3.2.x and 3.1.x.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
Fix interpreter crash caused by RUBY_INTERNAL_EVENT_NEWOBJ + Ractors
When a Ractor is created whilst a tracepoint for
RUBY_INTERNAL_EVENT_NEWOBJ is active, the interpreter crashes. This is
because during the early setup of the Ractor, the stdio objects are
created, which allocates Ruby objects, which fires the tracepoint.
However, the tracepoint machinery tries to dereference the control frame
(ec->cfp->pc), which isn't set up yet and so crashes with a null pointer
dereference.
Fix this by not firing GC tracepoints if cfp isn't yet set up.
---
gc.c | 1 +
test/objspace/test_ractor.rb | 17 +++++++++++++++++
2 files changed, 18 insertions(+)
create mode 100644 test/objspace/test_ractor.rb
Add guard to compaction test in WeakMap
Some platforms don't support compaction, so we should skip this test.
---
test/ruby/test_weakmap.rb | 2 ++
1 file changed, 2 insertions(+)
ObjectSpace::WeakMap: fix compaction support
[Bug #19529]
`rb_gc_update_tbl_refs` can't be used on `w->obj2wmap` because it's
not a `VALUE -> VALUE` table, but a `VALUE -> VALUE *` table, so
we need some dedicated iterator.
---
test/ruby/test_weakmap.rb | 8 ++++++++
weakmap.c | 37 ++++++++++++++++++++++++++++++++++++-
2 files changed, 44 insertions(+), 1 deletion(-)
Fix crash during compaction
[Bug #19529]
The fix for [Bug #19529] in commit 548086b contained a bug that crashes
on the following script:
```
wm = ObjectSpace::WeakMap.new
obj = Object.new
100.times do
wm[Object.new] = obj
GC.start
end
GC.compact
```
---
test/ruby/test_weakmap.rb | 10 ++++++++++
weakmap.c | 2 +-
2 files changed, 11 insertions(+), 1 deletion(-)
Fix incorrect size of WeakMap buffer
In wmap_final_func, j is the number of elements + 1 (since j also
includes the length at the 0th index), so we should resize the buffer
to size j and the new length is j - 1.
---
weakmap.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
