IO::Buffer#resize: Free internal buffer if new size is zero (#7569)
`#resize(0)` on an IO::Buffer with internal buffer allocated will
result in calling `realloc(data->base, 0)`. The behavior of `realloc`
with size = 0 is implementation-defined (glibc frees the object
and returns NULL, while BSDs return an inaccessible object). And
thus such usage is deprecated in standard C (upcoming C23 will make it
UB).
To avoid this problem, just `free`s the memory when the new size is zero.
---
io_buffer.c | 5 +++++
test/ruby/test_io_buffer.rb | 18 ++++++++++++++++++
2 files changed, 23 insertions(+)
Fix crash when allocating classes with newobj hook
We need to zero out the whole slot when running the newobj hook for a
newly allocated class because the slot could be filled with garbage,
which would cause a crash if a GC runs inside of the newobj hook.
For example, the following script crashes:
```
require "objspace"
GC.stress = true
ObjectSpace.trace_object_allocations {
100.times do
Class.new
end
}
```
[Bug #19482]
---
gc.c | 8 +++++++-
test/objspace/test_objspace.rb | 7 +++++++
2 files changed, 14 insertions(+), 1 deletion(-)
Pass -Werror=lto-type-mismatch for GCC LTO jobs
This helps to find possible LTO miscompilations earlier. See also
https://github.com/ruby/ruby/pull/7695.
---
.github/workflows/compilers.yml | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
give up checking old compilers
These old compilers needed old OSes, which are getting EODed. We
cannot maintain healthy binary of them (building compilers on our
own is quite haed). Let us just retire them. Fixes [Bug #19353]
---
.github/workflows/compilers.yml | 6 ------
1 file changed, 6 deletions(-)
YJIT: Generate Block::entry_exit with block entry PC
Previously, when Block::entry_exit is requested from any instruction
that is not the first one in the block, we generated the exit with an
incorrect PC. We should always be using the PC for the entry of the
block for Block::entry_exit.
It was a simple typo. The bug was [introduced][1] while we were
refactoring to use the current backend. Later, we had a chance to spot
this issue while [preparing][2] to enable unused variable warnings, but
didn't spot the issue.
Fixes [Bug #19463]
[1]: 27fcab995e
[2]: 31461c7e0e
---
test/ruby/test_yjit.rb | 41 +++++++++++++++++++++++++++++++++++++++++
yjit/src/codegen.rs | 4 ++--
2 files changed, 43 insertions(+), 2 deletions(-)
Fix interpreter crash caused by RUBY_INTERNAL_EVENT_NEWOBJ + Ractors
When a Ractor is created whilst a tracepoint for
RUBY_INTERNAL_EVENT_NEWOBJ is active, the interpreter crashes. This is
because during the early setup of the Ractor, the stdio objects are
created, which allocates Ruby objects, which fires the tracepoint.
However, the tracepoint machinery tries to dereference the control frame
(ec->cfp->pc), which isn't set up yet and so crashes with a null pointer
dereference.
Fix this by not firing GC tracepoints if cfp isn't yet set up.
---
gc.c | 1 +
test/objspace/test_ractor.rb | 17 +++++++++++++++++
2 files changed, 18 insertions(+)
create mode 100644 test/objspace/test_ractor.rb
Add guard to compaction test in WeakMap
Some platforms don't support compaction, so we should skip this test.
---
test/ruby/test_weakmap.rb | 2 ++
1 file changed, 2 insertions(+)
ObjectSpace::WeakMap: fix compaction support
[Bug #19529]
`rb_gc_update_tbl_refs` can't be used on `w->obj2wmap` because it's
not a `VALUE -> VALUE` table, but a `VALUE -> VALUE *` table, so
we need some dedicated iterator.
---
test/ruby/test_weakmap.rb | 8 ++++++++
weakmap.c | 37 ++++++++++++++++++++++++++++++++++++-
2 files changed, 44 insertions(+), 1 deletion(-)
Fix crash during compaction
[Bug #19529]
The fix for [Bug #19529] in commit 548086b contained a bug that crashes
on the following script:
```
wm = ObjectSpace::WeakMap.new
obj = Object.new
100.times do
wm[Object.new] = obj
GC.start
end
GC.compact
```
---
test/ruby/test_weakmap.rb | 10 ++++++++++
weakmap.c | 2 +-
2 files changed, 11 insertions(+), 1 deletion(-)
Fix incorrect size of WeakMap buffer
In wmap_final_func, j is the number of elements + 1 (since j also
includes the length at the 0th index), so we should resize the buffer
to size j and the new length is j - 1.
---
weakmap.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
[Bug #19471] `Regexp.compile` should handle keyword arguments
As well as `Regexp.new`, it should pass keyword arguments to the
`Regexp#initialize` method.
---
re.c | 2 +-
test/ruby/test_regexp.rb | 5 +++++
2 files changed, 6 insertions(+), 1 deletion(-)
Fix frozen status loss when moving objects
[Bug #19536]
When objects are moved between size pools, their frozen status is lost
in the shape. This will cause the frozen check to be bypassed when there
is an inline cache. For example, the following script should raise a
FrozenError, but doesn't on Ruby 3.2 and master.
class A
def add_ivars
@a = @b = @c = @d = 1
end
def set_a
@a = 10
end
end
a = A.new
a.add_ivars
a.freeze
b = A.new
b.add_ivars
b.set_a # Set the inline cache in set_a
GC.verify_compaction_references(expand_heap: true, toward: :empty)
a.set_a
---
shape.c | 2 +-
test/ruby/test_gc_compact.rb | 28 ++++++++++++++++++++++++++++
2 files changed, 29 insertions(+), 1 deletion(-)
[Bug #19161] Check for TLS usability
On all platforms using GCC, even other than darwin.
---
configure.ac | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
Fix indentation in vm_setivar_default
---
vm_insnhelper.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
[Bug #19469] Fix crash when resizing generic iv list
The following script can sometimes trigger a crash:
```ruby
GC.stress = true
class Array
def foo(bool)
if bool
@a = 1
@b = 2
@c = 1
else
@c = 1
end
end
end
obj = []
obj.foo(true)
obj2 = []
obj2.foo(false)
obj3 = []
obj3.foo(true)
```
This is because vm_setivar_default calls rb_ensure_generic_iv_list_size
to resize the iv list. However, the call to gen_ivtbl_resize reallocs
the iv list, and then inserts into the generic iv table. If the
st_insert triggers a GC then the old iv list will be read during
marking, causing a use-after-free bug.
Co-Authored-By: Jemma Issroff <jemmaissroff@gmail.com>
---
internal/variable.h | 2 +-
variable.c | 23 ++++++++++++++++++-----
vm_insnhelper.c | 4 ++--
3 files changed, 21 insertions(+), 8 deletions(-)
YJIT: Detect and reject `send(:alias_for_send, :foo)`
Previously, YJIT failed to put the stack into the correct shape when
`BasicObject#send` calls an alias method for the send method itself.
This can manifest as strange `NoMethodError`s in the final non-send
receiver, as [seen][1] with the kt-paperclip gem. I also found a case
where it makes YJIT fail the stack size assertion while compiling
`leave`.
YJIT's `BasicObject#__send__` implementation already rejects sends to
`send`, but didn't detect sends to aliases of `send`. Adjust the
detection and reject these cases.
Fixes [Bug #19464]
[1]: https://github.com/Shopify/yjit/issues/306
---
test/ruby/test_yjit.rb | 20 ++++++++++++++++++++
yjit/src/codegen.rs | 25 ++++++++++---------------
2 files changed, 30 insertions(+), 15 deletions(-)
[Bug #19459] Remove unnecessary always-true checks (#7362)
`length` is a required argument for `IO::Buffer#read` and
`IO::Buffer#write` methods, and `argc` is already checked with
`rb_check_arity`. Also fix the call-seq of `IO::Buffer#read`.
---
io_buffer.c | 24 +++++++-----------------
1 file changed, 7 insertions(+), 17 deletions(-)
Marshal.load: restore instance variables on Regexp
[Bug #19439]
The instance variables were restore on the Regexp source,
not the regexp itself.
Unfortunately we have a bit of a chicken and egg problem.
The source holds the encoding, and the encoding need to be set on
the source to be able to instantiate the Regexp.
So the instance variables have to be read on the `source`.
To correct this we transfert the instance variables after
instantiating the Regexp.
The only way to avoid this would be to read the instance variable
twice and rewind.
---
marshal.c | 20 ++++++++++++++++++--
spec/ruby/core/marshal/shared/load.rb | 11 +++++++++++
2 files changed, 29 insertions(+), 2 deletions(-)
YJIT: jit_prepare_routine_call() for String#+@ missing
We saw SEGVs due to this when running with StackProf, which needs a
correct PC for RUBY_INTERNAL_EVENT_NEWOBJ, the same event used for
ObjectSpace allocation tracing.
[Bug #19444]
---
test/ruby/test_yjit.rb | 27 +++++++++++++++++++++++++++
yjit/src/codegen.rs | 5 ++++-
2 files changed, 31 insertions(+), 1 deletion(-)
YJIT: Fix false assumption that String#+@ => ::String
Could return a subclass.
[Bug #19444]
---
test/ruby/test_yjit.rb | 17 +++++++++++++++++
yjit/src/codegen.rs | 10 +++++++---
2 files changed, 24 insertions(+), 3 deletions(-)
Remove ibf_dumper's WB_PROTECTED status
It doesn't have the right write barriers in place. For example, there is
rb_mark_set(dump->global_buffer.obj_table);
in the mark function, but there is no corresponding write barrier when
adding to the table in the
`ibf_dump_object() -> ibf_table_find_or_insert() -> st_insert()` code path.
To insert write barrier correctly, we need to store the T_STRUCT VALUE
inside `struct ibf_dump`. Instead of doing that, let's just demote it
to WB unproected for correctness. These dumper object are ephemeral so
there is not a huge benefit for having them WB protected.
Users of the bootsnap gem ran into crashes due to this issue:
https://github.com/Shopify/bootsnap/issues/436
Fixes [Bug #19419]
---
compile.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
YJIT: Use the system page size when the code page size is too small
(#7267)
Previously on ARM64 Linux systems that use 64 KiB pages
(`CONFIG_ARM64_64K_PAGES=y`), YJIT was panicking on boot due to a failed
assertion.
The assertion was making sure that code GC can free the last code page
that YJIT manages without freeing unrelated memory. YJIT prefers picking
16 KiB as the granularity at which to free code memory, but when the
system can only free at 64 KiB granularity, that is not possible.
The fix is to use the system page size as the code page size when the
system page size is 64 KiB. Continue to use 16 KiB as the code page size
on common systems that use 16/4 KiB pages.
Add asserts to code_gc() and free_page() about code GC's assumptions.
Fixes [Bug #19400]
---
yjit/src/asm/mod.rs | 78 ++++++++++++++++++++++++++++++++------------------
yjit/src/codegen.rs | 2 --
yjit/src/virtualmem.rs | 13 +++++++++
3 files changed, 63 insertions(+), 30 deletions(-)
YJIT: Fix assertion for partially mapped last pages (#7337)
Follows up [Bug #19400]
---
test/ruby/test_yjit.rb | 19 +++++++++++++++++++
yjit/src/asm/mod.rs | 2 +-
yjit/src/virtualmem.rs | 18 +++++++++++++-----
3 files changed, 33 insertions(+), 6 deletions(-)
Only emit circular dependency warning for owned thread shields [Bug
#19415]
If multiple threads attemps to load the same file concurrently
it's not a circular dependency issue.
So we check that the existing ThreadShield is owner by the current
fiber before warning about circular dependencies.
---
internal/thread.h | 1 +
load.c | 3 ++-
spec/ruby/core/kernel/shared/require.rb | 11 +++++++++++
spec/ruby/fixtures/code/concurrent_require_fixture.rb | 4 ++++
test/ruby/test_require.rb | 3 ---
thread.c | 11 +++++++++++
6 files changed, 29 insertions(+), 4 deletions(-)
create mode 100644 spec/ruby/fixtures/code/concurrent_require_fixture.rb
Use Thread.pass until thread.stop? to wait for thread to block
[Bug #19415]
It should be more reliable
---
spec/ruby/fixtures/code/concurrent_require_fixture.rb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
mkconfig: Map `includedir` only for system ruby
Only when installing to the system path on macOS, prepend '$(SDKROOT)'
and remap `includedir`.
Fix https://github.com/rbenv/ruby-build/discussions/2123
---
test/mkmf/test_config.rb | 4 ++--
test/test_rbconfig.rb | 9 ---------
tool/mkconfig.rb | 4 +++-
3 files changed, 5 insertions(+), 12 deletions(-)
[Bug #19399] Parsing invalid heredoc inside block parameter
Although this is of course invalid as Ruby code, allow to just parse
and tokenize.
---
ext/ripper/lib/ripper/lexer.rb | 2 +-
test/ripper/test_lexer.rb | 12 ++++++++++++
2 files changed, 13 insertions(+), 1 deletion(-)
[Bug #19398] Memory leak in WeakMap
There's a memory leak in ObjectSpace::WeakMap due to not freeing
the `struct weakmap`. It can be seen in the following script:
```
100.times do
10000.times do
ObjectSpace::WeakMap.new
end
# Output the Resident Set Size (memory usage, in KB) of the current Ruby process
puts `ps -o rss= -p #{$$}`
end
```
---
gc.c | 1 +
test/ruby/test_weakmap.rb | 9 +++++++++
2 files changed, 10 insertions(+)
YJIT: Crash with rb_bug() when panicking
Helps with getting good bug reports in the wild. Intended to be
backported to the 3.2.x series.
---
yjit/bindgen/src/main.rs | 3 +++
yjit/src/cruby_bindings.inc.rs | 1 +
yjit/src/yjit.rs | 29 +++++++++++++++++++++++++++--
3 files changed, 31 insertions(+), 2 deletions(-)
[PATCH 1/4] YJIT: Move CodegenGlobals::freed_pages into an Rc
This allows for supplying a freed_pages vec in Rust tests. We need it so we
can test scenarios that occur after code GC.
---
yjit/src/asm/mod.rs | 48 +++++++++++++++++++++++++++++++++------------
yjit/src/codegen.rs | 16 ++++-----------
2 files changed, 39 insertions(+), 25 deletions(-)
Subject: [PATCH 2/4] YJIT: other_cb is None in tests
Since the other cb is in CodegenGlobals, and we want Rust tests to be
self-contained.
---
yjit/src/asm/mod.rs | 1 +
1 file changed, 1 insertion(+)
Subject: [PATCH 3/4] YJIT: ARM64: Move functions out of arm64_emit()
---
yjit/src/backend/arm64/mod.rs | 180 +++++++++++++++++-----------------
1 file changed, 90 insertions(+), 90 deletions(-)
Subject: [PATCH 4/4] YJIT: ARM64: Fix long jumps to labels
Previously, with Code GC, YJIT panicked while trying to emit a B.cond
instruction with an offset that is not encodable in 19 bits. This only
happens when the code in an assembler instance straddles two pages.
To fix this, when we detect that a jump to a label can land on a
different page, we switch to a fresh new page and regenerate all the
code in the assembler there. We still assume that no one assembler has
so much code that it wouldn't fit inside a fresh new page.
[Bug #19385]
---
yjit/src/backend/arm64/mod.rs | 65 ++++++++++++++++++++++++++++++++---
1 file changed, 60 insertions(+), 5 deletions(-)
Fix parsing of regexps that toggle extended mode on/off inside regexp
This was broken in ec3542229b. That commit
didn't handle cases where extended mode was turned on/off inside the
regexp. There are two ways to turn extended mode on/off:
```
/(?-x:#y)#z
/x =~ '#y'
/(?-x)#y(?x)#z
/x =~ '#y'
```
These can be nested inside the same regexp:
```
/(?-x:(?x)#x
(?-x)#y)#z
/x =~ '#y'
```
As you can probably imagine, this makes handling these regexps
somewhat complex. Due to the nesting inside portions of regexps,
the unassign_nonascii function needs to be recursive. In
recursive mode, it needs to track both opening and closing
parentheses, similar to how it already tracked opening and
closing brackets for character classes.
When scanning the regexp and coming to `(?` not followed by `#`,
scan for options, and use `x` and `i` to determine whether to
turn on or off extended mode. For `:`, indicting only the
current regexp section should have the extended mode
switched, recurse with the extended mode set or unset. For `)`,
indicating the remainder of the regexp (or current regexp portion
if already recursing) should turn extended mode on or off, just
change the extended mode flag and keep scanning.
While testing this, I noticed that `a`, `d`, and `u` are accepted
as options, in addition to `i`, `m`, and `x`, but I can't see
where those options are documented. I'm not sure whether or not
handling `a`, `d`, and `u` as options is a bug.
Fixes [Bug #19379]
---
re.c | 153 +++++++++++++++++++++++++++++++++++++----------
test/ruby/test_regexp.rb | 56 +++++++++++++++++
2 files changed, 176 insertions(+), 33 deletions(-)
bignum.c: rb_int_parse_cstr handle `0` strings
[Bug #19390]
We shouldn't check the string length when skipping zeros, as the
string might only contains zero characters, resulting in an empty string.
---
bignum.c | 1 -
test/ruby/test_time.rb | 4 ++++
2 files changed, 4 insertions(+), 1 deletion(-)
