Fix memory leak in complemented method entries
[Bug #19894]
When a copy of a complemented method entry is created, there are two
issues:
1. IMEMO_FL_USER3 is not copied, so the complemented status is not
copied over.
2. In rb_method_entry_clone we increment both alias_count and
complemented_count. However, when we free the method entry in
rb_method_definition_release, we only decrement one of the two
counters, resulting in the rb_method_definition_t being leaked.
Co-authored-by: Adam Hess <adamhess1991@gmail.com>
---
method.h | 5 +++--
test/ruby/test_module.rb | 29 +++++++++++++++++++++++++++++
vm_method.c | 8 +++++---
3 files changed, 37 insertions(+), 5 deletions(-)
Fix write barrier order for `klass` to `cme` edge
Previously, the following crashes with
`CFLAGS=-DRGENGC_CHECK_MODE=2 -DRUBY_DEBUG=1 -fno-inline`:
$ ./miniruby -e 'GC.stress = true; Marshal.dump({})'
It crashes with a write barrier (WB) miss assertion on an edge from the
`Hash` class object to a newly allocated negative method entry.
This is due to usages of vm_ccs_create() running the WB too early,
before the method entry is inserted into the cc table, so before the
reference edge is established. The insertion can trigger GC and promote
the class object, so running the WB after the insertion is necessary.
Move the insertion into vm_ccs_create() and run the WB after the
insertion.
Discovered on CI:
http://ci.rvm.jp/results/trunk-asserts@ruby-sp2-docker/4391770
---
vm_eval.c | 3 +--
vm_insnhelper.c | 10 ++++++----
vm_method.c | 3 +--
3 files changed, 8 insertions(+), 8 deletions(-)
Fix visibility of alias of zsuper methods
This was broken by 71c746379d.
Fixes [Bug #18600]
---
test/ruby/test_alias.rb | 10 ++++++++++
vm_method.c | 1 +
2 files changed, 11 insertions(+)
An alias can suppress method redefinition warning
---
test/ruby/test_alias.rb | 11 +++++++++++
1 file changed, 11 insertions(+)
Fix memory leak at the same named alias [Bug #18516]
When aliasing a method to the same name method, set a separate bit
flag on that method definition, instead of the reference count
increment. Although this kind of alias has no actual effect at
runtime, is used as the hack to suppress the method re-definition
warning.
---
method.h | 1 +
test/ruby/test_alias.rb | 18 ++++++++++++++++++
vm_method.c | 9 ++++++++-
3 files changed, 27 insertions(+), 1 deletion(-)
Some callable method entries (cme) can be a key of `overloaded_cme_table`
and the keys should be pinned because the table is numtable (VALUE is a key).
Before the patch GC checks the cme is in `overloaded_cme_table` by looking up
the table, but it needs VM locking.
It works well in normal GC marking because it is protected by the VM lock,
but it doesn't work on `rb_objspace_reachable_objects_from` because it doesn't
use VM lock.
Now, the number of target cmes are small enough, I decide to pin down
all possible cmes instead of using looking up the table.
`overloaded_cme_table` keeps cme -> monly_cme pairs to manage
corresponding `monly_cme` for `cme`. The lifetime of the `monly_cme`
should be longer than `monly_cme`, but the previous patch losts the
reference to the living `monly_cme`.
Now `overloaded_cme_table` values are always root (keys are only weak
reference), it means `monly_cme` does not freed until corresponding
`cme` is invalidated.
To make managing easy, move `overloaded_cme_table` to `rb_vm_t`.
`def` (`rb_method_definition_t`) is shared by multiple callable
method entries (cme, `rb_callable_method_entry_t`).
There are two issues:
* old -> young reference: `cme1->def->mandatory_only_cme = monly_cme`
if `cme1` is young and `monly_cme` is young, there is no problem.
Howevr, another old `cme2` can refer `def`, in this case, old `cme2`
points young `monly_cme` and it violates gengc assumption.
* cme can have different `defined_class` but `monly_cme` only has
one `defined_class`. It does not make sense and `monly_cme`
should be created for a cme (not `def`).
To solve these issues, this patch allocates `monly_cme` per `cme`.
`cme` does not have another room to store a pointer to the `monly_cme`,
so this patch introduces `overloaded_cme_table`, which is weak key map
`[cme] -> [monly_cme]`.
`def::body::iseqptr::monly_cme` is deleted.
The first issue is reported by Alan Wu.
Previously, each of these methods returned self, but it is
more useful to return arguments, to allow for simpler method
decorators, such as:
```ruby
cached private def foo; some_long_calculation; end
```
Where cached sets up caching for the method.
For each of these methods, the following behavior is used:
1) No arguments returns nil
2) Single argument is returned
3) Multiple arguments are returned as an array
The single argument case is really the case we are trying to
optimize for, for the same reason that def was changed to return
a symbol for the method.
Idea and initial patch from Herwin Quarantainenet.
Implements [Feature #12495]
Builtin methods do not always have their mandatory_only_cme created (it
is only created when called with only mandatory parameters), so it could
be null. If we try to clear the cme, it will crash because it is null.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
Compare with the C methods, A built-in methods written in Ruby is
slower if only mandatory parameters are given because it needs to
check the argumens and fill default values for optional and keyword
parameters (C methods can check the number of parameters with `argc`,
so there are no overhead). Passing mandatory arguments are common
(optional arguments are exceptional, in many cases) so it is important
to provide the fast path for such common cases.
`Primitive.mandatory_only?` is a special builtin function used with
`if` expression like that:
```ruby
def self.at(time, subsec = false, unit = :microsecond, in: nil)
if Primitive.mandatory_only?
Primitive.time_s_at1(time)
else
Primitive.time_s_at(time, subsec, unit, Primitive.arg!(:in))
end
end
```
and it makes two ISeq,
```
def self.at(time, subsec = false, unit = :microsecond, in: nil)
Primitive.time_s_at(time, subsec, unit, Primitive.arg!(:in))
end
def self.at(time)
Primitive.time_s_at1(time)
end
```
and (2) is pointed by (1). Note that `Primitive.mandatory_only?`
should be used only in a condition of an `if` statement and the
`if` statement should be equal to the methdo body (you can not
put any expression before and after the `if` statement).
A method entry with `mandatory_only?` (`Time.at` on the above case)
is marked as `iseq_overload`. When the method will be dispatch only
with mandatory arguments (`Time.at(0)` for example), make another
method entry with ISeq (2) as mandatory only method entry and it
will be cached in an inline method cache.
The idea is similar discussed in https://bugs.ruby-lang.org/issues/16254
but it only checks mandatory parameters or more, because many cases
only mandatory parameters are given. If we find other cases (optional
or keyword parameters are used frequently and it hurts performance),
we can extend the feature.
I'm looking through the places where YJIT needs notifications. It looks
like these changes to gc.c and vm_callinfo.h have become unnecessary
since 84ab77ba592. This commit just makes the diff against upstream
smaller, but otherwise shouldn't change any behavior.
Added UJIT_CHECK_MODE. Set to 1 to double check method dispatch in
generated code.
It's surprising to me that we need to watch both cc and cme. There might
be opportunities to simplify there.
Point out that the method should be used for backwards compatibility
with code prior to Ruby 3.0 instead of Ruby 2.7. It's still needed
in Ruby 2.7. It isn't needed in Ruby 3.0, as the methods using it
could switch to delegating both positional and keyword arguments.
Add a link to the www.ruby-lang.org web page that goes into detail
describing when and how ruby2_keywords should be used.
Since refinement search is always performed, these entries should always
be public. The method entry that the refinement search returns decides
the visibility.
Fixes [Bug #17822]
To invalidate some callable method entries, we replace the entry in the
class. Most types of method entries are on the method table of the
origin class, but refinement entries without an orig_me are housed in
the method table of the class itself. They are there because refinements
take priority over prepended methods.
By unconditionally inserting a copy of the refinement entry into the
origin class, clearing the method cache created situations where there
are refinement entry duplicates in the lookup chain, leading to infinite
loops and other problems.
Update the replacement logic to use the right class that houses the
method entry. Also, be more selective about cache invalidation when
moving refinement entries for prepend. This avoids calling
clear_method_cache_by_id_in_class() before refinement entries are in the
place it expects.
[Bug #17806]
If a class has been refined but does not have an origin class,
there is a single method entry marked with VM_METHOD_TYPE_REFINED,
but it contains the original method entry. If the original method
entry is present, we shouldn't skip the method when searching even
when skipping refined methods.
Fixes [Bug #17519]
Previously, attempting to change the visibility of a method in a
singleton class for a class/module that is prepended to and refined
would raise a NoMethodError.
Fixes [Bug #17519]
negative cache on a class which does not have subclasses was not
invalidated, but it should be invalidated because other classes
can cache this negative cache.
[Bug #17553]
rb_funcall* (rb_funcall(), rb_funcallv(), ...) functions invokes
Ruby's method with given receiver. Ruby 2.7 introduced inline method
cache with static memory area. However, Ruby 3.0 reimplemented the
method cache data structures and the inline cache was removed.
Without inline cache, rb_funcall* searched methods everytime.
Most of cases per-Class Method Cache (pCMC) will be helped but
pCMC requires VM-wide locking and it hurts performance on
multi-Ractor execution, especially all Ractors calls methods
with rb_funcall*.
This patch introduced Global Call-Cache Cache Table (gccct) for
rb_funcall*. Call-Cache was introduced from Ruby 3.0 to manage
method cache entry atomically and gccct enables method-caching
without VM-wide locking. This table solves the performance issue
on multi-ractor execution.
[Bug #17497]
Ruby-level method invocation does not use gccct because it has
inline-method-cache and the table size is limited. Basically
rb_funcall* is not used frequently, so 1023 entries can be enough.
We will revisit the table size if it is not enough.
Method cache can be cleared during lazy sweeping. An object that will
be collected during lazy sweep *should not* have it's method cache
cleared. Soon-to-be-collected objects can be in an inconsistent state and
this can lead to a crash. This patch just leaves early if the object is
going to be collected.
Fixes [Bug #17536]
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Co-Authored-By: Alan Wu <XrXr@users.noreply.github.com>
separate some fields from rb_ractor_t to rb_ractor_pub and put it
at the beggining of rb_ractor_t and declare it in vm_core.h so
vm_core.h can access rb_ractor_pub fields.
Now rb_ec_ractor_hooks() is a complete inline function and no
MJIT related issue.
Also document that both :deprecated and :experimental are supported
:category option values.
The locations where warnings were marked as deprecation warnings
was previously reviewed by shyouhei.
Comment a couple locations where deprecation warnings should probably
be used but are not currently used because deprecation warning
enablement has not occurred at the time they are called
(RUBY_FREE_MIN, RUBY_HEAP_MIN_SLOTS, -K).
Add assert_deprecated_warn to test assertions. Use this to simplify
some tests, and fix failing tests after marking some warnings with
deprecated category.
