Commit graph

209 commits

Author SHA1 Message Date
John Hawthorn
f483befd90 Add shape_id to RBasic under 32 bit
This makes `RBobject` `4B` larger on 32 bit systems
but simplifies the implementation a lot.

[Feature #21353]

Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
2025-05-26 10:31:54 +02:00
Peter Zhu
e00c46017b Drop unnecessary compiler guards for memory_sanitizer
We unpoison slots allocated out of the GC, so we don't need to disable
the assertions that read from the memory.
2025-05-23 14:35:56 -04:00
Peter Zhu
9130023cf5 Remove dependency on bits.h in default.c when BUILDING_MODULAR_GC
We can assume that the compiler will have __builtin_clzll so we can implement
nlz_int64 using that.
2025-05-23 14:35:56 -04:00
Peter Zhu
3487117e55 [ruby/mmtk] Fix object ID in rb_gc_impl_define_finalizer
The 0th element of the finalizer table array should be the object ID.

75e4a82652
2025-05-21 14:35:36 +00:00
Peter Zhu
27b0638386 [ruby/mmtk] Fix object ID for finalizers
We should get the object ID for finalizers in rb_gc_impl_define_finalizer
instead of when we create the finalizer job in make_final_job because
when we are in multi-Ractor mode, object ID needs to walk the references
which allocates an identity hash table. We cannot allocate in make_final_job
because it is in a MMTk worker thread.

922f22a690
2025-05-21 02:30:33 +00:00
Peter Zhu
b043abc048 Only define RVALUE_OVERHEAD if undefined
This allows RVALUE_OVERHEAD to be defined elsewhere.
2025-05-20 12:57:23 -04:00
Jean Boussier
1e33a451bb gc: Execute run_final with the lock held
The finalizer table can't be read nor modified without the VM lock.
2025-05-16 20:16:52 +02:00
Jean Boussier
ec8900e3eb rb_gc_impl_copy_finalizer: generate a new object id
Fix a regression introduced by: https://github.com/ruby/ruby/pull/13155
2025-05-16 20:16:52 +02:00
Jean Boussier
a294427017 Add missing lock to rb_gc_impl_copy_finalizer 2025-05-16 20:16:52 +02:00
Peter Zhu
04f538c144 Remove dependency on sanitizers.h in default.c when BUILDING_MODULAR_GC 2025-05-15 14:13:53 -04:00
Jean Boussier
ed632cd0ba Add missing lock in rb_gc_impl_undefine_finalizer
The table is global so accesses must be synchronized.
2025-05-15 13:32:08 +02:00
Jean Boussier
3d1b8e7298 newobj_fill: don't assume RBasic size
The previous implementation assumed `RBasic` size is `2 * sizeof(VALUE)`,
might as well not make assumption and use a proper `sizeof`.

Co-Authored-By: John Hawthorn <john@hawthorn.email>
2025-05-15 13:26:26 +02:00
Alan Wu
92b218fbc3 YJIT: ZJIT: Allow both JITs in the same build
This commit allows building YJIT and ZJIT simultaneously, a "combo
build". Previously, `./configure --enable-yjit --enable-zjit` failed. At
runtime, though, only one of the two can be enabled at a time.

Add a root Cargo workspace that contains both the yjit and zjit crate.
The common Rust build integration mechanisms are factored out into
defs/jit.mk.

Combo YJIT+ZJIT dev builds are supported; if either JIT uses
`--enable-*=dev`, both of them are built in dev mode.

The combo build requires Cargo, but building one JIT at a time with only
rustc in release build remains supported.
2025-05-15 00:39:03 +09:00
Jean Boussier
f9c3feccf4 Rename id_to_obj_tbl -> id2ref_tbl
As well as associated functions, this should make it more obvious
what the purpose is.
2025-05-14 11:41:14 +02:00
John Hawthorn
3306d7d2f9 Only clear Ractor cache in child
This avoids a race condition where we were clearing the cache from
another ractor while it was in use. Oops!

Fixes this failure http://ci.rvm.jp/results/master@oci-aarch64/5750416
2025-05-09 16:15:54 -07:00
Jean Boussier
15e3675ecc Remove dead code in rb_gc_impl_ractor_cache_free
Followup: https://github.com/ruby/ruby/pull/13286
2025-05-09 11:13:52 +02:00
John Hawthorn
30ef0f180b Fix allocation count when forking with Ractors
After fork we reset to single ractor mode (which IMO we shouldn't do,
but it requires more work to fix) and so we need to add the pending
object counts back to the main heap.
2025-05-09 00:55:14 -07:00
Peter Zhu
c18bedcdbb Remove dependency on debug_counter.h when BUILDING_MODULAR_GC
This allows the default GC to not need debug_counter.h when building as a
modular GC.
2025-05-08 10:36:27 -04:00
Peter Zhu
3f5080e767 Stop checking for USE_DEBUG_COUNTER in default.c
We don't need to check for USE_DEBUG_COUNTER because the code is no-op
if USE_DEBUG_COUNTER is not enabled.
2025-05-08 10:36:27 -04:00
Jean Boussier
2d1241ba97 Get rid of RB_GC_VM_ID_TO_OBJ_TABLE_KEYS 2025-05-08 07:58:05 +02:00
Jean Boussier
f48e45d1e9 Move object_id in object fields.
And get rid of the `obj_to_id_tbl`

It's no longer needed, the `object_id` is now stored inline
in the object alongside instance variables.

We still need the inverse table in case `_id2ref` is invoked, but
we lazily build it by walking the heap if that happens.

The `object_id` concern is also no longer a GC implementation
concern, but a generic implementation.

Co-Authored-By: Matt Valentine-House <matt@eightbitraptor.com>
2025-05-08 07:58:05 +02:00
Jean Boussier
0ea210d1ea Rename ivptr -> fields, next_iv_index -> next_field_index
Ivars will longer be the only thing stored inline
via shapes, so keeping the `iv_index` and `ivptr` names
would be confusing.

Instance variables won't be the only thing stored inline
via shapes, so keeping the `ivptr` name would be confusing.

`field` encompass anything that can be stored in a VALUE array.

Similarly, `gen_ivtbl` becomes `gen_fields_tbl`.
2025-05-08 07:58:05 +02:00
Peter Zhu
3e94b5f9c0 Remove dependence on internal/hash.h for default GC 2025-05-07 11:47:13 -04:00
Étienne Barrié
cb772247e7 Improve correctness contention for allocated object counts
Currently the count of allocated object for a heap is incremented
without regards to parallelism which leads to incorrect counts.

By maintaining a local counter in the ractor newobj cache, and only
syncing atomically with some granularity, we can improve the correctness
without increasing contention.

The allocated object count is also synced when the ractor is freed.

Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-05-06 19:13:59 +02:00
Nobuyoshi Nakada
c772d2691d
Count metadata entries automatically from the names list 2025-04-25 19:40:04 +09:00
Jean Boussier
62a7f17157 Update RB_GC_OBJECT_METADATA_ENTRY_COUNT
In cb1ea54bbf I added one more
metadata flag, but didn't notice `RB_GC_OBJECT_METADATA_ENTRY_COUNT`
had to be incremented.

This should fix ASAN builds.

Interestingly, bdb25959fb already
caused the count to be off by one, so I had to increment it by
2.
2025-04-25 11:17:31 +02:00
Jean Boussier
cb1ea54bbf objspace_dump: Include shareable flag
Given that the currently planned ractor local GC implementation
performance will heavilly be influenced by the number of shareable
objects it would be valuable to be able to know how many of them
are in the heap.
2025-04-24 10:14:29 +02:00
Jean Boussier
1048b162de Eagerly store a copy of object_id in finalizer table.
This makes the finalizer table fully self contained, so GC no
longer need to delay cleaning the `obj_to_id_tbl`.
2025-04-23 11:54:29 +02:00
Jean Boussier
6d0dd7d863 rb_gc_impl_define_finalizer: unlock on early return 2025-04-23 07:04:41 +02:00
Jean Boussier
7c30bd50df Add missing lock in rb_gc_impl_define_finalizer
`objspace->finalizer_table` must be synchronized,
otherwise concurrent insertion from multiple ractors
will cause a crash.

Repro:

```ruby
ractors = 5.times.map do |i|
  Ractor.new do
    100_000.times.map do
      o = Object.new
      ObjectSpace.define_finalizer(o, ->(id) {})
      o
    end
  end
end

ractors.each(&:take)
```
2025-04-22 23:23:35 +02:00
dependabot[bot]
f6fddbd337 Bump crossbeam-channel from 0.5.13 to 0.5.15 in /gc/mmtk
Bumps [crossbeam-channel](https://github.com/crossbeam-rs/crossbeam) from 0.5.13 to 0.5.15.
- [Release notes](https://github.com/crossbeam-rs/crossbeam/releases)
- [Changelog](https://github.com/crossbeam-rs/crossbeam/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crossbeam-rs/crossbeam/compare/crossbeam-channel-0.5.13...crossbeam-channel-0.5.15)

---
updated-dependencies:
- dependency-name: crossbeam-channel
  dependency-version: 0.5.15
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-15 17:14:26 +09:00
Jean Boussier
0606046c1a Lazily create objspace->id_to_obj_tbl
This inverse table is only useful if `ObjectSpace._id2ref` is used,
which is extremely rare. The only notable exception is the `drb` gem
and even then it has an option not to rely on `_id2ref`.

So if we assume this table will never be looked up, we can just
not maintain it, and if it turns out `_id2ref` is called, we
can lock the VM and re-build it.

```
compare-ruby: ruby 3.5.0dev (2025-04-10T09:44:40Z master 684cfa42d7) +YJIT +PRISM [arm64-darwin24]
built-ruby: ruby 3.5.0dev (2025-04-10T10:13:43Z lazy-id-to-obj d3aa9626cc) +YJIT +PRISM [arm64-darwin24]
warming up..

|           |compare-ruby|built-ruby|
|:----------|-----------:|---------:|
|baseline   |     26.364M|   25.974M|
|           |       1.01x|         -|
|object_id  |     10.293M|   14.202M|
|           |           -|     1.38x|
```
2025-04-15 07:57:39 +09:00
Kunshan Wang
d17ab5a430 [ruby/mmtk] Do root scanning in scan_vm_specific_roots
We rely on scan_vm_specific_roots to reach all stacks via the following
path:

    VM -> ractors -> threads -> fibers -> stacks

0a6a835aaa
2025-04-08 03:12:50 +00:00
Peter Zhu
d4406f0627 Grow GC heaps independently
[Bug #21214]

If we allocate objects where one heap holds transient objects and another
holds long lived objects, then the heap with transient objects will grow
along the heap with long lived objects, causing higher memory usage.

For example, we can see this issue in this script:

    def allocate_small_object = []
    def allocate_large_object = Array.new(10)

    arys = Array.new(1_000_000) do
      # Allocate 10 small transient objects
      10.times { allocate_small_object }
      # Allocate 1 large object that is persistent
      allocate_large_object
    end

    pp GC.stat
    pp GC.stat_heap

Before this change:

    heap_live_slots: 2837243
    {0 =>
      {slot_size: 40,
       heap_eden_pages: 1123,
       heap_eden_slots: 1838807},
     2 =>
      {slot_size: 160,
       heap_eden_pages: 2449,
       heap_eden_slots: 1001149},
    }

After this change:

    heap_live_slots: 1094474
    {0 =>
      {slot_size: 40,
       heap_eden_pages: 58,
       heap_eden_slots: 94973},
     2 =>
      {slot_size: 160,
       heap_eden_pages: 2449,
       heap_eden_slots: 1001149},
    }
2025-04-07 09:41:11 -04:00
Matt Valentine-House
e4020133c8 Remove incorrect assertion
ractor_cache will always be NULL in this context
2025-04-01 21:45:08 +01:00
Jean Boussier
7db0e07134 Don't preserve object_id when moving object to another Ractor
That seemed like the logical thing to do to me, but ko1 disagree.
2025-03-31 12:01:55 +02:00
Jean Boussier
0350290262 Ractor: Fix moving embedded objects
[Bug #20271]
[Bug #20267]
[Bug #20255]

`rb_obj_alloc(RBASIC_CLASS(obj))` will always allocate from the basic
40B pool, so if `obj` is larger than `40B`, we'll create a corrupted
object when we later copy the shape_id.

Instead we can use the same logic than ractor copy, which is
to use `rb_obj_clone`, and later ask the GC to free the original
object.

We then must turn it into a `T_OBJECT`, because otherwise
just changing its class to `RactorMoved` leaves a lot of
ways to keep using the object, e.g.:

```
a = [1, 2, 3]
Ractor.new{}.send(a, move: true)
[].concat(a) # Should raise, but wasn't.
```

If it turns out that `rb_obj_clone` isn't performant enough
for some uses, we can always have carefully crafted specialized
paths for the types that would benefit from it.
2025-03-31 12:01:55 +02:00
Alan Wu
c576e83a24
Prefer FL_TEST_RAW() in GC on known on-heap objects
Was reading some assembly and noticed the dead branches generated for
FL_TEST(). Just a quick basic pass to change the obvious places; there
may be other opportunities.
2025-03-25 18:16:31 -04:00
Peter Zhu
6bb35a1de4 Make ruby_autocompact_compare_func static
It's not used outside of default.c.
2025-03-25 16:19:32 -04:00
Peter Zhu
e4c7eb1152 Make ruby_enable_autocompact static
It's not used outside of defaut.c
2025-03-25 16:19:32 -04:00
Peter Zhu
eb3f73ae9c [DOC] Use install-modular-gc in gc/README.md 2025-03-25 08:49:31 -04:00
Peter Zhu
319fcca656 Move rb_gc_impl_ractor_cache_free to shutdown section 2025-03-24 08:49:30 -04:00
Peter Zhu
a572ec1ba0 Move rb_gc_impl_objspace_free to shutdown section 2025-03-24 08:49:30 -04:00
Peter Zhu
31cf2684b0 Output object_id in object metadata for MMTk 2025-03-13 10:12:24 -04:00
Peter Zhu
bdb25959fb Move object_id to flags for ObjectSpace dumps
Moving object_id dumping from ObjectSpace to the GC flags allows ObjectSpace
to not assume the FL_SEEN_OBJ_ID flag and instead move it to the responsibility
of the GC.
2025-03-13 10:12:24 -04:00
Kunshan Wang
aa7b5e2df4 [ruby/mmtk] Trigger forced GC in GC.start
We now use `MMTK::handle_user_collection_request(true, ...)` to force
triggering a GC instead of enabling GC temporarily.

02ef47f818
2025-02-24 23:30:00 +00:00
Peter Zhu
0f1cb92345 [ruby/mmtk] Fix compatibility for Rust 1.85
9da566e26a
2025-02-20 19:35:38 +00:00
Peter Zhu
49e229b3fc Fix value of RB_GC_OBJECT_METADATA_ENTRY_COUNT
There are 7 entries in RB_GC_OBJECT_METADATA_ENTRY_COUNT.
2025-02-19 09:56:17 -05:00
Peter Zhu
5e45f2a0bc Add age to rb_gc_object_metadata
This will allow ObjectSpace.dump to output the age of the object.
2025-02-19 09:47:28 -05:00
Peter Zhu
5acfe30880 Implement rb_gc_object_metadata for MMTk 2025-02-19 09:47:28 -05:00