archive/ruby
1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2025-08-23 04:55:21 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
91898 commits 30 branches 1174 tags 508 MiB
Commit graph

604 commits

Author SHA1 Message Date
Peter Zhu
ff51dc5654 [Feature #20265] Remove rb_newobj_of and RB_NEWOBJ_OF 2024-03-14 12:53:04 -04:00
Jean Boussier
315bde5a0f Exception#set_backtrace accept arrays of Backtrace::Location 
[Feature #13557]

Setting the backtrace with an array of strings is lossy. The resulting
exception will return nil on `#backtrace_locations`.

By accepting an array of `Backtrace::Location` instance, we can rebuild
a `Backtrace` instance and have a fully functioning Exception.

Co-Authored-By: Étienne Barrié <etienne.barrie@gmail.com>
 2024-03-14 11:38:40 +01:00
Peter Zhu
6ad347a105 Don't directly read the SIZE_POOL_COUNT in shapes 
This removes the assumption about SIZE_POOL_COUNT for shapes.
 2024-03-13 09:55:52 -04:00
Peter Zhu
2fc551e34e Simplify NEWOBJ_OF macro 2024-03-13 09:05:52 -04:00
Peter Zhu
2c349cf4b6 Use NEWOBJ_OF_ec in NEWOBJ_OF_0 2024-03-11 19:30:09 -04:00
Jean Boussier
2d80b6093f Retire RUBY_MARK_UNLESS_NULL 
Marking `Qnil` or `Qfalse` works fine, having
an extra macro to avoid it isn't needed.
 2024-03-08 14:13:14 +01:00
Jean Boussier
d4f3dcf4df Refactor VM root modules 
This `st_table` is used to both mark and pin classes
defined from the C API. But `vm->mark_object_ary` already
does both much more efficiently.

Currently a Ruby process starts with 252 rooted classes,
which uses `7224B` in an `st_table` or `2016B` in an `RArray`.

So a baseline of 5kB saved, but since `mark_object_ary` is
preallocated with `1024` slots but only use `405` of them,
it's a net `7kB` save.

`vm->mark_object_ary` is also being refactored.

Prior to this changes, `mark_object_ary` was a regular `RArray`, but
since this allows for references to be moved, it was marked a second
time from `rb_vm_mark()` to pin these objects.

This has the detrimental effect of marking these references on every
minors even though it's a mostly append only list.

But using a custom TypedData we can save from having to mark
all the references on minor GC runs.

Addtionally, immediate values are now ignored and not appended
to `vm->mark_object_ary` as it's just wasted space.
 2024-03-06 15:33:43 -05:00
Jean Boussier
b4a69351ec Move FL_SINGLETON to FL_USER1 
This frees FL_USER0 on both T_MODULE and T_CLASS.

Note: prior to this, FL_SINGLETON was never set on T_MODULE,
so checking for `FL_SINGLETON` without first checking that
`FL_TYPE` was `T_CLASS` was valid. That's no longer the case.
 2024-03-06 13:11:41 -05:00
Jean Boussier
e626da82ea Don't pin named structs defined in Ruby 
[Bug #20311]

`rb_define_class_under` assumes it's called from C and that the
reference might be held in a C global variable, so it adds the
class to the VM root.

In the case of `Struct.new('Name')` it's wasteful and make
the struct immortal.
 2024-03-01 08:23:38 +01:00
Peter Zhu
5481dbef07 Make rb_define_finalizer_no_check private 2024-02-28 13:45:19 -05:00
Peter Zhu
dcc976add9 Remove unused rb_gc_id2ref_obj_tbl 2024-02-28 12:21:38 -05:00
Peter Zhu
78ae6dbb11 Remove rb_objspace_marked_object_p 
rb_objspace_marked_object_p is no longer used in the objspace module, so
we can remove it.
 2024-02-26 17:05:34 -05:00
Peter Zhu
7538703d1b Make rb_objspace_data_type_memsize private 
rb_objspace_data_type_memsize is not used in the objspace module, so we
can make it private.
 2024-02-26 17:05:34 -05:00
Peter Zhu
c9b6cd4223 Remove unused rb_objspace_each_objects_without_setup 2024-02-26 14:34:24 -05:00
Peter Zhu
e65315a725 Extract imemo functions from gc.c into imemo.c 2024-02-22 11:35:09 -05:00
Peter Zhu
330830dd1a Add IMEMO_NEW 
Rather than exposing that an imemo has a flag and four fields, this
changes the implementation to only expose one field (the klass) and
fills the rest with 0. The type will have to fill in the values themselves.
 2024-02-21 11:33:05 -05:00
John Hawthorn
1c97abaaba De-dup identical callinfo objects 
Previously every call to vm_ci_new (when the CI was not packable) would
result in a different callinfo being returned this meant that every
kwarg callsite had its own CI.

When calling, different CIs result in different CCs. These CIs and CCs
both end up persisted on the T_CLASS inside cc_tbl. So in an eval loop
this resulted in a memory leak of both types of object. This also likely
resulted in extra memory used, and extra time searching, in non-eval
cases.

For simplicity in this commit I always allocate a CI object inside
rb_vm_ci_lookup, but ideally we would lazily allocate it only when
needed. I hope to do that as a follow up in the future.
 2024-02-20 18:55:00 -08:00
yui-knk
e7ab5d891c Introduce NODE_REGX to manage regexp literal 2024-02-21 08:06:48 +09:00
yui-knk
89cfc15207 [Feature #20257] Rearchitect Ripper 
Introduce another semantic value stack for Ripper so that
Ripper can manage both Node and Ruby Object separately.
This rearchitectutre of Ripper solves these issues.
Therefore adding test cases for them.

* [Bug 10436] https://bugs.ruby-lang.org/issues/10436
* [Bug 18988] https://bugs.ruby-lang.org/issues/18988
* [Bug 20055] https://bugs.ruby-lang.org/issues/20055

Checked the differences of `Ripper.sexp` for files under `/test/ruby`
are only on test_pattern_matching.rb.
The differences comes from the differences between
`new_hash_pattern_tail` functions between parser and Ripper.
Ripper `new_hash_pattern_tail` didn’t call `assignable` then
`kw_rest_arg` wasn’t marked as local variable.
This is also fixed by this commit.

```
--- a/./tmp/before/test_pattern_matching.rb
+++ b/./tmp/after/test_pattern_matching.rb
@@ -3607,7 +3607,7 @@
                  [:in,
                   [:hshptn, nil, [], [:var_field, [:@ident, “a”, [984, 13]]]],
                   [[:binary,
-                    [:vcall, [:@ident, “a”, [985, 10]]],
+                    [:var_ref, [:@ident, “a”, [985, 10]]],
                     :==,
                     [:hash, nil]]],
                   nil]]],
@@ -3662,7 +3662,7 @@
                  [:in,
                   [:hshptn, nil, [], [:var_field, [:@ident, “a”, [993, 13]]]],
                   [[:binary,
-                    [:vcall, [:@ident, “a”, [994, 10]]],
+                    [:var_ref, [:@ident, “a”, [994, 10]]],
                     :==,
                     [:hash,
                      [:assoclist_from_args,
@@ -3813,7 +3813,7 @@
                    [:command,
                     [:@ident, “raise”, [1022, 10]],
                     [:args_add_block,
-                     [[:vcall, [:@ident, “b”, [1022, 16]]]],
+                     [[:var_ref, [:@ident, “b”, [1022, 16]]]],
                      false]]],
                   [:else, [[:var_ref, [:@kw, “true”, [1024, 10]]]]]]]],
                nil,
@@ -3876,7 +3876,7 @@
                      [:@int, “0”, [1033, 15]]],
                     :“&&“,
                     [:binary,
-                     [:vcall, [:@ident, “b”, [1033, 20]]],
+                     [:var_ref, [:@ident, “b”, [1033, 20]]],
                      :==,
                      [:hash, nil]]]],
                   nil]]],
@@ -3946,7 +3946,7 @@
                      [:@int, “0”, [1042, 15]]],
                     :“&&“,
                     [:binary,
-                     [:vcall, [:@ident, “b”, [1042, 20]]],
+                     [:var_ref, [:@ident, “b”, [1042, 20]]],
                      :==,
                      [:hash,
                       [:assoclist_from_args,
@@ -5206,7 +5206,7 @@
                      [[:assoc_new,
                        [:@label, “c:“, [1352, 22]],
                        [:@int, “0”, [1352, 25]]]]]],
-                   [:vcall, [:@ident, “r”, [1352, 29]]]],
+                   [:var_ref, [:@ident, “r”, [1352, 29]]]],
                   false]]],
                [:binary,
                 [:call,
@@ -5299,7 +5299,7 @@
                       [:assoc_new,
                        [:@label, “c:“, [1367, 34]],
                        [:@int, “0”, [1367, 37]]]]]],
-                   [:vcall, [:@ident, “r”, [1367, 41]]]],
+                   [:var_ref, [:@ident, “r”, [1367, 41]]]],
                   false]]],
                [:binary,
                 [:call,
@@ -5931,7 +5931,7 @@
              [:in,
               [:hshptn, nil, [], [:var_field, [:@ident, “r”, [1533, 11]]]],
               [[:binary,
-                [:vcall, [:@ident, “r”, [1534, 8]]],
+                [:var_ref, [:@ident, “r”, [1534, 8]]],
                 :==,
                 [:hash,
                  [:assoclist_from_args,
```
 2024-02-20 17:33:58 +09:00
Nobuyoshi Nakada
b1d70e4264
[Bug #20280] Check by rb_parser_enc_str_coderange 
Co-authored-by: Yuichiro Kaneko <spiketeika@gmail.com>
 2024-02-19 16:33:26 +09:00
Nobuyoshi Nakada
fcc55dc226
[Bug #20280] Raise SyntaxError on invalid encoding symbol 2024-02-19 16:33:26 +09:00
Peter Zhu
1d3b306753 Move rb_class_allocate_instance from gc.c to object.c 2024-02-14 13:43:02 -05:00
Aaron Patterson
c35fea8509
Specialize String#byteslice(a, b) (#9939) 
* Specialize String#byteslice(a, b)

This adds a specialization for String#byteslice when there are two
parameters.

This makes our protobuf parser go from 5.84x slower to 5.33x slower

```
Comparison:
decode upstream (53738 bytes):     7228.5 i/s
decode protobuff (53738 bytes):     1236.8 i/s - 5.84x  slower

Comparison:
decode upstream (53738 bytes):     7024.8 i/s
decode protobuff (53738 bytes):     1318.5 i/s - 5.33x  slower
```

* Update yjit/src/codegen.rs

---------

Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
 2024-02-13 16:20:27 +00:00
Jean Boussier
de1a586ecc proc.c: get rid of CLONESETUP 
[Bug #20253]

All the way down to Ruby 1.9, `Proc`, `Method`, `UnboundMethod`
and `Binding` always had their own specific clone and dup routine.

This caused various discrepancies with how other objects behave
on `dup` and `clone. [Bug #20250], [Bug #20253].

This commit get rid of `CLONESETUP` and use the the same codepath
as all other types, so ensure consistency.

NB: It's still not accepting the `freeze` keyword argument on `clone`.

Co-Authored-By: Étienne Barrié <etienne.barrie@gmail.com>
 2024-02-12 18:31:48 +01:00
yui-knk
33c1e082d0 Remove ruby object from string nodes 
String nodes holds ruby string object on `VALUE nd_lit`.
This commit changes it to `struct rb_parser_string *string`
to reduce dependency on ruby object.
Sometimes these strings are concatenated with other string
therefore string concatenate functions are needed.
 2024-02-09 14:20:17 +09:00
Peter Zhu
5e0c171451 Make io_fwrite safe for compaction 
[Bug #20169]

Embedded strings are not safe for system calls without the GVL because
compaction can cause pages to be locked causing the operation to fail
with EFAULT. This commit changes io_fwrite to use rb_str_tmp_frozen_no_embed_acquire,
which guarantees that the return string is not embedded.
 2024-02-05 11:11:07 -05:00
Nobuyoshi Nakada
8ba8e979c8
Parenthesize casted argument 2024-02-01 16:42:09 +09:00
S.H
f3df218f48
Introduced rb_node_const_decl_val function 
Introduce `rb_node_const_decl_val` function to allow `rb_ary_join` and
`rb_ary_reverse` functions to be removed from Universal Parser.
 2024-01-31 13:31:38 +09:00
Nobuyoshi Nakada
361b3efe16
Use UNDEF_P 2024-01-30 14:48:59 +09:00
S.H
9b40f42c22
Introduce NODE_ENCODING 
`__ENCODING__ `was managed by `NODE_LIT` with Encoding object. 

Introduce `NODE_ENCODING` for
1. `__ENCODING__` is detectable from AST Node.
2. Reduce dependency Ruby object for parse.y
 2024-01-27 08:11:10 +00:00
Nobuyoshi Nakada
d86c4e553e
Define IO_WITHOUT_GVL macro 2024-01-24 20:51:50 +09:00
yui-knk
ee7f63ebba Make lastline and nextline to be rb_parser_string 
This commit changes `struct parser_params` lastline and nextline
from `VALUE` (String object) to `rb_parser_string_t *` so that
dependency on Ruby Object is reduced.
`parser_string_buffer_t string_buffer` is added to `struct parser_params`
to manage `rb_parser_string_t` pointers of each line. All allocated line
strings are freed in `rb_ruby_parser_free`.
 2024-01-23 08:58:16 +09:00
KJ Tsanaktsidis
61da90c1b8 Mark asan fake stacks during machine stack marking 
ASAN leaves a pointer to the fake frame on the stack; we can use the
__asan_addr_is_in_fake_stack API to work out the extent of the fake
stack and thus mark any VALUEs contained therein.

[Bug #20001]
 2024-01-19 09:55:12 +11:00
KJ Tsanaktsidis
3cfcb45ecf Define special macros for asan/msan being enabled 
__has_feature is a clang-ism, and GCC has a different way to tell if
sanitizers are enabled. For this reason, I don't want to spray
__has_feature all over the codebase for other places where conditional
compilation based on sanitizers is required.

[Bug #20001]
 2024-01-19 09:55:12 +11:00
KJ Tsanaktsidis
cabdaebc70 Make stack bounds detection work with ASAN 
Where a local variable is used as part of the stack bounds detection, it
has to actually be on the stack. ASAN can put local variable on "fake
stacks", however, with addresses in different memory mappings. This
completely destroys the stack bounds calculation, and can lead to e.g.
things not getting GC marked on the machine stack or stackoverflow
checks that always fail.

The __asan_addr_is_in_fake_stack helper can be used to get the _real_
stack address of such variables, and thus perform the stack size
calculation properly

[Bug #20001]
 2024-01-19 09:55:12 +11:00
Xavier Noria
aad246feba s/SafeStringValue/StringValue/ 
The macro SafeStringValue() became just StringValue() in c5c05460ac,
and it is deprecated nowadays.

This patch replaces remaining macro usage. Some occurrences are left in
ext/stringio and ext/win32ole, they should be fixed upstream.

The macro itself is not deleted, because it may be used in extensions.
 2024-01-12 12:24:48 -08:00
yui-knk
52d9e55903 Statically allocate parser config 2024-01-12 21:17:41 +09:00
KJ Tsanaktsidis
396e94666b Revert "Pass down "stack start" variables from closer to the top of the stack" 
This reverts commit 4ba8f0dc99.
 2024-01-12 17:58:54 +11:00
KJ Tsanaktsidis
6af0f442c7 Revert "Make stack bounds detection work with ASAN" 
This reverts commit 6185cfdf38.
 2024-01-12 17:58:54 +11:00
KJ Tsanaktsidis
33a03cb236 Revert "Define special macros for asan/msan being enabled" 
This reverts commit bdafad8790.
 2024-01-12 17:58:54 +11:00
KJ Tsanaktsidis
688a6ff510 Revert "Mark asan fake stacks during machine stack marking" 
This reverts commit d10bc3a2b8.
 2024-01-12 17:58:54 +11:00
KJ Tsanaktsidis
d10bc3a2b8 Mark asan fake stacks during machine stack marking 
ASAN leaves a pointer to the fake frame on the stack; we can use the
__asan_addr_is_in_fake_stack API to work out the extent of the fake
stack and thus mark any VALUEs contained therein.

[Bug #20001]
 2024-01-12 17:29:48 +11:00
KJ Tsanaktsidis
bdafad8790 Define special macros for asan/msan being enabled 
__has_feature is a clang-ism, and GCC has a different way to tell if
sanitizers are enabled. For this reason, I don't want to spray
__has_feature all over the codebase for other places where conditional
compilation based on sanitizers is required.

[Bug #20001]
 2024-01-12 17:29:48 +11:00
KJ Tsanaktsidis
6185cfdf38 Make stack bounds detection work with ASAN 
Where a local variable is used as part of the stack bounds detection, it
has to actually be on the stack. ASAN can put local variable on "fake
stacks", however, with addresses in different memory mappings. This
completely destroys the stack bounds calculation, and can lead to e.g.
things not getting GC marked on the machine stack or stackoverflow
checks that always fail.

The __asan_addr_is_in_fake_stack helper can be used to get the _real_
stack address of such variables, and thus perform the stack size
calculation properly

[Bug #20001]
 2024-01-12 17:29:48 +11:00
KJ Tsanaktsidis
4ba8f0dc99 Pass down "stack start" variables from closer to the top of the stack 
The implementation of `native_thread_init_stack` for the various
threading models can use the address of a local variable as part of the
calculation of the machine stack extents:

* pthreads uses it as a lower-bound on the start of the stack, because
  glibc (and maybe other libcs) can store its own data on the stack
  before calling into user code on thread creation.
* win32 uses it as an argument to VirtualQuery, which gets the extent of
  the memory mapping which contains the variable

However, the local being used for this is actually allocated _inside_
the `native_thread_init_stack` frame; that means the caller might
allocate a VALUE on the stack that actually lies outside the bounds
stored in machine.stack_{start,end}.

A local variable from one level above the topmost frame that stores
VALUEs on the stack must be drilled down into the call to
`native_thread_init_stack` to be used in the calculation. This probably
doesn't _really_ matter for the win32 case (they'll be in the same
memory mapping so VirtualQuery should return the same thing), but
definitely could matter for the pthreads case.

[Bug #20001]
 2024-01-12 17:29:48 +11:00
Peter Zhu
057df4379f Free environ when RUBY_FREE_AT_EXIT 
The environ is malloc'd, so it gets reported as a memory leak. This
commit adds ruby_free_proctitle which frees it during shutdown when
RUBY_FREE_AT_EXIT is set.

    STACK OF 1 INSTANCE OF 'ROOT LEAK: <calloc in ruby_init_setproctitle>':
    5   dyld                                  0x18b7090e0 start + 2360
    4   ruby                                  0x10000e3a8 main + 100  main.c:58
    3   ruby                                  0x1000b4dfc ruby_options + 180  eval.c:121
    2   ruby                                  0x1001c5f70 ruby_process_options + 200  ruby.c:3014
    1   ruby                                  0x10035c9fc ruby_init_setproctitle + 76  setproctitle.c:105
    0   libsystem_malloc.dylib                0x18b8c7b78 _malloc_zone_calloc_instrumented_or_legacy + 100
 2024-01-11 10:09:53 -05:00
KJ Tsanaktsidis
25f5b83689 Fix crash when printing RGENGC_DEBUG=5 output from GC 
I was trying to debug an (unrelated) issue in the GC, and wanted to turn
on the trace-level GC output by compiling it with -DRGENGC_DEBUG=5.
Unfortunately, this actually causes a crash in newobj_init() because the
code there tries to log the obj_info() of the newly created object.
However, the object is not actually sufficiently set up for some of the
things that obj_info() tries to do:

* The instance variable table for a class is not yet initialized, and
  when using variable-length RVALUES, said ivar table is embedded in
  as-yet unitialized memory after the struct RValue. Attempting to read
  this, as obj_info() does, causes a crash.
* T_DATA variables need to dereference their ->type field to print out
  the underlying C type name, which is not set up until newobj_fill() is
  called.

To fix this, create a new method `obj_info_basic`, which dumps out only
the parts of the object that are valid before the object is fully
initialized.

[Fixes #18795]
 2024-01-11 10:44:57 +11:00
yui-knk
db476cc71c Introduce NODE_SYM to manage symbol literal 
`:sym` was managed by `NODE_LIT` with `Symbol` object.
This commit introduces `NODE_SYM` so that

1. Symbol literal is detectable from AST Node
2. Reduce dependency on ruby object
 2024-01-09 16:07:19 +09:00
yui-knk
7ffff3e043 Change numeric node value functions argument to NODE * 
Change the argument to align with other node value functions
like `rb_node_line_lineno_val`.
 2024-01-08 14:02:48 +09:00
S-H-GAMELINKS
1b8d01136c Introduce Numeric Node's 2024-01-07 09:24:34 +09:00