Commit graph

18546 commits

Author SHA1 Message Date
Kevin Newton
f38814564b [ruby/prism] Fix eval parsing depth
89bf7a4948
2023-12-15 15:19:50 +00:00
Kevin Newton
fe9b42f024 [ruby/prism] Invalid pinned locals in pattern matching
3a67b37a56
2023-12-15 15:03:49 +00:00
TSUYUSATO Kitsune
16830a4783 [ruby/prism] Add an error for in keyword in arguments
Fix https://github.com/ruby/prism/pull/2026

c4b41cd477
2023-12-15 13:25:54 +00:00
Satoshi Tagomori
04f7be6126 loading/testing in different processes for multiple runs 2023-12-15 21:36:27 +09:00
Koichi Sasada
406d4bb599 add a test
proposed at https://bugs.ruby-lang.org/issues/20050#note-5
2023-12-15 11:58:43 +09:00
Jeremy Evans
29e99c84ae Remove unused variables in test_call_op_asgn_keywords_mutable 2023-12-14 18:44:13 -08:00
Jemma Issroff
5587bd4b37 [PRISM] Implement safe navigation in CallNodes
This commit implements safe navigation for CallNodes,
CallAndWriteNodes and CallOperatorWriteNodes
2023-12-14 17:11:54 -05:00
Ufuk Kayserilioglu
d07fdc5ede [PRISM] Fix keyword hash handling in method calls
* Start using the renamed `PM_KEYWORD_HASH_NODE_FLAGS_SYMBOL_KEYS` flag
to check if all keys of the keyword hash node are symbols.
* For arguments passed as a hash, start marking them as `KW_SPLAT_MUT` only if the number of entries in the hash is greater than 1 (which is what the old compiler used to do).
2023-12-14 14:08:37 -05:00
Kevin Newton
295d97ab4d Pattern matching 2023-12-14 14:06:26 -05:00
Kevin Newton
019fff3a86 [ruby/prism] Fix parse result for nesting pattern matching
ee6fc9ee87
2023-12-14 18:54:46 +00:00
Jemma Issroff
7ac93e99a3 [PRISM] Account for multiple anonymous locals
This commit adjusts the local table size to be consistent regardless
of the number of anonymous locals.
2023-12-14 13:45:41 -05:00
Ufuk Kayserilioglu
01f21d5729 [ruby/prism] Fix the implementation of the flag on keyword hash nodes
The previous implementation was incorrect since it was just checking for all keys in assoc nodes to be static literals but the actual check is that all keys in assoc nodes must be symbol nodes.

This commit fixes that implementation, and, also, aliases the flag to `PM_KEYWORD_HASH_NODE_FLAGS_SYMBOL_KEYS` so that ruby/ruby can start using the new flag name.

I intend to later change the real flag name to `PM_KEYWORD_HASH_NODE_FLAGS_SYMBOL_KEYS` and remove the alias.

f5099c79ce
2023-12-14 18:05:54 +00:00
Jemma Issroff
e71f011713 [PRISM] Fix bugs in compiling optional keyword parameters
This PR fixes two bugs when compiling optional keyword parameters:
- It moves keyword parameter compilation to STEP 5 in the parameters
sequence, where the rest of compilation happens. This is important
because keyword parameter compilation relies on the value of
body->param.keyword->bits_start which gets set in an earlier step
- It compiles array and hash values for keyword parameters, which
it didn't previously
2023-12-14 11:46:54 -05:00
TSUYUSATO Kitsune
3658798dbb [ruby/prism] Make equality operators non-associative
Fix https://github.com/ruby/prism/pull/2073

0f747d9240
2023-12-14 16:39:05 +00:00
Jemma Issroff
8e1c148fd9 [PRISM] Use frozen flag on StringNode 2023-12-14 11:14:45 -05:00
Jeremy Evans
a18819e65f Fix op asgn method calls passing mutable keyword splats
When passing the keyword splat to [], it cannot be mutable, because
mutating the keyword splat inside [] would result in changes to the
keyword splat passed to []=.
2023-12-14 08:13:43 -08:00
Matt Valentine-House
a10c11a66b [PRISM] Add anon KW args to the block local table 2023-12-14 07:01:19 -05:00
Satoshi Tagomori
e51f9e9f75 rb_ext_resolve_symbol: C API to resolve and return externed symbols [Feature #20005]
This is a C API for extensions to resolve and get function symbols of other extensions.
Extensions can check the expected symbol is correctly loaded and accessible, and
use it if it is available.
Otherwise, extensions can raise their own error to guide users to setup their
environments correctly and what's missing.
2023-12-14 17:39:42 +09:00
Kevin Newton
b7e89d4b17 [ruby/prism] Fix hash pattern rest
43c4232cfc
2023-12-14 02:43:32 +00:00
Drew Stevenson
beefce1444 [rubygems/rubygems] Warn for duplicate meta data links
Match order of METADATA_LINK_KEYS to order used by rubygems.org in Links model.
Add missing download_uri key.

d2922cd6e9
2023-12-14 00:06:05 +00:00
Samuel Giddins
baf2ec2ca8 [rubygems/rubygems] Use match? when regexp match data is unused
Improved performance / reduced allocations

b04726c9a7
2023-12-13 22:00:26 +00:00
Ufuk Kayserilioglu
31c0ea20e5 [PRISM] Add a test with a non-static-literal hash key 2023-12-13 14:15:09 -05:00
eileencodes
110dbf62ac [Prism] Fix InterpolatedMatchLastLine Instructions
I looked at this at RubyConf with Kevin, and we noticed that there was a
`putobject` empty string missing from the instructions. I just got back
around to implementing this and pushing a PR and while doing that
noticed that we also have a `getspecial` when we want a `getglobal`.

This change adds the `putobject` instruction and replaces the
`getspecial` with a `getglobal`. If we look at the parsetree for the
following code:

```ruby
$pit = '.oo'; if /"#{$pit}"/mix; end
```

We can see it has a `NODE_GVAR` and the [Ruby
compiler](a4b43e9264/compile.c (L10024-L10030)) shows that should
use `getglobal.

```
 @ NODE_SCOPE (id: 14, line: 1, location: (1,0)-(22,36))
 +- nd_tbl: (empty)
 +- nd_args:
 |   (null node)
 +- nd_body:
     @ NODE_BLOCK (id: 12, line: 22, location: (22,0)-(22,36))
     +- nd_head (1):
     |   @ NODE_GASGN (id: 0, line: 22, location: (22,0)-(22,12))*
     |   +- nd_vid: :$pit
     |   +- nd_value:
     |       @ NODE_STR (id: 1, line: 22, location: (22,7)-(22,12))
     |       +- nd_lit: ".oo"
     +- nd_head (2):
         @ NODE_IF (id: 11, line: 22, location: (22,14)-(22,36))*
         +- nd_cond:
         |   @ NODE_MATCH2 (id: 10, line: 22, location: (22,14)-(22,36))
         |   +- nd_recv:
         |   |   @ NODE_DREGX (id: 2, line: 22, location: (22,17)-(22,31))
         |   |   +- nd_lit: "\""
         |   |   +- nd_next->nd_head:
         |   |   |   @ NODE_EVSTR (id: 4, line: 22, location: (22,19)-(22,26))
         |   |   |   +- nd_body:
         |   |   |       @ NODE_GVAR (id: 3, line: 22, location: (22,21)-(22,25))
         |   |   |       +- nd_vid: :$pit
         |   |   +- nd_next->nd_next:
         |   |       @ NODE_LIST (id: 7, line: 22, location: (22,26)-(22,27))
         |   |       +- as.nd_alen: 1
         |   |       +- nd_head:
         |   |       |   @ NODE_STR (id: 6, line: 22, location: (22,26)-(22,27))
         |   |       |   +- nd_lit: "\""
         |   |       +- nd_next:
         |   |           (null node)
         |   +- nd_value:
         |       @ NODE_GVAR (id: 9, line: 22, location: (22,14)-(22,36))
         |       +- nd_vid: :$_
         +- nd_body:
         |   @ NODE_BEGIN (id: 8, line: 22, location: (22,32)-(22,32))
         |   +- nd_body:
         |       (null node)
         +- nd_else:
             (null node)
```

I'm struggling with writing a failing test, but the before/after
instructions show that `getglobal` is correct here. I compared the
instructions for the  other `InterpolatedMatchLastLine` node tests
and they also used `getglobal`. I've edited the existing
`InterpolatedLastMatchLineNode` test so that it will use that instruction when
copied out of the test. Without the quotes it thinks it's just a
`MatchLastLineNode`.

Incorrect instructions:

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:1 (1,0)-(22,36)>
0000 putstring                              ".oo"                     (  22)[Li]
0002 setglobal                              :$pit
0004 putobject                              "\""
0006 getglobal                              :$pit
0008 dup
0009 objtostring                            <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE>
0011 anytostring
0012 putobject                              "\""
0014 toregexp                               7, 3
0017 getglobal                              :$_
0019 send                                   <calldata!mid:=~, argc:1, ARGS_SIMPLE>, nil
0022 branchunless                           30
0024 jump                                   26
0026 putnil
0027 jump                                   31
0029 pop
0030 putnil
0031 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:21 (21,0)-(21,36)>
0000 putstring                              ".oo"                     (  21)[Li]
0002 setglobal                              :$pit
0004 putobject                              "\""
0006 getglobal                              :$pit
0008 dup
0009 objtostring                            <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE>
0011 anytostring
0012 putobject                              "\""
0014 toregexp                               7, 3
0017 getspecial                             0, 0
0020 send                                   <calldata!mid:=~, argc:1, ARGS_SIMPLE>, nil
0023 branchunless                           31
0025 jump                                   27
0027 putnil
0028 jump                                   32
0030 pop
0031 putnil
0032 leave
```

Fixed instructions:

```
== disasm: #<ISeq:<compiled>@<compiled>:1 (1,0)-(22,36)>
0000 putstring                              ".oo"                     (  22)[Li]
0002 setglobal                              :$pit
0004 putobject                              "\""
0006 getglobal                              :$pit
0008 dup
0009 objtostring                            <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE>
0011 anytostring
0012 putobject                              "\""
0014 toregexp                               7, 3
0017 getglobal                              :$_
0019 send                                   <calldata!mid:=~, argc:1, ARGS_SIMPLE>, nil
0022 branchunless                           30
0024 jump                                   26
0026 putnil
0027 jump                                   31
0029 pop
0030 putnil
0031 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:21 (21,0)-(21,36)>
0000 putstring                              ".oo"                     (  21)[Li]
0002 setglobal                              :$pit
0004 putobject                              "\""
0006 getglobal                              :$pit
0008 dup
0009 objtostring                            <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE>
0011 anytostring
0012 putobject                              "\""
0014 toregexp                               7, 3
0017 getglobal                              :$_
0019 send                                   <calldata!mid:=~, argc:1, ARGS_SIMPLE>, nil
0022 branchunless                           30
0024 jump                                   26
0026 putnil
0027 jump                                   31
0029 pop
0030 putnil
0031 leave
```
2023-12-13 13:42:17 -05:00
Alan Wu
a3b48ac9ad Fix memory leak in Hash#compare_by_identity
We didn't free the old ST before overwriting it which caused a leak.
Found with RUBY_FREE_ON_EXIT.

Co-authored-by: Peter Zhu <peter@peterzhu.ca>
2023-12-13 09:43:09 -08:00
Takashi Kokubun
cc86fa8416 Skip an unstable test on MinGW
This test fails fairly frequently on MinGW:
1959892425 (step):11:168
1958562718 (step):11:168

and we aren't actively working on stabilizing tests for MinGW.
2023-12-13 09:40:54 -08:00
tomoya ishida
c83a648fc8 [ruby/irb] Remove unused lvar in mesure command test
(https://github.com/ruby/irb/pull/814)

320178b120
2023-12-13 17:05:35 +00:00
Jeremy Evans
0d53dba7ce Make String#chomp! raise ArgumentError for 2+ arguments if string is empty
String#chomp! returned nil without checking the number of passed
arguments in this case.
2023-12-13 07:05:21 -08:00
Matt Valentine-House
9267dbdd7a [PRISM] Generate instruction for when redo_label is set 2023-12-13 09:33:07 -05:00
Jemma Issroff
798a89fae1 [PRISM] If receiver on CallNode is SelfNode, use FCALL flags 2023-12-13 08:11:45 -05:00
eileencodes
1ad991c54d [PRISM] Fix attrset edge cases
In some cases code may look like an attrset ID but should actually
return the value of the method, not the passed values.

In ruby/prism#2051 a flag was added when we have a attribute write call.
I used this flag to add the proper instructions when we have a real
attrset instead of using `rb_is_attrset_id` which was kind of hacky
anyway.

The value that should be returned in the newly added test is 42, not 2.
Previously the changes we had made returned 2.

Related to ruby/prism#1715
2023-12-13 08:11:21 -05:00
tomoya ishida
745ab3e4c7 [ruby/irb] Warn and do nothing if block is passed to measure command
(https://github.com/ruby/irb/pull/813)

e79a90a1e6
2023-12-13 11:06:26 +00:00
Takashi Kokubun
20a09387d1 Skip a GC test for RJIT
It randomly fails like this:
1958616497
2023-12-12 23:24:19 -08:00
Vít Ondruch
1fa5dd883e [rubygems/rubygems] Test if the user dir is used for auto user installation
This is mainly to align this test case with the
`test_process_options_does_not_fallback_to_user_install_when_gem_home_
not_writable_and_no_user_install`, where the `install_dir` is checked
already.

02b1884b61
2023-12-13 12:16:55 +09:00
Vít Ondruch
402fd96ddc [rubygems/rubygems] Make sure --no-user-install is respected for auto user installation
The `options[:user_install]` might have three states:
* `true`: `--user-install`
* `false`: `--no-user-install` and
* `nil`: option was not specified

However, this had not been respected previously and the `false` and `nil`
were treated the same. This could lead to auto user installation despite
`--no-user-install` being specified on the command line.

Fixes https://github.com/rubygems/rubygems/pull/7237

9281545474
2023-12-13 12:16:55 +09:00
David Rodríguez
a7c9163b5d [rubygems/rubygems] Vendor timeout in RubyGems too
e2e7440ede
2023-12-13 12:16:55 +09:00
David Rodríguez
ce924ce1fb [rubygems/rubygems] Vendor net-http and net-protocol in RubyGems
99d91c9ed2
2023-12-13 12:16:55 +09:00
Jemma Issroff
196c24620e [PRISM] Correctly parse non-base 10 integers in Prism
This commit passes an `end` to rb_int_parse_cstr which allows us
to correctly parse non-base 10 integers which are enclosed in
parenthesis. Prior to this commit, we were getting a putobject nil
when compiling `(0o0)` for example.
2023-12-12 22:00:50 -05:00
KJ Tsanaktsidis
a8d2d93aff Add a test case for preregistering with different data
We want to make sure that if preregister is called with different data,
that the postponed job table is updated.
2023-12-13 13:35:05 +11:00
eileencodes
a6526342f3 [PRISM] Fix segv with regex once flag
When you have an interpolated regex with a `once` flag and local
variable is outside the block created by the `once` flag, Prism would
see a segv. This is because it was not taking the depth into account.

To fix this, we need to add 1 to the `local_depth_offset` on the
`scope`.

Fixes: ruby/prism#2047
2023-12-12 15:15:02 -05:00
Matt Valentine-House
43ede3f26c [PRISM] Compile NextNode arguments 2023-12-12 13:46:36 -05:00
Jemma Issroff
b9dfe04a73 [PRISM] Implementing forwarding of args for ForwardingSuperNode
ForwardingSuperNodes need to actually forward any applicable
arguments. This commit implements that logic, by using the data
stored on the local iseq about the parameters to forward the
appropriate arguments.
2023-12-12 13:45:21 -05:00
Kouhei Yanagita
06e2fbb826 [Bug #19114] Fix for multiple calls of TracePoint#enable 2023-12-12 09:36:37 -08:00
Kevin Newton
278ce27ee4 [ruby/prism] Flag for attribute write on calls
465731969c
2023-12-12 15:55:54 +00:00
Jeremy Evans
2f1d6da8c4
Fix op asgn calls with keywords
Examples of such calls:

```ruby
obj[kw: 1] += fo
obj[**kw] &&= bar
```

Before this patch, literal keywords would segfault in the compiler,
and keyword splat usage would result in TypeError.

This handles all cases I can think of:

* literal keywords
* keyword splats
* combined with positional arguments
* combined with regular splats
* both with and without blocks
* both popped and non-popped cases

This also makes sure that to_hash is only called once on the keyword
splat argument, instead of twice, and make sure it is called before
calling to_proc on a passed block.

Fixes [Bug #20051]

Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
2023-12-12 07:34:53 -08:00
Peter Zhu
185b7e92a3 Make WeakKeyMap safe for compaction during allocation
During allocation, the table may not have been allocated yet which would
crash in the st_foreach.
2023-12-12 09:01:21 -05:00
Peter Zhu
33cf8f640b Make WeakMap safe for compaction during allocation
During allocation, the table may not have been allocated yet which would
crash in the st_foreach.
2023-12-12 09:01:21 -05:00
Ufuk Kayserilioglu
d313c82f79 [ruby/prism] Update tests and snapshots
0663e2bcfa
2023-12-12 13:05:09 +00:00
Jeremy Evans
f49af3c969 Make Dir#chdir never yield args, and return block return value
If no block is given, return 0 instead of nil for consistency
with Dir.chdir and Dir.fchdir.
2023-12-11 22:31:16 -08:00
Nobuyoshi Nakada
305f66b859
Save $VERBOSE properly 2023-12-12 11:06:27 +09:00
Yusuke Endoh
73b43fffa1 Prevent a warning: assigned but unused variable - status 2023-12-12 10:31:37 +09:00