Commit graph

625 commits

Author SHA1 Message Date
Kevin Newton
1e52dde82a [PRISM] Match defined behavior for explicit block
Fixes [Bug #20748]
2024-09-16 11:53:56 -04:00
Kevin Newton
2d495300e2
[PRISM] Fix up pm_compile_branch_condition issue with single insn iseqs 2024-09-12 15:49:44 -04:00
Kevin Newton
9c461cd125 [PRISM] Check error type for parsing directory 2024-09-12 13:43:04 -04:00
Kevin Newton
ca61729fa7 Fix opening multibyte character filepath on Windows 2024-09-12 13:43:04 -04:00
Kevin Newton
d4af38ec9d Fix FILE_SHARE_* permissions for Windows in read_entire_file 2024-09-12 13:43:04 -04:00
Kevin Newton
d4ab1e4482 [PRISM] Move compile scope node to its own function 2024-09-12 13:43:04 -04:00
Kevin Newton
c4b43692db [PRISM] Move case node compilation into its own function 2024-09-12 13:43:04 -04:00
Kevin Newton
ea2af5782d Switch the default parser from parse.y to Prism
This commit switches the default parser to Prism. There are a
couple of additional changes related to this that are a part of
this as well to make this happen.

* Switch the default parser in parse.h
* Remove the Prism-specific workflow and add a parse.y-specific
  workflow to CI so that it continues to be tested
* Update a few test exclusions since Prism has the correct
  behavior but parse.y doesn't per
  https://bugs.ruby-lang.org/issues/20504.
* Skips a couple of tests on RBS which are failing because they
  are using RubyVM::AbstractSyntaxTree.of.

Fixes [Feature #20564]
2024-09-12 13:43:04 -04:00
Luke Gruber
5d358b660d Fix issue with super and forwarding arguments in prism_compile.c
Fixes [Bug #20720]
2024-09-11 16:41:46 -04:00
Kevin Newton
767d0a1716
[PRISM] Fix up compile warning for sign comparison 2024-09-03 13:16:24 -04:00
Kevin Newton
371432b2d7 [PRISM] Handle RubyVM.keep_script_lines 2024-08-29 20:27:01 -04:00
Yuta Saito
591a157b0f prism_compile.c: Fix read_entire_file() for platforms without file mmap
Apply similar fix to https://github.com/ruby/prism/pull/2944
2024-08-29 14:09:35 -04:00
Kevin Newton
079161e1ba [PRISM] Respect PM_REGULAR_EXPRESSION_FLAGS_FORCED_BINARY_ENCODING flag 2024-08-29 10:31:02 -04:00
Kevin Newton
14bb376b79 [PRISM] Copy the rest of the setup_args_dup_rest_p function 2024-08-29 10:29:34 -04:00
Alan Wu
fe440c5967 [PRISM] Use node flags for dup_rest detection instead of looping 2024-08-28 19:27:15 -04:00
Alan Wu
c3ffa7106b [PRISM] Set use_block parameter flag for forwardable methods
Match logic in compile.c:2133. Without this, the unused block warning
code allocates an array, causing TestAllocation to fail.
2024-08-28 17:17:33 -04:00
Kevin Newton
417bb8d4fd [PRISM] Field renaming
Rename some fields that do not quite make sense.

* CaseMatchNode#consequent -> CaseMatchNode#else_clause
* CaseNode#consequent -> CaseNode#else_clause
* IfNode#consequent -> IfNode#subsequent
* RescueNode#consequent -> RescueNode#subsequent
* UnlessNode#consequent -> UnlessNode#else_clause
2024-08-28 15:06:53 -04:00
Alan Wu
7c9bcdf397 [PRISM] Improve dup_rest optimization targeting
Part of implementing 3de20efc30 in
prism_compile.c. Down to 2 failures from 30 failures in
test/ruby/test_allocation.rb.
2024-08-28 13:34:37 -04:00
Alan Wu
1729f47e72 [PRISM] Wait for data before reading pipes and chardevs
With the parse.y parser, when a fifo (named pipe) is passed to
Kernel#load and friends, we wait for data to be available first before
reading. Note that with fifos, opening with `O_RDONLY|O_NONBLOCK` and
then reading will look like EOF with read(2) returning 0, but data can
become available later.

The prism compiler needs to match this behavior to pass
`test_loading_fifo_{fd_leak,threading_raise,threading_success}`. I chose
to use IO#read to do this.

An alternative way to match behavior would be to use open_load_file()
from ruby.c like parse.y, but I opted to only allocate an IO to deal
with threading when reading from pipes and character devices. The
memory mapping code seems to work fine for regular files.
2024-08-27 17:39:34 -04:00
eileencodes
7462cc7743 [PRISM] Fix allocations for keyword splat params
Fixes the following allocations tests:

* `test_keyword_and_keyword_splat_parameter`
* `test_keyword_parameter`
* `test_keyword_splat_parameter`
* `test_no_array_allocation_with_splat_and_nonstatic_keywords`
* `test_no_parameters`
* `test_positional_splat_and_keyword_splat_parameter`
* `test_ruby2_keywords`

* Checks for `first_chunk` and if `stack_length == 0` to match the
upstream parser. Otherwise, this optimization is skipped.
* Subtracts the index, otherwise it will skip the hash allocation for
the following: `keyword(*empty_array, a: 2, **empty_hash)`.
* Sets `dup_rest` in order to determine when to set the correct flags
* Doesn't set `VM_CALL_KW_SPLAT_MUT` flag unless `dup_rest` doesn't
match `initial_dup_rest`.

Given the following code:

```ruby
keyword(*empty_array, a: 2)
```

Instructions before:

```
== disasm: #<ISeq:test@test.rb:4 (4,0)-(8,3)>
local table (size: 2, argc: 1 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 2] empty_hash@0<Arg>[ 1] empty_array@1
0000 newarray                               0                         (   5)[LiCa]
0002 setlocal_WC_0                          empty_array@1
0004 putself                                                          (   7)[Li]
0005 getlocal_WC_0                          empty_array@1
0007 splatarray                             true
0009 putobject                              :a
0011 putobject                              2
0013 newhash                                2
0015 opt_send_without_block                 <calldata!mid:keyword, argc:2, ARGS_SPLAT|ARGS_SPLAT_MUT|FCALL|KW_SPLAT>
0017 leave                                                            (   8)[Re]
```

Instructions after:

```
== disasm: #<ISeq:test@test.rb:4 (4,0)-(8,3)>
local table (size: 2, argc: 1 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 2] empty_hash@0<Arg>[ 1] empty_array@1
0000 newarray                               0                         (   5)[LiCa]
0002 setlocal_WC_0                          empty_array@1
0004 putself                                                          (   7)[Li]
0005 getlocal_WC_0                          empty_array@1
0007 splatarray                             false
0009 putobject                              {:a=>2}
0011 opt_send_without_block                 <calldata!mid:keyword, argc:2, ARGS_SPLAT|FCALL|KW_SPLAT>
0013 leave                                                            (   8)[Re]
```

Differences:

* `splatarray` is `false` not `true
* `putobject`, `putobject`, `newhash` is simply `putobject` with
optimizations on
* Remove `ARGS_SPLAT_MUT` flag

Related: ruby/prism#2994

Co-authored-by: Kevin Newton <kddnewton@gmail.com>
2024-08-27 16:01:46 -04:00
Kevin Newton
773cf883ac [PRISM] Reset $. when done reading STDIN 2024-08-21 16:59:03 -04:00
Kevin Newton
f60499826f [PRISM] Fix TestTRICK#test_ksk_1
If an array element is a static literal that does not result in a
intermediate array, it still needs to be compiled normally.
2024-08-21 16:32:19 -04:00
eileencodes
7ad74d1686 [PRISM] Implement unused block warning
Related: ruby/prism#2935
2024-08-21 11:37:13 -04:00
Jean Boussier
ca5b7276c6 compile.c: don't allocate empty default values list
It just wastes memory.
2024-08-11 15:04:35 +02:00
Kevin Newton
1a18b03ee7 [PRISM] Add anon_* flags for iseqs with anonymous * and ** 2024-07-24 12:02:33 -04:00
eileencodes
b4a02502c2 [PRISM] Fix block param instructions when it has a recevier
In the following code:

```ruby
def foo(&blk)
  blk.call
end
```

Prism was using the `getblockparam` instruction but it should be
`getblockparamproxy`. In this case we have a receiver, if it's a local
variable read node, then it needs to go through the path to use
`getblockparamproxy`.

Before:

```
== disasm: #<ISeq:<main>@test2.rb:1 (1,0)-(3,3)>
0000 definemethod                           :foo, foo                 (   1)[Li]
0003 putobject                              :foo
0005 leave

== disasm: #<ISeq:foo@test2.rb:1 (1,0)-(3,3)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: 0, kw: -1@-1, kwrest: -1])
[ 1] blk@0<Block>
0000 getblockparam                          blk@0, 0                  (   2)[LiCa]
0003 opt_send_without_block                 <calldata!mid:call, argc:0, ARGS_SIMPLE>
0005 leave                                                            (   3)[Re]
```

After:

```
== disasm: #<ISeq:<main>@test2.rb:1 (1,0)-(3,3)>
0000 definemethod                           :foo, foo                 (   1)[Li]
0003 putobject                              :foo
0005 leave

== disasm: #<ISeq:foo@test2.rb:1 (1,0)-(3,3)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: 0, kw: -1@-1, kwrest: -1])
[ 1] blk@0<Block>
0000 getblockparamproxy                     blk@0, 0                  (   2)[LiCa]
0003 opt_send_without_block                 <calldata!mid:call, argc:0, ARGS_SIMPLE>
0005 leave                                                            (   3)[Re]
```

Fixes `test_getblockparamproxy` and `test_ifunc_getblockparamproxy` in
test_yjit.rb. Related to ruby/prism#2935.
2024-07-23 21:45:48 -04:00
eileencodes
70d4dcb76c [Prism] Use putnil for nil kwargs, not putobject {}
This addresses one of the issues in the `test_kw_splat_nil` failure, but
doesn't make the test pass because of other changes that need to be made
to Prism directly.

One issue was when we have the following code Prism was using
`putobject` with an empty hash whereas the parse.y parser used `putnil`.

```ruby
:ok.itself(**nil)
```

Before:

```
0000 putobject                              :ok                       (   1)[Li]
0002 putobject                              {}
0004 opt_send_without_block                 <calldata!mid:itself, argc:1, KW_SPLAT>
0006 leave
```

After:

```
== disasm: #<ISeq:<main>@test2.rb:1 (1,0)-(1,17)>
0000 putobject                              :ok                       (   1)[Li]
0002 putnil
0003 opt_send_without_block                 <calldata!mid:itself, argc:1, KW_SPLAT>
0005 leave
```

Related to ruby/prism#2935.
2024-07-23 21:45:40 -04:00
Kevin Newton
b6800515e8 [PRISM] Fix up ensure compilation, match compile.c 2024-07-23 12:48:48 -04:00
Peter Zhu
ec6f40e8f1 [PRISM] Use xcalloc for constants instead of calloc 2024-07-22 14:22:47 -04:00
Peter Zhu
5299672a5b [PRISM] Fix memory leak in constants
For example, the following code leaks:

    code = 1000.times.map { |i| "var#{i} = 1" }.join("\n")

    10.times do
      1000.times do
        RubyVM::InstructionSequence.compile_prism(code)
      end

      puts `ps -o rss= -p #{$$}`
    end

Before:

    70384
    88032
    103856
    115712
    125584
    132768
    144784
    152624
    165296
    180608

After:

    62368
    78784
    74512
    87712
    85072
    77728
    69424
    74992
    71264
    81440
2024-07-22 14:22:47 -04:00
Peter Zhu
e801fa5ce8 [PRISM] Fix compiler warning for min_tmp_array_size
prism_compile.c:5770:40: warning: comparison of integer expressions of different signedness: ‘size_t’ {aka ‘long unsigned int’} and ‘int’ [-Wsign-compare]
 5770 |                     if (tmp_array_size >= min_tmp_array_size) {
      |                                        ^~
2024-07-19 11:45:31 -04:00
Peter Zhu
b226c3407e [PRISM] Fix compiler warning for min_tmp_hash_length
prism_compile.c:1406:27: warning: comparison of integer expressions of different signedness: ‘size_t’ {aka ‘long unsigned int’} and ‘int’ [-Wsign-compare]
 1406 |                 if (count >= min_tmp_hash_length) {
      |                           ^~
2024-07-19 11:45:31 -04:00
eileencodes
69e65b9b5a Fix interpolated sybmol node instructions
If the symbol node is interpolated like this `:"#{foo}"` the instruction
sequence should be `putstring` followed by `intern`. In this case it was
a `putobject` causing the `test_yjit` tests to fail. Note that yjit is
not required to reproduce - the instructions are `putstring` and
`intern` for yjit and non-yjit with the original parser.

To fix I moved `pm_interpolated_node_compile` out of the else, and
entirely removed the conditional. `pm_interpolated_node_compile` knows
how / when to use `putstring` over `putobject` already. The `intern` is
then added by removing the conditional.

Before:

```
== disasm: #<ISeq:<main>@test2.rb:1 (1,0)-(1,11)>
0000 putobject                              :foo                      (   1)[Li]
0002 leave
```

After:

```
== disasm: #<ISeq:<main>@test2.rb:1 (1,0)-(1,11)>
0000 putstring                              "foo"                     (   1)[Li]
0002 intern
0003 leave
```

Fixes the test `TestYJIT#test_compile_dynamic_symbol`. Related to ruby/prism#2935
2024-07-18 21:15:43 -04:00
Kevin Newton
1fd1fb2aa5 [PRISM] Use KW_SPLAT_MUT when possible for method calls 2024-07-18 15:30:45 -04:00
Kevin Newton
53710be557 [PRISM] Use concattoarray instead of splatarray+concatarray 2024-07-18 15:30:45 -04:00
eileencodes
aa3030ac24 Fix empty hash instruction
When we have an empty hash the iseq should have a `newhash` but instead
had a `duphash`. To fix, check if the node's elements are equal to `0`.
If so we want a `newhash`, otherwise use the original `duphash`
instructions.

Before:

```
== disasm: #<ISeq:<main>@test2.rb:1 (1,0)-(1,2)>
0000 duphash                                {}                        (   1)[Li]
0002 leave
```

After:

```
== disasm: #<ISeq:<main>@test2.rb:1 (1,0)-(1,2)>
0000 newhash                                0                         (   1)[Li]
0002 leave
```

Fixes the test `TestYJIT#test_compile_newhash`. Related to ruby/prism#2935
2024-07-18 10:12:20 -04:00
Kevin Newton
2cc20c06e0
[PRISM] Use RSTRING_PTR for Ruby parsing with fgets 2024-07-17 15:45:07 -04:00
Kevin Newton
7de2c06352
[PRISM] Use RSTRING_LEN for Prism stream parsing 2024-07-17 14:13:16 -04:00
Nobuyoshi Nakada
644424941a [Bug #20457] [Prism] Remove redundant return flag 2024-07-17 14:06:11 -04:00
Kevin Newton
7993b88eee [PRISM] Use StringValuePtr for fgets for Prism stream parsing 2024-07-17 13:58:58 -04:00
Kevin Newton
b0a99d0da9 [PRISM] Properly compile branch conditions in their own sequence 2024-07-16 14:40:20 -04:00
Kevin Newton
90e945a7b7 [PRISM] Fix up ensure+loop+break 2024-07-16 14:40:20 -04:00
Kevin Newton
2911578ed7
[PRISM] Add missing rescue tracepoint for rescue modifier 2024-07-15 15:03:54 -04:00
Kevin Newton
8080de04be [PRISM] Optimize inner static literal hashes 2024-07-15 14:04:25 -04:00
Kevin Newton
c1e5358448 [PRISM] Optimize pushing large hash literals 2024-07-15 14:04:25 -04:00
Kevin Newton
b38493c572 [PRISM] Chunk sub-arrays of static literals in array literals
Co-authored-by: Adam Hess <HParker@github.com>
2024-07-15 14:04:25 -04:00
Kevin Newton
fb6d54143d [PRISM] Optimizations for compiling large arrays 2024-07-15 14:04:25 -04:00
Kevin Newton
1f6aeadc82 [PRISM] Fix Windows 2015 segfault 2024-07-11 14:25:54 -04:00
Kevin Newton
ac093f5a06 [PRISM] Fix up shareable constant casting 2024-07-11 14:25:54 -04:00
Kevin Newton
c1df15c3e6 [PRISM] Use node ids for error highlight 2024-07-11 14:25:54 -04:00