Commit graph

337 commits

Author SHA1 Message Date
TSUYUSATO Kitsune
48cb70fee9 [ruby/prism] Fix parsing unterminated empty string "
Fix https://github.com/ruby/prism/pull/2034

8280e577fa
2023-12-11 13:36:37 +00:00
TSUYUSATO Kitsune
a860e3605c [ruby/prism] Fix to parse a (endless-)range with binary operators
Fix https://github.com/ruby/prism/pull/2022
Fix https://github.com/ruby/prism/pull/2030

b78d8b6525
2023-12-11 13:34:48 +00:00
Aaron Patterson
0166040939 [ruby/prism] Hashes need to deoptimize based on their contents
If a hash has children that are not "basic" types of objects, then we
can't consider the hash to be a static literal.

Fixes: #2015

ca2523137d
2023-12-08 14:35:26 +00:00
Haldun Bayhantopcu
ce094e8556 [ruby/prism] Emit error for constant assignments in defs
864b06f90e
2023-12-06 22:15:16 +00:00
Kevin Newton
c5a4409f20 [ruby/prism] Simplify unterminated string
ef512ca914
2023-12-06 21:34:48 +00:00
TSUYUSATO Kitsune
cbb941f58a [ruby/prism] Fix closing loc for string literals
Fix https://github.com/ruby/prism/pull/1974

453d403593
2023-12-06 20:48:11 +00:00
Haldun Bayhantopcu
a57186b9d1 [ruby/prism] Fix one potential memory leak and silence one false positive report.
9608aa386e
2023-12-06 20:38:22 +00:00
Kevin Newton
82f18baa21 [ruby/prism] Provide flags for changing encodings
e838eaff6f
2023-12-06 14:23:38 -05:00
Jemma Issroff
018dbf18d5 [ruby/prism] Add locals_body_index to DefNode, BlockNode, LambdaNode
The locals_body_index gives the index in the locals array where
the locals from the body start. This allows compilers to easily
index past the parameters in the locals array.

5d4627b890
2023-12-06 09:55:48 -05:00
eileencodes
a33632e1ca [ruby/prism] Fix defined with new line
It's possible to write the following and have it be valid Ruby:

```
defined?("foo"
)
```

But Prism wasn't taking the new line into account. This adds an
`accept1` for a `PM_TOKEN_NEWLINE` to account for this. I've also
updated the fixtures and snapshots to test this.

b87f8eedc6
2023-12-05 20:47:22 +00:00
Haldun Bayhantopcu
94bf9f8037 [ruby/prism] Remove unnecessary match
694d3d7279
2023-12-05 15:25:20 +00:00
TSUYUSATO Kitsune
ff6706b08e [ruby/prism] Fix a typo
https://github.com/ruby/prism/pull/1949#discussion_r1412568793

a00f21d10a
2023-12-05 15:16:42 +00:00
TSUYUSATO Kitsune
975b74a8eb [ruby/prism] Fix argument order
https://github.com/ruby/prism/pull/1949#discussion_r1410733341

27635da821

Co-Authored-By: Kevin Newton <kddnewton@gmail.com>
2023-12-05 15:16:41 +00:00
TSUYUSATO Kitsune
3d4a4b2529 [ruby/prism] Fix some corner cases
d5453f168e
2023-12-05 15:16:41 +00:00
TSUYUSATO Kitsune
dcd75fd49d [ruby/prism] Fix to parse command-style method calls more correctly
Fix https://github.com/ruby/prism/pull/1468
Fix https://github.com/ruby/prism/pull/1575

To decide command-style method calls are allowed, this introduce a new
parameter `accepts_command_call` to `parse_expression` and some
functions.

Although one think this can be solved by operator precedence, it is
hard or impossible, because the precedence of command-style calls is skewed
(e.g. `! bar 1 ` is accepted, but `foo = ! bar 1` is rejected.)

One of the most complex examples is that:
(1) even though `foo = bar = baz 1` and `foo, bar = baz 1` is accepted,
(2) `foo, bar = baz = fuzz 1` is rejected.
To implement this behavior, this introduces a new binding power
`PM_BINDING_POWER_MULTI_ASSIGNMENT` and uses it for distinguish which single
assignments or multi assignments at their RHS.

d4dd49ca81
2023-12-05 15:16:40 +00:00
Lily Lyons
1227b6d912 [ruby/prism] Refactor pm_diagnostic_t and pm_comment_t to use pm_location_t
115b6a2fc6
2023-12-04 14:33:44 +00:00
TSUYUSATO Kitsune
2a65d83707 [ruby/prism] Check "void value expression" for array literals
Fix https://github.com/ruby/prism/pull/1978

194c997d0a
2023-12-04 14:00:23 +00:00
Haldun Bayhantopcu
562d949e02 [ruby/prism] Fix parsing heredoc ends
aa8c702271
2023-12-01 20:10:58 +00:00
Kevin Newton
cdb74d74af [ruby/prism] Change numbered parameters
Previously numbered parameters were a field on blocks and lambdas
that indicated the maximum number of numbered parameters in either
the block or lambda, respectively. However they also had a
parameters field that would always be nil in these cases.

This changes it so that we introduce a NumberedParametersNode that
goes in place of parameters, which has a single uint8_t maximum
field on it. That field contains the maximum numbered parameter in
either the block or lambda.

As a part of the PR, I'm introducing a new UInt8Field type that
can be used on nodes, which is just to make it a little more
explicit what the maximum values can be (the maximum is actually 9,
since it only goes up to _9). Plus we can do a couple of nice
things in serialization like just read a single byte.

2d87303903
2023-12-01 12:03:09 -05:00
TSUYUSATO Kitsune
ffeec108cf [ruby/prism] Add missing context names to the switch in debug_context
Now, `PM_DEBUG_LOGGING` can be available.

569ffacb5f
2023-12-01 15:15:28 +00:00
TSUYUSATO Kitsune
417d700406 [ruby/prism] Improve to handle unterminated strings
Fix https://github.com/ruby/prism/pull/1946

This fixes to set an error position for unterminated strings to the
opening delimiters. Previously, the error position was set to the end
of the delimiter.

The same fix applies to other string-like literals.

Additionally, this fixes https://github.com/ruby/prism/pull/1946; that is, it adds the last part of the
string even though the string literal does not terminate.

c1240baafd
2023-12-01 15:15:01 +00:00
TSUYUSATO Kitsune
0e599336a2 [ruby/prism] Fix not_provided comment
I don't know when it is changed, but the `not_provided` signature is
changed.

0255cc307d
2023-12-01 15:13:45 +00:00
Matt Valentine-House
ce5f5ca1d6 [PRISM] Remove transparent scope nodes 2023-12-01 15:04:13 +00:00
Kevin Newton
0c277f2b76 [ruby/prism] Group encodings into a single array
f4b7beadc9
2023-11-30 21:37:56 -05:00
Kevin Newton
6b234c1acf [ruby/prism] Do not expose encodings that do not need to be exposed
c52c7f37ea
2023-11-30 21:37:56 -05:00
Kevin Newton
ea409958b3 [ruby/prism] Remove ability to decode other encodings
98e218d989
2023-11-30 21:37:56 -05:00
Kevin Newton
19a321bff0 [ruby/prism] CESU encoding
2d5b9c2b3c
2023-11-30 21:37:56 -05:00
Kevin Newton
ca26e0e34b [ruby/prism] EUC-TW encoding
edfb54f039
2023-11-30 21:37:56 -05:00
Kevin Newton
32249c2cf1 [ruby/prism] GB18030 encoding
ca3ab7ec89
2023-11-30 21:37:56 -05:00
Kevin Newton
a9162a44c5 [ruby/prism] Emacs MULE encodings
4c06b6c42e
2023-11-30 21:37:56 -05:00
Kevin Newton
10d3897e13 [PRISM] Big5 encodings 2023-11-30 21:37:56 -05:00
Kevin Newton
700e172a50 [ruby/prism] EUC-KR encodings
ba5218385a
2023-11-30 21:37:56 -05:00
Kevin Newton
ddaa073058 [ruby/prism] Support other EUC-JP encodings
d040337ce9
2023-11-30 21:37:56 -05:00
Kevin Newton
9ba92327f2 [PRISM] Consolidate SJIS encodings 2023-11-30 21:37:56 -05:00
Kevin Newton
219c3c1c09 [ruby/prism] Add other UTF8 encodings
709fb6e09f
2023-11-30 21:37:56 -05:00
Haldun Bayhantopcu
0d1917aead [ruby/prism] Remove no-op assignment
4b3079d9e8
2023-11-30 17:42:24 +00:00
sid-707
2c64041ed5 [ruby/prism] Fix typo in comment
eb1a6ba263
2023-11-30 14:10:28 +00:00
Matt Boldt
9fc40d2b26 [ruby/prism] Add MacJapanese encoding
MacJapanese (also aliased as MacJapan) is a modified Shift_JIS
encoding, but is implemented identically in Ruby

9e0a097699
2023-11-29 12:08:15 -05:00
Dhaval
9fada99cb2 [ruby/prism] added CP950 encoding
9c2d1cf4ba
2023-11-29 11:15:50 -05:00
Jemma Issroff
b632732bcf [ruby/prism] Fixed comment on pm_parser_numbered_parameters_set
ceae5727b8
2023-11-29 15:24:41 +00:00
Jean Boussier
2af82e2316 [ruby/prism] Convert start line to signed integers
Ruby allows for 0 or negative line start, this is often used
with `eval` calls to get a correct offset when prefixing a snippet.

e.g.

```ruby
caller = caller_locations(1, 1).first
class_eval <<~RUBY, caller.path, caller.line - 2
  # frozen_string_literal: true
  def some_method
    #{caller_provided_code_snippet}
  end
RUBY
```

0d14ed1452
2023-11-29 13:56:19 +00:00
Jean Boussier
2653404840 [ruby/prism] Rename varint as varuint
Line numbers may be negative, so we need to introduce signed varint,
so renaming unsigned ones first avoid confusion.

90d862361e
2023-11-29 13:56:18 +00:00
TSUYUSATO Kitsune
a908cef53f [ruby/prism] Reject class/module defs in method params/rescue/ensure/else
Fix https://github.com/ruby/prism/pull/1936

232e77a003
2023-11-29 02:03:06 +00:00
Kevin Newton
4938390177 [ruby/prism] Implicit rest nodes
Fundamentally, `foo { |bar,| }` is different from `foo { |bar, *| }`
because of arity checks. This PR introduces a new node to handle
that, `ImplicitRestNode`, which goes in the `rest` slot of parameter
nodes instead of `RestParameterNode` instances.

This is also used in a couple of other places, namely:

* pattern matching: `foo in [bar,]`
* multi target: `for foo, in bar do end`
* multi write: `foo, = bar`

Now the only splat nodes with a `NULL` value are when you're
forwarding, as in: `def foo(*) = bar(*)`.

dba2a3b652
2023-11-28 22:33:50 +00:00
Jemma Issroff
04cbcd37b1 [ruby/prism] Add numbered_parameters field to BlockNode and LambdaNode
We are aware at parse time how many numbered parameters we have
on a BlockNode or LambdaNode, but prior to this commit, did not
store that information anywhere in its own right.

The numbered parameters were stored as locals, but this does not
distinguish them from other locals that have been set, for example
in `a { b = 1; _1 }` there is nothing on the AST that distinguishes
b from _1.

Consumers such as the compiler need to know information about how
many numbered parameters exist to set up their own tables around
parameters. Since we have this information at parse time, we should
compute it here, instead of deferring the work later on.

bf4a1e124d
2023-11-28 21:08:46 +00:00
TSUYUSATO Kitsune
c11dd34753 [ruby/prism] Reset current_param_name around closed scopes
It is for supporting `def foo(bar = (def baz(bar) = bar; 1)) = 2` case.

c789a833c5
2023-11-28 17:27:10 +00:00
TSUYUSATO Kitsune
f6fbb9fec5 [ruby/prism] Use 0 for the default valie of current_param_name
896915de24
2023-11-28 17:27:10 +00:00
TSUYUSATO Kitsune
b5796d7b11 [ruby/prism] Check circular references in default values of params
Fix https://github.com/ruby/prism/pull/1637

0172d69cba
2023-11-28 17:27:09 +00:00
TSUYUSATO Kitsune
9365b78d49 [ruby/prism] Fix to parse * as forwarding in foo[*] case
Fix https://github.com/ruby/prism/pull/1924

7cde900065
2023-11-28 13:25:59 +00:00
Kevin Newton
c798943a4a [ruby/prism] Move DATA parsing into its own parse result field
42b60b6e95
2023-11-28 13:25:48 +00:00