Commit graph

302 commits

Author SHA1 Message Date
TSUYUSATO Kitsune
16830a4783 [ruby/prism] Add an error for in keyword in arguments
Fix https://github.com/ruby/prism/pull/2026

c4b41cd477
2023-12-15 13:25:54 +00:00
Kevin Newton
019fff3a86 [ruby/prism] Fix parse result for nesting pattern matching
ee6fc9ee87
2023-12-14 18:54:46 +00:00
Ufuk Kayserilioglu
01f21d5729 [ruby/prism] Fix the implementation of the flag on keyword hash nodes
The previous implementation was incorrect since it was just checking for all keys in assoc nodes to be static literals but the actual check is that all keys in assoc nodes must be symbol nodes.

This commit fixes that implementation, and, also, aliases the flag to `PM_KEYWORD_HASH_NODE_FLAGS_SYMBOL_KEYS` so that ruby/ruby can start using the new flag name.

I intend to later change the real flag name to `PM_KEYWORD_HASH_NODE_FLAGS_SYMBOL_KEYS` and remove the alias.

f5099c79ce
2023-12-14 18:05:54 +00:00
TSUYUSATO Kitsune
3658798dbb [ruby/prism] Make equality operators non-associative
Fix https://github.com/ruby/prism/pull/2073

0f747d9240
2023-12-14 16:39:05 +00:00
Kevin Newton
b7e89d4b17 [ruby/prism] Fix hash pattern rest
43c4232cfc
2023-12-14 02:43:32 +00:00
Ufuk Kayserilioglu
e96f6126f2 [ruby/prism] Fix hash deopt based on contents
The previous implementation of hash deopt was based on clearing the static literal flag on a hash node if the element that was being added was an array, hash or range node, or if the element was not a static literal in the first place.

However, this is not correct. First of all, the elements added to a hash node will primarily be assoc nodes, but never array, hash or range nodes. Secondly, the static literal flag is set on assoc nodes, only if the value in an assoc node is a static literal, so the key is never checked. As a result, the static literal flag on a hash node would never be cleared if the key wasn't a static literal.

This commit fixes this by clearing the static literal flag if:
1. the element is not an assoc node,
2. the element is an assoc node, but the key is not a static literal, or
3. the element is an assoc node, the key is a static literal, but assoc node (and thus the value in assoc node) is not a static literal.

7f67109b36
2023-12-13 00:21:42 +00:00
Kevin Newton
f61df27b4c [ruby/prism] Use new flag setters for attribute write
1e07832778
2023-12-12 18:43:59 +00:00
Ufuk Kayserilioglu
1f22245ed5 [ruby/prism] Start using flag macros/methods exclusively
5f05a6fe83
2023-12-12 17:35:53 +00:00
Ufuk Kayserilioglu
bdb38dd9f2 [ruby/prism] Add methods for setting/unsetting and macros for testing a flags
e5f37d1407
2023-12-12 17:35:52 +00:00
Kevin Newton
278ce27ee4 [ruby/prism] Flag for attribute write on calls
465731969c
2023-12-12 15:55:54 +00:00
Ufuk Kayserilioglu
43229d531f [ruby/prism] Start KeywordHashNodes with STATIC_KEYS set, until hit an element that should clear it
7c7c486507
2023-12-12 13:05:09 +00:00
Kevin Newton
4095e7d2be [ruby/prism] Regexp terminator escapes
42a48a2ba9
2023-12-11 16:34:55 +00:00
Kevin Newton
c65de63913 [ruby/prism] Handle a non-interpolated dsym spanning a heredoc
b23136ebfd
2023-12-11 16:34:41 +00:00
Kevin Newton
261e8f28a0 [ruby/prism] Correct escapes when ctrl+meta+escape
ee68b17c01
2023-12-11 16:34:27 +00:00
Kevin Newton
b673b5b432 [ruby/prism] Split up CallNode in target position
In this commit we're splitting up the call nodes that were in target
positions (that is, for loop indices, rescue error captures, and
multi assign targets).

Previously, we would simply leave the call nodes in place. This had
the benefit of keeping the AST relatively simple, but had the
downside of not being very explicit. If a static analysis tool wanted
to only look at call nodes, it could easily be confused because the
method would have 1 fewer argument than it would actually be called
with.

This also brings some consistency to the AST. All of the nodes in
a target position are now *TargetNode nodes. These should all be
treated the same, and the call nodes can now be treated the same.

Finally, there is benefit to memory. Because being in a target
position ensures we don't have some fields, we can strip down the
number of fields on these nodes.

So this commit introduces two new nodes: CallTargetNode and
IndexTargetNode. For CallTargetNode we get to drop the opening_loc,
closing_loc, arguments, and block. Those can never be present. We
also get to mark their fields as non-null, so they will always be
seen as present.

The IndexTargetNode keeps around most of its fields but gets to
drop both the name (because it will always be []=) and the
message_loc (which was always super confusing because it included
the arguments by virtue of being inside the []).

Overall, this adds complexity to the AST at the expense of memory
savings and explicitness. I believe this tradeoff is worth it in
this case, especially because these are very much not common nodes
in the first place.

3ef71cdb45
2023-12-11 15:32:31 +00:00
TSUYUSATO Kitsune
48cb70fee9 [ruby/prism] Fix parsing unterminated empty string "
Fix https://github.com/ruby/prism/pull/2034

8280e577fa
2023-12-11 13:36:37 +00:00
TSUYUSATO Kitsune
a860e3605c [ruby/prism] Fix to parse a (endless-)range with binary operators
Fix https://github.com/ruby/prism/pull/2022
Fix https://github.com/ruby/prism/pull/2030

b78d8b6525
2023-12-11 13:34:48 +00:00
Aaron Patterson
0166040939 [ruby/prism] Hashes need to deoptimize based on their contents
If a hash has children that are not "basic" types of objects, then we
can't consider the hash to be a static literal.

Fixes: #2015

ca2523137d
2023-12-08 14:35:26 +00:00
Haldun Bayhantopcu
ce094e8556 [ruby/prism] Emit error for constant assignments in defs
864b06f90e
2023-12-06 22:15:16 +00:00
Kevin Newton
c5a4409f20 [ruby/prism] Simplify unterminated string
ef512ca914
2023-12-06 21:34:48 +00:00
TSUYUSATO Kitsune
cbb941f58a [ruby/prism] Fix closing loc for string literals
Fix https://github.com/ruby/prism/pull/1974

453d403593
2023-12-06 20:48:11 +00:00
Haldun Bayhantopcu
a57186b9d1 [ruby/prism] Fix one potential memory leak and silence one false positive report.
9608aa386e
2023-12-06 20:38:22 +00:00
Kevin Newton
82f18baa21 [ruby/prism] Provide flags for changing encodings
e838eaff6f
2023-12-06 14:23:38 -05:00
Jemma Issroff
018dbf18d5 [ruby/prism] Add locals_body_index to DefNode, BlockNode, LambdaNode
The locals_body_index gives the index in the locals array where
the locals from the body start. This allows compilers to easily
index past the parameters in the locals array.

5d4627b890
2023-12-06 09:55:48 -05:00
eileencodes
a33632e1ca [ruby/prism] Fix defined with new line
It's possible to write the following and have it be valid Ruby:

```
defined?("foo"
)
```

But Prism wasn't taking the new line into account. This adds an
`accept1` for a `PM_TOKEN_NEWLINE` to account for this. I've also
updated the fixtures and snapshots to test this.

b87f8eedc6
2023-12-05 20:47:22 +00:00
Haldun Bayhantopcu
94bf9f8037 [ruby/prism] Remove unnecessary match
694d3d7279
2023-12-05 15:25:20 +00:00
TSUYUSATO Kitsune
ff6706b08e [ruby/prism] Fix a typo
https://github.com/ruby/prism/pull/1949#discussion_r1412568793

a00f21d10a
2023-12-05 15:16:42 +00:00
TSUYUSATO Kitsune
975b74a8eb [ruby/prism] Fix argument order
https://github.com/ruby/prism/pull/1949#discussion_r1410733341

27635da821

Co-Authored-By: Kevin Newton <kddnewton@gmail.com>
2023-12-05 15:16:41 +00:00
TSUYUSATO Kitsune
3d4a4b2529 [ruby/prism] Fix some corner cases
d5453f168e
2023-12-05 15:16:41 +00:00
TSUYUSATO Kitsune
dcd75fd49d [ruby/prism] Fix to parse command-style method calls more correctly
Fix https://github.com/ruby/prism/pull/1468
Fix https://github.com/ruby/prism/pull/1575

To decide command-style method calls are allowed, this introduce a new
parameter `accepts_command_call` to `parse_expression` and some
functions.

Although one think this can be solved by operator precedence, it is
hard or impossible, because the precedence of command-style calls is skewed
(e.g. `! bar 1 ` is accepted, but `foo = ! bar 1` is rejected.)

One of the most complex examples is that:
(1) even though `foo = bar = baz 1` and `foo, bar = baz 1` is accepted,
(2) `foo, bar = baz = fuzz 1` is rejected.
To implement this behavior, this introduces a new binding power
`PM_BINDING_POWER_MULTI_ASSIGNMENT` and uses it for distinguish which single
assignments or multi assignments at their RHS.

d4dd49ca81
2023-12-05 15:16:40 +00:00
Lily Lyons
1227b6d912 [ruby/prism] Refactor pm_diagnostic_t and pm_comment_t to use pm_location_t
115b6a2fc6
2023-12-04 14:33:44 +00:00
TSUYUSATO Kitsune
2a65d83707 [ruby/prism] Check "void value expression" for array literals
Fix https://github.com/ruby/prism/pull/1978

194c997d0a
2023-12-04 14:00:23 +00:00
Haldun Bayhantopcu
562d949e02 [ruby/prism] Fix parsing heredoc ends
aa8c702271
2023-12-01 20:10:58 +00:00
Kevin Newton
cdb74d74af [ruby/prism] Change numbered parameters
Previously numbered parameters were a field on blocks and lambdas
that indicated the maximum number of numbered parameters in either
the block or lambda, respectively. However they also had a
parameters field that would always be nil in these cases.

This changes it so that we introduce a NumberedParametersNode that
goes in place of parameters, which has a single uint8_t maximum
field on it. That field contains the maximum numbered parameter in
either the block or lambda.

As a part of the PR, I'm introducing a new UInt8Field type that
can be used on nodes, which is just to make it a little more
explicit what the maximum values can be (the maximum is actually 9,
since it only goes up to _9). Plus we can do a couple of nice
things in serialization like just read a single byte.

2d87303903
2023-12-01 12:03:09 -05:00
TSUYUSATO Kitsune
ffeec108cf [ruby/prism] Add missing context names to the switch in debug_context
Now, `PM_DEBUG_LOGGING` can be available.

569ffacb5f
2023-12-01 15:15:28 +00:00
TSUYUSATO Kitsune
417d700406 [ruby/prism] Improve to handle unterminated strings
Fix https://github.com/ruby/prism/pull/1946

This fixes to set an error position for unterminated strings to the
opening delimiters. Previously, the error position was set to the end
of the delimiter.

The same fix applies to other string-like literals.

Additionally, this fixes https://github.com/ruby/prism/pull/1946; that is, it adds the last part of the
string even though the string literal does not terminate.

c1240baafd
2023-12-01 15:15:01 +00:00
TSUYUSATO Kitsune
0e599336a2 [ruby/prism] Fix not_provided comment
I don't know when it is changed, but the `not_provided` signature is
changed.

0255cc307d
2023-12-01 15:13:45 +00:00
Matt Valentine-House
ce5f5ca1d6 [PRISM] Remove transparent scope nodes 2023-12-01 15:04:13 +00:00
Kevin Newton
0c277f2b76 [ruby/prism] Group encodings into a single array
f4b7beadc9
2023-11-30 21:37:56 -05:00
Kevin Newton
6b234c1acf [ruby/prism] Do not expose encodings that do not need to be exposed
c52c7f37ea
2023-11-30 21:37:56 -05:00
Kevin Newton
ea409958b3 [ruby/prism] Remove ability to decode other encodings
98e218d989
2023-11-30 21:37:56 -05:00
Kevin Newton
19a321bff0 [ruby/prism] CESU encoding
2d5b9c2b3c
2023-11-30 21:37:56 -05:00
Kevin Newton
ca26e0e34b [ruby/prism] EUC-TW encoding
edfb54f039
2023-11-30 21:37:56 -05:00
Kevin Newton
32249c2cf1 [ruby/prism] GB18030 encoding
ca3ab7ec89
2023-11-30 21:37:56 -05:00
Kevin Newton
a9162a44c5 [ruby/prism] Emacs MULE encodings
4c06b6c42e
2023-11-30 21:37:56 -05:00
Kevin Newton
10d3897e13 [PRISM] Big5 encodings 2023-11-30 21:37:56 -05:00
Kevin Newton
700e172a50 [ruby/prism] EUC-KR encodings
ba5218385a
2023-11-30 21:37:56 -05:00
Kevin Newton
ddaa073058 [ruby/prism] Support other EUC-JP encodings
d040337ce9
2023-11-30 21:37:56 -05:00
Kevin Newton
9ba92327f2 [PRISM] Consolidate SJIS encodings 2023-11-30 21:37:56 -05:00
Kevin Newton
219c3c1c09 [ruby/prism] Add other UTF8 encodings
709fb6e09f
2023-11-30 21:37:56 -05:00