Commit graph

72 commits

Author SHA1 Message Date
Kevin Newton
f33c412ebc [ruby/yarp] Constant paths followed by an & should be lexed as a call
b0a2ba2c4d
2023-08-23 18:29:25 +00:00
Kevin Newton
c837e1adfb [ruby/yarp] Add LABEL lex state when lexing a keyword params
422bcd0ebf
2023-08-23 16:07:37 +00:00
Mike Dalessio
9c43ec621d [ruby/yarp] fix: newline tracking for a comment at EOF
62fb0bddf5
2023-08-23 15:42:20 +00:00
Mike Dalessio
8f0a8e579d [ruby/yarp] Remove unnecessary loop
86e8741ee3
2023-08-23 15:40:21 +00:00
Mike Dalessio
f902df128d [ruby/yarp] refactor: extract peek_addr()
In many places in the code we use the idiom:

    x < parser->end && *x == 'y'

which is essentially an extension of an existing pattern:

- `peek()` looks at `parser->current.end`
- `peek_at()` looks at `(parser->current.end + offset)`

This commit introduces a new inline function, `peek_addr`, which
accepts a pointer and encapsulates the address value check and
conditional dereferencing. The result is more readable code, and more
ubiquitous safety checks on pointer values, allowing us to rewrite the
above as:

    peek_addr(parser, x) == 'y'

Also:

- change the type of `peek_at()`'s offset argument from `size_t` to
  `ptrdiff_t` so that it can accept negative offsets.
- use `current_token_starts_line` in one place where the equivalent
  code is inline.
- use `peek` or `peek_at` to replace inline code in a few places

These changes simplify the code and make it easier to visually spot
patterns, particularly around line-endings (which will be a subject of
a future pull request).

4c608d53ea
2023-08-23 15:08:35 +00:00
Mike Dalessio
95e29b0423 [ruby/yarp] fix: avoid invalid memory read when CR is present without LF
2296c037de
2023-08-21 19:00:55 +00:00
Mike Dalessio
461f8eaba7 [ruby/yarp] fix: parsing a '%' expression with a CR but not a newline
Previously this failed an assertion and aborted.

a037d942a8
2023-08-21 19:00:52 +00:00
Jemma Issroff
7d26c03267
Manually resync YARP
YARP commits were synced out of order. We are doing this reset
commit to ensure that all files are currently correct and we can
proceed with monitoring syncs so this doesn't happen again.
2023-08-21 10:45:56 -07:00
Mike Dalessio
9ca547b9fe [ruby/yarp] prefactor: extract yp_newline_list_check_append
149c74291b
2023-08-21 10:43:24 -07:00
Mike Dalessio
926857eb1e [ruby/yarp] fix: support newline-terminated regular expressions
Previously, parsing a snippet like this:

    %r\nfoo\n

would result in tracking the second newline twice, resulting in a
failed runtime assertion.

Fixing that issue reveals another bug, which is that the _first_
newline was not being tracked at all. So we introduce a call to
yp_newline_list right when we construct the REGEXP_BEGIN token.

0d5d759091
2023-08-21 10:43:23 -07:00
Kevin Newton
e63bac3128 [ruby/yarp] Rename statements to body where appropriate
0aa7d9d10c
2023-08-21 10:43:21 -07:00
Kevin Newton
a7f40fc2d7 [ruby/yarp] Change AndWriteNode, OrWriteNode, OperatorWriteNode to contain write nodes
It makes it more difficult to find all locations where a variable
is written to if they're just read nodes. To keep things consistent
we should make them write nodes.

840e094045
2023-08-21 10:43:18 -07:00
Kevin Newton
988b0108fb [ruby/yarp] Consolidate OperatorAndWrite and OperatorOrWrite nodes
9e680a7598
2023-08-21 10:43:16 -07:00
Kevin Newton
4fb2b3b6bf [ruby/yarp] Ensure correct location with if/elsif, allow FOCUS env var, fix newlines on __END__
9da0bc4452
2023-08-21 10:43:13 -07:00
Mike Dalessio
05c59995fa [ruby/yarp] fix: newline tracking for nl-terminated % %q %Q
1e4940864b
2023-08-21 10:33:45 -07:00
Mike Dalessio
dcc8afe9a4 [ruby/yarp] prefactor: extract yp_newline_list_check_append
149c74291b
2023-08-21 10:33:43 -07:00
Mike Dalessio
ac819f4db9 [ruby/yarp] fix: support newline-terminated regular expressions
Previously, parsing a snippet like this:

    %r\nfoo\n

would result in tracking the second newline twice, resulting in a
failed runtime assertion.

Fixing that issue reveals another bug, which is that the _first_
newline was not being tracked at all. So we introduce a call to
yp_newline_list right when we construct the REGEXP_BEGIN token.

0d5d759091
2023-08-21 12:22:53 -04:00
Kevin Newton
1d0b627b70 [ruby/yarp] Rename statements to body where appropriate
0aa7d9d10c
2023-08-19 01:04:04 +00:00
Kevin Newton
07833049df [ruby/yarp] Consolidate operator write nodes
22b287e2b1
2023-08-18 20:25:00 +00:00
Kevin Newton
ec47fc9539 [ruby/yarp] Consolidate OperatorAndWrite and OperatorOrWrite nodes
9e680a7598
2023-08-18 20:24:56 +00:00
Mike Dalessio
e1505aebf2 [ruby/yarp] fix: multiple heredocs with embedded expressions with newlines
Set heredoc_end to NULL at the start of lexing a heredoc, to avoid
having state from the previous heredoc confuse the parser's current
location.

21ee304f0e
2023-08-18 18:30:49 +00:00
HParker
db076d8e84 [ruby/yarp] handle missing HEREDOC endline at start of heredoc
7b72493b6d
2023-08-18 18:05:09 +00:00
Mike Dalessio
2800d1fd37 [ruby/yarp] fix: unterminated % in arguments
efb3102369
2023-08-18 12:51:14 +00:00
Takashi Kokubun
f50e814dae Resync YARP to 11ad3fb590 2023-08-17 10:01:57 -07:00
Takashi Kokubun
40002dd7dc Resync YARP 2023-08-17 09:58:56 -07:00
Takashi Kokubun
3873b1eb39 Resync YARP 2023-08-16 17:47:32 -07:00
HParker
a8c70ed2b4 [ruby/yarp] add a diagnostic for *rest in order after optional state
908244ba12
2023-08-16 17:47:32 -07:00
Kevin Newton
bf723b21cc [ruby/yarp] More flip flop flags
Whenever you see a `not` or a `!`, the receiver of that method should
potentially be marked as a flip-flop.

ba5977a40a
2023-08-16 17:47:32 -07:00
Kevin Newton
fb287fa425 [ruby/yarp] Support the flipflop flag
6315890390
2023-08-16 17:47:32 -07:00
Jemma Issroff
86cc82a1ab [ruby/yarp] Improved comments, moved flags after type
4815d77e82
2023-08-16 17:47:32 -07:00
Jemma Issroff
383c47f538 [ruby/yarp] Separate yp_node_flags_t from yp_node_type_t
Prior to this commit, we folded the flags into the type. This created
extra overhead when calculating the type and setting the flags. This
commit separates them.

b783a5678c
2023-08-16 17:47:32 -07:00
Kevin Newton
89de544442 [ruby/yarp] Set constant path write node target type
771f7ed789
2023-08-16 17:47:32 -07:00
Kevin Newton
ee885237f6 [ruby/yarp] Fix string concat parsing
58f839a3eb
2023-08-16 17:47:32 -07:00
Kevin Newton
77e52735f0 [ruby/yarp] Fix warnings found from gcc-analyzer and clang-analyzer
1f9a6e05c4
2023-08-16 17:47:32 -07:00
Thomas E. Enebo
96aebb4265 [ruby/yarp] Address PR comments
- odd whitespace
- a couple of name changes
- properly read uint32_t when not properly aligned

3208ee3983
2023-08-16 17:47:32 -07:00
Kevin Newton
d6f9f3e498 [ruby/yarp] Rename RescueNode#exception to RescueNode#reference
ad0eb4b2c3
2023-08-16 17:47:32 -07:00
Kevin Newton
a793260c15 [ruby/yarp] Also rework regexp lexer to check terminators properly
4b157a8352
2023-08-16 17:47:32 -07:00
Kevin Newton
2ef54d3855 [ruby/yarp] Rework the list lexer to check terminators properly
60315d0f13
2023-08-16 17:47:32 -07:00
Thomas E. Enebo
7257145320 [ruby/yarp] Clang beats gcc in pedantry
Accidentally declared int32_t and not uint32_t and it was caught
on macos.

6581520793
2023-08-16 17:47:32 -07:00
Thomas E. Enebo
1e0749e032 [ruby/yarp] Addressed review comments.
Missing any tests and in fact this is untested so still a WIP.

5411abd651
2023-08-16 17:47:32 -07:00
Kevin Newton
0004565a91 [ruby/yarp] Use yp_memchr in regexp parsing
08081dd24f
2023-08-16 17:47:32 -07:00
Kevin Newton
3f64defe13 [ruby/yarp] Make sure lexing ? does not read off the end
d694e3ebf2
2023-08-16 17:47:32 -07:00
Kevin Newton
1ea9e444ec [ruby/yarp] Fix memory leak from constant write node creation
0071bee0bb
2023-08-16 17:47:32 -07:00
Kevin Newton
e66e60cf69 [ruby/yarp] Handle invalid multiple parameters
2f83265653
2023-08-16 17:47:32 -07:00
Kevin Newton
7a1c497426 [ruby/yarp] Add constant write node
a62f2f0153
2023-08-16 17:47:32 -07:00
Kevin Newton
9c61e0c831 [ruby/yarp] Ensure token cannot begin an expression
837522217c
2023-08-16 17:47:32 -07:00
Kevin Newton
3d032cf3e8 [ruby/yarp] Simplify creation of numerics
e5f6ffa23a
2023-08-16 17:47:32 -07:00
Benoit Daloze
b6f26c2e4a [ruby/yarp] Use common fields for yp_string_t
* Otherwise it is undefined behavior to access the field of another `.as`.
* Accessing the right `.as` field according mode would be extra overhead.

7dc41ee803
2023-08-16 17:47:32 -07:00
Kevin Newton
31960a9627 [ruby/yarp] Handle missing block parameters
bc9049418d
2023-08-16 17:47:32 -07:00
Kevin Newton
eca3b93437 [ruby/yarp] **nil can be a STAR_STAR token
33b06f252c
2023-08-16 17:47:32 -07:00