Essentially, this change updates `yp_unescape_calculate_difference` to
not create syntax errors, and we rely entirely on
`yp_unescape_manipulate_string` to report syntax errors.
To do that, this PR adds another (!) parameter to `unescape`:
`yp_list_t *error_list`. When present, `unescape` reports syntax
errors (and otherwise does not).
However, an edge case that needed to be addressed is reporting syntax
errors in this case:
?\u{1234 2345}
In a string context, it's possible to have multiple codepoints by
doing something like `"\u{1234 2345}"`; however, in the character
literal context, this is a syntax error -- only a single codepoint is
allowed.
Unfortunately, when `yp_unescape_manipulate_string` is called, there's
nothing to indicate that we are in a "character literal" context and
that only a single codepoint is valid.
To make this work, this PR:
- introduces a new static utility function in yarp.c,
`yp_char_literal_node_create_and_unescape`, which is called when
we're parsing `YP_TOKEN_CHARACTER_LITERAL`
- introduces a new (unexported) function,
`yp_unescape_manipulate_char_literal` which does the same thing as
`yp_unescape_manipulate_string` but tells `unescape` that only a
single codepoint is expected
f6a65840b5
* Consistent with ClassVariableWriteNode, ConstantWriteNode, InstanceVariableWriteNode, GlobalVariableWriteNode.
* Fixes desugaring of local variable with operators.
9a66737775
* Consistent with ClassVariableWriteNode, ConstantWriteNode, InstanceVariableWriteNode, LocalVariableWriteNode.
* Fixes desugaring of global variable with operators.
fb5a53fc0b
file
(https://github.com/ruby/yarp/pull/1371)
* refactor: move EOF check into yp_unescape_calculate_difference
parser_lex is a bit more readable when we can rely on that behavior
* fix: octal and hex digits at the end of a file
Previously this resulted in invalid memory access.
* fix: unicode strings at the end of a file
Previously this resulted in invalid memory access.
* Unterminated curly-bracket unicode is a syntax error
21cf11acb5
Previously this resulted in invalid memory access as well as a
cascading failed assertion:
src/enc/yp_unicode.c:2224: yp_utf_8_codepoint: Assertion `n >= 1' failed.
Found by the fuzzer.
a34c534440
This should make it easier on the sync to determine what changed
and hopefully result in fewer merge conflicts that have to be
manually resolved.
17d82afbfc
Introduce three new inline helper functions:
- `match_line_ending`
- `match_line_ending_at`
- `match_line_ending_addr`
These functions are similar in signature to the `peek*` functions, but
return the length of the line ending being inspected (or 0 if no line
ending was found).
These functions are then used to simplify how we're detecting line
endings throughout "src/yarp.c".
Also:
- test coverage backfilled for `__END__` comments with CRLF line endings.
- error message for invalid `%` tokens updated to not include
the potential line endings.
- some small refactorings for readability along the way
a00067386d
This commit is the initial sync of all files from ruby/yarp
into ruby/ruby. Notably, it does the following:
* Sync all ruby/yarp/lib/ files to ruby/ruby/lib/yarp
* Sync all ruby/yarp/src/ files to ruby/ruby/yarp/
* Sync all ruby/yarp/test/ files to ruby/ruby/test/yarp
