The purpose of this commit is to fix Bug #21188. We need to detect when
stdin has run in to an EOF case. Unfortunately we can't _call_ the eof
function on IO because it will block.
Here is a short script to demonstrate the issue:
```ruby
x = STDIN.gets
puts x
puts x.eof?
```
If you run the script, then type some characters (but _NOT_ a newline),
then hit Ctrl-D twice, it will print the input string. Unfortunately,
calling `eof?` will try to read from STDIN again causing us to need a
3rd Ctrl-D to exit the program.
Before introducing the EOF callback to Prism, the input loop looked
kind of like this:
```ruby
loop do
str = STDIN.gets
process(str)
if str.nil?
p :DONE
end
end
```
Which required 3 Ctrl-D to exit. If we naively changed it to something
like this:
```ruby
loop do
str = STDIN.gets
process(str)
if STDIN.eof?
p :DONE
end
end
```
It would still require 3 Ctrl-D because `eof?` would block. In this
patch, we're wrapping the IO object, checking the buffer for a newline
and length, and then using that to simulate a non-blocking eof? method.
This commit wraps STDIN and emulates a non-blocking `eof` function.
[Bug #21188]
Previously, endless method definitions in assignment contexts like
`x = def f = p 1` would fail to parse because command calls (method
calls without parentheses) were only accepted when the surrounding
binding power was less than `PM_BINDING_POWER_COMPOSITION`.
This fix specifically checks for assignment context and allows command
calls in those cases while maintaining the existing behavior for other
contexts. This ensures that:
- `x = def f = p 1` parses correctly (previously failed)
- `private def f = puts "Hello"` still produces the expected error
722af59ba3
This makes it hard to do version checks against this value. The current version checks work because there are so few possible values at the moment.
As an example, PR 3337 introduces new syntax for ruby 3.5 and uses `PM_OPTIONS_VERSION_LATEST` as its version guard. Because what is considered the latest changes every year, it must later be changed to `parser->version == parser->version == PM_OPTIONS_VERSION_CRUBY_3_5 || parser->version == PM_OPTIONS_VERSION_LATEST`, with one extra version each year.
With this change, the PR can instead write `parser->version >= PM_OPTIONS_VERSION_CRUBY_3_5` which is self-explanatory
and works for future versions.
8318a113ca
Strings concatenated with backslash may end up being frozen when they
shouldn't be. This commit fixes the issue. It required a change
upstream in Prism, but also a change to the Prism compiler in CRuby.
https://github.com/ruby/prism/pull/3606
[Bug #21187]
This reverts commit a495e6a44c.
This break extension builds:
```
/Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:321:in 'String#replace': can't modify frozen String: "$(SDKROOT)$(prefix)/include" (FrozenError)
from /Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:321:in 'RbConfig.expand'
from /Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:314:in 'block in RbConfig.expand'
from /Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:307:in 'String#gsub'
from /Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:307:in 'RbConfig.expand'
from /Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:314:in 'block in RbConfig.expand'
from /Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:307:in 'String#gsub'
from /Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:307:in 'RbConfig.expand'
from /Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:314:in 'block in RbConfig.expand'
from /Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:307:in 'String#gsub'
from /Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:307:in 'RbConfig.expand'
from /Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:325:in 'block in <module:RbConfig>'
from /Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:324:in 'Hash#each_value'
from /Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:324:in '<module:RbConfig>'
from /Users/hsbt/Documents/github.com/ruby/ruby/rbconfig.rb:11:in '<top (required)>'
from ./ext/extmk.rb:42:in 'Kernel#require'
from ./ext/extmk.rb:42:in '<main>'
make[1]: *** [ext/configure-ext.mk:70: ext/json/exts.mk] Error 1
```
When inner strings aren't frozen, we need to clear the flags on
interpolated string nodes so that we don't emit wrong instructions.
The compiler is currently incorrectly emitting frozen strings because
the parser is erroneously declaring interpolated strings as "frozen".
We need to fix this behavior in the parser so we can fix the compiler in
CRuby. This patch is a partial fix for [this bug](https://bugs.ruby-lang.org/issues/21187)
eda693f056
Previously, endless method definitions like `x = def f = p 1` would fail
to parse because command calls (method calls without parentheses) were
only accepted when the surrounding binding power was less than
`PM_BINDING_POWER_COMPOSITION` (8). In assignment contexts with binding
power 18, this condition was false, causing parse errors.
This fix ensures command calls are always accepted in endless method
bodies by passing `true` for `accepts_command_call`, making the method
body parse consistently regardless of where the method is defined.
70413ed4dd
When arithmetic expressions like `-1**2` are used in pattern matching contexts,
Ruby crashes with "Unexpected node type in pattern matching expression: PM_CALL_NODE".
This happens because the Prism parser creates `PM_CALL_NODE` for arithmetic operations,
but Ruby's pattern matching compiler doesn't handle call nodes.
This fix adds validation to reject `PM_CALL_NODE` in pattern contexts with a proper
syntax error.
365049a767
This can get triggered even if the list of statements only contains
a single statement. This is necessary to properly support compiling
```ruby
defined? (;a)
defined? (a;)
```
as "expression". Previously these were parsed as statements lists
with single statements in them.
b63b5d67a9
Expressions joined with `&&` are better asserted separately, so that
it is clear from the failure message which expression is false.
Also, `unsigned long` may not be enough for `ptrdiff_t`, e.g., Windows.
Saying that `ptrdiff_t` can be larger than `SIZE_MAX` means that
`PTRDIFF_MAX` is larger than `SIZE_MAX` and `ptrdiff_t` is sufficient
to represent `SIZE_MAX`, otherwise if `PTRDIFF_MAX` is smaller than
`SIZE_MAX`, `diff` will always be smaller than `SIZE_MAX` as well.
`diff` could be equal to `SIZE_MAX` only if `PTRDIFF_MAX` is equal to
`SIZE_MAX` and these assertions would pass, but I don't think there is
any platform where that is the case.
1fc6dfcada
When parent scopes around an eval are forwarding parameters (like
*, **, &, or ...) we need to know that information when we are in
the parser. As such, we need to support passing that information
into the scopes option. In order to do this, unfortunately we need
a bunch of changes.
The scopes option was previously an array of array of strings.
These corresponded to the names of the locals in the parent scopes.
We still support this, but now additionally support passing in a
Prism::Scope instance at each index in the array. This Prism::Scope
class holds both the names of the locals as well as an array of
forwarding parameter names (symbols corresponding to the forwarding
parameters). There is convenience function on the Prism module that
creates a Prism::Scope object using Prism.scope.
In JavaScript, we now additionally support an object much the same
as the Ruby side. In Java, we now have a ParsingOptions.Scope class
that holds that information. In the dump APIs, these objects in all
3 languages will add an additional byte for the forwarding flags in
the middle of the scopes serialization.
All of this is in service of properly parsing the following code:
```ruby
def foo(*) = eval("bar(*)")
```
21abb6b7c4
Prism shoudld throw a syntax error for endless methods when the method
name uses brackets. Previously it would not. This matches the behavior
of parse.y.
Fixes https://bugs.ruby-lang.org/issues/2101043c16a89ef
Prism was already disallowing arguments after block args, but in
parse.y, any comma after a block arg is a syntax error. This moves the
error handling into `PM_TOKEN_UAMPERSAND` where we can check if the
current type is `PM_TOKEN_COMMA`then raise an error. I've also updated
the tests to include the examplesfrom ruby/prism#3112.
Fixes: ruby/prism#3112754cf8eddc
Raise an exception when the same numbered param is used inside a child
block. For example, the following code should be a syntax error:
```ruby
-> { _1 + -> { _1 } }
```
Fixes https://github.com/ruby/prism/pull/3291d4fc441838
parse.y treats CRLF as a LF and basically "normalizes" them before
parsing. That means a string like `%\nfoo\r\n` is actually treated as
`%\nfoo\n` for the purposes of parsing. This happens on both the
opening side of the percent string as well as on the closing side. So
for example `%\r\nfoo\n` must be treated as `%\nfoo\n`.
To handle this in Prism, when we start a % string, we check if it starts
with `\r\n`, and then consider the terminator to actually be `\n`. Then
we check if there are `\r\n` as we lex the string and treat those as
`\n`, but only in the case the start was a `\n`.
Fixes: #3230
[Bug #20938]
e573ceaad6
Co-authored-by: John Hawthorn <jhawthorn@github.com>
Co-authored-by: eileencodes <eileencodes@gmail.com>
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
