Commit graph

2207 commits

Author SHA1 Message Date
Nobuyoshi Nakada
230267d1a8 Now bison 3.0 or later is required 2022-11-09 21:34:02 +09:00
yui-knk
f7db1affd1 Set default %printer for NODE nterms
Before:

```
Reducing stack by rule 639 (line 5062):
   $1 = token "integer literal" (1.0-1.1: 1)
-> $$ = nterm simple_numeric (1.0-1.1: )
```

After:

```
Reducing stack by rule 641 (line 5078):
   $1 = token "integer literal" (1.0-1.1: 1)
-> $$ = nterm simple_numeric (1.0-1.1: NODE_LIT)
```

`"<*>"` is supported by Bison 2.3b (2008-05-27) or later.
https://git.savannah.gnu.org/cgit/bison.git/commit/?id=12e3584054c16ab255672c07af0ffc7bb220e8bc

Therefore developers need to install Bison 2.3b+ to build ruby from
source codes if their Bison is older.

Minimum version requirement for Bison is changed to 3.0.

See: https://bugs.ruby-lang.org/issues/19068 [Feature #19068]
2022-11-08 12:30:03 +09:00
Nobuyoshi Nakada
546566d34b
Do not set $! to SyntaxError when error tolerant 2022-10-09 19:07:21 +09:00
yui-knk
8483737bbf Fix typos 2022-10-08 23:29:36 +09:00
yui-knk
50f5223236 Fix SEGV of dump parsetree
Assign internal_id to semantic value so that dump parsetree option
can render the tree for these codes without SEGV.

* `def m(&); end`
* `def m(*); end`
* `def m(**); end`
2022-10-08 22:30:50 +09:00
yui-knk
3531086095 "expr_value" can be error
So that "IF" node is kept in the case below

```
def m
  if
end
```

[Feature #19013]
2022-10-08 17:59:11 +09:00
yui-knk
4bfdf6d06d Move error from top_stmts and top_stmt to stmt
By this change, syntax error is recovered smaller units.
In the case below, "DEFN :bar" is same level with "CLASS :Foo"
now.

```
module Z
  class Foo
    foo.
  end

  def bar
  end
end
```

[Feature #19013]
2022-10-08 17:59:11 +09:00
yui-knk
4f24f3ea94 Treat "end" as reserved word with consideration of indent
"end" after "." or "::" is treated as local variable or method,
see `EXPR_DOT_bit` for detail.
However this "changes" where `bar` method is defined. In the example
below it is not module Z but class Foo.

```
module Z
  class Foo
    foo.
  end

  def bar
  end
end
```

[Feature #19013]
2022-10-08 17:59:11 +09:00
yui-knk
342d4c16d9 Generates "end" tokens if parser hits end of input
but "end" tokens are needed for correct language.

[Feature #19013]
2022-10-08 17:59:11 +09:00
yui-knk
fbbdbdd891 Add error_tolerant option to RubyVM::AST
If this option is enabled, SyntaxError is not raised and Node is
returned even if passed script is broken.

[Feature #19013]
2022-10-08 17:59:11 +09:00
Shugo Maeda
a8ad22d926
Suppress a warning on clang
The following warning appears without this fix:

```
parse.y:78:1: warning: unknown warning group '-Wpsabi', ignored
      [-Wunknown-warning-option]
RBIMPL_WARNING_IGNORED(-Wpsabi)
^
./include/ruby/internal/warning_push.h:103:39: note: expanded from macro
      'RBIMPL_WARNING_IGNORED'
                                      ^
./include/ruby/internal/warning_push.h:99:39: note: expanded from macro
      'RBIMPL_WARNING_PRAGMA2'
                                      ^
./include/ruby/internal/warning_push.h:98:39: note: expanded from macro
      'RBIMPL_WARNING_PRAGMA1'
                                      ^
./include/ruby/internal/warning_push.h:97:39: note: expanded from macro
      'RBIMPL_WARNING_PRAGMA0'
                                      ^
<scratch space>:49:27: note: expanded from here
 clang diagnostic ignored "-Wpsabi"
                          ^
1 warning generated.
```
2022-09-26 14:44:54 +09:00
S.H
960db13c47
Reuse opt_arg_append function 2022-09-14 23:10:21 +09:00
Kazuki Tsujimoto
db0e0dad11
Fix unexpected "duplicated key name" error in paren-less one line pattern matching
[Bug #18990]
2022-09-09 14:00:27 +09:00
Nobuyoshi Nakada
ace2eee544
[Bug #18963] Separate string contents by here document terminator 2022-08-28 09:29:24 +09:00
S.H
13d31331c8
Reuse nonlocal_var patterns 2022-08-22 18:52:36 +09:00
S-H-GAMELINKS
3541f32951 Reuse opt_nl rule 2022-08-19 09:51:06 +09:00
S-H-GAMELINKS
f095361758 Repalce to NIL_P macro 2022-08-19 09:47:43 +09:00
Nobuyoshi Nakada
844a0edbae [Bug #18962] Do not read again once reached EOF
`Ripper::Lexer#parse` re-parses the source code with syntax errors
when `raise_errors: false`.

Co-Authored-By: tompng <tomoyapenguin@gmail.com>
2022-08-12 15:58:18 +09:00
Kevin Backhouse
8c1808151f
Fix some UBSAN false positives (#6115)
* Fix some UBSAN false positives.
* ruby tool/update-deps --fix
2022-07-12 11:48:10 -07:00
Nobuyoshi Nakada
eaeb130b11 [Bug #18890] newline should be insignificant after pattern label 2022-07-06 08:32:36 +09:00
Nobuyoshi Nakada
982cda9a3e [Bug #18877] Let lex_ctxt not to eat escaped whitespace 2022-06-30 16:31:51 +09:00
Nobuyoshi Nakada
685efac059
[Bug #18884] class cannot be just followed by modifiers 2022-06-29 14:13:15 +09:00
Nobuyoshi Nakada
961543945f
Suppress notes for old gcc 2022-06-23 22:52:45 +09:00
S-H-GAMELINKS
420f3ced4d Using is_ascii_string to check encoding 2022-06-17 12:02:50 +09:00
Nobuyoshi Nakada
cd5cafa4a3 Respect the encoding of the source [Bug #18827]
Do not override the input string encoding at the time of preparation,
the source encoding is not determined from the input yet.
2022-06-17 01:48:52 +09:00
Nobuyoshi Nakada
bb12aa4d15
Remove unnecessary condition
`no_blockarg` is called for non-null `node` only.
2022-06-15 22:17:23 +09:00
Nobuyoshi Nakada
1a70973f75 ripper: Check if anonymous parameters defined [Bug #18828] 2022-06-14 20:41:41 +09:00
Jeremy Evans
ec3542229b
Ignore invalid escapes in regexp comments
Invalid escapes are handled at multiple levels.  The first level
is in parse.y, so skip invalid unicode escape checks for regexps
in parse.y.

Make rb_reg_preprocess and unescape_nonascii accept the regexp
options.  In unescape_nonascii, if the regexp is an extended
regexp, when "#" is encountered, ignore all characters until the
end of line or end of regexp.

Unfortunately, in extended regexps, you can use "#" as a non-comment
character inside a character class, so also parse "[" and "]"
specially for extended regexps, and only skip comments if "#" is
not inside a character class. Handle nested character classes as well.

This issue doesn't just affect extended regexps, it also affects
"(#?" comments inside all regexps.  So for those comments, scan
until trailing ")" and ignore content inside.

I'm not sure if there are other corner cases not handled.  A
better fix would be to redesign the regexp parser so that it
unescaped during parsing instead of before parsing, so you already
know the current parsing state.

Fixes [Bug #18294]

Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
2022-06-06 13:50:03 -07:00
Nobuyoshi Nakada
3ddf6ad4d2
Private local variables should shadow outer variables [Bug #18629] 2022-04-21 16:04:36 +09:00
Nobuyoshi Nakada
08b6aacc1a
Fix the wrong index of the previous component [Bug #18739] 2022-04-18 18:37:09 +09:00
Jeremy Evans
3bb70a6924 Fix using anonymous block in method accepting explicit keywords
Record block ID before vtable_pop, so the incorrect one doesn't
override it.

Fixes [Bug #18673]
2022-04-05 07:35:25 -07:00
Kazuki Tsujimoto
db6b23c76c
Find pattern is no longer experimental [Feature #18585] 2022-02-19 18:45:49 +09:00
Kazuki Tsujimoto
3200d97e95
Fix location of NODE_LIT in p_kw 2022-02-19 18:45:36 +09:00
S-H-GAMELINKS
e6b537e605 Reuse operation rule for operation2 2022-02-12 12:28:07 +09:00
S-H-GAMELINKS
c0651b4ae1 Reuse p_kwnorest rule for f_no_kwarg 2022-02-12 12:27:49 +09:00
Vladimir Dementyev
b633c9ac1c
Reduce p_args rules with p_rest 2022-01-11 19:58:57 +09:00
Nobuyoshi Nakada
6baa78bb78 Use chomp: option when chomp mode
Get rid of depending on using $/ internally in String#chomp!, and
chomp the separator at once.
2022-01-07 09:23:38 +09:00
Nobuyoshi Nakada
83b987054a Explicitly pass $/ when loop mode
Get rid of depending on using $/ internally in ARGF.gets.
2022-01-07 09:23:38 +09:00
Jeremy Evans
f53dfab95c Add support for anonymous rest and keyword rest argument forwarding
This allows for the following syntax:

```ruby
def foo(*)
  bar(*)
end
def baz(**)
  quux(**)
end
```

This is a natural addition after the introduction of anonymous
block forwarding.  Anonymous rest and keyword rest arguments were
already supported in method parameters, this just allows them to
be used as arguments to other methods.  The same advantages of
anonymous block forwarding apply to rest and keyword rest argument
forwarding.

This has some minor changes to #parameters output.  Now, instead
of `[:rest], [:keyrest]`, you get `[:rest, :*], [:keyrest, :**]`.
These were already used for `...` forwarding, so I think it makes
it more consistent to include them in other cases.  If we want to
use `[:rest], [:keyrest]` in both cases, that is also possible.

I don't think the previous behavior of `[:rest], [:keyrest]` in
the non-... case and `[:rest, :*], [:keyrest, :**]` in the ...
case makes sense, but if we did want that behavior, we'll have to
make more substantial changes, such as using a different ID in the
... forwarding case.

Implements [Feature #18351]
2021-12-30 14:37:42 -08:00
S.H
9b187fec58
Add rb_parser_set_pos function
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Co-authored-by: Marivaldo Cavalheiro <marivaldo@gmail.com>
2021-12-16 20:03:44 +09:00
Nobuyoshi Nakada
aa7c4c37d7 Fix arg_forward without parentheses [Bug #18267] 2021-12-15 07:01:20 +09:00
Nobuyoshi Nakada
3a6b79d0c0 Revert "Rename in_kwarg as in_argdef as unrelated to keywords"
This reverts commit b7f7117bdc, to
separate `in_argdef` from `in_kwarg`.
2021-12-15 07:01:20 +09:00
Nobuyoshi Nakada
637c3cfc5d Turn SET_LEX_STATE macro into an inline function call 2021-12-15 01:52:15 +09:00
Nobuyoshi Nakada
a3934cd1e5
Fix indent [ci skip] 2021-12-15 00:50:39 +09:00
Nobuyoshi Nakada
54f0e63a8c Remove NODE_DASGN_CURR [Feature #18406]
This `NODE` type was used in pre-YARV implementation, to improve
the performance of assignment to dynamic local variable defined at
the innermost scope.  It has no longer any actual difference with
`NODE_DASGN`, except for the node dump.
2021-12-13 12:53:03 +09:00
S.H
ec7f14d9fa
Add nd_type_p macro 2021-12-04 00:01:24 +09:00
Nobuyoshi Nakada
c14f230b26 Assign temporary ID to anonymous ID [Bug #18250]
Dumped iseq binary can not have unnamed symbols/IDs, and ID 0 is
stored instead.  As `struct rb_id_table` disallows ID 0, also for
the distinction, re-assign a new temporary ID based on the local
variable table index when loading from the binary, as well as the
parser.
2021-11-23 21:03:19 +09:00
Yusuke Endoh
feda058531 Refactor hacky ID tables to struct rb_ast_id_table_t
The implementation of a local variable tables was represented as `ID*`,
but it was very hacky: the first element is not an ID but the size of
the table, and, the last element is (sometimes) a link to the next local
table only when the id tables are a linked list.

This change converts the hacky implementation to a normal struct.
2021-11-21 08:59:24 +09:00
Jeremy Evans
4adb012926 Anonymous block forwarding allows a method to forward a passed
block to another method without having to provide a name for the
block parameter.

Implements [Feature #11256]

Co-authored-by: Yusuke Endoh mame@ruby-lang.org
Co-authored-by: Nobuyoshi Nakada nobu@ruby-lang.org
2021-11-18 14:17:57 -08:00
Yusuke Endoh
415671a282 parse.y: Fix memory leak at parse error
Local variable tables might leak at the parse error.
2021-11-12 17:36:39 +09:00