Commit graph

2272 commits

Author SHA1 Message Date
yui-knk
cfdbbd6726 kw_rest_arg nd_cflag has not been used since 9720136 2023-08-26 08:45:02 +09:00
Nobuyoshi Nakada
40efbc7e40 [Bug #19851] Ripper: Hide internal block argument ID 2023-08-26 02:08:53 +09:00
Nobuyoshi Nakada
fe73f9f24b
Replace only use of snprintf in parser 2023-08-25 23:34:02 +09:00
Nobuyoshi Nakada
1f76e42b85 [Bug #19848] Flush BOM
The token just after BOM needs to position at column 0, so that the
indentation matches closing line.
2023-08-25 20:07:10 +09:00
Nobuyoshi Nakada
6aa16f9ec1 Move SCRIPT_LINES__ away from parse.y 2023-08-25 18:23:05 +09:00
卜部昌平
26cb3b5617 config.h has to be the very beginning
This header defines several essential macros.
2023-08-25 17:27:53 +09:00
yui-knk
ce79887da0 The first arg of NEW_KW_ARG macro is always 0 2023-08-24 07:47:52 +09:00
Nobuyoshi Nakada
0d7e847153
Consider the special node when printing
Appreciation to the reporter, Huichiao Tsai <hctsai.cs10@nycu.edu.tw>.
2023-08-16 23:52:04 +09:00
Peter Zhu
0b8f15575a Fix memory leak for incomplete lambdas
[Bug #19836]

The parser does not free the chain of `struct vtable`, which causes
memory leaks.

The following script reproduces this issue:

```
10.times do
  100_000.times do
    Ripper.parse("-> {")
  end

  puts `ps -o rss= -p #{$$}`
end
```
2023-08-09 14:06:58 -04:00
Peter Zhu
5bc8fceca8 Fix memory leak in parser for incomplete tokens
[Bug #19835]

The parser does not free the `tbl` of the `struct vtable` when there are
leftover `lvtbl` in the parser. This causes a memory leak.

The following script reproduces this issue:

```
10.times do
  100_000.times do
    Ripper.parse("class Foo")
  end

  puts `ps -o rss= -p #{$$}`
end
```
2023-08-09 14:06:58 -04:00
Nobuyoshi Nakada
382678d411 [Bug #19788] Use the result of tCOLON2 event 2023-08-01 19:00:31 +09:00
Koichi Sasada
6a5c548218 remove strange line event
```ruby
  def helper_cant_rescue
    begin
      raise SyntaxError
    rescue
      cant_rescue # here
    end
  end
```

on this case, a line event is reported on `cant_rescue` line
because of node structure. it should not be reported.
2023-08-01 18:06:25 +09:00
Nobuyoshi Nakada
26aef1c736 Use lex_eol_p family 2023-07-26 11:39:29 +09:00
S-H-GAMELINKS
76ea8ecbf3 Supress warning that variable may be used uninitialized with ripper building 2023-07-20 21:55:44 +09:00
yui-knk
82cd70ef93 Use functions defined by parser_st.c to reduce dependency on st.c 2023-07-15 12:50:40 +09:00
S-H-GAMELINKS
acd9c208d5 Move some macro for universal parser 2023-07-09 15:00:52 +09:00
S-H-GAMELINKS
8b2a0ec8df Move ISASCII defination to parse.y 2023-07-08 15:26:55 +09:00
Nobuyoshi Nakada
8ddfc17720 Use uint_least32_t
The elements of `ruby_global_name_punct_bits` table are 32-bit masks.
2023-07-04 21:30:44 +09:00
S-H-GAMELINKS
3fd1968d6f Introduce script_lines function for refactor script_lines_defined and script_lines_get functions 2023-07-01 23:17:57 +09:00
Jeremy Evans
1bc8838d60
Handle unterminated unicode escapes in regexps
This fixes an infinite loop possible after ec3542229b.
For \u{} escapes in regexps, skip validation in the parser, and rely on the regexp
code to handle validation. This is necessary so that invalid unicode escapes in
comments in extended regexps are allowed.

Fixes [Bug #19750]

Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
2023-06-30 19:37:53 -07:00
Peter Zhu
58386814a7 Don't check for null pointer in calls to free
According to the C99 specification section 7.20.3.2 paragraph 2:

> If ptr is a null pointer, no action occurs.

So we do not need to check that the pointer is a null pointer.
2023-06-30 09:13:31 -04:00
Nobuyoshi Nakada
1344de5621
[Bug #19743] All but EOF can be read again after push-back 2023-06-22 20:10:13 +09:00
Nobuyoshi Nakada
6be402e172
[Bug #19736] Recover after unterminated interpolation 2023-06-20 20:10:46 +09:00
yui-knk
4f79c83a6a Remove coverage_enabled from parser_params
`yyparse` never changes the value of `coverage_enabled`.
`coverage_enabled` depends on only return value of `e_option_supplied`.
Therefore `parser_params` doesn't need to have `coverage_enabled.
2023-06-18 10:10:52 +09:00
yui-knk
d444f1b1fa Specify int bitfield as signed int bitfield
sunc treats int bitfield as unsigned int.
This commit will fix build failure on sunc.

* 20230617T100003Z.fail.html.gz
* 20230617T090011Z.fail.html.gz
2023-06-17 22:02:13 +09:00
yui-knk
19c62b400d Replace parser & node compile_option from Hash to bit field
This commit reduces dependency to CRuby object.
2023-06-17 16:41:08 +09:00
Nobuyoshi Nakada
81836c6cb9
Fix duplicate symbol errors when statically linking ripper 2023-06-12 20:22:01 +09:00
yui-knk
b481b673d7 [Feature #19719] Universal Parser
Introduce Universal Parser mode for the parser.
This commit includes these changes:

* Introduce `UNIVERSAL_PARSER` macro. All of CRuby related functions
  are passed via `struct rb_parser_config_struct` when this macro is enabled.
* Add CI task with 'cppflags=-DUNIVERSAL_PARSER' for ubuntu.
2023-06-12 18:23:48 +09:00
yui-knk
5f65e8c5d5 Rename rb_node_name to the original name
98637d421d changes the name of
the function. However this function is exported as global,
then change the name to origin one for keeping compatibility.
2023-05-24 20:54:48 +09:00
yui-knk
98637d421d Move ruby_node_name to node.c and rename prefix of the function 2023-05-23 18:05:35 +09:00
Nobuyoshi Nakada
91c004885f
[Bug #19025] Numbered parameter names are always local variables 2023-05-14 22:16:15 +09:00
Nobuyoshi Nakada
bdaa491565 Add user argument to some macros used by bison 2023-05-14 15:38:48 +09:00
S-H-GAMELINKS
b632566f7e Introduce anddot_multiple_assignment_check function 2023-05-14 10:32:25 +09:00
Nobuyoshi Nakada
b15e88e0fc
[Bug #19619] Preserve numbered parameters context
Preserve numbered parameters context across method definitions
2023-05-02 17:39:18 +09:00
Nobuyoshi Nakada
b82c06a711
Handle private AREF call in compile.c 2023-04-30 23:21:59 +09:00
Takashi Kokubun
4af9bd52cb Get rid of a breakpoint left in parse.y 2023-04-10 11:22:12 -07:00
Nobuyoshi Nakada
ac8a16237c
[Bug #19563] Yield words separators per lines
So that newlines across a here-doc terminator will be separated
tokens.

Cf. https://github.com/ruby/irb/pull/558
2023-04-07 23:13:56 +09:00
Kazuki Tsujimoto
4ac8d11724
* in an array pattern should not be parsed as nil in ripper
After 6c0925ba70, it was impossible
to distinguish between the presence or absence of `*`.

    # Before the commit
    Ripper.sexp('0 in []')[1][0][2][1]  #=> [:aryptn, nil, nil, nil, nil]
    Ripper.sexp('0 in [*]')[1][0][2][1] #=> [:aryptn, nil, nil, [:var_field, nil], nil]

    # After the commit
    Ripper.sexp('0 in []')[1][0][2][1]  #=> [:aryptn, nil, nil, nil, nil]
    Ripper.sexp('0 in [*]')[1][0][2][1] #=> [:aryptn, nil, nil, nil, nil]

This commit reverts it.
2023-04-01 16:35:24 +09:00
yui-knk
3488eda41d Fix gc_verify_internal_consistency error for pattern_matching in ripper
`gc_verify_internal_consistency` reports "found internal inconsistency"
for "test_pattern_matching.rb".

http://ci.rvm.jp/results/trunk-gc-asserts@ruby-sp2-docker/4501173

Ruby's parser manages objects by two different ways.

1. For parser

* markable node holds objects
* call `RB_OBJ_WRITTEN` with `p->ast` as parent
* `mark_ast_value` marks objects

2. For ripper

* unmarkable node, NODE_RIPPER/NODE_CDECL, holds objects
* call `rb_ast_add_mark_object`. This function calls `rb_hash_aset` then
  `RB_OBJ_WRITTEN` is called with `mark_hash` as parent
* `mark_hash` marks objects

However in current pattern_matching implementation

* markable node holds objects
* call `rb_ast_add_mark_object`

This commit fix it to be #2.

This was inconsistency however always `mark_hash` is
made young by `rb_ast_add_mark_object` call then objects
are not collected.
2023-03-31 09:38:34 +09:00
Nobuyoshi Nakada
6f122965cf [Bug #19547] Add token for unescaped backslash
This token is exposed only when `RubyVM::AbstractSyntaxTree` with
`keep_tokens` option.
2023-03-30 19:47:36 +09:00
Kazuki Tsujimoto
d51529244f
[Bug #19175] p_kw without a sub pattern should be `assignable' 2023-03-26 18:57:34 +09:00
Kazuki Tsujimoto
6c0925ba70
[Bug #19175] p_rest should be `assignable'
It should also check for duplicate names.
2023-03-26 18:56:21 +09:00
Nobuyoshi Nakada
67dd52d59c
[Bug #19539] Match heredoc identifier from end of line
Not to ignore leading spaces in indented heredoc identifier.
2023-03-19 01:35:21 +09:00
Takashi Kokubun
c5e9af9c9d Expand tabs in parse.y
I used the same script as https://github.com/ruby/ruby/pull/6094 but
for a .y file.
2023-03-09 09:32:11 -08:00
Nobuyoshi Nakada
538c3b9ab7
Suppress -Wunused-but-set-variable warning 2023-02-14 19:26:41 +09:00
Nobuyoshi Nakada
7b343d9c67 Extract body rules from endless method definitions 2023-02-01 16:17:12 +09:00
yui-knk
e82cef1762 Remove not used argument from tokenize_ident
This has not been used since 5e59be3edd
2023-01-25 10:52:37 +09:00
Nobuyoshi Nakada
41fbcc5193
Fix format specifiers for pointer differences 2023-01-07 11:47:50 +09:00
Nobuyoshi Nakada
cee5beab1d [Bug #19312] Return end-of-input at __END__ 2023-01-06 13:13:07 +01:00
Nobuyoshi Nakada
3becc4a105
[Bug #19291] Rewind to the previous line
When rewinding looking ahead after newline token, also reset the last
line string, the pointers to it, and the location, not only the line
number.
2023-01-02 16:12:08 +09:00