archive/ruby
1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2025-08-23 13:04:13 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
84479 commits 30 branches 1174 tags 508 MiB
Commit graph

97 commits

Author SHA1 Message Date
HASUMI Hitoshi
40ecad0ad7
[Universal parser] Fix -Wsuggest-attribute=format warnings 
Under a configuration including `cppflags=-DUNIVERSAL_PARSER`, warnings listed below show in build time:

```
node.c:396:30: warning: initialization left-hand side might be a candidate for a format attribute [-Wsuggest-attribute=format]
  396 |     bug_report_func rb_bug = ast->node_buffer->config->bug;
      |                              ^~~
```

```
ruby_parser.c:655:21: warning: initialization left-hand side might be a candidate for a format attribute [-Wsuggest-attribute=format]
  655 |     .compile_warn = rb_compile_warn,
      |                     ^~~~~~~~~~~~~~~
ruby_parser.c:656:24: warning: initialization left-hand side might be a candidate for a format attribute [-Wsuggest-attribute=format]
  656 |     .compile_warning = rb_compile_warning,
      |                        ^~~~~~~~~~~~~~~~~~
ruby_parser.c:657:12: warning: initialization left-hand side might be a candidate for a format attribute [-Wsuggest-attribute=format]
  657 |     .bug = rb_bug,
      |            ^~~~~~
ruby_parser.c:658:14: warning: initialization left-hand side might be a candidate for a format attribute [-Wsuggest-attribute=format]
  658 |     .fatal = rb_fatal,
      |              ^~~~~~~~
```

To fix, this patch suggests adding `__attribute__((format(printf, n, m)))` to those function declarations.
 2024-03-15 22:14:57 +09:00
HASUMI Hitoshi
9a19cfd4cd [Universal Parser] Reduce dependence on RArray in parse.y 
- Introduce `rb_parser_ary_t` structure to partly eliminate RArray from parse.y
  - In this patch, `parser_params->tokens` and `parser_params->ast->node_buffer->tokens` are now `rb_parser_ary_t *`
  - Instead, `ast_node_all_tokens()` internally creates a Ruby Array object from the `rb_parser_ary_t`
  - Also, delete `rb_ast_tokens()` and `rb_ast_set_tokens()` in node.c

- Implement `rb_parser_str_escape()`
  - This is a port of the `rb_str_escape()` function in string.c
  - `rb_parser_str_escape()` does not depend on `VALUE` (RString)
  - Instead, it uses `rb_parser_stirng_t *`
  - This function works when --dump=y option passed

- Because WIP of the universal parser, similar functions like `rb_parser_tokens_free()` exist in both node.c and parse.y. Refactoring them may be needed in some way in the future

- Although we considered redesigning the structure: `ast->node_buffer->tokens` into `ast->tokens`, we leave it as it is because `rb_ast_t` is an imemo. (We will address it in the future)
 2024-03-12 17:17:52 +09:00
Peter Zhu
01f9b2ae41 Use rb_str_to_interned_str in parse.y 
This commit changes rb_fstring to rb_str_to_interned_str in parse.y.
rb_fstring is private so it shouldn't be used by ripper.
 2024-02-23 13:33:46 -05:00
yui-knk
91cb303531 Remove not used universal parser macros and functions 2024-02-21 13:36:45 +09:00
yui-knk
e7ab5d891c Introduce NODE_REGX to manage regexp literal 2024-02-21 08:06:48 +09:00
Peter Zhu
c184aa8740 Use rb_gc_mark_and_move for imemo 2024-02-20 10:39:30 -05:00
S-H-GAMELINKS
fba647087b Remove uneeded Universal Parser properties 2024-02-20 19:02:24 +09:00
yui-knk
89cfc15207 [Feature #20257] Rearchitect Ripper 
Introduce another semantic value stack for Ripper so that
Ripper can manage both Node and Ruby Object separately.
This rearchitectutre of Ripper solves these issues.
Therefore adding test cases for them.

* [Bug 10436] https://bugs.ruby-lang.org/issues/10436
* [Bug 18988] https://bugs.ruby-lang.org/issues/18988
* [Bug 20055] https://bugs.ruby-lang.org/issues/20055

Checked the differences of `Ripper.sexp` for files under `/test/ruby`
are only on test_pattern_matching.rb.
The differences comes from the differences between
`new_hash_pattern_tail` functions between parser and Ripper.
Ripper `new_hash_pattern_tail` didn’t call `assignable` then
`kw_rest_arg` wasn’t marked as local variable.
This is also fixed by this commit.

```
--- a/./tmp/before/test_pattern_matching.rb
+++ b/./tmp/after/test_pattern_matching.rb
@@ -3607,7 +3607,7 @@
                  [:in,
                   [:hshptn, nil, [], [:var_field, [:@ident, “a”, [984, 13]]]],
                   [[:binary,
-                    [:vcall, [:@ident, “a”, [985, 10]]],
+                    [:var_ref, [:@ident, “a”, [985, 10]]],
                     :==,
                     [:hash, nil]]],
                   nil]]],
@@ -3662,7 +3662,7 @@
                  [:in,
                   [:hshptn, nil, [], [:var_field, [:@ident, “a”, [993, 13]]]],
                   [[:binary,
-                    [:vcall, [:@ident, “a”, [994, 10]]],
+                    [:var_ref, [:@ident, “a”, [994, 10]]],
                     :==,
                     [:hash,
                      [:assoclist_from_args,
@@ -3813,7 +3813,7 @@
                    [:command,
                     [:@ident, “raise”, [1022, 10]],
                     [:args_add_block,
-                     [[:vcall, [:@ident, “b”, [1022, 16]]]],
+                     [[:var_ref, [:@ident, “b”, [1022, 16]]]],
                      false]]],
                   [:else, [[:var_ref, [:@kw, “true”, [1024, 10]]]]]]]],
                nil,
@@ -3876,7 +3876,7 @@
                      [:@int, “0”, [1033, 15]]],
                     :“&&“,
                     [:binary,
-                     [:vcall, [:@ident, “b”, [1033, 20]]],
+                     [:var_ref, [:@ident, “b”, [1033, 20]]],
                      :==,
                      [:hash, nil]]]],
                   nil]]],
@@ -3946,7 +3946,7 @@
                      [:@int, “0”, [1042, 15]]],
                     :“&&“,
                     [:binary,
-                     [:vcall, [:@ident, “b”, [1042, 20]]],
+                     [:var_ref, [:@ident, “b”, [1042, 20]]],
                      :==,
                      [:hash,
                       [:assoclist_from_args,
@@ -5206,7 +5206,7 @@
                      [[:assoc_new,
                        [:@label, “c:“, [1352, 22]],
                        [:@int, “0”, [1352, 25]]]]]],
-                   [:vcall, [:@ident, “r”, [1352, 29]]]],
+                   [:var_ref, [:@ident, “r”, [1352, 29]]]],
                   false]]],
                [:binary,
                 [:call,
@@ -5299,7 +5299,7 @@
                       [:assoc_new,
                        [:@label, “c:“, [1367, 34]],
                        [:@int, “0”, [1367, 37]]]]]],
-                   [:vcall, [:@ident, “r”, [1367, 41]]]],
+                   [:var_ref, [:@ident, “r”, [1367, 41]]]],
                   false]]],
                [:binary,
                 [:call,
@@ -5931,7 +5931,7 @@
              [:in,
               [:hshptn, nil, [], [:var_field, [:@ident, “r”, [1533, 11]]]],
               [[:binary,
-                [:vcall, [:@ident, “r”, [1534, 8]]],
+                [:var_ref, [:@ident, “r”, [1534, 8]]],
                 :==,
                 [:hash,
                  [:assoclist_from_args,
```
 2024-02-20 17:33:58 +09:00
Nobuyoshi Nakada
b1d70e4264
[Bug #20280] Check by rb_parser_enc_str_coderange 
Co-authored-by: Yuichiro Kaneko <spiketeika@gmail.com>
 2024-02-19 16:33:26 +09:00
Nobuyoshi Nakada
fcc55dc226
[Bug #20280] Raise SyntaxError on invalid encoding symbol 2024-02-19 16:33:26 +09:00
Peter Zhu
a71d1ed838 Fix memory leak when parsing invalid hash symbol 
For example:

    10.times do
      100_000.times do
        eval('{"\xC3": 1}')
      rescue EncodingError
      end

      puts `ps -o rss= -p #{$$}`
    end

Before:

    32032
    48464
    66112
    84192
    100592
    117520
    134096
    150656
    167168
    183760

After:

    17120
    17120
    17120
    17120
    18560
    18560
    18560
    18560
    18560
    18560
 2024-02-13 11:05:56 -05:00
yui-knk
33c1e082d0 Remove ruby object from string nodes 
String nodes holds ruby string object on `VALUE nd_lit`.
This commit changes it to `struct rb_parser_string *string`
to reduce dependency on ruby object.
Sometimes these strings are concatenated with other string
therefore string concatenate functions are needed.
 2024-02-09 14:20:17 +09:00
yui-knk
68b57ceb46 Use bool to check ascii only in parse_ident 
No need to use ENC_CODERANGE to record ascii only or not.
 2024-02-03 09:15:41 +09:00
S.H
f3df218f48
Introduced rb_node_const_decl_val function 
Introduce `rb_node_const_decl_val` function to allow `rb_ary_join` and
`rb_ary_reverse` functions to be removed from Universal Parser.
 2024-01-31 13:31:38 +09:00
Nobuyoshi Nakada
e018036d89
Rename nd_head in RNode_RESBODY as nd_next 2024-01-28 11:12:22 +09:00
Nobuyoshi Nakada
0f98d284f3
Remove unused nd_resq from RNode_ENSURE 2024-01-28 11:11:13 +09:00
S.H
9b40f42c22
Introduce NODE_ENCODING 
`__ENCODING__ `was managed by `NODE_LIT` with Encoding object. 

Introduce `NODE_ENCODING` for
1. `__ENCODING__` is detectable from AST Node.
2. Reduce dependency Ruby object for parse.y
 2024-01-27 08:11:10 +00:00
Nobuyoshi Nakada
0610f555ea
Constify rb_global_parser_config 2024-01-14 17:55:11 +09:00
yui-knk
b35e21b388 Remove reference counter from rb_parser_config 
It's allocated outside of parser then no need to track
reference count in rb_parser_config.
 2024-01-12 21:17:41 +09:00
yui-knk
52d9e55903 Statically allocate parser config 2024-01-12 21:17:41 +09:00
yui-knk
db476cc71c Introduce NODE_SYM to manage symbol literal 
`:sym` was managed by `NODE_LIT` with `Symbol` object.
This commit introduces `NODE_SYM` so that

1. Symbol literal is detectable from AST Node
2. Reduce dependency on ruby object
 2024-01-09 16:07:19 +09:00
S-H-GAMELINKS
ad7aee35e4 Remove unneeded rb_parser_config_struct struct properties for Universal Parser 2024-01-07 21:16:31 +09:00
S-H-GAMELINKS
1b8d01136c Introduce Numeric Node's 2024-01-07 09:24:34 +09:00
yui-knk
7a050638b1 Introduce NODE_FILE 
`__FILE__` was managed by `NODE_STR` with `String` object.
This commit introduces `NODE_FILE` and `struct rb_parser_string` so that

1. `__FILE__` is detectable from AST Node
2. Reduce dependency ruby object
 2024-01-02 14:19:42 +09:00
yui-knk
1ade170a6c Introduce NODE_LINE 
`__LINE__` was managed by `NODE_LIT` with `Integer` object.
This commit introduces `NODE_LINE` so that

1. `__LINE__` is detectable from AST Node
2. Reduce dependency ruby object
 2023-12-29 18:32:27 +09:00
yui-knk
4374236e95 Add errno_ptr property for Universal Parser 2023-12-28 13:17:36 +09:00
yui-knk
73fa322497 Add ary_modify property for Universal Parser 2023-12-28 09:00:44 +09:00
Nobuyoshi Nakada
e6a6ea9dcf
Fix typo in a comment [ci skip] 2023-12-08 01:20:15 +09:00
Nobuyoshi Nakada
13c9cbe09e
Embed rb_args_info in rb_node_args_t 2023-10-30 00:19:43 +09:00
yui-knk
820957b1ee Remove unused macro 
`struct RNode_OP_ASGN22` was removed by 37a783a.
 2023-10-25 08:12:48 +09:00
yui-knk
08e25985d1 Expand OP_ASGN1 nd_args to nd_index and nd_rvalue 
ARGSCAT has been used for nd_args to hold index and rvalue,
because there was limitation on the number of members for Node.
We can easily change structure of node now, let's expand it.
 2023-10-20 07:56:20 +09:00
Nobuyoshi Nakada
5bbb6fd6c3 Add printf format attributes to rb_parser_config_t 2023-10-20 07:15:24 +09:00
yui-knk
606452d1a9 Remove not used fields from STR 2023-10-15 16:16:06 +09:00
yui-knk
a4e3d595cd Remove not used fields from XSTR 2023-10-15 16:16:06 +09:00
yui-knk
cab67d227a Remove not used fields from LIT 2023-10-15 16:16:06 +09:00
yui-knk
dc8742f5cc Remove not used fields from MATCH 2023-10-15 16:16:06 +09:00
Nobuyoshi Nakada
a405b28e85 Delete heredoc line mark references 2023-10-14 11:08:43 +09:00
Nobuyoshi Nakada
a075c55d0c Manage rb_strterm_t without imemo 2023-10-14 11:08:43 +09:00
yui-knk
f9fe7aeef4 Extract NODE_FL_NEWLINE access to macro 2023-10-11 19:22:34 +09:00
yui-knk
5245123a4b Remove not used fields from DEFN 2023-10-10 11:05:29 +09:00
yui-knk
b5d74f4f26 Remove not used fields from EVSTR 2023-10-08 16:34:32 +09:00
yui-knk
5810304c2e Remove not used fields from asgn nodes 2023-10-07 17:54:35 +09:00
yui-knk
5c779dc45d Remove not used fields from MATCH3 2023-10-06 15:32:33 +09:00
yui-knk
58fc45325f Remove not used fields from YIELD 2023-10-06 07:31:53 +09:00
yui-knk
f6a2af255b Remove not used fields from ZLIST 2023-10-06 07:31:22 +09:00
Nobuyoshi Nakada
a5cc6341c0
Remove NODE_VALUES 
This node type was added for the multi-value experiment back in 2004.
The feature itself was removed after a few years, but this is its
remnant.
 2023-10-06 03:39:58 +09:00
Nobuyoshi Nakada
efa18fd6b3 Chain nodes to exit only 2023-10-05 14:23:42 +09:00
Nobuyoshi Nakada
696022a0cb Differentiate NODE_BREAK/NODE_NEXT/NODE_RETURN 2023-10-05 14:23:42 +09:00
Nobuyoshi Nakada
f5f3b35b93 Remove unused nodes in NODE_RETURN and NODE_REDO 2023-10-05 14:23:42 +09:00
Nobuyoshi Nakada
70e1635950 Move internal NODE_DEF_TEMP to parse.y 2023-10-05 14:23:42 +09:00