* parse.y (kwd_append): Update the last location of
NODE_KW_ARG when NODE is appended to the last.
e.g. The locations of the first NODE_KW_ARG is fixed:
```
def a(b: 1, c: 2); end
```
* Before
```
NODE_KW_ARG (line: 1, first_lineno: 1, first_column: 6, last_lineno: 1, last_column: 10)
```
* After
```
NODE_KW_ARG (line: 1, first_lineno: 1, first_column: 6, last_lineno: 1, last_column: 16)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (opt_arg_append): Update the last location of
NODE_OPT_ARG when NODE is appended to the last.
e.g. The locations of the first NODE_OPT_ARG is fixed:
```
def a(b = 1, c = 2); end
```
* Before
```
NODE_OPT_ARG (line: 1, first_lineno: 1, first_column: 6, last_lineno: 1, last_column: 11)
```
* After
```
NODE_OPT_ARG (line: 1, first_lineno: 1, first_column: 6, last_lineno: 1, last_column: 18)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_op_assign_gen): Update the location of
lhs when NODE_OP_ASGN_OR/NODE_OP_ASGN_AND are generated.
When NODE_OP_ASGN_OR/NODE_OP_ASGN_AND are generated
a nd_value of lhs is set, so it is needed to update
a location of lhs to include a location of rhs (same as
node_assign_gen).
e.g. The locations of NODE_DASGN_CURR is fixed:
```
a ||= 1
```
* Before
```
NODE_DASGN_CURR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 1)
```
* After
```
NODE_DASGN_CURR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 7)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60903 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (words, symbols, qwords, qsymbols): unify empty list and
non-empty list.
* parse.y (parser_parse_string): always dispatch a word separator
at the beginning of list literals.
[ruby-core:83871] [Bug #14126]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60892 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (logop_gen): Update the last location of
NODE_AND/NODE_OR when NODE is appended to the last.
e.g. The locations of NODE_AND is fixed:
```
a && b && c
```
* Before
```
NODE_AND (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 6)
```
* After
```
NODE_AND (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 11)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60887 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_skip_words_sep): QWORDS_BEG should not include
the first separators in ripper.
* parse.y (parser_parse_string): WORDS_SEP should not include
the closing parentheses of a word list in ripper, should include
spaces at beginning of lines. [ruby-core:83864] [Bug #14126]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60883 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Use @2 to only include a range of tSTRING_CONTENT.
e.g. The locations of NODE_STR is fixed:
```
%w[a]
```
* Before
```
NODE_STR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 5)
```
* After
```
NODE_STR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 4)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60868 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: list_append uses the locations
of the second argument. So we should set the
locations of $2 before pass it to list_append.
e.g. The locations of NODE_ARRAY is fixed:
```
%w[a b]
```
* Before
```
NODE_ARRAY (line: 1, first_lineno: 1, first_column: 0, last_lineno: 0, last_column: -1)
```
* After
```
NODE_ARRAY (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 7)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60867 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Use @2 to not include a range of ' '.
e.g. The locations of NODE_LIT is fixed:
```
%i[a]
```
* Before
```
NODE_LIT (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 5)
```
* After
```
NODE_LIT (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 4)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60866 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: list_append uses the locations
of the second argument. So we should set the
locations of $2 before pass it to list_append.
e.g. The locations of NODE_ARRAY is fixed:
```
%i[a b]
```
* Before
```
NODE_ARRAY (line: 1, first_lineno: 1, first_column: 0, last_lineno: 0, last_column: -1)
```
* After
```
NODE_ARRAY (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 7)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60864 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Update the locations of NODE_FCALL
when nd_args is determined.
e.g. The locations of NODE_FCALL is fixed:
```
a 1
```
* Before
```
NODE_FCALL (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 1)
```
* After
```
NODE_FCALL (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 3)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60862 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (block_append_gen): Update the last
location of NODE_BLOCK when a tail is appended.
e.g. The locations of NODE_BLOCK is fixed:
```
a; b; c
```
* Before
```
NODE_BLOCK (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 4)
```
* After
```
NODE_BLOCK (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 7)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60855 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Update the locations of NODE_ITER
when nd_iter is determined.
e.g. The locations of NODE_ITER is fixed:
```
a {b}
```
* Before
```
NODE_ITER (line: 1, first_lineno: 1, first_column: 2, last_lineno: 1, last_column: 5)
```
* After
```
NODE_ITER (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 5)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60840 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (list_append_gen, list_concat): Update
the last location of NODE_ARRAY when an item is
appended or concatenated with another NODE_ARRAY.
e.g. The locations of NODE_ARRAY is fixed:
```
a(1,2,3)
```
* Before
```
NODE_ARRAY (line: 1, first_lineno: 1, first_column: 2, last_lineno: 1, last_column: 3)
```
* After
```
NODE_ARRAY (line: 1, first_lineno: 1, first_column: 2, last_lineno: 1, last_column: 7)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60829 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (node_assign_gen): NODE_CALL is not passed to
node_assign_gen. NODE_CALL is not assignable. Assignable
method call (array set and attr set) is represented by NODE_ATTRASGN.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60823 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (node_assign_gen): NODE_ATTRASGN is created
before rhs is created. It is needed to set location
after rhs is created to rhs range be included to the
location of NODE_ATTRASGN.
e.g. The locations of NODE_ATTRASGN is fixed:
```
a[1] = 2
```
* Before
```
NODE_ATTRASGN (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 4)
```
* After
```
NODE_ATTRASGN (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 8)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60821 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (node_assign_gen): In some case assignable nodes
are created before rhs is created. In this case it is
needed to set location after rhs is shifted to
rhs range be included to assignable nodes.
e.g. The locations of NODE_DASGN_CURR is fixed:
```
a = 10
```
* Before
```
NODE_DASGN_CURR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 1)
```
* After
```
NODE_DASGN_CURR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 6)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60816 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (k_return): prohibit return in class/module body except
for singleton class.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60790 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_op_assign_gen): Fix location of asgn node.
Assignable node (e.g. NODE_LASGN) is generated before rhs
is generated, so we reset the location when nd_value of
asgn is fixed.
e.g. :
```
a -= 10
```
* Before
```
NODE_DASGN_CURR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 1)
```
* After
```
NODE_DASGN_CURR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 7)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60781 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_params): Add prevline to store previous line.
* parse.y (yycompile0): Initialize prevline with 0.
* parse.y (parser_nextline): Store previous line on prevline.
* parse.y (parser_nextc): Check parser is on EOF or has nextline.
Now parser_yylex does not always set lex_p as lex_pend, we should check
EOF flag and nextline is set.
* parse.y (parser_yylex): Restore previous line, set lex_p and tokp
on '\n'. Before this commit, tokp is on the head of next line of '\n'
and lex_p is on the tail of next line when next token is '\n'.
By this behavior, in some case the last column of NODE_CALL (or NODE_QCALL) is
set to the last column of next line. NODE_CALL can be generated
via `primary_value call_op operation2 {} opt_paren_args` and opt_paren_args
can be none. If none is generated with next token '\n', the last column of
none is set to the last column of next line.
e.g. :
```
a.b
cd.ef
```
The location of NODE_CALL of first line is set to,
* Before
```
NODE_CALL (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 6)
```
* After
```
NODE_CALL (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 3)
```
* parse.y (parser_mark): GC mark prevline.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60780 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_tokadd_utf8): relax restriction to allow zero or
more codepoints. fixup r59417.
7e8b910 (commitcomment-25602670)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60766 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (node_newnode): Initialize last column of nodes
with -1 and lineno with 0 to make it easy to detect nodes
which we forget to set a column number or lineno.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60751 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* node.c (rb_node_init): Initialize last location with 0.
* node.h (struct rb_code_range_struct): Define a structure
which contains first location and last location of a node.
* node.h (struct RNode): Use rb_code_range_t to store last
location of a node.
* node.h (nd_column, nd_set_column, nd_lineno, nd_set_lineno):
Follow-up the change of struct RNode.
* node.h (nd_last_column, nd_set_last_column, nd_last_lineno, nd_set_last_lineno):
Define getter/setter macros for last location of RNode.
* parse.y : Set last location of tokens.
Thanks to Yusuke Endoh (mame) for design of data structures.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60750 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
`rb_ast_dispose` calls `rb_gc_writebarrier_remember`. Unless we call
it, the marked objects may not be GC'ed until `rb_ast_t` is GC'ed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60722 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (rb_strterm_heredoc_t): should not define sourceline as
union, to fix wrong read on big-endian platforms.
as rb_imemo_new stores all members as VALUEs, should read it as
VALUE too.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60720 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_params): removed in_single, which is referenced
with in_def always.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60716 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* node.h (NEW_CASE2): Omit first argument of NEW_CASE2,
bacause the first argument of NEW_CASE2 is always 0.
* parse.y: Ditto
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60714 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
r60655 changed the usage of bison by defining a customized YYLTYPE,
which seemed to cause a random stall at rb_thread_terminate_all.
Kazuki Tsujimoto investigated the issue and pointed out the commit that
caused it. Thanks!
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60660 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rb_code_range_t has two t_code_location_t, i.e., the first and last
locations.
This is used for YYLTYPE, tracked locations of bison, and will be also
used for representing the "range", the first and the last locations of
each NODE. Currently, each NODE keeps only the first location, though.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60655 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (rb_strterm_mark): do not define in ripper to get rid of
duplicate symbol with static-linked-ext.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60654 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This magic number has two meanings depending upon the context:
* "required keyword argument (no name)" on NODE_LASGN (`def foo(x:)`)
* "rest argument (no name)" on NODE_MASGN and NODE_POSTARG
('a, b, * = ary` or `a, b, *, z = ary`)
To show this intention explicitly, two macros are introduced:
NODE_SPECIAL_REQUIRED_KEYWORD and NODE_SPECIAL_NO_NAME_REST.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60650 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
`new_args_tail_gen` returned imemo, but the value was later accessed as
`NODE*`. This prevented change of NODE structure.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60647 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (rb_discard_node_gen): rb_discard_node() is not used in
ripper right now.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60645 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_set_line): removed no longer used function.
* parse.y (rb_strterm_heredoc_t): adjust type of sourceline to
ruby_sourceline.
* parse.y (rb_strterm_t): get rid of redefinition.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60644 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This is a follow-up of r60488. Not only the Ruby parser but also Ripper
now use NODE buffer instead of NODE objects managed by GC.
Now we can change the struct of NODEs so that it can keep detailed
location information, perhaps (not tried yet). Note that, however, the
first word of each NODE must be still compatible with RBasic structure.
This is because Ripper handles NODEs and other objects uniformly;
especially, it still uses `RB_TYPE_P(obj, T_NODE)` for distinguishing
between NODEs and other objects.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60639 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
