(https://github.com/ruby/irb/pull/484)
* Improve assert_indenting helper
Instead of putting assertions inside the `auto_indent` block, we
can just make `auto_indent` return the calculated space count, and use
it for assertion outside of the `auto_indent` block call.
This simplifies the setup code and makes the intention easier to
understand.
* Introduce assert_row_indenting helper
1. Helper users shouldn't need to write 2 assertions for the current and
the next line's indentation.
2. With this new approach, we can generate clearer error message for
both cases:
When the current line's space count doesn't match
```
Incorrect spaces calculation for line:
```
> def each_top_level_statement
```
All lines:
```
def each_top_level_statement
```
<0> expected but was
<nil>
```
When the next line's space count doesn't match
```
Incorrect spaces calculation for line after the current line:
```
def each_top_level_statement
>
```
All lines:
```
def each_top_level_statement
```
<3> expected but was
<2>
```
* Replace assert_indenting with assert_row_indenting
Some background for this refactor:
1. Through a RubyLex instance's lifetime, the context passed to its methods
should be the same.
Given that `Context` is only initialised in `Irb#initialize`,
this should be true.
2. When `RubyLex` is initialised, the context object should be accessible.
This is also true in all 3 of `RubyLex.new`'s invocations.
With the above observations, we should be able to store the context in `RubyLex`
as an instance variable. And doing so will make `RubyLex`'s instance methods
easier to use and maintain.
5c8d3df2df
(https://github.com/ruby/irb/pull/504)
* Simplify `RubyLex#set_prompt`
It's optional argument is never used by any caller.
* Remove the optional `p` argument from `RubyLex#set_input`
The argument is only used in a test case, which can be easily replaced by
a block argument.
It looks like tool/sync_default_gems.rb is not capable of cherry-picking
commits from ruby/irb. I just executed `tool/sync_default_gems.rb irb`
to fix the sync status.
I'm not sure if what's the cause. It could be related to some diff that
doesn't exist in ruby/ruby, or it might be related to non-linear history
due to merge commits. For next time, I'd like to at least exclude the
second possibility, so I disabled merge commits in ruby/irb.
conveniently
(https://github.com/ruby/irb/pull/429)
* Create a base TestIRB::TestCase class
* Save/restore encodings for tests that initializes InputMethod classes
Because `RelineInputMethod#initializes` calls `set_encoding`, which
changes stdio/out/err and Encoding's default encoding values, we need to
make sure any test that directly or indirectly (e.g. through Context)
initializes `RelineInputMethod` restores encodings.
`ReadlineInputMethod` also changes encodings but currently no tests
cover it.
* Remove unnecessary TestHelper module
Since we now have a base TestCase, without_rdoc can just live there.
c2874ec121
(https://github.com/ruby/irb/pull/428)
RubyLex is not designed to be used alone. It's usually used with an IRB
context, which requires workspace. So its tests should have access to
those components too.
608f261da4
(https://github.com/ruby/irb/pull/427)
* Make sure `RubyLex#set_input`'s context is always present in tests
In real-world scenarios, the context should always be non-nil:
https://github.com/ruby/irb/blob/master/lib/irb.rb#L489
So we should make sure our test setup reflects that.
* Make context a required keyword
Since in practice, `set_input`'s context should always be non-nil, its
parameters should reflect that.
And since `RubyLex#check_state` is only called by `#lex` and
`#set_input`, both of which now always require context, we can assume
its context should be non-nil too.
1aeeb86203
* Use local_variables for colorize, code_block_open check, nesting_level and assignment_expression check
* Check if expression is an assignment BEFORE evaluating it. evaluate might define new localvars and change result of assignment_expression?
* Add local_variables dependent code test
* pend local variable dependent test on truffleruby
code_block_open is not working on truffleruby
* Always pass context to RubyLex#lex
* Rename local_variable_assign_code generator method name
* Add assignment expression truncate test
* Add Context#local_variables and make generate_local_variables_assign_code more simple
* Update lib/irb/input-method.rb
Co-authored-by: Stan Lo <stan001212@gmail.com>
* Add a comment why assignment expression check should be done before evaluate
c8b3877281
Co-authored-by: Stan Lo <stan001212@gmail.com>
Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
* Fix backtick method def method call handled as backtick open
* Fix handling heredoc in check_string_literal
* Sort result of lexer.parse by pos in ruby<2.7. It's not sorted when the given code includes heredoc.
* Update lib/irb/ruby-lex.rb
Co-authored-by: Stan Lo <stan001212@gmail.com>
* Update lib/irb/ruby-lex.rb
Co-authored-by: Stan Lo <stan001212@gmail.com>
* Add check_string_literal test for heredoc code that does not end with newline
44bc712460
Co-authored-by: Stan Lo <stan001212@gmail.com>
Instead of accessing the struct as an array, access it via methods. There are other places inside of this file already using this API (for example e0a5c3d2b7/lib/irb/ruby-lex.rb (L829-L830)).
This commit moves all struct array-ish calls to use their method calls instead. It is also ~1.23 faster accessing values via a method instead of as an array according to this microbenchmark:
```ruby
Elem = Struct.new(:pos, :event, :tok, :state, :message) do
def initialize(pos, event, tok, state, message = nil)
super(pos, event, tok, State.new(state), message)
end
# ...
def to_a
a = super
a.pop unless a.empty?
a
end
end
class ElemClass
attr_accessor :pos, :event, :tok, :state, :message
def initialize(pos, event, tok, state, message = nil)
@pos = pos
@event = event
@tok = tok
@state = State.new(state)
@message = message
end
def to_a
if @message
[@pos, @event, @tok, @state, @message]
else
[@pos, @event, @tok, @state]
end
end
end
# stub state class creation for now
class State; def initialize(val); end; end
```
```ruby
Benchmark.ips do |x|
x.report("struct") { struct[1] }
x.report("class ") { from_class.event }
x.compare!
end; nil
```
```
Warming up --------------------------------------
struct 1.624M i/100ms
class 1.958M i/100ms
Calculating -------------------------------------
struct 17.139M (± 2.6%) i/s - 86.077M in 5.025801s
class 21.104M (± 3.4%) i/s - 105.709M in 5.015193s
Comparison:
class : 21103826.3 i/s
struct: 17139201.5 i/s - 1.23x (± 0.00) slower
```
https://github.com/ruby/irb/issues/55
If we had put multiple open braces on a line the with no closing brace
spaces_of_nest array keeps getting '0' added to it. This means that when
we pop off of this array we are saying that we should be in position zero
for the next line. This is an issue because we don't always want to be
in position 0 after a closing brace.
Example:
```
[[[
]
]
]
```
In the above example the 'spaces_of_nest' array looks like this after
the first line is entered: [0,0,0]. We really want to be indented 4
spaces for the 1st closing brace 2 for the 2nd and 0 for the 3rd. i.e.
we want it to be: [0,2,4].
We also saw this issue with a heredoc inside of an array.
```
[<<FOO]
hello
FOO
```
80c69c8272
This commit fixes the check_newline_depth_difference method to multiple
open braces on one line into account. Before this change we were
subtracting from the depth in check_newline_depth_difference on
every open brace. This is the right thing to do if the opening and
closing brace are on the same line. For example in a method definition we
have an opening and closing parentheses we want to add 1 to our depth,
and then remove it.
```
def foo()
end
```
However this isn't the correct behavior when the brace spans multiple
lines. If a brace spans multiple lines we don't want to subtract from
check_newline_depth_difference and we want to treat the braces the same
way as we do `end` and allow check_corresponding_token_depth to pop the
correct depth.
Example of bad behavior:
```
def foo()
[
]
puts 'bar'
end
```
Example of desired behavior:
```
def foo()
[
]
puts 'bar'
end
```
7dc8af01e0
The set_auto_indent method calculates the correct number of spaces for
indenting a line. We think there might be a few bugs in this method so
we are testing the current functionality to make sure nothing breaks
when we address those bugs.
Example test failure:
```
1) Failure:
TestIRB::TestRubyLex#test_auto_indent [/Users/Ben/Projects/irb/test/irb/test_ruby_lex.rb:75]:
Calculated the wrong number of spaces for:
def each_top_level_statement
initialize_input
catch(:TERM_INPUT) do
loop do
begin
prompt
unless l = lex
throw :TERM_INPUT if @line == ''
else
.
<10> expected but was
<12>.
```
752d5597ab
