Some background for this refactor:
1. Through a RubyLex instance's lifetime, the context passed to its methods
should be the same.
Given that `Context` is only initialised in `Irb#initialize`,
this should be true.
2. When `RubyLex` is initialised, the context object should be accessible.
This is also true in all 3 of `RubyLex.new`'s invocations.
With the above observations, we should be able to store the context in `RubyLex`
as an instance variable. And doing so will make `RubyLex`'s instance methods
easier to use and maintain.
5c8d3df2df
methods
(https://github.com/ruby/irb/pull/502)
* Remove unnecessary parameter defaults
These methods are always called with tokens specified. So their default
`@tokens` value is never used and is misleading.
* Remove unnecessary context default
* Require tokens for `RubyLex#check_state`
(https://github.com/ruby/irb/pull/504)
* Simplify `RubyLex#set_prompt`
It's optional argument is never used by any caller.
* Remove the optional `p` argument from `RubyLex#set_input`
The argument is only used in a test case, which can be easily replaced by
a block argument.
(https://github.com/ruby/irb/pull/350)
Simplify part of regex ``[_a-zA-Z0-9]`` with equivalent shorthand ``\w``.
Replace case-when with match ``$1`` or default value ``?"``, making intention more clear.
(https://github.com/ruby/irb/pull/449)
* Seamlessly integrate a few debug commands
* Improve the break command support
* Utilize skip_src option if available
* Add step and delete commands
* Write end-to-end tests for each debugger command
* Add documentation
* Add backtrace, info, catch commands
976100c1c2
(https://github.com/ruby/irb/pull/426)
* Allow non-identifier aliases
* Move the configuration to IRB.conf
* Avoid abusing method lookup for symbol aliases
* Add more alias tests
* A small optimization
* Assume non-nil Context
* Load IRB.conf earlier
e23db5132e
(https://github.com/ruby/irb/pull/427)
* Make sure `RubyLex#set_input`'s context is always present in tests
In real-world scenarios, the context should always be non-nil:
https://github.com/ruby/irb/blob/master/lib/irb.rb#L489
So we should make sure our test setup reflects that.
* Make context a required keyword
Since in practice, `set_input`'s context should always be non-nil, its
parameters should reflect that.
And since `RubyLex#check_state` is only called by `#lex` and
`#set_input`, both of which now always require context, we can assume
its context should be non-nil too.
1aeeb86203
* Use local_variables for colorize, code_block_open check, nesting_level and assignment_expression check
* Check if expression is an assignment BEFORE evaluating it. evaluate might define new localvars and change result of assignment_expression?
* Add local_variables dependent code test
* pend local variable dependent test on truffleruby
code_block_open is not working on truffleruby
* Always pass context to RubyLex#lex
* Rename local_variable_assign_code generator method name
* Add assignment expression truncate test
* Add Context#local_variables and make generate_local_variables_assign_code more simple
* Update lib/irb/input-method.rb
Co-authored-by: Stan Lo <stan001212@gmail.com>
* Add a comment why assignment expression check should be done before evaluate
c8b3877281
Co-authored-by: Stan Lo <stan001212@gmail.com>
Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
* Fix backtick method def method call handled as backtick open
* Fix handling heredoc in check_string_literal
* Sort result of lexer.parse by pos in ruby<2.7. It's not sorted when the given code includes heredoc.
* Update lib/irb/ruby-lex.rb
Co-authored-by: Stan Lo <stan001212@gmail.com>
* Update lib/irb/ruby-lex.rb
Co-authored-by: Stan Lo <stan001212@gmail.com>
* Add check_string_literal test for heredoc code that does not end with newline
44bc712460
Co-authored-by: Stan Lo <stan001212@gmail.com>
Instead of accessing the struct as an array, access it via methods. There are other places inside of this file already using this API (for example e0a5c3d2b7/lib/irb/ruby-lex.rb (L829-L830)).
This commit moves all struct array-ish calls to use their method calls instead. It is also ~1.23 faster accessing values via a method instead of as an array according to this microbenchmark:
```ruby
Elem = Struct.new(:pos, :event, :tok, :state, :message) do
def initialize(pos, event, tok, state, message = nil)
super(pos, event, tok, State.new(state), message)
end
# ...
def to_a
a = super
a.pop unless a.empty?
a
end
end
class ElemClass
attr_accessor :pos, :event, :tok, :state, :message
def initialize(pos, event, tok, state, message = nil)
@pos = pos
@event = event
@tok = tok
@state = State.new(state)
@message = message
end
def to_a
if @message
[@pos, @event, @tok, @state, @message]
else
[@pos, @event, @tok, @state]
end
end
end
# stub state class creation for now
class State; def initialize(val); end; end
```
```ruby
Benchmark.ips do |x|
x.report("struct") { struct[1] }
x.report("class ") { from_class.event }
x.compare!
end; nil
```
```
Warming up --------------------------------------
struct 1.624M i/100ms
class 1.958M i/100ms
Calculating -------------------------------------
struct 17.139M (± 2.6%) i/s - 86.077M in 5.025801s
class 21.104M (± 3.4%) i/s - 105.709M in 5.015193s
Comparison:
class : 21103826.3 i/s
struct: 17139201.5 i/s - 1.23x (± 0.00) slower
```
From Reidline to Reline
Update description used in take_corresponding_syntax_to_kw_do and is_the_in_correspond_to_a_for methods
Use possessive noun correctly
Second element
4fa9714d6f
This fixes at least an issue where irb will incorrectly assume
code opens a heredoc when it does not, such as this code:
```ruby
s1 = 'testing'
s2 = 'this'
s2 <<s1
p s1
s1
```
Ruby parses the `s2 <<s1` as `s2.<<(s1)`, not as a heredoc, because
`s2` is a local variable in scope. irb was using ripper without
letting ripper know that `s2` was a local variable, so ripper would
lex it as a heredoc instead of a method call.
Fix the situation by prepending a line at line 0 with all local
variable definitions in scope whenever lexing. This fixes the
heredoc issue, and potentially other issues that depend on whether
an identifier is a local variable or not.
Fixes [Bug #17530]
4ed2187f76
* using the appropriciate exception instead of `break` so that the session
can be continue after the `irb_source` and `irb_load` commands
* suppress extra new line due to one more `#prompt` call
bdefaa7cfd
