When parent scopes around an eval are forwarding parameters (like
*, **, &, or ...) we need to know that information when we are in
the parser. As such, we need to support passing that information
into the scopes option. In order to do this, unfortunately we need
a bunch of changes.
The scopes option was previously an array of array of strings.
These corresponded to the names of the locals in the parent scopes.
We still support this, but now additionally support passing in a
Prism::Scope instance at each index in the array. This Prism::Scope
class holds both the names of the locals as well as an array of
forwarding parameter names (symbols corresponding to the forwarding
parameters). There is convenience function on the Prism module that
creates a Prism::Scope object using Prism.scope.
In JavaScript, we now additionally support an object much the same
as the Ruby side. In Java, we now have a ParsingOptions.Scope class
that holds that information. In the dump APIs, these objects in all
3 languages will add an additional byte for the forwarding flags in
the middle of the scopes serialization.
All of this is in service of properly parsing the following code:
```ruby
def foo(*) = eval("bar(*)")
```
21abb6b7c4
Calculating code unit offsets for a source can be very expensive,
especially when the source is large. This commit introduces a new
class that wraps the source and desired encoding into a cache that
reuses pre-computed offsets. It performs quite a bit better.
There are still some problems with this approach, namely character
boundaries and the fact that the cache is unbounded, but both of
these may be addressed in subsequent commits.
2e3e1a4d4d
This PR tweaks inspect representation of `Prism::Location`.
## Before
During debugging, the meaning of `@location=2147483648` was unclear:
```console
$ ruby -Ilib -rprism -e 'p Prism.lex("puts :hi").value.map(&:first)[1]'
#<Prism::Token:0x000000010cd74e40 @source=#<Prism::ASCIISource:0x000000010cb5f808 @source="puts :hi", @start_line=1, @offsets=[0]>,
@type=:SYMBOL_BEGIN, @value=":", @location=2147483648>
```
## After
This PR clarifies the contents of the location object, aligning with what I think user expects:
```console
$ ruby -Ilib -rprism -e 'p Prism.lex("puts :hi").value.map(&:first)[1]'
#<Prism::Token:0x000000010e174d50 @source=#<Prism::ASCIISource:0x000000010df5efe8 @source="puts :hi", @start_line=1, @offsets=[0]>,
@type=:SYMBOL_BEGIN, @value=":", @location=#<Prism::Location @start_offset=5 @length=1 start_line=1>>
```
Although it is uncertain whether Prism will accept this change in the inspect representation, it is submitted here as a suggestion.
e7421ce1c5
Having the @newline instance variable in every node adds up, and
since it is so rarely used, we only want to add it when necessary.
Moving this into an autoloaded file and moving the instance variable
out of the default initializers reduces allocated memory because the
nodes are now smaller and some fit into the compact list. On my
machine, I'm seeing about an 8% drop.
eea92c07d2
When creating the Ruby AST, we were previously allocating Location
objects for every node and every inner location. Instead, this
commit changes it to pack both the start offset and length into a
single u64 and pass that into the nodes. Then, when the locations
are requested via a reader method, we lazily allocate the Location
objects.
de203dca83
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
LSPs need this because the protocol dictates that you return code
units for offsets. None of our existing APIs provided that
information, and since we hid the source it's not nearly as useful
for them. Now they can pass an encoding directly to:
* Location#start_code_units_offset
* Location#end_code_units_offset
* Location#start_code_units_column
* Location#end_code_units_column
4757a2cc06
Co-Authored-By: Vinicius Stock <vinicius.stock@shopify.com>
Split up the diagnostic levels so that error and warning levels
aren't mixed. Also fix up deconstruct_keys implementation.
bd3eeb308d
Co-authored-by: Benoit Daloze <eregontp@gmail.com>
