Commit graph

107 commits

Author SHA1 Message Date
Kevin Newton
e9f1324464 Sync to latest prism 2024-02-01 12:52:16 -05:00
Kevin Newton
731367d0ab [ruby/prism] Fix up CI
224ea85565
2024-01-30 18:45:19 +00:00
Aaron Patterson
8e708e4a07 Update forwarding locals for prism 2024-01-30 13:19:06 -05:00
Kevin Newton
e050097beb [ruby/prism] Raise diagnostics for parser
102b4a16f5
2024-01-29 16:09:47 +00:00
Kevin Newton
e256d44f98 [ruby/prism] Handle implicit rest in array pattern for parser gem
d3722d6660
2024-01-28 01:10:47 +00:00
Kevin Newton
f12ebe1188 [ruby/prism] Add parser translation
8cdec8070c
2024-01-27 19:59:42 +00:00
Kevin Newton
e337c9478a [ruby/prism] Error follow-up
Split up the diagnostic levels so that error and warning levels
aren't mixed. Also fix up deconstruct_keys implementation.

bd3eeb308d

Co-authored-by: Benoit Daloze <eregontp@gmail.com>
2024-01-27 18:46:16 +00:00
Benoit Daloze
c2e2d2398b [ruby/prism] Call #inspect on diagnostic levels
* So it's clear it is a Symbol.
  Before:
  ... @level=warning_verbose_true>
  After:
  ... @level=:warning_verbose_true>

84503643b9
2024-01-27 18:41:07 +00:00
Kevin Newton
5d9d07a491 [ruby/prism] Bring back #arg_rest local
9b6907b727
2024-01-27 14:13:35 +00:00
Benoit Daloze
de135bc247 [ruby/prism] Add level to warnings and errors to categorize them
* Fixes https://github.com/ruby/prism/issues/2082

7a74576357
2024-01-26 21:34:34 +00:00
Kevin Newton
ebf803aa19 [ruby/prism] Fix Ruby head build
149e2ff7f6
2024-01-25 13:58:07 +00:00
Adam Hess
ac4046d34b [ruby/prism] switch unless to if
29bdbf4212

Co-authored-by: Kevin Newton <kddnewton@gmail.com>
2024-01-19 22:57:46 +00:00
Adam Hess
cfabe9c51c [ruby/prism] Handle stovetop start in constant path fullname
3a216e63fe
2024-01-19 22:57:45 +00:00
Kevin Newton
da521fc92c [ruby/prism] Parsing rules document
57a9575543
2024-01-19 16:12:25 +00:00
Matthew Healy
a58e091686 [ruby/prism] Extract identifier lexing documentation to separate file
c9df17e3c0
2024-01-19 15:43:29 +00:00
Cameron Dutro
8cbba87ca8 [ruby/prism] Add parse options to JavaScript's parsePrism function
d7fe7c7ae7
2024-01-16 19:19:30 +00:00
Kevin Newton
d124124c16 [ruby/prism] Fix up gemspec
82e3126762
2024-01-16 19:01:36 +00:00
Jemma Issroff
8bf12d7b2f [ruby/prism] First pass at documenting information about the CRuby compiler
b575914b2f
2024-01-16 18:47:48 +00:00
Aaron Patterson
881c5a1846 [ruby/prism] Add a "repeated flag" to parameter nodes
It's possible to repeat parameters in method definitions like so:

```ruby
def foo(_a, _a)
end
```

The compiler needs to know to adjust the local table size to account for
these duplicate names.  We'll use the repeated parameter flag to account
for the extra stack space required

b443cb1f60

Co-Authored-By: Kevin Newton <kddnewton@gmail.com>
Co-Authored-By: Jemma Issroff <jemmaissroff@gmail.com>
2024-01-10 15:24:26 +00:00
Kevin Newton
80da9b1547 [ruby/prism] Clarify __END__ comment
3e36d5eabc
2024-01-09 19:02:26 +00:00
Kevin Newton
3a3fcf56d0 [ruby/prism] Fix rational when parsing non decimal integer
75d4331f7f
2024-01-03 17:18:02 +00:00
Kevin Newton
23beceedb7 [ruby/prism] IndexTargetNode should always have ATTRIBUTE_WRITE
Because this is a user-facing change, we also need to deal with the
fact that CRuby 3.3.0 was just released.

In order to support workflows that want to parse exactly as CRuby
parses in a specific version, this PR introduces a new option to
the options struct that is "version". This allows you to specify
that you want "3.3.0" parsing.

I'm not sure if this is the correct solution. Another solution is
to just fork and keep around the old branch for security patches.
Or we could keep around a copy of the source files within this
repository as another directory and only update when necessary.
There are a lot of potential solutions here.

Because this change is so small and the check for it is so minimal,
I've decided to go with this enum. If this ends up entirely
cluttering the codebase with version checks, we'll come up with
another solution. But for now this works, so we're going to go in
this direction for a bit until we determine it's no longer working.

d8c7e6bd10
2024-01-02 18:51:18 +00:00
Kevin Newton
8bbe835cd8
Bump prism to version 0.19.0 2023-12-14 15:06:09 -05:00
Kevin Newton
fe6ee5e921 [ruby/prism] Move flag position consistently to front
6e69a81737
2023-12-06 20:50:02 +00:00
Kevin Newton
ea9f89eeb6 [ruby/prism] Ripper compat docs update
5f70b32b02
2023-12-04 17:00:14 +00:00
Kevin Newton
492c82cb41 [ruby/prism] Prism.parse_success?(source)
A lot of tools use Ripper/RubyVM::AbstractSyntaxTree to determine
if a source is valid. These tools both create an AST instead of
providing an API that will return a boolean only.

This new API only creates the C structs, but doesn't bother
reifying them into Ruby/the serialization API. Instead it only
returns true/false, which is significantly more efficient.

7014740118
2023-12-01 20:53:34 +00:00
Kevin Newton
ec83bd7356 [ruby/prism] Provide heredoc? queries
e148e8fe6a
2023-12-01 18:46:52 +00:00
Kevin Newton
cdb74d74af [ruby/prism] Change numbered parameters
Previously numbered parameters were a field on blocks and lambdas
that indicated the maximum number of numbered parameters in either
the block or lambda, respectively. However they also had a
parameters field that would always be nil in these cases.

This changes it so that we introduce a NumberedParametersNode that
goes in place of parameters, which has a single uint8_t maximum
field on it. That field contains the maximum numbered parameter in
either the block or lambda.

As a part of the PR, I'm introducing a new UInt8Field type that
can be used on nodes, which is just to make it a little more
explicit what the maximum values can be (the maximum is actually 9,
since it only goes up to _9). Plus we can do a couple of nice
things in serialization like just read a single byte.

2d87303903
2023-12-01 12:03:09 -05:00
Martin Emde
cbe57caa24 [ruby/prism] Fix comments after HEREDOCs again.
The problem was deeper than just looking back a single token.
You can push the heredoc_end token way back into the list.
We need to save the last location of a heredoc end to see if
it's the last token in the file.

Fixes https://github.com/ruby/prism/pull/1954

91dfd4eecd
2023-12-01 15:17:20 +00:00
Kevin Newton
abb1fe2868 [PRISM] Consolidate prism encoding files 2023-11-30 21:37:56 -05:00
Kevin Newton
10d3897e13 [PRISM] Big5 encodings 2023-11-30 21:37:56 -05:00
Kevin Newton
7b5bb978fb [PRISM] Alias CP51932 to EUC-JP 2023-11-30 21:37:56 -05:00
Kevin Newton
9ba92327f2 [PRISM] Consolidate SJIS encodings 2023-11-30 21:37:56 -05:00
Martin Emde
aac8be8034 [ruby/prism] Fix lex_compat for <<HEREDOC # comment at EOF
Fixes https://github.com/ruby/prism/pull/1874

304dd78dd2
2023-11-30 14:10:04 +00:00
Matt Boldt
9fc40d2b26 [ruby/prism] Add MacJapanese encoding
MacJapanese (also aliased as MacJapan) is a modified Shift_JIS
encoding, but is implemented identically in Ruby

9e0a097699
2023-11-29 12:08:15 -05:00
Dhaval
9fada99cb2 [ruby/prism] added CP950 encoding
9c2d1cf4ba
2023-11-29 11:15:50 -05:00
Kevin Newton
4938390177 [ruby/prism] Implicit rest nodes
Fundamentally, `foo { |bar,| }` is different from `foo { |bar, *| }`
because of arity checks. This PR introduces a new node to handle
that, `ImplicitRestNode`, which goes in the `rest` slot of parameter
nodes instead of `RestParameterNode` instances.

This is also used in a couple of other places, namely:

* pattern matching: `foo in [bar,]`
* multi target: `for foo, in bar do end`
* multi write: `foo, = bar`

Now the only splat nodes with a `NULL` value are when you're
forwarding, as in: `def foo(*) = bar(*)`.

dba2a3b652
2023-11-28 22:33:50 +00:00
Kevin Newton
c798943a4a [ruby/prism] Move DATA parsing into its own parse result field
42b60b6e95
2023-11-28 13:25:48 +00:00
Kevin Newton
1f06d168ba [ruby/prism] Add new doc to gemspec
99dfca6c1d
2023-11-22 15:15:07 +00:00
heyogrady
e3ef05a434 [ruby/prism] Add CP949 encoding
9e78dfdf69
2023-11-22 09:12:35 -05:00
Benoit Daloze
018e6abf85 [ruby/prism] Move CallNode#name field between receiver and arguments
* The same order as in source code.
* CallOrWriteNode, CallOperatorWriteNode, CallAndWriteNode already have
  the correct order so it was also inconsistent with them.

4434e4bc22
2023-11-22 12:15:20 +00:00
Kevin Newton
7d6f812c31 [ruby/prism] Update to v0.18.0
1398879d79
2023-11-21 13:59:10 -05:00
Vinicius Stock
7a9bb6d2a7 [ruby/prism] Fix constant path full name when parent is not a
constant
(https://github.com/ruby/prism/pull/1742)

* Raise if constant path parts contains nodes that can't be used to build full name

* Fix typo in constant path error documentation

Co-authored-by: Tim Morgan <tim@timmorgan.org>

---------

d73a053262

Co-authored-by: Tim Morgan <tim@timmorgan.org>
2023-11-21 16:39:27 +00:00
Haldun Bayhantopcu
8966d06b96 [ruby/prism] Warning for ENDs in methods
(https://github.com/ruby/prism/pull/1899)

1b41c2d56c
2023-11-21 16:36:12 +00:00
Ryan Garver
e16ff17374 Rename the big5-hkscs stuff to something more generic and add UAO sharing common code.
Merge the Big5 extensions into pm_big5.c
2023-11-20 21:18:31 -05:00
Kevin Newton
f2ed7eaba0 [ruby/prism] Add character APIs for locations
(https://github.com/ruby/prism/pull/1809)

d493ccd093
2023-11-20 16:07:06 +00:00
Ryan Garver
cd4316a51f [ruby/prism] Big5 HKSCS encoding
3ca9823eb4
2023-11-18 13:52:17 -05:00
Maple Ong
d976e7cc4b [ruby/prism] Update gemspec
1a10f6f9c0
2023-11-16 17:13:17 +00:00
Kevin Newton
d2e7a70ee6 [ruby/prism] Track the then keyword for conditionals
fef0019a25
2023-11-15 23:08:11 +00:00
Kevin Newton
ed75518192
[ruby/prism] Rename librubyparser to libprism
librubyparser was an artifact of the prototype that was initially
named ruby-parser. Instead, this renames it to libprism to be
consistent with the actual name.

8600b06811
2023-11-14 16:22:03 -05:00