Commit graph

71 commits

Author SHA1 Message Date
Kevin Newton
82f18baa21 [ruby/prism] Provide flags for changing encodings
e838eaff6f
2023-12-06 14:23:38 -05:00
Lily Lyons
1227b6d912 [ruby/prism] Refactor pm_diagnostic_t and pm_comment_t to use pm_location_t
115b6a2fc6
2023-12-04 14:33:44 +00:00
Kevin Newton
492c82cb41 [ruby/prism] Prism.parse_success?(source)
A lot of tools use Ripper/RubyVM::AbstractSyntaxTree to determine
if a source is valid. These tools both create an AST instead of
providing an API that will return a boolean only.

This new API only creates the C structs, but doesn't bother
reifying them into Ruby/the serialization API. Instead it only
returns true/false, which is significantly more efficient.

7014740118
2023-12-01 20:53:34 +00:00
Kevin Newton
cdb74d74af [ruby/prism] Change numbered parameters
Previously numbered parameters were a field on blocks and lambdas
that indicated the maximum number of numbered parameters in either
the block or lambda, respectively. However they also had a
parameters field that would always be nil in these cases.

This changes it so that we introduce a NumberedParametersNode that
goes in place of parameters, which has a single uint8_t maximum
field on it. That field contains the maximum numbered parameter in
either the block or lambda.

As a part of the PR, I'm introducing a new UInt8Field type that
can be used on nodes, which is just to make it a little more
explicit what the maximum values can be (the maximum is actually 9,
since it only goes up to _9). Plus we can do a couple of nice
things in serialization like just read a single byte.

2d87303903
2023-12-01 12:03:09 -05:00
Kevin Newton
7174c62f6b [prism] Remove escaping C source files 2023-11-30 12:11:19 +09:00
Jean Boussier
2af82e2316 [ruby/prism] Convert start line to signed integers
Ruby allows for 0 or negative line start, this is often used
with `eval` calls to get a correct offset when prefixing a snippet.

e.g.

```ruby
caller = caller_locations(1, 1).first
class_eval <<~RUBY, caller.path, caller.line - 2
  # frozen_string_literal: true
  def some_method
    #{caller_provided_code_snippet}
  end
RUBY
```

0d14ed1452
2023-11-29 13:56:19 +00:00
Jean Boussier
2653404840 [ruby/prism] Rename varint as varuint
Line numbers may be negative, so we need to introduce signed varint,
so renaming unsigned ones first avoid confusion.

90d862361e
2023-11-29 13:56:18 +00:00
Kevin Newton
c798943a4a [ruby/prism] Move DATA parsing into its own parse result field
42b60b6e95
2023-11-28 13:25:48 +00:00
Kevin Newton
bd4a992f38 [ruby/prism] Correct template.rb comment
4d689fe1df
2023-11-27 17:30:54 +00:00
Hiroya Fujinami
bad351a930 [ruby/prism] Remove handling OptionalNodeField in set_newline_flag
template
(https://github.com/ruby/prism/pull/1905)

6f7cbc1ca9
2023-11-21 22:10:35 +00:00
Kevin Newton
7d6f812c31 [ruby/prism] Update to v0.18.0
1398879d79
2023-11-21 13:59:10 -05:00
Kevin Newton
f2ed7eaba0 [ruby/prism] Add character APIs for locations
(https://github.com/ruby/prism/pull/1809)

d493ccd093
2023-11-20 16:07:06 +00:00
Kevin Newton
75d85f3f6f [ruby/prism] Remove non-ASCII source characters
(https://github.com/ruby/prism/pull/1787)

5acc38a2f3
2023-11-20 15:53:22 +00:00
Kevin Newton
f479e629ab [ruby/prism] Revert "Ensure serialized file is little endian"
4cec275fff
2023-11-18 01:32:37 +00:00
Kevin Newton
7c99e43c3f [ruby/prism] Ensure serialized file is little endian
0c762ee68a
2023-11-17 20:11:25 +00:00
Kevin Newton
94f82a65f7 [ruby/prism] Add the ability to convert nodes to dot
3e4b4fb947
2023-11-12 02:53:33 +00:00
Marco Roth
e6916f1305 [ruby/prism] Implement JavaScript visitors
ea00a1b3c6
2023-11-11 22:01:06 +00:00
Nobuyoshi Nakada
368a1cb3c4
Do not use non-ASCII chars in sources
No encodings are guaranteed in C compilers, and other than UTF-8
encodings may be assumed in some platforms, e.g., CP932 on Windows
Japanese edition, and may result in compilation errors.
2023-11-05 02:14:26 +09:00
Kevin Newton
5a132c7864
Update prism to v0.17.1 2023-11-03 13:59:07 -04:00
Vinicius Stock
dd0330249e [ruby/prism] Always fully qualify types for RBIs and avoid using attributes
0d0d039f8e
2023-11-03 17:53:54 +00:00
Kevin Newton
c44c982c93
[ruby/prism] Bump to version 0.17.0
61a306cd5c
2023-11-03 11:17:37 -04:00
Kevin Newton
4b5f516f2e [ruby/prism] Split comment
We were previously holding a type field on Comment to tell what
kind of comment it was. Instead, let's just use actual classes for
this.

e76830ca6e
2023-11-03 14:35:08 +00:00
Kevin Newton
aab2a6a8a3
[ruby/prism] Fix up lint
4f3a3e3ec1
2023-11-03 10:13:50 -04:00
Kevin Newton
8587d9a8bf
[ruby/prism] Wire up options through the Java parser
13fa262669
2023-11-03 10:13:50 -04:00
Kevin Newton
d7d3243364
[ruby/prism] Properly support the start line option
33cc75a4b7
2023-11-03 10:13:50 -04:00
Kevin Newton
6496591194
[ruby/prism] Rename serialization APIs for consistency
5a2252e3ac
2023-11-03 10:13:49 -04:00
Kevin Newton
05f5c545d2
[ruby/prism] Wire up options through the FFI API
f0aa8ad93b
2023-11-03 10:13:49 -04:00
Kevin Newton
2a0f2b7763
[ruby/prism] Create an options struct for passing all of the possible options
99e81619de
2023-11-03 10:13:49 -04:00
Kevin Newton
ee7bf4643d [ruby/prism] Handle invalid constants in the constant pool
It's possible for us to parse a constant that is invalid in the
current encoding. To fix this, we wrap the intern function in an
rb_protect to ensure we always put a valid symbol into the AST.

e78398554d
2023-11-02 22:14:08 +00:00
Kevin Newton
3551abab06 [ruby/prism] Fix up lint
77d4056766
2023-11-01 13:10:29 -04:00
Kevin Newton
690f3bbf5d [ruby/prism] Last remaining missing C comments
e327449db6
2023-11-01 13:10:29 -04:00
Kevin Newton
51079a89af [ruby/prism] More documentation for ast.h
84e22c5581
2023-11-01 13:10:29 -04:00
Kevin Newton
17923cc876 [ruby/prism] Even more C file documentation
9c648ce615
2023-11-01 13:10:29 -04:00
Kevin Newton
6b3b530cc1 [ruby/prism] Documentation for nodes
69323d3df4
2023-11-01 13:10:29 -04:00
Kevin Newton
b12c795bdc [ruby/prism] Remove unnecessary PM_EMPTY_NODE_LIST
857f72e1f4
2023-11-01 13:10:29 -04:00
Kevin Newton
f791db4433 [ruby/prism] Documentation for defines, version, and prettyprint
e2e29bef2d
2023-11-01 13:10:29 -04:00
Kevin Newton
7c8d939680 [ruby/prism] Start better documenting C API
2b6e661bbc
2023-11-01 13:10:29 -04:00
Kevin Newton
79034fbd50 [ruby/prism] More Ruby docs
ca9a660f52
2023-11-01 13:10:29 -04:00
Kevin Newton
953138698e [ruby/prism] Docs for node.rb and parse_result.rb
085da4feb9
2023-11-01 13:10:29 -04:00
Kevin Newton
0a460b23e0 [ruby/prism] Add comments on flags
3abd09c803
2023-11-01 13:10:29 -04:00
Kevin Newton
5df14ecebe [ruby/prism] Fix up Ruby docs
8062849d0d
2023-11-01 13:10:29 -04:00
Kevin Newton
b0a2373e14 [ruby/prism] Share comment generation logic
6e5556dbdf
2023-10-31 12:28:58 +00:00
Kevin Newton
b413760d66
[prism] Update prism to latest 2023-10-30 12:35:39 -04:00
Ufuk Kayserilioglu
1d51e4cadf [ruby/prism] Improve comment generation in templates
The existing comment generation was hard to read and was making a lot of string manipulation. However, ERB files are already designed to do string manipulation, so we can use that instead.

So, instead of doing a split and a map, I opted to use the `#each_line` method to iterate over the lines of the file.

Also, in order to add an optional space padding at the beginning of the line, I opted to pad it with a space and to then right trim it. This makes sure that no space is left behind if the line is empty, but a space is added if the line is not empty.

5736711e70
2023-10-30 14:15:47 +00:00
Kevin Newton
c201dbc0ad [ruby/prism] Prism.parse_inline_comments
5b72f84480
2023-10-27 18:09:14 +00:00
Kevin Newton
26a05c4217 [ruby/prism] Use printf attribute and then fix warnings
3193902c43
2023-10-27 14:23:17 +00:00
Vinicius Stock
06173a41dd [ruby/prism] Add rbi_class to fields
568fb39e2c
2023-10-27 13:45:52 +00:00
Vinicius Stock
544f64075b [ruby/prism] Add RBI and RBS templates for Prism
b62305ff3e

Co-authored-by: Stan Lo <st0012@users.noreply.github.com>
2023-10-27 13:45:52 +00:00
Kevin Newton
21dc64534b [ruby/prism] Regenerate snapshots
160df9a687
2023-10-26 15:19:52 -04:00
Kevin Newton
d1bb858d47 [ruby/prism] Match existing Ruby prettyprint
6d8358c083
2023-10-26 15:19:43 -04:00