Commit graph

14659 commits

Author SHA1 Message Date
Cody Cutrer
fa334ecd40 [rubygems/rubygems] use full path for instance_eval
so that __dir__ and __FILE__ will work properly from within that gemfile
fragment, if eval_gemfile is given a relative path

d521bf9790
2024-02-15 20:26:03 +00:00
Benoit Daloze
8f17b3bd27 [ruby/prism] Avoid extra String copies in the FFI backend
* For Prism.parse_file the file contents would be read as native, then
  converted to a Ruby String, then converted to a native String for
  pm_serialize_parse().
* Refactor the logic to always use a pm_string for the source code and
  pass that to other native functions.

9002b3c47d
2024-02-15 20:25:35 +00:00
Koichi ITO
be82755d4a [ruby/prism] Support multi-versioning for Prism::Translation::Parser
## Summary

Fixes https://github.com/ruby/prism/pull/2356.

I'm working on integrating Prism into RuboCop.

This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named
in accordance with the following comments.
https://github.com/rubocop/rubocop/issues/12600#issuecomment-1932707748

Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode.
This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified.

Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3,
the expected name for an `lvar` is `:it`, not `:"0it"`.

### Expected AST

The following is an expected AST when parsing Ruby 3.3 code:

```console
$ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:send, nil, :it), :do_something))
```

### Actual AST

The following is an actual AST when parsing Ruby 3.3 code:

```console
$ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:lvar, :"0it"), :do_something))
```

`Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively.

And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future,
but the behavior will be maintained until RuboCop's support is finalized:
https://github.com/rubocop/rubocop/issues/12600#issuecomment-1933657732

## Additional Information

A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism.
For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`.

`Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem.

Therefore, logic to maintain compatibility between Parser and Prism is implemented.

62d3991e22
2024-02-15 20:21:13 +00:00
Yusuke Endoh
25d74b9527 Do not include a backtick in error messages and backtraces
[Feature #16495]
2024-02-15 18:42:31 +09:00
Benoit Daloze
c2d8d6eba6 Initialize the Prism::Source directly with all 3 fields for the C extension
* Faster that way:
  $ ruby -Ilib -rprism -rbenchmark/ips -e 'Benchmark.ips { |x| x.report("parse") { Prism.parse("1 + 2") } }'
  195.722k (± 0.5%) i/s
  rb_iv_set():
  179.609k (± 0.5%) i/s
  rb_funcall():
  190.030k (± 0.3%) i/s
  before this PR:
  183.319k (± 0.4%) i/s
2024-02-14 15:48:33 +00:00
Benoit Daloze
1b2708b123 [ruby/prism] Remove attr_writer's for ParseResult#start_line and #offsets
* As the user should not set these.
* Use #instance_variable_set/rb_iv_set() instead internally.

cace09fb8c
2024-02-14 15:48:33 +00:00
Benoit Daloze
f0f6ffef42 [ruby/prism] Serialize the newline_list to avoid recomputing it again later
* Fixes https://github.com/ruby/prism/issues/2380

4eaaa90114
2024-02-14 15:48:32 +00:00
Kevin Newton
12b0e67fec
[PRISM] Sync to latest prism 2024-02-14 09:58:36 -05:00
Stan Lo
c84581f1d9 [ruby/irb] Repurpose the help command to display the help message
(https://github.com/ruby/irb/pull/872)

See #787 for more details.

d9192d92d0
2024-02-14 13:47:45 +00:00
tomoya ishida
d0412599e0 [ruby/irb] Improve constant lookup in SourceFinder
(https://github.com/ruby/irb/pull/871)

87c279ccf2
2024-02-14 13:46:49 +00:00
Kevin Newton
b1964a9204 [ruby/prism] Add code unit APIs to location
LSPs need this because the protocol dictates that you return code
units for offsets. None of our existing APIs provided that
information, and since we hid the source it's not nearly as useful
for them. Now they can pass an encoding directly to:

* Location#start_code_units_offset
* Location#end_code_units_offset
* Location#start_code_units_column
* Location#end_code_units_column

4757a2cc06

Co-Authored-By: Vinicius Stock <vinicius.stock@shopify.com>
2024-02-13 20:10:25 +00:00
Kevin Newton
29d04bb0c4 [ruby/prism] Introduce version: "3.4.0"
This is effectively an alias for "latest" right now. In the future
it will change to be its own enum value.

2c86036022
2024-02-13 18:26:28 +00:00
Vinicius Stock
dedca31804 [ruby/prism] Fix full_name for constant path targets
84c10f3a2d
2024-02-13 14:09:19 +00:00
Stan Lo
2f0f95235a [ruby/irb] Fix SourceFinder's constant evaluation issue
(https://github.com/ruby/irb/pull/869)

Currently, if the signature's constant part is not defined, a NameError
would be raised.

```
irb(main):001> show_source Foo
(eval):1:in `<top (required)>': uninitialized constant Foo (NameError)

Foo
^^^
        from (irb):1:in `<main>'
```

This commit fixes the issue and simplifies the `edit` command's implementation.

8c16e029d1
2024-02-13 13:36:32 +00:00
Stan Lo
ec26786b1a [ruby/irb] Refactor eval_path and SourceFinder::Source
(https://github.com/ruby/irb/pull/870)

* Assign `@eval_path` through `irb_path=` method

This simplifies the original caching logic for the `eval_path` method
and makes it easier to understand.

* Refactor SourceFinder::Source

c63e4c4035
2024-02-13 13:33:36 +00:00
Otávio Schwanck dos Santos
e848848b58
[ruby/reline] C for vi mode
(https://github.com/ruby/reline/pull/472)

d197be7c44
2024-02-13 18:57:17 +09:00
License Update
785aa47cd6 [rubygems/rubygems] Update SPDX license list as of 2024-02-08
ec3de95d79
2024-02-13 01:27:18 +00:00
tomoya ishida
7af97dc71f [ruby/irb] Powerup show_source by enabling RubyVM.keep_script_lines
(https://github.com/ruby/irb/pull/862)

* Powerup show_source by enabling RubyVM.keep_script_lines

* Add file_content field to avoid reading file twice while show_source

* Change path passed to eval, don't change irb_path.

* Encapsulate source coloring logic and binary file check insode class Source

* Add edit command testcase when irb_path does not exist

* Memoize irb_path existence to reduce file existence check calculating eval_path

239683a937
2024-02-12 18:38:30 +00:00
Kevin Newton
94bc5ad30a [ruby/prism] ruby_parser translator
1925b970c7
2024-02-12 17:54:54 +00:00
Noah Gibbs
16b39072a5 [ruby/prism] Move Prism::RipperCompat to Prism::Translation::Ripper
c0331abe4f
2024-02-12 15:57:57 +00:00
tomoya ishida
06995eb45b [ruby/irb] Fix exit! command warning and method behavior
(https://github.com/ruby/irb/pull/868)

* Fix exit! command warning and method behavior

* Remove arg(0) from Kernel.exit and Kernel.exit!

372bc59bf5
2024-02-12 11:28:54 +00:00
KJ Tsanaktsidis
1d467f2255 Burn default ASAN options into the built Ruby
* We always need use_sigaltstack=0 because Ruby registers sigaltstack
  handlers
* We also need to disable leak detection (unless RUBY_FREE_AT_EXIT is
  set - I might experiment later with automatically enabling leak
  detection if RUBY_FREE_AT_EXIT is set).

Burning it into the built ruby binary in this way avoids people needing
to remember to start their Ruby program with these flags all the time.

We also need a small fix in mkmf to make sure that test programs also
don't have leak detection enabled (this is never desirable)

[Bug #20256]
2024-02-12 10:51:46 +11:00
Nobuyoshi Nakada
c9006ddb88 [ruby/optparse] [DOC] Add description of OptionParser#define_by_keywords
451dea51a0
2024-02-11 16:55:41 +00:00
Nobuyoshi Nakada
bbccabe6d6 [ruby/optparse] [DOC] Add missing documents
33956ce93f
2024-02-11 16:08:14 +00:00
Stan Lo
5c4657f883 [ruby/irb] Polish the exit! command and its tests
(https://github.com/ruby/irb/pull/867)

* Remove IRB.irb_exit! method

It's not necessary to introduce a new method just for the exit! command
at this moment.

* Rename ExitForcedAction to ForceExit

* Move force exit tests to a dedicated file

* Fix nested history saving with exit! command

Because we switched to use `Kernel#exit` instead of `exit!`, the outer
session's ensure block in `Irb#run` will be run, which will save the
history. This means the separate check to save history when force exiting
is no longer necessary.

* execute_lines helper should also capture IRB setup's output

This prevents setup warnings from being printed to test output
while allowing those output to be tested.

* Update readme

899d10ade1
2024-02-11 05:17:40 +00:00
Ignacio Chiazzo Cardarello
429eeb09f2 [ruby/irb] Introduce exit! command
(https://github.com/ruby/irb/pull/851)

* Added failing test for when writing history on exit

* Save history on exit

* Exit early when calling Kernel.exit

* use status 0 for kernel.exit

* Added test for nested sessions

* Update lib/irb.rb

---------

c0a5f31679

Co-authored-by: Stan Lo <stan001212@gmail.com>
2024-02-10 22:07:53 +00:00
Nobuyoshi Nakada
f960fbc102 [ruby/optparse] Search exactly when require_exact
To work with options defined as `--[no]-something`.

Fix https://bugs.ruby-lang.org/issues/20252
Fix https://github.com/ruby/optparse/pull/60

78afdab307
2024-02-11 00:47:41 +09:00
Noah Gibbs
8688256522 [ruby/prism] Update lib/prism/ripper_compat.rb
2c53e017c1

Co-authored-by: Kevin Newton <kddnewton@gmail.com>
2024-02-09 19:49:27 +00:00
Noah Gibbs
f635b4dd0e [ruby/prism] RipperCompat: add array-refs, assigns, symbols, strings
b771c7f2ec
2024-02-09 19:49:27 +00:00
Kevin Newton
cf1cd215c0 [ruby/prism] Significantly faster offset cache for parser
8cd92eef79
2024-02-09 16:27:05 +00:00
Nobuyoshi Nakada
db73226bf6 [ruby/optparse] Adjust arguments for lambda-callbacks
Rake uses [lambda] as callbacks.
Calling it without omitted argument raises an `ArgumentError`.

lambda: https://github.com/ruby/rake/blob/master/lib/rake/application.rb#L543

213cb03b59
2024-02-09 19:58:31 +09:00
Nobuyoshi Nakada
2c6767b71e [ruby/optparse] Respect default values in block parameters
Fix https://github.com/ruby/optparse/pull/55

9d53e74aa4
2024-02-09 19:58:19 +09:00
fatkodima
f7a407cabd [ruby/optparse] Fix require_exact to work with options defined as --[no]-something
4e346ad337
2024-02-09 03:31:13 +00:00
Nobuyoshi Nakada
50bcaa6286 [ruby/optparse] Escape backslashes
b14c2c644d
2024-02-09 03:21:24 +00:00
Petrik
2a57e6e6ed [ruby/rdoc] Don't document aliases with trailing :nodoc directive
Attribute readers and writers can be marked as `:nodoc` to keep them
undocumented:

```ruby
attr_reader :name # :nodoc:
```

For aliases this behaviour should be the same:

```ruby
alias_method :old :new # :nodoc:
```

30f14e8271
2024-02-09 01:07:17 +00:00
Noah Gibbs
1983949811 [ruby/prism] Looks like Symbol#name wasn't a thing in 2.7, so need to switch back to to_s
0b90c9a398
2024-02-08 16:02:06 +00:00
Noah Gibbs
0c73553052 [ruby/prism] Add Kevin's visit_all change and change the parent class to Compiler.
bbdba3f42d

Co-Authored-By: Kevin Newton <kddnewton@gmail.com>
2024-02-08 16:02:06 +00:00
Noah Gibbs
f8b8a6780c [ruby/prism] Update lib/prism/ripper_compat.rb
03addf2d3d

Co-authored-by: Kevin Newton <kddnewton@gmail.com>
2024-02-08 16:02:05 +00:00
Noah Gibbs
3f0aa55449 [ruby/prism] Update lib/prism/ripper_compat.rb
5eac08f699

Co-authored-by: Kevin Newton <kddnewton@gmail.com>
2024-02-08 16:02:05 +00:00
Noah Gibbs
c1aba5d97b [ruby/prism] Update lib/prism/ripper_compat.rb
8271ce5ec9

Co-authored-by: Kevin Newton <kddnewton@gmail.com>
2024-02-08 16:02:05 +00:00
Noah Gibbs
70bc4ce34f [ruby/prism] Update lib/prism/ripper_compat.rb
6e2bf9c8cd

Co-authored-by: Kevin Newton <kddnewton@gmail.com>
2024-02-08 16:02:05 +00:00
Noah Gibbs
6aceb91de0 [ruby/prism] Update lib/prism/ripper_compat.rb
bce0a5c916

Co-authored-by: Kevin Newton <kddnewton@gmail.com>
2024-02-08 16:02:04 +00:00
Noah Gibbs
366af4679e [ruby/prism] RipperCompat: support for more features.
* add bin/prism ripper to compare Ripper output
* block arg handling is quirky, do it per-call-site
* block required params
* boolean values
* various assign-operator support
* breaks, early fragile begin/rescue/end
* more fixtures being checked

31732cb720
2024-02-08 16:02:04 +00:00
Hiroshi SHIBATA
70bb9cf065 [rubygems/rubygems] rake vendor:install
c38a96ceae
2024-02-08 15:27:01 +09:00
Hiroshi SHIBATA
75c5e1a136 [rubygems/rubygems] Removed unnecessary disabling of Style/RedundantParentheses
2361527c45
2024-02-08 15:27:00 +09:00
David Rodriguez
c236212600
[rubygems/rubygems] Use deprecation helper for deprecation warning
d1963bf1a6
2024-02-08 14:43:56 +09:00
David Rodríguez
24d5e7176e
[rubygems/rubygems] Refactor lockfile generation
6a0c03c77f
2024-02-08 14:43:56 +09:00
David Rodríguez
debc5aaee5
[rubygems/rubygems] Remove unused parameter
085eda7147
2024-02-08 14:43:56 +09:00
Kevin Newton
0e1f22ac7e [ruby/prism] Dev-only CLI
We keep adding more scripts to /bin that are doing the same kinds
of processing. Instead, this commit consolidates them all into a
single CLI that shares the same logic so that we can consistently
read files in the same way.

It keeps around 2 binstubs for bin/lex and bin/parse since those
are the most used and I'm sure people have built up muscle memory
for those. Those scripts are now just wrappers for forwarding to
bin/prism.

bddcb9bf17
2024-02-07 21:26:14 +00:00
Noah Gibbs
1b68b459ca [ruby/prism] Commit Kevin's suggestion to simplify grabbing the operator.
874ba7a1f4

Co-Authored-By: Kevin Newton <kddnewton@gmail.com>
2024-02-07 19:42:14 +00:00