Commit graph

20692 commits

Author SHA1 Message Date
Kevin Newton
843c760a0f [PRISM] Enable passing syntax tests 2024-03-27 08:34:42 -04:00
Kevin Newton
8b2fc85970 [PRISM] Enable passing frozen string in array test 2024-03-27 08:34:42 -04:00
Kevin Newton
6f8a252e96 [PRISM] Enable passing heredoc test 2024-03-27 08:34:42 -04:00
Kevin Newton
e4b2109065 [PRISM] Fix ASCII-compatible check for eval encoding 2024-03-27 08:34:42 -04:00
Andrii Konchyn
8fa6c36492 [ruby/strscan] Omit tests for #scan_byte and #peek_byte on
TruffleRuby temporary
(https://github.com/ruby/strscan/pull/91)

The methods were added in #89 but they aren't implemented in TruffleRuby
yet. So let's omit them for now to have CI green.

844d963b56
2024-03-27 12:17:01 +09:00
Jun Aruga
8896ac0289
[ruby/openssl] Fix test_pkey_dsa.rb in FIPS.
Note that I created the `dsa2048.pem` and signature text
(`signature_encoded.txt`), that is used as a text to create the `signature0` in
the `test_sign_verify` by the following steps with the `openssl` CLI on FIPS
module.

```
$ OPENSSL_DIR="${HOME}/.local/openssl-3.3.0-dev-fips-debug-1f03d33ef5"
$ export OPENSSL_CONF="${OPENSSL_DIR}/ssl/openssl_fips.cnf"

$ "${OPENSSL_DIR}/bin/openssl" dsaparam -out dsaparam2048.pem 2048
$ "${OPENSSL_DIR}/bin/openssl" gendsa -out dsa2048.pem dsaparam2048.pem

$ echo -n "Sign me!" > data.txt
$ "${OPENSSL_DIR}/bin/openssl" dgst -sha256 -sign dsa2048.pem data.txt > signature.txt
$ cat signature.txt | base64 > signature_encoded.txt
```

Skip the `test_DSAPrivateKey_encrypted` on FIPS because AES-128-CBC, the
password based encryption used in the PEM format uses MD5 for deriving the
encryption key from the password, and MD5 is not FIPS-approved.
See also the comment on the `test/openssl/utils.rb#omit_on_fips`.

4bdcb419a9
2024-03-27 12:16:11 +09:00
crazeteam
b2b665eba5 [DOC] remove repetitive words in comments
Signed-off-by: crazeteam <lilujing@outlook.com>
2024-03-27 07:52:18 +09:00
Samuel Williams
a7ff264477
Don't clear pending interrupts in the parent process. (#10365) 2024-03-27 10:10:07 +13:00
Kevin Newton
0c62eb25b5 [PRISM] Use correct encoding for regular expression literals 2024-03-26 15:32:09 -04:00
Koichi ITO
4a78d75213 [ruby/prism] Fix an incorrect range of Prism::Location when PM_ERR_RETURN_INVALID
This PR fixes the following incorrect range of `Prism::Location` when `PM_ERR_RETURN_INVALID`.

It may be hard to tell from the text, but this Ruby error highlights `return`:

```console
$ ruby -e 'class Foo return end'
-e:1: Invalid return in class/module body
class Foo return end
-e: compile error (SyntaxError)
```

Previously, the error's `Prism::Location` pointed to `end`:

```console
$ bundle exec ruby -Ilib -rprism -ve 'p Prism.parse("class Foo return end").errors'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:return_invalid @message="invalid `return` in a class or module body"
@location=#<Prism::Location @start_offset=17 @length=3 start_line=1> @level=:fatal>]

After this fix, it will indicate `return`.

```console
$ bundle exec ruby -Ilib -rprism -ve 'p Prism.parse("class Foo return end").errors'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:return_invalid @message="invalid `return` in a class or module body"
@location=#<Prism::Location @start_offset=10 @length=6 start_line=1> @level=:fatal>]
```

For reference, here are the before and after of `Prism::Translation::Parser`.

Before:

```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("class Foo return end")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:18: error: invalid `return` in a class or module body
(string):1: class Foo return end
(string):1:                  ^~~
/Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/diagnostic/engine.rb:72:in `process':
invalid `return` in a class or module body (Parser::SyntaxError)
        from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:220:in `block in unwrap'
        from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:218:in `each'
        from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:218:in `unwrap'
        from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:49:in `parse'
        from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:33:in `parse'
        from -e:1:in `<main>'
```

After:

```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("class Foo return end")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:11: error: invalid `return` in a class or module body
(string):1: class Foo return end
(string):1:           ^~~~~~
/Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/diagnostic/engine.rb:72:in `process':
invalid `return` in a class or module body (Parser::SyntaxError)
        from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:220:in `block in unwrap'
        from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:218:in `each'
        from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:218:in `unwrap'
        from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:49:in `parse'
        from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:33:in `parse'
        from -e:1:in `<main>'
```

This PR ensures that the originally intended `return` is highlighted as it should be.

1f9af4d2ad
2024-03-26 14:41:27 -04:00
Kevin Newton
240fb3957b [ruby/prism] Freeze internal parts, again
50372fee5c
2024-03-26 12:11:09 -04:00
Andrew Konchin
e9152bc9da [ruby/prism] Enable ParametersSignatureTest on TruffleRuby
c7a7af1eac
2024-03-26 13:49:18 +00:00
Étienne Barrié
2b08406cd0 Expose rb_str_chilled_p
Some extensions (like stringio) may need to differentiate between
chilled strings and frozen strings.

They can now use rb_str_chilled_p but must check for its presence since
the function will be removed when chilled strings are removed.

[Bug #20389]

[Feature #20205]

Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
2024-03-26 12:54:54 +01:00
Koichi ITO
52cf6ec46b [ruby/prism] Fix typos
After finding the "if if" typo, some additional typos identified by running `codespell` are also being corrected:
https://github.com/codespell-project/codespell

e6a34cfeeb
2024-03-26 10:51:12 +00:00
Nobuyoshi Nakada
a850cd1a87 [Bug #20392] Block arguments duplication check at super 2024-03-26 17:37:47 +09:00
Koichi Sasada
3680981c7b skip test_gc_stress_at_startup
(maybe) from 9cf754b the test fails on some environments:

20240325T200004Z.fail.html.gz

```
  1) Failure:
TestGc#test_gc_stress_at_startup [/home/chkbuild/chkbuild/tmp/build/20240325T200004Z/ruby/test/ruby/test_gc.rb:771]:
[Bug #15784]
pid 1087168 killed by SIGSEGV (signal 11) (core dumped).

1. [3/3] Assertion for "success?"
   | Expected #<Process::Status: pid 1087168 SIGSEGV (signal 11) (core dumped)> to be success?.
```

20240325T203002Z.fail.html.gz
20240325T195003Z.fail.html.gz
20240325T210003Z.fail.html.gz
...

so just skipt it until it works.
2024-03-26 17:18:43 +09:00
Martin Emde
aa90013829 [rubygems/rubygems] Fix: vendor_gem takes a block
50cda56fc3
2024-03-25 19:45:52 +00:00
Kevin Newton
453de8c2bc [ruby/prism] Revert "Frozen parts"
48f2e8c169
2024-03-25 19:08:46 +00:00
Kevin Newton
eef272f154 [ruby/prism] Mark inner parts of interpolated* nodes as frozen
58a127cd5d
2024-03-25 17:36:12 +00:00
Kevin Newton
a08954569f
[ruby/prism] Fix up minimal build setting
98c85c4acb
2024-03-25 11:54:20 -04:00
Kevin Newton
86077fbcde [ruby/prism] Refactor regexp lexing to make it easier to support CLRF
60805d85ca
2024-03-25 11:52:12 -04:00
Kevin Newton
14ab698967 [ruby/prism] Handle CLRF inside heredoc contents
1fbac72485
2024-03-25 11:52:09 -04:00
Koichi ITO
56a2fad2a4 [ruby/prism] Fix incorrect paring when using invalid regexp options
Fixes https://github.com/ruby/prism/pull/2617.

There was an issue with the lexer as follows.
The following are valid regexp options:

```console
$ bundle exec ruby -Ilib -rprism -ve 'p Prism.lex("/x/io").value.map {|token| token[0].type }'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
[:REGEXP_BEGIN, :STRING_CONTENT, :REGEXP_END, :EOF]
```

The following are invalid regexp options. Unnecessary the `IDENTIFIER` token is appearing:

```console
$ bundle exec ruby -Ilib -rprism -ve 'p Prism.lex("/x/az").value.map {|token| token[0].type }'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
[:REGEXP_BEGIN, :STRING_CONTENT, :REGEXP_END, :IDENTIFIER, :EOF]
```

As a behavior of Ruby, when given `A` to `Z` and `a` to `z`, they act as invalid regexp options. e.g.,

```console
$ ruby -e '/regexp/az'
-e:1: unknown regexp options - az
/regexp/az
-e: compile error (SyntaxError)
```

Thus, it should probably not be construed as `IDENTIFIER` token.

Therefore, `pm_byte_table` has been adapted to accept those invalid regexp option values.
Whether it is a valid regexp option or not is checked by `pm_regular_expression_flags_create`.
For invalid regexp options, `PM_ERR_REGEXP_UNKNOWN_OPTIONS` is added to diagnostics.

d2a6096fcf
2024-03-25 12:16:32 +00:00
Justin Collins
f5a2f55aca [ruby/prism] Use Sexp#line_max not Sexp#max_line
for RubyParser translation

a37169621a
2024-03-25 12:13:52 +00:00
tomoya ishida
f53209f023 [ruby/irb] Cache RDoc::RI::Driver.new
(https://github.com/ruby/irb/pull/911)

* Cache RDoc::RI::Driver.new to improve performance and to avoid flaky test

* Insert sleep to fix flaky rendering test that renders document dialog

da84e6cb56
2024-03-25 11:48:11 +00:00
Stan Lo
5f334b67d2 [ruby/reline] Remove useless tests
(https://github.com/ruby/reline/pull/665)

The pasting tests hadn't been working since as early as v0.2.0. Since
what it tried to cover is already gone for such a long time, I think it's
better to write new ones if needed then to keep them around.

And since these tests are gone, the helper methods for just them are also gone.

0eedf0e4a0
2024-03-25 11:06:39 +00:00
Nobuyoshi Nakada
fdd7ffb70c [Bug #20389] Chilled string cannot be a shared root 2024-03-25 10:26:56 +09:00
tomoya ishida
82f4cff1f3 [ruby/irb] Fix indent test for new reline
(https://github.com/ruby/irb/pull/908)

7c16ce033e
2024-03-24 12:54:40 +00:00
tomoya ishida
b0eda83ee0 [ruby/reline] Add mode_string to prompt calculation dependencies
(https://github.com/ruby/reline/pull/658)

* Add mode_string to prompt calculation dependencies

* Update vi show-mode-in-prompt test

a0cee06ec5
2024-03-24 10:47:22 +00:00
tomoya ishida
b03705dbaf [ruby/reline] Refactor key actor test
(https://github.com/ruby/reline/pull/645)

* Add assertion assert_cursor_line to test helper

* Autofix key_actor test to use assert_cursor_line

* Rename the assertion to assert_line_around_cursor and remove other assertions for line and cursor

e4773800c6
2024-03-24 10:45:32 +00:00
tomoya ishida
3adaba0e81 [ruby/reline] Do not send color reset sequence when GeneralIO is
used
(https://github.com/ruby/reline/pull/661)

3719702808
2024-03-23 23:00:21 +00:00
Stan Lo
f46b77596d [ruby/reline] Make mutated string in yamatanooroti explicitly
mutable
(https://github.com/ruby/reline/pull/662)

This avoids the frozen literal warning in Ruby 3.4.

cccb985804
2024-03-23 22:50:57 +00:00
Nobuyoshi Nakada
8265a7531f
Use dedicated methods to abort
When `RUBY_DEBUG` is set, accessing a class in an invalid object will
cause a breakpoint trap instead of a segfault on some implementations.
2024-03-24 01:40:17 +09:00
Nobuyoshi Nakada
678cb80033
Move -test-/fatal/rb_fatal to -test-/fatal 2024-03-24 01:09:29 +09:00
Nobuyoshi Nakada
5a77397489
Ignore method chains succeeding git ls-files 2024-03-23 23:58:40 +09:00
Nobuyoshi Nakada
a8075caa4e [ruby/win32ole] Test constants for the backward compatibility
22facf50fd
2024-03-23 10:34:35 +00:00
Jeremy Evans
2dbcc123f4 Do not apply anon_rest optimization when passed array uses keyword-flagged hash
The optimization sets args->rest_dupped to avoid allocating an array,
but this is not safe if the splat array ends in a keyword flagged
hash.  Unset args->rest_dupped in this case.

Fixes [Bug #20388]
2024-03-22 16:54:07 -07:00
David Rodriguez
8ef923dc35 [rubygems/rubygems] Respect global umask when writing regular files
fd5cb7396f
2024-03-22 13:15:15 +00:00
Benoit Daloze
74995a1a77 [Feature #20275] Remove extra backtrace entries for rescue and ensure 2024-03-22 12:30:15 +01:00
yui-knk
8ba4d7d75f Fix unexpected node bug for shareable_constant_value: literal
[Bug #20339]
[Bug #20341]

`const_decl_path` changes the value of `NODE **dest`, LHS of an assignment,
with `NODE_LIT` created by `const_decl_path`. `shareable_literal_constant` calls
`const_decl_path` via `ensure_shareable_node` multiple times if RHS of an assignment
is array or hash. This means `NODE **dest` argument of `const_decl_path` can be `NODE_LIT`
from the second time then causes `[BUG] unexpected node: NODE_LIT` in
`rb_node_const_decl_val`.
This commit change to not update `NODE **dest` in `const_decl_path` to
fix the issue.
2024-03-21 11:29:09 +09:00
Kevin Newton
af7bf9e0d8 [ruby/prism] Provide options for reducing size
592128de4d
2024-03-20 17:32:03 -04:00
Jean Boussier
a008c56826 Avoid deprecation warnings in TestString 2024-03-20 12:20:02 +01:00
tomoya ishida
d7bc6f0eff [ruby/reline] Reline 0.5.0.pre
(https://github.com/ruby/reline/pull/614)

* Re-architecture LineEditor's internal state and rendering

* Fix test related to LineEditor re-architecture

* Bump to 0.5.0.pre.1

* Hide cursor only when updating screen. Frequent hide&show makes cursor flickering.

* Simplify rerender call from reline.rb

* Simplify handle_cleared

It only need to clear screen. line_editor.rerender will be called later.

* Add description of updating pasting_state inserts continuous_insertion_buffer

* Use meaningful block parameter

Co-authored-by: Stan Lo <stan001212@gmail.com>

* Fix use of `@cursor_y`

Fix bug updating `@cursor_y`. Do not use `@cursor_y` while updating dialog because it is not current cursor position but cursor position at last rendered time.

* Remove useless instance_variable_set in test

These instance variables are already removed from LineEditor

* Always initialize instance variables to avoid ruby 2.7 warning, remove unused instance variable

* Call update_dialogs from reline.rb before first render

* Combine state representing rendered screen information into `@rendered_screen`

* Rename editor_cursor_ to wrapped_cursor

It represents cursor position of word wrapped whole content

* Remove unused code, tweak, add comment

---------

3fa376217d

Co-authored-by: Stan Lo <stan001212@gmail.com>
2024-03-19 14:17:26 +00:00
Étienne Barrié
12be40ae6b Implement chilled strings
[Feature #20205]

As a path toward enabling frozen string literals by default in the future,
this commit introduce "chilled strings". From a user perspective chilled
strings pretend to be frozen, but on the first attempt to mutate them,
they lose their frozen status and emit a warning rather than to raise a
`FrozenError`.

Implementation wise, `rb_compile_option_struct.frozen_string_literal` is
no longer a boolean but a tri-state of `enabled/disabled/unset`.

When code is compiled with frozen string literals neither explictly enabled
or disabled, string literals are compiled with a new `putchilledstring`
instruction. This instruction is identical to `putstring` except it marks
the String with the `STR_CHILLED (FL_USER3)` and `FL_FREEZE` flags.

Chilled strings have the `FL_FREEZE` flag as to minimize the need to check
for chilled strings across the codebase, and to improve compatibility with
C extensions.

Notes:
  - `String#freeze`: clears the chilled flag.
  - `String#-@`: acts as if the string was mutable.
  - `String#+@`: acts as if the string was mutable.
  - `String#clone`: copies the chilled flag.

Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
2024-03-19 09:26:49 +01:00
Kevin Newton
c73dd96fe4
[PRISM] Resync 2024-03-18 09:58:53 -04:00
Nobuyoshi Nakada
0d5b16599a
[Bug #20218] Reject keyword arguments in index 2024-03-17 13:20:23 +09:00
Nobuyoshi Nakada
df5ef28233
[Bug #19918] Reject block passing in index 2024-03-17 13:18:37 +09:00
Koichi ITO
3605d6076d [ruby/prism] Fix token incompatibility for Prism::Translation::Parser::Lexer
This PR fixes token incompatibility for `Prism::Translation::Parser::Lexer` when using backquoted heredoc indetiner:

```ruby
<<-`  FOO`
a
b
     FOO
```

## Parser gem (Expected)

Returns `tXSTRING_BEG` as the first token:

```console
$ bundle exec ruby -Ilib -rparser/ruby33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Parser::Ruby33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
[s(:xstr,
  s(:str, "a\n"),
  s(:str, "b\n")), [], [[:tXSTRING_BEG, ["<<`", #<Parser::Source::Range example.rb 0...10>]],
[:tSTRING_CONTENT, ["a\n", #<Parser::Source::Range example.rb 11...13>]],
[:tSTRING_CONTENT, ["b\n", #<Parser::Source::Range example.rb 13...15>]],
[:tSTRING_END, ["  FOO", #<Parser::Source::Range example.rb 15...23>]], [:tNL, [nil, #<Parser::Source::Range example.rb 10...11>]]]]
```

## `Prism::Translation::Parser` (Actual)

Previously, the tokens returned by the Parser gem were different. The escaped backslash does not match in the `tSTRING_BEG` token and
value of `tSTRING_END` token.

```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Prism::Translation::Parser33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
[s(:xstr,
  s(:str, "a\n"),
  s(:str, "b\n")), [], [[:tSTRING_BEG, ["<<\"", #<Parser::Source::Range example.rb 0...10>]],
[:tSTRING_CONTENT, ["a\n", #<Parser::Source::Range example.rb 11...13>]],
[:tSTRING_CONTENT, ["b\n", #<Parser::Source::Range example.rb 13...15>]],
[:tSTRING_END, ["`  FOO`", #<Parser::Source::Range example.rb 15...23>]], [:tNL, [nil, #<Parser::Source::Range example.rb 10...11>]]]]
```

After this correction, the AST and tokens returned by the Parser gem are the same:

```console
$ bunlde exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = File.read("example.rb"); p Prism::Translation::Parser33.new.tokenize(buf)'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
[s(:xstr,
  s(:str, "a\n"),
  s(:str, "b\n")), [], [[:tXSTRING_BEG, ["<<`", #<Parser::Source::Range example.rb 0...10>]],
[:tSTRING_CONTENT, ["a\n", #<Parser::Source::Range example.rb 11...13>]],
[:tSTRING_CONTENT, ["b\n", #<Parser::Source::Range example.rb 13...15>]],
[:tSTRING_END, ["  FOO", #<Parser::Source::Range example.rb 15...23>]], [:tNL, [nil, #<Parser::Source::Range example.rb 10...11>]]]]
```

308f8d85a1
2024-03-16 17:55:38 +00:00
tomoya ishida
bda5b09937 [ruby/irb] Fix irb_history saved to current directory
(https://github.com/ruby/irb/pull/901)

* Always save irb_history in HOME or XDG_CONFIG_HOME

Also split irbrc search logic from irb_history search logic as a refactor

* Remove IRB.conf[:RC_NAME_GENERATOR] because it's not configurable

This conf is used to specify which irbrc to load. Need to configure before irbrc is loaded, so it's actually not configurable.
This conf is also used for history file search, but it is configurable by conf[:HISTORY_FILE].

* remove rc_file_test because it is tested with rc_files, remove useless test setup

* Make internal irbrc searching method private

11d03a6ff7
2024-03-16 15:20:03 +00:00
Nobuyoshi Nakada
aae9f5628c
Prefer the simple read/write File singleton methods 2024-03-16 23:44:07 +09:00