Commit graph

20692 commits

Author SHA1 Message Date
Kevin Newton
1be9a99837 [ruby/prism] Add a flag for arguments that contain forwarding
ebd2889bee
2024-09-11 16:35:10 +00:00
Kevin Newton
886fc69b1c [ruby/prism] Parse tempfile
31154a389a
2024-09-11 15:39:22 +00:00
Hiroshi SHIBATA
baac5376da
Use macos? helper 2024-09-11 18:13:01 +09:00
Hiroshi SHIBATA
74c3259ce7
macOS 10.13(High Sierra) is already EOL 2024-09-11 18:13:01 +09:00
ydah
d03e0d1c35 Implement BREAK, NEXT and REDO NODE locations 2024-09-11 18:01:16 +09:00
ydah
4e6091ce09 Implement WHILE and UNTIL NODE locations 2024-09-11 09:28:55 +09:00
Nobuyoshi Nakada
a79907ed5e [ruby/tmpdir] Reject empty parent path
628c5bdc59
2024-09-10 08:44:50 +00:00
Kasumi Hanazuki
3231ac6008 [ruby/resolv] test_dns: Fix FD leak
The listening TCP socket is closed by `with_udp_and_tcp` helper, but
the connected socket is leaking.

```
Leaked file descriptor: TestResolvDNS#test_multiple_servers_with_timeout_and_truncated_tcp_fallback: 12 : #<TCPSocket:fd 12, AF_INET, 127.0.0.1, 50888>
COMMAND     PID     USER   FD   TYPE    DEVICE SIZE/OFF NODE NAME
ruby    3248055 chkbuild   12u  IPv4 112546322      0t0  TCP localhost:50888->localhost:40112 (CLOSE_WAIT)
```

For the purpose of the test case to simulate a timeout over TCP
transport, we have to delay closing this socket until the end the test
case.

Fixup: https://github.com/ruby/resolv/pull/50

236c38bdb1
2024-09-10 08:34:37 +00:00
Nobuyoshi Nakada
2d12fbc4db Add predicates for platforms 2024-09-10 16:50:21 +09:00
Koichi ITO
7a65334528 [ruby/prism] Fix a token incompatibility for Prism::Translation::Parser::Lexer
This PR fixes a token incompatibility between Parser gem and `Prism::Translation::Parser` for double splat argument.

## Parser gem (Expected)

Returns `tDSTAR` token:

```console
$ bundle exec ruby -Ilib -rparser/ruby33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "def f(**foo) end"; p Parser::Ruby33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master eb144ef91e) [x86_64-darwin23]
[[:kDEF, ["def", #<Parser::Source::Range example.rb 0...3>]], [:tIDENTIFIER, ["f", #<Parser::Source::Range example.rb 4...5>]],
[:tLPAREN2, ["(", #<Parser::Source::Range example.rb 5...6>]], [:tDSTAR, ["**", #<Parser::Source::Range example.rb 6...8>]],
[:tIDENTIFIER, ["foo", #<Parser::Source::Range example.rb 8...11>]], [:tRPAREN, [")", #<Parser::Source::Range example.rb 11...12>]],
[:kEND, ["end", #<Parser::Source::Range example.rb 13...16>]]]
```

## `Prism::Translation::Parser` (Actual)

Previously, the parser returned `tPOW` token when parsing the following:

```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "def f(**foo) end"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master eb144ef91e) [x86_64-darwin23]
[[:kDEF, ["def", #<Parser::Source::Range example.rb 0...3>]], [:tIDENTIFIER, ["f", #<Parser::Source::Range example.rb 4...5>]],
[:tLPAREN2, ["(", #<Parser::Source::Range example.rb 5...6>]], [:tPOW, ["**", #<Parser::Source::Range example.rb 6...8>]],
[:tIDENTIFIER, ["foo", #<Parser::Source::Range example.rb 8...11>]], [:tRPAREN, [")", #<Parser::Source::Range example.rb 11...12>]],
[:kEND, ["end", #<Parser::Source::Range example.rb 13...16>]]]
```

After the update, the parser now returns `tDSTAR` token for the same input:

```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "def f(**foo) end"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master eb144ef91e) [x86_64-darwin23]
[[:kDEF, ["def", #<Parser::Source::Range example.rb 0...3>]], [:tIDENTIFIER, ["f", #<Parser::Source::Range example.rb 4...5>]],
[:tLPAREN2, ["(", #<Parser::Source::Range example.rb 5...6>]], [:tDSTAR, ["**", #<Parser::Source::Range example.rb 6...8>]],
[:tIDENTIFIER, ["foo", #<Parser::Source::Range example.rb 8...11>]], [:tRPAREN, [")", #<Parser::Source::Range example.rb 11...12>]],
[:kEND, ["end", #<Parser::Source::Range example.rb 13...16>]]]
```

With this change, the following code could be removed from test/prism/ruby/parser_test.rb:

```diff
-          when :tPOW
-            actual_token[0] = expected_token[0] if expected_token[0] == :tDSTAR
```

`tPOW` is the token type for the behavior of `a ** b`, and its behavior remains unchanged:

```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "a ** b"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master eb144ef91e) [x86_64-darwin23]
[[:tIDENTIFIER, ["a", #<Parser::Source::Range example.rb 0...1>]], [:tPOW, ["**", #<Parser::Source::Range example.rb 2...4>]],
[:tIDENTIFIER, ["b", #<Parser::Source::Range example.rb 5...6>]]]
```

66bde35a44
2024-09-09 19:01:30 +00:00
Peter Zhu
5a502c1873 Add keys to GC.stat and fix tests
This adds keys heap_empty_pages and heap_allocatable_slots to GC.stat.
2024-09-09 10:15:21 -04:00
Peter Zhu
b66d6e48c8 Switch sorted list of pages in the GC to a darray 2024-09-09 10:15:21 -04:00
Jean Boussier
16f241f0aa Implement String#append_as_bytes(String | Integer, ...)
[Feature #20594]

A handy method to construct a string out of multiple chunks.

Contrary to `String#concat`, it doesn't do any encoding negociation,
and simply append the content as bytes regardless of whether this
result in a broken string or not.

It's the caller responsibility to check for `String#valid_encoding?`
in cases where it's needed.

When passed integers, only the lower byte is considered, like in
`String#setbyte`.
2024-09-09 15:04:51 +02:00
zverok
d7b0f26963 Return back legacy Range#step behavior for symbol ranges 2024-09-09 17:46:13 +09:00
David Rodríguez
72e80c8f29 [rubygems/rubygems] Make gem exec use the standard GEM_HOME
032b3c518a
2024-09-09 08:46:01 +00:00
David Rodríguez
a304fe00f3 [rubygems/rubygems] Fix gem fetch always exiting with zero status code
5887e6dfa1
2024-09-09 08:44:27 +00:00
ydah
d52e599538 Implement WHEN NODE locations 2024-09-09 10:34:02 +09:00
otegami
903f3790ad [ruby/open-uri] Update error message for request_specific_fields option validation
Added `inspect` to the `request_specific_fields` value to provide
better visibility for users in the exception message.

f89ce5112d
2024-09-08 14:13:00 +00:00
otegami
0ac16215da [ruby/open-uri] Add test about request_specific_fields option
060886f312
2024-09-08 14:12:59 +00:00
Koichi ITO
4774284124 [ruby/prism] Fix a token incompatibility for Prism::Translation::Parser::Lexer
This PR fixes a token incompatibility between Parser gem and `Prism::Translation::Parser` for left parenthesis.

## Parser gem (Expected)

Returns `tLPAREN2` token:

```console
$ bundle exec ruby -Ilib -rparser/ruby33 \
-ve 'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "foo(:bar)"; p Parser::Ruby33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master eb144ef91e) [x86_64-darwin23]
[[:tIDENTIFIER, ["foo", #<Parser::Source::Range example.rb 0...3>]], [:tLPAREN2, ["(", #<Parser::Source::Range example.rb 3...4>]],
[:tSYMBOL, ["bar", #<Parser::Source::Range example.rb 4...8>]], [:tRPAREN, [")", #<Parser::Source::Range example.rb 8...9>]]]
```

## `Prism::Translation::Parser` (Actual)

Previously, the parser returned `tLPAREN` token when parsing the following:

```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "foo(:bar)"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master eb144ef91e) [x86_64-darwin23]
[[:tIDENTIFIER, ["foo", #<Parser::Source::Range example.rb 0...3>]], [:tLPAREN, ["(", #<Parser::Source::Range example.rb 3...4>]],
[:tSYMBOL, ["bar", #<Parser::Source::Range example.rb 4...8>]], [:tRPAREN, [")", #<Parser::Source::Range example.rb 8...9>]]]
```

After the update, the parser now returns `tLPAREN2` token for the same input:

```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \
'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "foo(:bar)"; p Prism::Translation::Parser33.new.tokenize(buf)[2]'
ruby 3.4.0dev (2024-09-01T11:00:13Z master eb144ef91e) [x86_64-darwin23]
[[:tIDENTIFIER, ["foo", #<Parser::Source::Range example.rb 0...3>]], [:tLPAREN2, ["(", #<Parser::Source::Range example.rb 3...4>]],
[:tSYMBOL, ["bar", #<Parser::Source::Range example.rb 4...8>]], [:tRPAREN, [")", #<Parser::Source::Range example.rb 8...9>]]]
```

The `PARENTHESIS_LEFT` token in Prism is classified as either `tLPAREN` or `tLPAREN2` in the Parser gem.
The tokens that were previously all classified as `tLPAREN` are now also classified to `tLPAREN2`.

With this change, the following code could be removed from `test/prism/ruby/parser_test.rb`:

```diff
-          when :tLPAREN
-            actual_token[0] = expected_token[0] if expected_token[0] == :tLPAREN2
```

04d6f3478d
2024-09-07 22:36:38 +00:00
Nobuyoshi Nakada
f97332a3a3
Preserve encoding in exception message of Float 2024-09-07 16:34:28 +09:00
Nobuyoshi Nakada
c1862cbb89
[Bug #20719] Float argument must be ASCII compatible 2024-09-07 16:06:14 +09:00
David Rodríguez
fe1bace43c [rubygems/rubygems] Fix gem install does-not-exist being super slow
Every time a gem is not found in the Compact Index API, RubyGems will
fallback to the full index, which is very slow. This is unnecessary
because both indexes should be providing the same gems, so if a gem
can't be found in the Compact Index API, it won't be found in the full
index.

We _do_ want a fallback to the full index, whenever the Compact Index
API is not implemented. To detect that, we check that the API responds
to the "/versions" endpoint, just like Bundler does.

Before:

```
$ time gem install fooasdsfafs
ERROR:  Could not find a valid gem 'fooasdsfafs' (>= 0) in any repository
gem  20,77s user 0,59s system 96% cpu 22,017 total
```

After:

```
$ time gem install fooasdsfafs
ERROR:  Could not find a valid gem 'fooasdsfafs' (>= 0) in any repository
gem  5,02s user 0,09s system 91% cpu 5,568 total
```

c0d6b9eea7
2024-09-06 18:44:37 +00:00
Jun Aruga
ad742de79b [ruby/openssl] Fix test_provider.rb in FIPS.
7bdbc52100
2024-09-06 16:58:18 +00:00
Nobuyoshi Nakada
30176e3f23 [rubygems/rubygems] Ensure that the lock file will be removed
2706acb271
2024-09-06 14:46:43 +00:00
Nobuyoshi Nakada
5afee4d795 [rubygems/rubygems] Remove the lock file for binstubs
https://github.com/rubygems/rubygems/pull/7806#issuecomment-2241662488

4f06ee234a
2024-09-06 14:46:42 +00:00
Nobuyoshi Nakada
214668fccb
[Feature #20707] Fix negative UTC offset conversion
In short, get rid of division and modulo of negative integers.
2024-09-06 15:53:46 +09:00
Jean Boussier
57e3fc32ea Move Time#xmlschema in core and optimize it
[Feature #20707]

Converting Time into RFC3339 / ISO8601 representation is an significant
hotspot for applications that serialize data in JSON, XML or other formats.

By moving it into core we can optimize it much further than what `strftime` will
allow.

```
compare-ruby: ruby 3.4.0dev (2024-08-29T13:11:40Z master 6b08a50a62) +YJIT [arm64-darwin23]
built-ruby: ruby 3.4.0dev (2024-08-30T13:17:32Z native-xmlschema 34041ff71f) +YJIT [arm64-darwin23]
warming up......

|                        |compare-ruby|built-ruby|
|:-----------------------|-----------:|---------:|
|time.xmlschema          |      1.087M|    5.190M|
|                        |           -|     4.78x|
|utc_time.xmlschema      |      1.464M|    6.848M|
|                        |           -|     4.68x|
|time.xmlschema(6)       |    859.960k|    4.646M|
|                        |           -|     5.40x|
|utc_time.xmlschema(6)   |      1.080M|    5.917M|
|                        |           -|     5.48x|
|time.xmlschema(9)       |    893.909k|    4.668M|
|                        |           -|     5.22x|
|utc_time.xmlschema(9)   |      1.056M|    5.707M|
|                        |           -|     5.40x|
```
2024-09-05 19:23:12 +02:00
Mari Imaizumi
be6d23646f [ruby/reline] Prevent a warning for `warning: literal string will be
frozen in the future`
(https://github.com/ruby/reline/pull/744)

69c95c8b6a
2024-09-05 12:22:34 +00:00
Étienne Barrié
bf9879791a Optimized instruction for Hash#freeze
If a Hash which is empty or only using literals is frozen, we detect
this as a peephole optimization and change the instructions to be
`opt_hash_freeze`.

[Feature #20684]

Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
2024-09-05 12:46:02 +02:00
Étienne Barrié
a99707cd9c Optimized instruction for Array#freeze
If an Array which is empty or only using literals is frozen, we detect
this as a peephole optimization and change the instructions to be
`opt_ary_freeze`.

[Feature #20684]

Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
2024-09-05 12:46:02 +02:00
Jun Aruga
2e5680d304 [ruby/openssl] Fix test_pkey_rsa.rb in FIPS.
* test_sign_verify
  I created the signature text (`signature_encoded.txt`), that is used as a
  text to create the `signature0` in the `test_sign_verify` by the following
  steps with the `openssl` CLI on FIPS module.
  ```
  $ OPENSSL_DIR="${HOME}/.local/openssl-3.4.0-dev-fips-debug-3c6e114959"
  $ export OPENSSL_CONF="${OPENSSL_DIR}/ssl/openssl_fips.cnf"

  $ echo -n "Sign me!" > data.txt
  $ "${OPENSSL_DIR}/bin/openssl" dgst -sha256 -sign test/openssl/fixtures/pkey/rsa2048.pem data.txt > signature.txt
  $ cat signature.txt | base64 > signature_encoded.txt
  ```

091f3eb421
2024-09-05 08:41:30 +00:00
ydah
32680f543c Implement AND/OR NODE operator locations 2024-09-05 13:03:28 +09:00
ydah
ab18b1b4f5 Implement VALIAS NODE keyword locations 2024-09-04 14:36:35 +09:00
Nobuyoshi Nakada
37d7ae06af
[Bug #20708] Retry open on EINTR
Co-Authored-By: Martin Dorey <martin.dorey@hds.com>
2024-09-04 10:42:48 +09:00
Jeremy Evans
08f14b8d4c
Allow Errno::EACCES when testing connection timeout
Some packaging systems that include support for running tests,
such as OpenBSD's, do not allow outbound network connections
during testing for security reasons. EACCES is the error raised by
OpenBSD in this case.
2024-09-03 18:21:01 -07:00
Durable Programming Team
675529b9c6 [rubygems/rubygems] standardize pretty-print output for Gem::Source and subclasses
6d5fbf82f1
2024-09-03 17:40:12 +00:00
tomoya ishida
f1349924df [ruby/irb] Fix easter_egg run without RDoc, fix input-method test
run without RDoc
(https://github.com/ruby/irb/pull/998)

* EasterEgg no longer depend on RDoc

* Run most of the input-method tests even if RDoc is not avialable

30fa1595d9
2024-09-03 15:45:40 +00:00
Mari Imaizumi
0889f64021 [ruby/reline] Add test for reset_variables in Reline::Config
(https://github.com/ruby/reline/pull/741)

* Fix reset variables

* Add assertion for reload

* Add helper method to get instance variable value of Reline::Config

386f619ff5
2024-09-03 15:32:29 +00:00
tomoya ishida
ad9d2c6435 [ruby/reline] Fix redisplay/insert_text called from pre_input_hook
(https://github.com/ruby/reline/pull/742)

* Fix redisplay/insert_text called from pre_input_hook

* Rename insert_pasted_text to insert_multiline_text

It is now used from Reline.insert_text which is not inserting pasted text

694a540939
2024-09-03 15:19:44 +00:00
ydah
a2243ee48b Implement ALIAS NODE keyword locations 2024-09-03 22:09:08 +09:00
ydah
af143d8a74 Implement UNDEF NODE keyword locations 2024-09-03 21:15:12 +09:00
zverok
245ed2fc89 Range#step: restore legacy behavior for String ranges 2024-09-03 16:21:42 +09:00
Étienne Barrié
f4883e7904 [flori/json] Use the compiled extension in test
148afef84c
2024-09-03 11:51:51 +09:00
Peter Zhu
1b82d63462 Fix flaky test_latest_gc_info_need_major_by
It's possible for a GC to run between the calls of GC.latest_gc_info,
which would cause the test to fail. We can disable GC so that GC only
triggers manually.
2024-09-02 18:14:48 -04:00
Nobuyoshi Nakada
c1fecc5eab [rubygems/rubygems] Simplify Gem.read_binary and Gem.write_binary
Since `Gem.open_file` no longer locks the target file and is same as
`File.open` now, simply `Gem.read_binary` should read in binary mode.
Also the body of `Gem.write_binary` is same as `File.binwrite`.

44df9045df
2024-09-02 17:28:50 +00:00
Hiroshi SHIBATA
eb144ef91e Skip show_doc tests if RDoc is not available 2024-09-01 20:00:13 +09:00
Hiroshi SHIBATA
4aa3491bd2 Skip RDoc related feature if Gem::RDoc is not available 2024-09-01 20:00:13 +09:00
Durable Programming Team
d6fc8f3d57 [rubygems/rubygems] fix @license typo preventing licenses from being correctly unmarshalled
d6ba7ef79f
2024-08-31 18:30:25 +00:00
Nobuyoshi Nakada
4a1ea9b63a [ruby/io-console] Store console IO in Ractor-local storage
Ractor requires a shareable class has shareable constants only, but IO
is not shareable unless frozen.

65e0ff895c
2024-08-31 08:43:59 +00:00