Commit graph

18546 commits

Author SHA1 Message Date
David Rodríguez
33bd956257 [rubygems/rubygems] Better approach to falling back to user installation when GEM_HOME not writable
f67bced16b
2023-12-07 13:56:22 +09:00
David Rodríguez
0f3f907e17 [rubygems/rubygems] Add some early assertions to make sure the test is progressing fine
If an error happens during the install command, it will fail in an
strange way right now.

2b6e0c703a
2023-12-07 13:56:22 +09:00
Vít Ondruch
aabf2ce181 [rubygems/rubygems] Use globals variables for standard input/output
Replace use of `STDIN`, `STDOUT` and `STDERR` constants by their
`$stdin`, `$stdout` and `$stderr` global variable equivalents.

This enables easier testing via standard means, such as `assert_output`
for minitest or `expect { print 'foo' }.to output.to_stdout` for RSpec
test suites.

a0a6fc1b76
2023-12-07 13:55:31 +09:00
Dustin Brown
d89280e8bf
Copy encoding flags when copying a regex [Bug #20039]
* 🐛 Fixes [Bug #20039](https://bugs.ruby-lang.org/issues/20039)

When a Regexp is initialized with another Regexp, we simply copy the
properties from the original. However, the flags on the original were
not being copied correctly. This caused an issue when the original had
multibyte characters and was being compared with an ASCII string.
Without the forced encoding flag (`KCODE_FIXED`) transferred on to the
new Regexp, the comparison would fail. See the included test for an
example.

Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
2023-12-06 19:25:29 -08:00
Soutaro Matsumoto
4f213ea1ba
Fix SEGV caused by GC::Profiler.raw_data (#9122) 2023-12-07 10:37:00 +09:00
Haldun Bayhantopcu
ce094e8556 [ruby/prism] Emit error for constant assignments in defs
864b06f90e
2023-12-06 22:15:16 +00:00
Kevin Newton
c5a4409f20 [ruby/prism] Simplify unterminated string
ef512ca914
2023-12-06 21:34:48 +00:00
Matt Valentine-House
c8b60c8ac2 [PRISM] Correct depth offset for block local vars
Blocks should always look at their own local table first, even when
defined inside an ensure/rescue or something else that uses depth
offset. We can ignore the depth offset if we're doing local lookups
inside a block
2023-12-06 21:12:08 +00:00
Kevin Newton
fe6ee5e921 [ruby/prism] Move flag position consistently to front
6e69a81737
2023-12-06 20:50:02 +00:00
TSUYUSATO Kitsune
cbb941f58a [ruby/prism] Fix closing loc for string literals
Fix https://github.com/ruby/prism/pull/1974

453d403593
2023-12-06 20:48:11 +00:00
Eric Mueller
a6c912a80d [rubygems/rubygems] Gem::UpdateSuggestion uses Gem::CIDetector
e5b0458342
2023-12-06 20:05:24 +00:00
Eric Mueller
b1b78c4f9f [rubygems/rubygems] Introduce the Gem::CIDetector
This is based on the list in Gem::UpdateSuggestion and Bundler::Fetcher;
these have similar purposes (determining whether/what CI we're executing
in), and can benefit from being combined and updated (they're both
slightly out of date).

Noteable changes:
* We'll consider ourselves to be on a CI in more cases - if CI_NAME or
  TASKCLUSTER_ROOT_URL are set specifically, since those represent two
  cases that were either overlooked or are no longer covered by the
  existing implementation. (Or possibly TaskCluster still does provide
  RUN_ID, but failed to document it)
* We will notice/track a few additional services in ci_strings (cirrus,
  dsari, taskcluster), stop tracking 'snap' (they went under in 2017),
  and update buildbox to buildkite (they've been called that for 8
  years, and the BUILDBOX envs have been deprecated for 3).
* We'll also sort/uniq/downcase the values (all of which only matter
  because of the special case of CI_NAME).

60652b942f
2023-12-06 20:05:23 +00:00
Kevin Newton
82f18baa21 [ruby/prism] Provide flags for changing encodings
e838eaff6f
2023-12-06 14:23:38 -05:00
Jemma Issroff
0316e666c0 [PRISM] Fix ReturnNodes
This code is almost exactly the same (fixed variable names) as
what exists already in compile.c
2023-12-06 12:25:49 -05:00
Peter Zhu
12e3b07455 Re-embed when removing Object instance variables
Objects with the same shape must always have the same "embeddedness"
(either embedded or heap allocated) because YJIT assumes so. However,
using remove_instance_variable, it's possible that some objects are
embedded and some are heap allocated because it does not re-embed heap
allocated objects.

This commit changes remove_instance_variable to re-embed Object
instance variables when it becomes small enough.
2023-12-06 11:34:07 -05:00
Jemma Issroff
f80262b14d [PRISM] Account for nil parent in Call{Operator,And,Or}PathWriteNodes
Prior to this commit, we were not accounting for the case of a nil
parent in a CallXPathWriteNode, for example ::A ||= 1. This commit
checks if the parent exists, and if not, uses Object as the inferred
parent
2023-12-06 11:31:43 -05:00
Jemma Issroff
018dbf18d5 [ruby/prism] Add locals_body_index to DefNode, BlockNode, LambdaNode
The locals_body_index gives the index in the locals array where
the locals from the body start. This allows compilers to easily
index past the parameters in the locals array.

5d4627b890
2023-12-06 09:55:48 -05:00
eileencodes
da1519b223 [PRISM] Implement PM_MATCH_PREDICATE_NODE for defined?
Ruby code:

```ruby
defined? 1 in 1
```

Instructions:

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,16)>
0000 putobject                              "expression"
0002 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,16)>
0000 putobject                              "expression"
0002 leave
```
2023-12-06 09:46:19 -05:00
eileencodes
4547108448 [PRISM] Implement PM_KEYWORD_HASH_NODE for defined?
Ruby code:

```ruby
defined? [a: [:b, :c]]
```

Instructions (without optimizations):

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,23)>
== catch table
| catch type: rescue st: 0001 ed: 0007 sp: 0000 cont: 0009
| == disasm: #<ISeq:defined guard in <compiled>@<compiled>:0 (0,0)-(-1,-1)>
| local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
| [ 1] "$!"@0
| 0000 putnil
| 0001 leave
|------------------------------------------------------------------------
0000 putnil
0001 putobject                              true
0003 branchunless                           9
0005 putobject                              "expression"
0007 swap
0008 pop
0009 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,23)>
== catch table
| catch type: rescue st: 0000 ed: 0009 sp: 0000 cont: 0009
| == disasm: #<ISeq:defined guard in <compiled>@<compiled>:0 (0,0)-(-1,-1)>
| local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
| [ 1] "$!"@0
| 0000 putnil
| 0001 leave
|------------------------------------------------------------------------
0000 putnil
0001 putobject                              true
0003 branchunless                           9
0005 putobject                              "expression"
0007 swap
0008 pop
0009 leave
```

Instructions (with optimizations):

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,23)>
== catch table
| catch type: rescue st: 0001 ed: 0003 sp: 0000 cont: 0005
| == disasm: #<ISeq:defined guard in <compiled>@<compiled>:0 (0,0)-(-1,-1)>
| local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
| [ 1] "$!"@0
| 0000 putnil
| 0001 leave
|------------------------------------------------------------------------
0000 putnil
0001 putobject                              "expression"
0003 swap
0004 pop
0005 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,23)>
== catch table
| catch type: rescue st: 0000 ed: 0005 sp: 0000 cont: 0005
| == disasm: #<ISeq:defined guard in <compiled>@<compiled>:0 (0,0)-(-1,-1)>
| local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
| [ 1] "$!"@0
| 0000 putnil
| 0001 leave
|------------------------------------------------------------------------
0000 putnil
0001 putobject                              "expression"
0003 swap
0004 pop
0005 leave
```
2023-12-06 09:46:19 -05:00
eileencodes
8649764522 [PRISM] Implement PM_SPLAT_NODE for defined?
In an array for `defined?` we need to check if there is a
`contains_splat` flag, if so bail early.

Ruby code:

```ruby
defined?([[*1..2], 3, *4..5])
```

Instructions:

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,29)>
0000 putobject                              "expression"
0002 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,29)>
0000 putobject                              "expression"
0002 leave
```
2023-12-06 09:46:19 -05:00
eileencodes
02961fdbab [PRISM] Implement PM_SOURCE_LINE_NODE for defined?
Ruby code:

```ruby
defined?(__LINE__)
```

Instructions:

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,18)>
0000 putobject                              "expression"
0002 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,18)>
0000 putobject                              "expression"
0002 leave
```
2023-12-06 09:46:19 -05:00
eileencodes
d474239537 [PRISM] Implement PM_SOURCE_FILE_NODE for defined?
Ruby code:

```ruby
defined?(__FILE__)
```

Instructions:

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,18)>
0000 putobject                              "expression"
0002 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,18)>
0000 putobject                              "expression"
0002 leave
```
2023-12-06 09:46:19 -05:00
eileencodes
41b117e50c [PRISM] Implement PM_SOURCE_ENCODING_NODE for `defined?
Ruby code:

```ruby
defined?(__ENCODING__)
```

Instructions:

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,22)>
0000 putobject                              "expression"
0002 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,22)>
0000 putobject                              "expression"
0002 leave
```
2023-12-06 09:46:19 -05:00
eileencodes
b04255deff [PRISM] Implement PM_IMAGINARY_NODE for defined?
Ruby Code:

```
defined?(1i)
```

Instructions:

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,12)>
0000 putobject                              "expression"
0002 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,12)>
0000 putobject                              "expression"
0002 leave
```
2023-12-06 09:46:19 -05:00
Matt Valentine-House
d6ec1aa0a7 [PRISM] Compile Rescue Modifier nodes 2023-12-06 07:02:04 -05:00
Haldun Bayhantopcu
34543f7c75 [ruby/prism] Update snapshot
5f0ea09785
2023-12-06 11:41:49 +00:00
Nobuyoshi Nakada
e03e872eea [ruby/rdoc] Only word-ending colon separates new definition
When followed by non-space characters, rather it looks like a URL or a
path name on Windows.

72c6560773
2023-12-06 09:35:44 +00:00
Nobuyoshi Nakada
8105ff9ac4 [ruby/rdoc] Use single quotes to keep backslash literally
4ac9be7f48
2023-12-06 09:35:43 +00:00
Jun Aruga
538ff35133 [rubygems/rubygems] test_bundled_ca.rb: Add Net::OpenTimeout as a offline case.
It seems that when DNS connection is enabled, but the TCP connection is
disabled in a way, the `Net::HTTP.connect` raises `Net::OpenTimeout`.
And I want to skip the tests in this case.

042faf74e7/lib/net/http.rb (L1601-L1608)

```
  1) Error:
TestBundledCA#test_accessing_new_index:
Net::OpenTimeout: execution expired
    /builddir/build/BUILD/ruby-3.0.4/lib/net/http.rb:987:in `initialize'
    /builddir/build/BUILD/ruby-3.0.4/lib/net/http.rb:987:in `open'
    /builddir/build/BUILD/ruby-3.0.4/lib/net/http.rb:987:in `block in connect'
    /builddir/build/BUILD/ruby-3.0.4/lib/timeout.rb:107:in `timeout'
    /builddir/build/BUILD/ruby-3.0.4/lib/net/http.rb:985:in `connect'
    /builddir/build/BUILD/ruby-3.0.4/lib/net/http.rb:970:in `do_start'
    /builddir/build/BUILD/ruby-3.0.4/lib/net/http.rb:959:in `start'
    /builddir/build/BUILD/ruby-3.0.4/lib/net/http.rb:1512:in `request'
    /builddir/build/BUILD/ruby-3.0.4/lib/net/http.rb:1270:in `get'
    /builddir/build/BUILD/ruby-3.0.4/test/rubygems/test_bundled_ca.rb:34:in `assert_https'
    /builddir/build/BUILD/ruby-3.0.4/test/rubygems/test_bundled_ca.rb:58:in `test_accessing_new_index'
...
```

5defe0a2f6
2023-12-06 02:49:01 +00:00
eileencodes
a33632e1ca [ruby/prism] Fix defined with new line
It's possible to write the following and have it be valid Ruby:

```
defined?("foo"
)
```

But Prism wasn't taking the new line into account. This adds an
`accept1` for a `PM_TOKEN_NEWLINE` to account for this. I've also
updated the fixtures and snapshots to test this.

b87f8eedc6
2023-12-05 20:47:22 +00:00
TSUYUSATO Kitsune
3260ed7e5e [ruby/prism] Add test cases
e91f8dbb99
2023-12-05 15:16:42 +00:00
TSUYUSATO Kitsune
3d4a4b2529 [ruby/prism] Fix some corner cases
d5453f168e
2023-12-05 15:16:41 +00:00
TSUYUSATO Kitsune
dcd75fd49d [ruby/prism] Fix to parse command-style method calls more correctly
Fix https://github.com/ruby/prism/pull/1468
Fix https://github.com/ruby/prism/pull/1575

To decide command-style method calls are allowed, this introduce a new
parameter `accepts_command_call` to `parse_expression` and some
functions.

Although one think this can be solved by operator precedence, it is
hard or impossible, because the precedence of command-style calls is skewed
(e.g. `! bar 1 ` is accepted, but `foo = ! bar 1` is rejected.)

One of the most complex examples is that:
(1) even though `foo = bar = baz 1` and `foo, bar = baz 1` is accepted,
(2) `foo, bar = baz = fuzz 1` is rejected.
To implement this behavior, this introduces a new binding power
`PM_BINDING_POWER_MULTI_ASSIGNMENT` and uses it for distinguish which single
assignments or multi assignments at their RHS.

d4dd49ca81
2023-12-05 15:16:40 +00:00
Matt Valentine-House
6b9622ebdf [PRISM] Implement Retry node. 2023-12-05 08:56:22 -05:00
Nobuyoshi Nakada
113f5d7fd7 [ruby/rdoc] Allow any single-word token upto 2 before C method implementation
Previously only unknown word `intern` is allowed between a single-word
token before a C method.  Now any single-word token, such as `inline`
which is used for `ArithmeticSequence` in enumerator.c, is allowed
instead.

3a214c1dd1
2023-12-05 08:48:19 +00:00
toshimaru
cda431f538 [ruby/rdoc] fix: fix NoMethodError when token_stream is nil
The change in #1055 might be a breaking change.
So, just simply wrap `token_stream` with `Array`

d8c19d7fa1

Co-authored-by: Jonathan Hefner <jonathan@hefner.pro>
2023-12-05 06:21:51 +00:00
David Rodríguez
7ab877761e [rubygems/rubygems] Instead of checking writability, try to write
Checking writability is prone to errors. For example:

$ mkdir -p foo/bar
$ chmod -w foo
$ touch foo/bar/baz # succeeds even if foo is not writable

6056138b6a
2023-12-05 14:28:41 +09:00
David Rodríguez
4a71852f75 [rubygems/rubygems] Set specs as pending at the beginning
And word them consistently with other specs.

eee7afefd3
2023-12-05 14:28:40 +09:00
David Rodríguez
a60718d107 [rubygems/rubygems] Simplify --no-user-install spec
This spec is ignoring the local `.gem` file, so most stuff is not
needed.

d52526800f
2023-12-05 14:28:40 +09:00
Hiroshi SHIBATA
84654bfbba [flori/json] cosmetics
39d6c854a4
2023-12-05 12:04:11 +09:00
Hiroshi SHIBATA
abc3d124f7 [flori/json] The modern Ruby uses utf-8 encodings by default
11b31210ac
2023-12-05 12:04:10 +09:00
tompng
70740deea7 [flori/json] Fix JSON.dump overload combination
41c2712a3b
2023-12-05 12:04:08 +09:00
Takashi Kokubun
e6b35e8a6d [flori/json] Overload kwargs in JSON.dump
936f280f9f
2023-12-05 12:04:08 +09:00
Jean Boussier
a22ed89438 [flori/json] JSON.dump: handle unenclosed hashes regression
Fix: https://github.com/flori/json/issues/553

We can never add keyword arguments to `dump` otherwise
existing code using unenclosed hash will break.

8e0076a3f2
2023-12-05 12:04:07 +09:00
Jemma Issroff
81a700853d [PRISM] Fixed redo node 2023-12-04 17:02:04 -05:00
eileencodes
7d371ca25d [PRISM] Handle percent literals for defined?
Tests all the possible percent literal with `defined?`. Implements the
missing `PM_X_STRING_NODE` for the `%x` literal.

Code:

```ruby
defined?(%x[1,2,3])
```

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,19)>
0000 putobject                              "expression"
0002 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,19)>
0000 putobject                              "expression"
0002 leave
```
2023-12-04 16:45:18 -05:00
eileencodes
09e8d37f5b [PRISM] Implement PM_INTERPOLATED_REGULAR_EXPRESSION_NODE
Implements `PM_INTERPOLATED_REGULAR_EXPRESSION_NODE` for `defined?`

Code:

```ruby
defined?(/#{1}/)
```

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,16)>
0000 putobject                              "expression"
0002 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,16)>
0000 putobject                              "expression"
0002 leave
```
2023-12-04 16:45:18 -05:00
eileencodes
33bc22efe5 [PRISM] Implement PM_INTERPOLATED_STRING_NODE
Implements `PM_INTERPOLATED_STRING_NODE` for `defined?`

Code:

```ruby
defined?("#{expr}")
```

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,19)>
0000 putobject                              "expression"
0002 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,19)>
0000 putobject                              "expression"
0002 leave
```
2023-12-04 16:45:18 -05:00
eileencodes
c23b2e5353 [PRISM] Fix PM_PARENTHESES_NODE
In #9101 I only accounted for an empty paren. This change implements the
`PM_PARENTHESES_NODE` for when it's `nil` and when it's an expression.

Code:

```ruby
defined?(("a"))
```

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,15)>
0000 putobject                              "expression"
0002 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,15)>
0000 putobject                              "expression"
0002 leave
```
2023-12-04 16:45:18 -05:00
eileencodes
569750f624 [Prism] Implement backref and numbered reference for defined?
This PR implements `PM_BACK_REFERENCE_READ_NODE` and
`PM_NUMBERED_REFERENCE_READ_NODE` for `defined?`. The following now
works:

* `PM_NUMBERED_REFERENCE_READ_NODE`

```
defined? $1
defined? $2
```

Instructions:

```
"********* RUBY *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,12)>
0000 putnil
0001 defined                                ref, :$1, "global-variable"
0005 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,12)>
0000 putnil
0001 defined                                ref, :$1, "global-variable"
0005 leave
```

* `PM_BACK_REFERENCE_READ_NODE`

```
defined? $'
defined? $`
defined? $&
```

Instructions:

```
"********* RUBY *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,12)>
0000 putnil
0001 defined                                ref, :$`, "global-variable"
0005 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,12)>
0000 putnil
0001 defined                                ref, :$`, "global-variable"
0005 leave
```
2023-12-04 12:03:22 -08:00