Commit graph

14675 commits

Author SHA1 Message Date
David Rodríguez
ee2c7bcae5 [rubygems/rubygems] Allow noop bundle install to work on read-only or protected folders
As long as there's nothing new to install and gems are already there.

If not, give a meaningful error about what happened.

This was how things already worked until
345ec45f5a, so this commit partially
reverts that change.

794b0ecb39
2025-02-21 11:31:49 +09:00
Hiroshi SHIBATA
425cfcff85 [rubygems/rubygems] bin/rubocop -a --only Style/MultipleComparison
677e17aa2e
2025-02-21 11:31:47 +09:00
David Rodríguez
cdbadafd25 [rubygems/rubygems] Fix resolver issue due to ill-defined version ranges being created
f2f2ac9680
2025-02-21 11:31:44 +09:00
Hiroshi SHIBATA
556be73cbb [rubygems/rubygems] gem server command is extracted from rubygems core feature now
b29f8d6a37
2025-02-20 12:58:45 +09:00
David Rodríguez
63600d79dc [rubygems/rubygems] Fix lockfile platforms inconveniently added on JRuby
When working with our repository on JRuby locally, I get the following
changes when running `bin/rake setup` in all of our lockfiles

```diff
diff --git a/tool/bundler/dev_gems.rb.lock b/tool/bundler/dev_gems.rb.lock
index 74550b2a40 100644
--- a/tool/bundler/dev_gems.rb.lock
+++ b/tool/bundler/dev_gems.rb.lock
@@ -66,6 +66,7 @@ PLATFORMS
   java
   ruby
   universal-java
+  universal-java-22
   x64-mingw-ucrt
   x86-linux
   x86_64-darwin
```

This is inconvenient, so I applied the same strategy we already use on
non JRuby implementations to not add the current platform to the
lockfile if a less specific platform is already there.

812b9cd1e8
2025-02-20 12:58:44 +09:00
David Rodríguez
5d8fe7a595
[rubygems/rubygems] Make sure empty gems are not reinstalled every time
Unfortunately this requires reverting a previous enhancement of
reinstalling gems if they incorrectly ship with an empty installation
dir.

However, there's no way to distinguish this kind of bad state from a gem
that's empty for real, for example, sorbet-static-and-runtime.

This reverts commit 9720a9b980, and adds a
spec to make sure empty gems are not reinstalled every time.

7c102394af
2025-02-19 11:04:09 +09:00
David Rodríguez
507de2226b [rubygems/rubygems] Fix Bundler incorrectly downgrading direct dependencies
There's no reason to call `converge_specs` when adding additional
lower bound requirements to prevent downgrades, and it actually causes
the extra requirements to be missed sometimes.

Loop over the originally locked specs directly, adding the additional
precaution of not adding the requirement if the Gemfile dependency has
changed and it no longer matches the locked spec.

5154506912
2025-02-18 12:12:54 +09:00
David Rodríguez
c5cdabc14a [rubygems/rubygems] Don't try to skip requirements to prevent downgrades
These don't really hurt, so I'm not sure why I introduced it.

85b6b405ac
2025-02-18 12:12:52 +09:00
David Rodríguez
c77354157f [rubygems/rubygems] Fix locked gems being upgraded when locked dependencies are incorrect
Resolver had internal logic to prioritize locked versions when sorting
versions, however part of it was not being actually hit because of how
unlocking worked in the resolver: a package was allow to be unlocked
when that was explicit requested or when the list of unlocks was empty.
That did not make a lot of sense and other cases were working because
the explicit list of unlocks was getting "artificially filled".

Now we consider a package unlocked when explicitly requested (`bundle
update <package>`), or when everything is being unlocked (`bundle
install` with no lockfile or `bundle update`).

This makes things simpler and gets the edge case added as a test case
working as expected.

b8e55087f0
2025-02-18 12:12:51 +09:00
David Rodríguez
592eb02d6c [rubygems/rubygems] Refactor finding dependency changes
d8c4754d8f
2025-02-18 12:12:49 +09:00
David Rodríguez
533e894cb5 [rubygems/rubygems] Improve processing and categorizing unlock information
516430c3ec
2025-02-18 12:12:48 +09:00
David Rodríguez
c8f4df4ae9 [rubygems/rubygems] Make Bundler never instantiate development dependencies
Bundler does not really have a concept of "development dependencies",
like RubyGems has. Bundler has the more generic concept of "groups".

Under the hood, the `gemspec` DSL will put gemspec development
dependencies under a `:development` Gemfile group, but there's no reason
to instantiate these as development dependencies, they are regular
runtime dependencies, except that they belong in a group named
:development.

By never instantiating development dependencies at all, we avoid having
to introduce hacks to "undo" the type Bundler does not know about, and I
also think the error messages read better.

9a06fa5bda
2025-02-17 11:04:39 +09:00
David Rodríguez
a55a2fc6e8
[rubygems/rubygems] Reset existing specs when using Bundler::SpecSet#[]=
We have a flaky failure where to equal Bundler specs sneak into the same
SpecSet. This seems like a vector where that could happen so trying this
in case it fixes the flaky.

a33aeb3c4d
2025-02-14 16:13:27 +09:00
David Rodríguez
a64c697b22
[rubygems/rubygems] Refactor SpecSet to not need reset
55af558124
2025-02-14 16:13:27 +09:00
David Rodríguez
061d8133ce
[rubygems/rubygems] Simplify hacks to integrate with RubyGems
33d91de732
2025-02-14 16:13:27 +09:00
David Rodríguez
c94cec27f7
[rubygems/rubygems] Simplify SpecSet#sorted
a3f365bbaa
2025-02-14 16:13:27 +09:00
David Rodríguez
e59c90118e
[rubygems/rubygems] Raise error when lockfile is missing deps in frozen mode
And avoid installing any gems.

c12700c7e4
2025-02-14 16:13:27 +09:00
johnnyshields
e11401f799
[rubygems/rubygems] Deprecate legacy windows platforms (mswin, mingw) in the Bundler DSL, in favor of using platform :windows
This commit is only deprecation and does not change/remove any actual functionality.

0ca6dc3984
2025-02-14 16:13:27 +09:00
David Rodríguez
181cab566c
[rubygems/rubygems] Fix legacy windows platform values no longer working
de8b3016db
2025-02-14 16:13:26 +09:00
Kevin Newton
127325a4ba [ruby/prism] No writing to numbered parameters
Fixes [Bug #21117]

19d4bab5a0
2025-02-13 20:04:02 +00:00
Nobuyoshi Nakada
4a67ef09cc
[Feature #21116] Extract RJIT as a third-party gem 2025-02-13 18:01:03 +09:00
David Rodríguez
ee03df26ba
[rubygems/rubygems] --prefer-local should resolve to latest version if no gems are available locally
Filtering out remote specs should only apply where there are locally
installed specs. Otherwise they should always be considered.

118f8389a1
2025-02-13 09:37:51 +09:00
David Rodríguez
151b436c9d
[rubygems/rubygems] Fix incorrect error message in frozen mode
When Bundler refuses to install in frozen mode, sometimes it would
incorrectly claim that some dependencies have been added to the Gemfile
when that's not really the case. Fix that by making sure
`locked_dependencies` always has all locked dependencies, even when
unlocking,

Additionally, the suggestion to run `bundle install` is also confusing
when unlocking, since `bundle update` is what has been run. So skip that
part as well when unlocking.

64d84ad7d8
2025-02-13 09:37:51 +09:00
David Rodríguez
d9267b4a71
[rubygems/rubygems] Fix type and gemfile not getting reported as invalid options
5b6077a1e8
2025-02-13 09:37:51 +09:00
David Rodríguez
e7720ef8d7
[rubygems/rubygems] Materializing specs for vendor/cache should not be strict
Platforms specific gems not compatible with the current Ruby should not
make `bundle cache` fail and should not get removed from the cache since
they still may be useful in other rubies.

717b43f565
2025-02-13 09:37:50 +09:00
David Rodríguez
5adbad731b
[rubygems/rubygems] Move logic to materialize gems for cache to a new method
And make the current `materialize_strictly` private.

3fc2129147
2025-02-13 09:37:50 +09:00
David Rodríguez
06e3943c38
[rubygems/rubygems] Remove unnecessary source change
We materializing for installation, we already do this, and we
materializing for cache, it does not seem necessary.

1a983c6cbc
2025-02-13 09:37:50 +09:00
David Rodríguez
bb764e42ba
[rubygems/rubygems] Extract some common materialization logic to a method
32982fcd33
2025-02-13 09:37:50 +09:00
David Rodríguez
7ac29372ca
[rubygems/rubygems] Make LazySpecification#__materialize__ private
And rename it to something better.

321174d1ad
2025-02-13 09:37:50 +09:00
David Rodríguez
135479a58d [rubygems/rubygems] Allocate strings from Requirement match only once
c65b8644e6

Co-authored-by:  Samuel Giddins <segiddins@segiddins.me>
2025-02-10 09:27:18 +09:00
yuuji.yaginuma
dd1fe03b8a [rubygems/rubygems] Add irb to a Gemfile for a newly created gem
I think we need this to silence the following warning when running
`bin/console` with Ruby 3.4

```
./bin/console:10: warning: irb was loaded from the standard library, but will no longer be part of the default gems starting from Ruby 3.5.0.
You can add irb to your Gemfile or gemspec to silence this warning.
```

c46230c856
2025-02-06 15:58:01 +09:00
David Rodríguez
ac093f4350 [rubygems/rubygems] Auto-heal empty installation directory
9720a9b980
2025-02-06 15:58:00 +09:00
David Rodríguez
78ef59acf7 [rubygems/rubygems] Refine messages about gem installations being missing
The previous wording was too specific, there may be situations when the
gem has actually never installed (so never deleted either).

e4a0d71fbe
2025-02-06 15:57:59 +09:00
David Rodríguez
385dc5dc16 [rubygems/rubygems] Don't potentially load remote metadata when expanding dependencies
For installed specifications, we can ignore any constraints they may
have, since we know they match the current version of Ruby or otherwise
would not be installed.

For remote specifications, we already resolve optimistically without
metadata and retry force-fetching it if necessary.

If in the future we support resolving against a Ruby version different
that the one being run, we'll probably need to change this but now it's
unnecessary and saves some memory.

### Before

Total allocated: 262.99 MB (3177437 objects)
Total retained:  115.91 MB (1297821 objects)

### After

Total allocated: 259.89 MB (3134199 objects)
Total retained:  115.05 MB (1283779 objects)

201c1863fc
2025-02-06 15:57:57 +09:00
David Rodríguez
a1716e2365 [rubygems/rubygems] Move expanding dependencies with metadata to specification classes
7f921aa46e
2025-02-06 15:57:57 +09:00
David Rodríguez
24f5e3010f [rubygems/rubygems] Metadata dependencies can be Gem::Dependency instances
They use less memory that way.

When resolving from scratch a Gemfile including only `"gem "rails", "~>
8.0.1"`, I get the following results:

### Before

Total allocated: 265.06 MB (3186053 objects)
Total retained:  116.98 MB (1302280 objects)

### After

Total allocated: 262.99 MB (3177437 objects)
Total retained:  115.91 MB (1297821 objects)

a4ef9c5f56
2025-02-06 15:57:56 +09:00
David Rodríguez
7fed6c887d [rubygems/rubygems] Remove unnecessary remapping of dependencies
Sometimes we'll resolve using bare `Gem::Dependency` instances rather
than `Bundler::Dependency` instances, which is fine, simpler, and saves
some memory.

When resolving from scratch a Gemfile including only `"gem "rails", "~>
8.0.1"`, I get the following results:

### Before

Total allocated: 277.48 MB (3384318 objects)
Total retained:  117.53 MB (1338657 objects)

### After

Total allocated: 265.06 MB (3186053 objects)
Total retained:  116.98 MB (1302280 objects)

c6dc2966c5
2025-02-06 15:57:55 +09:00
David Rodríguez
4c0cf2deed [rubygems/rubygems] Make Bundler::Dependency more memory efficient
When resolving from scratch a Gemfile including only `"gem "rails", "~>
8.0.1"`, I get the following results:

### Before

Total allocated: 288.21 MB (3498515 objects)
Total retained:  119.10 MB (1357976 objects)

### After

Total allocated: 277.44 MB (3383182 objects)
Total retained:  117.55 MB (1338622 objects)

2d3d6e5015
2025-02-06 15:57:55 +09:00
David Rodríguez
8e7883011a [rubygems/rubygems] Lazily parse dependencies in EndpointSpecification
Since not every dependency gets referenced.

When resolving from scratch a Gemfile including only `"gem "rails", "~>
8.0.1"`, I get the following results:

### Before

Total allocated: 295.01 MB (3624335 objects)
Total retained:  119.31 MB (1364474 objects)

### After
Total allocated: 288.21 MB (3498515 objects)
Total retained:  119.10 MB (1357976 objects)

61eee39d81

Co-authored-by: Samuel Giddins <segiddins@segiddins.me>
2025-02-06 15:57:54 +09:00
David Rodríguez
c83370671b Improve bundled gems warning messages
Currently evenn if the require actually fails, they suggest that the
file was actually loaded, which is confusing. I reworded them to reduce
this confusion.
2025-02-06 13:07:55 +09:00
David Rodríguez
03a0c4e079 Rename "gem" to "name"
The name "gem" could be confused with RubyGems activation method.
2025-02-06 13:07:55 +09:00
David Rodríguez
433f4e30b3 Simplify bundled gems warnings implementation
Most of the stuff is not actually necessary.
2025-02-06 13:07:55 +09:00
David Rodríguez
68bb6ceeaf Remove unnecessary SINCE_FAST_PATH constant
If anything, I think this may be causing some false positives.
2025-02-06 13:07:55 +09:00
Edouard CHIN
c204cf7cb1 [rubygems/rubygems] Deprecate CurrentRuby#maglev? and other related maglev methods:
- Follow up to https://github.com/rubygems/rubygems/pull/8430#discussion_r1927239555.

  The maglev platform was not supported by Bundler, so calling
  `gem "foo", platforms: ["maglev"]` would raise an error.

  The helpers added in the `CurrentRuby` class were used at a time
  when maglev was supported (as explained in 45ec86e2e5).
  Support of maglev was most likely dropped at some point and the helpers
  in the `CurrentRuby` class were not deprecated/removed.

  We decided to deprecate them now.

66388babf8
2025-02-05 12:48:44 +09:00
Hiroshi SHIBATA
470784cbd9 Expand stub-out scope of Fiddle.dlopen 2025-02-05 07:36:59 +09:00
Hiroshi SHIBATA
10d06b9a72 [ruby/resolv] Load win32/resolv with rake test
3ecfce3626
2025-02-04 04:55:16 +00:00
Nikita Shilnikov
e8cf441485 [ruby/weakref] Add missing block parameter
A block is part of the Delegator's contract. Ruby 3.4 issues a warning if a block is passed but unused. This commit fixes the warning by adding a block to the argument list.

9495ec9191
2025-02-03 01:58:50 +00:00
David Rodríguez
c0688c21fe [rubygems/rubygems] Raise a simpler error when RubyGems fails to activate a dependency
If you force uninstall a dependency but leave other gems depending on
it, those gems will fail to be activated.

In that case, RubyGems prints a rather complicated error:

```
$ rails --version
/Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems/specification.rb:1413:in 'block in Gem::Specification#activate_dependencies': Could not find 'activesupport' (= 8.0.1) among 478 total gem(s) (Gem::MissingSpecError)
Checked in 'GEM_PATH=/Users/deivid/.local/share/gem/ruby/3.4.0:/Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/gems/3.4.0' at: /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/gems/3.4.0/specifications/railties-8.0.1.gemspec, execute `gem env` for more information
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems/specification.rb:1399:in 'Array#each'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems/specification.rb:1399:in 'Gem::Specification#activate_dependencies'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems/specification.rb:1381:in 'Gem::Specification#activate'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems.rb:283:in 'block in Gem.activate_bin_path'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems.rb:282:in 'Thread::Mutex#synchronize'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems.rb:282:in 'Gem.activate_bin_path'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/bin/rails:25:in '<main>'
/Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems/dependency.rb:303:in 'Gem::Dependency#to_specs': Could not find 'activesupport' (= 8.0.1) - did find: [activesupport-7.1.3,activesupport-7.0.8.7] (Gem::MissingSpecVersionError)
Checked in 'GEM_PATH=/Users/deivid/.local/share/gem/ruby/3.4.0:/Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/gems/3.4.0' , execute `gem env` for more information
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems/specification.rb:1411:in 'block in Gem::Specification#activate_dependencies'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems/specification.rb:1399:in 'Array#each'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems/specification.rb:1399:in 'Gem::Specification#activate_dependencies'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems/specification.rb:1381:in 'Gem::Specification#activate'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems.rb:283:in 'block in Gem.activate_bin_path'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems.rb:282:in 'Thread::Mutex#synchronize'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems.rb:282:in 'Gem.activate_bin_path'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/bin/rails:25:in '<main>'
```

With this commit, the error becomes a bit simpler to parse:

```
$ rails --version
/Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems/specification.rb:1421:in 'block in Gem::Specification#activate_dependencies': Could not find 'activesupport' (= 8.0.1) among 478 total gem(s) (Gem::MissingSpecError)
Checked in 'GEM_PATH=/Users/deivid/.local/share/gem/ruby/3.4.0:/Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/gems/3.4.0' at: /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/gems/3.4.0/specifications/railties-8.0.1.gemspec, execute `gem env` for more information
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems/specification.rb:1407:in 'Array#each'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems/specification.rb:1407:in 'Gem::Specification#activate_dependencies'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems/specification.rb:1389:in 'Gem::Specification#activate'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems.rb:283:in 'block in Gem.activate_bin_path'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems.rb:282:in 'Thread::Mutex#synchronize'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/lib/ruby/site_ruby/3.4.0/rubygems.rb:282:in 'Gem.activate_bin_path'
	from /Users/deivid/.asdf/installs/ruby/3.4.1/bin/rails:25:in '<main>'
```

And also, we reduce exception based control flow in our code.

7e48c49f2d
2025-02-03 10:05:34 +09:00
David Rodríguez
451d848a76
Stop generating binstubs for Bundler itself 2025-01-31 14:34:55 +09:00
David Rodríguez
56e2ef2468 [rubygems/rubygems] Fix bundle console unnecessarily trying to load IRB twice
f9bf58573f
2025-01-31 14:34:29 +09:00