Commit graph

149 commits

Author SHA1 Message Date
David Rodríguez
18ac783ea6 [rubygems/rubygems] Revert adding loaded specs to Gem::Specification.stubs and Gem::Specification.stubs_for
The rationale is that:

* The change has caused realworld issues. See for example
https://github.com/ruby/did_you_mean/issues/117 and specifically [this
comment](https://github.com/ruby/did_you_mean/issues/117#issuecomment-482733159)
for a great explanation of the issue it caused for `did_you_mean`.

* The change also causes problems for our development workflows. For
example, because of it, our `bundler` specs cannot currently be run with
`bin/rake` and we have to use `bin/rspec` or `bin/parallel_spec`
directly. The explanation for this is:

  - Our specs install test dependencies to `tmp` before running specs.
  - `rake` is one of these test dependencies.
  - Before installing each test dependency, we check whether it has
  matching installed specs: 2bbcdcde08/bundler/spec/support/rubygems_ext.rb (L109-L114).

  - Normally, if `rake` has not yet been installed to `tmp`, this check
  fails and `rake` is installed, but since the loaded specs are now
  added to `Gem::Specification.stubs` and `rake`'s specification _is_
  loaded because we're running through `bin/rake`, the check incorrectly
  assumes that `rake` is already installed to `tmp` and skips
  installation.
  - At a later point the specs check whether `rake` is actually
  installed and fail if it's not: 2bbcdcde08/bundler/spec/support/builders.rb (L372-L383)

Essentially, both of the issues are the same. If at runtime we change
the location of gems, we'll _want_ to not consider loaded specifications
when dealing with the new gem location, because the loaded
specifications have not been loaded from there. Loaded specifications is
something different from installed stub specifications and those should
not be mixed.

The PR still seemed to have fixed an issue, so I did my archaeology job
and investigated the original issue to double check if reverting is ok.
The logs for the original error can be found here:
https://ci.appveyor.com/project/rubygems/rubygems/build/1172/job/ogubyucpljcv22ux.

So I installed ruby 2.4.4, checked out the commit reference before the
offending PR, and the exact error reproduced. 🎉

```
$ rake test
/home/deivid/Code/rubygems/lib/rubygems/resolver.rb:231:in `search_for': Unable to resolve dependency: user requested 'bundler (= 1.16.2)' (Gem::UnsatisfiableDependencyError)
  from /home/deivid/Code/rubygems/lib/rubygems/resolver.rb:283:in `block in sort_dependencies'
  from /home/deivid/Code/rubygems/lib/rubygems/resolver.rb:277:in `each'
  from /home/deivid/Code/rubygems/lib/rubygems/resolver.rb:277:in `sort_by'
  from /home/deivid/Code/rubygems/lib/rubygems/resolver.rb:277:in `with_index'
  from /home/deivid/Code/rubygems/lib/rubygems/resolver.rb:277:in `sort_dependencies'
  from /home/deivid/Code/rubygems/lib/rubygems/resolver/molinillo/lib/molinillo/delegates/specification_provider.rb:52:in `block in sort_dependencies'
  from /home/deivid/Code/rubygems/lib/rubygems/resolver/molinillo/lib/molinillo/delegates/specification_provider.rb:69:in `with_no_such_dependency_error_handling'
  from /home/deivid/Code/rubygems/lib/rubygems/resolver/molinillo/lib/molinillo/delegates/specification_provider.rb:51:in `sort_dependencies'
  from /home/deivid/Code/rubygems/lib/rubygems/resolver/molinillo/lib/molinillo/resolution.rb:165:in `initial_state'
  from /home/deivid/Code/rubygems/lib/rubygems/resolver/molinillo/lib/molinillo/resolution.rb:106:in `start_resolution'
  from /home/deivid/Code/rubygems/lib/rubygems/resolver/molinillo/lib/molinillo/resolution.rb:64:in `resolve'
  from /home/deivid/Code/rubygems/lib/rubygems/resolver/molinillo/lib/molinillo/resolver.rb:42:in `resolve'
  from /home/deivid/Code/rubygems/lib/rubygems/resolver.rb:188:in `resolve'
  from /home/deivid/Code/rubygems/lib/rubygems/request_set.rb:396:in `resolve'
  from /home/deivid/Code/rubygems/lib/rubygems/request_set.rb:408:in `resolve_current'
  from /home/deivid/Code/rubygems/lib/rubygems.rb:243:in `finish_resolve'
  from /home/deivid/Code/rubygems/lib/rubygems/rdoc.rb:13:in `<top (required)>'
  from /home/deivid/Code/rubygems/lib/rubygems/core_ext/kernel_require.rb:54:in `require'
  from /home/deivid/Code/rubygems/lib/rubygems/core_ext/kernel_require.rb:54:in `require'
  from /home/deivid/Code/rubygems/lib/rubygems/test_case.rb:1563:in `<top (required)>'
  from /home/deivid/Code/rubygems/test/rubygems/test_bundled_ca.rb:2:in `require'
  from /home/deivid/Code/rubygems/test/rubygems/test_bundled_ca.rb:2:in `<top (required)>'
  from /home/deivid/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/rake_test_loader.rb:15:in `require'
  from /home/deivid/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/rake_test_loader.rb:15:in `block in <main>'
  from /home/deivid/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/rake_test_loader.rb:4:in `select'
  from /home/deivid/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/rake_test_loader.rb:4:in `<main>'
rake aborted!
Command failed with status (1)

Tasks: TOP => test
```

Now the explanation of the error:

* Rubygems base `TestCase` class requires `bundler` because some tests
use `bundler`:
2bbcdcde08/lib/rubygems/test_case.rb (L26)

* That `require` (our custom rubygems require) would activate the
default bundler spec (1.16.1 for ruby 2.4.4) but then overwrite it with
a 1.16.2 version (the locally provided bundler those days) due to [this
old
hack](9f7bf0ac3a/lib/bundler/version.rb (L7-L23)).

* Rubygems base `TestCase` class requires `rubygems/rdoc`:
2bbcdcde08/lib/rubygems/test_case.rb (L1536)

* And that file ends up calling `Gem.finish_resolve`:
2bbcdcde08/lib/rubygems/rdoc.rb (L13)

* `Gem.finish_resolve` adds the currently loaded specs to the
resolution:
2bbcdcde08/lib/rubygems.rb (L235)

* That means it would try to resolve bundler 1.16.2, but no
specification for that version was installed since the default was
1.16.1. That explains why upgrading to rubygems 2.7.7 fixed the issue,
since it provided bundler 1.16.2 by default so there was not bundler
version discrepancy.

After understanding the error, I conclude that:

* Only this part of the original patch was actually needed to resolve
the error, not any of the changes in `Gem::Specification.stubs` and
`Gem::Specification.stubs_for`:

```diff
diff --git a/lib/rubygems/test_case.rb b/lib/rubygems/test_case.rb
index f1cd3d274c..92c848e870 100644
--- a/lib/rubygems/test_case.rb
+++ b/lib/rubygems/test_case.rb
@@ -13,6 +13,15 @@ else
   require 'rubygems'
 end

+# If bundler gemspec exists, add to stubs
+bundler_gemspec = File.expand_path("../../../bundler/bundler.gemspec", __FILE__)
+if File.exist?(bundler_gemspec)
+  Gem::Specification.dirs.unshift File.dirname(bundler_gemspec)
+  Gem::Specification.class_variable_set :@@stubs, nil
+  Gem::Specification.stubs
+  Gem::Specification.dirs.shift
+end
+
 begin
   gem 'minitest'
 rescue Gem::LoadError
```

So, I propose to revert adding loaded specification to
`Gem::Specification.stubs` and `Gem::Specification.stubs_for` because I
think it's safe, it fixes the issues caused by their addition, and it
simplifies `Gem::Specification` code, which is already complicated
enough.

5269cd617c
2020-05-08 14:13:29 +09:00
Vít Ondruch
acc86570dd [rubygems/rubygems] Let @@stubs_by_name to be incrementally populated again.
Originally, the call to `.stubs_for` allowed to incrementally populate
the `@@stubs_by_name` (especially see the `"#{name}-*.gemspec"` pattern
in 4fa03bb7aac9f25f44394e818433fdda9962ae8d). Now it looks like it
expects that all stubs are loaded, but the `.stubs_for` still matches
the .gemspec files by the `name` pattern:

6d45e0f7ac/lib/rubygems/specification.rb (L845)

I think this was done by mistake incrementally by PR #1239 and
4cee8ca9199ac7b3ab8647e0b78615f55d3eb02b. I think the best option is to
get back to the original implementation, to let RubyGems incrementally
populate the array. Other option would be to replace the logic in
`.stub_for` by call to `.stubs`, but the means the performance
improvement from the original commit was lost.

4d0e18185a
2020-05-08 14:13:29 +09:00
David Rodríguez
314d7f897c Remove builder gem requirement for gem regenerate_index
This requirement was introduced 14 years ago in
7ce7039b390440754954df5efea619e9f57ef823, and I don't think it's
necessary anymore. I made several tests introducing UTF-8 characters in
gemspec files and generating indexes out of them, and couldn't find any
issues. Gemspecs are read with UTF-8 encoding these days.
2020-05-08 14:13:29 +09:00
bronzdoc
7fb694fdb0 [rubygems/rubygems] Show gemspec location when a Gem::MissingSpecError is raised while trying to activate a gem
4da54a8e8b
2020-05-08 14:13:29 +09:00
David Rodríguez
2205ae8af5 Normalize heredoc delimiters 2020-05-08 07:38:50 +09:00
Josef Šimánek
39322317f3 [rubygems/rubygems] Do not use set.
1a72c8796f
2020-05-08 07:38:50 +09:00
Josef Šimánek
856cbbdd52 [rubygems/rubygems] Track removed methods calls and warn during build time.
move rubyforge_project= to removed methods

223f7fd470
2020-05-08 07:38:50 +09:00
Josef Šimánek
4dd46dbad0 [rubygems/rubygems] Deprecate rubyforge_project attribute only during build time.
c44eb21648
2020-05-08 07:38:50 +09:00
bronzdoc
7db538a7c9 [rubygems/rubygems] Rename version horizon deprecation methods
6afd914fda
2020-05-08 07:38:50 +09:00
bronzdoc
ff37dd7e9e [rubygems/rubygems] Modify files to use new version horizon deprecations
4fe5bb5bf3
2020-05-08 07:38:50 +09:00
David Rodríguez
061add792e
[rubygems/rubygems] Enable Style/ExtraSpacing and auto-correct
6fa0b1b679
2020-03-30 12:48:23 +09:00
David Rodríguez
ba9dcdab36
[rubygems/rubygems] Enable Style/PercentLiteralDelimiters cop in rubygems
So it matches the style used by bundler.

ab0580fd65
2020-03-30 12:42:10 +09:00
David Rodríguez
96064e6f1c
Sync rubygems with current master (#2889) 2020-03-24 15:39:24 +09:00
Hiroshi SHIBATA
600a715c9b
Merge the current master branch of rubygems/rubygems.
Just started to develop RubyGems 3.2.0.
2020-02-01 11:14:57 +09:00
Hiroshi SHIBATA
7d463e360b Merge RubyGems 3.1.0.pre3
* Fix gem pristine not accounting for user installed gems. Pull request
    #2914 by Luis Sagastume.
  * Refactor keyword argument test for Ruby 2.7. Pull request #2947 by
    SHIBATA Hiroshi.
  * Fix errors at frozen Gem::Version. Pull request #2949 by Nobuyoshi
    Nakada.
  * Remove taint usage on Ruby 2.7+. Pull request #2951 by Jeremy Evans.
  * Check Manifest.txt is up to date. Pull request #2953 by David Rodríguez.
  * Clarify symlink conditionals in tests. Pull request #2962 by David
    Rodríguez.
  * Update command line parsing to work under ps. Pull request #2966 by
    David Rodríguez.
  * Properly test `Gem::Specifications.stub_for`. Pull request #2970 by
    David Rodríguez.
  * Fix Gem::LOADED_SPECS_MUTEX handling for recursive locking. Pull request
    #2985 by MSP-Greg.
2019-11-11 16:59:49 +09:00
Nobuyoshi Nakada
c4ec817a7a
Gem::Specification#to_ruby needs OpenSSL 2019-09-26 22:56:32 +09:00
bronzdoc
5c872b297c
[rubygems/rubygems] Avoid adding OpenSSL::PKey::RSA instances
ba021fb4be
2019-09-26 17:48:01 +09:00
bronzdoc
2e9c078707
[rubygems/rubygems] Fix indentation in case statement
8ac0647659
2019-09-26 17:48:00 +09:00
bronzdoc
8f2379b0c5
[rubygems/rubygems] Make ruby_code method handle OpenSSL::PKey::RSA objects
b1d825ab3a
2019-09-26 17:48:00 +09:00
Ellen Marie Dash
508afe2c26
[rubygems/rubygems] Set SOURCE_DATE_EPOCH env var if not provided.
Fixes #2290.

1. `Gem::Specification.date` returns SOURCE_DATE_EPOCH when defined,
2. this commit makes RubyGems set it _persistently_ when not provided.

This combination means that you can build a gem, check the build time,
and use that value to generate a new build -- and then verify they're
the same.

d830d53f59
2019-09-26 17:48:00 +09:00
David Rodríguez
d9e6315177
[rubygems/rubygems] Bump rubocop to 0.74.0 and fix new offenses
d4fc383497
2019-09-05 18:48:15 +09:00
bronzdoc
87bc29fe28 [rubygems/rubygems] Revert cadb66037d9b58c80fc795f39384d533229a1f73
5c3158d975
2019-08-05 18:44:53 +09:00
Vít Ondruch
a3b784b3a0
[rubygems/rubygems] Move default specifications dir definition out of BasicSpecification.
This was never the right place. The method got there just by evolution,
not by design. Move it within default methods, where it suits better.

Since this method is presumably used just internally, it should be safe
to deprecate it and remove later.

0c0dd9458a
2019-07-31 07:47:02 +08:00
bronzdoc
a34168f2ac
[rubygems/rubygems] Autorequire was used by old RubyGems, it is neither supported nor functional.
cadb66037d
2019-07-31 07:47:02 +08:00
Nobuyoshi Nakada
a2f0331c4e
[rubygems/rubygems] Simplify #to_ruby code
Separate #add_runtime_dependency and #add_development_dependency
availability condition from #specification_version availability,
which is not related to directly.
Also check if the former method is available, instead of comparing
the version.

5cccc2b836
2019-07-31 07:47:02 +08:00
bronzdoc
60d3f9ef9f
[rubygems/rubygems] Remove useless TODO comment
44bc809dc8
2019-07-31 07:47:01 +08:00
bronzdoc
b2a54744f2
[rubygems/rubygems] check_version_conflict don't have a 'dep' variable available for use
1783cf0fd3
2019-07-31 07:47:01 +08:00
David Rodríguez
df0b41fc72
[rubygems/rubygems] Explicitly deprecate rubyforge_project
9094740109
2019-07-31 07:47:01 +08:00
Benoit Daloze
05ae2c11fa
[rubygems/rubygems] Do not replace the cache entry if there is already one
* That way, multiple lookups for the same file always return the same object.

50a431b6db
2019-07-31 07:47:00 +08:00
Benoit Daloze
2453d16f5e
[rubygems/rubygems] Synchronize access to the Gem::Specification::LOAD_CACHE Hash
* It's accessed concurrently, notably when installing a gem with a C extension.

543294d7dd
2019-07-31 07:47:00 +08:00
bronzdoc
6c5170634a
[rubygems/rubygems] Lazy require stringio
82f0d4ca69
2019-07-31 07:47:00 +08:00
Hiroshi SHIBATA
56660de3c6 Merge rubygems master from upstream.
I picked the commit from 3c469e0da538428a0ddd94f99aa73c32da22e8ba
2019-06-01 13:50:41 +03:00
David Rodríguez
d0a5467320 Update rubygems with latest upstream changes
Closes: https://github.com/ruby/ruby/pull/2154
2019-04-28 11:07:45 +09:00
hsbt
f28be7e02d Merge rubygems/rubygems from upstream.
The current master branch is
  97b264f0fa

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67415 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-02 11:48:18 +00:00
hsbt
4ae3df42f7 Merge RubyGems master@9be7858f7f17eae3058204f3c03e4b798ba18b9c
This version contains the some style changes by RuboCop.

    * 9d810be0ed
    * 61ea98a727
    * 795893dce3
    * 9be7858f7f

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67074 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-02-14 12:59:03 +00:00
hsbt
3dc6efbe9c Merge rubygems master targeted RubyGems 3.1.0.
1172320540

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66904 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-01-22 06:28:04 +00:00
hsbt
7f9bf068fc Merge rubygems@21f12a8 from upstream.
* [BudlerVersionFinder] set .filter! and .compatible? to match only on major versions https://github.com/rubygems/rubygems/pull/2515
  * Fix broken symlink that points to ../* https://github.com/rubygems/rubygems/pull/2516

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66347 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-12 05:07:50 +00:00
hsbt
a94cbf81e1 Merge rubygems master@221bdea
* Fixed https://github.com/rubygems/rubygems/pull/2506

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66188 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-04 08:00:57 +00:00
hsbt
5dbc6583c9 Merge rubygems upstream from 2c499655f2
https://github.com/rubygems/rubygems/pull/2493

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66065 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-28 03:08:14 +00:00
hsbt
5335ce0e06 Merge master branch from rubygems/rubygems upstream.
* Enable Style/MethodDefParentheses in Rubocop
    https://github.com/rubygems/rubygems/pull/2478
  * Enable Style/MultilineIfThen in Rubocop
    https://github.com/rubygems/rubygems/pull/2479
  * Fix required_ruby_version with prereleases and improve error message
    https://github.com/rubygems/rubygems/pull/2344
  * Fix bundler rubygems binstub not properly looking for bundler
    https://github.com/rubygems/rubygems/pull/2426

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65904 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-21 10:20:47 +00:00
hsbt
615ac35934 Merge rubygems master branch from github.com/rubygems/rubygems.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65294 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-22 00:27:02 +00:00
hsbt
85d461456c Merge master branch from rubygems upstream.
* It's preparation to release RubyGems 3.0.0.beta2 and Ruby 2.6.0
    preview 3.
  * https://github.com/rubygems/rubygems/compare/v3.0.0.beta1...fad2eb15a282b19dfcb4b48bc95b8b39ebb4511f

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64555 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-27 10:05:04 +00:00
hsbt
8da8d4b043 Merge RubyGems 3.0.0.beta1.
* It drop to support < Ruby 2.2
  * Cleanup deprecated methods and classes.
  * Mark obsoleted methods to deprecate.
  * and other enhancements.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63528 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-30 13:01:35 +00:00
hsbt
c6da9cadb3 Merge RubyGems 2.7.7
see release details here: https://blog.rubygems.org/2018/05/18/2.7.7-released.html

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63461 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-18 01:39:13 +00:00
hsbt
7619cb3d7d Merge RubyGems 2.7.6 from upstream.
It fixed some security vulnerabilities.

  http://blog.rubygems.org/2018/02/15/2.7.6-released.html

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62422 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-16 08:08:06 +00:00
hsbt
500f15e507 Merge rubygems-2.6.14 changes.
It fixed http://blog.rubygems.org/2017/10/09/unsafe-object-deserialization-vulnerability.html

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60149 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-10 08:58:22 +00:00
hsbt
c00e84327f Merge rubygems master.
This is RC version of Rubygems 2.7.0.
  688fb7e83c

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60133 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-08 01:32:18 +00:00
hsbt
c5da5b1ea2 Merge rubygems-2.6.13.
see details for this update:
  http://blog.rubygems.org/2017/08/27/2.6.13-released.html

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59672 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-08-28 08:31:28 +00:00
naruse
d7181f726f confirm current process equals to gemhome's pid
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58648 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-10 07:23:06 +00:00
naruse
8972de9ff7 dump stub specifications if nil
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58642 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-09 17:41:10 +00:00