Overriding the version constant feels too magic and creates a set of
problems. For example, Bundler will lock the simulated version, and that
can cause issues when the lockfile is used under an environment not
simulating Bundler 4 (it will try to auto-install and auto-switch to a
version that does not exist).
On top of that, it can only be configured with an ENV variable which is
not too flexible.
This commit takes a different approach of using a setting, which is
configurable through ENV or `bundle config`, and pass the simulated
version to `Bundler::FeatureFlag`. The real version is still the one set
by `VERSION`, but anything that `Bundler::FeatureFlag` controls will use
the logic of the "simulated version".
In particular, all feature flags and deprecation messages will respect
the simulated version, and this is exactly the set of functionality that
we want users to be able to easily try before releasing it.
8129402193
Since we no longer pass ruby CLI flags in our spec commands, we no
longer need the previous workaround and can get the realworld code
tested.
fd92c855fb
Currently to test Bundler 3 mode we have to actually edit the version
file to simulate we're running a future version. This is inconvenient.
Instead, allow passing an environment variable, `BUNDLER_3_MODE`, to set
the "working mode" Bundler should use.
This can now be set easily by end users to enable them to try out the
changes in the future version and give us feedback.
It's unclear how version auto-switching should work when this
environment variable is set, so the auto-switching feature will be
disabled in that case.
4e92e9b209
Resolver had internal logic to prioritize locked versions when sorting
versions, however part of it was not being actually hit because of how
unlocking worked in the resolver: a package was allow to be unlocked
when that was explicit requested or when the list of unlocks was empty.
That did not make a lot of sense and other cases were working because
the explicit list of unlocks was getting "artificially filled".
Now we consider a package unlocked when explicitly requested (`bundle
update <package>`), or when everything is being unlocked (`bundle
install` with no lockfile or `bundle update`).
This makes things simpler and gets the edge case added as a test case
working as expected.
b8e55087f0
While working on something else I noticed:
* Usage of uppercased "RUBY" and "JAVA" as platforms, when those don't
really exist.
* Usage of some test gems with "1.0" as gemspec version and "1.0.0" as
actual version.
This commit fixes both inconsistencies to make things more expectable.
e3ec32e247
If two platform specific variants have different dependencies, then
resolution may fallback to the non platform specific variant. However,
the platform specific variants that have the same dependencies as the
non specific one can still be kept.
Do a pass to complete those after resolution.
Looks for the CHECKSUMS section in the lockfile, activating the feature
only if the section exists. Without a CHECKSUMS section, Bundler will
continue as normal, validating checksums when gems are installed while
checksums from the compact index are present.
2353cc93a4
Since we started locking the specific platform in the lockfile, that has
created an annoying situation for users that don't develop on Linux.
They will create a lockfile on their machines, locking their local
platform, for example, darwin. But then that lockfile won't work
automatically when deploying to Heroku for example, because the lockfile
is frozen and the Linux platform is not included.
There's the chance though that resolving against two platforms (Linux +
the local platform) won't succeed while resolving for just the current
platform will. So, instead, we check other platform specific variants
available for the resolution we initially found, and lock those
platforms and specs too if they satisfy the resolution.
This is only done when generating new lockfiles from scratch, existing
lockfiles should keep working as before, and it's only done for "ruby
platforms", i.e., not Java or Windows which have their own complexities,
and so are excluded.
With this change, we expect that MacOS users can bundle locally and
deploy to Heroku without needing to do anything special.
5f24f06bc5
Improve error reporting for checksums, raises a new error class.
Solve for multi-source checksum errors.
Add CHECKSUMS to tool/bundler/(dev|standard|rubocop)26_gems.rb
26ceee0e76
Co-authored-by: Samuel Giddins <segiddins@segiddins.me>
This gets the specs passing, and handles the fact that we expect
checkums to be pinned only to a particular source
This also avoids reading in .gem files during lockfile generation,
instead allowing us to query the source for each resolved gem to grab
the checksum
Finally, this opens up a route to having user-stored checksum databases,
similar to how other package managers do this!
Add checksums to dev lockfiles
Handle full name conflicts from different original_platforms when adding checksums to store from compact index
Specs passing on Bundler 3
86c7084e1c
We lock the checksum for each resolved spec under a new CHECKSUMS
section in the lockfile.
If the locked spec does not resolve for the local platform, we preserve
the locked checksum, similar to how we preserve specs.
Checksum locking only makes sense on install. The compact index
information is only available then.
bde37ca6bf
I've never seen this error in real life, and if it was happening, I
think it's either some server side issue that would need to be fixed or
some transient issue. We should move away from the full index, since
it's slow, so let's stop recommending it.
Following up on https://github.com/rubygems/rubygems/pull/6355, which
turned a crash into a nicer error message, this commit auto-heals the
corrupt lockfile instead.
In this particular case (a corrupt Gemfile.lock with missing
dependencies) the LazySpecification will not have accurate dependency
information, we have to materialize the SpecSet to determine there are
missing dependencies. We've already got a way to handle this, via
`SpecSet#incomplete_specs`, but it wasn't quite working for this case
because we'd get to `@incomplete_specs += lookup[name]` and
`lookup[name]` would be empty for the dependency.
With this commit we catch it a bit earlier, marking the parent spec
containing the missing dependency as incomplete.
486ecb8f20
I did a bad thing (script that edits the Gemfile.lock directly) and
ended up with a Gemfile.lock that was completely missing some indirect
dependencies. While this is my fault and an error is reasonable, I
noticed that the error got progressively less friendly in recent
versions of bundler.
Something similar came up in https://github.com/rubygems/rubygems/issues/6210,
and this commit would have helped with that case as well
(although we've already handled this a different way with #6219).
Details:
---
Back on Bundler 2.2.23, a corrupt lockfile like this would cause a helpful error:
```
Unable to find a spec satisfying minitest (>= 5.1) in the set. Perhaps the lockfile is corrupted?
```
Bundler 2.3.26 gave a helpful warning:
```
Warning:
Your lockfile was created by an old Bundler that left some things out.
Because of the missing DEPENDENCIES, we can only install gems one at a time,
instead of installing 16 at a time.
You can fix this by adding the missing gems to your Gemfile, running bundle
install, and then removing the gems from your Gemfile.
The missing gems are:
* minitest depended upon by activesupport
```
But then continued on and crashed while trying to report the unmet
dependency:
```
--- ERROR REPORT TEMPLATE -------------------------------------------------------
NoMethodError: undefined method `full_name' for nil:NilClass
lib/bundler/installer/parallel_installer.rb:127:in `block (2 levels) in check_for_unmet_dependencies'
...
```
Bundler 2.4.0 and up crash as above when jobs=1, but crash
even harder when run in parallel:
```
--- ERROR REPORT TEMPLATE -------------------------------------------------------
fatal: No live threads left. Deadlock?
3 threads, 3 sleeps current:0x00007fa6b6704660 main thread:0x00007fa6b6704660
* #<Thread:0x000000010833b130 sleep_forever>
rb_thread_t:0x00007fa6b6704660 native:0x0000000108985600 int:0
* #<Thread:0x0000000108dea630@Parallel Installer Worker #0 tmp/1/gems/system/gems/bundler-2.5.0.dev/lib/bundler/worker.rb:90 sleep_forever>
rb_thread_t:0x00007fa6b67f67c0 native:0x0000700009a62000 int:0
* #<Thread:0x0000000108dea4a0@Parallel Installer Worker #1 tmp/1/gems/system/gems/bundler-2.5.0.dev/lib/bundler/worker.rb:90 sleep_forever>
rb_thread_t:0x00007fa6b67f63c0 native:0x0000700009c65000 int:0
<internal:thread_sync>:18:in `pop'
tmp/1/gems/system/gems/bundler-2.5.0.dev/lib/bundler/worker.rb:42:in `deq'
...
```
Changes
---
This commit fixes the confusing thread deadlock crash by detecting if
dependencies are missing such that we'll never be able to enqueue. When
that happens we treat it as a failure so the install can finish.
That gets us back to the `NoMethodError`, which this commit fixes by
using a different warning in the case where no spec is found.
d73001a21d
We have a check for a corrupt lockfile right before installing. However,
the check accounted for locked specs not satisfying locked dependencies,
but not for locked specs missing for some locked dependencies.
Instead of fixing this check, I decided to remove it in favor of
automatically detecting the situation and re-resolve to automatically
fix the lockfile rather than printing a warning but leave the problem
there.
4a7a584252
