This has the following benefits:
* Avoid duplicated work in some specs that first build a repo, and then
overwrite it with a completely different set of gems.
* Reduce RSpec nesting and improve readability.
* The change also made surfaces several specs that were incorrect since
they were unintentionally not testing the right thing.
ed430883e0
This message is printed when running `bundle lock --add-platform`. This
command affects the lockfile, not the gemfile, and I think it's better
to use "You are adding" rather than "You added", because the addition is
happening during the current invocation (as opposed to other log
messages that talk about a change made to the Gemfile prior to running
the command).
aba1e55f5b
Due to a typo in the spec, the issue was not caught initially. If
Bundler does not need to re-resolve, `bundle lock` is a noop so Bundler
does not add checksums.
To fix the issue, we do something similar to what `bundle install` does,
just without actually installation. First set the domain (local or
remote) according to whether a re-resolve is necessary, and then
materialize lazy specifications into real specifications, so that
checksums are actually fetched from each source.
84b6f4ee96
`bundle lock --print --update` can take a long time to fetch sources and
resolve the lock file.
Before, `--print` caused output to be completely silenced, so nothing
was printed at all until the resolved lock file is finally emitted to
stdout.
With this change, `--print` now prints progress to stderr. E.g.:
```
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
```
This provides a better user experience, especially when
`lock --print --update` takes several seconds or more.
The lock file is still printed to stdout, so tools consuming the lock
file on stdout will not be affected.
6719baa700
When resolving on truffleruby, and multiple platforms are included in
the lockfile, Bundler will not respect existing platforms, but always
force ruby variants. That means removal of existing version specific
variants, introducing lockfile churn between implementations.
To prevent this, we introduce the distinction between
`Dependency#force_ruby_platform`, only settable via Gemfile, and
`Dependency#default_force_ruby_platform`, which is always true on
truffleruby for certain dependency names. This way, when resolving
lockfile gems for other platforms on truffleruby, we keep platform
specific variants in the lockfile.
However, that introduces the problem that if only platform specific
variants are locked in the lockfile, Bundler won't be able to
materialize on truffleruby because the generic variant will be missing.
To fix this additional problem, we make sure the generic "ruby" platform
is always added when resolving on truffleruby.
Instead of having to do a complete pass after resolve.
To do this, we add to the ruby group all the platform specs with the
same dependencies as the ruby specs.
e50415f2a6
Even if all gems are properly installed and no resolve is needed, we
recently started always reading all packages in `vendor/cache` and
extracting specifications from them.
This commit fixes the problem by longer making considering cached specs
the default and only enable them when a resolve is actually needed.
edeb2c42bf
To make sure we can always update to the latest resolvable version for
each gem explicitly requested for update, we first run a full update,
and then add explicit exact requirements to the resolved versions. This
may lead into conflicts, but our resolver already automatically parses
those and unlocks additional gems to fix them.
01c0bf34f0
Looks for the CHECKSUMS section in the lockfile, activating the feature
only if the section exists. Without a CHECKSUMS section, Bundler will
continue as normal, validating checksums when gems are installed while
checksums from the compact index are present.
2353cc93a4
Since we started locking the specific platform in the lockfile, that has
created an annoying situation for users that don't develop on Linux.
They will create a lockfile on their machines, locking their local
platform, for example, darwin. But then that lockfile won't work
automatically when deploying to Heroku for example, because the lockfile
is frozen and the Linux platform is not included.
There's the chance though that resolving against two platforms (Linux +
the local platform) won't succeed while resolving for just the current
platform will. So, instead, we check other platform specific variants
available for the resolution we initially found, and lock those
platforms and specs too if they satisfy the resolution.
This is only done when generating new lockfiles from scratch, existing
lockfiles should keep working as before, and it's only done for "ruby
platforms", i.e., not Java or Windows which have their own complexities,
and so are excluded.
With this change, we expect that MacOS users can bundle locally and
deploy to Heroku without needing to do anything special.
5f24f06bc5
Daily Bundler CI against ruby-head is failing because ruby-head now
uses bigdecimal 3.1.5, so that gets locked by this spec.
This change should make the test stable until bigdecimal 99.1.5 is
bundled with Ruby :)
830326041f
Improve error reporting for checksums, raises a new error class.
Solve for multi-source checksum errors.
Add CHECKSUMS to tool/bundler/(dev|standard|rubocop)26_gems.rb
26ceee0e76
Co-authored-by: Samuel Giddins <segiddins@segiddins.me>
This gets the specs passing, and handles the fact that we expect
checkums to be pinned only to a particular source
This also avoids reading in .gem files during lockfile generation,
instead allowing us to query the source for each resolved gem to grab
the checksum
Finally, this opens up a route to having user-stored checksum databases,
similar to how other package managers do this!
Add checksums to dev lockfiles
Handle full name conflicts from different original_platforms when adding checksums to store from compact index
Specs passing on Bundler 3
86c7084e1c
1. Use the checksum provided by the server if provided: provides security
knowing if the gem you downloaded matches the gem on the server
2. Calculate the checksum from the gem on disk: provides security knowing
if the gem has changed between installs
3. In some cases, neither is possible in which case we don't put anything
in the checksum and we maintain functionality as it is today
Add the checksums to specs in the index if we already have them
Prior to checksums, we didn't lose any information when overwriting specs
in the index with stubs. But now when we overwrite EndpointSpecifications
or RemoteSpecifications with more generic specs, we could lose checksum
info. This manually sets checksum info so we keep it in the index.
de00a4f153
We lock the checksum for each resolved spec under a new CHECKSUMS
section in the lockfile.
If the locked spec does not resolve for the local platform, we preserve
the locked checksum, similar to how we preserve specs.
Checksum locking only makes sense on install. The compact index
information is only available then.
bde37ca6bf
The `lock` command is specifically designed to manage the lockfile, so
running it should take precedence over any "frozen" setting.
Besides that, "frozen" is not specifically designed as "lockfile cannot
be updated" but as "installation of gems should be prevented if gemfile
is not in sync with the lockfile".
The lock command does not install any gems and preserves the property of
the lockfile being in sycn with its gemfile, so I think frozen should
not influence it.
The current behavior is quite confusing when frozen is set. On an app
where rubocop can get lockfile updates
```
$ bundle lock --update rubocop
Writing lockfile to /path/to/Gemfile.lock
```
Completely silent, it makes you think that it has written the lockfile,
but still no updates.
In verbose mode, it gives a bit more information, but still confusing
and unexpected, and does not change the lockfile:
```
$ bundle lock --update rubocop --verbose
Running `bundle lock --update "rubocop" --verbose` with bundler 2.4.20
Frozen, using resolution from the lockfile
Writing lockfile to /path/to/Gemfile.lock
```
With this commit, it updates the lockfile as expected.
1d501ae8ea
When the latest allowed minor of `dep` adds a new dependency, that new
dependency would be incorrectly resolved to the latest minor of the
first major version.
fd50c9d4f3
While working on locking multiple platforms by default, I got an
infinite resolution loop in one of our resolver specs.
The culprit ended up being that when dealing with lockfile specs with
incomplete dependencies (spec appears in lockfile, but its dependencies
don't), those specs were not being properly expired and that tripped up
resolution.
The issue for some reason only manifests when dealing with multiple
lockfile platforms, that's why it only manifested when working on
locking multiple platforms by default.
4ca72913bb
Bundler is very conservative by default, trying to preserve versions
from the lockfile as possible, and never downgrading them. However, when
it runs into a resolution error, it still tries to find a valid
resolution.
This fallback behavior was too "brute-force" though, completely
unrestricting any gem found in the resolution conflict, and that could
lead to direct dependencies being downgraded in some edge cases.
Instead, unlock things a bit more carefully:
* First try unlocking fully pinned indirect dependencies, but leave a
lower bound requirement in place to prevent downgrades.
* Then try unlocking any fully pinned dependency, also leaving a lower
bound requirement in place.
* Finally completely unrestrict dependencies if nothing else worked.
7f55ed8302
Since Bundler 2.4, we will try to checkout any branch specified in the
Gemfile, while until Bundler 2.3 we would directly checkout the locked
revision.
This should not make any difference in most situations, but in some edge
cases, like if the branch specified in the `Gemfile` has been renamed,
but the locked revision still exist, it causes an error now while before
it would update the lockfile without issues.
I debated which behavior was best, since I was not sure. But my
conclusion is that if the situation does not require expiring the
lockfile source in favor of the Gemfile source, we should use the locked
revision directly and proceed happily. So I restored Bundler 2.3
behavior.
I think this is consistent with how yanked gems are handled, for example.
Of course, if explicitly updating the git source itself, or all gems, we
will still get any errors like missing branches related to the git source.
This was working fine for direct dependencies using
`force_ruby_platform` explicitly through Gemfile, but not for indirect
dependencies. In general, indirect dependencies do not have this
property set, but in truffleruby this is different and the default value
is to have it set.
This should be a very rare edge case, however, it does happen when using
a .dev version of Bundler because in that case, that's the only version
that the resolver considers, and it should not be ignored.
We could've special cased this specifically for Bundler, but I think it
does make sense for every gem.
