[Bug #19985] Raise LoadError with the converted feature name
`Kernel#require` converts feature name objects that have the `to_path`
method such as `Pathname`, but had used the original object on error
and had resulted in an unexpected `TypeError`.
---
load.c | 14 +++++++++++---
test/ruby/test_require.rb | 26 +++++++++++++++++++++-----
2 files changed, 32 insertions(+), 8 deletions(-)
Speed up rebuilding the loaded feature index
Rebuilding the loaded feature index slowed down with the bug fix
for #17885 in 79a4484a07. The
slowdown was extreme if realpath emulation was used, but even when
not emulated, it could be about 10x slower.
This adds loaded_features_realpath_map to rb_vm_struct. This is a
hidden hash mapping loaded feature paths to realpaths. When
rebuilding the loaded feature index, look at this hash to get
cached realpath values, and skip calling rb_check_realpath if a
cached value is found.
Fixes [Bug #19246]
---
load.c | 27 +++++++++++++++++++++++----
vm.c | 2 ++
vm_core.h | 1 +
3 files changed, 26 insertions(+), 4 deletions(-)
Fix autoload status of statically linked extensions
Previously, for statically-linked extensions, we used
`vm->loading_table` to delay calling the init function until the
extensions are required. This caused the extensions to look like they
are in the middle of being loaded even before they're required.
(`rb_feature_p()` returned true with a loading path output.) Combined
with autoload, queries like `defined?(CONST)` and `Module#autoload?`
were confused by this and returned nil incorrectly. RubyGems uses
`defined?` to detect if OpenSSL is available and failed when OpenSSL was
available in builds using `--with-static-linked-ext`.
Use a dedicated table for the init functions instead of adding them to
the loading table. This lets us remove some logic from non-EXTSTATIC
builds.
[Bug #19115]
---
load.c | 55 +++++++++++++++++++++++++++++++++++-----------
test/ruby/test_autoload.rb | 18 +++++++++++++++
vm.c | 3 +++
vm_core.h | 9 ++++++++
4 files changed, 72 insertions(+), 13 deletions(-)
Only emit circular dependency warning for owned thread shields [Bug
#19415]
If multiple threads attemps to load the same file concurrently
it's not a circular dependency issue.
So we check that the existing ThreadShield is owner by the current
fiber before warning about circular dependencies.
---
internal/thread.h | 1 +
load.c | 3 ++-
spec/ruby/core/kernel/shared/require.rb | 11 +++++++++++
spec/ruby/fixtures/code/concurrent_require_fixture.rb | 4 ++++
test/ruby/test_require.rb | 3 ---
thread.c | 11 +++++++++++
6 files changed, 29 insertions(+), 4 deletions(-)
create mode 100644 spec/ruby/fixtures/code/concurrent_require_fixture.rb
Use Thread.pass until thread.stop? to wait for thread to block
[Bug #19415]
It should be more reliable
---
spec/ruby/fixtures/code/concurrent_require_fixture.rb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
rb_provide_feature: Prevent $LOADED_FEATURES from being copied
[Bug #18599]
`vm->loaded_features` and `vm->loaded_features_snapshot` both share the
same root. When a feature is pushed into `loaded_features`, the sharing
is broken and `loaded_features` is copied.
So an application requiring 1000 files, will allocate 1000 arrays of
increasing size, which is very wasteful.
To avoid this, we first clear the snapshot, so that `loaded_features`
can directly be pushed into.
Co-Authored-By: Peter Zhu <peter.zhu@shopify.com>
---
load.c | 4 ++++
1 file changed, 4 insertions(+)
Fix TAG_THROW through require [Bug #18562]
Previously this was being incorrectly swapped with TAG_RAISE in the next
line. This would end up checking the T_IMEMO throw_data to the exception
handling (which calls Module#===). This happened to not break existing
tests because Module#=== returned false when klass is NULL.
This commit handles throw from require correctly by jumping to the tag
retaining the TAG_THROW state.
---
load.c | 2 +-
test/ruby/test_exception.rb | 21 +++++++++++++++++++++
2 files changed, 22 insertions(+), 1 deletion(-)
Instead of always using a new anonymous module for Kernel#load if
the wrap argument is not false/nil, use the given module if a module
is provided.
Implements [Feature #6210]
This appears to be only necessary on Solaris, but this commit
enables it unconditionally to test breakage. The following
commit will switch to only enabling it on Solaris.
This fixes issues with paths being loaded twice in certain cases
when symlinks are used.
It took me multiple attempts to get this working. My original
attempt tried to convert paths to realpaths before adding them
to $LOADED_FEATURES. Unfortunately, this doesn't work well
with the loaded feature index, which is based off load paths
and not realpaths. While I was able to get require working, I'm
fairly sure the loaded feature index was not being used as
expected, which would have significant performance implications.
Additionally, I was never able to get that approach working with
autoload when autoloading a non-realpath file. It also broke
some specs.
This takes a more conservative approach. Directly before loading the
file, if the file with the same realpath has been required, the
loading of the file is skipped. The realpaths are stored as
fstrings in a hidden hash.
When rebuilding the loaded feature index, the hash of realpaths
is also rebuilt. I'm guessing this makes rebuilding process
slower, but I don think that is a hot path. In general, modifying
loaded features is only done when reloading, and that tends to be
in non-production environments.
Change test_require_with_loaded_features_pop test to use 30 threads
and 300 iterations, instead of 4 threads and 1000 iterations.
I saw only sporadic failures with 4/1000, but consistent failures
30/300 threads. These failures were due to the fact that the
concurrent deletions from $LOADED_FEATURES in other threads can
result in rb_ary_entry returning nil when rebuilding the loaded
features index.
To avoid concurrency issues when rebuilding the loaded features
index, the building of the index itself is left alone, and
afterwards, a separate loop is done on a copy of the loaded feature
snapshot in order to rebuild the realpaths hash.
Fixes [Bug #17885]
Instead of relying on setting an unsetting ruby_verbose, which is
not thread-safe, restructure require_internal and load_lock to
accept a warn argument for whether to warn, and add
rb_require_internal_silent to require without warnings. Use
rb_require_internal_silent when loading encoding.
Note this does not modify ruby_debug and errinfo handling, those
remain thread-unsafe.
Also silent requires when loading transcoders.
This fixes issues with paths being loaded twice in certain cases
when symlinks are used.
It took me multiple attempts to get this working. My original
attempt tried to convert paths to realpaths before adding them
to $LOADED_FEATURES. Unfortunately, this doesn't work well
with the loaded feature index, which is based off load paths
and not realpaths. While I was able to get require working, I'm
fairly sure the loaded feature index was not being used as
expected, which would have significant performance implications.
Additionally, I was never able to get that approach working with
autoload when autoloading a non-realpath file. It also broke
some specs.
This takes a more conservative approach. Directly before loading the
file, if the file with the same realpath has been required, the
loading of the file is skipped. The realpaths are stored as
fstrings in a hidden hash.
When rebuilding the loaded feature index, the hash of realpaths
is also rebuilt. I'm guessing this makes rebuilding process
slower, but I don think that is a hot path. In general, modifying
loaded features is only done when reloading, and that tends to be
in non-production environments.
Change test_require_with_loaded_features_pop test to use 30 threads
and 300 iterations, instead of 4 threads and 1000 iterations.
I saw only sporadic failures with 4/1000, but consistent failures
30/300 threads. These failures were due to the fact that the
concurrent deletions from $LOADED_FEATURES in other threads can
result in rb_ary_entry returning nil when rebuilding the loaded
features index.
To avoid concurrency issues when rebuilding the loaded features
index, the building of the index itself is left alone, and
afterwards, a separate loop is done on a copy of the loaded feature
snapshot in order to rebuild the realpaths hash.
Fixes [Bug #17885]
If $LOADED_FEATURES is changed in the just required file, also the
index table needs to be updated before loaded_features_snapshot is
reset. If the snapshot is reset without updating the table, the
name of the added feature will not be found.
When attempting to require a file without an extension that has
already been required or provided with an .so extension, only
look for files with an .rb extension. There is no point in
trying to find files with an .so extension, since we already
know one has been loaded.
Previously, attempting to require such a file scanned the load
path twice, once for .rb and once for .so. Now it only scans
once for .rb. The scan once for .rb cannot be avoided, since
the .rb file would take precedence and should be loaded if it
exists.
Fixes [Bug #10902]
When looking for libraries to load with a feature name without
extension, `.rb` files are given priority. However, since the
feature index arrays were not in that order of priority, but in
the order in which they were loaded, a lower priority extension
library might be returned. In that case, the `.rb` file had to be
searched for again from the `$LOAD_PATH`, resulting in poor
performance.
C extensions can violate the ractor-safety, so only ractor-safe
C extensions (C methods) can run on non-main ractors.
rb_ext_ractor_safe(true) declares that the successive
defined methods are ractor-safe. Otherwiwze, defined methods
checked they are invoked in main ractor and raise an error
if invoked at non-main ractors.
[Feature #17307]
Update and format the Kernel#load documentation to separate the
three cases (absolute path, explicit relative path, other), and
also document that it raises LoadError on failure.
Fixes [Bug #16988]
Saves comitters' daily life by avoid #include-ing everything from
internal.h to make each file do so instead. This would significantly
speed up incremental builds.
We take the following inclusion order in this changeset:
1. "ruby/config.h", where _GNU_SOURCE is defined (must be the very
first thing among everything).
2. RUBY_EXTCONF_H if any.
3. Standard C headers, sorted alphabetically.
4. Other system headers, maybe guarded by #ifdef
5. Everything else, sorted alphabetically.
Exceptions are those win32-related headers, which tend not be self-
containing (headers have inclusion order dependencies).
TAG_FATAL represents interpreter closing state and ec->errinfo
contains FIXNUM (eTerminateSignal, etc). If we need to change the
state, then errinfo is also changed because TAG_RAISE assumes that
ec->errinfo contains a Exception object.
Without this patch, TAG_FATAL is ignored and no ec->errinfo change
so that it causes critical issue.
[Bug #16177]
This removes the security features added by $SAFE = 1, and warns for access
or modification of $SAFE from Ruby-level, as well as warning when calling
all public C functions related to $SAFE.
This modifies some internal functions that took a safe level argument
to no longer take the argument.
rb_require_safe now warns, rb_require_string has been added as a
version that takes a VALUE and does not warn.
One public C function that still takes a safe level argument and that
this doesn't warn for is rb_eval_cmd. We may want to consider
adding an alternative method that does not take a safe level argument,
and warn for rb_eval_cmd.
Looking at the list of symbols inside of libruby-static.a, I found
hundreds of functions that are defined, but used from nowhere.
There can be reasons for each of them (e.g. some functions are
specific to some platform, some are useful when debugging, etc).
However it seems the functions deleted here exist for no reason.
This changeset reduces the size of ruby binary from 26,671,456
bytes to 26,592,864 bytes on my machine.
We can check the function pointer passed to rb_define_global_function
like we do so in rb_define_method. It turns out that almost anybody
is misunderstanding the API.
After 5e86b005c0, I now think ANYARGS is
dangerous and should be extinct. There is only one usage of
MEMO::u3::func in load.c (where void Init_Foobar(vodi) is registered)
so why not just be explicit.
After 5e86b005c0, I now think ANYARGS is
dangerous and should be extinct. This commit uses rb_gvar_getter_t /
rb_gvar_setter_t for rb_define_hooked_variable /
rb_define_virtual_variable which revealed lots of function prototype
inconsistencies. Some of them were literally decades old, going back
to dda5dc00cf.
