test/runner.rb and tool/lib/test/unit/parallel.rb must use the same
settings. However, some settings were copied and pasted, while some were
added only to test/runner.rb.
This changeset creates tool/test/init.rb for all settings of test-unit,
which is loaded not only by test/runner.rb but also
tool/lib/test/unit/parallel.rb.
Background: the GEM_HOME environment variable was removed in
test/runner.rb, which prohibit `require "rake"` (note that rake is a
bundled gem). However the parallel mode didn't refrect this setting,
i.e., `require "rake"` was allowed.
This leads to an inconsistency, which actually affected a test test
defines s test class *only when* `require "rake"` is successful.
(test/rubygems/test_gem_package_task.rb)
1851105563 (step):8:1714
```
/home/runner/work/ruby/ruby/src/tool/lib/test/unit.rb:729:in `const_get': uninitialized constant TestGemPackageTask (NameError)
suites.map! {|r| ::Object.const_get(r[:testcase])}
^^^^^^^^^^
```
to @__passed__, @@__current__.
@passed is redefined in a few test suites, and this could lead to bugs.
Also rename @options (Runner#options) to @__runner_options__, which is
only used in make test-tool anyway.
This test skipped sometimes due to failure to load 'rake/packagetask'.
This is due to manipulation of $LOAD_PATH by other rubygems tests. If
rake is loaded before any rubygems tests run, then it works fine.
To reproduce the skipping behavior:
$ make test-all TESTOPTS="-j6 --test-order=sorted test/rubygems/test_*.rb"
Ignoring debug-1.7.1 because its extensions are not built. Try: gem pristine debug --version 1.7.
Revert "Clear gem paths for each test"
This reverts commit 6698b580dd.
When a test worker hangs and timeouts, the test runner crashes with the
following stack trace:
ruby/tool/lib/test/unit.rb:1747:in `puke': undefined method `backtrace' for Timeout::Error:Class (NoMethodError)
from ruby/tool/lib/test/unit.rb:790:in `block in _run_parallel'
from ruby/tool/lib/test/unit.rb:788:in `each'
This commit adds handling for Timeout::Error and outputs a message.
* So deprecated methods/constants/functions are dealt with early,
instead of many tests breaking suddenly when removing a deprecated
method/constant/function.
* Follows https://bugs.ruby-lang.org/issues/17591
On the parallel test, workers can be killed because of timeout
and the information for the retrying can be inconsistent.
This patch will skip if the inconsistency is found and report
as an error.
http://ci.rvm.jp/results/trunk-asserts@phosphorus-docker/3834082
If sole `filter` option doesn't seem including test case name,
match with method name only.
And if the filter is a Regexp or String, it never matches method
name symbols.
Lets consider the following scenario:
~~~
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):001:0> p suite
OpenSSL::TestEC
=> OpenSSL::TestEC
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):002:0> p all_test_methods
["test_ECPrivateKey", "test_ECPrivateKey_encrypted", "test_PUBKEY", "test_check_key", "test_derive_key", "test_dh_compute_key", "test_dsa_sign_asn1_FIPS186_3", "test_ec_group", "test_ec_key", "test_ec_point", "test_ec_point_add", "test_ec_point_mul", "test_generate", "test_marshal", "test_sign_verify", "test_sign_verify_raw"]
=>
["test_ECPrivateKey",
"test_ECPrivateKey_encrypted",
"test_PUBKEY",
"test_check_key",
"test_derive_key",
"test_dh_compute_key",
"test_dsa_sign_asn1_FIPS186_3",
"test_ec_group",
"test_ec_key",
"test_ec_point",
"test_ec_point_add",
"test_ec_point_mul",
"test_generate",
"test_marshal",
"test_sign_verify",
"test_sign_verify_raw"]
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):003:0> p filter
/\A(?=.*)(?!.*(?-mix:(?-mix:memory_leak)|(?-mix:OpenSSL::TestEC.test_check_key)))/
=> /\A(?=.*)(?!.*(?-mix:(?-mix:memory_leak)|(?-mix:OpenSSL::TestEC.test_check_key)))/
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):004:0> method = "test_check_key"
=> "test_check_key"
~~~
The intention here is to exclude the `test_check_key` test case.
Unfortunately this does not work as expected, because the negative filter
is never checked:
~~~
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):005:0> filter === method
=> true
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):006:0> filter === "#{suite}##{method}"
=> false
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):007:0> filter === method || filter === "#{suite}##{method}"
=> true
~~~
Therefore always filter against the fully qualified method name
`#{suite}##{method}`, which should provide the expected result.
However, if plain string filter is used, keep checking also only the
method name.
This resolves [Bug #16936].
