When a test worker hangs and timeouts, the test runner crashes with the
following stack trace:
ruby/tool/lib/test/unit.rb:1747:in `puke': undefined method `backtrace' for Timeout::Error:Class (NoMethodError)
from ruby/tool/lib/test/unit.rb:790:in `block in _run_parallel'
from ruby/tool/lib/test/unit.rb:788:in `each'
This commit adds handling for Timeout::Error and outputs a message.
* So deprecated methods/constants/functions are dealt with early,
instead of many tests breaking suddenly when removing a deprecated
method/constant/function.
* Follows https://bugs.ruby-lang.org/issues/17591
On the parallel test, workers can be killed because of timeout
and the information for the retrying can be inconsistent.
This patch will skip if the inconsistency is found and report
as an error.
http://ci.rvm.jp/results/trunk-asserts@phosphorus-docker/3834082
If sole `filter` option doesn't seem including test case name,
match with method name only.
And if the filter is a Regexp or String, it never matches method
name symbols.
Lets consider the following scenario:
~~~
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):001:0> p suite
OpenSSL::TestEC
=> OpenSSL::TestEC
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):002:0> p all_test_methods
["test_ECPrivateKey", "test_ECPrivateKey_encrypted", "test_PUBKEY", "test_check_key", "test_derive_key", "test_dh_compute_key", "test_dsa_sign_asn1_FIPS186_3", "test_ec_group", "test_ec_key", "test_ec_point", "test_ec_point_add", "test_ec_point_mul", "test_generate", "test_marshal", "test_sign_verify", "test_sign_verify_raw"]
=>
["test_ECPrivateKey",
"test_ECPrivateKey_encrypted",
"test_PUBKEY",
"test_check_key",
"test_derive_key",
"test_dh_compute_key",
"test_dsa_sign_asn1_FIPS186_3",
"test_ec_group",
"test_ec_key",
"test_ec_point",
"test_ec_point_add",
"test_ec_point_mul",
"test_generate",
"test_marshal",
"test_sign_verify",
"test_sign_verify_raw"]
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):003:0> p filter
/\A(?=.*)(?!.*(?-mix:(?-mix:memory_leak)|(?-mix:OpenSSL::TestEC.test_check_key)))/
=> /\A(?=.*)(?!.*(?-mix:(?-mix:memory_leak)|(?-mix:OpenSSL::TestEC.test_check_key)))/
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):004:0> method = "test_check_key"
=> "test_check_key"
~~~
The intention here is to exclude the `test_check_key` test case.
Unfortunately this does not work as expected, because the negative filter
is never checked:
~~~
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):005:0> filter === method
=> true
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):006:0> filter === "#{suite}##{method}"
=> false
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):007:0> filter === method || filter === "#{suite}##{method}"
=> true
~~~
Therefore always filter against the fully qualified method name
`#{suite}##{method}`, which should provide the expected result.
However, if plain string filter is used, keep checking also only the
method name.
This resolves [Bug #16936].
Needs to override Test::Unit::Runner#run, so that RunCount#run
runs which increments @@run_count. Previously it worked because
these methods were inserted between Test::Unit::Runner#run and
MiniTest::Unit#run.
