Commit graph

143 commits

Author SHA1 Message Date
Vít Ondruch
5086c25f60 Properly exclude test cases.
Lets consider the following scenario:

~~~
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):001:0> p suite
OpenSSL::TestEC
=> OpenSSL::TestEC

irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):002:0> p all_test_methods
["test_ECPrivateKey", "test_ECPrivateKey_encrypted", "test_PUBKEY", "test_check_key", "test_derive_key", "test_dh_compute_key", "test_dsa_sign_asn1_FIPS186_3", "test_ec_group", "test_ec_key", "test_ec_point", "test_ec_point_add", "test_ec_point_mul", "test_generate", "test_marshal", "test_sign_verify", "test_sign_verify_raw"]
=>
["test_ECPrivateKey",
 "test_ECPrivateKey_encrypted",
 "test_PUBKEY",
 "test_check_key",
 "test_derive_key",
 "test_dh_compute_key",
 "test_dsa_sign_asn1_FIPS186_3",
 "test_ec_group",
 "test_ec_key",
 "test_ec_point",
 "test_ec_point_add",
 "test_ec_point_mul",
 "test_generate",
 "test_marshal",
 "test_sign_verify",
 "test_sign_verify_raw"]

irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):003:0> p filter
/\A(?=.*)(?!.*(?-mix:(?-mix:memory_leak)|(?-mix:OpenSSL::TestEC.test_check_key)))/
=> /\A(?=.*)(?!.*(?-mix:(?-mix:memory_leak)|(?-mix:OpenSSL::TestEC.test_check_key)))/

irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):004:0> method = "test_check_key"
=> "test_check_key"
~~~

The intention here is to exclude the `test_check_key` test case.
Unfortunately this does not work as expected, because the negative filter
is never checked:

~~~
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):005:0> filter === method
=> true

irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):006:0> filter === "#{suite}##{method}"
=> false

irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):007:0> filter === method || filter === "#{suite}##{method}"
=> true
~~~

Therefore always filter against the fully qualified method name
`#{suite}##{method}`, which should provide the expected result.

However, if plain string filter is used, keep checking also only the
method name.

This resolves [Bug #16936].
2021-10-27 19:56:19 +09:00
Koichi Sasada
8a49c7e481 Revert "introduce check code for mysterious EBADF"
This reverts commit 7864efa105.
2021-10-26 17:06:57 +09:00
Koichi Sasada
d1424f7f7e Revert "check other IO#close calls"
This reverts commit a1c4cab11d.
2021-10-26 17:06:57 +09:00
Koichi Sasada
a1c4cab11d check other IO#close calls
http://ci.rvm.jp/results/trunk@ruby-iga/3690333
> tool/lib/test/unit/parallel.rb:68:in `close': Bad file descriptor (Errno::EBADF)
2021-10-25 17:38:23 +09:00
Koichi Sasada
7864efa105 introduce check code for mysterious EBADF
parallel test randomly failed with EBADF.
This patch checks wich suite causes this error.

ex) http://ci.rvm.jp/results/trunk@ruby-iga/3690219

```
/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:88:in `close': Bad file descriptor (Errno::EBADF)
	/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:88:in `ensure in _run_suite'
	/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:89:in `_run_suite'
	/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:30:in `block in _run_suites'
	/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:29:in `map'
	/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:29:in `_run_suites'
	/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:128:in `run'
	/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:211:in `<main>'
```
2021-10-25 15:47:09 +09:00
Nobuyoshi Nakada
6c74976aef Remove Test::Unit::TestCase.make_my_diffs_pretty!
`Test::Unit::CoreAssertions#mu_pp` is defined always using
`pretty_inspect`.
2021-10-18 17:36:06 +09:00
Nobuyoshi Nakada
d1b8544b4d Remove Test::Unit::Assertions#exception_details
Never used since fa0f3eff22.
2021-10-18 17:36:06 +09:00
Nobuyoshi Nakada
c7eac1be51
Retry hung up tests verbosely 2021-10-17 22:40:31 +09:00
Nobuyoshi Nakada
13716898df Retry hung tests after parallel runs 2021-10-17 16:33:58 +09:00
Nobuyoshi Nakada
478187e9a3 Timeout parallel test worker processes 2021-10-17 16:33:58 +09:00
Nobuyoshi Nakada
5de6e25144
Replace unpack with unpack1 2021-10-17 00:48:45 +09:00
Nobuyoshi Nakada
370fff98ce
Prefer require_relative 2021-10-15 18:47:25 +09:00
Nobuyoshi Nakada
679054208e
Use __dir__ 2021-10-15 15:07:31 +09:00
Nobuyoshi Nakada
cec61d16d2
Run JIT tests first when random order instead of no-sort 2021-10-10 19:23:52 +09:00
Nobuyoshi Nakada
fb9ea391cf
Default the test order to random 2021-10-10 18:30:33 +09:00
Nobuyoshi Nakada
fc913ad21d
Exclude also core_assertions.rb from backtraces 2021-10-09 16:17:37 +09:00
Nobuyoshi Nakada
7c98e673d7
Fix a typo since 688f2e1a89 2021-10-05 18:24:34 +09:00
Nobuyoshi Nakada
c4570acc86
Refactor ordering of tests
* Split the sorting types into classes.
* Apply the same sorting to method sorting under the parallel
  test.
2021-10-04 20:46:47 +09:00
Nobuyoshi Nakada
34d6720fee
Prohibit test method redefinition 2021-09-17 08:54:18 +09:00
Hiroshi SHIBATA
ec6f04c092
Use capture_output instead of capture_io 2021-09-13 21:14:42 +09:00
Hiroshi SHIBATA
be04006c7d
Rename capture_io to capture_output and deprecate to use capture_io 2021-09-13 20:56:34 +09:00
Hiroshi SHIBATA
455978cd13
Removed unused mu_pp method 2021-09-13 20:41:03 +09:00
Hiroshi SHIBATA
f5993331b2
Renamed skip to pend and prepared to deprecate skip method 2021-09-13 19:09:24 +09:00
Hiroshi SHIBATA
dbdceb8a19 Removed workaround for test_order 2021-09-13 14:39:23 +09:00
Hiroshi SHIBATA
2982464a8e
Revert "Resolved conflict initializers"
This reverts commit 62db6e47b6.

  308183fffa is the correct solution
  about this.
2021-09-13 11:07:32 +09:00
Nobuyoshi Nakada
bbd105c445
Exclude option_parser from Test::Unit::Runner#inspect 2021-09-12 03:51:19 +09:00
Nobuyoshi Nakada
308183fffa
Prepend the modules in Test::Unit
Needs to override Test::Unit::Runner#run, so that RunCount#run
runs which increments @@run_count.  Previously it worked because
these methods were inserted between Test::Unit::Runner#run and
MiniTest::Unit#run.
2021-09-12 03:45:58 +09:00
Hiroshi SHIBATA
8d0315a2bb
Removed unsed assertions for rubygems 2021-09-11 21:05:15 +09:00
Hiroshi SHIBATA
d58e0ffc24
Removed unused alias for run_test 2021-09-11 20:35:18 +09:00
Hiroshi SHIBATA
a13ee1f33c
Removed rubinius support from test suite 2021-09-11 18:10:15 +09:00
Hiroshi SHIBATA
aebd0a873e
Removed maglev related code because it's not active status now 2021-09-11 18:10:15 +09:00
Nobuyoshi Nakada
952806cdec
Update comments for minitest 2021-09-11 17:37:49 +09:00
Nobuyoshi Nakada
cd83e716d0
Update an option message 2021-09-11 17:32:42 +09:00
Nobuyoshi Nakada
911f62e681
Directly alias orig_run_suite 2021-09-11 17:19:02 +09:00
Nobuyoshi Nakada
46bde60df3
Remove method filter hack for minitest 2021-09-11 17:12:27 +09:00
Hiroshi SHIBATA
f73f9e1f89
Replace minitest to test for backtrace_filter 2021-09-11 15:17:38 +09:00
Hiroshi SHIBATA
b178929469
Fixed uninitialized constant Test::Unit::Assertions::MIN_MEASURABLE.
I'm not sure why it's failed under the chkbuild and rubyci.

  20210911T000007Z.fail.html.gz
2021-09-11 11:21:37 +09:00
Hiroshi SHIBATA
d05383812a Replace Test::Assertion and Test::Skip to Test::Unit::AssertionFailedError and Test::Unit::PendedError 2021-09-11 08:48:03 +09:00
Hiroshi SHIBATA
7cec81e073 Removed needless require 2021-09-11 08:48:03 +09:00
Hiroshi SHIBATA
151a1cd45e Fixed inconsistent require order for assertions and core_assertions 2021-09-11 08:48:03 +09:00
Hiroshi SHIBATA
a3b0518672 Removed obsoleted constant 2021-09-11 08:48:03 +09:00
Hiroshi SHIBATA
9b026ca39b Integrate Minitest to Test 2021-09-11 08:48:03 +09:00
Hiroshi SHIBATA
0598e87abf Use prepend for override to status line 2021-09-11 08:48:03 +09:00
Hiroshi SHIBATA
8ee38bdd7d Removed duplicated autorun method 2021-09-11 08:48:03 +09:00
Hiroshi SHIBATA
62db6e47b6 Resolved conflict initializers 2021-09-11 08:48:03 +09:00
Hiroshi SHIBATA
7ef0a4634c Removed unused _run_suites 2021-09-11 08:48:03 +09:00
Hiroshi SHIBATA
988eda744c Removed deprecated process_args 2021-09-11 08:48:03 +09:00
Hiroshi SHIBATA
206af4727c Merge poke methods 2021-09-11 08:48:03 +09:00
Hiroshi SHIBATA
1fd2a66a93 Merge run methods 2021-09-11 08:48:03 +09:00
Hiroshi SHIBATA
374cc7231e Removed VERSION constant 2021-09-11 08:48:03 +09:00