Commit graph

512 commits

Author SHA1 Message Date
Nobuyoshi Nakada
983f2688db
Hack to avoid leak checker 2022-05-25 18:48:01 +09:00
Takashi Kokubun
ead96e7b44
Rename test_jit to test_mjit
to avoid confusion with YJIT
2022-05-20 21:32:55 -07:00
Hiroshi SHIBATA
c506ddac6c
Added assert_true and assert_false same as test-unit gem 2022-04-20 14:13:10 +09:00
Alan Wu
21c8b9eb99 Copyedit parallel test harness crash message
Mea culpa.

Co-authored-by: Kaíque Kandy Koga <kaiquekandykoga@gmail.com>
2022-04-06 18:39:27 -04:00
Alan Wu
399ef463f7 Copyedit parallel test harness crash message 2022-04-06 18:11:43 -04:00
Kazuhiro NISHIYAMA
d9c6f8d45c
Try to fix NoMethodError on slow environments
```
  1) Failure:
TestParallel::TestParallel#test_hungup [/home/user/ruby/tool/test/testunit/test_parallel.rb:215]:
Expected /^Retrying hung up testcases\.+$/ to match "Run options: \n" +
"  --seed=43403\n" +
"  --ruby\n" +
"  \"./miniruby -I../lib -I. -I.ext/common ../tool/runruby.rb --extout=.ext -- --disable-gems\"\n" +
"  -j\n" +
"  t1\n" +
"  --worker-timeout=1\n" +
"\n" +
"# Running tests:\n" +
"\n" +
"/home/user/ruby/tool/lib/test/unit.rb:687:in `block in _run_parallel': undefined method `<' for nil:NilClass (NoMethodError)\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:538:in `block in quit_workers'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:537:in `reject!'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:537:in `quit_workers'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:687:in `_run_parallel'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:810:in `_run_suites'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:849:in `_run_suites'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:1479:in `_run_anything'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:1263:in `_run_anything'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:1654:in `run_tests'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:1641:in `block in _run'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:1640:in `each'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:1640:in `_run'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:1682:in `run'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:1034:in `run'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:882:in `run'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:154:in `run'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:1761:in `run'\n" +
"\tfrom /home/user/ruby/tool/lib/test/unit.rb:1765:in `run'\n" +
"\tfrom /home/user/ruby/tool/test/testunit/tests_for_parallel/runner.rb:14:in `<main>'\n".
```
2022-04-05 19:23:38 +09:00
Nobuyoshi Nakada
8780f15fd7 [ruby/date] Use assert_deprecated_warn
c55004715a
2022-02-25 20:39:47 +09:00
Koichi Sasada
00c7a0d491 fix parallel test timeout retrying
On the parallel test, workers can be killed because of timeout
and the information for the retrying can be inconsistent.
This patch will skip if the inconsistency is found and report
as an error.

http://ci.rvm.jp/results/trunk-asserts@phosphorus-docker/3834082
2022-02-16 17:57:04 +09:00
Jesse Chavez
99d02caed3 [ruby/logger] Fix log rotation inter-process lock failed.
Issue only occurs in JRuby 9.3.0.0 and Windows and the full
console output is:

log rotation inter-process lock failed. D:\log.txt -> D:\log.txt.0: The process cannot access the file because it is being used by another process.
log writing failed. closed stream
log writing failed. closed stream
...

19fc734638
2022-01-27 12:33:06 +09:00
Koichi Sasada
cac6fe9023 add --stderr-on-failure option to test-all
Now all failure messages are printed to stdout. This option
makes all failure messages printed into stderr.
2022-01-26 09:56:09 +09:00
David Rodríguez
de678cd51e
[ruby/pathname] Officially drop support for ruby 2.6 or older
The gem doesn't even install on old rubies, but since the gemspec claims
it's supported, `gem install pathname` will try to install it and print
an error.

This commit doesn't fix the above issue. The only way to fix it would be
to restore support and release a new version that actually supports old
rubies. However, such a change has been proposed and ignored for a long
time.

So this issue proposes to leave that broken but at least bring the
gemspec manifest and the CI matrix in sync to hopefully avoid this issue
from happening again in the future.

3ee010b538
2022-01-25 08:50:31 +09:00
Koichi Sasada
374904b25f remain RUBY_ON_BUG for child processes.
`RUBY_ON_BUG` is useful for child processes created by the test
process.
2022-01-19 23:17:14 +09:00
Hiroshi SHIBATA
3515867381 Removed skip alias in test suite 2022-01-17 10:39:24 +09:00
Takashi Kokubun
1a63468831
Prepare for removing RubyVM::JIT (#5262) 2021-12-13 23:07:46 -08:00
Nobuyoshi Nakada
438220f063
Let pretty_inspect work once at first
For older pp.rb which did not need io/console, and dealing with
`LoadError`.
2021-11-30 14:16:33 +09:00
Nobuyoshi Nakada
d400c44a55
Load io/console earlier
Something goes wrong at loading libraries inside `mu_pp` in the
test overriding `Class.inherited`.
2021-11-30 13:15:22 +09:00
Nobuyoshi Nakada
4acc7563a1 Consider environment variable case-insensitiveness 2021-11-29 23:00:20 +09:00
Nobuyoshi Nakada
3ff0a0b40c
Filter method names only if filtering method name only
If sole `filter` option doesn't seem including test case name,
match with method name only.
And if the filter is a Regexp or String, it never matches method
name symbols.
2021-11-09 15:20:44 +09:00
Vít Ondruch
5086c25f60 Properly exclude test cases.
Lets consider the following scenario:

~~~
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):001:0> p suite
OpenSSL::TestEC
=> OpenSSL::TestEC

irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):002:0> p all_test_methods
["test_ECPrivateKey", "test_ECPrivateKey_encrypted", "test_PUBKEY", "test_check_key", "test_derive_key", "test_dh_compute_key", "test_dsa_sign_asn1_FIPS186_3", "test_ec_group", "test_ec_key", "test_ec_point", "test_ec_point_add", "test_ec_point_mul", "test_generate", "test_marshal", "test_sign_verify", "test_sign_verify_raw"]
=>
["test_ECPrivateKey",
 "test_ECPrivateKey_encrypted",
 "test_PUBKEY",
 "test_check_key",
 "test_derive_key",
 "test_dh_compute_key",
 "test_dsa_sign_asn1_FIPS186_3",
 "test_ec_group",
 "test_ec_key",
 "test_ec_point",
 "test_ec_point_add",
 "test_ec_point_mul",
 "test_generate",
 "test_marshal",
 "test_sign_verify",
 "test_sign_verify_raw"]

irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):003:0> p filter
/\A(?=.*)(?!.*(?-mix:(?-mix:memory_leak)|(?-mix:OpenSSL::TestEC.test_check_key)))/
=> /\A(?=.*)(?!.*(?-mix:(?-mix:memory_leak)|(?-mix:OpenSSL::TestEC.test_check_key)))/

irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):004:0> method = "test_check_key"
=> "test_check_key"
~~~

The intention here is to exclude the `test_check_key` test case.
Unfortunately this does not work as expected, because the negative filter
is never checked:

~~~
irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):005:0> filter === method
=> true

irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):006:0> filter === "#{suite}##{method}"
=> false

irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):007:0> filter === method || filter === "#{suite}##{method}"
=> true
~~~

Therefore always filter against the fully qualified method name
`#{suite}##{method}`, which should provide the expected result.

However, if plain string filter is used, keep checking also only the
method name.

This resolves [Bug #16936].
2021-10-27 19:56:19 +09:00
Koichi Sasada
8a49c7e481 Revert "introduce check code for mysterious EBADF"
This reverts commit 7864efa105.
2021-10-26 17:06:57 +09:00
Koichi Sasada
d1424f7f7e Revert "check other IO#close calls"
This reverts commit a1c4cab11d.
2021-10-26 17:06:57 +09:00
Koichi Sasada
a1c4cab11d check other IO#close calls
http://ci.rvm.jp/results/trunk@ruby-iga/3690333
> tool/lib/test/unit/parallel.rb:68:in `close': Bad file descriptor (Errno::EBADF)
2021-10-25 17:38:23 +09:00
Koichi Sasada
7864efa105 introduce check code for mysterious EBADF
parallel test randomly failed with EBADF.
This patch checks wich suite causes this error.

ex) http://ci.rvm.jp/results/trunk@ruby-iga/3690219

```
/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:88:in `close': Bad file descriptor (Errno::EBADF)
	/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:88:in `ensure in _run_suite'
	/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:89:in `_run_suite'
	/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:30:in `block in _run_suites'
	/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:29:in `map'
	/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:29:in `_run_suites'
	/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:128:in `run'
	/tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:211:in `<main>'
```
2021-10-25 15:47:09 +09:00
Nobuyoshi Nakada
6c74976aef Remove Test::Unit::TestCase.make_my_diffs_pretty!
`Test::Unit::CoreAssertions#mu_pp` is defined always using
`pretty_inspect`.
2021-10-18 17:36:06 +09:00
Nobuyoshi Nakada
d1b8544b4d Remove Test::Unit::Assertions#exception_details
Never used since fa0f3eff22.
2021-10-18 17:36:06 +09:00
Nobuyoshi Nakada
c7eac1be51
Retry hung up tests verbosely 2021-10-17 22:40:31 +09:00
Nobuyoshi Nakada
13716898df Retry hung tests after parallel runs 2021-10-17 16:33:58 +09:00
Nobuyoshi Nakada
478187e9a3 Timeout parallel test worker processes 2021-10-17 16:33:58 +09:00
Nobuyoshi Nakada
5de6e25144
Replace unpack with unpack1 2021-10-17 00:48:45 +09:00
Nobuyoshi Nakada
370fff98ce
Prefer require_relative 2021-10-15 18:47:25 +09:00
Nobuyoshi Nakada
679054208e
Use __dir__ 2021-10-15 15:07:31 +09:00
Pavel Rosický
94882df3a2
[ruby/digest] jruby support
2e9dc14693
2021-10-12 20:46:13 +09:00
Nobuyoshi Nakada
cec61d16d2
Run JIT tests first when random order instead of no-sort 2021-10-10 19:23:52 +09:00
Nobuyoshi Nakada
fb9ea391cf
Default the test order to random 2021-10-10 18:30:33 +09:00
Nobuyoshi Nakada
c423cc932e
Refactor Test::Unit::CoreAssertions#assert_nothing_raised
* Separate exception classes to be rescued or reraised
* Use the filtered backtrace in the failure message
* Raise a new `AssertionFailedError` with the original backtrace
2021-10-09 18:14:21 +09:00
Nobuyoshi Nakada
fc913ad21d
Exclude also core_assertions.rb from backtraces 2021-10-09 16:17:37 +09:00
Nobuyoshi Nakada
3932227d96
Unify Test::Unit::Assertions#message
Merge `Test::Unit::CoreAssertions#message`.
2021-10-09 16:17:01 +09:00
Nobuyoshi Nakada
489c22b17f
Let Test::Unit::CoreAssertions::AllFailures#for yield the key
Similar to `Test::Unit::CoreAssertions::AllFailures#foreach`.
2021-10-09 14:28:27 +09:00
Nobuyoshi Nakada
7c98e673d7
Fix a typo since 688f2e1a89 2021-10-05 18:24:34 +09:00
Nobuyoshi Nakada
c4570acc86
Refactor ordering of tests
* Split the sorting types into classes.
* Apply the same sorting to method sorting under the parallel
  test.
2021-10-04 20:46:47 +09:00
Nobuyoshi Nakada
34d6720fee
Prohibit test method redefinition 2021-09-17 08:54:18 +09:00
Hiroshi SHIBATA
ec6f04c092
Use capture_output instead of capture_io 2021-09-13 21:14:42 +09:00
Hiroshi SHIBATA
be04006c7d
Rename capture_io to capture_output and deprecate to use capture_io 2021-09-13 20:56:34 +09:00
Hiroshi SHIBATA
455978cd13
Removed unused mu_pp method 2021-09-13 20:41:03 +09:00
Hiroshi SHIBATA
f5993331b2
Renamed skip to pend and prepared to deprecate skip method 2021-09-13 19:09:24 +09:00
Nobuyoshi Nakada
aaa9805e7e
Add unique token to separated runner
Same as Test::Unit::CoreAssertions#assert_no_memory_leak.
2021-09-13 15:47:45 +09:00
Hiroshi SHIBATA
dbdceb8a19 Removed workaround for test_order 2021-09-13 14:39:23 +09:00
Hiroshi SHIBATA
2982464a8e
Revert "Resolved conflict initializers"
This reverts commit 62db6e47b6.

  308183fffa is the correct solution
  about this.
2021-09-13 11:07:32 +09:00
Nobuyoshi Nakada
bbd105c445
Exclude option_parser from Test::Unit::Runner#inspect 2021-09-12 03:51:19 +09:00
Nobuyoshi Nakada
308183fffa
Prepend the modules in Test::Unit
Needs to override Test::Unit::Runner#run, so that RunCount#run
runs which increments @@run_count.  Previously it worked because
these methods were inserted between Test::Unit::Runner#run and
MiniTest::Unit#run.
2021-09-12 03:45:58 +09:00