Commit graph

209 commits

Author SHA1 Message Date
Nobuyoshi Nakada
04925c6608
Add --keep-repeating option
It directs the program to continue repeating the tests the specified
number of times, even if any tests fail along the way.
2025-06-15 23:59:46 +09:00
Nobuyoshi Nakada
36a04de9f0
Dump with debugger before killing stuck worker 2025-06-15 20:24:30 +09:00
Nobuyoshi Nakada
f7af75d3d9 Show test task names longer 2025-03-06 16:58:13 +09:00
Naoto Ono
48b6c60969
[Misc #20661] Stop retrying tests in make test-all command by default (#11271)
[Misc #20661] Stop retrying tests in make test-all command by default
2025-02-27 06:12:02 +00:00
Hiroshi SHIBATA
a450e9304d Extract Benchmark.measure on assert_cpu_usage_low 2025-01-10 13:27:16 +09:00
Nobuyoshi Nakada
14c72e96ae
Remove an unnecessary blank line added by mistake [ci skip] 2024-09-10 17:19:02 +09:00
Nobuyoshi Nakada
2d12fbc4db Add predicates for platforms 2024-09-10 16:50:21 +09:00
Nobuyoshi Nakada
e884d65094
Ignore blocks to Test::Unit::Assertions#pend silently
Although `pend` in test-unit gem expects the block does not pass if it
is given, our `pend` should ignore the block silently as same as it
just skips the test with no message by default.  Add an unused
parameter, to suppress the warning for the block to be ignored.
2024-09-04 14:49:25 +09:00
Hiroshi SHIBATA
64a005df3b make test-tool with Windows is broken. We should use :KILL with that platform.
```
[ 14/191] TestParallel::TestParallel#test_hungup = 11.02 s
  1) Failure:
TestParallel::TestParallel#test_hungup [C:/Users/hsbt/DevDrive/github.com/ruby/ruby/tool/test/testunit/test_parallel.rb:211]:
Expected /^Retrying hung up testcases\.+$/ to match "Run options: \n" +
"  --seed=35582\n" +
"  --ruby\n" +
"  \".\\\\ruby.exe -I../../ruby/lib -I.ext/x64-mswin64_140 -I. ../../ruby/tool/runruby.rb --extout=.ext -- --disable-gems\"\n" +
"  -j\n" +
"  t1\n" +
"  --worker-timeout=1\n" +
"\n" +
"# Running tests:\n" +
"\n" +
"[1/1] 50340=test4test_hungup.\n" +
"C:/Users/hsbt/DevDrive/github.com/ruby/ruby/tool/lib/test/unit.rb:418:in 'Process.kill': Invalid argument (Errno::EINVAL)\n" +
```
2024-08-22 11:29:48 +09:00
Yusuke Endoh
671f6e78b4 Use 20min for parallel test worker timeout 2024-08-07 10:17:43 +09:00
Yusuke Endoh
fdfde6ab2e Use SIGSEGV to kill a parallel test worker (to dump backtrace) 2024-08-07 09:44:06 +09:00
Yusuke Endoh
e62c2ce0e0 Wait a bit longer for worker quit 2024-08-07 03:54:47 +09:00
Yusuke Endoh
47c45d5c18 Add a log when SIGKILL is sent to a test parallel worker 2024-08-07 02:41:27 +09:00
Yusuke Endoh
2c35a37918 Fix a typo 2024-08-07 02:34:46 +09:00
Yusuke Endoh
c0938fd24c Use 3min for test parallel worker timeout again 2024-08-06 21:11:41 +09:00
Yusuke Endoh
062f85e141 Dump all-thread backtraces when test parallel worker exceeds time limit 2024-08-06 21:11:41 +09:00
Yusuke Endoh
24d46a6781 Extend the default timeout of parallel testing to one hour 2024-08-06 16:59:24 +09:00
Yusuke Endoh
30be7fb85d Extend the default timeout of parallel testing 2024-08-06 15:00:12 +09:00
Naoto Ono
8ede84aa95 Move the file location of launchable.rb 2024-07-08 10:15:04 +09:00
Naoto Ono
5b78925455 Integrate Launchable into make btest 2024-07-08 10:15:04 +09:00
Alan Wu
0dcb0fdfcd
Enable LeakChecker for RJIT previously disabled for MJIT (#10998)
RJIT doesn't spawn subprocesses so there should now be no need to
special case it.
2024-06-14 00:05:38 -07:00
Nobuyoshi Nakada
aabe718e64
Show the caller location of assertion methods
Not only defined in `Test::Unit` or `CoreAssertions`, also show the
caller location of assertion methods defined in the current class or
ancestors.
2024-05-08 17:24:36 +09:00
Kevin Newton
b4b39a6199 [PRISM] Enable test_methoddef_endless_command 2024-05-07 14:24:31 -04:00
Naoto Ono
56ecd8038a Launchable: Add lineNumber field 2024-04-03 16:17:30 +09:00
Naoto Ono
3d1d1435c1 Launchable: Refactor the logic of JsonStreamWriter 2024-04-02 21:55:32 +09:00
Nobuyoshi Nakada
114c0b71be
Inform failures in parallel tests before retrying
Displays for each failure which test it actually occurred in.  The
output destination follows the --{stdout,stderr}-on-failure option.
2024-02-29 18:05:00 +09:00
Naoto Ono
3371936b6f Add Launchable into CI 2024-02-23 14:10:01 +01:00
Nobuyoshi Nakada
1fa4d15ca7
Add Test::Unit::TestCase#method_name
For the compatibility with test-unit gem.
2024-02-21 15:45:53 +09:00
Yusuke Endoh
9d1b000bd1 Show the method owner in backtraces
```
test.rb:1:in 'Object#toplevel_meth': unhandled exception
        from test.rb:4:in 'Foo.class_meth'
        from test.rb:6:in 'Foo#instance_meth'
        from test.rb:11:in 'singleton_meth'
        from test.rb:13:in '<main>'
```

[Feature #19117]
2024-02-15 19:11:58 +09:00
Naoto Ono
a7ebe467a7 Add newline between end and def 2024-01-31 16:14:50 +09:00
Naoto Ono
45b53dd897 Add the ability to generate Launchable test reports 2024-01-31 16:14:50 +09:00
Takashi Kokubun
c25705c267 Get rid of obsoleted RJITFirst
It was renamed from test_mjit, but we did not maintain it as test_rjit.
We test RJIT very differently.
2023-12-22 20:20:42 -08:00
Yusuke Endoh
5bf75c20a2 Refactor the settings of test-all out
test/runner.rb and tool/lib/test/unit/parallel.rb must use the same
settings. However, some settings were copied and pasted, while some were
added only to test/runner.rb.
This changeset creates tool/test/init.rb for all settings of test-unit,
which is loaded not only by test/runner.rb but also
tool/lib/test/unit/parallel.rb.

Background: the GEM_HOME environment variable was removed in
test/runner.rb, which prohibit `require "rake"` (note that rake is a
bundled gem). However the parallel mode didn't refrect this setting,
i.e., `require "rake"` was allowed.
This leads to an inconsistency, which actually affected a test test
defines s test class *only when* `require "rake"` is successful.
(test/rubygems/test_gem_package_task.rb)

1851105563 (step):8:1714
```
  /home/runner/work/ruby/ruby/src/tool/lib/test/unit.rb:729:in `const_get': uninitialized constant TestGemPackageTask (NameError)

                suites.map! {|r| ::Object.const_get(r[:testcase])}
                                         ^^^^^^^^^^
```
2023-11-15 19:27:10 +09:00
lukeg
7717684d16 tool test/unit/testcase: rename vars @passed, @@current
to @__passed__, @@__current__.
@passed is redefined in a few test suites, and this could lead to bugs.

Also rename @options (Runner#options) to @__runner_options__, which is
only used in make test-tool anyway.
2023-10-26 12:39:13 +09:00
Nobuyoshi Nakada
38b79b05fd
Fallback job status to normal if no tty 2023-10-21 19:47:05 +09:00
Nobuyoshi Nakada
4f4016497e
Handle Timeout::Error reported from workers 2023-10-18 23:41:23 +09:00
Nobuyoshi Nakada
ff4c98f125
Fix total count when filters given 2023-10-18 12:19:52 +09:00
Nobuyoshi Nakada
f13068afaa
Escape method names containing control characters 2023-10-18 10:36:42 +09:00
Nobuyoshi Nakada
776d4dec69 Manage parallel test workers after timeout 2023-10-18 02:55:42 +09:00
Hiroshi SHIBATA
141102b0b0 Expose Test.filter_backtrace for the default gems. 2023-08-29 15:05:30 +09:00
Nobuyoshi Nakada
69292676a2
Add message for --ruby option [ci skip] 2023-08-08 09:21:45 +09:00
Nobuyoshi Nakada
4589056384 Exit with a failure if any test files failed to load 2023-06-03 20:22:10 +09:00
Takashi Kokubun
23ec248e48 s/mjit/rjit/ 2023-03-06 23:44:01 -08:00
Takashi Kokubun
2e875549a9 s/MJIT/RJIT/ 2023-03-06 23:44:01 -08:00
lukeg
ba55706fb6 fix test/rubygems/test_gem_package_task.rb when in -j mode
This test skipped sometimes due to failure to load 'rake/packagetask'.
This is due to manipulation of $LOAD_PATH by other rubygems tests. If
rake is loaded before any rubygems tests run, then it works fine.

To reproduce the skipping behavior:
  $ make test-all TESTOPTS="-j6 --test-order=sorted test/rubygems/test_*.rb"
2023-02-28 06:34:27 +09:00
Hiroshi SHIBATA
ba45be3c43
Supressing warnings messages like:
Ignoring debug-1.7.1 because its extensions are not built. Try: gem pristine debug --version 1.7.

  Revert "Clear gem paths for each test"

  This reverts commit 6698b580dd.
2023-01-18 16:14:00 +09:00
Nobuyoshi Nakada
6698b580dd Clear gem paths for each test
So that rubygems can find the bundled rake.
2023-01-15 23:57:43 +09:00
Nobuyoshi Nakada
d752cf7601 Use class methods of File over Kernel.open and IO.read 2022-12-01 02:28:49 +09:00
Nobuyoshi Nakada
ab01b8f23f
jobserver option may not be at the last 2022-11-07 10:08:09 +09:00
Nobuyoshi Nakada
180d37c817
Set autoclose: for inherited FDs only [ci skip] 2022-11-07 09:06:15 +09:00