We don't show succeeding tests in the summary, and for all intents and purposes,
these tests have succeeded, in that they behave as expected. I've seen the
output confuse people on multiple occasions, for example GH-17105.
Closes GH-17109
There is now a workaround in `system_with_timeout()` to avoid issues
with quotes and spaces in the filenames of the executable by using
`start`[1]. However, calling `start` will not propagate the process
status of the actual process. Thus, calling `proc_get_status()`
is pretty meaningless, and especially Microsoft errors cannot be
detected (typically, access violations etc.), and as such no "Termsig"
message is output.
We fix this by executing `exit` after the started command has finished.
[1] <a6d7d5234b/run-tests.php (L1157-L1162)>
Closes GH-15378.
This was an error in thinking on my side. The reason these tests are
unrepeatable is because --repeat executes the same request in the same process,
and does not run the CLEAN section in between runs. This is not the case when
retrying tests.
We could potentially make CLEAN tests repeatable by including the CLEAN section
in the tested script. This does however not work for all tests (e.g. tests that
set open_basedir).
Closes GH-12072
This fixes builds without cgi or phpdbg:
./configure --disable-cgi --disable-phpdbg
make
./sapi/cli/php run-tests.php
Otherwise, deprecation warnings (since PHP-8.1) are emitted:
Deprecated: escapeshellarg(): Passing null to parameter #1 ($arg) of
type string is deprecated in run-tests.php...
CURL: 404: Page Not Found
IMAP: Can't create a temporary mailbox: [ALREADYEXISTS] Mailbox already exists
Sockets: socket_bind(): Unable to bind address [98]: Address already in use
We have lots of spurious failures in CI, many of them with the "all" CONFLICT.
We're limiting the retrying to specific error messages. In the future we may
also provide a FLAKY section to retry specific tests.
Closes GH-10892
Multiple tests had to be changed to escape the arguments in shell
commands. Some tests are skipped because they behave differently with
spaces in the path versus without. One notable example of this is the
hashbang test which does not work because spaces in hashbangs paths are
not supported in Linux.
Co-authored-by: Michael Voříšek <mvorisek@mvorisek.cz>
run-tests.php with `-jN` can hang if a parallel worker dies without notice. This
can happen due to fatal errors in the worker, or if the worker is killed.
- run-tests.php (main process)
\_ run-tests.php (worker #0) // main process hangs if this one crashes
\_ test-001.php (test-001.phpt)
