Commit graph

711 commits

Author SHA1 Message Date
Ilija Tovilo
75970077b0
Migrate i386 to GitHub actions
Closes GH-9856
2022-11-03 14:34:54 +01:00
Remi Collet
fe4aba6588
typo 2022-03-29 10:20:21 +02:00
Remi Collet
23cce68d24
run-tests.php: fix TypeError: Unsupported operand types: string * int <n> is mandatory for --show-slow and --set-timeout use <n> in help message instead of confusing [n] 2022-03-29 09:54:40 +02:00
Christoph M. Becker
87d9e02f01
Don't truncate subsecond precision in run-tests.php JUNIT output
When run-tests.php has been typed[1], the type of `$time` has been
chosen to be `int`.  This, however, leads to truncation, and the
somewhat relevant subsecond precision is lost.  We fix that by
changing the type to `float`, although `int|string` would be more
appropriate, but requires PHP ≥ 7.4.0.  Another option would be to
move the `number_format()` formatting into `junit_mark_test_as()`.

[1] <11274f53e7>

Closes GH-7836.
2021-12-27 22:15:33 +01:00
Bob Weinand
c916613f13 Fix null handling in test runner when printing env 2021-06-04 16:29:14 +02:00
Bob Weinand
76a4ea5e2a Export ENV in .sh file generated by run-tests 2021-05-31 10:47:14 +02:00
Nikita Popov
090627048c Revert "Remove no longer used "log_errors_max_len" ini directive (#6838)"
This reverts commit d2d227e547.

This is an ABI break.
2021-05-08 20:54:53 +02:00
Michael Voříšek
d2d227e547
Remove no longer used "log_errors_max_len" ini directive (#6838) 2021-05-08 13:05:13 -05:00
Christoph M. Becker
fb683030bf Merge branch 'PHP-7.4' into PHP-8.0
* PHP-7.4:
  Capture Microsoft-defined HRESULT exit codes exit codes
2021-02-25 12:03:43 +01:00
Dylan K. Taylor
a480bf8093 Capture Microsoft-defined HRESULT exit codes exit codes
The lack of such a check leads to false-passes of tests on Windows
which expect no output, but produce a segfault or similar issue. I
discovered this a while ago due to bad tests in an extension I maintain.

Closes GH-6722.
2021-02-25 12:01:04 +01:00
Christoph M. Becker
8162c3d1fd Merge branch 'PHP-7.4' into PHP-8.0
* PHP-7.4:
  run-tests: fixed exit code not being set on BORKED tests
2021-02-23 13:22:50 +01:00
Christoph M. Becker
67bb06cc9b Merge branch 'PHP-7.3' into PHP-7.4
* PHP-7.3:
  run-tests: fixed exit code not being set on BORKED tests
2021-02-23 13:15:30 +01:00
Dylan K. Taylor
729cd8bacd run-tests: fixed exit code not being set on BORKED tests
When no test paths are specified this shows up when 'make test' is used on a PECL extension without specifying tests to run (or in php-src too, I guess...)

Closes GH-6717.
2021-02-23 12:50:03 +01:00
Nikita Popov
cbdd21a22d Color LEAK&FAIL like a failure 2021-02-17 16:40:19 +01:00
Nikita Popov
15073d8e1d Fixed bug #80437
Only chmod if the file has been created.
2020-11-30 11:40:46 +01:00
bashkarev
f17b2a75a5 Don't set xdebug.default_enable in run-tests
This throws a deprecation warning in XDebug 3.

Closes GH-6324.
2020-10-26 13:10:26 +01:00
Nikita Popov
b2700811f3 Skip arginfo test under msan
Msan is missing interceptors for some functions that result in
false positives.
2020-10-15 10:40:23 +02:00
Nikita Popov
9623756ab7 Remove unused --temp-urlbase option from run-tests.php
This was part of the removed HTML functionality.
2020-09-23 12:34:59 +02:00
Berbe
bfeb2f6abc Fix: Gracefully handle empty user input in run-tests.php
Closes GH-6016.
2020-08-24 11:46:01 +02:00
Tyson Andre
0461c1574a Add run-tests.php --context [n] option.
Mentioned in https://github.com/php/php-src/pull/5965#discussion_r467621123

This PR proposes 3 lines of context so the impact can be seen in tests.
Other `diff` programs show around 3 lines of context.
(This helps indicate exactly which position a test should be updated
to add a new expected line at)

Use the mapping for choosing order to display diffs

Properly include context in cases where the expected output had more lines than
the actual output, e.g.

```
--FILE--
A
A1
A
C
NEARBY
--EXPECTF--
A
B
A1
B
A
B
A
B
NEARBY
```

Closes GH-5968
2020-08-16 09:31:38 -04:00
Tyson Andre
7be61bec80 Colorize --show-diff, only on stdout
Don't add colors to the saved `.diff` file.

Related to a41cf3e1d2
2020-08-13 11:27:32 -04:00
Tyson Andre
45d69fb809 Support NO_COLOR environment variable in run-tests.php
And add a --color option for run-tests.php

See https://no-color.org/

> an informal standard is hereby proposed:
>
> All command-line software which outputs text with ANSI color added should check
> for the presence of a `NO_COLOR` environment variable that, when present
> (regardless of its value), prevents the addition of ANSI color.
2020-08-12 09:37:45 -04:00
Nikita Popov
a41cf3e1d2 Revert "Make diff section contents red(-)/green(+) in run-tests.php"
This reverts commit 58489bd3b9.

This also uses ANSI escape code coloring for .diff files, where it
is not meaningful.
2020-08-10 09:38:48 +02:00
Tyson Andre
58489bd3b9 Make diff section contents red(-)/green(+) in run-tests.php
This uses green for lines with `+` and red for lines with `-`.

Colors(Red and Green) would make the failure causes stand out visually when scrolling through errors.

Closes GH-5965
2020-08-09 20:59:43 -04:00
Tyson Andre
e79adf6381 Use short array values in run-tests.php
The run-tests.php file already requires php 7 syntax.
Done automatically with phpcbf with no other changes done/planned.

This file is already using short arrays in other places.

run-tests has already been substantially refactored in PHP 8.0
compared to PHP 7.4.

This makes merge conflicts more likely, but changes to run-tests.php on older
versions are less frequent bug fixes.
2020-08-09 16:09:55 -04:00
George Peter Banyard
5d72e40dfb Add colours to test results in run-tests.php
The test runner will attempt to colourise the result of each test

The --no-color option is introduced to disable this feature.

Closes GH-5901
2020-08-07 18:17:34 +02:00
George Peter Banyard
11274f53e7 Add type information to test-runner
Closes GH-5942
2020-08-06 20:37:36 +02:00
Tyson Andre
07db64156e [RFC] Make string length for getTraceAsString() configurable
Add a `zend.exception_string_param_max_len` ini setting.
(same suffix as `log_errors_max_len`)

Allow values between 0 and 1000000 bytes.
For example, with zend.exception_string_param_max_len=0,
"" would represent the empty string, and "..." would represent something
longer than the empty string.
Previously, this was hardcoded as exactly 15 bytes.

Discussion: https://externals.io/message/110717

Closes GH-5769
2020-07-25 09:50:57 -04:00
Dmitry Stogov
69b1f3d53d Use small counter values for testing 2020-07-08 16:49:51 +03:00
Paul Dragoonis
5d7ff25311 Removing HTML Functionality from run-tests.php
As discussed on GH-5632, the HTML functionality does not appear
to be in active use. For HTML rendering of test results, it is
suggested to instead use the JUnit integration, in combination
with your favorite JUnit viewer.

Closes GH-5705.
2020-06-19 09:59:38 +02:00
Nikita Popov
c260613c6f Optimize parallel test runner startup
Don't connect to each worker individually: First spawn all processes
and then establish connections in any order.

This avoids a ~1s wait when running on many cores.
2020-06-16 11:39:03 +02:00
Gerard Roche
d1dde094f2 run-tests: refactor 2020-05-22 09:58:04 +02:00
Gerard Roche
2ef88f517d run-tests: echo message and exit if no tests found
The test runner currently defaults to running the entire test suite if
no selected tests can be found. This can be unexpected.

For example the ext/mysqlnd/ directory has no tests, if you specify that
directory when testing the entire test suite will be run.

    run-tests.php [options] ext/mysqlnd/

Closes GH-5605.
2020-05-22 09:37:46 +02:00
Nikita Popov
0374cf08c7 Add --file-cache-prime/use options to run-tests
--file-cache-prime populates the file cache,
--file-cache-use uses the file cache.

And fix a number of tests to run under file cache or disabled
timestamp validation.
2020-05-20 15:29:32 +02:00
Nikita Popov
ec1e8540d8 Slightly deduplicate run-tests code 2020-05-20 11:58:42 +02:00
Derick Rethans
3707d75f4b Merge branch 'PHP-7.4' 2020-05-18 13:51:47 +01:00
Derick Rethans
d2797505cd Put Xdebug 3 into 'off' mode for run-tests.php 2020-05-18 13:51:40 +01:00
Derick Rethans
74a4a5ae55 Put Xdebug 3 into 'off' mode for run-tests.php 2020-05-18 13:49:12 +01:00
Gerard Roche
ecc0a87ff2 run-tests: extract usage message
Put the usage message near the top of the script,
into a separate function.

Closes GH-5558.
2020-05-13 09:54:55 +02:00
Gerard Roche
10edee7f03 run-tests: cs fixes (cleanup)
I used php-cs-fixer to do the cs fixes. The configuration I used is
posted below. The reason I disabled some of the rules is because they
create too much noise and would make it difficult to review. But please
feel free to close this PR and run the php-cs-fixer yourself.

    <?php

    $config = PhpCsFixer\Config::create();
    $config->setRiskyAllowed(false);
    $config->setRules([
        '@PSR2' => true,
        '@Symfony' => true,
        'array_syntax' => false,
        'binary_operator_spaces' => false,
        'blank_line_before_statement' => false,
        'concat_space' => false,
        'increment_style' => false,
        'phpdoc_align' => false,
        'single_quote' => false,
        'trailing_comma_in_multiline_array' => false,
        'unary_operator_spaces' => false,
        'yoda_style' => false,
    ]);

    $finder = PhpCsFixer\Finder::create();
    $finder->in(getcwd());

    $finder->exclude('Zend');
    $finder->exclude('build');
    $finder->exclude('ext');
    $finder->exclude('pear');
    $finder->exclude('sapi');
    $finder->exclude('scripts');
    $finder->exclude('win32');

    $config->setFinder($finder);

    return $config;

Closes GH-5557.
2020-05-13 09:52:44 +02:00
Gerard Roche
16f23cd159 run-tests: remove use of FILE_BINARY constant
The FILE_BINARY (and FILE_TEXT) constants are not really valid or useful
constants. It looks like they were added in 5.2.7 and have "no effect,
and are only available for forward compatibility."

See: https://www.php.net/manual/en/filesystem.constants.php

The default value of the file_put_contents() flags parameter is 0 and
FILE_BINARY is set to 0, so removing it doesn't change functionality.

P.S. Maybe those constants should be deprecated or removed in 8.0.

Closes GH-5556.
2020-05-12 16:59:25 +02:00
Christoph M. Becker
8cb237345a Revert "Show eventual output of clean sections"
This reverts commit 5eb4ab07f2.
The temporary hack has served its purpose.
2020-04-30 09:51:10 +02:00
Christoph M. Becker
daaaf2b4ff Merge branch 'PHP-7.4'
* PHP-7.4:
  Pass worker ID to clean scripts
2020-04-29 13:33:32 +02:00
Christoph M. Becker
9a98569efe Pass worker ID to clean scripts
On Windows, reusing/sharing of OPcache instances with different
configuration is not necessarily supported, so we have to make that it
does not happen for the clean scripts, by using `$orig_ini_settings`
instead of `$clean_params`.
2020-04-29 13:30:28 +02:00
Christoph M. Becker
5eb4ab07f2 Show eventual output of clean sections
This is a hack to investigate why mysqli_insert_packet_overflow.phpt
intermittently fails on AppVeyor, and will be reverted in due time.
See <https://github.com/php/php-src/pull/5447#issuecomment-620508790>.
2020-04-28 16:08:17 +02:00
Flávio Heleno
1baa58317f Fixed run-tests.php for PHP 7.2
Flexible heredoc syntax is only available since PHP 7.3.

Closes GH-5444.
2020-04-27 11:55:40 +02:00
Benjamin Eberlei
ec88bbc4c6 Enhance test failure SH script to allow gdb, valgrind, rr as alternative modes. 2020-04-18 16:14:00 +02:00
Nikita Popov
fd5dc55759 Force short_open_tag=0 in run-tests.php
Make sure we don't accidentially add tests depending on short tags.
2020-04-15 11:27:03 +02:00
Nikita Popov
a939805641 Use serialize_precision for var_dump()
var_dump() is debugging functionality, so it should print
floating-point numbers accurately. We do this by switching
to serialize_precision, which (by default) will print with
as much precision as necessary to preserve the exact value
of the float.

This also affects debug_zval_dump().

Closes GH-5172.
2020-02-25 09:51:32 +01:00
Christoph M. Becker
7d6384d1f7 Merge branch 'PHP-7.4'
* PHP-7.4:
  Create a new console for each test worker on Windows
2020-02-17 18:54:32 +01:00