Why:
- When tests are run in parallel, they are organized by groups--it is
useful to be able to see logs organized by each test group.
What:
- Set an environment variable in ComposerLaunchParallel,
`MW_PHPUNIT_SPLIT_GROUP_ID`
- If the environment variable is set, modify the file names used for
logs in DevelopmentSettings to use `split-group-{group ID}` in the
file name
- CI script for checking if any errors have been logged was updated
in I4283478a89d2a8d3a0502f836459b3fb21d99113
Bug: T375852
Change-Id: If5e08558a6efbb0139c320fda6f53ec73597ba60
Make it clearer in the class documentation that we can’t use most
MediaWiki services, including Shell::command(); that said, as far as I
can tell we should be able to use Shellbox (as it’s a separate library),
so do that.
Change-Id: I73f29214a1527bbbd3fd5c22b7476bdaa4627b03
In T361190 and Quibble 1.9.0, we introduced parallel execution of
PHPUnit tests to speed up the CI jobs. The existing implementation
is purely Python/Quibble, and cannot directly be used by developers
locally. With this patch, we re-implement the parallel test
execution already implemented in CI as a composer task so that the
parallel tests can be run locally.
The `phpunit:parallel:extensions` command expects to be run after
`phpunit:prepare-parallel:extensions`, and expects to find 8 test
suites with the names `split_group_X` (for X in 0 through 7) in the
PHPUnit configuration file. 8 here is currently a hard-coded number
that corresponds to the number of parallel test executions we need
to saturate the CPU of a 4-core developer machine, and experimentally
leads to a good speed-up versus the serial execution.
When this command runs, it forks to launch 8 parallel processes,
each running one of the `split_group_X` suites. The parent process
waits for the children to complete, buffers the output, collects the
exit statuses, then dumps the buffered output and exits with a
non-zero status if any of the child processes failed (i.e. if there
were test failures).
We introduce `phpunit:prepare-parallel:default` as a complement to
`phpunit:prepare-parallel:extensions`, and the two commands
`phpunit:parallel:database` and `phpunit:parallel:databaseless`.
This creates four possible combinations - two different test suites,
and two different test groups. This is a similar setup to that which
we have in CI - the Database and non-Database tests are run in
separate groups, and some jobs use the `extensions` suite while
others just use the default suite.
The `phpunit:parallel:...` commands will fail with a helpful message
if no `split_group_`s are found in the active PHPUnit configuration.
To help test whether the split test runs are really running all the
tests in the suite, we generate and store the PHPUnit results cache
file. Comparing the results cache files from linear versus parallel
runs should tell us if all the tests have been executed.
Bug: T365976
Change-Id: If106802f08edd5d4c841bb7970c69b88ab3bb39b
