In T361190 and Quibble 1.9.0, we introduced parallel execution of
PHPUnit tests to speed up the CI jobs. The existing implementation
is purely Python/Quibble, and cannot directly be used by developers
locally. With this patch, we re-implement the test splitting logic
already implemented in CI as a composer task so that the parallel
tests can be run locally.
There are a couple of different approaches to running PHPUnit tests
in parallel. The different approaches have been discussed at length
in T50217. Ideally, we would just install the `paratest` extension
and use that to parallelise the execution. Unfortunately we have
complex test suites (specifically Parser tests and the Scribunto
test suite) that dynamically create tests as they run, which makes
it hard for `paratest` to work out which tests will run.
To overcome this limitation, we use the `phpunit --list-tests`
function to create a list of test classes that would be included in
the execution of the test suite, then scan the filesystem for
classes named in the `tests-list.xml` output. The classes we find
are then collected into smaller groups (`split_group_X`) which we
can run in parallel in separate processes.
We split into 7-8 groups here, as that experimentally leads to an
even spread of the tests and consumes 100% of all cores on a 4-core
processor.
Because `ParserIntegrationTest.php` is a single test class that
generates thousands of integration tests, we put that in its own
bucket rather than allocating it round-robin to one of the split
buckets. This again helps to keep the buckets roughly the same size.
The current implementation only supports splitting the `extensions`
test suite. We need to do some more development and testing to
support splitting other suites.
The new composer command `phpunit:prepare-parallel:extensions` will
generate a `phpunit.xml` file with the same contents as
`phpunit.xml.dist`, but with the split-group suites added. The
result of running all of the split groups should be the same as the
result of running the whole test suite.
Bug: T365976
Change-Id: I2d841ab236c5367961603bb526319053551bec2e
1e2851d8d1