1e2851d8d1
In T361190 and Quibble 1.9.0, we introduced parallel execution of PHPUnit tests to speed up the CI jobs. The existing implementation is purely Python/Quibble, and cannot directly be used by developers locally. With this patch, we re-implement the test splitting logic already implemented in CI as a composer task so that the parallel tests can be run locally. There are a couple of different approaches to running PHPUnit tests in parallel. The different approaches have been discussed at length in T50217. Ideally, we would just install the `paratest` extension and use that to parallelise the execution. Unfortunately we have complex test suites (specifically Parser tests and the Scribunto test suite) that dynamically create tests as they run, which makes it hard for `paratest` to work out which tests will run. To overcome this limitation, we use the `phpunit --list-tests` function to create a list of test classes that would be included in the execution of the test suite, then scan the filesystem for classes named in the `tests-list.xml` output. The classes we find are then collected into smaller groups (`split_group_X`) which we can run in parallel in separate processes. We split into 7-8 groups here, as that experimentally leads to an even spread of the tests and consumes 100% of all cores on a 4-core processor. Because `ParserIntegrationTest.php` is a single test class that generates thousands of integration tests, we put that in its own bucket rather than allocating it round-robin to one of the split buckets. This again helps to keep the buckets roughly the same size. The current implementation only supports splitting the `extensions` test suite. We need to do some more development and testing to support splitting other suites. The new composer command `phpunit:prepare-parallel:extensions` will generate a `phpunit.xml` file with the same contents as `phpunit.xml.dist`, but with the split-group suites added. The result of running all of the split groups should be the same as the result of running the whole test suite. Bug: T365976 Change-Id: I2d841ab236c5367961603bb526319053551bec2e
61 lines
1.9 KiB
PHP
61 lines
1.9 KiB
PHP
<?php
|
|
|
|
declare( strict_types = 1 );
|
|
|
|
namespace MediaWiki\Composer\PhpUnitSplitter;
|
|
|
|
/**
|
|
* @license GPL-2.0-or-later
|
|
*/
|
|
class TestSuiteBuilder {
|
|
|
|
private static function sortByTimeDescending( TestDescriptor $a, TestDescriptor $b ): int {
|
|
if ( $a->getDuration() === $b->getDuration() ) {
|
|
return 0;
|
|
}
|
|
return ( $a->getDuration() > $b->getDuration() ? -1 : 1 );
|
|
}
|
|
|
|
private static function smallestGroup( array $suites ): int {
|
|
$min = 10000;
|
|
$minIndex = 0;
|
|
$groups = count( $suites );
|
|
for ( $i = 0; $i < $groups; $i++ ) {
|
|
if ( $suites[$i]["time"] < $min ) {
|
|
$min = $suites[$i]["time"];
|
|
$minIndex = $i;
|
|
}
|
|
}
|
|
return $minIndex;
|
|
}
|
|
|
|
public function buildSuites( array $testDescriptors, int $groups ): array {
|
|
$suites = array_fill( 0, $groups, [ "list" => [], "time" => 0 ] );
|
|
$roundRobin = 0;
|
|
usort( $testDescriptors, [ self::class, "sortByTimeDescending" ] );
|
|
foreach ( $testDescriptors as $testDescriptor ) {
|
|
if ( !$testDescriptor->getFilename() ) {
|
|
// We didn't resolve a matching file for this test, so we skip it
|
|
// from the suite here. This only happens for "known" missing test
|
|
// classes (see PhpUnitXmlManager::EXPECTED_MISSING_CLASSES) - in
|
|
// all other cases a missing test file will throw an exception during
|
|
// suite building.
|
|
continue;
|
|
}
|
|
if ( $testDescriptor->getDuration() === 0 ) {
|
|
// If no explicit timing information is available for a test, we just
|
|
// drop it round-robin into the next bucket.
|
|
$nextSuite = $roundRobin;
|
|
$roundRobin = ( $roundRobin + 1 ) % $groups;
|
|
} else {
|
|
// If we have information about the test duration, we try and balance
|
|
// out the tests suites by having an even amount of time spent on
|
|
// each suite.
|
|
$nextSuite = self::smallestGroup( $suites );
|
|
}
|
|
$suites[$nextSuite]["list"][] = $testDescriptor->getFilename();
|
|
$suites[$nextSuite]["time"] += $testDescriptor->getDuration();
|
|
}
|
|
return $suites;
|
|
}
|
|
}
|