2011-05-31 20:30:12 +00:00
|
|
|
<?php
|
2016-10-12 05:36:03 +00:00
|
|
|
use Wikimedia\ScopedCallback;
|
2011-05-31 20:30:12 +00:00
|
|
|
|
|
|
|
|
/**
|
2011-11-12 21:32:39 +00:00
|
|
|
* The UnitTest must be either a class that inherits from MediaWikiTestCase
|
2013-02-15 10:24:31 +00:00
|
|
|
* or a class that provides a public static suite() method which returns
|
2011-05-31 20:30:12 +00:00
|
|
|
* an PHPUnit_Framework_Test object
|
2013-02-15 10:24:31 +00:00
|
|
|
*
|
2011-05-31 20:30:12 +00:00
|
|
|
* @group Parser
|
2015-11-26 20:25:03 +00:00
|
|
|
* @group ParserTests
|
2011-05-31 20:30:12 +00:00
|
|
|
* @group Database
|
|
|
|
|
*/
|
Refactor parser tests
Merge the PHPUnit parser test runner with the old parserTests.inc,
taking the good bits of both. Reviewed, pared down and documented the
setup code. parserTests.php is now a frontend to a fully featured
parser test system, with lots of developer options, whereas PHPUnit
provides a simpler interface with increased isolation between test
cases.
Performance of both frontends is much improved, perhaps 2x faster for
parserTests.php and 10x faster for PHPUnit.
General:
* Split out the pre-Setup.php global variable configuration from
phpunit.php into a new class called TestSetup, also called it from
parserTests.php.
* Factored out the setup of TestsAutoLoader into a static method in
Maintenance.
* In Setup.php improved "caches" debug output.
PHPUnit frontend:
* Delete the entire contents of NewParserTest and replace it with a
small wrapper around ParserTestRunner. It doesn't inherit from
MediaWikiTestCase anymore since integrating the setup code was an
unnecessary complication.
* Rename MediaWikiParserTest to ParserTestTopLevelSuite and made it an
instantiable TestSuite class instead of just a static method. Got rid
of the eval(), just construct TestCase objects directly with a
specified name, it works just as well.
* Introduce ParserTestFileSuite for per-file setup.
* Remove parser-related options from phpunit.php, since we don't
support them anymore. Note that --filter now works just as well as
--regex used to.
* Add CoreParserTestSuite, equivalent to ExtensionsParserTestSuite,
for clarity.
* Make it possible to call MediaWikiTestCase::setupTestDB() more than
once, as is implied by the documentation.
parserTests.php frontend:
* Made parserTests.php into a Maintenance subclass, moved CLI-specific
code to it.
* Renamed ParserTest to ParserTestRunner, this is now the generic
backend.
* Add --upload-dir option which sets up an FSFileBackend, similar
to the old default behaviour
Test file reading and interpretation:
* Rename TestFileIterator to TestFileReader, and make it read and buffer
an entire file, instead of iterating.
* The previous code had an associative array representation of test
specifications. Used this form more widely to pass around test data.
* Remove the idea of !!hooks copying hooks from $wgParser, this is
unnecessary now that all extensions use ParserFirstCallInit. Resurrect
an old interpretation of the feature which was accidentally broken: if
a named hook does not exist, skip all tests in the file.
* Got rid of the "subtest" idea for tidy variants, instead use a
human-readable description that appears in the output.
* When all tests in a file are filtered or skipped, don't create the
articles in them. This greatly speeds up execution time when --regex
matches a small number of tests. It may possibly break extensions, but
they would have been randomly broken anyway since there is no
guarantee of test file execution order.
* Remove integrated testing of OutputPage::addCategoryLinks() category
link formatting, life is complicated enough already. It can go in
OutputPageTest if that's a thing we really need.
Result recording and display:
* Make TestRecorder into a generic plugin interface for progress output
etc., which needs to be abstracted for PHPUnit integration.
* Introduce MultiTestRecorder for recorder chaining, instead of using
a long inheritance chain. All test recorders now directly inherit from
TestRecorder.
* Move all console-related code to the new ParserTestPrinter.
* Introduce PhpunitTestRecorder, which is the recorder for the PHPUnit
frontend. Most events are ignored since they are never emitted in the
PHPUnit frontend, which does not call runTests().
* Put more information into ParserTestResult and use it more often.
Setup and teardown:
* Introduce a new API for setup/teardown where setup functions return a
ScopedCallback object which automatically performs the corresponding
teardown when it goes out of scope.
* Rename setUp() to staticSetup(), rewrite. There was a lot of cruft in
here which was simply copied from Setup.php without review, and had
nothing to do with parser tests.
* Rename setupGlobals() to perTestSetup(), mostly rewrite. For
performance, give staticSetup() precedence in cases where they were
both setting up the same thing.
* In support of merged setup code, allow Hooks::clear() to be called
from parserTests.php.
* Remove wgFileExtensions -- it is only used by UploadBase which we
don't call.
* Remove wgUseImageResize -- superseded by MockMediaHandlerFactory which
I imported from NewParserTest.
* Import MockFileBackend from NewParserTest. But instead of
customising the configuration globals, I injected services.
* Remove thumbnail deletion from upload teardown. This makes glob
handling as in the old parserTests.php unnecessary.
* Remove math file from upload teardown, math is actually an extension
now! Also, the relevant parser tests were removed from the Math
extension two years ago in favour of unit tests.
* Make addArticle() private, and introduce addArticles() instead, which
allows setup/teardown to be done once for each batch of articles
instead of every time.
* Remove $wgNamespaceAliases and $wgNamespaceProtection setup. These were
copied in from Setup.php in 2010, and are redundant since we do
actually run Setup.php.
* Use NullLockManager, don't set up a temporary directory just for
this alone.
Fuzz tests:
* Use the new TestSetup class.
* Updated for ParserTestRunner interface change.
* Remove some obsolete references to fuzz tests from the two frontends
where they used to reside.
Bug: T41473
Change-Id: Ia8e17008cb9d9b62ce5645e15a41a3b402f4026a
2016-09-08 01:25:22 +00:00
|
|
|
class ParserTestTopLevelSuite extends PHPUnit_Framework_TestSuite {
|
|
|
|
|
/** @var ParserTestRunner */
|
|
|
|
|
private $ptRunner;
|
|
|
|
|
|
|
|
|
|
/** @var ScopedCallback */
|
|
|
|
|
private $ptTeardownScope;
|
2011-05-31 20:30:12 +00:00
|
|
|
|
2013-05-10 19:20:12 +00:00
|
|
|
/**
|
|
|
|
|
* @defgroup filtering_constants Filtering constants
|
|
|
|
|
*
|
|
|
|
|
* Limit inclusion of parser tests files coming from MediaWiki core
|
|
|
|
|
* @{
|
|
|
|
|
*/
|
2011-05-31 20:30:12 +00:00
|
|
|
|
2013-05-10 19:20:12 +00:00
|
|
|
/** Include files shipped with MediaWiki core */
|
|
|
|
|
const CORE_ONLY = 1;
|
|
|
|
|
/** Include non core files as set in $wgParserTestFiles */
|
|
|
|
|
const NO_CORE = 2;
|
|
|
|
|
/** Include anything set via $wgParserTestFiles */
|
2017-10-06 19:10:53 +00:00
|
|
|
const WITH_ALL = self::CORE_ONLY | self::NO_CORE;
|
2013-05-10 19:20:12 +00:00
|
|
|
|
|
|
|
|
/** @} */
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Get a PHPUnit test suite of parser tests. Optionally filtered with
|
|
|
|
|
* $flags.
|
|
|
|
|
*
|
|
|
|
|
* @par Examples:
|
|
|
|
|
* Get a suite of parser tests shipped by MediaWiki core:
|
|
|
|
|
* @code
|
2016-09-08 01:07:06 +00:00
|
|
|
* ParserTestTopLevelSuite::suite( ParserTestTopLevelSuite::CORE_ONLY );
|
2013-05-10 19:20:12 +00:00
|
|
|
* @endcode
|
|
|
|
|
* Get a suite of various parser tests, like extensions:
|
|
|
|
|
* @code
|
2016-09-08 01:07:06 +00:00
|
|
|
* ParserTestTopLevelSuite::suite( ParserTestTopLevelSuite::NO_CORE );
|
2013-05-10 19:20:12 +00:00
|
|
|
* @endcode
|
|
|
|
|
* Get any test defined via $wgParserTestFiles:
|
|
|
|
|
* @code
|
2016-09-08 01:07:06 +00:00
|
|
|
* ParserTestTopLevelSuite::suite( ParserTestTopLevelSuite::WITH_ALL );
|
2013-05-10 19:20:12 +00:00
|
|
|
* @endcode
|
|
|
|
|
*
|
2014-07-24 12:55:43 +00:00
|
|
|
* @param int $flags Bitwise flag to filter out the $wgParserTestFiles that
|
2016-09-08 01:07:06 +00:00
|
|
|
* will be included. Default: ParserTestTopLevelSuite::CORE_ONLY
|
2013-05-10 19:20:12 +00:00
|
|
|
*
|
|
|
|
|
* @return PHPUnit_Framework_TestSuite
|
|
|
|
|
*/
|
|
|
|
|
public static function suite( $flags = self::CORE_ONLY ) {
|
Refactor parser tests
Merge the PHPUnit parser test runner with the old parserTests.inc,
taking the good bits of both. Reviewed, pared down and documented the
setup code. parserTests.php is now a frontend to a fully featured
parser test system, with lots of developer options, whereas PHPUnit
provides a simpler interface with increased isolation between test
cases.
Performance of both frontends is much improved, perhaps 2x faster for
parserTests.php and 10x faster for PHPUnit.
General:
* Split out the pre-Setup.php global variable configuration from
phpunit.php into a new class called TestSetup, also called it from
parserTests.php.
* Factored out the setup of TestsAutoLoader into a static method in
Maintenance.
* In Setup.php improved "caches" debug output.
PHPUnit frontend:
* Delete the entire contents of NewParserTest and replace it with a
small wrapper around ParserTestRunner. It doesn't inherit from
MediaWikiTestCase anymore since integrating the setup code was an
unnecessary complication.
* Rename MediaWikiParserTest to ParserTestTopLevelSuite and made it an
instantiable TestSuite class instead of just a static method. Got rid
of the eval(), just construct TestCase objects directly with a
specified name, it works just as well.
* Introduce ParserTestFileSuite for per-file setup.
* Remove parser-related options from phpunit.php, since we don't
support them anymore. Note that --filter now works just as well as
--regex used to.
* Add CoreParserTestSuite, equivalent to ExtensionsParserTestSuite,
for clarity.
* Make it possible to call MediaWikiTestCase::setupTestDB() more than
once, as is implied by the documentation.
parserTests.php frontend:
* Made parserTests.php into a Maintenance subclass, moved CLI-specific
code to it.
* Renamed ParserTest to ParserTestRunner, this is now the generic
backend.
* Add --upload-dir option which sets up an FSFileBackend, similar
to the old default behaviour
Test file reading and interpretation:
* Rename TestFileIterator to TestFileReader, and make it read and buffer
an entire file, instead of iterating.
* The previous code had an associative array representation of test
specifications. Used this form more widely to pass around test data.
* Remove the idea of !!hooks copying hooks from $wgParser, this is
unnecessary now that all extensions use ParserFirstCallInit. Resurrect
an old interpretation of the feature which was accidentally broken: if
a named hook does not exist, skip all tests in the file.
* Got rid of the "subtest" idea for tidy variants, instead use a
human-readable description that appears in the output.
* When all tests in a file are filtered or skipped, don't create the
articles in them. This greatly speeds up execution time when --regex
matches a small number of tests. It may possibly break extensions, but
they would have been randomly broken anyway since there is no
guarantee of test file execution order.
* Remove integrated testing of OutputPage::addCategoryLinks() category
link formatting, life is complicated enough already. It can go in
OutputPageTest if that's a thing we really need.
Result recording and display:
* Make TestRecorder into a generic plugin interface for progress output
etc., which needs to be abstracted for PHPUnit integration.
* Introduce MultiTestRecorder for recorder chaining, instead of using
a long inheritance chain. All test recorders now directly inherit from
TestRecorder.
* Move all console-related code to the new ParserTestPrinter.
* Introduce PhpunitTestRecorder, which is the recorder for the PHPUnit
frontend. Most events are ignored since they are never emitted in the
PHPUnit frontend, which does not call runTests().
* Put more information into ParserTestResult and use it more often.
Setup and teardown:
* Introduce a new API for setup/teardown where setup functions return a
ScopedCallback object which automatically performs the corresponding
teardown when it goes out of scope.
* Rename setUp() to staticSetup(), rewrite. There was a lot of cruft in
here which was simply copied from Setup.php without review, and had
nothing to do with parser tests.
* Rename setupGlobals() to perTestSetup(), mostly rewrite. For
performance, give staticSetup() precedence in cases where they were
both setting up the same thing.
* In support of merged setup code, allow Hooks::clear() to be called
from parserTests.php.
* Remove wgFileExtensions -- it is only used by UploadBase which we
don't call.
* Remove wgUseImageResize -- superseded by MockMediaHandlerFactory which
I imported from NewParserTest.
* Import MockFileBackend from NewParserTest. But instead of
customising the configuration globals, I injected services.
* Remove thumbnail deletion from upload teardown. This makes glob
handling as in the old parserTests.php unnecessary.
* Remove math file from upload teardown, math is actually an extension
now! Also, the relevant parser tests were removed from the Math
extension two years ago in favour of unit tests.
* Make addArticle() private, and introduce addArticles() instead, which
allows setup/teardown to be done once for each batch of articles
instead of every time.
* Remove $wgNamespaceAliases and $wgNamespaceProtection setup. These were
copied in from Setup.php in 2010, and are redundant since we do
actually run Setup.php.
* Use NullLockManager, don't set up a temporary directory just for
this alone.
Fuzz tests:
* Use the new TestSetup class.
* Updated for ParserTestRunner interface change.
* Remove some obsolete references to fuzz tests from the two frontends
where they used to reside.
Bug: T41473
Change-Id: Ia8e17008cb9d9b62ce5645e15a41a3b402f4026a
2016-09-08 01:25:22 +00:00
|
|
|
return new self( $flags );
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
function __construct( $flags ) {
|
|
|
|
|
parent::__construct();
|
|
|
|
|
|
|
|
|
|
$this->ptRecorder = new PhpunitTestRecorder;
|
|
|
|
|
$this->ptRunner = new ParserTestRunner( $this->ptRecorder );
|
|
|
|
|
|
2013-08-24 15:06:25 +00:00
|
|
|
if ( is_string( $flags ) ) {
|
2013-05-10 19:20:12 +00:00
|
|
|
$flags = self::CORE_ONLY;
|
|
|
|
|
}
|
2017-07-07 22:26:21 +00:00
|
|
|
global $IP;
|
2013-05-10 19:20:12 +00:00
|
|
|
|
2013-08-24 15:06:25 +00:00
|
|
|
$mwTestDir = $IP . '/tests/';
|
2013-05-10 19:20:12 +00:00
|
|
|
|
|
|
|
|
# Human friendly helpers
|
2013-08-24 15:06:25 +00:00
|
|
|
$wantsCore = ( $flags & self::CORE_ONLY );
|
|
|
|
|
$wantsRest = ( $flags & self::NO_CORE );
|
2013-05-10 19:20:12 +00:00
|
|
|
|
|
|
|
|
# Will hold the .txt parser test files we will include
|
2016-02-17 09:09:32 +00:00
|
|
|
$filesToTest = [];
|
2013-05-10 19:20:12 +00:00
|
|
|
|
|
|
|
|
# Filter out .txt files
|
2017-07-07 22:26:21 +00:00
|
|
|
$files = ParserTestRunner::getParserTestFiles();
|
2017-07-08 07:14:13 +00:00
|
|
|
foreach ( $files as $extName => $parserTestFile ) {
|
2013-05-10 19:20:12 +00:00
|
|
|
$isCore = ( 0 === strpos( $parserTestFile, $mwTestDir ) );
|
2011-05-31 20:30:12 +00:00
|
|
|
|
2013-08-24 15:06:25 +00:00
|
|
|
if ( $isCore && $wantsCore ) {
|
2013-05-10 19:20:12 +00:00
|
|
|
self::debug( "included core parser tests: $parserTestFile" );
|
2017-07-08 07:14:13 +00:00
|
|
|
$filesToTest[$extName] = $parserTestFile;
|
2013-08-24 15:06:25 +00:00
|
|
|
} elseif ( !$isCore && $wantsRest ) {
|
2013-05-10 19:20:12 +00:00
|
|
|
self::debug( "included non core parser tests: $parserTestFile" );
|
2017-07-08 07:14:13 +00:00
|
|
|
$filesToTest[$extName] = $parserTestFile;
|
2013-05-10 19:20:12 +00:00
|
|
|
} else {
|
|
|
|
|
self::debug( "skipped parser tests: $parserTestFile" );
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
self::debug( 'parser tests files: '
|
2013-08-24 15:06:25 +00:00
|
|
|
. implode( ' ', $filesToTest ) );
|
2013-05-10 19:20:12 +00:00
|
|
|
|
2016-02-17 09:09:32 +00:00
|
|
|
$testList = [];
|
2013-06-05 16:14:27 +00:00
|
|
|
$counter = 0;
|
2017-07-08 07:14:13 +00:00
|
|
|
foreach ( $filesToTest as $extensionName => $fileName ) {
|
|
|
|
|
if ( is_int( $extensionName ) ) {
|
|
|
|
|
// If there's no extension name because this is coming
|
|
|
|
|
// from the legacy global, then assume the next level directory
|
|
|
|
|
// is the extension name (e.g. extensions/FooBar/parserTests.txt).
|
|
|
|
|
$extensionName = basename( dirname( $fileName ) );
|
|
|
|
|
}
|
2015-09-07 05:30:13 +00:00
|
|
|
$testsName = $extensionName . '__' . basename( $fileName, '.txt' );
|
2013-06-05 16:14:27 +00:00
|
|
|
$parserTestClassName = ucfirst( $testsName );
|
2015-09-30 06:08:31 +00:00
|
|
|
|
2016-10-13 05:34:26 +00:00
|
|
|
// Official spec for class names: https://secure.php.net/manual/en/language.oop5.basic.php
|
2013-06-05 16:14:27 +00:00
|
|
|
// Prepend 'ParserTest_' to be paranoid about it not starting with a number
|
2015-09-30 06:08:31 +00:00
|
|
|
$parserTestClassName = 'ParserTest_' .
|
|
|
|
|
preg_replace( '/[^a-zA-Z0-9_\x7f-\xff]/', '_', $parserTestClassName );
|
|
|
|
|
|
2013-06-05 16:14:27 +00:00
|
|
|
if ( isset( $testList[$parserTestClassName] ) ) {
|
Refactor parser tests
Merge the PHPUnit parser test runner with the old parserTests.inc,
taking the good bits of both. Reviewed, pared down and documented the
setup code. parserTests.php is now a frontend to a fully featured
parser test system, with lots of developer options, whereas PHPUnit
provides a simpler interface with increased isolation between test
cases.
Performance of both frontends is much improved, perhaps 2x faster for
parserTests.php and 10x faster for PHPUnit.
General:
* Split out the pre-Setup.php global variable configuration from
phpunit.php into a new class called TestSetup, also called it from
parserTests.php.
* Factored out the setup of TestsAutoLoader into a static method in
Maintenance.
* In Setup.php improved "caches" debug output.
PHPUnit frontend:
* Delete the entire contents of NewParserTest and replace it with a
small wrapper around ParserTestRunner. It doesn't inherit from
MediaWikiTestCase anymore since integrating the setup code was an
unnecessary complication.
* Rename MediaWikiParserTest to ParserTestTopLevelSuite and made it an
instantiable TestSuite class instead of just a static method. Got rid
of the eval(), just construct TestCase objects directly with a
specified name, it works just as well.
* Introduce ParserTestFileSuite for per-file setup.
* Remove parser-related options from phpunit.php, since we don't
support them anymore. Note that --filter now works just as well as
--regex used to.
* Add CoreParserTestSuite, equivalent to ExtensionsParserTestSuite,
for clarity.
* Make it possible to call MediaWikiTestCase::setupTestDB() more than
once, as is implied by the documentation.
parserTests.php frontend:
* Made parserTests.php into a Maintenance subclass, moved CLI-specific
code to it.
* Renamed ParserTest to ParserTestRunner, this is now the generic
backend.
* Add --upload-dir option which sets up an FSFileBackend, similar
to the old default behaviour
Test file reading and interpretation:
* Rename TestFileIterator to TestFileReader, and make it read and buffer
an entire file, instead of iterating.
* The previous code had an associative array representation of test
specifications. Used this form more widely to pass around test data.
* Remove the idea of !!hooks copying hooks from $wgParser, this is
unnecessary now that all extensions use ParserFirstCallInit. Resurrect
an old interpretation of the feature which was accidentally broken: if
a named hook does not exist, skip all tests in the file.
* Got rid of the "subtest" idea for tidy variants, instead use a
human-readable description that appears in the output.
* When all tests in a file are filtered or skipped, don't create the
articles in them. This greatly speeds up execution time when --regex
matches a small number of tests. It may possibly break extensions, but
they would have been randomly broken anyway since there is no
guarantee of test file execution order.
* Remove integrated testing of OutputPage::addCategoryLinks() category
link formatting, life is complicated enough already. It can go in
OutputPageTest if that's a thing we really need.
Result recording and display:
* Make TestRecorder into a generic plugin interface for progress output
etc., which needs to be abstracted for PHPUnit integration.
* Introduce MultiTestRecorder for recorder chaining, instead of using
a long inheritance chain. All test recorders now directly inherit from
TestRecorder.
* Move all console-related code to the new ParserTestPrinter.
* Introduce PhpunitTestRecorder, which is the recorder for the PHPUnit
frontend. Most events are ignored since they are never emitted in the
PHPUnit frontend, which does not call runTests().
* Put more information into ParserTestResult and use it more often.
Setup and teardown:
* Introduce a new API for setup/teardown where setup functions return a
ScopedCallback object which automatically performs the corresponding
teardown when it goes out of scope.
* Rename setUp() to staticSetup(), rewrite. There was a lot of cruft in
here which was simply copied from Setup.php without review, and had
nothing to do with parser tests.
* Rename setupGlobals() to perTestSetup(), mostly rewrite. For
performance, give staticSetup() precedence in cases where they were
both setting up the same thing.
* In support of merged setup code, allow Hooks::clear() to be called
from parserTests.php.
* Remove wgFileExtensions -- it is only used by UploadBase which we
don't call.
* Remove wgUseImageResize -- superseded by MockMediaHandlerFactory which
I imported from NewParserTest.
* Import MockFileBackend from NewParserTest. But instead of
customising the configuration globals, I injected services.
* Remove thumbnail deletion from upload teardown. This makes glob
handling as in the old parserTests.php unnecessary.
* Remove math file from upload teardown, math is actually an extension
now! Also, the relevant parser tests were removed from the Math
extension two years ago in favour of unit tests.
* Make addArticle() private, and introduce addArticles() instead, which
allows setup/teardown to be done once for each batch of articles
instead of every time.
* Remove $wgNamespaceAliases and $wgNamespaceProtection setup. These were
copied in from Setup.php in 2010, and are redundant since we do
actually run Setup.php.
* Use NullLockManager, don't set up a temporary directory just for
this alone.
Fuzz tests:
* Use the new TestSetup class.
* Updated for ParserTestRunner interface change.
* Remove some obsolete references to fuzz tests from the two frontends
where they used to reside.
Bug: T41473
Change-Id: Ia8e17008cb9d9b62ce5645e15a41a3b402f4026a
2016-09-08 01:25:22 +00:00
|
|
|
// If there is a conflict, append a number.
|
2013-06-05 16:14:27 +00:00
|
|
|
$counter++;
|
|
|
|
|
$parserTestClassName .= $counter;
|
|
|
|
|
}
|
|
|
|
|
$testList[$parserTestClassName] = true;
|
2012-12-07 01:11:38 +00:00
|
|
|
|
Refactor parser tests
Merge the PHPUnit parser test runner with the old parserTests.inc,
taking the good bits of both. Reviewed, pared down and documented the
setup code. parserTests.php is now a frontend to a fully featured
parser test system, with lots of developer options, whereas PHPUnit
provides a simpler interface with increased isolation between test
cases.
Performance of both frontends is much improved, perhaps 2x faster for
parserTests.php and 10x faster for PHPUnit.
General:
* Split out the pre-Setup.php global variable configuration from
phpunit.php into a new class called TestSetup, also called it from
parserTests.php.
* Factored out the setup of TestsAutoLoader into a static method in
Maintenance.
* In Setup.php improved "caches" debug output.
PHPUnit frontend:
* Delete the entire contents of NewParserTest and replace it with a
small wrapper around ParserTestRunner. It doesn't inherit from
MediaWikiTestCase anymore since integrating the setup code was an
unnecessary complication.
* Rename MediaWikiParserTest to ParserTestTopLevelSuite and made it an
instantiable TestSuite class instead of just a static method. Got rid
of the eval(), just construct TestCase objects directly with a
specified name, it works just as well.
* Introduce ParserTestFileSuite for per-file setup.
* Remove parser-related options from phpunit.php, since we don't
support them anymore. Note that --filter now works just as well as
--regex used to.
* Add CoreParserTestSuite, equivalent to ExtensionsParserTestSuite,
for clarity.
* Make it possible to call MediaWikiTestCase::setupTestDB() more than
once, as is implied by the documentation.
parserTests.php frontend:
* Made parserTests.php into a Maintenance subclass, moved CLI-specific
code to it.
* Renamed ParserTest to ParserTestRunner, this is now the generic
backend.
* Add --upload-dir option which sets up an FSFileBackend, similar
to the old default behaviour
Test file reading and interpretation:
* Rename TestFileIterator to TestFileReader, and make it read and buffer
an entire file, instead of iterating.
* The previous code had an associative array representation of test
specifications. Used this form more widely to pass around test data.
* Remove the idea of !!hooks copying hooks from $wgParser, this is
unnecessary now that all extensions use ParserFirstCallInit. Resurrect
an old interpretation of the feature which was accidentally broken: if
a named hook does not exist, skip all tests in the file.
* Got rid of the "subtest" idea for tidy variants, instead use a
human-readable description that appears in the output.
* When all tests in a file are filtered or skipped, don't create the
articles in them. This greatly speeds up execution time when --regex
matches a small number of tests. It may possibly break extensions, but
they would have been randomly broken anyway since there is no
guarantee of test file execution order.
* Remove integrated testing of OutputPage::addCategoryLinks() category
link formatting, life is complicated enough already. It can go in
OutputPageTest if that's a thing we really need.
Result recording and display:
* Make TestRecorder into a generic plugin interface for progress output
etc., which needs to be abstracted for PHPUnit integration.
* Introduce MultiTestRecorder for recorder chaining, instead of using
a long inheritance chain. All test recorders now directly inherit from
TestRecorder.
* Move all console-related code to the new ParserTestPrinter.
* Introduce PhpunitTestRecorder, which is the recorder for the PHPUnit
frontend. Most events are ignored since they are never emitted in the
PHPUnit frontend, which does not call runTests().
* Put more information into ParserTestResult and use it more often.
Setup and teardown:
* Introduce a new API for setup/teardown where setup functions return a
ScopedCallback object which automatically performs the corresponding
teardown when it goes out of scope.
* Rename setUp() to staticSetup(), rewrite. There was a lot of cruft in
here which was simply copied from Setup.php without review, and had
nothing to do with parser tests.
* Rename setupGlobals() to perTestSetup(), mostly rewrite. For
performance, give staticSetup() precedence in cases where they were
both setting up the same thing.
* In support of merged setup code, allow Hooks::clear() to be called
from parserTests.php.
* Remove wgFileExtensions -- it is only used by UploadBase which we
don't call.
* Remove wgUseImageResize -- superseded by MockMediaHandlerFactory which
I imported from NewParserTest.
* Import MockFileBackend from NewParserTest. But instead of
customising the configuration globals, I injected services.
* Remove thumbnail deletion from upload teardown. This makes glob
handling as in the old parserTests.php unnecessary.
* Remove math file from upload teardown, math is actually an extension
now! Also, the relevant parser tests were removed from the Math
extension two years ago in favour of unit tests.
* Make addArticle() private, and introduce addArticles() instead, which
allows setup/teardown to be done once for each batch of articles
instead of every time.
* Remove $wgNamespaceAliases and $wgNamespaceProtection setup. These were
copied in from Setup.php in 2010, and are redundant since we do
actually run Setup.php.
* Use NullLockManager, don't set up a temporary directory just for
this alone.
Fuzz tests:
* Use the new TestSetup class.
* Updated for ParserTestRunner interface change.
* Remove some obsolete references to fuzz tests from the two frontends
where they used to reside.
Bug: T41473
Change-Id: Ia8e17008cb9d9b62ce5645e15a41a3b402f4026a
2016-09-08 01:25:22 +00:00
|
|
|
// Previously we actually created a class here, with eval(). We now
|
|
|
|
|
// just override the name.
|
|
|
|
|
|
2013-05-10 19:20:12 +00:00
|
|
|
self::debug( "Adding test class $parserTestClassName" );
|
Refactor parser tests
Merge the PHPUnit parser test runner with the old parserTests.inc,
taking the good bits of both. Reviewed, pared down and documented the
setup code. parserTests.php is now a frontend to a fully featured
parser test system, with lots of developer options, whereas PHPUnit
provides a simpler interface with increased isolation between test
cases.
Performance of both frontends is much improved, perhaps 2x faster for
parserTests.php and 10x faster for PHPUnit.
General:
* Split out the pre-Setup.php global variable configuration from
phpunit.php into a new class called TestSetup, also called it from
parserTests.php.
* Factored out the setup of TestsAutoLoader into a static method in
Maintenance.
* In Setup.php improved "caches" debug output.
PHPUnit frontend:
* Delete the entire contents of NewParserTest and replace it with a
small wrapper around ParserTestRunner. It doesn't inherit from
MediaWikiTestCase anymore since integrating the setup code was an
unnecessary complication.
* Rename MediaWikiParserTest to ParserTestTopLevelSuite and made it an
instantiable TestSuite class instead of just a static method. Got rid
of the eval(), just construct TestCase objects directly with a
specified name, it works just as well.
* Introduce ParserTestFileSuite for per-file setup.
* Remove parser-related options from phpunit.php, since we don't
support them anymore. Note that --filter now works just as well as
--regex used to.
* Add CoreParserTestSuite, equivalent to ExtensionsParserTestSuite,
for clarity.
* Make it possible to call MediaWikiTestCase::setupTestDB() more than
once, as is implied by the documentation.
parserTests.php frontend:
* Made parserTests.php into a Maintenance subclass, moved CLI-specific
code to it.
* Renamed ParserTest to ParserTestRunner, this is now the generic
backend.
* Add --upload-dir option which sets up an FSFileBackend, similar
to the old default behaviour
Test file reading and interpretation:
* Rename TestFileIterator to TestFileReader, and make it read and buffer
an entire file, instead of iterating.
* The previous code had an associative array representation of test
specifications. Used this form more widely to pass around test data.
* Remove the idea of !!hooks copying hooks from $wgParser, this is
unnecessary now that all extensions use ParserFirstCallInit. Resurrect
an old interpretation of the feature which was accidentally broken: if
a named hook does not exist, skip all tests in the file.
* Got rid of the "subtest" idea for tidy variants, instead use a
human-readable description that appears in the output.
* When all tests in a file are filtered or skipped, don't create the
articles in them. This greatly speeds up execution time when --regex
matches a small number of tests. It may possibly break extensions, but
they would have been randomly broken anyway since there is no
guarantee of test file execution order.
* Remove integrated testing of OutputPage::addCategoryLinks() category
link formatting, life is complicated enough already. It can go in
OutputPageTest if that's a thing we really need.
Result recording and display:
* Make TestRecorder into a generic plugin interface for progress output
etc., which needs to be abstracted for PHPUnit integration.
* Introduce MultiTestRecorder for recorder chaining, instead of using
a long inheritance chain. All test recorders now directly inherit from
TestRecorder.
* Move all console-related code to the new ParserTestPrinter.
* Introduce PhpunitTestRecorder, which is the recorder for the PHPUnit
frontend. Most events are ignored since they are never emitted in the
PHPUnit frontend, which does not call runTests().
* Put more information into ParserTestResult and use it more often.
Setup and teardown:
* Introduce a new API for setup/teardown where setup functions return a
ScopedCallback object which automatically performs the corresponding
teardown when it goes out of scope.
* Rename setUp() to staticSetup(), rewrite. There was a lot of cruft in
here which was simply copied from Setup.php without review, and had
nothing to do with parser tests.
* Rename setupGlobals() to perTestSetup(), mostly rewrite. For
performance, give staticSetup() precedence in cases where they were
both setting up the same thing.
* In support of merged setup code, allow Hooks::clear() to be called
from parserTests.php.
* Remove wgFileExtensions -- it is only used by UploadBase which we
don't call.
* Remove wgUseImageResize -- superseded by MockMediaHandlerFactory which
I imported from NewParserTest.
* Import MockFileBackend from NewParserTest. But instead of
customising the configuration globals, I injected services.
* Remove thumbnail deletion from upload teardown. This makes glob
handling as in the old parserTests.php unnecessary.
* Remove math file from upload teardown, math is actually an extension
now! Also, the relevant parser tests were removed from the Math
extension two years ago in favour of unit tests.
* Make addArticle() private, and introduce addArticles() instead, which
allows setup/teardown to be done once for each batch of articles
instead of every time.
* Remove $wgNamespaceAliases and $wgNamespaceProtection setup. These were
copied in from Setup.php in 2010, and are redundant since we do
actually run Setup.php.
* Use NullLockManager, don't set up a temporary directory just for
this alone.
Fuzz tests:
* Use the new TestSetup class.
* Updated for ParserTestRunner interface change.
* Remove some obsolete references to fuzz tests from the two frontends
where they used to reside.
Bug: T41473
Change-Id: Ia8e17008cb9d9b62ce5645e15a41a3b402f4026a
2016-09-08 01:25:22 +00:00
|
|
|
$this->addTest( new ParserTestFileSuite(
|
|
|
|
|
$this->ptRunner, $parserTestClassName, $fileName ) );
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
public function setUp() {
|
|
|
|
|
wfDebug( __METHOD__ );
|
|
|
|
|
$db = wfGetDB( DB_MASTER );
|
|
|
|
|
$type = $db->getType();
|
|
|
|
|
$prefix = $type === 'oracle' ?
|
|
|
|
|
MediaWikiTestCase::ORA_DB_PREFIX : MediaWikiTestCase::DB_PREFIX;
|
|
|
|
|
MediaWikiTestCase::setupTestDB( $db, $prefix );
|
|
|
|
|
$teardown = $this->ptRunner->setDatabase( $db );
|
|
|
|
|
$teardown = $this->ptRunner->setupUploads( $teardown );
|
|
|
|
|
$this->ptTeardownScope = $teardown;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
public function tearDown() {
|
|
|
|
|
wfDebug( __METHOD__ );
|
|
|
|
|
if ( $this->ptTeardownScope ) {
|
|
|
|
|
ScopedCallback::consume( $this->ptTeardownScope );
|
2011-05-31 20:30:12 +00:00
|
|
|
}
|
|
|
|
|
}
|
2013-05-10 19:20:12 +00:00
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Write $msg under log group 'tests-parser'
|
|
|
|
|
* @param string $msg Message to log
|
|
|
|
|
*/
|
|
|
|
|
protected static function debug( $msg ) {
|
2017-09-09 20:50:23 +00:00
|
|
|
wfDebugLog( 'tests-parser', wfGetCaller() . ' ' . $msg );
|
2013-05-10 19:20:12 +00:00
|
|
|
}
|
2011-05-31 20:30:12 +00:00
|
|
|
}
|