Indexing these files takes about 5-10% of the run time when generating
code coverage reports for core. It has no coverage at the moment and
probably won't any time soon given it's just variable declarations.
At least for now, exclude it to significantly speed up generating of
coverage reports.
Change-Id: I0357e004e96d4f4b19036474c8353ddb778e31f3
As of a few releases ago, there are no longer any skins in core
in that directory (base classes and fallback are under includes/).
Removing this makes running phpunit coverage faster locally,
when there are skins installed. This should be excluded, similar
to how extensions/ is not indexed for the core report, either.
Change-Id: Idc01578bf22febc684dbcb7aeb9ec9475e930ae4
Skins, like extensions, can register PHPUnit tests.
The testsuite interface in PHPUnit is designed to execute one run only,
so this should be based on the semantic entry of "skin jobs".
Changing the Jenkins job to run both doesn't work (only one value accepted).
Working around by running phpunit twice also seems wrong.
Bug: T170880
Change-Id: Ic04f4a842b2912574826b01b00d49db93d51c99b
I was bored. What? Don't look at me that way.
I mostly targetted mixed tabs and spaces, but others were not spared.
Note that some of the whitespace changes are inside HTML output,
extended regexps or SQL snippets.
Change-Id: Ie206cc946459f6befcfc2d520e35ad3ea3c0f1e0
This also fixed the testLoggersAreRestoredOnTearDown
test which was broken, and factors it out into
3 seperate tests
Bug: T151081
Change-Id: I00d29b501fa84db22e3bcd3d5642c49b2e99d4a0
Merge the PHPUnit parser test runner with the old parserTests.inc,
taking the good bits of both. Reviewed, pared down and documented the
setup code. parserTests.php is now a frontend to a fully featured
parser test system, with lots of developer options, whereas PHPUnit
provides a simpler interface with increased isolation between test
cases.
Performance of both frontends is much improved, perhaps 2x faster for
parserTests.php and 10x faster for PHPUnit.
General:
* Split out the pre-Setup.php global variable configuration from
phpunit.php into a new class called TestSetup, also called it from
parserTests.php.
* Factored out the setup of TestsAutoLoader into a static method in
Maintenance.
* In Setup.php improved "caches" debug output.
PHPUnit frontend:
* Delete the entire contents of NewParserTest and replace it with a
small wrapper around ParserTestRunner. It doesn't inherit from
MediaWikiTestCase anymore since integrating the setup code was an
unnecessary complication.
* Rename MediaWikiParserTest to ParserTestTopLevelSuite and made it an
instantiable TestSuite class instead of just a static method. Got rid
of the eval(), just construct TestCase objects directly with a
specified name, it works just as well.
* Introduce ParserTestFileSuite for per-file setup.
* Remove parser-related options from phpunit.php, since we don't
support them anymore. Note that --filter now works just as well as
--regex used to.
* Add CoreParserTestSuite, equivalent to ExtensionsParserTestSuite,
for clarity.
* Make it possible to call MediaWikiTestCase::setupTestDB() more than
once, as is implied by the documentation.
parserTests.php frontend:
* Made parserTests.php into a Maintenance subclass, moved CLI-specific
code to it.
* Renamed ParserTest to ParserTestRunner, this is now the generic
backend.
* Add --upload-dir option which sets up an FSFileBackend, similar
to the old default behaviour
Test file reading and interpretation:
* Rename TestFileIterator to TestFileReader, and make it read and buffer
an entire file, instead of iterating.
* The previous code had an associative array representation of test
specifications. Used this form more widely to pass around test data.
* Remove the idea of !!hooks copying hooks from $wgParser, this is
unnecessary now that all extensions use ParserFirstCallInit. Resurrect
an old interpretation of the feature which was accidentally broken: if
a named hook does not exist, skip all tests in the file.
* Got rid of the "subtest" idea for tidy variants, instead use a
human-readable description that appears in the output.
* When all tests in a file are filtered or skipped, don't create the
articles in them. This greatly speeds up execution time when --regex
matches a small number of tests. It may possibly break extensions, but
they would have been randomly broken anyway since there is no
guarantee of test file execution order.
* Remove integrated testing of OutputPage::addCategoryLinks() category
link formatting, life is complicated enough already. It can go in
OutputPageTest if that's a thing we really need.
Result recording and display:
* Make TestRecorder into a generic plugin interface for progress output
etc., which needs to be abstracted for PHPUnit integration.
* Introduce MultiTestRecorder for recorder chaining, instead of using
a long inheritance chain. All test recorders now directly inherit from
TestRecorder.
* Move all console-related code to the new ParserTestPrinter.
* Introduce PhpunitTestRecorder, which is the recorder for the PHPUnit
frontend. Most events are ignored since they are never emitted in the
PHPUnit frontend, which does not call runTests().
* Put more information into ParserTestResult and use it more often.
Setup and teardown:
* Introduce a new API for setup/teardown where setup functions return a
ScopedCallback object which automatically performs the corresponding
teardown when it goes out of scope.
* Rename setUp() to staticSetup(), rewrite. There was a lot of cruft in
here which was simply copied from Setup.php without review, and had
nothing to do with parser tests.
* Rename setupGlobals() to perTestSetup(), mostly rewrite. For
performance, give staticSetup() precedence in cases where they were
both setting up the same thing.
* In support of merged setup code, allow Hooks::clear() to be called
from parserTests.php.
* Remove wgFileExtensions -- it is only used by UploadBase which we
don't call.
* Remove wgUseImageResize -- superseded by MockMediaHandlerFactory which
I imported from NewParserTest.
* Import MockFileBackend from NewParserTest. But instead of
customising the configuration globals, I injected services.
* Remove thumbnail deletion from upload teardown. This makes glob
handling as in the old parserTests.php unnecessary.
* Remove math file from upload teardown, math is actually an extension
now! Also, the relevant parser tests were removed from the Math
extension two years ago in favour of unit tests.
* Make addArticle() private, and introduce addArticles() instead, which
allows setup/teardown to be done once for each batch of articles
instead of every time.
* Remove $wgNamespaceAliases and $wgNamespaceProtection setup. These were
copied in from Setup.php in 2010, and are redundant since we do
actually run Setup.php.
* Use NullLockManager, don't set up a temporary directory just for
this alone.
Fuzz tests:
* Use the new TestSetup class.
* Updated for ParserTestRunner interface change.
* Remove some obsolete references to fuzz tests from the two frontends
where they used to reside.
Bug: T41473
Change-Id: Ia8e17008cb9d9b62ce5645e15a41a3b402f4026a
Since in several cases, with an all-in-one commit, git's file rename
detection failed, I split the renames out into their own commit to
make review easier. Some changes here won't make complete sense without
the following commit.
* Moved TestsAutoLoader to tests/common/. It will be joined by a friend.
* Renamed ParserTest to ParserTestRunner, since the former name was
overly generic.
* Renamed TestFileIterator to TestFileReader. Please see the subsequent
commit for rationale.
* Moved parserTests.php to tests/parser/. It was the only file left in
tests/, and it should have been moved to tests/parser years ago,
analogous to phpunit.php.
* Renamed NewParserTest to ParserIntegrationTest. This was a tricky one,
apparently the name has to end in "Test" or else the structure test
will fail. Analogous to ParserMethodsTest etc. Rationale: because it's
not new anymore.
* Renamed MediaWikiParserTest to ParserTestTopLevelSuite and moved it to
the suites directory. A more descriptive name. Being in suites/
shields it from StructureTests, and is correct anyway.
Change-Id: Iddc6eaf815fdd64b3addb8570b4b6303ab99d634
It's very common these days for test runs on ordinary rigs to skip hundreds of
tests, which makes getting the part of the results you care about (the test
failures) an arduous trip up hundreds of lines in your terminal buffer.
Change-Id: Iaf714b86473379e1ad32e4d2d93e83018a757782
The MediaWiki test suite is painfully slow and delays merging of
changes. More than half of the time is spent in
ParserTest_Parser⁄parserTests::testParserTest which is the PHPUnit
wrapping class for the parser tests.
This patch let us extract the parser tests so we can run them
independently. By running them parallely with the rest of the tests,
that will speed up the gate processing time.
Mark the MediaWikiParserTest and NewParserTest class as belonging to the
test group 'ParserTests'. Will let us filter them out via PHPUnit
option --exclude-group
Introduce a new PHPUnit test suite 'parsertests' which loads the
MediaWiki core parser tests wrapper 'MediaWikiParserTest' and the suite
which loads the extensions parser tests (ExtensionsParserTestSuite.php).
This way we can run solely the parser tests with:
cd tests/phpunit
php phpunit.php --testsuite ParserTests
Wikimedia CI can then be configure to run two jobs:
A) php phpunit.php --exclude-group ParserTests
B) php phpunit.php --testsuite ParserTests
Bug: T114314
Change-Id: Ie819bab43163995048c073691c4c5d258f797c02
To add integration jobs for the MediaWiki skins, we will use the 'skins'
test suite. Add the structure and less validator tests to it. The
skins would then be tested using something like:
php tests/phpunit/phpunit.php --testsuite skins
Bug: T68926
Change-Id: I1be4454b1aa7935ec2e9ac3703eee3907e5101e8
The tests in the structure PHPUnit testsuite are used to verify the
Autoloader, ResourceLoader definitions and some basic other tests.
They are not run by Jenkins for MediaWiki extensions since we invoke:
phpunit.php --testsuite extensions
Add the structure directory to the 'extensions' testsuite. That will
start enforcing them on all extensions, so need to be merged with care.
We will want to backport this patch on all release branches we support
and make sure they pass for all extensions/branches :-/
Bug: T78798
Change-Id: Icb88fb3f10d203cfcb61c36c9c92864b92d6739c
Add the LessFileCompilationTest test case class, which represents the
validation of a LESS file by compilation.
Add the LessTestSuite test suite, which tests all LESS files registered
with the ResourceLoader, and use this to rewrite the checkLess.php
maintenance script.
Bug: 54665
Change-Id: Iedb8dc31e4817d8b4e40b655cf9b8fb092979e90
This has the effect of excluding the tests themselves from the code coverage
and including files that no test loads.
Change-Id: I65674ca3569d1ae6484b6769366213d71c0bdbc4
To better control what we are really covering, this patch uses PHPUnit
forceCoversAnnotation option. That makes it to record covering if and
only if @covers is used.
Also rewrapped the options to uses tabulations.
Change-Id: I467a5077a65b93cce08072a8d088b0b9c652a426
Parser tests are registeredd by appending one or more .txt files to the
$wgParserTestFiles global setting. Since this is shared with MediaWiki
core, I have made MediaWikiParserTest a factory of PHPUnit testsuite
which would filter in/out extensions tests.
The `extensions` test suite now has a second test suite builder which is
simply a wrapper around MediaWikiParserTest factory.
Play cases:
$ php phpunit.php --group Parser --tap
Runs any parser tests including the ones coming from extensions.
With an extension having parser tests such as Cite:
$ php phpunit.php --testsuite extensions --tap
// Extensions tests are run including parser tests.
bug: 42506
Change-Id: Icc3e9d30706b32149aa9dd18552e4241ec4af67e
The phpunit root directory has two test file:
AutoLoaderTest.php
StructureTest.php
The later was registered in phpunit under the `structure` test suite
while the former was not registered and hence never run (bug 47750).
This patch moves both files under the `structure` subdirectory and
change the suite to look in that directory. That will avoid us having
to manually maintain a list of test files.
Updated the __DIR__ in StructureTest.php.
Change-Id: I419c9157f32bdf7e1ff26a42f4bb3f3922b7be37
default .. 2s -> 10s
medium ... 10s -> 30s
I originally thought that using PHPUnit timeout system was a good idea
but that is causing more harm than being helpful. Now default to ten
seconds.
Change-Id: If10a9b6df967130838a253d986b2ed623d8e3b0a
PHPUnit has a timeout protection system which let us put tests in three
categories having different timeout (default: 1s, 10s, 60s). The timeout
only happens when using strict mode and having PHPUnit Invoker
installed.
Recently, the continuous integration server has been upgraded and the
PHPUnit Invoker was installed to let us run a serie of tests related to
dumping the database to XML. They do need some timeout system. Suddenly,
some tests started failing.
By default all tests are in the 1 second timeout group. Since we use a
sqlite backend and the server hard disk can get busy, some test can take
more than one second to execute.
This patch raise the default (small group) timeout to 2seconds and
explicitly define the value for the two other groups (medium and large)
using PHPUnit default values.
Change-Id: I2f5613b0c0215023c413dec8e84804175d8c8d6e
Adds tests/phpunit to the include_path to let PHPUnit find out our files.
Also add to change the suite.xml paths from './path' to 'path' so
stream_resolve_path() correctly find out the suite by using include_path
suite.xml now comes with colors=true. Under Windows, we override the
setting by forging a --no-colors. One can still force color usage
under windows by using --colors.
lack of fuzz testing and other parser test options, and the DB creation is still a little flaky.
A MediaWikiPHPUnitCommand class had to be created to allow for custom CLI parameters.
