Some of Wikibase’s self-proclaimed unit tests don’t actually work when
run as pure unit tests, and as long as the unit-ness isn’t enforced in
CI, I doubt that’s going to change – but it makes running the tests on
their own annoying (I noticed this again in T362505). Let’s add a way to
override the detection via an environment variable.
Change-Id: I9aae6b2446d0471d44474654b57a2d48849a4d88
Paratest takes additional options that are not recognized by PHPUnit.
While paratest does unset them when invoking PHPUnit, we were previously
passing them through in the bootstrap file, which results in an uncaught
exception.
So, make a list of a few known paratest options and make PHPUnit ignore
them.
This is obviously doomed to fail miserably as soon as someone tries to
pass another paratest option that is ot listed here, but since we aren't
officially supporting paratest at the moment, it should be fine for the
time being.
Bug: T50217
Change-Id: I4c197676d9410adebc47baf50c699ec7cb411ed8
Turns out that XDebug causes the subprocess to hang, causing phpunit to
hang. Unsetting the relevant environment variables solves the problem.
Bug: T358097
Change-Id: I0d4e148aa5215ec055747636625c18032bbfbc49
Leave `composer phpunit` as the main PHPUnit entry point, a single
config file (phpunit.xml.dist) and a single bootstrap (bootstrap.php).
Deprecate the `composer phpunit:entrypoint`, making it print a
deprecation message and then fall back to `composer phpunit`. Deprecate
boostrap.integration.php as well. Update the release notes accordingly.
Add a temporary PHPUnit extension, only enabled in suite.xml, that
prints a deprecation message. Define a new constant in phpunit.php to
avoid printing 2 deprecation messages when phpunit.php is used (as it
auto-defaults to suite.xml).
Add some documentation to bootstrap.php.
Bug: T227900
Depends-On: Ie91c9f2882f12fd8ea822529528dd1dd9efe89db
Change-Id: If2ffe0522422fd6aadcc08988789d98c2403fc48
Do that in the bootstrap and remove
initializeForStandardPhpunitEntrypointIfNeeded. The problem with the
latter is that it runs after data providers, so it wouldn't be possible
to read config etc. there. You may argue that dataproviders shouldn't do
complex things such as reading config. That's a valid concern, but it's
not easy to achieve in the current state of things. This decision might
be reconsidered if the MW settings infrastructure is improved and moved
away from globals.
Also, doing a deferred initialization in MediaWikiIntegrationTestCase
means that (unit) tests that run before said initialization and the ones
running after would see different states of the system (one where config
has been loaded, one where MediaWiki might not even be installed). Doing
it earlier guarantees that there won't be such differences inside a test
run.
Ideally, we would use PHPUnit events to determine if we need settings
once we have a list of tests to execute. But that can't happen until we
migrate to PHPUnit 10.
Print a message saying whether MW settings before starting PHPUnit, as
this might be useful when debugging failures.
Change-Id: I467f82ed9a88cf2cfa63f76abac9c264904eb492
In a future with a single bootstrap and config, we won't be able to run
hooks as early as in ExtensionTestSuite::suite. On top of that, if the
config file does not fully load the MW config (as is the case with
bootstrap.php), it won't even be possible to run hooks at all.
So move the relevant code from ExtensionsTestSuite to
getPHPUnitExtensionsAndSkins, collect the output in bootstrap.php and
pass it to ExtensionsTestSuite.
This hack wouldn't be necessary if the UnitTestList hook didn't exist,
but fortunately there are plans to get rid of it (T298509).
Keep the old code in place when entering from an entry point that
doesn't load bootstrap.php (i.e., phpunit.php and composer
phpunit:entrypoint).
Bug: T227900
Change-Id: Idf72db24dbd66bb66baf51564a7504d2bc035e8c
When running unit tests, the bootstrap would previously load all
extensions and skins in the filesystem. This was OK for an initial
implementation, but is not acceptable if we want to eventually do that
for all PHPUnit entry points (once we'll have a single config and
bootstrap). Instead, it's desirable to only load the extensions
specified in LocalSettings.php. The problem is that it's pretty much
impossible to load LocalSettings.php without also loading the rest of
MediaWiki, with all the side effects this might have.
This patch introduces a helper script that loads all the config, then
prints what extensions and skins were loaded. The bootstrap file runs
this script via proc_open and then reads the list of extensions to
load. Because the script is run in a separate process, any side effects
only affect the spawned process, not the one where PHPUnit is running.
Currently, there doesn't seem to be a better way to obtain the list of
extensions loaded in LocalSettings.php without all the other side
effects. YAML settings
(https://www.mediawiki.org/wiki/Manual:YAML_settings_file_format) would
probably help, but that's very far from becoming the only supported
config format (if it will ever be).
Also add two TestSuite implementations to replace the '*' wildcard in the
extensions:unit and skins:unit suites. These use the same list of loaded
extensions to determine where to look for tests.
And last but not least: my most sincere apologies to you if the hack
you're seeing here has ruined your day. If you think a better approach
exists, please tell me and I'll be so relieved!
Bug: T227900
Change-Id: Ib578644b8a4c0b64dca607afb9eb8204ca7fc660
Add utility methods to TestSetup and call them from the bootstrap file.
Also call the same method to load settings from
MediaWikiIntegrationTestCase when invoked via the unit tests entry
point. This also fixes a couple bug in the deferred setup scenario where
setLoadTestClassesAndNamespaces wasn't called, and the error handler
wasn't reset (T227900#9003435).
Make MediaWikiIntegrationTestCase not try to load settings more than
once. Trying to do so should probably be mostly harmless because of the
require_once usage, but it's also unnecessary and this change avoids any
chance of unwanted side effects.
Also use MW_INSTALL_PATH instead of $IP in bootstrap.php.
Bug: T227900
Change-Id: I7090976435e7e2d1264ee345e2670baaccf532ea
- Move handling of PHPUNIT_WIKI to the common bootstrap, the values
might be needed in integration tests.
- Add method to TestSetup for checking composer.lock, and use it in both
bootstraps. This cannot be moved to the common bootstrap yet, because
it needs to run after loading MW.
- In MediaWikiIntegrationTestCase::
initializeForStandardPhpunitEntrypointIfNeeded, use global $IP instead
of redefining it.
Bug: T227900
Change-Id: I050dcc36b814abd7b909eaaf6d356ecd0d3a0c80
- Remove display_errors override. This was copied from the Maintenance
class in Id6d7e9db, but it's unnecessary here because we configure
PHPUnit to fail if a PHP notice/warning is emitted. This causes the
warning/notice to be converted to a failed assertion handled by
PHPUnit.
- Remove code that turned off output buffering, also copied from
Maintenance. I'd say it's unlikely for output buffering to enabled at
that point, and most importantly, we would enable output buffering a
bit later anyway.
- Call ob_start earlier, so that it also works for anything that prints
messages even before dataProviders are run (e.g., setup or
bootstrap code).
- Move the ob_start call to the common bootstrap, as unit tests are
still affected by the same issue that the call aimed to fix.
- Print the PHP version immediately, and do that for unit tests too.
- Get rid of MediaWikiCliOptions in favour of private code in
MediaWikiIntegrationTestCase, which is the only class supposed to
access those settings.
- Set wgCommandLineMode in the common bootstrap. Assume that nobody is
changing its value randomly in config files because that'd be
ill-advised.
- Inline code for setting memory_limit and max_execution_time, and run
it only after loading the settings. Doing that earlier is pointless,
because these values are already set in suite.xml.
- Update obsolete comment mentioning PHPUnitMaintClass, which is long
gone. In the current code, this is the only way (besides PHPUnit
hooks/events) to run code before PHP exits. Also remove the second
part because we're now on PHPUnit 9, so this should be actionable now.
Remove link to the PHPUnit documentation as it no longer works. There
doesn't seem to be an equivalent link for PHPUnit 9, perhaps because
hooks were replaced with events. There //is// an equivalent link in
the PHPUnit 10 documentation (for events), but we're not yet using it.
Change-Id: I79bbbfcfe4eab3ff5ae81b2deb6450ba06ad7611
- Consolidate the common parts into a shared bootstrap.common.php file
- Rearrange the code a bit to clarify when certain things (like globals)
are actually needed.
- Replace the bootstrap class with two global functions, and inline two
other functions.
This change should be a no-op for both unit and integration tests.
Change-Id: Ie20617ac80d12f99ceaeae53b852cc6ed76b3feb
Follows-up I58ff3c193190d78a. Small step toward not run-time reading
the global (and using Config here is non-trivial/unsafe).
Change-Id: Ic527e493baabe700c50f75fadaa5b51615a5e597
This change cleans up when and how we register autoloader info for
extensions.
This bumps CACHE_VERSION to 8, to avoid issues in case this patch gets
reverted: the new code doesn't copy the map of class files into
$info['globals']['AutoloadClasses']. If old code was to read a new cache
entry, autoloading would fail.
BREAKING CHANGE: The following methods have been removed from
ExtensionRegistry without deprecation and without replacement.
They had been introduced in 1.35 for use in the testing framework,
and were not in use by any known extension:
- exportAutoloadClassesAndNamespaces
- exportTestAutoloadClassesAndNamespaces
NOTE: breaks the hack that SocialProfile has in place for T243861.
Bug: T240535
Change-Id: I6e1ceac034c443d9475f1adc1babecddd6af6d05
DefaultSettings.php has been replaced by MainConfigSchema.
Loading DefaultSettings.php is deprecated.
Code that needs to have access to configuration defaults should use the
ConfigSchema service object.
Bug: T300129
Change-Id: I7b2c0ca95a78990be1cdb9dd9ace92f6dcf1af15
The global variable $IP has been replaced by the MW_INSTALL_PATH
constant. Clarify the continued use of $IP on Setup.php.
Change-Id: I157abfd9049fb8382da53005a084ab86f47e8d8a
TestSetup seems a nice place for this function. This way, it can also be
reused in the other boostrap file whilst we migrate the entrypoint.
Also, replace the check in MediaWikiIntegrationTestCase with another
constant; this also makes it easier to understand when exactly that code
should run.
Bug: T90875
Change-Id: I7858d982378ab4b6f11c4e9bf955d83d1acbc85d
Application logic should use the BaseDirectory config variable.
Framework code should use MW_INSTALL_PATH to locate files should.
NOTE: Update https://www.mediawiki.org/wiki/Manual:$IP
Bug: T300301
Depends-On: I7142af16d692f26e90673b058029f572c1ea3991
Change-Id: Ib4caa80bb7007c4c7960a2fd370cf5da7d9ba344
wfDetectLocalSettingsFile already defines MW_CONFIG_FILE, defining
it again in bootstrap.php triggers a warning.
Bug: T301503
Change-Id: I0f2892dd803e4d7be0e3e8c8bca26b1251bc6f14
This allows a file other than LocalSettings.php to be used as the primary
settings file by setting the MW_CONFIG_FILE environment variable.
This also allows the primary settings file to use YAML or JSON format.
Using static configuration files should be the default in the future.
However, YAML files in the document root could easily be exposed to the
public. Better not to encourage that, and require them to be enabled
explicitly and loaded from a different place.
Bug: T294750
Change-Id: I7747f83481cb05a6d05f819be652259951183819
In order to allow configuration default to come from JsonSchema,
and dynamic defaults need to be initialized in Setup.php.
This movesd the initialialization of $wgExtensionDirectory and $wgStyleDirectory
to Setup.php
Bug: T294788
Change-Id: I11f5723aa299caa210cf6a1f5b7436f191b1ffc2
When invoking `composer phpunit:unit` without any configuration, we lack
custom logging or developer enhancements such as $wgDevelopmentWarnings.
Inject `includes/DevelopmentSettings.php` in the PHPUnit bootstrap.
The $wgMWLoggerDefaultSpi is set to LegacyLogger which requires four
global variables:
- `$wgDebugLogFile`
- `$wgDebugLogGroups`
- `$wgDBerrorLog`
- `$wgDBerrorLogTZ`
Allow `$wgDebugLogFile` and `$wgDebugLogGroups` globals to be passed to
unit tests. `$wgDBerrorLog` and `$wgDBerrorLogTZ` are not, since I don't
think that applies to unit tests.
This change does not affect integration tests run via phpunit.php, where
developers may include DevelopmentSettings.php from LocalSettings.php in
a custom order before/after their own changes, or not at all.
Bug: T291227
Change-Id: I792ca0be3456f83b86fb4ebddae4117bf387be3a
Right now, for autoloading, it depends on a complex logic of
DirectoryIterator and a hard-coded set of extension.json names.
This is not useful and also doesn't pick up Wikibase extension.json
files because they are extension-repo.json and extension-client.json
which in turn causes all sorts of warnings
This patch fixes that
Bug: T243124
Change-Id: Iaf47a6f485f4ba2769b4e384278c8121366dd9a5
applyInitialConfig() was introduced to merge config setup between
phpunit.php and parserTests.php, but this test entry point was missed.
Change-Id: I94c8a31056127f2e3d6a0ddf47d18b9bae114abb
This introduces two new keys to the extension schema, analogous to
the existing autoloader keys: TestAutoloadNamespaces and
TestAutoloadClasses.
The classes and namespaces defined by these keys are only loaded by
the registry when ExtensionRegistry::setLoadTestClassesAndNamespaces
is called. The phpunit wrapper does this in PHPUnitMaintClass::setup.
The unit test bootstrap file calls
ExtensionRegistry::exportTestAutoloadClassesAndNamespaces instead,
similar to how it loads the regular classes and namespaces.
Bug: T196090
Change-Id: I88cf69663362fd599c20649b1df641907a02678d
This code didn't work because the $GLOBALS array is exposed by reference.
Once this reference was broken by unset(), the rest just manipulated a
local array that happens to be called "GLOBALS". It must not be unset or
re-assigned. It can only be changed in-place.
Before this, the execution of a MediaWikiUnitTestCase test stored a
copy of GLOBALS in unitGlobals, then lost the GLOBALS pointer and
created a new variable called "GLOBALS". As such, the tearDown() function
didn't do what it meant to do, either – which then results in odd
failures like T230023
Rewrite it as follows:
* In setup, store the current GLOBALS keys and values, then reduce
GLOBALS to only the whitelisted keys and values.
* In teardown, restore the original state.
* As optimisation, do this from setUpBeforeClass as well, so that
there are relatively few globals to reset between tests.
(Thanks @Simetrical!)
The following tests were previously passing by accident under
MediaWikiUnitTestCase but actually did depend on global config.
* MainSlotRoleHandlerTest (…, ContentHandler, $wgContentHandlers)
* SlotRecordTest (…, ContentHandler, $wgContentHandlers)
* WikiReferenceTest (wfParseUrl, $wgUrlProtocols)
* DifferenceEngineSlotDiffRendererTest (DifferenceEngine, wfDebug, …)
* SlotDiffRendererTest (…, ContentHandler, $wgContentHandlers)
* FileBackendDBRepoWrapperTest (wfWikiID, "Backend domain ID not provided")
* JpegMetadataExtractorTest (…, wfDebug, …, LoggerFactory, …)
* ParserFactoryTest (…, wfDebug, …, LoggerFactory, InvalidArgumentException)
* MediaWikiPageNameNormalizerTest (…, wfDebug, …, LoggerFactory, …)
* SiteExporterTest (SiteImporter, wfLogWarning, …)
* SiteImporterTest (Site::newForType, $wgSiteTypes)
* ZipDirectoryReaderTest (…, wfDebug, …, LoggerFactory, …)
Bug: T230023
Change-Id: Ic22075bb5e81b7c2c4c1b8647547aa55306a10a7
Also use wgExtensionDirectory and wgStyleDirectory in bootstrap.php, and add
clarifying comment about why ExtensionRegistry->readFromQueue is not used.
Follows-Up: I237a9f82e4d1b05cf2f08b3e4bb7ffcd8d47111c
Bug: T226911
Change-Id: I9c8e70006600c6eebbdc46dc7483a4481859ea16
Prior to this patch, one could not run this command from within an extension
directory: `../../vendor/bin/phpunit -c ../../phpunit.xml.dist tests/phpunit`,
because our code in bootstrap.php loaded "LocalSettings.php" instead of
$IP/LocalSettings.php.
Note that making this change does not result in loading LocalSettings.php when
running unit tests.
Bug: T87781
Change-Id: Iee8440f11a2bb255567646289a1986d636c12765
For now only extensions with skin/extension.json which follow PSR-4 are
supported.
DefaultSettings is loaded in bootstrap to work around errors when unit tests are
run followed by integration tests (e.g. `vendor/bin/phpunit`)
Usage:
- composer phpunit:unit will run all unit tests in core and any tests in
`{extensions/skins}/tests/phpunit/unit` which also extend
MediaWikiUnitTestCase, and which have an extension.json file
- Pass a specific directory to only run unit tests for a particular extension,
e.g. `composer phpunit:unit -- extensions/GrowthExperiments`
Bug: T226911
Change-Id: I237a9f82e4d1b05cf2f08b3e4bb7ffcd8d47111c
This changeset resumes work on T89432 and related tickets
by porting an initial set of tests to the new unit test suite
separated out in I69b92db3e70093570e05cc0a64c7780a278b321a.
The tests were only ported if they worked immediately without
requiring any changes other than changing the test case class
to MediaWikiUnitTestCase and moving the test to the new suite.
If a test failed for any reason (even trivial misconfiguration),
it was NOT ported.
With this change, the unit tests suite now consits of a total
of 455 tests. As before, you can run these tests via the following
command:
$ composer phpunit:unit
Bug: T84948
Bug: T89432
Bug: T87781
Change-Id: Ibb8175981092d7f41864e641cc3c118af70a5c76
Following discussion in Ibb8175981092d7f41864e641cc3c118af70a5c76, this patch
proposes to further reduce the scope of what unit tests may access, by removing
the loading of DefaultSettings and GlobalFunctions.php. This also has the
implied effect of disabling the storage backend, as well as the global service
locator.
MediaWikiTestCase is renamed to MediaWikiIntegrationTestCase so it's scope and
purpose is more clear. Whether we still need to keep `@group Database`
annotation around is debatable, as it's unclear to me what the performance costs
are of implying database access for all tests which extend IntegrationTestCase.
As far as I can tell, `@group Database` is primarily used in CI to run faster
tests before slower ones, and with the new UnitTestCase the annotation seems
redundant.
To run all testsuites, use `composer phpunit`. Other composer scripts:
- `composer phpunit:unit` to run unit tests
- `composer phpunit:integration` to run integration tests
- `composer phpunit:coverage` to generate code coverage reports from unit
tests (requires XDebug).
Note that you can pass arguments to composer scripts with `--`, e.g. `composer
phpunit:integration --exclude-group Dump`.
Other changes:
- Rename bootstrap.php to bootstrap.maintenance.php so it's clear it's part of
the legacy PHPUnit-as-maintenance-class setup
- Create new bootstrap.php which loads the minimal configuration necessary for
the tests, and do additional setup in the run() method of the unit/integration
test case classes
- Move the unit-tests.xml file to phpunit.xml.dist in preparation for this being
the default test configuration
For a follow-up patch:
- Find unit/integration tests for extensions/skins
- Migrate other test suites from suite.xml
- Support running all tests via vendor/bin/phpunit
Bug: T84948
Bug: T89432
Bug: T87781
Change-Id: Ie717b0ecf4fcfd089d46248f14853c80b7ef4a76
This will be called right before the shutdown procedure begins,
so that all variables and classes still exist, and the hooks
can actually work.
See task for details.
Bug: T219673
Change-Id: Id6b2b4b7958b78822a0868720f7b0ba46d7b4951
Follows-up I1343872de7, Ia533aedf63 and I2df2f80b81.
Also updated usage in text in documentation and the
installer LocalSettingsGenerator.
Most of them were handled by this regex:
- find: (require|include|require_once|include_once)\s*\(\s*(.+?)\s*\)\s*;$
- replace: $1 $2;
Change-Id: I6b38aad9a5149c9c43ce18bd8edbab14b8ce43fa
- Remove check for version, that version is already enforced in phpunit.php,
so there is not point showing a warning for it is useless
- Remove call to MessageCache::destroyInstance(), there is no need for it,
since $wgMessageCacheType is set in phpunit.php before running Setup.php
- Remove includes of bootstrap.php in LanguageSrTest.php and LanguageUzTest.php
Change-Id: I4b2db6b3e6f001175e1a407c5add2972aade5e60
We can now do this since we finally switched to PHP 5.3 for MW 1.20 and get rid of the silly dirname(__FILE__) stuff :)
Change-Id: Id9b2c9cd2e678197aa81c78adced5d1d31ff57b1
As Setup.php assigns variables based on the cache config, bootstrap.php was late on reseting them, as some objects were already created.
So we could end up with a SqlBagOStuff created there, which when later accessed (such as trying to invalidate the cache for a user) would
-as any non-sqlite SqlBagOStuff- open a new db connection. Which is precisely what we shall not be done when dealing with temporary tables
(and would indeed fail miserably due to not finding unittest_objectcache table).
In summary, reenabling temporary tables disabled in r79411.
* MessageCache::singleton() calls wfGetMessageCacheStorage() directly instead of using $messageMemc, just in case this would be called before that variable is set
* Per TimStarling: also removed deprecated methods in MessageCache class: addMessages() and related, [get|set|enable|disable]Transform(), loadAllMessages(), loadMessageFile() and some others. Same for the legacyData stuff in LocalisationCache that was only used by MessageCache::addMessages() and related.
* Converted remaining extensions
