Commit graph

375 commits

Author SHA1 Message Date
Amir Sarabadani
440297fa3a Clean up most of b/c for phpunit4
We don't support phpunit4 anymore:
https://w.wiki/9UT

Change-Id: Ia7c932998f1a49af1e322ab9e1dc249b7bd2e8b2
2019-10-06 00:14:35 +02:00
Timo Tijhof
29e0183a56 phpunit: Repair GLOBALS reset in MediaWikiUnitTestCase
This code didn't work because the $GLOBALS array is exposed by reference.
Once this reference was broken by unset(), the rest just manipulated a
local array that happens to be called "GLOBALS". It must not be unset or
re-assigned. It can only be changed in-place.

Before this, the execution of a MediaWikiUnitTestCase test stored a
copy of GLOBALS in unitGlobals, then lost the GLOBALS pointer and
created a new variable called "GLOBALS". As such, the tearDown() function
didn't do what it meant to do, either – which then results in odd
failures like T230023

Rewrite it as follows:

* In setup, store the current GLOBALS keys and values, then reduce
  GLOBALS to only the whitelisted keys and values.

* In teardown, restore the original state.

* As optimisation, do this from setUpBeforeClass as well, so that
  there are relatively few globals to reset between tests.
  (Thanks @Simetrical!)

The following tests were previously passing by accident under
MediaWikiUnitTestCase but actually did depend on global config.

* MainSlotRoleHandlerTest (…, ContentHandler, $wgContentHandlers)
* SlotRecordTest (…, ContentHandler, $wgContentHandlers)
* WikiReferenceTest (wfParseUrl, $wgUrlProtocols)
* DifferenceEngineSlotDiffRendererTest (DifferenceEngine, wfDebug, …)
* SlotDiffRendererTest (…, ContentHandler, $wgContentHandlers)
* FileBackendDBRepoWrapperTest (wfWikiID, "Backend domain ID not provided")
* JpegMetadataExtractorTest (…, wfDebug, …, LoggerFactory, …)
* ParserFactoryTest (…, wfDebug, …, LoggerFactory, InvalidArgumentException)
* MediaWikiPageNameNormalizerTest (…, wfDebug, …, LoggerFactory, …)
* SiteExporterTest (SiteImporter, wfLogWarning, …)
* SiteImporterTest (Site::newForType, $wgSiteTypes)
* ZipDirectoryReaderTest (…, wfDebug, …, LoggerFactory, …)

Bug: T230023
Change-Id: Ic22075bb5e81b7c2c4c1b8647547aa55306a10a7
2019-09-02 20:58:34 +01:00
Aaron Schulz
26b1ba3989 parser: document what 0/null rev IDs do when given to Parser::parse
This was extremely obscure and tricky to figure out before

Change-Id: Ibdbb669431819eecd34de508735864e977d51021
2019-08-21 11:13:49 -07:00
Aryeh Gregor
8ec24b607a Introduce MovePageFactory
This will help make MovePage more testable.

In the course of abstracting the logic out of ParserFactoryTest to
FactoryArgTestTrait so it could be used in MovePageFactoryTest, I made
them all unit tests instead of integration. This required some
modification to the Parser constructor so that it didn't access
MediaWikiServices unnecessarily.

Change-Id: Idaa1633f32dfedfa37516bb9180cfcfbe7ca31aa
2019-08-19 20:25:31 +03:00
Tim Starling
212ae934cd Revert rename of mSpeculativeRevId to speculativeRevIdUsed
And add a test which is confirmed to fail on HHVM prior to this change
with the error message "serialize(): "" returned as member variable from
__sleep() but does not exist".

Bug: T229366
Change-Id: I236bb4d64bc2e9f7756885e8c418399804eac5e1
2019-07-31 02:42:27 +00:00
Amir Sarabadani
d23af35764 Unset all globals unneeded for unit tests, assert correct directory
* Unset globals to avoid tests that look like unit tests but actually rely on
  globals
* move some tests out of unit directory so that the test suite will pass.
* Assert that tests which extend MediaWikiUnitTestCase are in a directory with
  "/unit/" in its path name

Depends-On: I67b37b1bde94eaa3d4298d9bd98ac57995ce93b9
Depends-On: I90921679518ee95fe393f8b1bbd9134daf0ba032
Bug: T87781
Change-Id: I16691fc8ac063705ba0c2bc63b96c4534ca8660b
2019-07-09 14:09:29 -04:00
Amir Sarabadani
7ec9745444 Split SanitizerTest to unit and integration tests
Out of 150 tests of SanitizerTest.php, 100 of them are pure unit tests
they are moved to the new file in the new structure, the rest stay

Change-Id: I366d37607abff4bcd624a56fb8b2299729fbc088
2019-07-08 09:48:07 +02:00
daniel
dbce648a15 Ensure canHaveTalkPage returns false when getTalkPage would fail.
This causes Title::getTalkPage and NamespaceInfo::getTitle() to throw
an MWException when called on a LinkTarget that is an interwiki link
or a relative section link. These methods were already throwing
MWException when called on a link to a Special page.

Bug: T224814
Change-Id: I525c186a5b8b8fc22bca195da48afead3bfbd402
2019-07-03 10:40:10 +02:00
Máté Szabó
344481f60d Move trivially compatible tests to the unit tests suite
This changeset resumes work on T89432 and related tickets
by porting an initial set of tests to the new unit test suite
separated out in I69b92db3e70093570e05cc0a64c7780a278b321a.
The tests were only ported if they worked immediately without
requiring any changes other than changing the test case class
to MediaWikiUnitTestCase and moving the test to the new suite.
If a test failed for any reason (even trivial misconfiguration),
it was NOT ported.

With this change, the unit tests suite now consits of a total
of 455 tests. As before, you can run these tests via the following
command:
$ composer phpunit:unit

Bug: T84948
Bug: T89432
Bug: T87781
Change-Id: Ibb8175981092d7f41864e641cc3c118af70a5c76
2019-06-30 15:23:53 +02:00
Aaron Schulz
ca00e47f53 revision: rename various $wikiId fields/parameters to $dbDomain
These fields are passed to methods like LoadBalancer::getConnection() and are
already expected to be DB domains. Update various comments as well.

Fix a few minor IDEA warnings.

Change-Id: I7cf76700690aec449872a80d30b5ba540d2bf315
2019-06-28 21:30:25 +00:00
C. Scott Ananian
bda42cef3c Deprecate Sanitizer::setupAttributeWhitelist/attributeWhitelist
These methods should be made private in the next release, but
hard-deprecate them for 1.34.

Tweak the return value of the attribute whitelist to be an
associative rather than a sequential array, which makes the
lookup of allowed attributes more efficient and avoids an
array_flip for every html element sanitized.

Bug: T221677
Change-Id: I17d734937accec6c2679dbe17328cf9554bd556a
2019-06-20 14:42:20 -04:00
C. Scott Ananian
53fe91ded5 Hard deprecate Preprocessor_DOM
The Preprocessor_DOM implementation doesn't interact well with PHP memory
profiling, and has some limitations not present in the Preprocessor_Hash
implementation (see T216664).  There is no reason to keep around two
versions of the preprocessor: it just complicates on-going wikitext
feature development.

Hard deprecate use of Preprocessor_DOM, so we can remove the redundant
code in a future release.

Bug: T204945
Depends-On: Id38c9360e4d02b570996dbf7a660f964f02f1a2c
Change-Id: Ica5d1ad5b1e677542962fc36d582a793f941155e
2019-06-14 12:21:40 -04:00
Legoktm
4e35134f7a Revert "Separate MediaWiki unit and integration tests"
This reverts commit 0a2b996278.

Reason for revert: Broke postgres tests.

Change-Id: I27d8e0c807ad5f0748b9611a4f3df84cc213fbe1
2019-06-13 23:00:08 +00:00
Máté Szabó
0a2b996278 Separate MediaWiki unit and integration tests
This changeset implements T89432 and related tickets and is based on exploration
done at the Prague Hackathon. The goal is to identify tests in MediaWiki core
that can be run without having to install & configure MediaWiki and its dependencies,
and provide a way to execute these tests via the standard phpunit entry point,
allowing for faster development and integration with existing tooling like IDEs.

The initial set of tests that met these criteria were identified using the work Amir did in
I88822667693d9e00ac3d4639c87bc24e5083e5e8. These tests were then moved into a new subdirectory
under phpunit/ and organized into a separate test suite. The environment for this suite
is set up via a PHPUnit bootstrap file without a custom entry point.

You can execute these tests by running:
$ vendor/bin/phpunit -d memory_limit=512M -c tests/phpunit/unit-tests.xml

Bug: T89432
Bug: T87781
Bug: T84948
Change-Id: Iad01033a0548afd4d2a6f2c1ef6fcc9debf72c0d
2019-06-13 22:56:31 +02:00
Max Semenik
214b37ff07 SECURITY: blacklist CSS var()
Bug: T208881
Change-Id: I9a4ced2bc47eb5f96cf35e693bf5261c48acb126
2019-06-06 16:15:55 +00:00
Thiemo Kreuz
8a4e15db7f Add missing newline between <?php and namespace/use section
The rest of the codebase is using this code formatting standard.

Change-Id: I4d2ba61757a7e28d40096d9dc5915005c340d4f2
2019-06-03 13:36:48 +00:00
Aryeh Gregor
7149153754 Don't pass Config to Parser(Factory)
Change-Id: I5996b7cad8c8a61518d2997e955a4547c64a73a5
2019-05-20 11:19:21 -05:00
Aryeh Gregor
e6df285854 Remove all $wgParser use from core
Bug: T160811

Change-Id: I0556c04d33386d0339e02e2bf7a1ee74d97c2abd
2019-04-17 15:16:50 +03:00
Aaron Schulz
fdbb64f354 Avoid extra parse/save delay for users with non-canonical parser options
If {{REVISIONID}} results in a re-parse, that re-parse will be post-send
unless the user has canonical parser options and will need the output for
page views anyway (e.g. the refresh after editing).

Also make getPreparedEdit() allow lazy-loading of the parser output via
a callback. A magic __get() method handles objects created the new way
but accessed by other code the old way.

Bug: T216306
Change-Id: I2012437c45dd605a6c0868dea47cf43dc67061d8
2019-04-14 02:00:16 +00:00
Erik Bernhardson
aef02d516d Improve RemexStripTagHandler working with tables
HTML, generated by some infoboxes and perhaps other places, gets
stripped in a way that merges words together that should not be
merged. Add tr, th, and td to the list of tags that should force
word separation.

Bug: T218001
Change-Id: Ib374339628b1f543ea4e07f24aa3e3b76f3117b5
2019-03-14 13:11:59 -07:00
Timo Tijhof
c6f3440832 resourceloader: Remove addModuleScripts, and deprecate getModuleScripts.
The addModuleScripts() methods were deprecated in 1.31 and 1.32,
these are now removed.

The getModuleScripts() are now deprecated as well, always returning
an empty array. To be removed in 1.34.

Depends on commits for bundled/wmf-deployed extensions that
remove the last few remaining callers to the deprecated functions
in: 3D, Collection, Flow, GlobalUserPage, and Wikibase.

Bug: T188689
Depends-On: If9f0bc6aef85117587fa1929f34f8861c8d80314
Depends-On: Ia8d41b97fbf6822f5f8f7ac889408acce1ac9a3a
Depends-On: I503b919739ea474ff33726815b0da55e2f7e2724
Depends-On: I236ef637fd03b810a46eb361e25067a037e9d183
Depends-On: I62e17779753b977a452cc0c9694947941e999cc3
Change-Id: I5a19b8f164ccf666485d2971202194b747f882df
2019-03-05 16:54:08 +00:00
Umherirrender
f45d3a797a Add missing @covers to parser related tests
Change-Id: I615ba41c9a6f235f65991549cb27003438115fe9
2019-02-01 21:15:40 +00:00
C. Scott Ananian
6db35b3c98 Remove most support for configuring Tidy, including Raggett
Remex is pure PHP so there is no reason to use an external tidy any
more. Configuration variables and implementation classes were
deprecated in 1.32 or earlier.  We've kept only $wgTidyConfig
which can be used for experimental features or debugging Remex.

Bug: T198214
Change-Id: I99d48f858d97b6e1d1e6cd76a42c960cc2c61f9f
2018-11-15 12:22:06 -05:00
jenkins-bot
7a3eb1f3a6 Merge "Hard deprecate codepaths where tidy is disabled" 2018-11-13 23:54:24 +00:00
Brad Jorsch
d65e96b763 Use new externallinks.el_index_60 field
This adds a method to LinkFilter to build the query conditions necessary
to properly use it, and adjusts code to use it.

This also takes the opportunity to clean up the calculation of el_index:
IPs are handled more sensibly and IDNs are canonicalized.

Also weird edge cases for invalid hosts like "http://.example.com" and
corresponding searches like "http://*..example.com" are now handled more
regularly instead of being treated as if the extra dot were omitted,
while explicit specification of the DNS root like "http://example.com./"
is canonicalized to the usual implicit specification.

Note that this patch will break link searches for links where the host
is an IP or IDN until refreshExternallinksIndex.php is run.

Bug: T59176
Bug: T130482
Change-Id: I84d224ef23de22dfe179009ec3a11fd0e4b5f56d
2018-11-12 22:33:18 +00:00
C. Scott Ananian
54ac31f94d Hard deprecate codepaths where tidy is disabled
Future parsers will not support the output generated with tidy disabled.

Parser tests using untidied output will also be deprecated (and
rewritten) in a follow-up patch.

No new release notes necessary since user-visible tidy configuration
was deprecated previously (in 1.32), and individual methods which had
disabled tidy during execution were individually release-noted as they
were updated.

Bug: T198214
Depends-On: I0f417f75a49dfea873e9a2f44d81796a48b9f428
Depends-On: If5c619cdd3e7f786687cfc2ca166074d9197ca11
Change-Id: I592e0e0dfef7d929f05c60ffe4d60e09725b39cc
2018-11-05 18:49:16 +00:00
Tim Starling
a6a017cea4 Fix use of non-existent variable Parser::$config
Fix bug from Ib4394f370cb561ccf195338a1c2e9e465dcb3dc3

Add test.

Bug: T208000
Change-Id: Ia81cca1b64afef2af3cb8dff19719a7f0de9d306
2018-10-25 16:27:55 -07:00
jenkins-bot
b53f299af2 Merge "Revert "Unwrap HTML loaded from parser cache"" 2018-10-11 18:11:55 +00:00
Gergő Tisza
617dbc4c18 Revert "Unwrap HTML loaded from parser cache"
This reverts commit 1bb5b58eb1.
A month has passed, the workaround for old parser cache entries
should not be needed anymore.

Bug: T203716
Change-Id: I446b47cc6b4c43aaae33675d62086d842b04ddcb
2018-10-09 18:29:20 +00:00
Brad Jorsch
dff469a408 Re-namespace RevisionStore and RevisionRecord classes
During development a lot of classes were placed in MediaWiki\Storage\.
The precedent set would mean that every class relating to something
stored in a database table, plus all related value classes and such,
would go into that namespace.

Let's put them into MediaWiki\Revision\ instead. Then future classes
related to the 'page' table can go into MediaWiki\Page\, future classes
related to the 'user' table can go into MediaWiki\User\, and so on.

Note I didn't move DerivedPageDataUpdater, PageUpdateException,
PageUpdater, or RevisionSlotsUpdate in this patch. If these are kept
long-term, they probably belong in MediaWiki\Page\ or MediaWiki\Edit\
instead.

Bug: T204158
Change-Id: I16bea8927566a3c73c07e4f4afb3537e05aa04a5
2018-10-09 10:22:48 -04:00
jenkins-bot
f340c7271c Merge "ParserOutput::getCacheTime should stay the same after the first call." 2018-10-04 16:58:00 +00:00
jenkins-bot
a4120368bb Merge "Only use "*Test.php" for actual PHPUnit tests" 2018-10-04 15:43:12 +00:00
daniel
414215ccac ParserOutput::getCacheTime should stay the same after the first call.
Previously, getCacheTime would default to the current time, potentially
causing the return value to change over subsequent calls. With this change,
the value is determined on the first call, and then remembered for subsequent
calls.

Bug: T205464
Change-Id: If240161c71d523ad5b0d33b9378950e0bebceb6e
2018-10-04 13:08:56 +02:00
Aryeh Gregor
2bb3551082 Only use "*Test.php" for actual PHPUnit tests
Otherwise I get errors every time I try to run PHPUnit on includes/ or
includes/parser, because it tries to run ParserIntegrationTest.php and
fails.  Apparently the <exclude> in suite.xml doesn't work if PHPUnit is
invoked on a directory.

Bug: T201278
Change-Id: I7d09576bee2705d8516152e8fa671da8dac40233
2018-10-03 20:22:22 +03:00
jenkins-bot
67dc908246 Merge "Preserve whitespace in search index text content" 2018-10-01 18:22:48 +00:00
Gergő Tisza
6e8d39c6e7
Add constant for the name of the 'main' slot for MCR
Bug: T202142
Change-Id: I97a74e5a029b014f3c2195188936d5c8233c1b7f
2018-09-24 16:52:12 -07:00
Erik Bernhardson
0d779c1ac6 Preserve whitespace in search index text content
Certain html tags imply a word break, but our html stripping doesn't
understand that at all. Adjust the html stripping to inject whitespace
for all block level tags (per MDN) along with the <br> element.

Bug: T195389
Change-Id: I9fbfac765ea88628e4f9b2794fb54e1cd0060203
2018-09-14 11:10:35 -07:00
Gergő Tisza
1bb5b58eb1
Unwrap HTML loaded from parser cache
Partially and temporarily reverts I1641b7995 to deal with cached
HTML the same way the old code did.

Bug: T203716
Change-Id: I29846a6513f6b580b429c0bfe6c310ada50b28bb
2018-09-07 18:49:34 +02:00
daniel
465954aa23 Provide new, unsaved revision to PST to fix magic words.
This injects the new, unsaved RevisionRecord object into the Parser used
for Pre-Save Transform, and sets the user and timestamp on that revision,
to allow {{subst:REVISIONUSER}} and {{subst:REVISIONTIMESTAMP}} to function.

Bug: T203583
Change-Id: I31a97d0168ac22346b2dad6b88bf7f6f8a0dd9d0
2018-09-06 18:33:44 +02:00
daniel
d8c409dd16 Make HTML generation in RenderedRevision optional
This allows optimization for situations in which a caller
needs the meta-data of a ParserOutput, and the respective
ContentHandler can provide that meta-data without generating
HTML output.

Bug: T194048
Change-Id: I786d294d18a6a2e3cea61577313e21b578c44f1e
2018-08-31 10:48:41 +00:00
daniel
e9f71517f7 [MCR] Introduce RevisionRenderer
RevisionRenderer is the MCR replacement for Content::getParserOutput,
as outlined in <https://www.mediawiki.org/wiki/User:Daniel_Kinzler_(WMDE)/MCR-PageUpdater>.

Note: This change also introduces quite a bit of code for
merging ParserOutput objects.

Bug: T194048
Change-Id: I871978bf79f67c9e7954fb3fc8528d6e365f2cc1
2018-08-30 19:15:12 +02:00
daniel
0dc7ba02b4 Apply content wrapping in ParserOutput::getText()
Instead of applying wrapping the the parser and unwrapping in
ParserOutput::getText(), turn this around and apply wrapping in getText(),
and only if desired.

This avoids search&replace logic for unwrapping, and it also makes it a lot
easier to merge the output of multiple slots for MCR output.

This changes behavior in two hopefully irrelevant ways:
1) the limit report comments will be inside the wrapper div, instead of
following it.
2) if HTML with a wrapper div is explicitly injected into a ParserOutput
object, it will not be possible to unwrap the text.

Bug: T174035
Change-Id: I1641b7995af9bd297f1acd610d583fbf874f34e0
2018-08-29 16:46:25 +02:00
Aryeh Gregor
90d4f56fe4 Mass conversion of $wgContLang to service
Brought to you by vim macros.

Bug: T200246
Change-Id: I79e919f4553e3bd3eb714073fed7a43051b4fb2a
2018-08-11 22:44:29 -06:00
Aryeh Gregor
bca6085920 Use ParserFactory in a bunch of places
I wasn't sure how to convert the rest of the occurrences in core (there
are a significant number).

Bug: T200881
Change-Id: I114bba946cd3ea8a293121e275588c3c4d174f94
2018-08-11 00:16:17 -06:00
Aryeh Gregor
62515f7b15 Introduce ParserFactory service
Bug: T200881
Change-Id: I257e78200983cb10afb76de1f07dd1b9d531c52a
2018-08-11 00:15:52 -06:00
Aryeh Gregor
355e21590a Use setContentLang() instead of setMwGlobals()
This changes behavior in some tests by making them set $wgLanguageCode
as well as $wgContLang, but that seems like a good thing.

Bug: T200246
Change-Id: I936888f46ff9fefe2707efba837e2ce3a7ca5e3f
2018-07-26 11:35:58 +00:00
Brad Jorsch
c6810d74d1 Deprecate ContentHandler::makeParserOptions()
Having a different ParserOptions for each content model isn't feasible
in an MCR world. And the only thing using this was Wikibase, which has
been fixed to do what it needs in a different way.

Bug: T194263
Change-Id: I01373b29ee25fa9346c6b0317155be4ccdc8c515
2018-07-13 14:32:59 -04:00
Brad Jorsch
c3900e06be Resolve used lazy options in ParserOptions::optionsHash()
If a lazy option is passed to ParserOptions::optionsHash(), we should
resolve the option so the hash can incorporate the proper value instead
of omitting it.

Also, completely unrelatedly, refactor the hook overriding in the unit
test because people won't stop whining about it in code review.

Change-Id: I2df78ed90875c229090b503b65f20fbbbba7f237
2018-05-15 06:54:55 +00:00
James D. Forrester
846f4f58f5 Remove $wgExperimentalHtmlIds and related code, deprecated in 1.30
Bug: T139744
Change-Id: Ia15d5ab6e7637fd40d5c3399822a3dbeb7b383b5
2018-05-01 14:34:02 -07:00
James D. Forrester
78e5c8d7af Default installations to using RemexHtml for tidying
This combines two changes – defaulting tidying to on, previously off, and
defaulting the tidying library to RemexHtml, previously the tidy binary.
Config options are going to be a bit of a mess until we drop support for
the old tidy binary config route.

Bug: T185753
Depends-On: I0a8973f508fbf65160177b003260831639828eeb
Change-Id: I6879a77a78d780c7c056d807dde20682c6097d1b
2018-04-10 10:51:34 -07:00