Originally we created a Parser object on every request, and so care
was taken to make Parser construction lightweight. In particular,
all potentially costly initialization was moved into a separate
Parser::firstCallInit() method. Starting with 1.32, parser construction
has instead been done lazily, via the ParserFactory registered with
MediaWikiServices. The extra complexity associated with the old manual
lazy initialization of Parser is therefore no longer needed.
Deprecate Parser::firstCallInit() as part of a general plan to refactor
the Parser class to allow subclasses and alternate parser implementations.
Add some tests to assert that parsers are being created lazily, and are
not being created when they are not needed.
Bug: T250444
Change-Id: Iffd2b38a2f848dad88010d243250b37506b2c715
Parser::Options(), Parser::OutputType(), and Parser::Title() have been
deprecated. All of these had incomplete replacements with either a
::get* method or a ::set* method (and in the case of Title, both).
Add the missing getters or setters where required.
Only Parser::Title() has been hard deprecated. Replacing the other
uses in deployed code requires the newly-added Parser::getOutputType()
or Parser::setOptions() methods, so we can't replace those methods in
our deployed code after this patch has been merged.
Code search:
https://codesearch.wmflabs.org/deployed/?q=-%3E%28OutputType%7CTitle%7COptions%29%5C%28&i=nope&files=&repos=
Bug: T236809
Change-Id: I0b4d5f170216597afb259cedbb13b8028d284715
Disabling tidy has been deprecated since 1.33. This cleans up the code
paths which still used untidy output.
Bug: T198214
Change-Id: I821ef3b8f59b272d983583d407b2f0794fe1e791
This allows them to be more easily reused by other Parser implementations
(ie, Parsoid), and helps keep Parser.php compact.
Bug: T236813
Change-Id: I68fb1e786374e445b7df047934c532d7e10b8e94
The `tidy` option is no longer a "canonical override" (hasn't been for
a while). Replace it with the sole remaining "canonical override",
which is 'enableLimitReport'. Also use `setInterfaceMessage` rather
than `setTidy` when testing, since setting tidy to false has been
deprecated since 1.33.
Change-Id: I416116c25f102a5ee95785e9f2204f2b83d4dace
We formally deprecated running the parser with tidy disabled in 1.33, but
I missed a hard-deprecation on this code path.
Be careful not to spam the logs with deprecation warnings triggered by
(deprecated) API requests, though.
Change-Id: I10f64e76ec0c5aee8b26fc00bc11ebb0e39f961b
This adds methods to ParserOutput ::addExtraCSPStyleSrc,
::addExtraCSPDefaultSrc, and ::addExtraCSPScriptSrc, to easily
allow parser tags/functions to add additional CSP sources if their
tag needs it. Previously such an extension would need to use
and OutputPage hook. This is modeled on how addModules() works.
The immediate use case is for Kartographer (T240960), although
its expected that lots of extensions might do something like this,
especially extensions used outside of Wikimedia.
Change-Id: I24e5f0b4edff58025a0c2a3e1a9aa3f62eb7db7b
HTML5 says id attributes should not have whitespace, where
whitespace is defined as LF, CR, FF, TAB or SPACE (oddly enough
VT does not count). Firefox in my testing actually was fine with
these except CR. Nonetheless we should follow the spec, so this converts
these whitespace characters to _. I don't think this will
cause any back-compat issues, since its very hard to make these
characters in wikitext (other than space which was already
being converted) and basically requires either Lua or html entities
to make these (with FF seeming to be impossible).
Bug: T238385
Depends-On: Ie6fa40798f06a358f6082110b4d8cc0028c80321
Change-Id: Ie2b7c9429691e2c491c3359d5b400d8f078aa789
Currently if you combine a valid percent encoding and a non
escaped character that is reserved in urls in a headline, the toc
link does not work. E.g. ==`%41== needs #`%2541 but we currently
generate #`%41 which matches ==`A== instead.
Tested in firefox and chrome
Bug: T238385
Change-Id: Ice2bbf79bed612d488ed6feb7510035e9dfb33af
1. into class Parser
2. into class LinkHoderArray
3. into class DefaultPreferencesFactory
Add more tests for DefaultPreferencesFactory:
1. testVariantsSupport verifies that converter with variants is used correctly.
2. Test testUserGroupMemberships verifies that membership is used correctly
Bug: T243320, T243321, T243317
Change-Id: I1e5c37e18332d0d32391c74c06e3d84862e48df8
This patch tries to make assertions in tests more readable by using more
self-documenting assertions as provided by modern PHPUnit versions. Among
a few others, these two main changes are done:
* I found a lot of assertions with the expected value being the *second*
parameter. I did not changed all of them. Only some that can be replaced
with assertNull() and such.
* I try to replace all `assertTrue( is_…() )` with dedicated assertions.
Change-Id: I1fc72188fbd0edacf13886e7f9a9eacbd85f13c2
Remove the deprecated Preprocessor_DOM class, which was hard-deprecated
in 1.34. This begins to simplify parser configuration and reduce redundant
code paths, but I've left two things for cleanup in a future patch:
1. The `preprocessorClass` configuration option to the parser, exposed
in `$wgParserConf`, ServiceWiring, ParserFactory, etc. There is no reason
for this to be exposed as configurable, but I've left this clean up to a
future patch.
2. The `$wgMaxGeneratedPPNodeCount` configuration, exposed also in
ParserOptions. Only Preprocessor_DOM calculated this count, and since
we are only using Preprocessor_Hash now, this configuration has no effect.
But since this value was exposed in ParserOptions and elsewhere, I've
deprecated where needed but left this clean up to a future patch.
Bug: T204945
Change-Id: I727f003f9a42d0c92bcbcce8a8289d5af6cd1298
This was done automatically by replacing every assertContains with
string *needle*. Then verifying the results.
Bug: T192167
Change-Id: Id8cbbf3b01e948f80046714183cc299f86be21fd
The following public methods were renamed and made private; the old name
is hard-deprecated and calls the new renamed private method:
Parser::doMagicLinks() => handleMagicLinks()
Parser::doDoubleUnderscore() => handleMagicLinks()
Parser::doHeadings() => handleHeadings()
Parser::doAllQuotes() => handleAllQuotes()
Parser::replaceExternalLinks() => handleExternalLinks()
Parser::replaceInternalLinks() => handleInternalLinks()
Parser::replaceInternalLinks2() => handleInternalLinks2()
Parser::getVariableValue() => expandMagicVariable()
Parser::initialiseVariables() => initializeVariables()
Parser::formatHeadings() => finalizeHeadings()
Parser::test{Pst,Preprocess,Srvus}() => fuzzTest{Pst,Preprocess,Srvus}()
Additionally, the following methods are not used externally, but are
used outside the Parser class by core code. They have been marked
@internal:
Parser::doQuotes() (used by {{#displaytitle}}),
Parser::getExternalLink{Rel,Attribs}() (used by Linker),
Parser::normalizeLinkUrl() (used by Special:LinkSearch and elsewhere).
Parser::{brace,arg,extension}Substitution() (used by PPFrame)
Code search query:
https://codesearch.wmflabs.org/deployed/?q=do%28MagicLinks%7CDoubleUnderscore%7CHeadings%7CAllQuotes%29%7Creplace%28ExternalLinks%7CInternalLinks%28%7C2%29%29%7CgetVariableValue%7CinitialiseVariables%7CformatHeadings%7Ctest%28Pst%7CPreprocess%7CSrvus%29%7CdoQuotes%7CgetExternalLink%28Rel%7CAttribs%29%7CnormalizeLinkUrl%7C%28brace%2Carg%2Cextension%29Substitution&i=nope&files=&repos=
Bug: T236810
Change-Id: I19a43ffc5dcfdd2981b51079c33422c964acb076
Add public, protected or private to function missing a visibility
Enable the tests folder for the phpcs sniff
Change-Id: Ibefce76ea9984c47e08c94889ea2eafca7565e2c
This code didn't work because the $GLOBALS array is exposed by reference.
Once this reference was broken by unset(), the rest just manipulated a
local array that happens to be called "GLOBALS". It must not be unset or
re-assigned. It can only be changed in-place.
Before this, the execution of a MediaWikiUnitTestCase test stored a
copy of GLOBALS in unitGlobals, then lost the GLOBALS pointer and
created a new variable called "GLOBALS". As such, the tearDown() function
didn't do what it meant to do, either – which then results in odd
failures like T230023
Rewrite it as follows:
* In setup, store the current GLOBALS keys and values, then reduce
GLOBALS to only the whitelisted keys and values.
* In teardown, restore the original state.
* As optimisation, do this from setUpBeforeClass as well, so that
there are relatively few globals to reset between tests.
(Thanks @Simetrical!)
The following tests were previously passing by accident under
MediaWikiUnitTestCase but actually did depend on global config.
* MainSlotRoleHandlerTest (…, ContentHandler, $wgContentHandlers)
* SlotRecordTest (…, ContentHandler, $wgContentHandlers)
* WikiReferenceTest (wfParseUrl, $wgUrlProtocols)
* DifferenceEngineSlotDiffRendererTest (DifferenceEngine, wfDebug, …)
* SlotDiffRendererTest (…, ContentHandler, $wgContentHandlers)
* FileBackendDBRepoWrapperTest (wfWikiID, "Backend domain ID not provided")
* JpegMetadataExtractorTest (…, wfDebug, …, LoggerFactory, …)
* ParserFactoryTest (…, wfDebug, …, LoggerFactory, InvalidArgumentException)
* MediaWikiPageNameNormalizerTest (…, wfDebug, …, LoggerFactory, …)
* SiteExporterTest (SiteImporter, wfLogWarning, …)
* SiteImporterTest (Site::newForType, $wgSiteTypes)
* ZipDirectoryReaderTest (…, wfDebug, …, LoggerFactory, …)
Bug: T230023
Change-Id: Ic22075bb5e81b7c2c4c1b8647547aa55306a10a7
This will help make MovePage more testable.
In the course of abstracting the logic out of ParserFactoryTest to
FactoryArgTestTrait so it could be used in MovePageFactoryTest, I made
them all unit tests instead of integration. This required some
modification to the Parser constructor so that it didn't access
MediaWikiServices unnecessarily.
Change-Id: Idaa1633f32dfedfa37516bb9180cfcfbe7ca31aa
And add a test which is confirmed to fail on HHVM prior to this change
with the error message "serialize(): "" returned as member variable from
__sleep() but does not exist".
Bug: T229366
Change-Id: I236bb4d64bc2e9f7756885e8c418399804eac5e1
* Unset globals to avoid tests that look like unit tests but actually rely on
globals
* move some tests out of unit directory so that the test suite will pass.
* Assert that tests which extend MediaWikiUnitTestCase are in a directory with
"/unit/" in its path name
Depends-On: I67b37b1bde94eaa3d4298d9bd98ac57995ce93b9
Depends-On: I90921679518ee95fe393f8b1bbd9134daf0ba032
Bug: T87781
Change-Id: I16691fc8ac063705ba0c2bc63b96c4534ca8660b
Out of 150 tests of SanitizerTest.php, 100 of them are pure unit tests
they are moved to the new file in the new structure, the rest stay
Change-Id: I366d37607abff4bcd624a56fb8b2299729fbc088
This causes Title::getTalkPage and NamespaceInfo::getTitle() to throw
an MWException when called on a LinkTarget that is an interwiki link
or a relative section link. These methods were already throwing
MWException when called on a link to a Special page.
Bug: T224814
Change-Id: I525c186a5b8b8fc22bca195da48afead3bfbd402
This changeset resumes work on T89432 and related tickets
by porting an initial set of tests to the new unit test suite
separated out in I69b92db3e70093570e05cc0a64c7780a278b321a.
The tests were only ported if they worked immediately without
requiring any changes other than changing the test case class
to MediaWikiUnitTestCase and moving the test to the new suite.
If a test failed for any reason (even trivial misconfiguration),
it was NOT ported.
With this change, the unit tests suite now consits of a total
of 455 tests. As before, you can run these tests via the following
command:
$ composer phpunit:unit
Bug: T84948
Bug: T89432
Bug: T87781
Change-Id: Ibb8175981092d7f41864e641cc3c118af70a5c76
These fields are passed to methods like LoadBalancer::getConnection() and are
already expected to be DB domains. Update various comments as well.
Fix a few minor IDEA warnings.
Change-Id: I7cf76700690aec449872a80d30b5ba540d2bf315
These methods should be made private in the next release, but
hard-deprecate them for 1.34.
Tweak the return value of the attribute whitelist to be an
associative rather than a sequential array, which makes the
lookup of allowed attributes more efficient and avoids an
array_flip for every html element sanitized.
Bug: T221677
Change-Id: I17d734937accec6c2679dbe17328cf9554bd556a
The Preprocessor_DOM implementation doesn't interact well with PHP memory
profiling, and has some limitations not present in the Preprocessor_Hash
implementation (see T216664). There is no reason to keep around two
versions of the preprocessor: it just complicates on-going wikitext
feature development.
Hard deprecate use of Preprocessor_DOM, so we can remove the redundant
code in a future release.
Bug: T204945
Depends-On: Id38c9360e4d02b570996dbf7a660f964f02f1a2c
Change-Id: Ica5d1ad5b1e677542962fc36d582a793f941155e
