Commit graph

1081 commits

Author SHA1 Message Date
Peter Ovchyn
61e0908fa2 languages: Introduce LanguageConverterFactory
Done:
* Replace LanguageConverter::newConverter by LanguageConverterFactory::getLanguageConverter
* Remove LanguageConverter::newConverter from all subclasses
* Add LanguageConverterFactory integration tests which covers all languages by their code.
* Caching of LanguageConverters in factory
* Make all tests running (hope that's would be enough)
* Uncomment  the deprecated functions.
* Rename FakeConverter to TrivialLanguageConverter
* Create ILanguageConverter to have shared ancestor
* Make the LanguageConverter class abstract.
* Create table with mapping between lang code and converter instead of using name convention
* ILanguageConverter @internal
* Clean up code

Change-Id: I0e4d77de0f44e18c19956a1ffd69d30e63cf51bf
Bug: T226833, T243332
2020-02-03 11:38:03 +02:00
C. Scott Ananian
a20d4094f8 Deprecate/make private internal callbacks invoked inside Parser.php
Parser::magicLinkCallback() and Parser::makeFreeExternalLink() are not
called from outside the parser and have been made private.

Parser::attributeStripCallback() did have one use outside MediaWiki core,
so it has been deprecated rather than made private.  The one use inside
the parser has been converted to an anonymous callback.

Code search:
https://codesearch.wmflabs.org/search/?q=magicLinkCallback%7CmakeFreeExternalLink%7CattributeStripCallback&i=nope&files=&repos=

Change-Id: Id909b745f9dea1481d09de4747f4aa32648bf85b
2020-01-30 14:00:36 -05:00
C. Scott Ananian
70c4ea9b1a Make private Parser::incrementIncludeSize(), which is not used outside the class
Code search:
https://codesearch.wmflabs.org/search/?q=incrementIncludeSize&i=fosho&files=&repos=

Change-Id: I14da911be23ef2d806c876ebb9ac9cf1fe782e24
2020-01-30 13:08:47 -05:00
C. Scott Ananian
dc4fd76e69 Hard-deprecate Parser::doBlockLevels(), which Parsoid won't support
This is an odd "partial parse" endpoint which won't be supported by
the unified Parsoid parser.  Most places it appears in our codebase
it is just a name-check, mostly complaining about how broken it is.
It is not called by any WMF-deployed code.

Code search:
https://codesearch.wmflabs.org/search/?q=doBlockLevels&i=fosho&files=&repos=

Change-Id: Iad2a1e5ed593c1f6299b8b45e4390b415537c84c
2020-01-30 12:58:13 -05:00
C. Scott Ananian
0fc8f7cf83 Hard-deprecate Parser::fetchTemplate()
This is a follow up to I6f7c3e31ff394781fbaaef267a2772de05f85bf2, which
deprecated Parser::fetchTemplate but forgot to include the wfDeprecated()
call.

Code search:
https://codesearch.wmflabs.org/search/?q=fetchTemplate%5C%28&i=nope&files=&repos=

Change-Id: I912b538e6e77c35498308c168316803ddde1f137
2020-01-30 12:50:19 -05:00
C. Scott Ananian
25f3b8ad68 Deprecate Parser::fetchTemplate() in favor of ::fetchTemplateAndTitle()
This reduces the API surface of Parser, in preparation for further
refactoring of the Parser class.  Parser::fetchTemplateAndTitle() was
introduced in MW 1.6.0 in 2007, so is always appropriate as a replacement.

Code search:
https://codesearch.wmflabs.org/search/?q=fetchTemplate%5C%28&i=nope&files=&repos=

Depends-On: I2865b12b55aa1412a861fb8db10239018a1d1580
Change-Id: I6f7c3e31ff394781fbaaef267a2772de05f85bf2
2020-01-28 20:00:54 -05:00
C. Scott Ananian
a8f3eb96ce Hard deprecate Parser::setFunctionTagHook()
This is not used in production code.

Code search:
https://codesearch.wmflabs.org/search/?q=setFunctionTagHook%7CmFunctionTagHooks&i=nope&files=&repos=

Change-Id: Ie03b1687d1c65be23213035d7cb4011ef763b69a
2020-01-28 16:20:40 -08:00
C. Scott Ananian
9f0adbdb81 Remove Parser::setTransparentTagHook()
The transparent tag hooks are not used anywhere in public git.

Codesearch link:
https://codesearch.wmflabs.org/search/?q=setTransparentTagHook%7CreplaceTransparentTags%7CmTransparentTagHooks&i=nope&files=&repos=

Depends-On: I0df0ca2fc7ed05dc6dc413bd0a6d38533b66e428
Change-Id: I8384f47bbcde7604efa41b83db0dbc4f429219a8
2020-01-28 16:20:40 -08:00
jenkins-bot
eaebf057f3 Merge "Remove $wgMaxGeneratedPPNodeCount" 2020-01-29 00:14:55 +00:00
C. Scott Ananian
e149a38d4f Remove $wgMaxGeneratedPPNodeCount
This no longer has any effect.  The getter and setter in ParserOptions
aren't used in any public git code, and so have been removed without
deprecation.

Bug: T204945
Followup-To: I727f003f9a42d0c92bcbcce8a8289d5af6cd1298
Change-Id: Id48effcba48d1ae1621a4e17a70e65b73f2473b7
2020-01-28 16:54:53 -05:00
jenkins-bot
5e462e5efe Merge "Remove all traces of preprocessorClass from $wgParserConf" 2020-01-26 21:48:23 +00:00
jenkins-bot
f9a08c0c83 Merge "Remove Preprocessor_DOM, deprecated in 1.34" 2020-01-26 02:02:44 +00:00
C. Scott Ananian
0eb9f18d8d Remove all traces of preprocessorClass from $wgParserConf
Bug: T204945
Followup-To: I727f003f9a42d0c92bcbcce8a8289d5af6cd1298
Change-Id: I23821686ad666728d7d196e089f40be4b76b7390
2020-01-25 13:15:16 -05:00
C. Scott Ananian
5cbb64f56a Remove Preprocessor_DOM, deprecated in 1.34
Remove the deprecated Preprocessor_DOM class, which was hard-deprecated
in 1.34.  This begins to simplify parser configuration and reduce redundant
code paths, but I've left two things for cleanup in a future patch:

1. The `preprocessorClass` configuration option to the parser, exposed
in `$wgParserConf`, ServiceWiring, ParserFactory, etc.  There is no reason
for this to be exposed as configurable, but I've left this clean up to a
future patch.

2. The `$wgMaxGeneratedPPNodeCount` configuration, exposed also in
ParserOptions.  Only Preprocessor_DOM calculated this count, and since
we are only using Preprocessor_Hash now, this configuration has no effect.
But since this value was exposed in ParserOptions and elsewhere, I've
deprecated where needed but left this clean up to a future patch.

Bug: T204945
Change-Id: I727f003f9a42d0c92bcbcce8a8289d5af6cd1298
2020-01-25 11:22:45 -05:00
C. Scott Ananian
b01642a27c Mark @internal a number of Parser methods not used outside MW core
These should not be considered a part of the public MW API.

Note that interwikiTransclude is used in maintenance scripts in the
Math extension, but that seems to be a reasonable use of an @internal
method.

Code search link:
https://codesearch.wmflabs.org/deployed/?q=limitationWarn%7CisCurrentRevisionOfTitleCached%7CinterwikiTransclude%7CargSubstitution%7CrenderImageGallery%7CgetFlatSectionInfo%7CmarkerSkipCallback%7CparseWidthParam&i=nope&files=&repos=

Change-Id: I50e52fd57ba3b5ad7cc7f2b21ec772374268179f
2020-01-25 16:12:04 +00:00
Thiemo Kreuz
ea01e7cfbb parser: Remove unused ParserAfterUnstrip hook
Introduced in 2015 via If83b062. Unfortunately the patch fails to mention
why the hook was introduced. A review comment asking for the missing
information was ignored.

The hook was introduced exclusively for the Cite extension, see I0e136f9.
The Cite change got reverted a few days later via I1a71854. As far as I
can tell the hook (oddly specific and underspecified anyway) was never
used again, neither for Cite nor for anything else.

Now unused: https://codesearch.wmflabs.org/search/?q=ParserAfterUnstrip

I suggest to just remove it with no deprecation phase.

My team is currently working on Cite, essentially turning it upside down.
I'm not aware of a need for this hook.

Change-Id: If68a88d7afff60385ff1e33fd13716b20d01a313
2020-01-25 09:20:52 +01:00
James D. Forrester
0958a0bce4 Coding style: Auto-fix MediaWiki.Usage.IsNull.IsNull
Change-Id: I90cfe8366c0245c9c67e598d17800684897a4e27
2020-01-10 14:17:13 -08:00
James D. Forrester
4f2d1efdda Coding style: Auto-fix MediaWiki.Classes.UnsortedUseStatements.UnsortedUse
Change-Id: I94a0ae83c65e8ee419bbd1ae1e86ab21ed4d8210
2020-01-10 09:32:25 -08:00
Aaron Schulz
6de44014de parser: further restrict "revisionid" by namespace in $wgMiserMode
Follows-up c537eb1868.

The restrictions used to only apply to "content" namespaces, which
is typically just NS_MAIN. Wikis like Commons and Wikidata use custom
values, NS_FILE and $LexemeNamespace, respectively.

Now, only yield the integer ID for user, project, and talk pages.
This means that NS_FILE, NS_CATEGORY, and other namespaces with
content intended for readers, only yield '' or '-' for this work.

Bug: T235957
Change-Id: Id0110c9ee91ad5750164e9a047bcd6d7621930c1
2020-01-06 11:16:51 -08:00
Kunal Mehta
99007e96c7 Use namespaced IPUtils class
Change-Id: I047e099a93203a59093946d336a143d899d0271f
2020-01-01 02:36:49 -08:00
Daimona Eaytoy
598c4d7fcb build: Upgrade phan to 0.9.0
Scalar casts are still allowed (for now), because there's a huge amount
of false positives. Ditto for invalid array offsets.

Thoughts about the rest: luckily, many false positives with array offsets
have gone. Moreover, since *Internal issues are suppressed in the base
config, we can remove inline suppressions.

Unfortunately, there are a couple of new issues about array additions
with only false positives, because apparently they don't take
branches into account.

Change-Id: I5a3913c6e762f77bfdae55051a395fae95d1f841
2019-12-07 20:16:19 +00:00
jenkins-bot
097ad02261 Merge "REST compare: bundle section offset information" 2019-11-13 01:18:01 +00:00
Thiemo Kreuz
cf42131019 Streamline/fix code by using short ?: and ?? syntax
Change-Id: I27dc0c8a1a184571ed96abdea22af953510457cd
2019-11-11 11:17:13 +01:00
Tim Starling
8299d7ae8f REST compare: bundle section offset information
Bug: T236964
Change-Id: Ib97b0fa0de682404fb9df2ecef55a3526c54a31f
2019-11-11 16:25:35 +11:00
James D. Forrester
8255ecd074 Parser: Drop ::disableCache(), deprecated in 1.28
Change-Id: Ide0b28f821c55fb008e50c199032f1c34334ac1d
2019-11-05 17:07:00 -05:00
jenkins-bot
ee2751a70b Merge "Minor cleanups" 2019-11-05 10:46:25 +00:00
C. Scott Ananian
8dbc866e91 Hard deprecate Parser::disableCache()
Among deployed extensions, only used in the Quiz extension.

Full list of uses (some false positives):
https://codesearch.wmflabs.org/search/?q=disableCache&i=nope&files=&repos=

Depends-On: I956a88120d07d76d1afa9d06e95d31055f9b07f1
Change-Id: I6419754de6cbc01af07b6c0eafb8396bd720a58d
2019-11-04 21:22:11 +00:00
C. Scott Ananian
3397bd43e7 Parser: remove/make private historically-public methods
These were hard-deprecated in 1.34.  This continues work to clean up
the Parser API.

Bug: T236810
Change-Id: I0b2258a63d73370fd770612a6ca437509676d71c
2019-11-04 15:43:20 -05:00
C. Scott Ananian
cff5c92cb9 Parser: Add Title type hints
This reverts commit c16b0a533e0d7350bceca7ccbe490efa3e3ce4c4, now
that an appropriate period of time has passed since this behavior
was deprecated.

Bug: T235392
Change-Id: I076ae0dbfbbb8be731367e4641f9c8aacf7586e9
2019-11-04 14:08:26 -05:00
Max Semenik
f1c9cf8879 Minor cleanups
* Identifier case
* Returning a void function result
* Unused variable
* Missing documentation

Change-Id: Ibfd2fc5ae1d91c7c9c6a34bcd4523384d3bca576
2019-11-03 17:10:23 -08:00
Tim Starling
68d87dba9f Remove Parser::serializeHalfParsedText() and helpers
It was deprecated in 1.31, It was a failed experiment, intended to
improve performance, but it never worked and makes the code
much more complex. I also removed two LinkHolderArray helpers which were
accidentally not marked deprecated -- codesearch confirms that they are
not called by anything.

Change-Id: I74754265dde388b590f0b4c995cb4a35a1318299
2019-11-01 15:22:29 +11:00
jenkins-bot
74a646c0f2 Merge "Parser: Remove ::getConverterLanguage(), deprecated in 1.32" 2019-11-01 01:55:28 +00:00
Fomafix
14b853672d Parser: Remove ::getConverterLanguage(), deprecated in 1.32
Use Parser::getTargetLanguage() instead.

Change-Id: Id1075b91f84d2c1766754410476aafb7c9144604
2019-11-01 01:43:15 +00:00
C. Scott Ananian
023bb7677f Deprecate Parser::replaceLinkHolders / replaceLinkHoldersText
Because this method is used in the ImageMap extension, we can't
immediately hard-deprecate replaceLinkHolders.

Code search:
https://codesearch.wmflabs.org/deployed/?q=replaceLinkHolders&i=nope&files=&repos=

Bug: T236810
Change-Id: If02130ffc86d0d9db2a455efcd29641a8206f0a0
2019-10-29 16:53:35 -04:00
C. Scott Ananian
736b6b27f3 Deprecate additional public methods of Parser
Affected:
Parser::armorLinks()
Parser::makeKnownLinkHolder()
Parser::getImageParams()
Parser::parseLinkParameter()
Parser::stripAltText()

It wasn't obvious there was a better name for these, so just rename
them with a `...Private` suffix and deprecate the old names.  When
the deprecated public methods are removed we'll rename the private
methods to remove the `...Private` suffix again.

Code search:
https://codesearch.wmflabs.org/deployed/?q=armorLinks%7CmakeKnownLinkHolder%7CgetImageParams%7CparseLinkParameter%7CstripAltText&i=nope&files=&repos=

Bug: T236810
Change-Id: I44458490fa86abd0ead048a4c94021da6be333f0
2019-10-29 16:53:35 -04:00
C. Scott Ananian
d7cd12e81f Deprecate Parser::splitWhitespace() / Parser::createAssocArgs()
These methods are not used anywhere in deployed code.

Parser::createAssocArgs() is used in Extension:DataTable2, but it should
just copy the implementation from Parser, it's very little code.

Code search:
https://codesearch.wmflabs.org/deployed/?q=splitWhitespace%7CcreateAssocArgs&i=nope&files=&repos=

Bug: T236810
Change-Id: I4e6a39e2ecf7a3a568e26ad1d8ce1166a44a5ad9
2019-10-29 16:52:22 -04:00
C. Scott Ananian
dcae22c8fa Deprecate Parser::areSubpagesAllowed() / Parser::maybeDoSubpageLink()
These are unused outside the Parser and are so short it's not worth
renaming them to make them private; just hard-deprecate the methods
and inline the implementation in the small # of places they appear.

Code search:
https://codesearch.wmflabs.org/deployed/?q=areSubpagesAllowed%7CmaybeDoSubpageLink&i=nope&files=&repos=

Bug: T236810
Change-Id: Ia06c65409a3158b083bcc59c9f6e347945b375c0
2019-10-29 16:50:21 -04:00
C. Scott Ananian
045ec0a408 Deprecate Parser implementation methods (will be private in next release)
The following public methods were renamed and made private; the old name
is hard-deprecated and calls the new renamed private method:

Parser::doMagicLinks() => handleMagicLinks()
Parser::doDoubleUnderscore() => handleMagicLinks()
Parser::doHeadings() => handleHeadings()
Parser::doAllQuotes() => handleAllQuotes()
Parser::replaceExternalLinks() => handleExternalLinks()
Parser::replaceInternalLinks() => handleInternalLinks()
Parser::replaceInternalLinks2() => handleInternalLinks2()
Parser::getVariableValue() => expandMagicVariable()
Parser::initialiseVariables() => initializeVariables()
Parser::formatHeadings() => finalizeHeadings()
Parser::test{Pst,Preprocess,Srvus}() => fuzzTest{Pst,Preprocess,Srvus}()

Additionally, the following methods are not used externally, but are
used outside the Parser class by core code.  They have been marked
@internal:

Parser::doQuotes() (used by {{#displaytitle}}),
Parser::getExternalLink{Rel,Attribs}() (used by Linker),
Parser::normalizeLinkUrl() (used by Special:LinkSearch and elsewhere).
Parser::{brace,arg,extension}Substitution() (used by PPFrame)

Code search query:
https://codesearch.wmflabs.org/deployed/?q=do%28MagicLinks%7CDoubleUnderscore%7CHeadings%7CAllQuotes%29%7Creplace%28ExternalLinks%7CInternalLinks%28%7C2%29%29%7CgetVariableValue%7CinitialiseVariables%7CformatHeadings%7Ctest%28Pst%7CPreprocess%7CSrvus%29%7CdoQuotes%7CgetExternalLink%28Rel%7CAttribs%29%7CnormalizeLinkUrl%7C%28brace%2Carg%2Cextension%29Substitution&i=nope&files=&repos=

Bug: T236810
Change-Id: I19a43ffc5dcfdd2981b51079c33422c964acb076
2019-10-29 14:00:29 -04:00
jenkins-bot
7a54595d10 Merge "Parser: Hard deprecate getConverterLanguage" 2019-10-28 08:48:44 +00:00
Fomafix
9bae9db4cc Parser: Hard deprecate getConverterLanguage
getConverterLanguage is deprecated since MediaWiki 1.32.
getConverterLanguage always return a Language object and never null.

Change-Id: Ia0480c76416ef1e925619d9e85c7134c2ecf2296
Depends-On: Iea4771161d129c49f6482e1a6822e1324bf2fb49
2019-10-27 19:58:41 +00:00
jenkins-bot
a10deed7cb Merge "Parser.php: Export an experimental convenience method for Parsoid/PHP" 2019-10-18 20:48:56 +00:00
C. Scott Ananian
dd9e6124b4 Deprecate setting Parser::mTitle to null
This never happens in core code; however extensions have slipped into
a state of sin.

Bug: T235392
Change-Id: Ia254949cd8b3bc162b11dcc911dcce40d91bf1b7
2019-10-18 15:46:53 -04:00
Subramanya Sastry
c90e20f1cb Parser.php: Export an experimental convenience method for Parsoid/PHP
* This ensures we run all necessary hooks that extensions expect
  will be run for their content.

* We will rip out this method once we decide / discover that we don't
  need it anymore.

Change-Id: I683bd6c3cc2c3f4d65326adab78cdf70ae39a51a
2019-10-18 11:30:39 -05:00
Fomafix
d91a136ae7 Revert "Parser: Add Title type hints"
This change reverts most of commit 3dff713fe4.

Especially the return type hints for getTitle() and Title() are reduced
to allow the type null as return value, because SematicMediaWiki uses
this by

	$this->parser->getTitle() instanceof Title

to check if there is a valid Title object.

The parameter type hints for setTitle() and Title() are kept.

Bug: T235392
Change-Id: I72ac1c9d37059876dbc7cd38158e7abd212da8fe
2019-10-18 08:28:49 +02:00
Daimona Eaytoy
e3412efac3 Unsuppress PhanParamReqAfterOpt, use PHP71 nullable types
These were all checked with codesearch to ensure nothing is overriding
these methods.
For the most part, I've updated the signature to use nullable types; for
two Pager's, I've just made all parameters non-optional, because you're
already forced to pass them with a required parameter at the end.

Bug: T231636
Change-Id: Ie047891f55fcd322039194cfa9a8549e4f1f6f14
2019-10-10 11:53:58 +02:00
James D. Forrester
74f8ae0a24 [BREAKING CHANGE] Services: Convert Parser's static to a const now HHVM is gone
Change-Id: I54eafc84694bf68fe791b482865a1763c0439cbf
2019-10-09 12:26:48 -07:00
Fomafix
3dff713fe4 Parser: Add Title type hints
getTitle() and Title() will never return NULL anymore. When mTitle is
not set then a TypeError be thrown at runtime. The explicit local
checks for is_null( $this->mTitle ) are not needed anymore when
$this-getTitle() instead of $this->mTitle is used.

Change-Id: I09a178e5c6938954edb2949f13660227d6a01fbc
2019-09-26 16:34:12 +02:00
jenkins-bot
c96b8354fc Merge "Parser: Use 'Special:Badtitle/Parser' instead of 'NO TITLE'" 2019-09-26 14:15:53 +00:00
Fomafix
a6e8182b9b Parser: Use 'Special:Badtitle/Parser' instead of 'NO TITLE'
Change-Id: I818ac7d4766c3e52b56c2bd4abddb88e4140003b
2019-09-22 21:35:15 +02:00
Daimona Eaytoy
b138a9b228 Fix using null for a non-nullable argument
These are reported by phan as PhanTypeMismatchArgumentNullableInternal
when null_casts_as_any_type is disabled.

Change-Id: I85076ee31c1bfc59a19600e84da0d915e425890a
2019-09-19 16:55:03 +00:00