This does not move the actual limit report data into
ParserOptions yet, that should be done separately
given that it will require serialization changes.
Let's get this change settled first before messing
with serialization.
This unifies canonical and non-canonical ParserOptions,
so ParserCache can now be used with both. It is hard
to say how this will affect the ParserCache capacity,
so we should monitor it after releasing this.
Change-Id: I154c0a77a5b0287b5572614d56339fb57ac56c33
We moved the ToC insertion from the parser to ParserOutput::getText()
in T287767 but forgot to ensure that the ToC contents are properly
language converted -- this happens *after* the call to
ParserOutput::setTOCHTML() in the old Parser code.
This is a quick and dirty fix, which does the language conversion
but probably misses a few corner cases of the original behavior
(marked by XXX comment). For example, it doesn't disable language
conversion on interface messages -- but there shouldn't be any
ToC on interface messages. Not heeding __NOCONTENTCONVERT__
in the article is a legit problem, but probably not as bad as
the UBN regression we're fixing. We'll clean this up in
a followup (T295209), but it will involve passing some additional
information from the Parser to ParserOutput which won't be
present in "old" parser cache entries anyway.
This is an UBN and this patch is the quickest way to ensure that
existing parser cache content renders correctly. It's
preferable to the alternative
(Iffcff96fd9b4749794ac78414c1801979a652792) which handles all the
corner cases but can't fix up existing parser cache content,
which has "always" been stored without language conversion.
Bug: T295187
Change-Id: Ic14b3a49a8ee7ed600485d4f8a363a206035a847
* Do not store table of contents in parser output
* Instead inject table of contents via strpos where needed
inside Article based on Skin "toc" option
* Use <mw:tocplace> as a TOC placeholder; for Parsoid compatibility
this will be replaced with a <meta> tag in a followup patch.
Bug: T287767
Change-Id: I44045b3b9e78e7ab793da3f37e3c0dbc91cd7d39
Encourage localization and factor out common code by taking a message
key as the first argument to ::addWarningMsg() instead of a wikitext
string. This also plays nicer with Parsoid by separating out the
localization code from the parse.
Bug: T293515
Change-Id: I6a7c04c67ac586ab00d4edcbb3d09485a7794e23
This is a uniform mechanism to access a number of bespoke boolean
flags in ParserOutput. It allows extensibility in core (by adding new
field names to ParserOutputFlags) without exposing new getter/setter
methods to Parsoid. It replaces the ParserOutput::{get,set}Flag()
interface which (a) doesn't allow access to certain flags, and (b) is
typically called with a string rather than a constant, and (c) has a
very generic name. (Note that Parser::setOutputFlag() already called
these "output flags".)
In the future we might unify the representation so that we store
everything in $mFlags and don't have explicit properties in
ParserOutput, but those representation details should be invisible to
the clients of this API. (We might also use a proper enumeration
for ParserOutputFlags, when PHP supports this.)
There is some overlap with ParserOutput::{get,set}ExtensionData(), but
I've left those methods as-is because (a) they allow for non-boolean
data, unlike the *Flag() methods, and (b) it seems worthwhile to
distingush properties set by extensions from properties used by core.
Code search:
https://codesearch.wmcloud.org/search/?q=%5BOo%5Dut%28put%29%3F%28%5C%28%5C%29%29%3F-%3E%28g%7Cs%29etFlag%5C%28&i=nope&files=&excludeFiles=&repos=
Bug: T292868
Change-Id: I39bc58d207836df6f328c54be9e3330719cebbeb
This function, added in 1.34, does not appear to be used outside core:
https://codesearch.wmcloud.org/search/?q=-%3EgetAllFlags\%28\%29&i=nope&files=&excludeFiles=&repos=
It currently exposes representation details of ParserOutput (that is,
whether a boolean flag is stored in an explicit property, in the
$mFlags array, or in the $mExtensionData array) and so it would be
best not to expose it outside core so as to facilitate any future
change in the internal representation of ParserOutput.
Bug: T292868
Change-Id: I7b6d309425ff01dc211334b848068d0b9c0f9261
This moves the implementation of ParserOutput::addTrackingCategory()
to the TrackingCategories class as a non-static method. This makes
invocation from ParserOutput awkward, but when invoking as
Parser::addTrackingCategory() all the necessary services are
available. As a result, we've also soft-deprecated
ParserOutput::addTrackingCategory(); new users should use the
TrackingCategories::addTrackingCategory() method, or else
Parser::addTrackingCategory() if the parser object is available.
The Parser class is already kind of bloated as it is (alas), but there
aren't too many callsites which invoke
ParserOutput::addTrackingCategory() and don't have the corresponding
Parser object handy; see:
https://codesearch.wmcloud.org/search/?q=%5BOo%5Dutput%28%5C%28%5C%29%29%3F-%3EaddTrackingCategory%5C%28&i=nope&files=&excludeFiles=&repos=
Change-Id: I697ce188a912e445a6a748121575548e79aabac6
Make ::setCategory() consistent with the corresponding singular method,
which is ::addCategory(), not ::addCategoryLink(). Also, don't return
a value.
This renaming is in preparation for factoring out a write-only base
class from ParserOutput suitable to be used by Parsoid.
Note that OutputPage does distinguish a 'category link' from a
'category list', and there are separate OutputPage::getCategories()
and OutputPage::getCategoryLinks() methods. However, the category
map in ParserOutput isn't exactly the same as either of these:
it's actually a map (or list of pairs) of category name to sort key.
Rename ParserOutput::getCategoryLinks() to ::getCategoryNames()
in order to clarify that the concept involved is not the same as
the OutputPage "category links" methods.
Code search:
https://codesearch.wmcloud.org/deployed/?q=-%3E(get%7Cset)CategoryLinks%5C(&i=nope&files=&excludeFiles=&repos=
(Note that many of the code search matches are for the methods in
OutputPage, which we are trying to disambiguate here.)
Bug: T287216
Change-Id: Idb383d3d9ef7b76f8a0208a057a3cb8c639465c9
Usages of Title in the public interface of ParsrOutput are being
replaced with LinkTarget or PageReference, as appropriate.
Change-Id: I571cf18fd7e666a59ef3947b09c2e75357bcac04
- Added a test where ParserOutput objects with CacheTime
properties set are unserialized from previous versions.
- Generate new serialization tests for 1.38
Now all serialization in production is JSON, so changing
property visibility shouldn't affect ParserCache.
Bug: T263851
Depends-On: I283340ff559420ceee8f286ba3ef202c01206a23
Change-Id: I70d6feb1c995a0a0f763b21261141ae8ee6dc570
Follow up to the soft deprecation in I9822c60c180d204bd30cb4447a1120155d456da4.
Code search:
https://codesearch.wmcloud.org/deployed/?q=%28allow%7Cprevent%29Clickjacking&i=nope&files=&excludeFiles=&repos=
Note that the dependencies here are actually cleanups to
OutputPage::preventClickjacking(), not ParserOutput::preventClickjacking().
It can be hard to tell the difference between these two methods when
reading code (especially when parameter types are omitted) and so we're
being extra cautious by cleaning up both of these together.
Bug: T287216
Depends-On: I626e89a1bff2d9c535b828a5b25eff863c91f858
Depends-On: I23e8b35540d2da34c0a3d335069c518b4b72a333
Depends-On: I2538ac8f37fcde183f496e49a8de0ed25a38e508
Change-Id: I0f9b1fe755f7d3d6a9b18fa21a738dda7566c211
The ::getProperty() naming is too generic and doesn't clearly indicate
that these are "page properties" (which have their own table in the DB).
As part of refactoring a clean API out of ParserOutput which can be used
by Parsoid, clean up the naming here.
Soft-deprecation in this patch, there are a handful of external users
which need to be cleaned up before we hard-deprecate.
Bug: T287216
Change-Id: Ie963eea5aa0f0e984ced7c4dfa0fd65d57313cfa
The value in the attribute displaytitle must contain valid HTML. The
sanitizer of the {{DISPLAYTITLE}} parser ensures that only valid HTML
is accepted.
If there is no {{DISPLAYTITLE}} in the wikitext then displaytitle
falls back to $title->getPrefixedText(). Here an HTML encoding of
special characters is necessary. This affects only the replacement of
& by & because other special characters like < and > are not
allowed in the title.
This change affects the displaytitle fallback on the following places:
* ApiParse
* ApiQueryInfo
* InfoAction
* Parser
The displaytitle fallback in OutputPage is also updated to this
behavior although
Sanitizer::normalizeCharReferences( Sanitizer::removeHTMLtags( $html )
also replaces & by &.
Also add test cases with & in the displaytitle to:
* ApiParseTest
* ApiQueryInfoTest
* parserTests
Bug: T291985
Change-Id: I8ee1e2731d9bfa49725d663b34986e7e3073e4ca
Because in PHP is "0" == false.
Also
* Combine $this->mOutput->setTitleText calls.
* Avoid inverted logic. Use
if ( !A && !B && !C && D )
instead of
if ( !( A || B || C || !D ) )
* Document false as possible return value.
Change-Id: I92c343b74a9b313b10a2c9b31717a3727aed4cde
This name is consist with the rest of the setter and getter methods
in ParserOutput. Renamed the methods in OutputPage, ImageHistoryList,
ImageHistoryPseudoPager, and ContribsPager as well for consistency;
it also makes chasing down lingering references in codesearch easier.
Soft-deprecated the old name for 1.38. Hard-deprecation will follow,
but there are a number of users in production that should be chased
down first.
Code search:
https://codesearch.https://codesearch.wmcloud.org/deployed/?q=(allow%7Cprevent)Clickjacking&i=nope&files=&excludeFiles=&repos=
Bug: T287216
Change-Id: I9822c60c180d204bd30cb4447a1120155d456da4
This method is used only twice outside this class: in ParserOutputTest
(which can use the standard backdoor to access private members) and
in the advancedbacklinks extension (which should just cut-and-paste
the code, it's quite small and has no dependencies).
Code search:
https://codesearch.wmcloud.org/search/?q=isLinkInternal&i=nope&files=&excludeFiles=&repos=
Bug: T287216
Change-Id: Iddc7f7b691be2cb97a5dba60de7037e2b12206dc
Update/Create override classes of ContentHandler.
Soft-deprecate and remove method from Content and classes that override them.
Bug: T287158
Change-Id: Idfcfbfe1a196cd69a04ca357281d08bb3d097ce2
$wgShellLocale was a flawed solution to the problem of locale
dependence. MediaWiki has its own concept of locale (the Language
hierarchy) and any kind of dependence on the server's libc locale is
incorrect and harmful, leading to bugs. Developers have an expectation
that functions like strtolower() will work in a certain way, and
respecting the locale set in the environment at install time violates
this expectation.
The problems with using C as a locale, which led to $wgShellLocale, are:
* escapeshellarg() will strip non-ASCII characters. This can be worked
around by not using it. The security vulnerability it was trying to
fix can be prevented in another way.
* Shell commands like rsvg will fail to correctly interpret UTF-8
arguments. This is the reason for the putenv(). On Linux, this can
be fixed by using C.UTF-8, which we didn't know at the time. On
Windows, the problem is not relevant (there are unrelated issues
with UTF-8 arguments).
Bug: T291234
Change-Id: Ib5ac0e7bc720dcc094303a358ee1c7bbdcfc6447
