This method was renamed in 1.37 to ::hasReducedExpiry() and only has a
single use in core (and no uses outside of core, as far as I can
tell). Rename the single use and deprecate the old name.
Code search:
https://codesearch.wmcloud.org/search/?q=hasDynamicContent&i=nope
Change-Id: Ie2bea78e31433a01a5590becc06f32294b04522e
This reverts commit 2bcb3fe567.
Reason for revert: this is a good change,
just needed more work to not break CI
Change-Id: I23768bee242e3cf81b1493a740cf070e7ad1e224
This does not move the actual limit report data into
ParserOptions yet, that should be done separately
given that it will require serialization changes.
Let's get this change settled first before messing
with serialization.
This unifies canonical and non-canonical ParserOptions,
so ParserCache can now be used with both. It is hard
to say how this will affect the ParserCache capacity,
so we should monitor it after releasing this.
Change-Id: I154c0a77a5b0287b5572614d56339fb57ac56c33
We moved the ToC insertion from the parser to ParserOutput::getText()
in T287767 but forgot to ensure that the ToC contents are properly
language converted -- this happens *after* the call to
ParserOutput::setTOCHTML() in the old Parser code.
This is a quick and dirty fix, which does the language conversion
but probably misses a few corner cases of the original behavior
(marked by XXX comment). For example, it doesn't disable language
conversion on interface messages -- but there shouldn't be any
ToC on interface messages. Not heeding __NOCONTENTCONVERT__
in the article is a legit problem, but probably not as bad as
the UBN regression we're fixing. We'll clean this up in
a followup (T295209), but it will involve passing some additional
information from the Parser to ParserOutput which won't be
present in "old" parser cache entries anyway.
This is an UBN and this patch is the quickest way to ensure that
existing parser cache content renders correctly. It's
preferable to the alternative
(Iffcff96fd9b4749794ac78414c1801979a652792) which handles all the
corner cases but can't fix up existing parser cache content,
which has "always" been stored without language conversion.
Bug: T295187
Change-Id: Ic14b3a49a8ee7ed600485d4f8a363a206035a847
* Do not store table of contents in parser output
* Instead inject table of contents via strpos where needed
inside Article based on Skin "toc" option
* Use <mw:tocplace> as a TOC placeholder; for Parsoid compatibility
this will be replaced with a <meta> tag in a followup patch.
Bug: T287767
Change-Id: I44045b3b9e78e7ab793da3f37e3c0dbc91cd7d39
Encourage localization and factor out common code by taking a message
key as the first argument to ::addWarningMsg() instead of a wikitext
string. This also plays nicer with Parsoid by separating out the
localization code from the parse.
Bug: T293515
Change-Id: I6a7c04c67ac586ab00d4edcbb3d09485a7794e23
This is a uniform mechanism to access a number of bespoke boolean
flags in ParserOutput. It allows extensibility in core (by adding new
field names to ParserOutputFlags) without exposing new getter/setter
methods to Parsoid. It replaces the ParserOutput::{get,set}Flag()
interface which (a) doesn't allow access to certain flags, and (b) is
typically called with a string rather than a constant, and (c) has a
very generic name. (Note that Parser::setOutputFlag() already called
these "output flags".)
In the future we might unify the representation so that we store
everything in $mFlags and don't have explicit properties in
ParserOutput, but those representation details should be invisible to
the clients of this API. (We might also use a proper enumeration
for ParserOutputFlags, when PHP supports this.)
There is some overlap with ParserOutput::{get,set}ExtensionData(), but
I've left those methods as-is because (a) they allow for non-boolean
data, unlike the *Flag() methods, and (b) it seems worthwhile to
distingush properties set by extensions from properties used by core.
Code search:
https://codesearch.wmcloud.org/search/?q=%5BOo%5Dut%28put%29%3F%28%5C%28%5C%29%29%3F-%3E%28g%7Cs%29etFlag%5C%28&i=nope&files=&excludeFiles=&repos=
Bug: T292868
Change-Id: I39bc58d207836df6f328c54be9e3330719cebbeb
This function, added in 1.34, does not appear to be used outside core:
https://codesearch.wmcloud.org/search/?q=-%3EgetAllFlags\%28\%29&i=nope&files=&excludeFiles=&repos=
It currently exposes representation details of ParserOutput (that is,
whether a boolean flag is stored in an explicit property, in the
$mFlags array, or in the $mExtensionData array) and so it would be
best not to expose it outside core so as to facilitate any future
change in the internal representation of ParserOutput.
Bug: T292868
Change-Id: I7b6d309425ff01dc211334b848068d0b9c0f9261
This moves the implementation of ParserOutput::addTrackingCategory()
to the TrackingCategories class as a non-static method. This makes
invocation from ParserOutput awkward, but when invoking as
Parser::addTrackingCategory() all the necessary services are
available. As a result, we've also soft-deprecated
ParserOutput::addTrackingCategory(); new users should use the
TrackingCategories::addTrackingCategory() method, or else
Parser::addTrackingCategory() if the parser object is available.
The Parser class is already kind of bloated as it is (alas), but there
aren't too many callsites which invoke
ParserOutput::addTrackingCategory() and don't have the corresponding
Parser object handy; see:
https://codesearch.wmcloud.org/search/?q=%5BOo%5Dutput%28%5C%28%5C%29%29%3F-%3EaddTrackingCategory%5C%28&i=nope&files=&excludeFiles=&repos=
Change-Id: I697ce188a912e445a6a748121575548e79aabac6
Make ::setCategory() consistent with the corresponding singular method,
which is ::addCategory(), not ::addCategoryLink(). Also, don't return
a value.
This renaming is in preparation for factoring out a write-only base
class from ParserOutput suitable to be used by Parsoid.
Note that OutputPage does distinguish a 'category link' from a
'category list', and there are separate OutputPage::getCategories()
and OutputPage::getCategoryLinks() methods. However, the category
map in ParserOutput isn't exactly the same as either of these:
it's actually a map (or list of pairs) of category name to sort key.
Rename ParserOutput::getCategoryLinks() to ::getCategoryNames()
in order to clarify that the concept involved is not the same as
the OutputPage "category links" methods.
Code search:
https://codesearch.wmcloud.org/deployed/?q=-%3E(get%7Cset)CategoryLinks%5C(&i=nope&files=&excludeFiles=&repos=
(Note that many of the code search matches are for the methods in
OutputPage, which we are trying to disambiguate here.)
Bug: T287216
Change-Id: Idb383d3d9ef7b76f8a0208a057a3cb8c639465c9
Usages of Title in the public interface of ParsrOutput are being
replaced with LinkTarget or PageReference, as appropriate.
Change-Id: I571cf18fd7e666a59ef3947b09c2e75357bcac04
- Added a test where ParserOutput objects with CacheTime
properties set are unserialized from previous versions.
- Generate new serialization tests for 1.38
Now all serialization in production is JSON, so changing
property visibility shouldn't affect ParserCache.
Bug: T263851
Depends-On: I283340ff559420ceee8f286ba3ef202c01206a23
Change-Id: I70d6feb1c995a0a0f763b21261141ae8ee6dc570
Follow up to the soft deprecation in I9822c60c180d204bd30cb4447a1120155d456da4.
Code search:
https://codesearch.wmcloud.org/deployed/?q=%28allow%7Cprevent%29Clickjacking&i=nope&files=&excludeFiles=&repos=
Note that the dependencies here are actually cleanups to
OutputPage::preventClickjacking(), not ParserOutput::preventClickjacking().
It can be hard to tell the difference between these two methods when
reading code (especially when parameter types are omitted) and so we're
being extra cautious by cleaning up both of these together.
Bug: T287216
Depends-On: I626e89a1bff2d9c535b828a5b25eff863c91f858
Depends-On: I23e8b35540d2da34c0a3d335069c518b4b72a333
Depends-On: I2538ac8f37fcde183f496e49a8de0ed25a38e508
Change-Id: I0f9b1fe755f7d3d6a9b18fa21a738dda7566c211
The ::getProperty() naming is too generic and doesn't clearly indicate
that these are "page properties" (which have their own table in the DB).
As part of refactoring a clean API out of ParserOutput which can be used
by Parsoid, clean up the naming here.
Soft-deprecation in this patch, there are a handful of external users
which need to be cleaned up before we hard-deprecate.
Bug: T287216
Change-Id: Ie963eea5aa0f0e984ced7c4dfa0fd65d57313cfa
Because in PHP is "0" == false.
Also
* Combine $this->mOutput->setTitleText calls.
* Avoid inverted logic. Use
if ( !A && !B && !C && D )
instead of
if ( !( A || B || C || !D ) )
* Document false as possible return value.
Change-Id: I92c343b74a9b313b10a2c9b31717a3727aed4cde
This name is consist with the rest of the setter and getter methods
in ParserOutput. Renamed the methods in OutputPage, ImageHistoryList,
ImageHistoryPseudoPager, and ContribsPager as well for consistency;
it also makes chasing down lingering references in codesearch easier.
Soft-deprecated the old name for 1.38. Hard-deprecation will follow,
but there are a number of users in production that should be chased
down first.
Code search:
https://codesearch.https://codesearch.wmcloud.org/deployed/?q=(allow%7Cprevent)Clickjacking&i=nope&files=&excludeFiles=&repos=
Bug: T287216
Change-Id: I9822c60c180d204bd30cb4447a1120155d456da4
This method is used only twice outside this class: in ParserOutputTest
(which can use the standard backdoor to access private members) and
in the advancedbacklinks extension (which should just cut-and-paste
the code, it's quite small and has no dependencies).
Code search:
https://codesearch.wmcloud.org/search/?q=isLinkInternal&i=nope&files=&excludeFiles=&repos=
Bug: T287216
Change-Id: Iddc7f7b691be2cb97a5dba60de7037e2b12206dc
New option 'absoluteURLs' was added to getText method
of the ParserOutput object that replaces all links
in the page HTML with absolute URLs.
Removing the action=render special case from Title
seems safe cause we will end up replacing the result
with absolute URL if we're in a render action no matter
where Title::getLocalUrl was called from.
This change is safely revertable from the perspective
of ParserCache.
Bug: T263581
Change-Id: Id660e1026192f40181587199d3418568f0fdb6d3
== History of WikiPage::triggerOpportunisticLinksUpdate ==
* 2007 (r19095; T10575; b3a8d488a8)
Introduces the "cascading protection" feature.
This commit added code to Article.php, in a conditional branch
where we encountered a ParserCache "miss" and thus have done a
fresh parse. The code in question would query which templates
we ended up using, and if that differed from what the database
said (e.g. stored during the last actual edit or links update),
then a new LinksUpdate is ad-hoc constructed and executed.
I could not find it anywhere explicitly spelled out, but my best
guess is that the reason for this is to make sure that if the page
in question contains wikitext that trancludes a different page based
on the current date and time (such as how most Wikipedia main pages
transclude news information and "Did you know" information based on
dated subpages that are prepared in advance), then we don't just
want to re-render the page after a day has passed, we also want to
re-do the links update to ensure the search index, category links,
and "WhatLinksHere" is correct, and thus by extent, to make sure
that cascading protection from the main page does in fact apply
to the "current" set of subpages and templates actually in-use.
* 2007 (r19227; 0c0c0eff81)
This adds an optimisation to the added logic that limits it to
pages that satisfy `mTitle->areRestrictionsCascading()`.
Thus for most articles, which aren't protected at all, we don't
run LinksUpdate mid-request after a cache miss page view.
Because of this commit, the pre-2007 status quo remained unaltered
and has remains unaltered to this very day: We don't re-index
categories and WhatLinksHere etc, unless an article edit or
propagating template edit takes place.
* 2009 (r52888; 1353a8ba29)
Introduces the PoolCounter feature.
The logic in question moves to Article::doCascadeProtectionUpdates().
* 2015 (Iea952d4d2e66; df5ef8b5d7).
The logic in question is changed, motivated by wanting to avoid
DB writes during page views.
* Instead of executing LinksUpdate mid-request, we now queue a
RefreshLinksJob on the JobQueue, and utilize a newly added
`prioritize => true` parameter.
This commit also introduces a new feature, which is to queue
RefreshLinksJob also for pages that do not have cascading
protection, but that do satisfy a new boolean method
called `$parserOutput->hasDynamicContent()`, which is set when
the Parser encounters TTL-reducing magic words and functions
such as {{CURRENTDAY}} and {{#time}}. For this new case, however,
the `prioritize` parameter is not set, and this feature is disabled
in WMF production (and other farms that enable wgMiserMode).
This commit also renamed doCascadeProtectionUpdates()
to triggerOpportunisticLinksUpdate().
This commit also removed various documentation comments, which
I've partly restored in this patch, the patch you're looking at
now.
== Actual changes ==
* Rename hasDynamicContent() to hasReducedExpiry() and keep the
previous method as a non-deprecated wrapper.
This change is motivated by T280605, in which I intent to make use
of a Parser hook that reduces the cache expiry. There are numerous
extensions in WMF production that already do this, and thus the
assumption that these have "dynamic content" is already false in
some cases. I'm not yet sure how or if to refactor this so to allow
reducing of the TTL *without* causing this side-effect, but as a
first step we can make the method more obvious in its impact
and behaviour.
I've also updated two of the callers that I think will benefit from
this more explicit name and (current) implementation detail.
Bug: T280605
Change-Id: I85bdff7f86911f8ea5b866e3639f08ddd3f3bf6f
This is micro-optimization of closure code to avoid binding the closure
to $this where it is not needed.
Created by I25a17fb22b6b669e817317a0f45051ae9c608208
Change-Id: I0ffc6200f6c6693d78a3151cb8cea7dce7c21653
Apparently I've missed a few instances where dynamic properties
were still written. To fix those we need some grace period for
ParserCache to expire when we would need to have get fallback,
so undeprecate reading dynamic properties.
Bug: T263851
Change-Id: I123605f7b5b907cc1b0ae6f183f3015b41835e8b
One major difference with what we've had before is that now we
actually write class names into the serialization - given that
this new mechanism is extencible, we can't establish any kind
of mapping of allowed classes. I do not think it's a problem
though.
Bug: T264394
Change-Id: Ia152f3b76b967aabde2d8a182e3aec7d3002e5ea
CacheTime::mUsedOptions and ParserOutput::mAccessedOptions
do exactly the same thing and has to be merged into a single property.
This patch adds forward-compatibility and needs to be deployed
at least one train before the patch which actually merges the properties.
Change-Id: Ic9d71a443994e2545ebf2a826b9155c82961cb88
This introduces a mechanism for encoding binary data in
strings set via setProperty(). This is needed to accommodate compressed
data as used by TemplateData, which uses gzip compression to make the
data fit into the page_props table.
Bug: T266200
Change-Id: I19fa0dea8c25d93fcdec9dc5ddd6f3c9c162b621
This adds JSON serialization and deserialization capabilities
to CacheTime and ParserOutput.
NOTE: JSON serialization is disabled for now. Merging this patch
should not change behavior in production.
Bug: T263579
Change-Id: I18187e8bce573d21f6f1bd29106e07c63a6d2f4d
