This is the second patch of a series of patches to remove
ParserOutput::getText() calls from core. This series of patches should
be functionally equivalent to I2b4bcddb234f10fd8592570cb0496adf3271328e.
This patch replaces the calls to getText where the legacy parser is
called directly by creating a pipeline and invoking it on the generated.
These should probably eventually use the Content framework to generate
output instead of using Parser directly (T371008), which will also allow
them to transparently support Parsoid.
Bug: T293512
Change-Id: I45951a49e57a8031887ee6e4546335141d231c18
I believe this makes the code less brittle, and also makes it a bit
more obvious what these strings are meant to represent.
Change-Id: Ia39b5c80af4b495931d0a68fd091b783645dd709
And deprecated aliases for the the no namespaced classes.
ReplicatedBagOStuff that already is deprecated isn't moved.
Bug: T353458
Change-Id: Ie01962517e5b53e59b9721e9996d4f1ea95abb51
Changes to the use statements done automatically via script
Addition of missing use statement done manually
Change-Id: Iae45fa269363be8ee05c598ea6926514ce817762
This is a follow-up to: I0b683461212a357c7eb09ddec59c87539e323c65
and I40a8372a76f33c5f62ea73bb1180dd7c47412c89 which explicitly for
backward compatibility reasons supports IBufferingStatsdDataFactory.
Now that we've fully switched to StatsFactory together with the
`copyToStatsdAt()` method, we're fine to fully remove this `instanceof`
logic.
Bug: T356815
Change-Id: I164d82904b6d3fb575cb973c14f9454569bf09ac
The presence of this mock means that, valuable debug messages are
missing from PHPUnit output. Considering how high-level that is,
that is quite a bit.
Follows-up a longer-than-needed debug session from I5413825a0
and Ie532c17e5b. Most of the time was actually not spent adding this
back in, but trying to figure out how it was that the messages
were missing.
Bug: T358901
Change-Id: I41fa8dce1ea37ea69632917bcb591b8327d35e7d
* Inject from ServiceWiring via PoolCounterFactory.
* In production we use PoolCounterClient (which is backed by
our poolcounterd service), we also offer PoolCounterRedis for
third parties. Replace the local logger in the Redis variant in
favour of the built-in one.
* Update PoolCounterWork to adopt this as well. Expose it via getter
to here, so that DI works all the way, including for the existing
test.
Bug: T358901
Change-Id: I5413825a0172b186d58e85bbc3cc93697b174c27
PoolCounterWorkArticleView was not designed for use by all callers of
getParserOutput. It provides stampede protection but does not generally
prevent duplicate concurrent parsing, and it may result in stale cache
entries being returned to the caller. This is acceptable for page views,
but not other use cases like editing or updating secondary derived data.
Bug: T352837
Change-Id: Ie532c17e5b86e8e1adbb57ecd5c5c6405b83bf8f
Set the render ID for each parse stored into cache so that we are able
to identify a specific parse when there are dependencies (for example
in an edit based on that parse). This is recorded as a property added
to the ParserOutput, not the parent CacheTime interface. Even though
the render ID is /related/ to the CacheTime interface, CacheTime is
also used directly as a parser cache key, and the UUID should not be
part of the lookup key.
In general we are trying to move the location where these cache
properties are set as early as possible, so we check at each location
to ensure we don't overwrite a previously-set value. Eventually we
can convert most of these checks into assertions that the cache
properties have already been set (T350538). The primary location for
setting cache properties is the ContentRenderer.
Moved setting the revision timestamp into ContentRenderer as well, as
it was set along the same code paths. An extra parameter was added to
ContentRenderer::getParserOutput() to support this.
Added merge code to ParserOutput::mergeInternalMetaDataFrom() which
should ensure that cache time, revision, timestamp, and render id are
all set properly when multiple slots are combined together in MCR.
In order to ensure the render ID is set on all codepaths we needed to
plumb the GlobalIdGenerator service into ContentRenderer, ParserCache,
ParserCacheFactory, and RevisionOutputCache. Eventually (T350538) it
should only be necessary in the ContentRenderer.
Bug: T350538
Bug: T349868
Followup-To: Ic9b7cc0fcf365e772b7d080d76a065e3fd585f80
Change-Id: I72c5e6f86b7f081ab5ce7a56f5365d2f75067a78
Some less trivial cases. Also update variable names.
This doesn't really change anything, but I hope it helps promote
getConnectionProvider() as the common way to do this.
Follow-up to 8604c384f6.
Change-Id: I6657d783375fac5c7fa856b884ff1fb09285e94c
Two of the classes in this directory have already namespaced to
MediaWiki\PoolCounter.
Bug: T353458
Change-Id: Ie41f8d935f7623bb40040a5eb78f99c6d7b7b75e
== Skin::wrapHTML ==
Skin::wrapHTML no longer has to perform any guessing of the
ParserOutput language. Nor does it have to special wiki pages vs
special pages in this regard. Yay, code removal.
== ImagePage ==
On URLs like /wiki/File:Example.jpg, the main output handler is
ImagePage::view. This calls the parent Article::view to handle most of
its output. Article::view obtains the ParserOptions, and then fetches
ParserOutput, and then adds `<div class=mw-parser-output>` and its
metadata to OutputPage.
Before this change, ImagePage::view was creating a wrapper based
on "predicting" what language the ParserOutput will contain. It
couldn't call the new OutputPage::getContentLanguage or some
equivalent as Article::view wouldn't have populated that yet.
This leaky abstraction is fixed by this change as now the `<div>`
from ParserOutput no longer comes with a "please wrap it properly"
contract that Article subclasses couldn't possibly implement correctly
(it coudln't wrap it after the fact because Article::view writes to
OutputPage directly).
RECENT (T310445):
A special case was recently added for file pages about translated SVGs.
For those, we decide which language to use for the "fullMedia" thumb
atop the page. This was recently changed as part of T310445 from a
hardcoded $wgLanguageCode (site content lang) to new problematic
Title::getPageViewLanguage, which tries to guestimate the page
language of the rendered ParserOutput and then gets the preferred
variant for the current user. The motivation for this was to support
language variants but used Title::getPageViewLanguage as a kitchen
sink to achieve that minor side-effect. The only part of this
now-deprecated method that we actually need is
LanguageConverter::getPreferredVariant().
Test plan: Covered by ImagePageTest.
== Skin mainpage-title ==
RECENT (T331095, T298715):
A special case was added to Skin::getTemplateData that powers the
mainpage-title interface message feature. This is empty by default,
but when created via MediaWiki:mainpage-title allows interface admins
to replace the H1 with a custom and localised page heading.
A few months ago, in Ifc9f0a7174, Title::getPageViewLanguage was
applied here to support language variants. Replace with the same
fix as for ImagePage. Revert back to Message::inContentLanguage()
but refactor to inLanguage() via MediaWikiServices::getContentLanguage
so that LanguageConverter::getPreferredVariant can be applied.
== EditPage ==
This was doing similar "predicting" of the ParserOutput language to
create an empty preview placeholder for use by preview.js. Now that
ApiParse (via ParserOutput::getText) returns a usable element without
any secret "you magically know the right class, lang, and dir" contract,
this placeholder is no longer needed.
Test Plan:
* EditPage: Default preview
1. index.php?title=Main_Page&action=edit
2. Show preview
3. Assert <div class="mw-content-ltr mw-parser-output" lang=en dir=ltr>
* EditPage: JS preview
1. Preferences > Editing > Show preview without reload
2. index.php?title=Main_Page&action=edit
3. Show preview
4. Assert <div class="mw-content-ltr mw-parser-output" lang=en dir=ltr>
5. Type something and 'Show preview' again
6. Assert old element gone, new text is shown, and new element
attributes are the same as the above.
== McrUndoAction ==
Same as EditPage basically, but without the JS preview use case.
== DifferenceEngine ==
Test:
1. Open /w/index.php?title=Main_Page&diff=0
(this shows the latest diff, can do manually by viewing
/wiki/Main_Page, click "View history", click "Compare selected revisions")
2. Assert <div class="mw-content-ltr mw-parser-output" lang=en dir=ltr>
3. Open /w/index.php?title=Main_Page&diff=0&action=render
4. Assert <div class="mw-content-ltr mw-parser-output" lang=en dir=ltr>
== Special:ExpandTemplates ==
Test:
1. /wiki/Special:ExpandTemplates
2. Write "Hello".
3. "OK"
4. Assert <div class="mw-content-ltr mw-parser-output" lang=en dir=ltr>
Bug: T341244
Depends-On: Icd9c079f5896ee83d86b9c2699636dc81d25a14c
Depends-On: I4e7484b3b94f1cb6062e7cef9f20626b650bb4b1
Depends-On: I90b88f3b3a3bbeba4f48d118f92f54864997e105
Change-Id: Ib130a055e46764544af0f1a46d2bc2b3a7ee85b7
* This reverts commit c1b82097.
* This reverts commit 56025174.
* This updates a test change from commit c8d0470f.
* Now that ParsoidOutputAccess has become a thin wrapper over
ParserOutputAccess and the code has landed in production without
needing to be reverted, we can revert the above hacks as soon as the
hits from the 'parsoid' instance start to go down to a small number.
As of the time of creating of this patch, of the combined hits to the
'parsoid' and 'parsoid_pcache' instance, over 90% are now from the
'parsoid_pcache' instance. We can wait for a couple more days to
watch how this number changes.
* Note that once we deploy this patch, the accesses which would have
hit in the 'parsoid' instance (with this hack) will instead result
in a cache miss thus adding the full parse latency to REST API
requests (whether by VisualEditor or by other clients). So, we need
to figure out what the cutoff point is. While 3 weeks is a guaranteed
switchover timeframe (because all entries in 'parsoid' cache will
expire at that time and we'll get no more hits from there after that),
note that we are at < 10% hits in this cache just 4 days after the
train rollout. So, there is a good chance we could get beyond 95%
by the end of this week.
Bug: T347632
Change-Id: Ibd741b92b860b4d4b03ca220863debaf53fab44a
For example:
* assertStatusGood = ok without any errors or warnings
* assertStatusWarning = ok, but not good, i.e. there is a warning
Change-Id: I4b3ec7a3c5b028c0505e1371c297a9c47e448b42
* ParsoidOutputAccess used a 'parsoid' ParserCache instance and did not
set the 'useParsoid' parser option for tier 2 ParserOutput cache key
computations.
* ParserOutputAccess uses 'pcache' for legacy parser output and
'parsoid-pcache' for Parsoid parser output objects based on whether
'useParsoid' parser option is true or false.
* 'parsoid-pcache' is right now very sparsely populated since useParsoid
is only used for testing.
* In Ic9b7cc0fcf36, where we make ParsoidOutputAccess a thin wrapper
over ParserOutputAccess, all Parsoid parser output requests will go
to ParserOutputAccess's 'parsoid-pcache' instance which is sparsely
populated and hence will result in a lot of cold cache misses.
* To eliminate this scenario, this patch adds hardcoded hacks to both
ParserOutputAccess and ParserCache to query the 'parsoid' PC instance
on cache misses to the 'parsoid-pcache' instance. Over a 3-week
period, as 'parsoid-pcache' fills up, there will be fewer and fewer
access to the 'parsoid' PC instance which will also expire. At the
end of that period, we can remove this hack.
T347632 tracks removal of these hacks.
* Added new PHP unit test verifying that the hack work as intended.
Bug: T332931
Change-Id: I7f933fd61bf358c6ea0e0c1202231cac618f9e8d
This patch makes two major changes:
- In the PoolCounter chain, we simply inject CP and call it directly
and as result, there is no need for ILBF::getChronologyProtectorTouched
- Instead of injecting CP callback to LB, just pass the object down the
chain which leads to simpler and more stable code.
Bug: T275713
Change-Id: If78f4498d98e256015e54cc46561cb11b2947058
This class is used heavily basically everywhere, moving it to Utils
wouldn't make much sense. Also with this change, we can move
StatusValue to MediaWiki\Status as well.
Bug: T321882
Depends-On: I5f89ecf27ce1471a74f31c6018806461781213c3
Change-Id: I04c1dcf5129df437589149f0f3e284974d7c98fa
Triggering an opportunistic LinksUpdate on every cache miss of the
current revision might not be appropriate in some cases.
Some functions like ContentHandler::getParserOutputForIndexing might
be called after all LinksUpdates but if these functions do explicitely
disallow populating the parser cache via OPT_NO_UPDATE_CACHE we might
enter a case where involved jobs would trigger themselves forever.
It is happening in the case of the CirrusSearch extension that listens
to LinksUpdate and is relying on
ContentHandler::getParserOutputForIndexing to fetch the parser output.
Introduce a new option ParserOutputAccess::OPT_LINKS_UPDATE to be
more intentional on whether such cascading LinksUpdate might occur
or not on cache misses.
Change the default to not trigger a LinksUpdate on every cache miss
and enable it only when rendering the article view (Article::view).
It does not seem ideal that this behavior is owned by the ParserCache
and further refactoring might be needed to separate these concerns.
Bug: T329842
Change-Id: Ib3c3ca935f316ea880ff6c6b393fa80166e42bd3
Uses flag to detect which cache instance to use based on ParserOptions
and sets the primary and secondary caches accordingly. This ensures
that the ParserCacheMetadata cache used by the ParserCache is also
appropriately forked for Parsoid, as Parsoid may consult different
options in the ParserCache than core does.
A follow up patch will attempt to refactor this to be less
parsoid-specific.
Bug: T327769
Bug: T330677
Co-authored-by: Alangi Derick <alangiderick@gmail.com>
Change-Id: Id580b97ad9a0b90bbe56d4de3c2f999274fe329b
* Unnecessary regex modifier. I agree with this inspection which flags
/s modifiers on regexes that don't use a dot.
* Property declared dynamically.
* Unused local variable. But it's acceptable for an unused local
variable to take the return value of a method under test, when it is
being tested for its side-effects. And it's acceptable for an unused
local variable to document unused list expansion elements, or the
nature of array keys in a foreach.
Change-Id: I067b5b45dd1138c00e7269b66d3d1385f202fe7f
Just methods where adding "static" to the declaration was enough, I
didn't do anything with providers that used $this.
Initially by search and replace. There were many mistakes which I
found mostly by running the PHPStorm inspection which searches for
$this usage in a static method. Later I used the PHPStorm "make static"
action which avoids the more obvious mistakes.
Bug: T332865
Change-Id: I47ed6692945607dfa5c139d42edbd934fa4f3a36
* Allow for DI of config and PoolCounterConnectionManager.
* Manage the PoolCounterConnectionManager singleton without using
a global/static field.
* Allow for test overrides (in ParserOutputAccessTest) without needing
to bring all of ObjectFactory and 'factory' and thus exposing class
constructors to stable interface (except not really since the args
are hardcoded in practice).
Bug: T201223
Change-Id: I514fee20b388f04f9c85c5a1373845d621c65395
PageUpdateStatus provides clean access the the newly created
RevisionRecord.
Depends-On: Ia08c586198082ea47e8313d0d41835f9830fb29e
Change-Id: Id6963842321c4eaa3d7d029ad0b769f73433c103
This is a quick find & replace of calls to the deprecated method
ParserOptions::newCanonical() when the context is the string literal
'canonical'. This can be safely replaced by called newFromAnon().
Change-Id: If7bb68459b11e0c5f5de188f10fdae85ad1a78bf
All revision related classes are namespaced MediaWiki\Revision
instead of MediaWiki\Storage since 1.32. The old namespaced
class names are deprecated and only kept for backwards-compatibility.
Bug: T305784
Change-Id: I34e492d84d9fc4bc78481667202716d93b3c43cb
Usually we opt to break access control in a test, rather than expose
internals in production classes.
Change-Id: I7e393d2569e8784e2c8eb7ed29d60aab58b9bd83
This ensures that assertions work in a uniform way,
and provides meaningful messages in cause of failure.
Change-Id: Ic01715b9a55444d3df6b5d4097e78cb8ac082b3e
This guards against duplicate parses.
These happen when a page is parsed but an extension needs the
ParserOutput again in the same request when it hasn't made into the
ParserCache yet, or if it is considered uncachable. In that case we
still want to allow re-use within the same process.
Bug: T301310
Change-Id: I1ddd967a40b760b1e53f1fd227cb0d0732f78ec1
Still needs to downcast to WikiPage in 2 places:
1. To check get a ContentHandler and check if content model
is cacheable. We probably should just make all content models
cacheable.
2. To call WikiPage::triggerOpportunisticLinksUpdate. I have
an elaborate plan for this one, but it will be done separately.
Change-Id: Ifd9ab0155dc1fad0c1608dafea05d16292afd057
ParserOptions not updated cause they depend on Title::getLanguage
implementation.
Tests converted to not require a DB anymore. Can't be proper unit
tests yet due to globals in ParserOptions and fake time hacks,
but exec time does go down from 70 seconds to 9 seconds.
Page content model is still emitted in the metrics since
it was considered useful. Should be removed when we get
something like a page type concept.
Change-Id: Ib16fd0b5b87ffc3cb4d21f4aa43d1203cb7206d2
Since ConvertibleTimestamp interprets "" as "now", setting the epoch to
"" resulted in the cache expiring after one second. This would cause the
test to fail of the system clock ticket over the second mark before the
cache was accessed, resulting in spurious test failures.
Bug: T271081
Change-Id: I4ffe4570f9a7a1a09460ad00fedc14b17adddf05
My personal best practice is to not document @params when there
is a @dataProvider. I mean, these test…() functions are not
meant to be called from anywhere. They do not really need
documentation. @param tags don't do much but duplicate what the
@dataProvider does. This is error-prone, as demonstrated by the
examples in this patch.
This patch also removes @throws tags from tests. A test…() can
never throw an exception. Otherwise the test would fail.
Most of these are found by the not yet released I10559d8.
Change-Id: I3782bca43f875687cd2be972144a7ab6b298454e
