Commit graph

30 commits

Author SHA1 Message Date
David Causse
ce511406e0 ParserCache: add an option to explicitly trigger links update
Triggering an opportunistic LinksUpdate on every cache miss of the
current revision might not be appropriate in some cases.
Some functions like ContentHandler::getParserOutputForIndexing might
be called after all LinksUpdates but if these functions do explicitely
disallow populating the parser cache via OPT_NO_UPDATE_CACHE we might
enter a case where involved jobs would trigger themselves forever.

It is happening in the case of the CirrusSearch extension that listens
to LinksUpdate and is relying on
ContentHandler::getParserOutputForIndexing to fetch the parser output.

Introduce a new option ParserOutputAccess::OPT_LINKS_UPDATE to be
more intentional on whether such cascading LinksUpdate might occur
or not on cache misses.

Change the default to not trigger a LinksUpdate on every cache miss
and enable it only when rendering the article view (Article::view).
It does not seem ideal that this behavior is owned by the ParserCache
and further refactoring might be needed to separate these concerns.

Bug: T329842
Change-Id: Ib3c3ca935f316ea880ff6c6b393fa80166e42bd3
2023-05-16 11:32:55 +02:00
jenkins-bot
7b98392676 Merge "ParserOutputAccess: Fork primary and secondary caches for parsoid" 2023-03-27 08:09:07 +00:00
C. Scott Ananian
2caf69797c ParserOutputAccess: Fork primary and secondary caches for parsoid
Uses flag to detect which cache instance to use based on ParserOptions
and sets the primary and secondary caches accordingly.  This ensures
that the ParserCacheMetadata cache used by the ParserCache is also
appropriately forked for Parsoid, as Parsoid may consult different
options in the ParserCache than core does.

A follow up patch will attempt to refactor this to be less
parsoid-specific.

Bug: T327769
Bug: T330677
Co-authored-by: Alangi Derick <alangiderick@gmail.com>
Change-Id: Id580b97ad9a0b90bbe56d4de3c2f999274fe329b
2023-03-26 21:46:07 -04:00
Tim Starling
be3018b268 Just another 80 or so PHPStorm inspection fixes (#4)
* Unnecessary regex modifier. I agree with this inspection which flags
  /s modifiers on regexes that don't use a dot.
* Property declared dynamically.
* Unused local variable. But it's acceptable for an unused local
  variable to take the return value of a method under test, when it is
  being tested for its side-effects. And it's acceptable for an unused
  local variable to document unused list expansion elements, or the
  nature of array keys in a foreach.

Change-Id: I067b5b45dd1138c00e7269b66d3d1385f202fe7f
2023-03-25 00:39:06 +00:00
Tim Starling
5e30a927bc tests: Make some PHPUnit data providers static
Just methods where adding "static" to the declaration was enough, I
didn't do anything with providers that used $this.

Initially by search and replace. There were many mistakes which I
found mostly by running the PHPStorm inspection which searches for
$this usage in a static method. Later I used the PHPStorm "make static"
action which avoids the more obvious mistakes.

Bug: T332865
Change-Id: I47ed6692945607dfa5c139d42edbd934fa4f3a36
2023-03-24 02:53:57 +00:00
Timo Tijhof
04a1760b54 poolcounter: Create PoolCounterFactory service class
* Allow for DI of config and PoolCounterConnectionManager.

* Manage the PoolCounterConnectionManager singleton without using
  a global/static field.

* Allow for test overrides (in ParserOutputAccessTest) without needing
  to bring all of ObjectFactory and 'factory' and thus exposing class
  constructors to stable interface (except not really since the args
  are hardcoded in practice).

Bug: T201223
Change-Id: I514fee20b388f04f9c85c5a1373845d621c65395
2023-01-19 07:33:58 +00:00
daniel
166b569444 Make use of the new PageUpdateStatus
PageUpdateStatus provides clean access the the newly created
RevisionRecord.

Depends-On: Ia08c586198082ea47e8313d0d41835f9830fb29e
Change-Id: Id6963842321c4eaa3d7d029ad0b769f73433c103
2022-11-30 17:56:58 +00:00
Derick Alangi
00b38e1345 ParserOutputAccessTest: Inline services & avoid creating variables
The services can just be inlined instead of creating containers that
are only used once.

Change-Id: Ib29a376df9d2cce50c169ba5724d5a9baf65f7d3
2022-09-01 22:25:22 +01:00
Derick Alangi
5e8cd2c838
Migrate from setMwGlobals() to overrideConfigValue(s)
Change-Id: I3f167d0e7d59a5aa091c3095a7d96c889d6e7e78
2022-08-02 10:14:10 +01:00
Derick Alangi
d01e3ed739 Replace deprecated calls ParserOptions::newCanonical( 'canonical' )
This is a quick find & replace of calls to the deprecated method
ParserOptions::newCanonical() when the context is the string literal
'canonical'. This can be safely replaced by called newFromAnon().

Change-Id: If7bb68459b11e0c5f5de188f10fdae85ad1a78bf
2022-06-16 14:22:24 +01:00
Umherirrender
2909d06a08 Use new namespace for revision related classes
All revision related classes are namespaced MediaWiki\Revision
instead of MediaWiki\Storage since 1.32. The old namespaced
class names are deprecated and only kept for backwards-compatibility.

Bug: T305784
Change-Id: I34e492d84d9fc4bc78481667202716d93b3c43cb
2022-04-14 23:03:43 +02:00
Adam Wight
f21b52bec3 Remove test-only method
Usually we opt to break access control in a test, rather than expose
internals in production classes.

Change-Id: I7e393d2569e8784e2c8eb7ed29d60aab58b9bd83
2022-04-06 10:01:52 +02:00
daniel
e239b02a5e Add convenience methods for asserting status.
This ensures that assertions work in a uniform way,
and provides meaningful messages in cause of failure.

Change-Id: Ic01715b9a55444d3df6b5d4097e78cb8ac082b3e
2022-03-16 22:44:25 +01:00
Amir Sarabadani
206f6cbd50 ParserOutputAccess: Add process cache within the service class
This guards against duplicate parses.

These happen when a page is parsed but an extension needs the
ParserOutput again in the same request when it hasn't made into the
ParserCache yet, or if it is considered uncachable. In that case we
still want to allow re-use within the same process.

Bug: T301310
Change-Id: I1ddd967a40b760b1e53f1fd227cb0d0732f78ec1
2022-02-16 12:18:21 +00:00
Reedy
6e29611642 Remove or replace usages of "sanity"
Still some more to go...

Bug: T254646
Change-Id: Ia117f01e443c35b4765f3275cab4f2707e1be96f
2021-11-21 16:42:31 +00:00
Petr Pchelko
cd66d7c335 Convert ParserOutputAccess to PageRecord.
Still needs to downcast to WikiPage in 2 places:

1. To check get a ContentHandler and check if content model
is cacheable. We probably should just make all content models
cacheable.
2. To call WikiPage::triggerOpportunisticLinksUpdate. I have
an elaborate plan for this one, but it will be done separately.

Change-Id: Ifd9ab0155dc1fad0c1608dafea05d16292afd057
2021-04-05 07:46:34 -06:00
Petr Pchelko
f642215aed Convert ParserCache to PageRecord
ParserOptions not updated cause they depend on Title::getLanguage
implementation.

Tests converted to not require a DB anymore. Can't be proper unit
tests yet due to globals in ParserOptions and fake time hacks,
but exec time does go down from 70 seconds to 9 seconds.

Page content model is still emitted in the metrics since
it was considered useful. Should be removed when we get
something like a page type concept.

Change-Id: Ib16fd0b5b87ffc3cb4d21f4aa43d1203cb7206d2
2021-04-02 21:14:54 -06:00
daniel
ece39dbf60 Set proper epoch in cache tests.
Since ConvertibleTimestamp interprets "" as "now", setting the epoch to
"" resulted in the cache expiring after one second. This would cause the
test to fail of the system clock ticket over the second mark before the
cache was accessed, resulting in spurious test failures.

Bug: T271081
Change-Id: I4ffe4570f9a7a1a09460ad00fedc14b17adddf05
2021-02-03 23:08:25 +01:00
Thiemo Kreuz
b655f382db Remove broken/outdated @param/@throws tags from @dataProviders
My personal best practice is to not document @params when there
is a @dataProvider. I mean, these test…() functions are not
meant to be called from anywhere. They do not really need
documentation. @param tags don't do much but duplicate what the
@dataProvider does. This is error-prone, as demonstrated by the
examples in this patch.

This patch also removes @throws tags from tests. A test…() can
never throw an exception. Otherwise the test would fail.

Most of these are found by the not yet released I10559d8.

Change-Id: I3782bca43f875687cd2be972144a7ab6b298454e
2021-01-21 03:42:42 +00:00
Umherirrender
0347fd0631 Improve some function documentation in tests
Also fix some whitespaces

Change-Id: Ibed50a4f07442d3f299cf545c16f5dbb5f27a411
2021-01-14 22:13:55 +01:00
daniel
00a3439dce Introduce RevisionOutputCache
Bug: T267981
Change-Id: Ib1dc641ed10d786918362b25bd655780d5844ba1
2020-12-14 16:50:28 +00:00
Petr Pchelko
4417b13d58 Make ParserCache respect ParserOptions::isSafeToCache
Bug: T269154
Change-Id: I8e9ecd2787aa8d172e708ba64ea936e63fbc6b36
2020-12-02 14:02:36 -06:00
daniel
2c7ba6f62b PoolWorkArticleViewOld: use WANObjectCache
Use WANObjectCache instead of the local cluster object cache.

Bug: T268278
Change-Id: Ic16feffecaf4b75c284c6ef34de42ac113e625f8
2020-11-30 16:38:56 +00:00
Petr Pchelko
dbdc2a3cd3 Introduce JsonCodec to help with serialization/deserialization
Change-Id: I5433090ae8e2b3f2a4590cc404baf838025546ce
2020-11-19 08:32:21 -07:00
daniel
195bc9715d PoolWorkArticleView: inject logger
Bug: T267832
Change-Id: I7f4763d0e812d076188bb1a4ca2c333f50dffbee
2020-11-18 17:08:37 +01:00
daniel
a2ae4192c0 ParserOutputAccess: cache ouput for old revisions
DEPLOY: Set $wgOldRevisionParserCacheExpireTime = 0 in production first!

Bug: T267832
Depends-On: I3c73f5d9f6a54e2736600e8f9506659a3fb0e7f6
Change-Id: I0fe275b4991f1bf89c7bb587132bc4fb0ea862e2
2020-11-17 20:52:35 +00:00
daniel
ed41864370 Extract PoolWorlArticleViewCurrent
Extracts a specialized subclass for rendering the current revision
from PoolWorlArticleView, which then no longer knowes about caching.

In the next step, we will add a subclass that implements caching for old
revisions.

Bug: T267832
Change-Id: I56fb365962951e6c723a01cf9243dbc0094b5581
2020-11-17 20:17:02 +01:00
daniel
175d548e61 Clean up PoolWorkArticleView
PoolWorkArticleView needs some cleanup before we can make it
cache output for old revisiosn (T244058).

This patch does doe following:
* apply dependency injection
* remove backwards compatibility code for legacy constructor calls
* mark PoolWorkArticleView as @internal (unused in extensions)
* remove audience check (to be done by caller)
* no longer set $wgUseFileCache to false.

For $wgUseFileCache, it seems like this has had no effect for a long
time. It would be set to false only on a cache miss during a page view.
But the file cache is only updated via HtmlCacheUpdater on edit and
purge.

Bug: T244058
Change-Id: Ief467562af0aa2f88ff7b42469d0273d2a1dcf7a
2020-11-16 12:06:20 +01:00
Petr Pchelko
7c68ae9296 Safe ParserOutput extension data and JsonUnserializable helper.
One major difference with what we've had before is that now we
actually write class names into the serialization - given that
this new mechanism is extencible, we can't establish any kind
of mapping of allowed classes. I do not think it's a problem
though.

Bug: T264394
Change-Id: Ia152f3b76b967aabde2d8a182e3aec7d3002e5ea
2020-11-10 11:21:09 -07:00
daniel
67d0986211 Introduce ParserOutputAccess
Encapsulate logic for getting rendered page content, for any revision,
with caching and pooling hidden away.

Introducing such a service object will also give us a leverage point for
supporting output transformations. Output transformations are currently
implemented partially in ParserOutput, partially in Parser, and partially
duplicated in Parsoid.

Bug: T267234
Change-Id: I566d7a7936633823ba68b5aecbc8c2d88949b4f8
2020-11-10 15:12:12 +01:00