Commit graph

98 commits

Author SHA1 Message Date
addshore
bc86b698cd parser: Add new pcache metrics, split by page content model
Change-Id: I31c3c5b863309ffcc4424c43891b577b3fb7a753
2019-02-11 20:48:56 +00:00
daniel
d8c409dd16 Make HTML generation in RenderedRevision optional
This allows optimization for situations in which a caller
needs the meta-data of a ParserOutput, and the respective
ContentHandler can provide that meta-data without generating
HTML output.

Bug: T194048
Change-Id: I786d294d18a6a2e3cea61577313e21b578c44f1e
2018-08-31 10:48:41 +00:00
Fomafix
6a022c8d20 Add type hint for ParserOutput
EditPage::getPreviewLimitReport is called by EditPage::showEditForm
with $output = null. Specify this in the @param tag and allow this by a
default value.

Change-Id: Iec8905aab736a1f254a57853c7cab935d008653e
2018-07-30 09:23:59 +02:00
Thiemo Kreuz
e6b6920cff Fix PHPDoc type hints in CacheTime, ParserOptions, and related
I'm intentionally not touching any code in this patch, only
documentation.

Change-Id: I6975194c218760031789d5335dfbb330017dc6fc
2018-04-18 15:10:31 +00:00
Brad Jorsch
2791fb0861 Hard-deprecate ParserOutput stateful transform methods
This also removes all the in-core calls that had been kept for the
benefit of extensions, and causes them to not have any effect since
anything that had been calling them was already either a no-op or will
probably be broken now that nothing in core is setting or checking the
flags.

Change-Id: Id22c1a5a6d6a249debb14063ae3f8838d105b634
2018-02-13 12:28:36 -05:00
Kunal Mehta
399adec9ad Turn ParserCache into a service, deprecate $parserMemc
ParserCache is already a singleton, making it a good candidate for a
service. $parserMemc is an odd global (it lacks the "wg" prefix) and is
ripe for deprecation.

The following are now deprecated:
* $parserMemc global
* ParserCache::singleton()
* wfGetParserCacheStorage()

A ParserCache::getCacheStorage() method was added for cases where direct
access to the underlying BagOStuff object is necessary.

Usage of $parserMemc will emit deprecation warnings through the
DeprecatedGlobal class mechanism. All usage in core was migrated.

Also take this opportunity to inject the $wgCacheEpoch global value into
ParserCache. This will require an update to the FlaggedRevs extension.

Change-Id: I2ac7afff0d8522214329248c3d1cdccd0f72bbd4
2017-07-05 19:56:49 -07:00
Brad Jorsch
84694a9d59 Remove ParserOptions::legacyOptions() and cleanup related code
ParserOptions::legacyOptions() has been sitting around since 1.17.
Originally it seems to have been intended as a way to avoid a mass cache
invalidation (similar to optionsHashPre30() from I7fb9ffca9). That code
was mostly removed in 1.23, but legacyOptions() was left behind because
it was also being used in a few places as "all cache-varying options"
(despite it not being documented for that purpose) where we'd rather
have any key than no key at all.

This patch creates an actual ParserOptions::allCacheVaryingOptions()
method for those use cases and deprecates the long-obsolete
legacyOptions().

It also makes more explicit the use of the "all cache-varying options"
fallback in ParserCache::getKey(), and doesn't bother trying to use that
fallback in ParserCache::get() where it no longer makes sense.

Change-Id: Ife1e54744155136a570210c03fe907f18f8e8ece
2017-07-04 01:28:57 +00:00
Brad Jorsch
27fd0920a1 Remove ParserOptions::optionsHashPre30()
The pre-1.30 version of ParserOptions::optionsHash() was kept
temporarily as ParserOptions::optionsHashPre30() to prevent a cache
stampede on WMF sites when the hash format was changed in I7fb9ffca9.

Now that the cache has been rebuilt, it's no longer needed and we should
clean it up instead of leaving it forever to bitrot.

Change-Id: I037d8dfdefe72a295547bd331bc1454e69cb418d
2017-06-28 00:18:59 +00:00
Brad Jorsch
da43a0ae34 ParserCache: Delete old-style key when saving
It was noticed that disk usage on the parser cache machines was
increasing since shortly after wmf.4 was redeployed everywhere on the
9th. One theory is that I7fb9ffca9 causes this by making reparses for an
existing old-style cache entry start writing the new-style key where
they would previously have overwritten the old-style key. On that
theory, let's delete that old-style key (that should now be useless) on
save.

I'm assuming here that firing a blind delete for keys that probably
don't exist in the cache (i.e. every new edit) isn't going to hurt
anything. If that's not the case, we'd need to check existence before
deleting.

Bug: T167784
Change-Id: Ie5efb05722cb7da2a90da195a1f244468177175d
2017-06-14 13:42:36 +00:00
Brad Jorsch
0facbe3e3d Try harder to avoid parser cache pollution
* ParserOptions is reorganized so it knows all the options and their
  defaults, and can report whether the non-key options are at their
  defaults.
* Definition of the "canonical" ParserOptions (which is unfortunately
  different from the "default" ParserOptions) is moved from
  ContentHandler to ParserOptions.
* WikiPage uses this to throw an exception if it's asked to cache
  with options that aren't used in the cache key.
* ParserCache gets some temporary code to try to avoid a massive cache
  stampede on upgrade.

Bug: T110269
Change-Id: I7fb9ffca96e6bd04db44d2d5f2509ec96ad9371f
Depends-On: I4070a8f51927121f690469716625db4a1064dea5
2017-06-05 14:17:28 +00:00
Kunal Mehta
ff8a0c788b parser: Avoid deprecated wfMemcKey()
Tested that parser cache keys stay the same, before and after this
change.

Also use the more obvious ObjectCache::getLocalClusterInstance() instead
of looking up the main cache type in config and using
ObjectCache::getInstance().

Change-Id: Icef646b3c05e732ef4079d6900e6bce111debf2b
2017-05-25 12:05:49 -07:00
James D. Forrester
9635dda73a includes: Replace implicit Bugzilla bug numbers with Phab ones
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.

Change-Id: I6f59febaf8fc96e80f8cfc11f4356283f461142a
2017-02-21 18:13:24 +00:00
Aaron Schulz
c84ba4d864 Avoid page_touched update for HTTP GET action=purge requests
This will still clear the local DC parser cache and the CDN cache
in all DCs. Therefore, the next page views served by the local
DC will reflect the refreshed content, as will further GET/HEAD
requests by the client that issued the purge using GET/HEAD.

If the problem was imply a stale CDN cache entry, then all
DCs will be up-to-date. If the problem is stale parser cache,
then a proper POST purge request is required to refresh all DCs.

Bug: T92357
Change-Id: I9af12ca8cfff73298f404fd3e2dd4f546621c546
2016-09-13 05:24:46 +00:00
jenkins-bot
3f7164062f Merge "ParserCache: Don't try to save to nothing if disabled" 2016-05-03 01:59:57 +00:00
Kunal Mehta
a9a5b2c2c4 ParserCache: Don't try to save to nothing if disabled
If the ParserCache is disabled via $wgParserCacheType = CACHE_NONE,
don't bother trying to save anything because it won't do anything.

This also removes the misleading "Saved in parser cache" HTML comment
from the rendered output since it isn't being saved at all.

Change-Id: I5603d2b2866f0079cdcd837b1f1a4efc00bc8ed9
2016-05-02 18:24:17 -07:00
jenkins-bot
086fd2799f Merge "In ParserCache, respect $useOutdated" 2016-05-02 14:42:29 +00:00
Aaron Schulz
af349f79c8 Fix timestamp check in ApiStashEdit::checkCache
* The wrong time method was used, and it also was not set.
* Bumped the threshold a bit while at it, which was basically
  0-5 minutes before.

Bug: T133332
Change-Id: Ide3e66f551aa6e50410c562e5c917141d59b7f64
2016-04-28 22:01:03 +00:00
Aaron Schulz
72d334bdf6 Allow for logging cases when parser cache is rejected
Bug: T133659
Change-Id: I779eeffc2ae0533c1929b189a19339a0dc4986f5
2016-04-26 20:25:38 -07:00
Tim Starling
a8f60c1343 In ParserCache, respect $useOutdated
$useOutdated was broken by Ifcc4d2f67f3b77, which forces the cached entry
to always have the same revision ID as the latest revision. This
probably caused the "pool queue is full" errors reported after Prince's
death was announced.

Bug: T133659
Change-Id: Ibd111bed203dd72103d772d8dcba4f0d943124fd
2016-04-26 09:41:36 +00:00
Reedy
1834ee3d8e Fix numerous class/function casing
Change-Id: I23982bfa0548c9ea3bdb432be7982f1563930715
2016-03-18 23:14:49 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Aaron Schulz
54991403ff objectcache: Add BagOStuff::READ_VERIFIED flag to get()
* This lets multiwrite backends upgrade cached items
  to higher tiers using UPGRADE_TTL.
* This is useful for memcached/sql tiers or apc/memcached.

Change-Id: I34b30ce8b54f8de36429d48c80f6768aed310272
2015-10-05 16:03:00 -07:00
Amir E. Aharoni
bd30ccd795 Make lines shorter to pass phpcs in some files under includes/parser
This doesn't fix all the files under includes/parser -
some of them deserve their own patches.

Bug: T102614
Change-Id: I2fcbc19ee337e1b7db4635b5e5f324c651b4d144
2015-09-26 18:19:11 +00:00
Aaron Schulz
fd8e85fc2d Support async writes to secondary MultiWriteBagOStuff stores
* This is useful for ParserCache, as it tries to focus on memcached
  and use other caches (e.g. mariadb) for the long-tail of less used
  content, as setup on WMF. The class uses BagOStuff in a way that is
  compatible with this approach.

Bug: T109751
Change-Id: Ia64eb44a9b52a988fde27b468d604d9163bed4b4
2015-09-18 21:35:43 -07:00
Aaron Schulz
5d7ffb0767 Fixed ParserCache IDE warnings
Change-Id: I1785f4207379b873e4ce7312c0aae7a4300e7a06
2015-08-21 04:22:59 +00:00
Jackmcbarn
c4f69827bc Fix some PHPCS issues
Change-Id: I6289ce362f8dfde7baf2b665f082820898844973
2015-06-26 01:32:28 -04:00
Kunal Mehta
e4c41d5126 Document that ParserCache::get() may be passed a WikiPage or Article
This is terrible, but at least it is no longer lying.

Change-Id: Id1cc1616b60dbde45a12ce9a23b76282efd1c6a9
2015-06-24 01:21:10 +00:00
Ori Livneh
207dfd2adf Add RejectParserCacheValue hook
Add a new hook, 'RejectParserCacheValue', which allows extensions to reject an
otherwise-successful parser cache lookup. The intent is to allow extensions to
manage the eviction of archaic HTML output from the cache.

Change-Id: I660679a48c46608f859bd52b31d6a888aabcc9ac
2015-06-23 11:23:57 -07:00
Aaron Schulz
8af83f4ff8 Use instanceof in ParserCache::getKey to help IDEs
Change-Id: I772f53ee28ade5da499fe05259a17fed5cc52adb
2015-06-10 14:09:20 -07:00
Ori Livneh
b31e567b78 hierarchicalize(!) stat names
Graphite expects name components to be dot-separated, so our habit of using
dashes doesn't really make sense. Change metric names to be more compatible
with Graphite, except the job queue's, since that will require a gdash
dashboard definition migration.

Change-Id: I77d0ff7606a8fc88434e4352d23415a9a8f4725a
2015-06-03 16:27:13 -07:00
daniel
d39e1e24d1 Introduce ParserCacheSaveComplete hook.
Rationale: give extensions a way to track which "renderings"
of a page exist in the cache. This is particularly relevant
for multi-lingual wikis that splpit the parser cache by user
language on some pages. In that case, hooking into
ParserAfterParse or LinksUpdateComplete is insufficient to
track all language specific renderings.

Bug: T99511
Change-Id: Iebf526098ca837a7df637c650097119495000c81
2015-05-25 13:35:23 +00:00
Aaron Schulz
d9505b9dc1 Updated ParserCache doc types
Change-Id: I71fead62a4a498e40b2aa57e6d2701409bf7c7c0
2015-05-01 23:07:18 -07:00
Aaron Schulz
4111ff0dc3 Removed obsolete "containsOldMagic" code
Change-Id: Id225347e0599a6f79b30b0793cce7d97daed46f2
2015-02-15 14:41:49 -08:00
Chad Horohoe
aa21e125a3 Remove obvious function-level profiling
Xhprof generates this data now. Custom profiling of various
sub-function units are kept.

Calls to profiler represented about 3% of page execution
time on Special:BlankPage (1.5% in/out); after this change
it's down to about 0.98% of page execution time.

Change-Id: Id9a1dc9d8f80bbd52e42226b724a1e1213d07af7
2015-01-07 11:14:24 -08:00
addshore
61c989cfc0 Fix phpcs issues in parser
This fixes all issues except for:
 - class names
 - line length

Change-Id: Ie91b010d5b3eec49d3b80b6e93b125a901ef43c6
2014-08-12 01:00:15 +00:00
umherirrender
dd8921c9d9 Cleanup some docs (includes/[m-r])
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling

Change-Id: I8ebfbcea0e2ae2670553822acedde49c1aa7e98d
2014-07-24 19:43:25 +02:00
Siebrand Mazeland
dfc7416fbe Various documentation updates for includes/parser/
Change-Id: I16dd3a792cc83f8c80b3652d42c055730f6d177a
2014-05-11 18:18:26 +02:00
Siebrand Mazeland
2527cca6de Fix most CodeSniffer issues in includes/parser/
Remaining are the classes containing underscores and possibly a few other
issues that will be addressed soonish.

Change-Id: Icf56374c71afc134420ebbcfecf12dcb29dc9564
2014-05-11 08:44:52 +00:00
umherirrender
7f9fd63901 Fixed some @params documentation (includes/parser)
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.

Change-Id: I49f8f48b521878de7abd9cc40efdeff6cf9a37e0
2014-04-22 01:38:39 +02:00
umherirrender
725d9d125d Removed unneeded spaces and colons in @param and friends
Also swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.

Change-Id: Ic36c8c7820a6c2d603f1138130670c6bf6a1ca59
2014-04-08 16:02:49 +00:00
Brad Jorsch
78aad9802d Include parsed revision ID in parser cache
One theory for what's behind bug 46014 is that the vandal submits the
edit, then someone (maybe the vandal) gets into the branch of
Article::view that uses PoolWorkArticleView, then ClueBot comes along
and reverts before the PoolWorkArticleView actually executes. Once that
PoolWorkArticleView actually does execute, it overwrites the parser
cache entry from ClueBot's revert with the one from the old edit.

To detect this sort of thing, let's include the revision id in the
parser cache entry and consider it expired if that doesn't match. Which
makes sense to do anyway.

And for good measure, let's have PoolWorkArticleView not save to the
parser cache if !$isCurrent.

Bug: 46014
Change-Id: Ifcc4d2f67f3b77f990eb2fa45417a25bd6c7b790
2014-04-01 12:15:34 -04:00
jenkins-bot
c131c00ea5 Merge "Document parser cache key control." 2014-02-08 19:40:57 +00:00
Siebrand Mazeland
0eb2e0c6d8 Update documentation for ParserCache::save()
The second parameter is a WikiPage, not an Article. Rename the variable
to $page to avoid confusion.

Change-Id: I4655aa9d8c7469805362c8ec43559bb5f9be4b06
2014-01-02 14:13:19 +00:00
tonythomas01
247b8ede25 Removed trailing whitespace from ParserCache.php
Follow up from Change I705a056665.

Change-Id: I31bdede4347b1d55be113e8ba7c44f3683c2ab46
2013-12-30 20:22:49 +00:00
tonythomas01
3006f76fdf Removed const ParserCache::try116cache
Version 1.16 is long ago and there seems no need to keep the const
ParserCache::try116cache in core.
Removed the if(self:: try116cache) block

Bug: 59127
Change-Id: I705a056665441f81516a4dbb6fe317a44da91d43
2013-12-31 00:36:58 +05:30
daniel
22e75a7ca7 Document parser cache key control.
This documents how implementations of the Content interface
can control the parser cache key by calling ParserOutput::recordOption()

Change-Id: I9e8ad1e33e06aceea6e63c7d5575679f84ce4004
2013-12-03 12:56:29 +01:00
Siebrand Mazeland
3f968db2cc Update some documentation in CacheTime and ParserCache
Change-Id: I31be0d4639f2a7189e15b622b34595564fad2a3d
2013-11-17 20:25:37 +00:00
Aaron Schulz
9750150c2e Reduce chance for parser cache race conditions
* Move cache time definition to happen before parse so it is less likely
  to override the cache with a stale value and a high cache timestamp.

bug: 46014
Change-Id: I021d27885ac260d89232ff11ec3cccf976e44f0f
2013-09-24 13:57:51 -07:00
umherirrender
6c38a5eb72 Fixed spacing in logging/parser/profiler/rl/revdel/search folder
Added spaces before if, foreach
Added some braces for one line statements

Change-Id: I11bbcfa351e945b7bde10c2105d61a3cf5622205
2013-04-20 17:38:24 +02:00
umherirrender
570dda0455 fix some spacing
Change-Id: I88e73d47a552918880514d88a876296a6cb80d88
2013-02-04 20:09:18 +00:00