Commit graph

132 commits

Author SHA1 Message Date
Umherirrender
3f1a52805e Use short type bool/int in param documentation
Enable the phpcs sniffs for this and used phpcbf

Change-Id: Iaa36687154ddd2bf663b9dd519f5c99409d37925
2017-08-20 13:20:59 +02:00
Umherirrender
ace44e2064 Use correct variable name in @param documentation
For some varargs a variable name is added with suffix ,... as seen for
many other varargs

Some @param are swapped, because there are in the wrong order

Enable Sniff MediaWiki.Commenting.FunctionComment.ParamNameNoMatch

Change-Id: I60fec6025bce824d5c67563ab7b65ad6cd628ad8
2017-08-11 19:27:19 +02:00
Umherirrender
a9007e8baf Add missing & to @param documentation to match functon call
Change-Id: I81e68310abcbc59964b22e0e74842d509f6b1fb9
2017-08-11 18:47:46 +02:00
Aaron Schulz
82e2c924e4 Remove "@author Aaron Schulz" annotations
Bug: T139301
Change-Id: Ib5248e8e27d60611c7373bce4b29dd5e85aa3489
2017-06-27 15:24:14 -07:00
jenkins-bot
95b63ec8a1 Merge "objectcache: Use faster getArrayCopy() in WAN cache methods" 2017-06-10 15:38:30 +00:00
Aaron Schulz
914274c58c objectcache: Use faster getArrayCopy() in WAN cache methods
Change-Id: I5f0543a59bdc59656c54fab4d7c4ae33fd233f9a
2017-06-10 15:26:37 +00:00
Aaron Schulz
cceefaadee Remove unused key in WANObjectCache::newEmpty()
Change-Id: I5d11736b95fbba1debcf80f80cbdb4753caf15f7
2017-06-08 07:27:14 -07:00
Aaron Schulz
83fcb86dfa objectcache: add getMultiWithUnionSetCallback() method
This supports callbacks that fetch all the missing values at once.

Change-Id: I74747cc06f97edc9163178180597e6651743b048
2017-05-30 23:34:28 +00:00
Aaron Schulz
6ddaf52d32 objectcache: respect process cache in getMultiWithSetCallback()
Backend cache queries are now avoided if they are fresh in memory.

Also factor out some code into private methods for clarity.

Change-Id: Ib343fde3dbf63f39e8cb09eca6278811a7d9738b
2017-05-26 15:34:45 -07:00
Aaron Schulz
0c24cefa12 objectcache: fix cache warmup bug in getMultiWithSetCallback()
The warmup cache was not properly prefixed and was also using the entity
IDs instead of the cache keys. Thus, it effectively just wasted a
getMulti() query and resulted in the usual separate GETs anyway.

Added some unit tests for this.

Change-Id: I75b7a31214b515511856f9d95db32e8881d80ccc
2017-05-26 04:28:13 +00:00
Aaron Schulz
cbff84a5b9 Add missing ampersand to getMultiWithSetCallback() callback
Also fixed a php example block comment

Change-Id: Ib6967180ad0d45aec7b7d4b6d383723e2c95fefa
2017-05-25 03:40:26 -07:00
Aaron Schulz
d46d8bb58d Add dynomite comments to WANObjectCache
Change-Id: I7ac9961e6acdecf90b766643138bb64b5ea9ec4a
2017-03-04 11:31:11 -08:00
Aaron Schulz
2e5eb693de objectcache: add WANObjectCacheReaper for assuring purges
* This fixes keys based on some sort of change log.
  Updates are wrapped in a mutex and keep track of the
  last known good position.
* Make WANObjectReapUpdate class that cleans up title
  related keys using the recentchanges table. This triggers
  as a deferred updates on RC view.

Change-Id: I7f14b9ca2533032147e62b1a3cc004a23da86579
2017-01-27 02:21:06 +00:00
jenkins-bot
2116f0926b Merge "Reduce WAN object cache logging spam for reject set() calls" 2017-01-10 22:00:13 +00:00
jenkins-bot
8b3265736f Merge "Add an refresh probability comment to worthRefreshPopular()" 2017-01-10 21:42:09 +00:00
Aaron Schulz
8972104ba2 Reduce WAN object cache logging spam for reject set() calls
Change-Id: I276c91b3b3b47d337f9c1d91a8348ca8f3d57194
2017-01-10 13:20:02 -08:00
Aaron Schulz
b0e0f6e3b3 Add an refresh probability comment to worthRefreshPopular()
Change-Id: Ia6717881c7d5c19481c87ade5972c0ff641d6e11
2017-01-07 19:00:00 -08:00
Aaron Schulz
02d6133fd8 Avoid IDEA warning about $keys being both a parameter and loop variable
Change-Id: Id562053a71f135fa863cb88b5f081982c01575cf
2017-01-07 16:10:23 -08:00
Hashar
4d487d1942 Revert "objectcache: detect default getWithSetCallback() set options"
Causes HHVM 3.12.7 to segfault and or:
Fatal error: Stack overflow in includes/libs/objectcache/BagOStuff.php on line 754

This reverts commit b47ce21cec.

Bug: T150833
Change-Id: Ia52fdf65f5e386c0eaa1046328680fb5c8a9081a
2016-11-16 10:35:27 +00:00
Aaron Schulz
b47ce21cec objectcache: detect default getWithSetCallback() set options
This works by setting a callback to return the cache set
options. The callback will watch DB reads and create a
merged result from said usage.

This handles callers that are missing getCacheSetOptions().

Change-Id: Ia264f011e45e8cf105480955dad7e2c4c2357b73
2016-11-16 04:53:53 +00:00
Aaron Schulz
43ff2a83b5 objectcache: avoid using process cache in nested callbacks
Because the process cache can be lagged by virtue of blind TTL,
the HOLDOFF_TTL might not be enough to account for it, so avoid
using it when already inside a callback.

Also split of the tests from the MediaWiki test class, so this
does not require DB access anymore.

Change-Id: I743a1233a5efc7f036fad140a9ff8a30b32f8f27
2016-10-20 22:13:29 +00:00
Aaron Schulz
fdce245e9f objectcache: use a default "since" timestamp in getWithSetCallback()
* Renamed mDoneWrites to be clearer at what type it is.
* Also cleaned up a few callers of this method

Change-Id: I45856b210c289c2e2f193cc4328a208e20b4e0a8
2016-10-14 23:31:09 +00:00
Aaron Schulz
ec0b9d55c3 Fix WANObjectCache doc typo
Change-Id: I74ef6598d86bdf43ae35f776878cfbb8e9695907
2016-10-13 01:15:12 -07:00
Aaron Schulz
24200e88d2 objectcache: add WANObjectCache::getMultiWithSetCallback()
This does what it says on the tin, e.g. a way to fetch
multiple keys at once. Callbacks are still called in a
serial manner when needed.

Change-Id: I8e24a6de7f46499a53ec41636c5a4f106b9b3d09
2016-10-05 15:50:18 +00:00
Aaron Schulz
ad0f205feb Add more version notes to WANObjectCache
Change-Id: Ia1b74b568d2b5137c7096a3b4549ac2cd4c7fa93
2016-09-09 13:41:03 -07:00
Aaron Schulz
843c112884 objectcache: add "staleTTL" option to WANObjectCache::set()
Change-Id: Ib9386a8ad4ab98d07d0a3784e3afffeeca9bbd6e
2016-09-09 01:05:55 -07:00
Aaron Schulz
55b4d2f620 Cleanup and re-indent some WANObjectCache docs
Change-Id: Ibf7092297ce53fd21e478a6f87a16f9b4373ad85
2016-09-09 00:57:57 -07:00
Aaron Schulz
6d1d6c90f1 objectcache: expose "minTime" in WAN cache and call it "minAsOf" instead
Change-Id: Iabb639dddb9458d961dbbaee6a0dfb2323c2739a
2016-09-08 13:42:17 -07:00
Aaron Schulz
8db4c163e8 objectcache: pass in the $oldValue as-of time in getWithSetCallback()
This lets callers use adaptive TTLs on the near-expiration
preemptive refreshes if the new and current values match,
using the as-of time as $mtime.

Change-Id: Ie541c35f890c9f789d1accf9f2a43506daaf31f0
2016-09-08 18:36:25 +00:00
Aaron Schulz
b9f4168b56 objectcache: Avoid doxygen misformatting for pcGroup comments
Change-Id: I970bc9ccb04b046c563a357d87f814da652e23d2
2016-09-08 01:22:31 +00:00
Aaron Schulz
5316e7d700 objectcache: add "pcGroup" option to WANObjectCache::getWithSetCallback()
This lets callers have their own process cache but keep it
managed in a central location, making it easier to reset
when DB transaction snapshot are flushed or when unit tests
want to clear caches between runs.

Use this to replace the Revision text process cache.

Change-Id: Ic61ee9140d4ce9836cc4650adb5bb75a291fea18
2016-09-07 16:35:15 -07:00
James D. Forrester
960e4d09ca Documentation: Replace misuse of 'later' when we meant 'latter'
Change-Id: I01bfdb8e0bfaa132533732873f7c840cea7935eb
2016-09-07 13:13:42 -07:00
Aaron Schulz
3348200473 Make adaptiveTTL() less strict about $mtime type
Callers using wfTimestamp( TS_UNIX, ... ) where getting $minTTL
due to the output being a string number.

Change-Id: I6b67a941940f40ef9a543f11d0dbccacafaaa53b
2016-09-07 09:18:37 -07:00
jenkins-bot
c118951f57 Merge "objectcache: add "hotTTR" and "ageNew" options to getWithSetCallback()" 2016-09-07 07:04:59 +00:00
Aaron Schulz
afcc7d772b objectcache: add "hotTTR" and "ageNew" options to getWithSetCallback()
These control worthRefreshPopular(), which will try to make sure that
heavily used keys are kept more consistent than others.

Change-Id: I2d351b50462a96d0fa8ca8fa6d28d51d3a5c94ff
2016-09-07 06:40:43 +00:00
Aaron Schulz
950cf6016c Rename DB_SLAVE constant to DB_REPLICA
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.

The old constant is an alias now.

Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
2016-09-05 22:55:53 -07:00
Aaron Schulz
16266edff3 Change "slave" => "replica DB" in /includes
Change-Id: Icb716219c9335ff8fa447b1733d04b71d9712bf9
2016-09-05 21:01:01 +00:00
Aaron Schulz
52511952de objectcache: add and use adaptiveTTL() method
* This better handles delayed/lost cache purges by
  having lower TTLs for entries that often changes.
* Use this for foreign upload description page caches,
  we purges are never received from the source wiki.
* Also use this for User and LocalFile cache TTLs.
* Also move the Database::getCacheSetOptions() call in
  User *before* doing the queries, which is preferred.
* Fixed some IDEA errors too, like the undeclared
  mApiBase field.

Change-Id: I70f8ebb29ac853c2a530d9eedb9e7facc1b7b710
2016-08-31 03:40:56 +00:00
Aaron Schulz
d110a67082 Clarify some WANObjectCache docs
Change-Id: I819a630ebba4122a757f04fbf702122069af4fb1
2016-08-18 20:53:20 -07:00
jenkins-bot
b33cbac370 Merge "objectcache: add mcrouter support to WANObjectCache" 2016-08-19 02:57:18 +00:00
Aaron Schulz
70547f3fa3 objectcache: add mcrouter support to WANObjectCache
* Update documentation about relay methods.
* Change interim key set() to add() to avoid broadcasting it.
* Remove the behavior of doing purges synchronously in the
  local DC first before relay. In both the event relayer and
  mcrouter case, they will be asynchronous. It was hardly
  even possible to use such behavior since loads come from
  slave DBs, which do not see changes right after COMMIT.

Bug: T97562
Change-Id: I7759c82ae6e1b72fc227882a99c9a712a46374f6
2016-08-18 22:42:11 +00:00
Aaron Schulz
6da50011e4 objectcache: expand getWithSetCallback() docs a bit
Change-Id: Ifc8304848fd5e4b8f7ea5555271de09609cdbe10
2016-08-15 02:07:31 -07:00
Aaron Schulz
5f921702d2 Avoid stack overflow in LoadBalancer with CACHE_DB WAN/server cache
* Add the ability to expose key BagOStuff attributes like whether the
  emulation SQL cache is being used. Several callers use hacks to detect
  this and can be updated later.
* Fallback to a safe empty WAN cache in the CACHE_DB case. This fixes
  a regression from f4bf52e843.
* Also add this protection to the server cache, which could break
  similarly before too.
* Also fix CACHE_ANYTHING by avoid the recursion risk from
  fc1d4d7960 by just checking the disabled service map.

Bug: T123829
Bug: T141804
Change-Id: I17ee26138f69e01ec1aaddb55ab27caa4d542193
2016-08-02 13:06:49 +00:00
Aaron Schulz
cbd2f3937f objectcache: Add callable doc comments for WANObjectCache
Change-Id: I28a705ae12e065dc8a3921e33c873f9a7fbb7490
2016-07-25 18:40:39 +00:00
Aaron Schulz
1a081c6707 objectcache: Add "busyValue" option to WANObjectCache::getWithSetCallback
This is useful for avoiding stampedes in the one case that lockTSE
does not alone cover, which is when the key does not exist or is
tombstoned.

Also avoid saving interim values unless the key is tombstoned
since there is no point in doing that otherwise.

Change-Id: I70997e90217a0979e0589afa7a5107b0e623c7cf
2016-07-22 22:24:38 +00:00
Aaron Schulz
fd4940a1b6 Fix WAN cache stashing bug due to missing list() call
Fixes regression from 0a38dbc809.

Bug: T137877
Change-Id: I4ea960966c084ada36dc54d4ec8729126c17ccdc
2016-06-15 09:40:34 -07:00
Aaron Schulz
6aaf0f9359 Avoid getWithSetCallback() warnings on unversioned key migration
If an unversioned key is present, this generated a warning
previously.

Bug: T137244
Change-Id: I92df9b9fe0436348b5b27e684d239238d57c9b5f
2016-06-08 10:25:25 -07:00
Aaron Schulz
e30b0199ac Fix key prefix comment after 0a38dbc809
Change-Id: I9085d1f0020917b431d9ef7afd4de93e54df0abc
2016-05-31 22:27:46 -07:00
Aaron Schulz
0a38dbc809 objectcache: Support key versioning in WANObjectCache
* getWithSetCallback() takes a 'version' parameter.
* If the value at a key has a different version, then
  getWithSetCallback() will automatically use a separate
  key. Which value "wins" the main key does not matter.
* Purges are handled by using the main key as a sort of
  check key (with no hold-off). Note that this key is always
  purged on delete().
* Changed stash keys to track the same info as other keys
  both for consistency and because this change needs the
  generation timestamp. Renamed the stash prefix to avoid
  corrupt results with Het Deploy.
* This is useful for things like the User class that use
  versioning and have cross-wiki key access and purges.
  Currently, bumps to version must be deployed to all wikis
  at once, which this aims to avoid.

Change-Id: I26ae62f116e32b48bcf06bc13f8b9e79ae976745
2016-05-25 18:03:21 +00:00
jenkins-bot
f3d45a2d72 Merge "Document WAN cache FLD_* constants" 2016-05-12 19:44:17 +00:00