Commit graph

19 commits

Author SHA1 Message Date
Aaron Schulz
8db4c163e8 objectcache: pass in the $oldValue as-of time in getWithSetCallback()
This lets callers use adaptive TTLs on the near-expiration
preemptive refreshes if the new and current values match,
using the as-of time as $mtime.

Change-Id: Ie541c35f890c9f789d1accf9f2a43506daaf31f0
2016-09-08 18:36:25 +00:00
Aaron Schulz
5316e7d700 objectcache: add "pcGroup" option to WANObjectCache::getWithSetCallback()
This lets callers have their own process cache but keep it
managed in a central location, making it easier to reset
when DB transaction snapshot are flushed or when unit tests
want to clear caches between runs.

Use this to replace the Revision text process cache.

Change-Id: Ic61ee9140d4ce9836cc4650adb5bb75a291fea18
2016-09-07 16:35:15 -07:00
Aaron Schulz
3348200473 Make adaptiveTTL() less strict about $mtime type
Callers using wfTimestamp( TS_UNIX, ... ) where getting $minTTL
due to the output being a string number.

Change-Id: I6b67a941940f40ef9a543f11d0dbccacafaaa53b
2016-09-07 09:18:37 -07:00
Aaron Schulz
52511952de objectcache: add and use adaptiveTTL() method
* This better handles delayed/lost cache purges by
  having lower TTLs for entries that often changes.
* Use this for foreign upload description page caches,
  we purges are never received from the source wiki.
* Also use this for User and LocalFile cache TTLs.
* Also move the Database::getCacheSetOptions() call in
  User *before* doing the queries, which is preferred.
* Fixed some IDEA errors too, like the undeclared
  mApiBase field.

Change-Id: I70f8ebb29ac853c2a530d9eedb9e7facc1b7b710
2016-08-31 03:40:56 +00:00
Aaron Schulz
70547f3fa3 objectcache: add mcrouter support to WANObjectCache
* Update documentation about relay methods.
* Change interim key set() to add() to avoid broadcasting it.
* Remove the behavior of doing purges synchronously in the
  local DC first before relay. In both the event relayer and
  mcrouter case, they will be asynchronous. It was hardly
  even possible to use such behavior since loads come from
  slave DBs, which do not see changes right after COMMIT.

Bug: T97562
Change-Id: I7759c82ae6e1b72fc227882a99c9a712a46374f6
2016-08-18 22:42:11 +00:00
Aaron Schulz
1a081c6707 objectcache: Add "busyValue" option to WANObjectCache::getWithSetCallback
This is useful for avoiding stampedes in the one case that lockTSE
does not alone cover, which is when the key does not exist or is
tombstoned.

Also avoid saving interim values unless the key is tombstoned
since there is no point in doing that otherwise.

Change-Id: I70997e90217a0979e0589afa7a5107b0e623c7cf
2016-07-22 22:24:38 +00:00
Aaron Schulz
fd4940a1b6 Fix WAN cache stashing bug due to missing list() call
Fixes regression from 0a38dbc809.

Bug: T137877
Change-Id: I4ea960966c084ada36dc54d4ec8729126c17ccdc
2016-06-15 09:40:34 -07:00
Aaron Schulz
0a38dbc809 objectcache: Support key versioning in WANObjectCache
* getWithSetCallback() takes a 'version' parameter.
* If the value at a key has a different version, then
  getWithSetCallback() will automatically use a separate
  key. Which value "wins" the main key does not matter.
* Purges are handled by using the main key as a sort of
  check key (with no hold-off). Note that this key is always
  purged on delete().
* Changed stash keys to track the same info as other keys
  both for consistency and because this change needs the
  generation timestamp. Renamed the stash prefix to avoid
  corrupt results with Het Deploy.
* This is useful for things like the User class that use
  versioning and have cross-wiki key access and purges.
  Currently, bumps to version must be deployed to all wikis
  at once, which this aims to avoid.

Change-Id: I26ae62f116e32b48bcf06bc13f8b9e79ae976745
2016-05-25 18:03:21 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Aaron Schulz
ba06ccb394 Fix get()/getMulti() check key race condition in WANObjectCache
If a set() happened around the exact same time as check key was
initialized via get(), the curTTL in future get() calls could
sometimes be 0 instead of a negative value, even though hold-off
should apply.

Change-Id: Ide1fd65112aff425a4798e2ec406d71f2a8e84a7
2015-12-04 14:46:18 -08:00
Aaron Schulz
2ca76c62d0 Remove redundant testGetMulti() assertion
Change-Id: Iadbf39de9c6be2baeb7c2f7c1dce39e9f2bed2e4
2015-12-04 21:03:09 +00:00
Aaron Schulz
680f152715 Add 0 hold-off TTL support to WANObjectCache::delete()
This removes the peculiar >= 1 restriction

Change-Id: I255dc6ef9750ad2ed6234e3fcfed9ae22a038748
2015-11-23 21:46:36 -08:00
Aaron Schulz
545ffd2d4c Fudge testGetWithSeveralCheckKeys() in both directions to be deterministic
Change-Id: I0f25cbd4555033f492b1b5d4ab55669240f19fc8
2015-11-20 14:21:31 -08:00
Timo Tijhof
a0cce5e4b6 objectcache: Implement check keys per cache key in WANObjectCache::getMulti()
To allow batch queries for multiple keys that themselves have different check
keys. Previously check keys always applied to all keys being retrieved.

Change-Id: I9e5ba198d79020ce05a802a510762e29fcfb2f1b
2015-11-17 04:15:33 +00:00
Aaron Schulz
6ae5dde077 objectcache: Add more WAN cache check key unit tests
Change-Id: If9bfef463b294391a456c5f04f48858f9dbe73fd
2015-11-17 03:28:18 +00:00
Timo Tijhof
6c3b45d4d3 objectcache: Add more @covers to WANObjectCache unit tests
Has only ~ 30% coverage according to PHPUnit, but mostly due to
lack of @covers annotations.

Change-Id: I871b45d8d8ee22c7cda644c450d78f577584e78c
2015-11-16 21:49:52 +00:00
Aaron Schulz
6d0108d533 Fix slow callbacks in getWithSetCallback() using lockTSE
* Keys that take a long time to generate would run into
  the MAX_SNAPSHOT_LAG check and have set() no-op. This
  would be fine except that leaves no key there to figure
  out the time since expiry and therefore whether to use
  the mutex, so it didn't. This now saves the keys but with
  a FLG_STALE bit set, making the next caller that sees it
  perform a regeneration (unless it can't get the mutex).
* Bumped LOCK_TTL so that keys can stay locked much longer.
* This is easy to test via sleep(5) in tagUsageStatistics()
  and two Special:Tags browser tabs.

Bug: T91535
Change-Id: I549e70ace3d2e7da40d3c4346ebacc36024cd522
2015-10-28 23:31:37 +00:00
Aaron Schulz
05a3ee2468 Make WAN cache HOLDOFF_TTL smaller by combining db/snapshot lag
* In the common ~0 second lag case, transactions up to ~7 seconds
  long will have normal set() behavior (instead of just ~5 seconds).
* Like wise with ~0 second transactions tolerating ~7 seconds of
  lag (instead of just ~5).
* The lower hold-off time lets caching resume 3 seconds sooner.

Change-Id: I21e2a0a6915492cec422b6a6af5541937745c15b
2015-10-27 11:38:54 -07:00
Aaron Schulz
cce813a922 Move MultiWriteBagOStuff to /libs
Also moved related tests files to /libs.

Change-Id: I806eeaa30205733d497adde933baf0c4157f7aae
2015-10-24 12:15:26 -07:00
Renamed from tests/phpunit/includes/objectcache/WANObjectCacheTest.php (Browse further)