Commit graph

57 commits

Author SHA1 Message Date
jenkins-bot
285080c311 Merge "objectcache: make RedisBagOStuff pass all tests" 2018-06-02 01:35:26 +00:00
jenkins-bot
966e34619b Merge "objectcache: add setMockTime() method to BagOStuff/WANObjectCache" 2018-06-01 13:19:24 +00:00
Aaron Schulz
acace9a049 objectcache: add setMockTime() method to BagOStuff/WANObjectCache
Change-Id: I3e5760814fb7dbe628eb0d979d690c3275fc3c15
2018-06-01 03:46:58 +00:00
Aaron Schulz
13f7232bf4 objectcache: make RedisBagOStuff pass all tests
* Provide a default lock-based BagOStuff::cas implementation
* Make RedisBagOStuff::merge() use mergeViaCas()
* Use the raw unserialized string as the redis CAS token to
  avoid any bad interaction with __wakeup() methods changing
  field values every time
* Make RedisBagOStuff::incr() return false when there is no
  such key, not null
* Rewrite merge() test to make the order of write/cas phase
  of the parent and child merge() calls well defined instead
  of arbitrary usleep() calls
* Avoid cache key reuse in test runs

Change-Id: I388ec173cf3858bb2fc7a8c8a00cda68703074ce
2018-05-31 20:43:10 -07:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
Aaron Schulz
f81853ed53 objectcache: add BagOStuff comment additions about access scope
Change-Id: Id23859a58ea3bde0338ba4d22ce12ffcbbf4480a
2018-05-17 05:23:24 +00:00
Aaron Schulz
93120703f6 objectcache: fix some makeKey/makeGlobalKey IDEA warnings
Change-Id: I5fefda9676a685167326c63c34b2b7df8be2e7cd
2017-11-18 19:57:26 -08:00
Aaron Schulz
317d8476da objectcache: Mention colons in BagOStuff key generation
Change-Id: I76967245c32aac4eea8d6672e4fc1fac5e5f3d07
2017-10-20 03:26:25 +00:00
Umherirrender
86463f3df6 build: Updating mediawiki/mediawiki-codesniffer to 0.12.0
Change-Id: Iefaae5043fa77d5d556c31079549dab8f61bd3ef
2017-09-10 21:11:37 +02:00
Umherirrender
3f1a52805e Use short type bool/int in param documentation
Enable the phpcs sniffs for this and used phpcbf

Change-Id: Iaa36687154ddd2bf663b9dd519f5c99409d37925
2017-08-20 13:20:59 +02:00
Umherirrender
ace44e2064 Use correct variable name in @param documentation
For some varargs a variable name is added with suffix ,... as seen for
many other varargs

Some @param are swapped, because there are in the wrong order

Enable Sniff MediaWiki.Commenting.FunctionComment.ParamNameNoMatch

Change-Id: I60fec6025bce824d5c67563ab7b65ad6cd628ad8
2017-08-11 19:27:19 +02:00
Umherirrender
a9007e8baf Add missing & to @param documentation to match functon call
Change-Id: I81e68310abcbc59964b22e0e74842d509f6b1fb9
2017-08-11 18:47:46 +02:00
Umherirrender
be42e09aa8 build: Prepare for mediawiki/mediawiki-codesniffer to 0.9.0
The used phpcs has a bug, so the version 0.9.0 could not be enforced at the moment.
Will be fixed in next version, see T167168

Changed:
- Remove duplicate newline at end of file
- Add space between function and ( for closures
- and -> &&, or -> ||

Change-Id: I4172fb08861729bccd55aecbd07e029e2638d311
2017-06-26 17:14:31 +00:00
Timo Tijhof
50a035cd6b Clean up get_class($this) -> static::class in /includes/cache and /includes/libs
* get_class()        -> __CLASS__ (same as self::class)
* get_called_class() -> static::class
* get_class($this)   -> static::class

Change-Id: I22c62851bcc88405f4bdd77258d6e28ec1a14924
2017-03-07 21:06:06 +00:00
Kunal Mehta
61adc1e146 Use namespaced ScopedCallback
The un-namespaced \ScopedCallback is deprecated.

Change-Id: Ie014d5a775ead66335a24acac9d339915884d1a4
2016-10-17 15:46:05 -07:00
Kunal Mehta
090d0267da Use wikimedia/wait-condition-loop
Since the WaitConditionLoop class was first introduced in 1.28 (current
master), no back-compat alias is added.

Bug: T146256
Depends-On: Ia84774d83da79fea1e167fe065c69549981f753b
Change-Id: Ibd4f15c87105b8caccbd1f661b74b6efa012b77f
2016-10-01 00:25:43 -07:00
Aaron Schulz
d32b64235f Switch some callers to WaitConditionLoop
Also fixed up backwards documentation

Change-Id: I00c36aa751a79ca86a754e049a6da78cbb417b81
2016-09-08 19:29:18 -07:00
Aaron Schulz
e99bb1b7dc objectcache: allow for callbacks to mask SYNC_WRITE latency
Change-Id: I908222ad3788ebe330aa58831cda139da32becd8
2016-09-01 13:44:18 -07:00
Aaron Schulz
55f6639c2e objectcache: Add changeTTL() method
This can change the TTL without fetching the object
so that keys can be renewed or set to expire soon.

Change-Id: Id1c2c9f89b3445cfa34263057dc5029cbe170833
2016-08-11 19:55:32 -07:00
Aaron Schulz
5f921702d2 Avoid stack overflow in LoadBalancer with CACHE_DB WAN/server cache
* Add the ability to expose key BagOStuff attributes like whether the
  emulation SQL cache is being used. Several callers use hacks to detect
  this and can be updated later.
* Fallback to a safe empty WAN cache in the CACHE_DB case. This fixes
  a regression from f4bf52e843.
* Also add this protection to the server cache, which could break
  similarly before too.
* Also fix CACHE_ANYTHING by avoid the recursion risk from
  fc1d4d7960 by just checking the disabled service map.

Bug: T123829
Bug: T141804
Change-Id: I17ee26138f69e01ec1aaddb55ab27caa4d542193
2016-08-02 13:06:49 +00:00
Kunal Mehta
4a8ae0a0b4 Use callable type-hint in a few places
Mostly places which immediately had a:
 if ( !is_callable( $callback ) ) {
  throw new Exception(...);
 }
check at the beginning of the function.

Change-Id: Ia78663b2231629010816bd1cda8814b996968d1d
2016-06-07 16:39:22 -07:00
Aaron Schulz
ac5512b48d Let BagOStuff::merge() callbacks override the TTL
This is useful if the TTL depends on what is in cache now.

Change-Id: Idde6eb9e156f9ae847f9bc7e3b1f3f511f39ffe4
2016-05-11 13:50:33 -07:00
Aaron Schulz
629f063bb9 Avoid spamming the duplicate key fetch log in BagOStuff::merge()
Change-Id: I68b10208f222f81f173f6e282c9ecf0bfcdde1fc
2016-04-29 12:03:50 -07:00
Bryan Davis
450bb6723d BagOStuff: Optionally log duplicate key lookups
Add new 'reportDupes' parameter to BagOStuff. This parameter enables scheduling
of a callback after the current web request ends. This callback will emit warning
log messages for all keys that were requested more than once.

The default ObjectCache factory for MediaWiki enables this option by default.
Not by default for plain BagOStuff instances, however. E.g. `new HashBagOStuff()`.

It also set 'asyncHandler' for all classes now (not just MultiWriteBagOStuff).

Bug: T128125
Co-Authored-By: Timo Tijhof <krinklemail@gmail.com>
Change-Id: I8a2b06cf54d2acf5950eed71756ecdf50e224be1
2016-04-22 04:21:20 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Ricordisamoa
60be0296c3 Stop doing $that = $this in includes/libs
Closures support $this as of PHP 5.4

Change-Id: I1b5a5d7e619029684cb8d2a8d150fcc13051c2e0
2016-02-10 18:26:25 +01:00
Brad Jorsch
25dbd91513 Clean up after Ie161e0f
Ie161e0f was done in a hurry, and so didn't do things in the best ways.
This introduces a new "CachedBagOStuff" that transparently handles all
the logic that had been copy-pasted all over in Ie161e0f.

The differences between CachedBagOStuff and MultiWriteBagOStuff are:
* CachedBagOStuff supports only one "backend".
* There's a flag for writes to only go to the in-memory cache.
* The in-memory cache is always updated.
* Locks go to the backend cache (with MultiWriteBagOStuff, it would wind
  up going to the HashBagOStuff used for the in-memory cache).

Change-Id: Iea494729bd2e8c6c5ab8facf4c241232e31e8215
2016-02-03 21:45:18 +00:00
Aaron Schulz
5ffaa6ae0d Avoid "Unable to set value to APCBagOStuff" exceptions
* This can happen due to incr/add races. Use incrWithInit()
  instead to handle such cases.
* Also made BagOStuff:incrWithInit() return the new value like incr().

Change-Id: I0e3b02a4cff7c20544a9db2eaabd3f61e5a470b1
2015-12-21 20:41:43 -08:00
Timo Tijhof
e8275758fe objectcache: Introduce IExpiringStore for convenient TTL constants
Also consistently use self:: instead of BagOStuff:: for constants
referenced within the BagOStuff class.

Change-Id: I20fde9fa5cddcc9e92fa6a02b05dc7effa846742
2015-10-28 04:07:25 +00:00
Ori Livneh
0c9fb12265 Escape colons in BagOStuff key segments
For the sake of safety and correctness, the following BagOStuff::makeKey()
invocations should return distinct keys:

   $cache->makeKey( 'ab:', 'cd' );
   $cache->makeKey( 'ab', ':cd' );

That is not currently the case, because while we use ':' as a key path
separator, we don't escape ':' in the input supplied to makeKey(). So, make
BagOStuff::makeKeyInternal() URL-encode colons.

To prevent this from messing up the max. key length calculations, reproduce
this logic in MemcachedBagOStuff::makeKeyInternal(), in lieu of having the
method call its parent.

Change-Id: I83ea7e7336a1c9e64aa42284c2517089a736efe5
2015-10-23 20:26:49 -07:00
Aaron Schulz
fb7022c54d Bound BagOStuff::lock() RTT estimate for sanity
The timestamp difference might be 0 (or in rare cases, less)

Change-Id: I0298b413df637d9da5ff33c078ad49dadc8a46ce
2015-10-21 20:14:54 -07:00
jenkins-bot
409e7d6a70 Merge "objectcache: Fix grammar error in BagOStuff::getWithToken doc" 2015-10-22 02:33:16 +00:00
Aaron Schulz
5c8ef13306 Add WRITE_SYNC flag to BagOStuff::set()/merge()
* This blocks on writing to all replicas
  and returns false if any failed.
* This is useful if ChronologyProtector is to work across
  domains by having the writes go everywhere so that later
  reads will see them (and be local at the same time).
* Redundant doc comments were also removed.

Change-Id: I9ed098d563c64dba605e7809bc96731da3b3e79d
2015-10-22 01:44:09 +00:00
Timo Tijhof
92b197dc59 objectcache: Fix grammar error in BagOStuff::getWithToken doc
Follows-up 1fa1235d32.

Change-Id: I19a4430609423f5ff32a1014ed21131175469adf
2015-10-22 02:35:35 +01:00
Aaron Schulz
d4895d5c70 Add make(Global)Key() method to WANObjectCache
Change-Id: I8e739fbe8614c9bd1c3595bbf16c8cd423aff7cc
2015-10-19 12:55:30 -07:00
Ori Livneh
6916548490 Add makeKey and makeGlobalKey to BagOStuff
* Add a string `keyspace` member to BagOStuff instances. The default
  implementation, meant for simple key/value stores, treats the key space
  as a string prefix to prepend to keys. By default, its value is `local`,
  but any instance created via ObjectCache::newFromParams() (or or one of
  its callers) will have that default to $wgCachePrefix / wfWikiID().
* Add `makeKey` and `makeGlobalKey` methods to the base BagOStuff class.
  These methods are not static to allow for BagOStuff types which require
  a configured instance to know the underlying storage engine's key semantics.
* Make wfMemcKey() and wfGlobalCacheKey() delegate to these methods on the main
  ObjectCache instance.

Change-Id: Ib7fc2f939be3decfa97f66af8c2431c51039905f
2015-10-13 09:32:48 -04:00
Aaron Schulz
b39fcc855c Add get() flags support to BagOStuff::getWithSetCallback()
Change-Id: Ib6046f5bc5d8c0481f90b56c381ba761fee558d6
2015-10-07 20:51:00 -07:00
Aaron Schulz
1fa1235d32 Clean up BagOStuff::get() interface
* Callers of get() no longer have to contend with
  the annoying $casToken parameter, which is there
  but totally unusable to non-BagOStuff code.
* The default get() now delegates to doGet(),
  which callers must implement instead. They can
  ignore the overhead of generating $casToken if
  they do not implement cas(), which applies to
  callers that use the stock merge(). If cas() is
  used for merge(), then getWithToken() must be
  implemented.
* Also add BagOStuff::READ_LATEST to mergeViaCas()
  for sanity, as that missing before.
  Likewise with mergeViaLock().

Change-Id: I4efce6a9ab4b1eadd2f161dff641004a7239c516
2015-10-07 02:54:57 +00:00
Aaron Schulz
355ba8530e objectcache: Add BagOStuff::getWithSetCallback() convenience method
Change-Id: I9cc162ff1cc48c1c500f2999327bd18ba235bfd0
2015-10-05 17:26:32 -07:00
Aaron Schulz
54991403ff objectcache: Add BagOStuff::READ_VERIFIED flag to get()
* This lets multiwrite backends upgrade cached items
  to higher tiers using UPGRADE_TTL.
* This is useful for memcached/sql tiers or apc/memcached.

Change-Id: I34b30ce8b54f8de36429d48c80f6768aed310272
2015-10-05 16:03:00 -07:00
Aaron Schulz
c9b1fe1896 Added reentrant lock support to BagOStuff
* Also fixed HashBagOStuff::delete() return value for non-keys

Change-Id: I9a977750c6fc6b8406bc1c9366a6b1b34bf48b6b
2015-08-25 01:17:45 +00:00
Kunal Mehta
221acd4ee4 BagOStuff: Don't try to access a protected variable in a closure
Follows up 78f1fee559.

Change-Id: Id356509cc55a5953351dfd98896cf01055739ee0
2015-08-23 15:31:36 -07:00
jenkins-bot
27155a28ea Merge "BagOStuff: Don't use $this in a closure" 2015-08-21 20:50:29 +00:00
Kunal Mehta
78f1fee559 BagOStuff: Don't use $this in a closure
Change-Id: Ib9fb92a70adc2b4d75c30148e17dff6f7765afe1
2015-08-21 20:41:11 +00:00
Aaron Schulz
7b997f4ae5 Fixed IDE error in mergeViaLock()
Change-Id: I50b5a22bf353272ad6960181db07c2f4ec857f3c
2015-08-21 20:37:02 +00:00
Aaron Schulz
d8931781e8 Fixed BagOStuff getScopedLock() docs
Change-Id: I4d6d3211e93ee1c279728ff787acff558a47fbee
2015-08-19 19:28:01 +00:00
Aaron Schulz
97c7a897c8 Avoid some possible deadlocks on account creation
* This uses a non-blocking $wgMemc lock to reserve the user
  name in question. This should prevent two threads from
  reaching LOCK IN SHARE MODE and getting stuck on INSERT.
  The lock is global to better cover auth plugins.
* This adds a BagOStuff::getScopedLock() convenience method.
  It uses less queries than LockManager by being EX only.
* Avoid extra lock attempt in lock() in non-blocking mode.
* Removed (un)lock() HashBagOStuff overrides that made it
  behave differently than other caches (wrt to re-entrance).

Bug: T106850
Change-Id: Iecf95206d712367f5d202f76ab0eaa9d7bdabf2b
2015-08-17 21:33:50 -07:00
Aaron Schulz
4065e4d1ab Made BagOStuff::merge() avoid retries on I/O errors
Change-Id: Ia2fd61132322ed3fa8909d60d9f7eb42e39cb443
2015-08-07 04:31:15 +00:00
Aaron Schulz
c3dec1193c Added BagOStuff READ_LATEST flag for replicated stores
Bug: T88493
Change-Id: I7ea050a2eabba635f2aadb4e33b6f8fbfb1b01a8
2015-08-04 16:03:37 -07:00
Aaron Schulz
4fe54ae2db Removed partly obsolete and redundant BagOStuff comment
Change-Id: I265912276af8f2bcdba4e49b4cc522df0a39a5db
2015-05-25 20:33:55 -07:00