Commit graph

271 commits

Author SHA1 Message Date
Aaron Schulz
1fa1235d32 Clean up BagOStuff::get() interface
* Callers of get() no longer have to contend with
  the annoying $casToken parameter, which is there
  but totally unusable to non-BagOStuff code.
* The default get() now delegates to doGet(),
  which callers must implement instead. They can
  ignore the overhead of generating $casToken if
  they do not implement cas(), which applies to
  callers that use the stock merge(). If cas() is
  used for merge(), then getWithToken() must be
  implemented.
* Also add BagOStuff::READ_LATEST to mergeViaCas()
  for sanity, as that missing before.
  Likewise with mergeViaLock().

Change-Id: I4efce6a9ab4b1eadd2f161dff641004a7239c516
2015-10-07 02:54:57 +00:00
Aaron Schulz
7c07943eab Update more docs and type hints to use IDatabase
Change-Id: I8c8d85b32a8aba21e14d2a2dde4c25eb509186c1
2015-10-06 18:49:52 -07:00
jenkins-bot
480f4e6f75 Merge "objectcache: Add BagOStuff::READ_VERIFIED flag to get()" 2015-10-05 23:13:24 +00:00
Aaron Schulz
54991403ff objectcache: Add BagOStuff::READ_VERIFIED flag to get()
* This lets multiwrite backends upgrade cached items
  to higher tiers using UPGRADE_TTL.
* This is useful for memcached/sql tiers or apc/memcached.

Change-Id: I34b30ce8b54f8de36429d48c80f6768aed310272
2015-10-05 16:03:00 -07:00
Aaron Schulz
a88df43d3f Database debug log cleanup (remove wgDebugDumpSqlLength/wgDebugDBTransactions)
* Simplify the debug log call and use queries group
* Remove $wgDebugDumpSqlLength, as profiler output
  already has shortened query strings (one can use
  profiling without DBO_DEBUG)
* Removed $wgDebugDBTransactions as BEGIN/COMMIT already show
* Removed PostgresTransactionState as it was only used for
  $wgDebugDBTransactions handling
* This cuts down on lots of global variable usage

Change-Id: I185adb1694441d074dea965960429b4910727620
2015-10-05 12:47:09 -07:00
Marius Hoch
e6e4ffd2f4 Fix ObjectCache::newAccelerator with string fallback in PHP 5.3
Because in PHP5.3:

php > $a = "hash";
php > echo isset( $a['fallback'] );
1
php > echo $a['fallback'];
h

This will fix using MediaWiki with MySQL on PHP5.3
if neither APC, xcache nor wincache are available.

Change-Id: Iebf034be75b282e2654cd298713455caf062eda4
2015-10-03 19:15:33 +02:00
Timo Tijhof
9afd767226 objectcache: Implement 'fallback' param for newAccelerator()
This way it can be used in $wgObjectCaches using 'factory', whilst
also specifying a fallback.

Change-Id: I8d4de1e2a394e7e915513fae91808fe32a21f75d
2015-10-02 12:01:38 -07:00
jenkins-bot
9dfb45cc7c Merge "Fix Memcached key decode" 2015-09-30 00:43:19 +00:00
Matthias Mullie
8ca796ea99 Fix Memcached key decode
Flow had a key: flowdb:flow_ref:wiki:by-source:v3:Parser\'s_"broken"_+_(page)_&_grill:testwiki:1:4.7
the '+' in there was not being encoded (it only does /[\x00-\x20\x25\x7f]+/)
but coming back, it was decoded into ' '.
getMulti() shows a key=>value array or results. Since key was different,
we couldn't find what we had requested.

Bug: T110326
Change-Id: Ia92edd73d0eb7fe0d35e38e7e7af8174fb85cbcc
2015-09-28 19:37:53 +02:00
Amir E. Aharoni
c6f066b88c Make long lines shorter to pass phpcs in 6 files
Bug: T102614
Change-Id: I3159c33ddd16bded94a70f363258a294f407fc58
2015-09-26 17:07:06 +00:00
jenkins-bot
8aafaa30df Merge "Fixes for RedisBagOStuff when using twemproxy" 2015-09-22 14:48:08 +00:00
Aaron Schulz
6303f0396f Merge "Cleaned up $asyncWrites member variable in MultiWriteBagOStuff" 2015-09-21 18:55:15 +00:00
Aaron Schulz
551940b382 Cleaned up $asyncWrites member variable in MultiWriteBagOStuff
Change-Id: I27e48aef77ac0f6453b7837f2b2686b7b5b726d4
2015-09-21 11:41:44 -07:00
Ori Livneh
481d5a3221 MultiWriteBagOStuff: require nonempty 'caches' param
MultiWriteBagOStuff::__construct() barfs if the 'caches' parameter is unset,
but it permits it to be an empty array. This means that subsequent operations
that need to reference the first (primary) cache backend need to check that
it is there at all.

Since there is not much sense in having a MultiWriteBagOStuff object with zero
cache backends, make the constructor require a nonempty array for 'caches', and
remove the subsequent checks that are made redundant by this new restriction.

Change-Id: I30f3e0dcbfe67570a368e64b8233cc0ba7f90b2f
2015-09-21 11:34:15 -07:00
Aaron Schulz
fd8e85fc2d Support async writes to secondary MultiWriteBagOStuff stores
* This is useful for ParserCache, as it tries to focus on memcached
  and use other caches (e.g. mariadb) for the long-tail of less used
  content, as setup on WMF. The class uses BagOStuff in a way that is
  compatible with this approach.

Bug: T109751
Change-Id: Ia64eb44a9b52a988fde27b468d604d9163bed4b4
2015-09-18 21:35:43 -07:00
Aaron Schulz
45fea5f477 Fixes for RedisBagOStuff when using twemproxy
* Use the nx/ex flags for the redis SET method
  to implement add() correctly. This also handes a
  prior FIXME comment.
* Made merge() work via locking instead of cas()
  since Redis::MULTI does not work with twemproxy.
  Locking, which uses add(), does, and works better
  than it did before.
* Removed some pointless newlines.

Change-Id: I652bd0ad2c594097d2cb1ab77f291e8bd27ad14f
2015-09-16 10:20:38 -07:00
Matthias Mullie
58c81f5b86 Decode Memcached keys before returning getMulti result
Keys are sent to Memcached encoded. However, getMulti()
will respond in [key => value] format. The keys it
responds with should not be the encoded versions, or
callers won't be able to map them to the results.

Bug: T111138
Change-Id: I0d821b1219a492be8e93453f0249c78f18e24533
2015-09-07 05:43:49 +00:00
Aaron Schulz
2cdcea5c9e Improved getMainStashInstance() docs
Change-Id: I49e5c112478927387244214917fb06907c22186f
2015-08-28 16:24:13 -07:00
jenkins-bot
f0c88b2163 Merge "Added reentrant lock support to BagOStuff" 2015-08-25 01:28:24 +00:00
Aaron Schulz
c9b1fe1896 Added reentrant lock support to BagOStuff
* Also fixed HashBagOStuff::delete() return value for non-keys

Change-Id: I9a977750c6fc6b8406bc1c9366a6b1b34bf48b6b
2015-08-25 01:17:45 +00:00
Timo Tijhof
62e3da7e58 objectcache: Document the primary entry points people should use
There are imho too many ObjectCache methods, $wg*Cache* variables,
CACHE_ types, and wfGet*Cache() functions to know which ones
should actually be used publicly. This should make it easier to
know which ones should be used.

The difference between them was also harder to understand with the
main documentation sections about each of these living in
conceptually different places:
- classes (WANObjectCache)
- methods (ObjectCache::getMainStashInstance)
- variables (DefaultSettings: wgMainCacheType, wgMainWANCache, wgMainStash)

Also:
* Correct wfGetCache() documentation.
* Add the missing keywords for 'public' visibility.

Change-Id: I3b05fdb8b7888bf7e3f05bdca36538f3484556b8
2015-08-25 03:11:15 +02:00
Timo Tijhof
79ce51d1c9 objectcache: Make first parameter of newAccelerator optional
Makes it more convenient to use.

Change-Id: I1e11f7a759bd2816e47d1c2453cbe39b8f44b2f0
2015-08-24 23:43:19 +02:00
Aaron Schulz
4f0b16b914 Fixed BufferingStatsdDataFactory::timing() callers to use ms
* The interface actually demands this

Change-Id: I1e334c2696a8a8eca73a6ae7f71428190cad3107
2015-08-19 01:01:49 +00:00
Aaron Schulz
3bf391e842 Added statsd metrics for ObjectCacheSessionHandler
Change-Id: I86e967a027e2010f962831b55a9fceab02d8a19e
2015-08-13 13:04:28 -07:00
Aaron Schulz
08039b2df4 Restored I4afaecd8: "Avoiding writing sessions for no reason"
* After 203d2c9c11, the refresh updates should actually trigger

This reverts commit e48fec5a8a.

Change-Id: I8427ed5b3a5bb80033cbdf071a33f8a3999ecb97
2015-08-11 21:46:05 +00:00
jenkins-bot
e341ad3ec5 Merge "Fixed @var doc in MultiWriteBagOStuff" 2015-08-11 20:29:45 +00:00
Aaron Schulz
012db9afc2 Fixed @var doc in MultiWriteBagOStuff
Change-Id: I53384460b7a6ccda50a23e3ab1fc614198e82d1a
2015-08-11 13:10:28 -07:00
Ori Livneh
2d607c444b Fix argument handling in MultiWriteBagOStuff::get()
Follow-up for I7ea050a2ea.

Change-Id: Ibaa4cf01ee913c7a725d1719c00308ef3c6b0aff
2015-08-11 13:02:44 -07:00
Aaron Schulz
203d2c9c11 Moved ObjectCacheSessionHandler renewal logic to wfSetupSession()
* This should trigger more reliably than the shutdown function
  callback, which is really only there for sanity to make sure
  session close/write happens.

Change-Id: I9a1aa76de121ba8de33b3fa850bd223929fae404
2015-08-10 19:10:38 +00:00
Aaron Schulz
c3dec1193c Added BagOStuff READ_LATEST flag for replicated stores
Bug: T88493
Change-Id: I7ea050a2eabba635f2aadb4e33b6f8fbfb1b01a8
2015-08-04 16:03:37 -07:00
Ori.livneh
e48fec5a8a Revert I4afaecd8: Avoiding writing sessions for no reason
I4afaecd8 reduced the frequency at which the TTL of sessions keys is reset, on
the assumption that we were doing it a whole lot more often than necessary.
We can now assume that this assumption was false, because an uncommitted revert
of this change which I pushed to production earlier caused the rate of session
loss errors in the logs to plummet.

Bug: T102199
Change-Id: Ie67f4ca000afcf3d4f44155c13f91cd4c286866d
2015-07-31 23:30:11 +00:00
Matthew Flaschen
16c81325ed array_shift returns null on empty array, not false
Change-Id: I690f9dd0de11ff60f9cba9f401c6e456e7ba8bd6
2015-07-30 21:27:07 -04:00
Ori Livneh
1d526cae81 RedisBagOStuff: if no alternatives, skip master link status check
If RedisBagOStuff::getConnection() is able to establish a connection, only
check the master link status if automatic failover is enabled and if there are
other viable servers left to consider. If there are no servers left to
consider, or if automatic failover is not configured, just return the
connection handle without subjecting it to further tests.

This will have the side-effect of making RedisBagOStuff compatible with
Nutcracker, which does not implement the INFO command. This is because when
MediaWiki is configured to use Nutcracker, the server pool will consist of a
single server (namely, Nutcracker itself), and thus there will be no other
server to consider, so INFO will never be executed.

Change-Id: I3812ec5a0b22df122bdf44350bc0496574c02ce8
2015-07-28 16:22:57 -07:00
jenkins-bot
b95d41c62b Merge "objectcache: Ensure an integer put into memcached comes out an integer" 2015-07-09 14:20:24 +00:00
jenkins-bot
8195fd3551 Merge "Improved fail-over in ReplicatedBagOStuff for redis" 2015-07-08 06:44:42 +00:00
jenkins-bot
5de749d207 Merge "Made SqlBagOStuff set() wrap setMulti()" 2015-07-02 08:14:36 +00:00
Erik Bernhardson
508d8ee79b objectcache: Ensure an integer put into memcached comes out an integer
Some parts of resourceloader put an integer into the BagOStuff
and rejects any value it pulls back out that is not an integer.
This patch updates the memcached client behind the BagOStuff to
flag integer values and convert them back to integer when recieved.
The end result is shaving several seconds off the time it takes to
request a CSS or JS file in certain configurations.

Bug: 60563
Change-Id: Ie4dc8cf2c2cd232a2c167b4075ff4254b656bcbc
2015-07-01 03:51:55 +00:00
jenkins-bot
7bc770bbd8 Merge "Made SqlBagOStuff unset DB handles on connection errors" 2015-06-28 00:28:30 +00:00
Aaron Schulz
29dbdc3a64 Made SqlBagOStuff set() wrap setMulti()
* This also makes sure garbageCollect() is called on writes from
  time to time, since set() is used much more often

Change-Id: I5b6cc686bd6204fc3d6ff8b003be8b8d6a493e24
2015-06-24 13:16:50 -07:00
jenkins-bot
799ca9856b Merge "Made ReplicatedBagOStuff wrapping the SQL class the default stash" 2015-06-24 20:09:53 +00:00
Aaron Schulz
4825b4c08e Made ReplicatedBagOStuff wrapping the SQL class the default stash
* To make this work, a slaveOnly flag was added to SqlBagOStuff
  and it no longer prunes expired items on get(), it just treats
  them as missing. The use of garbageCollect() for the writeFactory
  cache and the use of replace() on updates already makes this
  a non-issue.
* This also makes the default $wgMainStash non-bogus.

Bug: T88493
Change-Id: I9d1f31305e08430de29a3cd521cdb10a82dffd10
2015-06-24 19:37:03 +00:00
Aaron Schulz
674858617a Made SqlBagOStuff unset DB handles on connection errors
* A "gone away" error can trigger ping(), which can fail,
  triggering a full open(). If this fails (as it often will),
  mConn is false and the DatabaseBase object is not usable.

Bug: T103435
Change-Id: Id2aaed87d2642e73c7811076da908180ee5ab5a4
2015-06-23 12:28:03 -07:00
Aaron Schulz
646fdc978c Added pre-emptive session renewal to avoid "random" submission errors
* The session will be renewed on access if it will expire in 10 minutes
* This avoids a problem both with 532ef78 + PHP <= 5.5 or any PHP >= 5.6

Bug: T102199
Change-Id: I8cad100bc35e86fa95ad144bcad338f3cb5d82a1
2015-06-18 21:51:15 +00:00
umherirrender
d8821f2b0b Fixed spacing
- Removed space after casts
- Removed spaces in array index
- Added spaces around string concat
- Added space after words: switch, foreach
- else if -> elseif
- Removed parentheses around require_once, because it is not a function
- Added newline at end of file
- Removed double spaces
- Added spaces around operations
- Removed repeated newlines

Bug: T102609
Change-Id: Ib860222b24f8ad8e9062cd4dc42ec88dc63fb49e
2015-06-17 20:22:32 +00:00
Kunal Mehta
f6e5079a69 Use mediawiki/at-ease library for suppressing warnings
wfSuppressWarnings() and wfRestoreWarnings() were split out into a
separate library. All usages in core were replaced with the new
functions, and the wf* global functions are marked as deprecated.

Additionally, some uses of @ were replaced due to composer's autoloader
being loaded even earlier.

Ie1234f8c12693408de9b94bf6f84480a90bd4f8e adds the library to
mediawiki/vendor.

Bug: T100923
Change-Id: I5c35079a0a656180852be0ae6b1262d40f6534c4
2015-06-11 18:49:29 +00:00
Timo Tijhof
7b20e0ef7a objectcache: Minor code clean up in ObjectCache.php
* Whitespace.
* Simplify logic.
* Apply coding conventions to documentation blocks (empty line
  before annotations, no empty lines between annotation, consistent
  order of annotations).

Change-Id: I3e5268d6a6295643d5725c66ea2a01bccf610ed8
2015-06-10 04:53:41 +01:00
jenkins-bot
af2f7e00c3 Merge "Added ObjectCache::getMainStashInstance() and $wgMainStash" 2015-05-19 07:44:44 +00:00
Aaron Schulz
793d01401c Added ObjectCache::getMainStashInstance() and $wgMainStash
* This provides factory/config more appropriate for stashes
* Cleaned up some neighboring config comments while at it

Bug: T88493
Bug: T97620
Change-Id: Id370ee50be6e493a4700c858df863a183abc05fd
2015-05-18 11:39:14 -07:00
Aaron Schulz
42bf202b19 Added RedisBagOStuff FIXME comments
Change-Id: I6f4f85166584439a38d632356f3ed95e4fd4fd6f
2015-05-15 23:00:16 +00:00
Aaron Schulz
e45d07d7fd Improved fail-over in ReplicatedBagOStuff for redis
Bug: T96123
Change-Id: I3f37417a1a4f010b092a7718d43bfcdfe16e8f4c
2015-05-02 13:45:41 -07:00