Inject the DeferredUpdates::addCallableUpdate method via the
ObjectCache. This brings it closer to being able to move to /libs.
Change-Id: Ifa0d893002c3d709a4dc7346c263a92162274bd7
* Add a string `keyspace` member to BagOStuff instances. The default
implementation, meant for simple key/value stores, treats the key space
as a string prefix to prepend to keys. By default, its value is `local`,
but any instance created via ObjectCache::newFromParams() (or or one of
its callers) will have that default to $wgCachePrefix / wfWikiID().
* Add `makeKey` and `makeGlobalKey` methods to the base BagOStuff class.
These methods are not static to allow for BagOStuff types which require
a configured instance to know the underlying storage engine's key semantics.
* Make wfMemcKey() and wfGlobalCacheKey() delegate to these methods on the main
ObjectCache instance.
Change-Id: Ib7fc2f939be3decfa97f66af8c2431c51039905f
Provide a complement to ObjectCache::getMainWANInstance() and
ObjectCache::getMainStashInstance() which gets the default ObjectCache
instance.
Change-Id: Ib16ee40908b159e60be41a308db49a9291b5de0d
* Callers of get() no longer have to contend with
the annoying $casToken parameter, which is there
but totally unusable to non-BagOStuff code.
* The default get() now delegates to doGet(),
which callers must implement instead. They can
ignore the overhead of generating $casToken if
they do not implement cas(), which applies to
callers that use the stock merge(). If cas() is
used for merge(), then getWithToken() must be
implemented.
* Also add BagOStuff::READ_LATEST to mergeViaCas()
for sanity, as that missing before.
Likewise with mergeViaLock().
Change-Id: I4efce6a9ab4b1eadd2f161dff641004a7239c516
* This lets multiwrite backends upgrade cached items
to higher tiers using UPGRADE_TTL.
* This is useful for memcached/sql tiers or apc/memcached.
Change-Id: I34b30ce8b54f8de36429d48c80f6768aed310272
* Simplify the debug log call and use queries group
* Remove $wgDebugDumpSqlLength, as profiler output
already has shortened query strings (one can use
profiling without DBO_DEBUG)
* Removed $wgDebugDBTransactions as BEGIN/COMMIT already show
* Removed PostgresTransactionState as it was only used for
$wgDebugDBTransactions handling
* This cuts down on lots of global variable usage
Change-Id: I185adb1694441d074dea965960429b4910727620
Because in PHP5.3:
php > $a = "hash";
php > echo isset( $a['fallback'] );
1
php > echo $a['fallback'];
h
This will fix using MediaWiki with MySQL on PHP5.3
if neither APC, xcache nor wincache are available.
Change-Id: Iebf034be75b282e2654cd298713455caf062eda4
Flow had a key: flowdb:flow_ref:wiki:by-source:v3:Parser\'s_"broken"_+_(page)_&_grill:testwiki:1:4.7
the '+' in there was not being encoded (it only does /[\x00-\x20\x25\x7f]+/)
but coming back, it was decoded into ' '.
getMulti() shows a key=>value array or results. Since key was different,
we couldn't find what we had requested.
Bug: T110326
Change-Id: Ia92edd73d0eb7fe0d35e38e7e7af8174fb85cbcc
MultiWriteBagOStuff::__construct() barfs if the 'caches' parameter is unset,
but it permits it to be an empty array. This means that subsequent operations
that need to reference the first (primary) cache backend need to check that
it is there at all.
Since there is not much sense in having a MultiWriteBagOStuff object with zero
cache backends, make the constructor require a nonempty array for 'caches', and
remove the subsequent checks that are made redundant by this new restriction.
Change-Id: I30f3e0dcbfe67570a368e64b8233cc0ba7f90b2f
* This is useful for ParserCache, as it tries to focus on memcached
and use other caches (e.g. mariadb) for the long-tail of less used
content, as setup on WMF. The class uses BagOStuff in a way that is
compatible with this approach.
Bug: T109751
Change-Id: Ia64eb44a9b52a988fde27b468d604d9163bed4b4
* Use the nx/ex flags for the redis SET method
to implement add() correctly. This also handes a
prior FIXME comment.
* Made merge() work via locking instead of cas()
since Redis::MULTI does not work with twemproxy.
Locking, which uses add(), does, and works better
than it did before.
* Removed some pointless newlines.
Change-Id: I652bd0ad2c594097d2cb1ab77f291e8bd27ad14f
Keys are sent to Memcached encoded. However, getMulti()
will respond in [key => value] format. The keys it
responds with should not be the encoded versions, or
callers won't be able to map them to the results.
Bug: T111138
Change-Id: I0d821b1219a492be8e93453f0249c78f18e24533
There are imho too many ObjectCache methods, $wg*Cache* variables,
CACHE_ types, and wfGet*Cache() functions to know which ones
should actually be used publicly. This should make it easier to
know which ones should be used.
The difference between them was also harder to understand with the
main documentation sections about each of these living in
conceptually different places:
- classes (WANObjectCache)
- methods (ObjectCache::getMainStashInstance)
- variables (DefaultSettings: wgMainCacheType, wgMainWANCache, wgMainStash)
Also:
* Correct wfGetCache() documentation.
* Add the missing keywords for 'public' visibility.
Change-Id: I3b05fdb8b7888bf7e3f05bdca36538f3484556b8
* This should trigger more reliably than the shutdown function
callback, which is really only there for sanity to make sure
session close/write happens.
Change-Id: I9a1aa76de121ba8de33b3fa850bd223929fae404
I4afaecd8 reduced the frequency at which the TTL of sessions keys is reset, on
the assumption that we were doing it a whole lot more often than necessary.
We can now assume that this assumption was false, because an uncommitted revert
of this change which I pushed to production earlier caused the rate of session
loss errors in the logs to plummet.
Bug: T102199
Change-Id: Ie67f4ca000afcf3d4f44155c13f91cd4c286866d
If RedisBagOStuff::getConnection() is able to establish a connection, only
check the master link status if automatic failover is enabled and if there are
other viable servers left to consider. If there are no servers left to
consider, or if automatic failover is not configured, just return the
connection handle without subjecting it to further tests.
This will have the side-effect of making RedisBagOStuff compatible with
Nutcracker, which does not implement the INFO command. This is because when
MediaWiki is configured to use Nutcracker, the server pool will consist of a
single server (namely, Nutcracker itself), and thus there will be no other
server to consider, so INFO will never be executed.
Change-Id: I3812ec5a0b22df122bdf44350bc0496574c02ce8
Some parts of resourceloader put an integer into the BagOStuff
and rejects any value it pulls back out that is not an integer.
This patch updates the memcached client behind the BagOStuff to
flag integer values and convert them back to integer when recieved.
The end result is shaving several seconds off the time it takes to
request a CSS or JS file in certain configurations.
Bug: 60563
Change-Id: Ie4dc8cf2c2cd232a2c167b4075ff4254b656bcbc
* This also makes sure garbageCollect() is called on writes from
time to time, since set() is used much more often
Change-Id: I5b6cc686bd6204fc3d6ff8b003be8b8d6a493e24
* To make this work, a slaveOnly flag was added to SqlBagOStuff
and it no longer prunes expired items on get(), it just treats
them as missing. The use of garbageCollect() for the writeFactory
cache and the use of replace() on updates already makes this
a non-issue.
* This also makes the default $wgMainStash non-bogus.
Bug: T88493
Change-Id: I9d1f31305e08430de29a3cd521cdb10a82dffd10
* A "gone away" error can trigger ping(), which can fail,
triggering a full open(). If this fails (as it often will),
mConn is false and the DatabaseBase object is not usable.
Bug: T103435
Change-Id: Id2aaed87d2642e73c7811076da908180ee5ab5a4
* The session will be renewed on access if it will expire in 10 minutes
* This avoids a problem both with 532ef78 + PHP <= 5.5 or any PHP >= 5.6
Bug: T102199
Change-Id: I8cad100bc35e86fa95ad144bcad338f3cb5d82a1
- Removed space after casts
- Removed spaces in array index
- Added spaces around string concat
- Added space after words: switch, foreach
- else if -> elseif
- Removed parentheses around require_once, because it is not a function
- Added newline at end of file
- Removed double spaces
- Added spaces around operations
- Removed repeated newlines
Bug: T102609
Change-Id: Ib860222b24f8ad8e9062cd4dc42ec88dc63fb49e
wfSuppressWarnings() and wfRestoreWarnings() were split out into a
separate library. All usages in core were replaced with the new
functions, and the wf* global functions are marked as deprecated.
Additionally, some uses of @ were replaced due to composer's autoloader
being loaded even earlier.
Ie1234f8c12693408de9b94bf6f84480a90bd4f8e adds the library to
mediawiki/vendor.
Bug: T100923
Change-Id: I5c35079a0a656180852be0ae6b1262d40f6534c4
* Whitespace.
* Simplify logic.
* Apply coding conventions to documentation blocks (empty line
before annotations, no empty lines between annotation, consistent
order of annotations).
Change-Id: I3e5268d6a6295643d5725c66ea2a01bccf610ed8
* This provides factory/config more appropriate for stashes
* Cleaned up some neighboring config comments while at it
Bug: T88493
Bug: T97620
Change-Id: Id370ee50be6e493a4700c858df863a183abc05fd
This class handles caching across distance sites where purges
must reach both. It also aims to make purging more reliable in
the face of network glitches and node consistent hash ejection.
bug: T88492
Change-Id: I686811b3075bf22e2f4de45127e8461e54648ead
Move the MWLogger PSR-3 logging related classes into the
MediaWiki\Logger namespace. Create shim classes to ease migration of
existing MWLoggerFactory usage to the namespaced classes.
Bug: T93406
Change-Id: I359cc81fbd2dcf8937742311dcc7d3dee08747b0
* Set a custom profiler with no expectations to avoid tripping
the "0 write" expection. This avoids useless log entries.
Change-Id: Iac849a729eb36b1a8affb0dbc8b8c195fab4b03a
The base BagOStuff class and some of the implementations are
not dependent on MediaWiki and can be in the libs folder.
Change-Id: I299f9abb778abc65a70461a2aff3015c02a1f04d
* merge() using the locking version by default. The callers that implement cas()
override merge() to use the cas-based version. Those that do not no longer
need to define the unused dummy method.
* Removed some redundant doc blocks.
Change-Id: I49f7dd5432efa8d76f4758c273f0859376ddafb7
ObjectCache::newFromParams() will handle a 'loggroup' parameter
specially, getting an instance of \Psr\Log\LoggerInterface to
pass to the BagsOStuff.
BagOStuff implementations can use $this->logger which will be
an implementation of \Psr\Log\LoggerInterface. As this is set in
BagOStuff::__construct(), all subclasses must now call the
parent constructor.
The goal of this is to make the logging in BagOStuff non-MediaWiki
specific, in the hopes of separating it out into a separate library
in the future.
Change-Id: I8a8e278e6f028814499d8457d6d5341d03eabc7a
Xhprof generates this data now. Custom profiling of various
sub-function units are kept.
Calls to profiler represented about 3% of page execution
time on Special:BlankPage (1.5% in/out); after this change
it's down to about 0.98% of page execution time.
Change-Id: Id9a1dc9d8f80bbd52e42226b724a1e1213d07af7
Partial deletion percentage progess was computed using
$remainingSeconds. This is incorrect since $remainingSeconds
is going to *decrease* ( instead of increase ) while the
process continues.
The correct implementation uses 'processed seconds' instead.
Change-Id: Ic745f6e8375a85da543de36703fad6d31f62ea90
Avoid an unnecessary serialize() call on get() requests, which makes it
faster and also happens to work around a nasty crash bug which is
causing lots of segfaults right now.
CAS semantics should be the same, if the caller handles a CAS token in
an opaque way and does not assume it is a string. I could only find one
caller of cas() in core and WMF deployed extensions, and that is
BagOStuff::mergeViaCas(), which does correctly handle CAS tokens of any
type.
Bug: 71724
Change-Id: I0b0b5f0157c005ec7f76210dfb8576c056314733
- Added space after reserved words: function, foreach, if
- Combined 'else if' into elseif
- Added braces to one-line statements
- Added spaces after comma, before parentheses
Change-Id: Ie5bbf680d6fbe0f0872dab2700c16b1394906a72
Fixes "Fatal error: Argument 4 passed to Memcached::cas() must be an
instance of int, float given".
Change-Id: Ibf1ea638ec1a4dcf009cdaea8aa66008c74ff30b
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling
Change-Id: I8ebfbcea0e2ae2670553822acedde49c1aa7e98d
Database::select() can return false, so we should check for it before
attempting to iterate on the result or to call methods on it.
Change-Id: I0862493305e5b2784422e0e94b3e62e734267795
- Added/removed spaces around parenthesis
- Added space after switch/if/foreach
- changed else if to elseif
Change-Id: I99cda543e0e077320091addd75c188cb6e3a42c2
* Callers still sees some decent time ordering since writes block
until committed. This will not blindly flush transactions though.
* Basically reverts 59e8032
Change-Id: Ib9a91ac023f8e3fafb0bf0eef9dca6e31b867be9
* This avoids breaking the main transaction in finishWrite().
* Also removed an unused variable in SqlBagOStuff.
Change-Id: Ia330ac362b080c1338616d95b793032c4752dc23
* This method is currently hit when bots use the maxlag API parameter
* Also added a $fallback param to ObjectCache::newAccelerator so code
does not have to keep reinventing that logic.
Change-Id: I45bf1085301c5cf69b29b494a146c6a067534581
It does not work with unserialize because RedisBagOStuff->unserialize
expects either strings of digits, or serialized strings. A plain integer
is neither.
incr docs were incorrect, so fix and expand by copying from parent class.
Change-Id: Ifc9c7a36f221b251ceea0d9a79ae6c3ce5753dd4
SqlBagOStuff::getMulti currently returns an array with false values for missed
cache keys.
Default BagOStuff::getMulti (and other specific implementations like memcached),
however, just omit missed values.
e.g.:
$cache->getMulti( array( 'goodkey' 'badkey' ) );
should return
array( 'goodkey' => 'value' );
instead of
array( 'goodkey' => 'value', 'badkey' => false );
Bug: 65637
Change-Id: Ib2de06a0f76421094707f7ad4218346087f4dc55
Includes implementions for Redis, Sql and MemcachedPecl,
other types will fallback to using $this->set repeatedly.
Change-Id: I0924a197b28ee69e883128ccd672343e5c041929
* This can limit how many workers work on a given file at once,
regardless of which specific thumbnail parameters are requested.
* Also added a BagOStuff::incrWithInit() method per DRY.
* Tweaked the register_shutdown_function() callback to avoid
File/MediaTransformOutput references, since they may involve
TempFSFile objects.
Change-Id: Ic01461cb974ab23b179ac3b60cf6de12e36360f6
Changed closure to capital word Closure in doc and type hint,
also changed callback in docs to callable
Change-Id: I52c8e8f13d38a837052101c38b9986be780ca057
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.
Change-Id: I86fd10e3f2d4bb80e7432533038d124693acfb3c
Variants included 'in <version>', 'as of <version>' and just the
version number.
Some @deprecated annotations do not have the version number at all,
I want to hunt them down separately.
Change-Id: I8208c6097098f4735d4f51bc42254675f1f27f6d
- Added spaces after if/foreach/catch
- Added new line before end of file
- Added or removed spaces before/after parenthesis, comma
- Added spaces around string concat
Change-Id: I0590070f1b3542108e242730e8d9a3ba9831e94f
* Almost nothing actually uses this (one extension)
as it doesn't really solve race conditions like merge().
* Removed subclassed versions too.
Change-Id: I4d4123afcc0576a90a3cbf1e4e6daff7ebce39c3
Isn't useful outside of testing as the comments state, but it's
barely even useful for that. People who are testing the cache
code can use CACHE_DB if CACHE_MEMCACHED is too hard.
Change-Id: Ief0aa148376957fdd844c8bb585a133b854a012c
mehcache isn't a caching system, it's a pun.
I'm removing Ehcache support because it's an experiment didn't pan out
and the code's just sitting here abandoned now.
Change-Id: Ief5728b4c61bb5ad8f5a5f0b55415b23ef6727eb
For caches where CAS is not natively supported, we have a workaround, where the
CAS token is (based on) the stored value.
To confirm the data can be written to cache, the CAS token is compared against
"whatever is currently in cache", so we pull the cached data and rebuild the
value.
In the case of objects, we have now pulled & rebuilt (unserialized) 2 objects
that are actually the same object (for CAS purpose - it's the correct value to
overwrite), but in terms of ===, they're 2 different values.
This patch should make sure CAS tokens are always a serialized version of the
value we're saving (where no native CAS exists); these serialized versions can
reliably be compared.
Bug: 59941
Change-Id: I2760416c48f2ceb7a0e0c874dd70ec07b4dccdfc
* This works with local memcached (e.g. unix:///var/run/memcached/memcached.sock:0 )
and noticeably increases memcached mediawiki performance
Change-Id: Ie08c151caa09eb0a4269df88965d71c2367c398b
* Avoid serializing scalars like the memcached client does.
This means incr/decr can be fast and preserve the TTL.
* Properly call unwatch() in the cas() method.
* Made the expiry set on add() actually atomic.
* Also cleaned up the profiling calls.
bug: 56069
Change-Id: I3e0d1c4888062544c54aef32085f8ce608ff5423
The Line continuation Coding conventions prefers the closing parenthesis
on the same line than the beginning curly braces. This is done for ifs
and functions.
Also move some boolean operator from the end of a line to the beginning
and changed some indentation to make the condition hopefully better
readable.
Change-Id: Id0437b06bde86eb5a75bc59eefa19e7edb624426
