Replaces \TestingAccessWrapper (defined in core) with
\Wikimedia\TestingAccessWrapper (defined in the composer package
wikimedia/testing-access-wrapper).
See https://gerrit.wikimedia.org/r/#/q/topic:librarize-testing-access-wrapper
for downstream patches.
The core version of the class is kept around for a while to avoid
circular dependency problems.
Bug: T163434
Change-Id: I52cc257e593da3d6c3b01a909e554a950225aec8
* Fix typo that disabled testNewAnythingNoAccel().
Follows-up c5a0fa5bed, accidentally committed a local hack
to disable the test.
* Add missing case other types falling back and no DB.
* Add missing case of no other types and no DB.
Change-Id: If158f21053f0b3741f2625fe4455fdb31955a22f
If $wgMainCacheType = CACHE_ACCEL, but there is no APC, then its
possible that CACHE_ANYTHING will default to CACHE_NONE because
that's what CACHE_ACCEL would do.
Possibly also T147161
Bug: T160519
Change-Id: I9ac2d071437b35a0f9cd3678e2279628f7b1931e
I was bored. What? Don't look at me that way.
I mostly targetted mixed tabs and spaces, but others were not spared.
Note that some of the whitespace changes are inside HTML output,
extended regexps or SQL snippets.
Change-Id: Ie206cc946459f6befcfc2d520e35ad3ea3c0f1e0
Follows-up c4e698dc2. The tests are failing on Travis CI because
php5-redis isn't installed by default. These tests shouldn't need
that to be installed (and even then, we should skip gracefully as
it is an optional dependency).
> RedisBagOStuffTest::testUnserialize with data set #0
> Exception: RedisConnectionPool requires a Redis client library.
> ./mediawiki/includes/clientpool/RedisConnectionPool.php:86
Fixed by using a mock and disabling the original constructor.
Change-Id: Icced7c30a75516c2118489ad29eac2aa5cff80ad
If3e20c6 and the following patches introduced a breaking change and
cause a regression in Wikibase because we are using the version number
constant as part of a cache key prefix. Currently the Wikibase version
is set to "0.5 alpha".
Space characters were allowed before and encoded as "%20". This does
not happen any more.
Change-Id: Ia2fd4ed6738a10e02050bced947ef5d4e8b98980
* Follow-up to 0c9fb12265
* Make sure colons actually get escaped
* Added more unit tests
* Also fixed the test actual/expected order
Change-Id: Ie04ea6059ee1eb6d1da8f30fefdec52fa49d38fb
For the sake of safety and correctness, the following BagOStuff::makeKey()
invocations should return distinct keys:
$cache->makeKey( 'ab:', 'cd' );
$cache->makeKey( 'ab', ':cd' );
That is not currently the case, because while we use ':' as a key path
separator, we don't escape ':' in the input supplied to makeKey(). So, make
BagOStuff::makeKeyInternal() URL-encode colons.
To prevent this from messing up the max. key length calculations, reproduce
this logic in MemcachedBagOStuff::makeKeyInternal(), in lieu of having the
method call its parent.
Change-Id: I83ea7e7336a1c9e64aa42284c2517089a736efe5
Currently we have to undo any transformations we apply to keys in getMulti() by
iterating over the response array keys reversing any changes. This is hairy and
complicated. So --
* Replace calls to MemcachedBagOStuff::encodeKey() with calls to a new method,
MemcachedBagOStuff::validateKeyEncoding(). The new method only validates that
the key is compatible with memcached. If it is not, it throws an exception.
If it is, it returns the key unmodified.
* Remove MemcachedBagOStuff::{encode,decode}Key(), since they no longer serve a
purpose, and have no callers.
Change-Id: If3e20c6a1a1b42fc1f2823aa660328e37c26eccb
The motivation for this patch came from seeing the following error in the
memcached log:
memcached ERROR: Memcached error for key "flowdb:flow_ref:wiki:by-source:\
v3:0:tawikinews:பூமிக்கு_அச்சுறுத்தலான_சிறுகோள்களைக்_கண்டுபிடிக்கும்_முயற்சியில்_தனியார்_நிறுவன0jம்:4.7" \
on server ":": A BAD KEY WAS PROVIDED/CHARACTERS OUT OF RANGE
I submitted a fix to Flow in I26e531f6, but I noticed that AbuseFilter had a
similar issue (fixed by Aaron in I27b51a4b), so I started thinking about how
to solve this more generally:
* The regular expression we current use to sanitize keys does not cover
characters outside the ASCII range, but such characters can be illegal
if one of their constituent bytes (when taken by itself) is an ASCII
control character. So change the regular expression to cover any and all
characters that fall outside the range \x22-\x7e (and '#' -- see below).
* Enforce a key length limit of 255 bytes, which is the maximum length
permitted by the memcached protocol. The Tamil segment in the key above is 84
characters, but 233 bytes in UTF-8, which become 684 characters when
URL-encoded. To fix this, try to shrink any segment that would push the total
key length over the limit by md5()ing it. If the end result is *still* over
the limit (this would happen if, for example, $args consists of many short
strings), then concatenate all args together and MD5 them.
* MD5'd arguments are prefixed with '#'. Any "organic" '#'s in the key segments
are URL-encoded.
Change-Id: Ia46987d3b0a09bb6b1952abd936d4c72ea7c56a0
* This blocks on writing to all replicas
and returns false if any failed.
* This is useful if ChronologyProtector is to work across
domains by having the writes go everywhere so that later
reads will see them (and be local at the same time).
* Redundant doc comments were also removed.
Change-Id: I9ed098d563c64dba605e7809bc96731da3b3e79d
Inject the DeferredUpdates::addCallableUpdate method via the
ObjectCache. This brings it closer to being able to move to /libs.
Change-Id: Ifa0d893002c3d709a4dc7346c263a92162274bd7
* Add a string `keyspace` member to BagOStuff instances. The default
implementation, meant for simple key/value stores, treats the key space
as a string prefix to prepend to keys. By default, its value is `local`,
but any instance created via ObjectCache::newFromParams() (or or one of
its callers) will have that default to $wgCachePrefix / wfWikiID().
* Add `makeKey` and `makeGlobalKey` methods to the base BagOStuff class.
These methods are not static to allow for BagOStuff types which require
a configured instance to know the underlying storage engine's key semantics.
* Make wfMemcKey() and wfGlobalCacheKey() delegate to these methods on the main
ObjectCache instance.
Change-Id: Ib7fc2f939be3decfa97f66af8c2431c51039905f
* Make getWithSetCallback() accept a TTL field for this
* Make ChangeTag callers use this flag to avoid hundreds of
duplicate queries at Special:Tags
Change-Id: Ic1ed28294f5d557e02f39a7f20d36766244b9ded
* Put 'checkKeys' param in opts array instead of as a separate parameter.
It's neat to not have to skip unnamed/positional arguments that are optional.
* Move TTL to be before callback instead of after. This avoids dangling integers
toward the bottom of a large code block that have no obvious meaning.
Matches BagOStuff::getWithSetCallback.
Add unit test for lockTSE to confirm
Change-Id: I60d6b64fc5dff14108741bafe801fd1fde16d1df
Flow had a key: flowdb:flow_ref:wiki:by-source:v3:Parser\'s_"broken"_+_(page)_&_grill:testwiki:1:4.7
the '+' in there was not being encoded (it only does /[\x00-\x20\x25\x7f]+/)
but coming back, it was decoded into ' '.
getMulti() shows a key=>value array or results. Since key was different,
we couldn't find what we had requested.
Bug: T110326
Change-Id: Ia92edd73d0eb7fe0d35e38e7e7af8174fb85cbcc
* This is useful for ParserCache, as it tries to focus on memcached
and use other caches (e.g. mariadb) for the long-tail of less used
content, as setup on WMF. The class uses BagOStuff in a way that is
compatible with this approach.
Bug: T109751
Change-Id: Ia64eb44a9b52a988fde27b468d604d9163bed4b4
Keys are sent to Memcached encoded. However, getMulti()
will respond in [key => value] format. The keys it
responds with should not be the encoded versions, or
callers won't be able to map them to the results.
Bug: T111138
Change-Id: I0d821b1219a492be8e93453f0249c78f18e24533
* This uses a non-blocking $wgMemc lock to reserve the user
name in question. This should prevent two threads from
reaching LOCK IN SHARE MODE and getting stuck on INSERT.
The lock is global to better cover auth plugins.
* This adds a BagOStuff::getScopedLock() convenience method.
It uses less queries than LockManager by being EX only.
* Avoid extra lock attempt in lock() in non-blocking mode.
* Removed (un)lock() HashBagOStuff overrides that made it
behave differently than other caches (wrt to re-entrance).
Bug: T106850
Change-Id: Iecf95206d712367f5d202f76ab0eaa9d7bdabf2b
* This still allows If-Modified-Since logic but does not
need to broadcast initialization values just because a
key fell out of cache. The value can differ between DCs
anyway via skew, this just lets them drift more. Actual
purge events are still broadcasted, which is what matters.
* The User class has now been simplified given this change.
* Added more general comments to getCheckKeyTime().
Change-Id: Ic1f4bbb1947e0d1dd47499c9e9dc86991c30580c
Fixed PHP runtime warnings:
> Declaration of ReplicatedBagOStuff::getMulti() should be compatible with BagOStuff.
> Declaration of ReplicatedBagOStuff::decr() should be compatible with BagOStuff.
Change-Id: Icf1a0bf2c30408c4a5bef2de0b69ae2162b234d5
- Removed space after casts
- Removed spaces in array index
- Added spaces around string concat
- Added space after words: switch, foreach
- else if -> elseif
- Removed parentheses around require_once, because it is not a function
- Added newline at end of file
- Removed double spaces
- Added spaces around operations
- Removed repeated newlines
Bug: T102609
Change-Id: Ib860222b24f8ad8e9062cd4dc42ec88dc63fb49e
This class handles caching across distance sites where purges
must reach both. It also aims to make purging more reliable in
the face of network glitches and node consistent hash ejection.
bug: T88492
Change-Id: I686811b3075bf22e2f4de45127e8461e54648ead
Can't be implemented as a test case because that's not being
run before MediaWikiTestCase::tearDown.
Also there's no parent destructor, so no need to call one.
Also fixed BagOStuffTest.
Change-Id: Ifd8659b6c8a748c6716099f8822e2c50ab51bda7
