* Add BagOStuffTest subclasses for all core BagOStuff subclasses,
replacing PHPUNIT_USE_BAGOSTUFF, as suggested in a todo comment.
* Add config $wgEnableRemoteBagOStuffTests which causes all tests
enabled by $wgObjectCache to execute, which means that the memcached
tests are executed by default.
I have verified all except RESTBagOStuff and WinCacheBagOStuff. The
memcached tests fail against memcached 1.5.x but pass against memcached
1.6.x.
Bug: T90875
Change-Id: Id74b5226669f8cb857f859fbc35bc58ab001e873
The new style of checking for the last error during a section of
calls is more robust since it allows nesting of callers. Typically,
an external caller will want to watch a section of code that will
involve zero or more internally watched sections. Errors that are
seen internally (leading to a failing response) should also be
visible externally.
Replace internal BagOStuff clearLastError() calls.
Replace WANObjectCache clearLastError() calls. Such a class should not
clear the error codes since the class is effectively "internal". Callers
that are more meaningfully "external" might want to check the errors.
Cleanup "last" error handling for proxy backends.
Change-Id: I281817a85602967c0ec2bdd23a5d8be101680b64
Add a multi-primary mode option that supports MySQL DB setups
that use circular replication with STATEMENT formatted binlogs.
The `modtoken` column is only used when multi-primary mode is
explicitly enabled in configuration. The column is used by write
queries to determine the "winning" version of keys, with the goal
of approximating "Last-Write-Wins" eventual consistency.
Writes with different timestamps can be handled by picking the
one with the highest timestamp as the "winner". Writes with the
same timestamp, from different primary DBs, can be handled by
picking the one from the primary DB with the highest server_id.
Writes with the same token timestamp from the same primary DB can
be handled by picking the last write to appear in the binlog.
The delete() operation uses tombstones in multi-primary mode,
since there must be a key version to actually compare with the
versions from other operations.
Also:
* Remove "LOCK IN SHARE MODE" that was made obsolete by the
CONN_TRX_AUTOCOMMIT flag. For the SQLite transaction case,
it is serializable anyway.
* Simplified handleWriteError() to match handleReadError()
and merged them into handleDBError().
Changes from d56a686f83:
* Keep using replace() for set() in non-multi-primary mode.
* Fix old encoding issues when incrementing keys with postgres.
Bug: T274174
Change-Id: I25b1278586f99fdace8f2a081f6f460f734e5d83
Empty leading components should still be colon separated. Although the
keyspace and collection should never be empty strings, it does not hurt
to handle that case anyway for safety.
Change-Id: Ic89acaf1fe5c34e58323b2913a9a26f973868590
Add a multi-primary mode option that supports MySQL DB setups
that use circular replication with STATEMENT formatted binlogs.
The `modtoken` column is only used when multi-primary mode is
explicitly enabled in configuration. The column is used by write
queries to determine the "winning" version of keys, with the goal
of approximating "Last-Write-Wins" eventual consistency.
Writes with different timestamps can be handled by picking the
one with the highest timestamp as the "winner". Writes with the
same timestamp, from different primary DBs, can be handled by
picking the one from the primary DB with the highest server_id.
Writes with the same token timestamp from the same primary DB can
be handled by picking the last write to appear in the binlog.
The delete() operation uses tombstones in multi-primary mode,
since there must be a key version to actually compare with the
versions from other operations.
Also:
* Remove "LOCK IN SHARE MODE" that was made obsolete by the
CONN_TRX_AUTOCOMMIT flag. For the SQLite transaction case,
it is serializable anyway.
* Simplified handleWriteError() to match handleReadError()
and merged them into handleDBError().
Bug: T274174
Change-Id: Icc5eff9a032dd3403b5718058f20e38f8ea84af5
Ended up using
grep -Prl '\->setMethods\(' . | xargs sed -r -i 's/setMethods\(/onlyMethods\(/g'
special-casing setMethods( null ) -> onlyMethods( [] )
and then manual fix of failing test (from PS2 onwards).
Bug: T278010
Change-Id: I012dca7ae774bb430c1c44d50991ba0b633353f1
* parent::setUp() should be first, and ::tearDown()
should be last
* Move tests that directly extend PHPUnit\Framework\TestCase
to /unit
Change-Id: I1172855c58f4f52a8f624e6d596ec43beb8c93ff
Add "generic" key methods for quickly deriving keys from
key component lists in a bijective manor. This is useful
for BagOStuff classes that wrap other BagOStuff instances
or for parsing keys to get stats.
Make the proxy BagOStuff classes (ReplicatedBagOStuff,
MultiWriteBagOStuff, CachedBagOStuff) use "generic" keys
so that they can convert to appropriate keys when making
backing cache instance method calls.
Make EmptyBagOStuff, HashBagOStuff, APCUBagOStuff,
RedisBagOStuff, and RESTBagOStuff use "generic" keys rather
than those of MediumSpecificBagOStuff::makeKeyInternal().
This lets proxy BagOStuff classes bypass key conversions
when used with instances of these classes as backing stores.
Also:
* Fix missing incr(), incrWithInit(), and decr() return
values in MultiWriteBagOStuff.
* Make MultiWriteBagOfStuff, ReplicatedBagOStuff, and
CachedBagOStuff use similar backend method forwarding
styles by using a new BagOStuff method.
* Improved various related bits of documentation.
Bug: T250239
Bug: T235705
Change-Id: I1eb897c2cea3f5b756dd1e3c457b7cbd817599f5
Previously, the refresh probability ramp-up logic assumed that
keys were always assigned logical TTLs >= "lowTTL". Thus, if a
key was saved with a TTL of 3 seconds and "lowTTL" was 10 seconds,
then the probability of a refresh at t=0 seconds would incorrectly
equal the value it should have at t=7.
Also:
* Use class constants for key metadata array keys. This makes it
easy to track the usage of options and is less prone to typos.
* Improve some documentation slightly.
Bug: T264787
Change-Id: I7267e8639617fb8dc0850530465ff9d1b899b592
This makes User::pingLimiter() include the expiry time in the payload of
the cache key that holds the current count. This allows us to ignore
stale counts.
Until now, we have been relying on the cache implementation to expire
the relevant keys in time. This however seems to fail sometimes.
Bug: T246991
Change-Id: Ifa3c558b4449f1ca133d0064781f26ac1bf59425
The name change happened some time ago, and I think its
about time to start using the name name!
(Done with a find and replace)
My personal motivation for doing this is that I have started
trying out vscode as an IDE for mediawiki development, and
right now it doesn't appear to handle php aliases very well
or at all.
Change-Id: I412235d91ae26e4c1c6a62e0dbb7e7cf3c5ed4a6
Create a simple APCUBagOStuff subclass and also fix
APCUBagOStuff::incrWithInit() default $init value.
Change-Id: If84963fe7dcfedd6edebfb8785235263e0868ece
Done with `composer fix` and suppressing the rest (i.e. sniffs for
global variables, which for core should be suppressed anyway).
Additionally, add `-p` to `phpcbf`, as otherwise it just seems stuck.
Change-Id: Ide8d6cdd083655891b6d654e78440fbda81ab2bc
This makes it easier to rollout one keyspace/project at a time even
if some keys are shared and receive purges. The shared keys can all
be done as the last step.
Also, simplify getMulti() to no longer need extractBaseKey().
Make the "warm up cache" logic a bit easier to follow and less
likely to copy values around.
Change-Id: I8b602ddf5dd1feaada45fb0af202c5603836a8dd
If a key always takes a very long time to regenerate, is popular,
and does not use lockTSE, it still needs to be cacheable. Since a
value cannot be more up-to-date than the time it takes to regenerate
it, take the "lower the TTL" approach for these cases. Use "walltime"
to narrow down the "reject the set()" case based on regeneration time.
This is already provided by getWithSetCallback() automatically.
Bug: T244877
Change-Id: Id43fb02738b28dad3bc922057efb7eee0272d0e1
This is useful for grouping related keys on the same servers to reduce
the need for cache server connections and availability. A cache key that
uses "lockTSE" can already involve accessing several keys during the
read/write cache-aside paths:
a) The value key itself
b) The check key (named after the main key, a common pattern)
c) The mutex key (used if the value looks stale)
d) The cool-off key (used if regeneration took a while)
Any problems accessing the first two could cause extra value regenerations.
Problems with the mutex key could lead to stampedes due to threads assuming
another thread was regerating a soon-to-expire value when, in fact, none was.
A similar problem could happen with cool-off keys, with threads assuming
that another saved the newly regenerated value when, in fact, none did.
The use of hash stops puts the tiny related keys on the same server as the
main cache key that they serve. This is only for hash-based routing, and not
route prefix routing (e.g. All*Route still sends the key to multiple child
routes, but the PoolRoute/HashRoute function will hash differently).
The option is not enabled by default yet.
Change-Id: I37e92a88f356ef1e2a2b7728263040e2f6f09a13
We're observing WANObjectCacheTest fail sometimes, possibly due to a
1/825 chance of a cache key being counted as "popular" combined with an
mt_srand() call making this more likely. I tweaked the options to
getWithSetCallback() to avoid the affected mt_rand() call altogether.
I also added a lowTTL option to another getWithSetCallback() call,
eliminating a condition with a 3% chance. It didn't appear to matter
which branch the test took, but I'd rather avoid non-deterministic
behaviour.
There are a few remaining coin tosses being logged, but they all occur
with a chance of less than 10^-5.
Bug: T238197
Change-Id: Iadd3dc00d44b8490210fec97fe1f49236326300f
To help isolate the referenced bug, which is a sporadic test failure
in testGetWithSetCallback().
Bug: T238197
Change-Id: If35d60340c804b6bfe1e9ddfcf53c76373c794b1
Add public, protected or private to function missing a visibility
Enable the tests folder for the phpcs sniff
Change-Id: Ibefce76ea9984c47e08c94889ea2eafca7565e2c
Use the stricter comparisons from assertSame instead of
assertEquals.
In some cases this revealed a test that was asserting a value
different from the one actually returned:
* testSetAndGet:
The test failed with the test case (null, 3), where it tries
to store the value `null`, but gets back `false`, which is wrong.
This appears to be a genuine bug, filed as T234583.
Change-Id: Ic3dcc7fa2e8749b0f2d68917a8ac728dda26b6ca
assertEquals( false, … ) still succeeds when the actual value is 0, null,
an empty string, even an empty array. All these should be reported as a
failure, I would argue.
Note this patch previously also touched assertSame( false ). I reverted
these. The only benefit would have been consistency within this codebase,
but there is no strict reason to prefer one over the other. assertFalse()
and assertSame( false ) are functionally identical.
Change-Id: Ic5f1c7d504e7249002d3184520012e03313137b4
The benefit of using count() is that the test would still succeed if
the return vfalue is not an array, but an iterable object. It seems
this is not needed.
Change-Id: I23529f6990aebe0cce86e236a21820fe74993204
assertEquals( null, … ) still succeeds when the actual value is 0, false,
an empty string, even an empty array. All these should be reported as a
failure, I would argue.
Note this patch previously also touched assertSame( null ). I reverted
these. The only benefit would have been consistency within this codebase,
but there is no strict reason to prefer one over the other. assertNull()
and assertSame( null ) are functionally identical.
Change-Id: I92102e833a8bc6af90b9516826abf111e2b79aac
assertSame() is guaranteed to not do any type conversion. This can be
critical when acciden tially comparing, for example, 0 to 0.0.
Change-Id: Iffcc9bda69573623ba14af655dcd697d0fcce525
