This is useful for grouping related keys on the same servers to reduce
the need for cache server connections and availability. A cache key that
uses "lockTSE" can already involve accessing several keys during the
read/write cache-aside paths:
a) The value key itself
b) The check key (named after the main key, a common pattern)
c) The mutex key (used if the value looks stale)
d) The cool-off key (used if regeneration took a while)
Any problems accessing the first two could cause extra value regenerations.
Problems with the mutex key could lead to stampedes due to threads assuming
another thread was regerating a soon-to-expire value when, in fact, none was.
A similar problem could happen with cool-off keys, with threads assuming
that another saved the newly regenerated value when, in fact, none did.
The use of hash stops puts the tiny related keys on the same server as the
main cache key that they serve. This is only for hash-based routing, and not
route prefix routing (e.g. All*Route still sends the key to multiple child
routes, but the PoolRoute/HashRoute function will hash differently).
The option is not enabled by default yet.
Change-Id: I37e92a88f356ef1e2a2b7728263040e2f6f09a13
We're observing WANObjectCacheTest fail sometimes, possibly due to a
1/825 chance of a cache key being counted as "popular" combined with an
mt_srand() call making this more likely. I tweaked the options to
getWithSetCallback() to avoid the affected mt_rand() call altogether.
I also added a lowTTL option to another getWithSetCallback() call,
eliminating a condition with a 3% chance. It didn't appear to matter
which branch the test took, but I'd rather avoid non-deterministic
behaviour.
There are a few remaining coin tosses being logged, but they all occur
with a chance of less than 10^-5.
Bug: T238197
Change-Id: Iadd3dc00d44b8490210fec97fe1f49236326300f
To help isolate the referenced bug, which is a sporadic test failure
in testGetWithSetCallback().
Bug: T238197
Change-Id: If35d60340c804b6bfe1e9ddfcf53c76373c794b1
Add public, protected or private to function missing a visibility
Enable the tests folder for the phpcs sniff
Change-Id: Ibefce76ea9984c47e08c94889ea2eafca7565e2c
Use the stricter comparisons from assertSame instead of
assertEquals.
In some cases this revealed a test that was asserting a value
different from the one actually returned:
* testSetAndGet:
The test failed with the test case (null, 3), where it tries
to store the value `null`, but gets back `false`, which is wrong.
This appears to be a genuine bug, filed as T234583.
Change-Id: Ic3dcc7fa2e8749b0f2d68917a8ac728dda26b6ca
assertEquals( false, … ) still succeeds when the actual value is 0, null,
an empty string, even an empty array. All these should be reported as a
failure, I would argue.
Note this patch previously also touched assertSame( false ). I reverted
these. The only benefit would have been consistency within this codebase,
but there is no strict reason to prefer one over the other. assertFalse()
and assertSame( false ) are functionally identical.
Change-Id: Ic5f1c7d504e7249002d3184520012e03313137b4
The benefit of using count() is that the test would still succeed if
the return vfalue is not an array, but an iterable object. It seems
this is not needed.
Change-Id: I23529f6990aebe0cce86e236a21820fe74993204
assertEquals( null, … ) still succeeds when the actual value is 0, false,
an empty string, even an empty array. All these should be reported as a
failure, I would argue.
Note this patch previously also touched assertSame( null ). I reverted
these. The only benefit would have been consistency within this codebase,
but there is no strict reason to prefer one over the other. assertNull()
and assertSame( null ) are functionally identical.
Change-Id: I92102e833a8bc6af90b9516826abf111e2b79aac
assertSame() is guaranteed to not do any type conversion. This can be
critical when acciden tially comparing, for example, 0 to 0.0.
Change-Id: Iffcc9bda69573623ba14af655dcd697d0fcce525
Make the default $init value for incrWithInit() be $value.
This is far less suprising and also makes the operation
easier to replicate without conflicts.
Make decr() definitions more explicit since various cache
drivers do not handle negative incr() values (e.g. memcached).
Change-Id: I2b8d642656cc91c841abbd7a55d97eba101b027a
Add MediumSpecificBagOStuff::getValueOrSegmentList() helper method.
Also:
* Use $keysMissing variable correctly in CachedBagOStuff::getMulti()
to avoid extra overhead.
* Optimize mergeViaCas() when the current value matches the new one.
Change-Id: I5c4bd74379bc459216ac0278150ce3aecff3b851
Partly a follow-up to 88640fd902.
Use real time in changeTTL() tests to fix all remaining
failures for BagOStuff sub-classes.
Change-Id: I537d665d6c8770a68a5a79233a913f1714881dfb
Use real time for testing absolute expirations with changeTTL().
Otherwise, backends like memcached or redis will fail since
they do not use the mock time.
Also:
* Make SqlBagOStuff actually override changeTTLMulti() by
using the right method name
* Check TTL_INDEFINITE more explicitly for clarity
* Rename TTL conversion methods for clarity
* Use isRelativeExpiration() in MemcachedBagOStuff
Change-Id: I9365ceb31d4e7bef65906363d42b8c3020a66346
This should help fix the following issues that various repos
are getting from Phan as of late:
> Call with 5 arg(s) to \BagOStuff::makeKey() which only takes 2 arg(s)
> defined at ../../includes/libs/objectcache/BagOStuff.php:456
> <source="PhanParamTooMany"/>
Bug: T228563
Depends-On: I5cfba063821101325a5a7359e6b8ad71a0fb1b2f
Depends-On: Ifa5b96735376f2fbe3680799f960616ba8d357ff
Change-Id: Ic9df7f3ad7f356c7cbdfe1edfbe35821b931dda6
Also simplify the code by removing the is_callable() check and relying on
regular PHP errors instead of an exception for bad callbacks.
Change-Id: I084b0132c5fb05f1941a6d6839cfa74e2cf677f0
Also fix some IDEA warnings in redis classes and make it easy for IDEs
to recognize the Redis (phpredis) class calls to RedisConnRef proxies.
Bug: T113916
Change-Id: If45a37da412ac37e8c07dc3d1053826aa0a62077
As long as get()s are disallowed from the process cache, the sets() should
at least still be up-to-date, so there is little reason to prevent them.
Change-Id: Ic62c8380801130de7f8412cddcf85b246e33b3cd
Add WANObjectCache::multiRemap() as an array_combine() wrapper for
easily working with IDs after getMultiWith(Union)SetCallback() calls.
Make the enforcement of uniqueness in makeMultiKeys() stricter and
discourage poor key design in comments. Add WANObjectCache::hash256()
method for getting good key component hashes.
Also avoid pointless use of ArrayIterator::getArrayCopy().
Change-Id: I61ffdbf4af4374864bac180df590b4dddc8da56b
Add FLD_VALUE_VERSION key to the value wrapper array to hold the version
number used in getWithSetCallback(). Remove the VFLD_* wrapper array from
FLD_VALUE for versioned values.
Keys stored with the old VFLD_VERSION and VFLD_DATA fields will be seen as
having the wrong version. The previous WAN cache code will see the new keys
that use FLD_VALUE_VERSION as having the wrong version too. In either case,
the usual variant key logic applies, so there should not be any issues.
This means that moving from a non-versioned to a versioned cache key is no
longer a breaking change when, for the same key, some code passes a version
number to getWithSetCallback() while other code does not.
Also:
* Make "pcTTL" respect the version number for sanity
* Make sure set() respects TTL_UNCACHEABLE for completeness
* Track slow regeneration callback runtime in FLD_GENERATION_TIME
* Remove is_callable() check overhead and rely on PHP Error instances
* Refactor unwrap() to return a more immediately useful value
* Simplify getNonProcessCachedKeys() signature by using $opts
* Split out PURGE_* constants for purge entries since those keys are
never stored in any serialize value but are only in PHP arrays
* Rename doGetWithSetCallback() to be more succinct
* Rename and reorganize some variables for clarity
Change-Id: I4060b19583cdfd9fa36c91d7014441eeef4b3609
In SqlBagOStuff:
* Add modifyMulti() helper method to reduce code duplication
* Improve atomicity of add(), cas(), and changeTTL() queries
* Avoid integer serialization and improve atomicity of incr()
* Optimize new BagOStuff::changeTTLMulti() method
In BagOStuff:
* Add changeTTLMulti() method for subclasses to optimize
* Make set() ignore WRITE_ALLOW_SEGMENTS for integers so incr() works
* Strip WRITE_ALLOW_SEGMENTS flag from the setMulti() call in set() to
avoid triggering bogus sanity check exceptions
* Fix BagOStuffTest::testSetSegmentable failures via the above changes
* Enforce WRITE_ALLOW_SEGMENTS sanity check in setMulti() for all the
subclasses by using a final wrapper method
* Add WRITE_ALLOW_SEGMENTS sanity check to deleteMulti()
Bug: T113916
Change-Id: I25d1790fa9b0d1837643efccfa94a12043cfbf42
This changeset resumes work on T89432 and related tickets
by porting an initial set of tests to the new unit test suite
separated out in I69b92db3e70093570e05cc0a64c7780a278b321a.
The tests were only ported if they worked immediately without
requiring any changes other than changing the test case class
to MediaWikiUnitTestCase and moving the test to the new suite.
If a test failed for any reason (even trivial misconfiguration),
it was NOT ported.
With this change, the unit tests suite now consits of a total
of 455 tests. As before, you can run these tests via the following
command:
$ composer phpunit:unit
Bug: T84948
Bug: T89432
Bug: T87781
Change-Id: Ibb8175981092d7f41864e641cc3c118af70a5c76
This changeset implements T89432 and related tickets and is based on exploration
done at the Prague Hackathon. The goal is to identify tests in MediaWiki core
that can be run without having to install & configure MediaWiki and its dependencies,
and provide a way to execute these tests via the standard phpunit entry point,
allowing for faster development and integration with existing tooling like IDEs.
The initial set of tests that met these criteria were identified using the work Amir did in
I88822667693d9e00ac3d4639c87bc24e5083e5e8. These tests were then moved into a new subdirectory
under phpunit/ and organized into a separate test suite. The environment for this suite
is set up via a PHPUnit bootstrap file without a custom entry point.
You can execute these tests by running:
$ vendor/bin/phpunit -d memory_limit=512M -c tests/phpunit/unit-tests.xml
Bug: T89432
Bug: T87781
Bug: T84948
Change-Id: Iad01033a0548afd4d2a6f2c1ef6fcc9debf72c0d
