I noticed this in CI during development for Ia893ddb36427eb5e9bff,
however, it is reproducible on latest master as well. Locally, it
happens about 1/30 times:
```
$ php phpunit.php --filter testLockTSESlow unit/includes/libs/objectcache/
Using PHP 8.1.13
PHPUnit 9.5.27 by Sebastian Bergmann and contributors.
There was 1 failure:
1) WANObjectCacheTest::testLockTSESlow with data set #0 (array('hash_tag'), '{')
Callback not used (mutex not acquired, stale value used)
Failed asserting that 3 is identical to 2.
/w/tests/phpunit/unit/includes/libs/objectcache/WANObjectCacheTest.php:1311
/w/tests/phpunit/MediaWikiUnitTestCase.php:11
```
* Remove randomisation from test. The HashBag is empty at this point,
and a dedicated instance is created for each test case. Remove doubt
over conflicts or other uncertainty, and make it easier to debug at
the same time.
* Increase mocked time gap from 2s to 32s similar to what the
previous assertion in the same test does already. 2s does not actually
go past the reduced logical TTL which is 30s at this point.
It seems to still pass on master in most cases, but it makes sense
that it can fail. I could not reproduce failures after this change.
Change-Id: I6e491ae3c4f41413e7a952aa0796a787d559ac48
Loosen up the code coverage a bit so that we can more clearly identity
what logic truly isn't excercised by tests. It also restores some
gaps that were left behind by various refactors when code moved around
or out into various private methods.
Change-Id: I4e65f92ea1eb3f0e781ada4fb2ced0f8ef9d3cc4
This avoids the latency of waiting for a response.
Change yieldStampedeLock() to use delete() instead of changeTTL() since
the PECL memcached client does not support WRITE_BACKGROUND with TOUCH.
Also, simply relayVolatilePurges() into relayVolatilePurge() and tweak
some related documentation for WANObjectCache
Bug: T302623
Change-Id: I16f011b730f34332aa54a48c579de3c6606dafda
This allows for large string value storage similar to that
of direct BagOStuff callers.
Bug: T309126
Change-Id: I22d2d8bb520ef2ea6ccd28016d34db36fe27c3df
Make set() and getWithSetCallback() wrap setMainValue(), avoiding some
duplicate option array resolution logic.
Track set() calls via statsd, now that it can be distinguished from
getWithSetCallback() invocations.
Change-Id: Ia1e7b244b1db3b76314d718b3aea9e6e708235c0
Introduced in 2017 with I7f14b9ca2533032 (2e5eb693) but remains
unused at WMF, and disabled by default.
Follows-up I62107789fa (9e49260fc9) which added reap to LinkCache
test cases in 2021.
Change-Id: I0654c29a671467dd6b366f462d1c09b90a273413
Follows-up Ia550ef7fe3 (4d3549ad71) which added reap to NameTableStore
in 2018, which was then (mostly) removed again with I63e61e1ab (2c7e4adcea)
for T198561, except one call was left behind in reloadMap(). I've
changed this to match the existing delete call in acquireId().
To be removed in I0654c29a671467dd6.
Change-Id: I7541c9c6d1ef70d552944aca94cbc4d0f7ec0d5b
And also assertNotRegExp -> assertDoesNotMatchRegularExpression. The
methods were renamed in PHPUnit 9.
Done automatically with:
grep -rl assertRegExp tests/ | xargs sed -r -i "s/>assertRegExp\(/>assertMatchesRegularExpression\(/"
grep -rl assertNotRegExp tests/ | xargs sed -r -i "s/>assertNotRegExp\(/>assertDoesNotMatchRegularExpression\(/"
Split out from Ifdba0f9e98eb6bce4590b7eb73170c51a697d7c6 so that it
remains smaller and easier to review.
Also make a test use MediaWikiUnitTestCase (it's already in the unit/
dir) so that it can access the forward-compat method.
Bug: T243600
Change-Id: Ifa279d5f201d7abeebece292141ebface8278046
It took me a good half hour to understand what these tests are doing,
and another half hour to build up a modicum of confidence that the
current behaviour is the indeed the desired behaviour.
* Remove random strings.
* Remove indirection and computed expected values as much as possible
in favour of inlined literals that explicitly expect outcomes.
* Remove confusing second argument from the data provider
getMultiWithUnionSetCallback_provider, which wasn't used, and
add labels to it.
* Remove some of the existing inline comments.
* Add some inline comments with what I've understood.
* Fix bug in "C: Repeat case with recently created check keys",
which claimed to be observing that caches were ignored due to the
check key being too new, when in fact it was a cache miss simply
because the key didn't exist yet upto this point. Make it actually
repeating by adding a second call afterwards to observe the
stated behaviour.
Bug: T303092
Change-Id: I874a20ecd49464f7b0e6a0b579c000dc0e804c0a
The examples and comments were misleading with regard to the handling
of missing entities.
Bug: T303092
Change-Id: Ib672fd18d1270c35253683eb54b48be056e6dc3c
Tweak testLockTSESlow() tests to reduce mt_rand() dependency.
Also, rename last few $now variables to $mockWallClock and use
integral floats to make it easier to find raw microtime() values
via debugging statements.
Change-Id: Ie2b125e42d3ac37104f5b87b75a9a81558956518
Handle edge case of fast callbacks that happen to run just before
the transaction age reaches MAX_READ_LAG. Such callbacks will no
longer be considered "slow".
Distinguish slow and fast regenerations for the case when TTL
mitigation is triggered by the combination of snapshot and
replication lag.
Reduce the cases where TTL_UNCACHEABLE is used as the mitigation
TTL, to avoid operational risk.
Simplify the mitigation TTL code by reducing needless variation
between the "lockTSE" and non-"lockTSE" cases.
Track the key and time of the last 10 get() calls that returned
either a stale value or no value at all. Use this this estimate
the walltime in set(), if it was not provided.
Bug: T285070
Change-Id: Ie6c88fbf7dbfc831d67bfb6cde082c123555a8cf
… including PHPDoc tags like `@return <type> $variableName`.
A return value doesn't have a variable name. I can see that
some people do this intentionally, repeating the variable
name that was used in the final `return $var;` at the end
of a method. This can indeed be helpful. I leave a lot of
these untouched and removed them only when it's obviously
wrong, or does not provide any additional information in
addition to what the code already says.
Change-Id: Ia18cd9f25ef658b08ad25b97a744897e2a8deffc
It is not entirely meaningless. It might be an indicator that
the number of calls to a method is intentionally unlimited.
This is similar to e.g. an @inheritDoc PHPDoc comment that
marks a method as being "intentionally undocumented".
However, what's the meaning of being "intentionally
unconstrained"? Let's just not have any constraint then.
I feel all these ->expects( $this->any() ) bloat the test
code so much that it's never worth it.
Change-Id: I9925e7706bd03e1666f6eb0b284cb42b0dd3be23
Ended up using
grep -Prl '\->setMethods\(' . | xargs sed -r -i 's/setMethods\(/onlyMethods\(/g'
special-casing setMethods( null ) -> onlyMethods( [] )
and then manual fix of failing test (from PS2 onwards).
Bug: T278010
Change-Id: I012dca7ae774bb430c1c44d50991ba0b633353f1
Also remove the unused 'region' option.
Added release notes, and updated the notes for coalesceKeys as well
to reside in the configuration section instead.
Change-Id: I219ea67416c6d121ce1488d99dab9a3a089f27d9
Renamed several methods for clarity and updated various comments
Use explicit number_format() instead of casting floats to strings
when making purge values
Fix LocalRepoTest mocking of WANObjectCache::makeGlobalKey()
Bug: T264604
Change-Id: Ib03b97fcab8a34d38d3a8a12da6e28e12feef4ad
* Split makePurgeValue() into one for "check" keys and one for
tombstones.
* Simplify tombstones by ommitting the zeroed-out holdoff component.
* Remove unused and deprecated HOLDOFF_NONE constant.
* Clarify various comments.
Change-Id: If6cc34538b1ea30764836b6870ba8648424ce9d7
Key coalescing hints are now unconditionally enabled, using
the mcrouter style by default. This can be changed by setting
"coalesceScheme" (e.g. in case of experimental dynomite use).
Change-Id: I4866ed66d42d7d7da2b3edc2f99b8ba35fe3d511
This reverts commit 4e94fa8812.
The extra serialization call here is not needed since storing
a value naturally results in serialization and thus exception
throwing already (hence the fatal error at T273242).
Performing serialization twice comes with a significant cost,
which has in past led to notable regressions.
Adding a try-catch makes sense, but this can be placed at a
higher level instead, as is already done by Ibfcbf0eceb4b7a36
which was already merged and back-ported, making this obsolete.
Other use cases:
- Continuous integration where HashBag is used, which doesn't
need serialize and thus doesn't cover the Closure issue.
Cost of serialize was removed there for performance reasons,
but if we want to add it back, let's do so there in the natural
place where it once was (or in MediumBag, which might be preferable
so that we can use that same code path to gradually switch to JSON
encoding and warn/throw for non-jsonic values).
- Regular getWithSet calls (rather than the async refresh from which
the fatal is currently seen). For regular getWithSet calls we already
get a useful stack trace that points directly to the line of code
where the value and cache key are computed, so additional logging
does not seem that valuable. But for consistency we could move the
try-catch from Ibfcbf0eceb4b7a36e33a8 a little lower to cover all
fetchOrRegenerate() calls, not just those from scheduleAsyncRefresh.
Bug: T273242
Change-Id: Ic649016dde7b58323a028ce6d4fe617ef62a1ff8
Update SQL, REST, and redis subclasses to emit call count and
payload size metrics for cache key operations. These metrics
are bucketed by cache key collection (similar to WANCache).
Bug: T235705
Change-Id: Icaa3fa1ae9c8b0f664c26ce70b7e1c4fc5f92767
My personal best practice is to not document @params when there
is a @dataProvider. I mean, these test…() functions are not
meant to be called from anywhere. They do not really need
documentation. @param tags don't do much but duplicate what the
@dataProvider does. This is error-prone, as demonstrated by the
examples in this patch.
This patch also removes @throws tags from tests. A test…() can
never throw an exception. Otherwise the test would fail.
Most of these are found by the not yet released I10559d8.
Change-Id: I3782bca43f875687cd2be972144a7ab6b298454e
These are not only 100% identical to the actual code, but also:
* It's error-prone. Some are already wrong.
* These test…() functions are not meant to be called from
anywhere. What is the target audience for this documentation?
* There is a @dataProvider. What such @param tags actually do is
document the provider, but in an odd place. Just looking at
the provider should give the same information.
* The MediaWiki CodeSniffer allows to skip @param when there is
a @dataProvider, for the reasone listed.
Change-Id: I0f6f42f9a15776df944a0da48a50f9d5a2fb6349
* parent::setUp() should be first, and ::tearDown()
should be last
* Move tests that directly extend PHPUnit\Framework\TestCase
to /unit
Change-Id: I1172855c58f4f52a8f624e6d596ec43beb8c93ff
This changeset implements T89432 and related tickets and is based on exploration
done at the Prague Hackathon. The goal is to identify tests in MediaWiki core
that can be run without having to install & configure MediaWiki and its dependencies,
and provide a way to execute these tests via the standard phpunit entry point,
allowing for faster development and integration with existing tooling like IDEs.
The initial set of tests that met these criteria were identified using the work Amir did in
I88822667693d9e00ac3d4639c87bc24e5083e5e8. These tests were then moved into a new subdirectory
under phpunit/ and organized into a separate test suite. The environment for this suite
is set up via a PHPUnit bootstrap file without a custom entry point.
You can execute these tests by running:
$ vendor/bin/phpunit -d memory_limit=512M -c tests/phpunit/unit-tests.xml
Bug: T89432
Bug: T87781
Bug: T84948
Change-Id: Iad01033a0548afd4d2a6f2c1ef6fcc9debf72c0d
2019-06-13 22:56:31 +02:00
Renamed from tests/phpunit/includes/libs/objectcache/WANObjectCacheTest.php (Browse further)
