Commit graph

26 commits

Author SHA1 Message Date
Aaron Schulz
278bcbb7a6 Reduce microtime()/mt_rand() fragility in WANObjectCacheTest
Tweak testLockTSESlow() tests to reduce mt_rand() dependency.

Also, rename last few $now variables to $mockWallClock and use
integral floats to make it easier to find raw microtime() values
via debugging statements.

Change-Id: Ie2b125e42d3ac37104f5b87b75a9a81558956518
2021-10-09 03:16:57 +00:00
Aaron Schulz
c7b6548bc5 objectcache: improve the WANObjectCache "lag mitigation" TTL logic
Handle edge case of fast callbacks that happen to run just before
the transaction age reaches MAX_READ_LAG. Such callbacks will no
longer be considered "slow".

Distinguish slow and fast regenerations for the case when TTL
mitigation is triggered by the combination of snapshot and
replication lag.

Reduce the cases where TTL_UNCACHEABLE is used as the mitigation
TTL, to avoid operational risk.

Simplify the mitigation TTL code by reducing needless variation
between the "lockTSE" and non-"lockTSE" cases.

Track the key and time of the last 10 get() calls that returned
either a stale value or no value at all. Use this this estimate
the walltime in set(), if it was not provided.

Bug: T285070
Change-Id: Ie6c88fbf7dbfc831d67bfb6cde082c123555a8cf
2021-09-14 00:17:56 +00:00
Timo Tijhof
11ed63fe89 objectcache: Update WANObjectCache test to not require fraction in purge value
In prep for the next commit and to confirm that it passes
both before and after.

Change-Id: I4d203843a8cb2486584df1601fec589389110f73
2021-09-11 19:48:08 +01:00
DannyS712
ae4efe66c6 Don't pass unneeded variables into anonymous functions
Change-Id: Iad4d1dc4727828f9e0120c8bf99dc245a29ee14d
2021-07-13 19:48:26 +00:00
Thiemo Kreuz
2ba01c7ee7 Remove some more comments that literally repeat the code
… including PHPDoc tags like `@return <type> $variableName`.
A return value doesn't have a variable name. I can see that
some people do this intentionally, repeating the variable
name that was used in the final `return $var;` at the end
of a method. This can indeed be helpful. I leave a lot of
these untouched and removed them only when it's obviously
wrong, or does not provide any additional information in
addition to what the code already says.

Change-Id: Ia18cd9f25ef658b08ad25b97a744897e2a8deffc
2021-06-18 21:23:56 +00:00
Thiemo Kreuz
b95a07380a Remove meaningless ->expects( $this->any() ) from all tests
It is not entirely meaningless. It might be an indicator that
the number of calls to a method is intentionally unlimited.
This is similar to e.g. an @inheritDoc PHPDoc comment that
marks a method as being "intentionally undocumented".

However, what's the meaning of being "intentionally
unconstrained"? Let's just not have any constraint then.

I feel all these ->expects( $this->any() ) bloat the test
code so much that it's never worth it.

Change-Id: I9925e7706bd03e1666f6eb0b284cb42b0dd3be23
2021-04-23 11:58:58 +02:00
Thiemo Kreuz
e2ec44d73f Replace ->exactly( 1 ) with ->once() in tests
It's shorter, the more canonical form, and easier to read
because of this.

Change-Id: I9c68a49c1f31bddfac91a74a6592fe3545ca00a7
2021-04-22 09:42:28 +02:00
Daimona Eaytoy
535d7abf59 phpunit: Mass-replace setMethods with onlyMethods and adjust
Ended up using
  grep -Prl '\->setMethods\(' . | xargs sed -r -i 's/setMethods\(/onlyMethods\(/g'

special-casing setMethods( null ) -> onlyMethods( [] )

and then manual fix of failing test (from PS2 onwards).

Bug: T278010
Change-Id: I012dca7ae774bb430c1c44d50991ba0b633353f1
2021-04-16 20:15:00 +02:00
Timo Tijhof
a6a25a1efc objectcache: Replace 'cluster' and 'mcrouterAware' with 'broadcastRoutingPrefix'
Also remove the unused 'region' option.

Added release notes, and updated the notes for coalesceKeys as well
to reside in the configuration section instead.

Change-Id: I219ea67416c6d121ce1488d99dab9a3a089f27d9
2021-04-09 23:04:41 +00:00
Timo Tijhof
16ac6980f8 objectcache: Implement 'onHostRoutingPrefix' option in WANObjectCache
Renamed several methods for clarity and updated various comments

Use explicit number_format() instead of casting floats to strings
when making purge values

Fix LocalRepoTest mocking of WANObjectCache::makeGlobalKey()

Bug: T264604
Change-Id: Ib03b97fcab8a34d38d3a8a12da6e28e12feef4ad
2021-04-09 00:27:13 +00:00
Aaron Schulz
19de9147d1 objectcache: cleanups to broadcast/relay code in WANObectCache
* Split makePurgeValue() into one for "check" keys and one for
  tombstones.
* Simplify tombstones by ommitting the zeroed-out holdoff component.
* Remove unused and deprecated HOLDOFF_NONE constant.
* Clarify various comments.

Change-Id: If6cc34538b1ea30764836b6870ba8648424ce9d7
2021-04-07 08:22:40 +00:00
Aaron Schulz
74ebf0ec9c objectcache: replace "coalesceKeys" with "coalesceScheme"
Key coalescing hints are now unconditionally enabled, using
the mcrouter style by default. This can be changed by setting
"coalesceScheme" (e.g. in case of experimental dynomite use).

Change-Id: I4866ed66d42d7d7da2b3edc2f99b8ba35fe3d511
2021-04-01 19:52:07 -07:00
jenkins-bot
2d2fe36d3c Merge "objectache: Revert "throw on Closure" in WANObjectCache" 2021-02-09 16:58:36 +00:00
Krinkle
0ec2643767 objectache: Revert "throw on Closure" in WANObjectCache
This reverts commit 4e94fa8812.

The extra serialization call here is not needed since storing
a value naturally results in serialization and thus exception
throwing already (hence the fatal error at T273242).
Performing serialization twice comes with a significant cost,
which has in past led to notable regressions.

Adding a try-catch makes sense, but this can be placed at a
higher level instead, as is already done by Ibfcbf0eceb4b7a36
which was already merged and back-ported, making this obsolete.

Other use cases:

- Continuous integration where HashBag is used, which doesn't
  need serialize and thus doesn't cover the Closure issue.
  Cost of serialize was removed there for performance reasons,
  but if we want to add it back, let's do so there in the natural
  place where it once was (or in MediumBag, which might be preferable
  so that we can use that same code path to gradually switch to JSON
  encoding and warn/throw for non-jsonic values).

- Regular getWithSet calls (rather than the async refresh from which
  the fatal is currently seen). For regular getWithSet calls we already
  get a useful stack trace that points directly to the line of code
  where the value and cache key are computed, so additional logging
  does not seem that valuable. But for consistency we could move the
  try-catch from Ibfcbf0eceb4b7a36e33a8 a little lower to cover all
  fetchOrRegenerate() calls, not just those from scheduleAsyncRefresh.

Bug: T273242
Change-Id: Ic649016dde7b58323a028ce6d4fe617ef62a1ff8
2021-02-09 16:27:48 +00:00
jenkins-bot
726f87c0c6 Merge "WANObjectCache: throw on Closure" 2021-02-09 13:35:31 +00:00
daniel
4e94fa8812 WANObjectCache: throw on Closure
Alternative to Ia91d9f8533d4cc58564473ee9a375ba6be90eb64

Bug: T273242
Change-Id: I76a6667a1fcdc35ff972ebcdf1ad0b95ef07e015
2021-02-09 11:10:41 +00:00
Umherirrender
a1de8b8700 Tests: Mark more more closures as static
Result of a new sniff I25a17fb22b6b669e817317a0f45051ae9c608208

Bug: T274036
Change-Id: I695873737167a75f0d94901fa40383a33984ca55
2021-02-09 02:55:57 +00:00
Reedy
729f20afc8 Tests: Mark more closures as static
Bug: T274036
Change-Id: I911d3041cebe417d162934223b46ea295c6d20e3
2021-02-07 13:26:56 +01:00
Umherirrender
62002cdcf1 build: Update mediawiki/mediawiki-codesniffer to 35.0.0
Change-Id: Idb413be4b8cba8611afdc022af59810ce1a4531e
2021-01-31 13:34:38 +00:00
jenkins-bot
f5ba23f9b4 Merge "objectcache: add statsd key metrics to BagOStuff classes" 2021-01-26 00:15:06 +00:00
Aaron Schulz
57325ba3bd objectcache: add statsd key metrics to BagOStuff classes
Update SQL, REST, and redis subclasses to emit call count and
payload size metrics for cache key operations. These metrics
are bucketed by cache key collection (similar to WANCache).

Bug: T235705
Change-Id: Icaa3fa1ae9c8b0f664c26ce70b7e1c4fc5f92767
2021-01-25 14:36:29 -08:00
Thiemo Kreuz
b655f382db Remove broken/outdated @param/@throws tags from @dataProviders
My personal best practice is to not document @params when there
is a @dataProvider. I mean, these test…() functions are not
meant to be called from anywhere. They do not really need
documentation. @param tags don't do much but duplicate what the
@dataProvider does. This is error-prone, as demonstrated by the
examples in this patch.

This patch also removes @throws tags from tests. A test…() can
never throw an exception. Otherwise the test would fail.

Most of these are found by the not yet released I10559d8.

Change-Id: I3782bca43f875687cd2be972144a7ab6b298454e
2021-01-21 03:42:42 +00:00
Thiemo Kreuz
2f66b3754f tests: Remove @param docs from test code that just repeat the signature
These are not only 100% identical to the actual code, but also:
* It's error-prone. Some are already wrong.
* These test…() functions are not meant to be called from
  anywhere. What is the target audience for this documentation?
* There is a @dataProvider. What such @param tags actually do is
  document the provider, but in an odd place. Just looking at
  the provider should give the same information.
* The MediaWiki CodeSniffer allows to skip @param when there is
  a @dataProvider, for the reasone listed.

Change-Id: I0f6f42f9a15776df944a0da48a50f9d5a2fb6349
2021-01-21 03:41:23 +00:00
DannyS712
6a93b0ca93 More misc test cleanup
* parent::setUp() should be first, and ::tearDown()
  should be last
* Move tests that directly extend PHPUnit\Framework\TestCase
  to /unit

Change-Id: I1172855c58f4f52a8f624e6d596ec43beb8c93ff
2020-12-24 00:52:06 +00:00
Legoktm
4e35134f7a Revert "Separate MediaWiki unit and integration tests"
This reverts commit 0a2b996278.

Reason for revert: Broke postgres tests.

Change-Id: I27d8e0c807ad5f0748b9611a4f3df84cc213fbe1
2019-06-13 23:00:08 +00:00
Máté Szabó
0a2b996278 Separate MediaWiki unit and integration tests
This changeset implements T89432 and related tickets and is based on exploration
done at the Prague Hackathon. The goal is to identify tests in MediaWiki core
that can be run without having to install & configure MediaWiki and its dependencies,
and provide a way to execute these tests via the standard phpunit entry point,
allowing for faster development and integration with existing tooling like IDEs.

The initial set of tests that met these criteria were identified using the work Amir did in
I88822667693d9e00ac3d4639c87bc24e5083e5e8. These tests were then moved into a new subdirectory
under phpunit/ and organized into a separate test suite. The environment for this suite
is set up via a PHPUnit bootstrap file without a custom entry point.

You can execute these tests by running:
$ vendor/bin/phpunit -d memory_limit=512M -c tests/phpunit/unit-tests.xml

Bug: T89432
Bug: T87781
Bug: T84948
Change-Id: Iad01033a0548afd4d2a6f2c1ef6fcc9debf72c0d
2019-06-13 22:56:31 +02:00
Renamed from tests/phpunit/includes/libs/objectcache/WANObjectCacheTest.php (Browse further)