Commit graph

156 commits

Author SHA1 Message Date
addshore
959bc315f2 MediaWikiTestCase to MediaWikiIntegrationTestCase
The name change happened some time ago, and I think its
about time to start using the name name!
(Done with a find and replace)

My personal motivation for doing this is that I have started
trying out vscode as an IDE for mediawiki development, and
right now it doesn't appear to handle php aliases very well
or at all.

Change-Id: I412235d91ae26e4c1c6a62e0dbb7e7cf3c5ed4a6
2020-06-30 17:02:22 +01:00
Thiemo Kreuz
5f3a92385b Fix visibility of setUp/tearDown
Change-Id: I636be48eb9f713680abac35d46091f7b49374696
2020-06-16 21:02:05 +02:00
Aaron Schulz
55d147e4bb objectcache: Split out BagOStuffTestBase class for easier subclassing
Create a simple APCUBagOStuff subclass and also fix
APCUBagOStuff::incrWithInit() default $init value.

Change-Id: If84963fe7dcfedd6edebfb8785235263e0868ece
2020-06-13 20:15:23 +00:00
Daimona Eaytoy
2b37cfaf18 build: Bump mediawiki-codesniffer to 31.0.0
Done with `composer fix` and suppressing the rest (i.e. sniffs for
global variables, which for core should be suppressed anyway).

Additionally, add `-p` to `phpcbf`, as otherwise it just seems stuck.

Change-Id: Ide8d6cdd083655891b6d654e78440fbda81ab2bc
2020-05-30 14:56:28 +00:00
Reedy
a8b006426e Fix tests/ PSR12.Properties.ConstantVisibility.NotFound
Change-Id: I0beed1a35e046705fb84c9d1f63cf92afd009bb4
2020-05-16 04:30:21 +01:00
Aaron Schulz
c95c6f7470 objectcache: add "non-global" mode to WANObjectCache "coalesceKeys"
This makes it easier to rollout one keyspace/project at a time even
if some keys are shared and receive purges. The shared keys can all
be done as the last step.

Also, simplify getMulti() to no longer need extractBaseKey().
Make the "warm up cache" logic a bit easier to follow and less
likely to copy values around.

Change-Id: I8b602ddf5dd1feaada45fb0af202c5603836a8dd
2020-05-08 17:01:31 +00:00
Aaron Schulz
9ec57d7e5b objectcache: make WANObjectCache::set() handle very slow regeneration
If a key always takes a very long time to regenerate, is popular,
and does not use lockTSE, it still needs to be cacheable. Since a
value cannot be more up-to-date than the time it takes to regenerate
it, take the "lower the TTL" approach for these cases. Use "walltime"
to narrow down the "reject the set()" case based on regeneration time.
This is already provided by getWithSetCallback() automatically.

Bug: T244877
Change-Id: Id43fb02738b28dad3bc922057efb7eee0272d0e1
2020-04-14 22:53:38 +00:00
Thiemo Kreuz
e1dd371e11 Make use of PHPUnit's assertCount feature where possible
… and avoid assertEmpty() on arrays, in favor of a much more strict
assertSame( [] ).

Change-Id: I20266b0b1fc38a3a87666ba1b0793cb2b37d94a9
2020-03-02 15:58:41 +00:00
Aaron Schulz
b9b3f366bb objectcache: fix "coalesceKeys" option name in WANObjectCache
Follow-up to 85bc62c5a8

Fix related unit tests that otherwise break as a result

Change-Id: I28b3a1537d319c68a7c12c578e1acfb916f3ec99
2020-02-13 03:17:34 +00:00
Aaron Schulz
85bc62c5a8 objectcache: add "coalesceKeys" option to WANObjectCache for key grouping
This is useful for grouping related keys on the same servers to reduce
the need for cache server connections and availability. A cache key that
uses "lockTSE" can already involve accessing several keys during the
read/write cache-aside paths:
a) The value key itself
b) The check key (named after the main key, a common pattern)
c) The mutex key (used if the value looks stale)
d) The cool-off key (used if regeneration took a while)

Any problems accessing the first two could cause extra value regenerations.
Problems with the mutex key could lead to stampedes due to threads assuming
another thread was regerating a soon-to-expire value when, in fact, none was.
A similar problem could happen with cool-off keys, with threads assuming
that another saved the newly regenerated value when, in fact, none did.

The use of hash stops puts the tiny related keys on the same server as the
main cache key that they serve. This is only for hash-based routing, and not
route prefix routing (e.g. All*Route still sends the key to multiple child
routes, but the PoolRoute/HashRoute function will hash differently).

The option is not enabled by default yet.

Change-Id: I37e92a88f356ef1e2a2b7728263040e2f6f09a13
2020-02-06 20:27:08 +00:00
jenkins-bot
51280df815 Merge "objectcache: fix cache pollution in WANObectCache Multi* methods" 2020-01-30 18:17:03 +00:00
Aaron Schulz
527fd0109f objectcache: fix cache pollution in WANObectCache Multi* methods
This was triggered by bad reference handling during preemptive refreshes

Bug: T235188
Change-Id: I239a3e1922f478c74c94d8d2debff28f525c7c31
2020-01-29 21:08:39 +00:00
jenkins-bot
0a46bef2db Merge "objectcache: fix storage of null values in WANObjectCache" 2020-01-29 04:14:20 +00:00
Aaron Schulz
4fb5210b62 objectcache: fix storage of null values in WANObjectCache
Bug: T234583
Change-Id: I38a531b9a0acb95d7884519f3381b48cd9d8faa0
2020-01-24 22:49:26 +00:00
James D. Forrester
5e9fca47b9 Coding style: Auto-fix MediaWiki.Usage.PHPUnit*
Change-Id: I86fc55a4fc8ceafe368692173211bbcd6d8581d7
2020-01-10 10:17:12 +00:00
jenkins-bot
f73d62d47d Merge "Add more logging to getWithSetCallback()" 2019-12-20 00:36:31 +00:00
Daimona Eaytoy
e501bb3b50 Manual fix of the remaining assertInternalType & Co.
Bug: T192167
Change-Id: I3b8176745f49b1d4bfe6f36978b7879d546f68a9
2019-12-15 00:20:10 +00:00
Daimona Eaytoy
7c9e3db1e6 Fixes for PHPUnit 8 compatibility
Bug: T192167
Change-Id: Ic14f5debc53e55d67146dc96279d26dfd52b4000
2019-12-10 17:02:06 +00:00
Tim Starling
29e33aa50a Fix random WANObjectCacheTest failures
We're observing WANObjectCacheTest fail sometimes, possibly due to a
1/825 chance of a cache key being counted as "popular" combined with an
mt_srand() call making this more likely. I tweaked the options to
getWithSetCallback() to avoid the affected mt_rand() call altogether.

I also added a lowTTL option to another getWithSetCallback() call,
eliminating a condition with a 3% chance. It didn't appear to matter
which branch the test took, but I'd rather avoid non-deterministic
behaviour.

There are a few remaining coin tosses being logged, but they all occur
with a chance of less than 10^-5.

Bug: T238197
Change-Id: Iadd3dc00d44b8490210fec97fe1f49236326300f
2019-11-20 05:00:48 +00:00
Tim Starling
24812e4096 Add more logging to getWithSetCallback()
To help isolate the referenced bug, which is a sporadic test failure
in testGetWithSetCallback().

Bug: T238197
Change-Id: If35d60340c804b6bfe1e9ddfcf53c76373c794b1
2019-11-18 16:32:47 +11:00
Max Semenik
48a323f702 tests: Add explicit return type void to setUp() and tearDown()
Bug: T192167
Depends-On: I581e54278ac5da3f4e399e33f2c7ad468bae6b43
Change-Id: I3a21fb55db76bac51afdd399cf40ed0760e4f343
2019-10-30 14:31:22 -07:00
James D. Forrester
83d76f4cb5 phpcs: Enable MediaWiki.Commenting.PhpunitAnnotations.ForbiddenExpectedException* and make pass
Change-Id: I63f97497714a32236268be6965c5e181dade6c58
2019-10-14 12:48:48 -07:00
Umherirrender
5bd311b1a2 Add public as visibility in tests folder
Add public, protected or private to function missing a visibility
Enable the tests folder for the phpcs sniff

Change-Id: Ibefce76ea9984c47e08c94889ea2eafca7565e2c
2019-10-10 21:55:37 +02:00
Timo Tijhof
28c470adc7 objectcache: Stricter assertions for WANObjectCache tests
Use the stricter comparisons from assertSame instead of
assertEquals.

In some cases this revealed a test that was asserting a value
different from the one actually returned:

* testSetAndGet:
  The test failed with the test case (null, 3), where it tries
  to store the value `null`, but gets back `false`, which is wrong.
  This appears to be a genuine bug, filed as T234583.

Change-Id: Ic3dcc7fa2e8749b0f2d68917a8ac728dda26b6ca
2019-10-06 02:10:24 +00:00
Max Semenik
bc3878e33a Begin cleaning up PHPUnit 4 code from tests
This process will be broken up into several parts for reviewability.

Bug: T192167
Change-Id: Ie415fd3308384a5ca2b3de24ba037785f8a3a714
2019-10-04 14:19:05 -07:00
Thiemo Kreuz
e4272518f7 tests: Replace PHPUnit's loose assertEquals(false) with assertFalse()
assertEquals( false, … ) still succeeds when the actual value is 0, null,
an empty string, even an empty array. All these should be reported as a
failure, I would argue.

Note this patch previously also touched assertSame( false ). I reverted
these. The only benefit would have been consistency within this codebase,
but there is no strict reason to prefer one over the other. assertFalse()
and assertSame( false ) are functionally identical.

Change-Id: Ic5f1c7d504e7249002d3184520012e03313137b4
2019-10-04 00:30:36 +00:00
Daimona Eaytoy
c12b32c9a3 HHVM removal: Kill HHVM_VERSION checks in PHPUnit tests
Change-Id: I1d18b2d6956d326e5f9e50cd01ce2bb683a5b653
2019-10-03 15:35:21 +00:00
Thiemo Kreuz
cff596804a tests: Replace assertions on count() == 0 with strict === []
The benefit of using count() is that the test would still succeed if
the return vfalue is not an array, but an iterable object. It seems
this is not needed.

Change-Id: I23529f6990aebe0cce86e236a21820fe74993204
2019-09-30 16:20:34 +02:00
jenkins-bot
864c4df8b9 Merge "tests: Replace PHPUnit's loose assertEquals(null) with assertNull()" 2019-09-27 22:43:09 +00:00
Thiemo Kreuz
c2211946f7 tests: Replace PHPUnit's loose assertEquals(null) with assertNull()
assertEquals( null, … ) still succeeds when the actual value is 0, false,
an empty string, even an empty array. All these should be reported as a
failure, I would argue.

Note this patch previously also touched assertSame( null ). I reverted
these. The only benefit would have been consistency within this codebase,
but there is no strict reason to prefer one over the other. assertNull()
and assertSame( null ) are functionally identical.

Change-Id: I92102e833a8bc6af90b9516826abf111e2b79aac
2019-09-27 19:15:38 +00:00
Aaron Schulz
223b7a3717 objectcache: fully respect "pcTTL" in WANObjectCache instead of using INF when >= 0
This was broken since 611e2d5596

Change-Id: I612eaf211ff698d5ab1c911aa58195b7bc44f00c
2019-09-24 02:56:39 -07:00
jenkins-bot
095c50ade1 Merge "tests: Prefer assertSame() when comparing the integer 0" 2019-09-20 15:33:02 +00:00
Thiemo Kreuz
32a429e8c4 tests: Prefer assertSame() when comparing the integer 0
assertSame() is guaranteed to not do any type conversion. This can be
critical when acciden tially comparing, for example, 0 to 0.0.

Change-Id: Iffcc9bda69573623ba14af655dcd697d0fcce525
2019-09-19 15:35:23 +00:00
Timo Tijhof
9244cdb977 objectcache: Escape dots from cache keys in StatsD metrics
Bug: T232907
Change-Id: Ia61256e24fe35803e69a83a1ca235e18297c75cd
2019-09-14 18:16:55 +00:00
Aaron Schulz
bee2a20903 objectcache: make WANObjectCache::relayPurge() actually use $holdoff
Fixes a regression introduced by 70547f3fa3 in 2016.

Change-Id: I5996b63c31ac3b382e838a6858b8585bdc8c585c
2019-08-24 22:41:39 +00:00
Aaron Schulz
94eb0c946f objectcache: improve BagOStuff arithmetic method signatures
Make the default $init value for incrWithInit() be $value.
This is far less suprising and also makes the operation
easier to replicate without conflicts.

Make decr() definitions more explicit since various cache
drivers do not handle negative incr() values (e.g. memcached).

Change-Id: I2b8d642656cc91c841abbd7a55d97eba101b027a
2019-08-24 12:50:37 -07:00
jenkins-bot
90d6f495bd Merge "objectcache: fix WRITE_ALLOW_SEGMENTS in BagOStuff cas() and add() methods" 2019-08-20 15:50:58 +00:00
Aaron Schulz
942f708b61 objectcache: fix WRITE_ALLOW_SEGMENTS in BagOStuff cas() and add() methods
Add MediumSpecificBagOStuff::getValueOrSegmentList() helper method.

Also:
* Use $keysMissing variable correctly in CachedBagOStuff::getMulti()
  to avoid extra overhead.
* Optimize mergeViaCas() when the current value matches the new one.

Change-Id: I5c4bd74379bc459216ac0278150ce3aecff3b851
2019-08-20 10:55:42 -04:00
Aaron Schulz
d421f22eb6 objectcache: make newFromId()/newWANcacheFromId()/getDefaultKeyspace() private
Change-Id: I30b99f23dbd9637ca8178d3a3650b4c38ec43e7d
2019-08-20 13:11:27 +00:00
Aaron Schulz
49025f52b2 objectcache: clean up MemcachedBagOStuff expiry handling
Partly a follow-up to 88640fd902.

Use real time in changeTTL() tests to fix all remaining
failures for BagOStuff sub-classes.

Change-Id: I537d665d6c8770a68a5a79233a913f1714881dfb
2019-08-10 12:12:12 -07:00
Aaron Schulz
88640fd902 objectcache: fix failing tests for non-HashBagOStuff backends
Use real time for testing absolute expirations with changeTTL().
Otherwise, backends like memcached or redis will fail since
they do not use the mock time.

Also:
* Make SqlBagOStuff actually override changeTTLMulti() by
  using the right method name
* Check TTL_INDEFINITE more explicitly for clarity
* Rename TTL conversion methods for clarity
* Use isRelativeExpiration() in MemcachedBagOStuff

Change-Id: I9365ceb31d4e7bef65906363d42b8c3020a66346
2019-08-07 14:07:22 +00:00
Aaron Schulz
47aa48f073 objectcache: make "busyValue" stricter to avoid callback ambigiuity
Change-Id: I01a1503ff5b37d65ef148fef79270505d8eb3146
2019-07-27 02:21:56 -04:00
jenkins-bot
f577358f43 Merge "objectcache: let BagOStuff::getWithSetCallback() callbacks modify the TTL" 2019-07-25 23:26:05 +00:00
jenkins-bot
7af9c25c88 Merge "objectcache: Use variadic signature for makeKey()" 2019-07-23 19:00:17 +00:00
Timo Tijhof
460e68ab3b objectcache: Use variadic signature for makeKey()
This should help fix the following issues that various repos
are getting from Phan as of late:

> Call with 5 arg(s) to \BagOStuff::makeKey() which only takes 2 arg(s)
> defined at ../../includes/libs/objectcache/BagOStuff.php:456
> <source="PhanParamTooMany"/>

Bug: T228563
Depends-On: I5cfba063821101325a5a7359e6b8ad71a0fb1b2f
Depends-On: Ifa5b96735376f2fbe3680799f960616ba8d357ff
Change-Id: Ic9df7f3ad7f356c7cbdfe1edfbe35821b931dda6
2019-07-23 20:44:06 +02:00
Aaron Schulz
f3919c7c79 objectcache: let BagOStuff::getWithSetCallback() callbacks modify the TTL
Also simplify the code by removing the is_callable() check and relying on
regular PHP errors instead of an exception for bad callbacks.

Change-Id: I084b0132c5fb05f1941a6d6839cfa74e2cf677f0
2019-07-23 07:33:40 -07:00
Aaron Schulz
735ec8d704 objectcache: reorganize WANObjectCache fields and avoid exposing internal constants
Change-Id: I95771fc8d032939e71adba3a416894004ea0847d
2019-07-20 16:04:08 -07:00
Aaron Schulz
bebe30333d objectcache: add MediumSpecificBagOStuff base class for non-proxy subclasses
This make it much clearer what needs to be overridden

Change-Id: I3073f8a0605f557c6a3a93d0d8401cddd0fb8dbe
2019-07-19 13:01:46 -07:00
jenkins-bot
ee801e1611 Merge "objectcache: clean up RedisBagOStuff and optimize changeTTLMulti()" 2019-07-18 02:07:31 +00:00
Aaron Schulz
d8b952ae47 objectcache: clean up RedisBagOStuff and optimize changeTTLMulti()
Also fix some IDEA warnings in redis classes and make it easy for IDEs
to recognize the Redis (phpredis) class calls to RedisConnRef proxies.

Bug: T113916
Change-Id: If45a37da412ac37e8c07dc3d1053826aa0a62077
2019-07-18 01:46:49 +00:00