Commit graph

109 commits

Author SHA1 Message Date
Daimona Eaytoy
25ba3b3794 Document more scalar types for BagOStuff::makeKey and Language::formatNum
These were spotted by phan's stricter type checks. All make*Key methods
can also take integers, and that's pretty common across core and
extensions.
Language::formatNum can also take numbers (and tests cover this case);
in this case, move the cast to the top of the method, so that we're
passing a string to strtr, too.

The methods touched in this patch are the main source of false positives
in both core and extensions.

Change-Id: I6d538ce3ca7fd2d495c2bafbab7cc279da69db1c
2019-12-15 00:02:04 +00:00
Daimona Eaytoy
327e8ea416 Unsuppress phan issues part 6
Bug: T231636
Depends-On: I50377746f01749b058c39fd8229f9d566224cc43
Change-Id: I2cd24e73726394e3200a570c45d5e86b6849bfa9
2019-09-01 09:48:45 +00:00
Aaron Schulz
94eb0c946f objectcache: improve BagOStuff arithmetic method signatures
Make the default $init value for incrWithInit() be $value.
This is far less suprising and also makes the operation
easier to replicate without conflicts.

Make decr() definitions more explicit since various cache
drivers do not handle negative incr() values (e.g. memcached).

Change-Id: I2b8d642656cc91c841abbd7a55d97eba101b027a
2019-08-24 12:50:37 -07:00
Aaron Schulz
b70e54e986 objectcache: add BagOStuff::fieldHasFlags() convenience method
Change-Id: Id2bf4ec0dd0999f988b70a895003c4b1aaae51a2
2019-08-22 15:07:14 -07:00
Aaron Schulz
88640fd902 objectcache: fix failing tests for non-HashBagOStuff backends
Use real time for testing absolute expirations with changeTTL().
Otherwise, backends like memcached or redis will fail since
they do not use the mock time.

Also:
* Make SqlBagOStuff actually override changeTTLMulti() by
  using the right method name
* Check TTL_INDEFINITE more explicitly for clarity
* Rename TTL conversion methods for clarity
* Use isRelativeExpiration() in MemcachedBagOStuff

Change-Id: I9365ceb31d4e7bef65906363d42b8c3020a66346
2019-08-07 14:07:22 +00:00
jenkins-bot
f577358f43 Merge "objectcache: let BagOStuff::getWithSetCallback() callbacks modify the TTL" 2019-07-25 23:26:05 +00:00
Timo Tijhof
460e68ab3b objectcache: Use variadic signature for makeKey()
This should help fix the following issues that various repos
are getting from Phan as of late:

> Call with 5 arg(s) to \BagOStuff::makeKey() which only takes 2 arg(s)
> defined at ../../includes/libs/objectcache/BagOStuff.php:456
> <source="PhanParamTooMany"/>

Bug: T228563
Depends-On: I5cfba063821101325a5a7359e6b8ad71a0fb1b2f
Depends-On: Ifa5b96735376f2fbe3680799f960616ba8d357ff
Change-Id: Ic9df7f3ad7f356c7cbdfe1edfbe35821b931dda6
2019-07-23 20:44:06 +02:00
Aaron Schulz
f3919c7c79 objectcache: let BagOStuff::getWithSetCallback() callbacks modify the TTL
Also simplify the code by removing the is_callable() check and relying on
regular PHP errors instead of an exception for bad callbacks.

Change-Id: I084b0132c5fb05f1941a6d6839cfa74e2cf677f0
2019-07-23 07:33:40 -07:00
Aaron Schulz
bebe30333d objectcache: add MediumSpecificBagOStuff base class for non-proxy subclasses
This make it much clearer what needs to be overridden

Change-Id: I3073f8a0605f557c6a3a93d0d8401cddd0fb8dbe
2019-07-19 13:01:46 -07:00
jenkins-bot
3509329707 Merge "objectcache: optimize MemcachedPeclBagOStuff::*Multi() write methods" 2019-07-18 15:28:13 +00:00
Aaron Schulz
efba2a4afc objectcache: optimize MemcachedPeclBagOStuff::*Multi() write methods
Make use of Memcached::OPT_NO_BLOCK and Memcached::OPT_BUFFER_WRITES
at selective points to avoid slow internal blocking for loops in the
PECL extension methods. As this mode makes the return values less
meaningful, this is only triggered only if WRITE_BACKGROUND is
provided (it only catches some glaring format or size errors that the
client runs into).

Leave changeTTLMulti() to the default as the above approach would not
make it faster, since the diver does not buffer for that command.

Expand mctest.php to include *Multi() methods and also break down the
single key method timings by method.

Change-Id: I11fc58e1a2296778b2526a0b2f3103fef1baba0c
2019-07-15 19:45:37 -07:00
Aaron Schulz
b72a00383a objectcache: make BagOStuff::getMulti() preserve order and omit keys with missing segments
A key missing segments should count as non-existing, meaning that it should
not appear in the return value at all

Change-Id: I0f08b3dba77e0d335ebb2d75061927ebd36570d2
2019-07-14 00:31:57 +00:00
Aaron Schulz
383460afa3 objectcache: normalize BagOStuff method overriding pattern for *Multi() methods
Change-Id: I1bebb60307b1a166461cb5f9a55a79194cc0e363
2019-07-11 04:32:35 -07:00
Aaron Schulz
20a9c8d24e bagostuff: optimize SqlBagOStuff and fix failing segmentation tests
In SqlBagOStuff:
* Add modifyMulti() helper method to reduce code duplication
* Improve atomicity of add(), cas(), and changeTTL() queries
* Avoid integer serialization and improve atomicity of incr()
* Optimize new BagOStuff::changeTTLMulti() method

In BagOStuff:
* Add changeTTLMulti() method for subclasses to optimize
* Make set() ignore WRITE_ALLOW_SEGMENTS for integers so incr() works
* Strip WRITE_ALLOW_SEGMENTS flag from the setMulti() call in set() to
  avoid triggering bogus sanity check exceptions
* Fix BagOStuffTest::testSetSegmentable failures via the above changes
* Enforce WRITE_ALLOW_SEGMENTS sanity check in setMulti() for all the
  subclasses by using a final wrapper method
* Add WRITE_ALLOW_SEGMENTS sanity check to deleteMulti()

Bug: T113916
Change-Id: I25d1790fa9b0d1837643efccfa94a12043cfbf42
2019-07-09 22:53:18 -07:00
Aaron Schulz
4245a2c200 objectcache: deleteObjectsExpiringBefore() signature and code improvements
Make the method use a callable type hint for $progressCallback.

Also make the SQLBagOStuff subclass shuffle() the array of server and table
indexes in case the limit keeps getting applied. Only garbage collect on DBs
that were going to be written to anyway. Also use DeferredUpdates if possible.

Change-Id: I723e6377c26750ff98e33f7ab103c6162ae65f43
2019-07-09 02:32:06 -07:00
jenkins-bot
e4975895e6 Merge "objectcache: tweak SqlBagOStuff purging to happen on write only" 2019-07-08 23:07:02 +00:00
Aaron Schulz
670687fae0 objectcache: add IStoreKeyEncoder interface for key generation helper methods
If a key is stored in both BagOStuff and WANObjectCache, it useful to have
one common method for generating the cache and storage keys.

Change-Id: I8b77651746a55e8a98a79db298226e334d15a494
2019-07-08 22:33:07 +00:00
Aaron Schulz
c860f99a6e objectcache: tweak SqlBagOStuff purging to happen on write only
Also adjust the frequency default and put in a 100 row limit for
randomized purges. This adds a $limit parameter to the BagOStuff
method deleteObjectsExpiringBefore().

Change-Id: I66bfcc67e8e118f8c659dbbee13d54bf2cd937f5
2019-07-06 00:04:13 -07:00
Aaron Schulz
e2e2eb49d6 objectcache: optimize lock() and unlock() methods in SqlBagOStuff
Also clean up base method in BagOStuff.

Bug: T113916
Change-Id: I3a1c6afe531a8eae34608bc7fe0aa6f9f4d439fe
2019-07-04 14:00:02 +00:00
jenkins-bot
699920cc66 Merge "objectcache: add object segmentation support to BagOStuff" 2019-06-13 18:19:17 +00:00
Aaron Schulz
b09b3980f9 objectcache: add object segmentation support to BagOStuff
Use it for ApiStashEdit so that large PaserOutput can be stored.

Add flag to allow for value segmentation on set() in BagOStuff.
Also add a flag for immediate deletion of segments on delete().

BagOStuff now has base serialize()/unserialize() methods.

Bug: T204742
Change-Id: I0667a02612526d8ddfd91d5de48b6faa78bd1ab5
2019-06-11 16:14:17 +01:00
Lucas Werkmeister
d7e71b5e71 objectcache: Make BagOStuff::getCurrentTime() public
If another cache wraps a BagOStuff, it may need to know about the
BagOStuff’s notion of the current time, which may be mocked and differ
from the real time.

Change-Id: I40f4085d4cf549314394d3140cccb1fe4e48690a
2019-06-09 17:16:12 +00:00
Aaron Schulz
62b8db6ea5 objectcache: remove some unneeded code from BagOStuff::mergeViaCas
Change-Id: Ib92090fe4e1b1eb2c4f9e031f8476b24189392c8
2019-03-27 19:33:07 -07:00
Aaron Schulz
962495b17a objectcache: merge getWithToken() into doGet() for simplicity
Change-Id: I581f866521e1086ca350973d9cdeff6656f48fe8
2019-03-27 13:03:25 -07:00
Aaron Schulz
d5afd5f909 objectcache: minor refactoring to BagOStuff
Split up expiry functions and moved add() near other primitive writes.

Change-Id: I97bb91e648b797b1ab5a6934d212689b1e67f7c6
2019-03-26 15:14:42 -07:00
Aaron Schulz
e626d014ac objectcache: remove BagOStuff::mergeViaLock() and update RESTBagOStuff
All subclasses are now using the CAS variant, which respects $attempts.

Change-Id: Ia7ec6a7f3337cabe95c54c1142c3c5464c1794e7
2019-03-24 00:35:23 -07:00
Aaron Schulz
c2150cdc3c objectcache: make BagOStuff::changeTTL() more atomic
Change-Id: I86493498ef3359046b12d51795cf7875af6c3a6c
2019-03-20 09:46:33 -07:00
Aaron Schulz
560774814d Make BagOStuff::incr abstract to discourage bad implementations
Callers should really use atomic TTL-preserving implementations
so that calling code works correctly. The old default base class
code did not do either.

Change-Id: Icf66db05e48b86c8d481dc08dc9041bd1fa6dbe9
2019-03-18 16:09:26 -07:00
jenkins-bot
4d1478c76f Merge "Fix WatchedItemStore last-seen stashing logic" 2019-03-15 21:46:07 +00:00
Aaron Schulz
6612673b20 objectcache: make BagOStuff::add() abstract to discourage non-atomic versions
Change-Id: If3c3fbf21207b0c74cad8a29fa5bbabe0af896e3
2019-03-15 00:38:28 +00:00
Aaron Schulz
e2b03d5d30 objectcache: add BagOStuff::deleteMulti() method for consistency
Also:
* Make the BagOStuff tests actually pass for memcached and sql
* Add more unit tests for BagOStuff
* Make SqlBagOStuff::add() more atomic

Change-Id: Ic1eec0990a66b595b57c646498c3bd229442230c
2019-03-14 23:12:30 +00:00
Aaron Schulz
588a464682 Fix WatchedItemStore last-seen stashing logic
This should be the "last revision seen" timestamp, which is
different than the "first revision not seen" timestamp in the DB.

Also make sure that SpecialWatchlist accounts for the stash values.

Relatedly, better document the callback usage in BagOStuff::merge().

Change-Id: I98b03a5cd40fec5b4a2633d499ff77079d264e3c
2019-03-14 14:52:29 -07:00
Aaron Schulz
547f207fa6 objectcache: make the $flags argument appear more consistently in BagOStuff
Change-Id: I08879ede5e9f0ab227497bb1dab89ea61b65abce
2019-03-12 22:55:18 +00:00
jenkins-bot
501ca07005 Merge "Streamline code involving .= string concatenations" 2019-03-11 12:39:04 +00:00
Thiemo Kreuz
cfda5e7efa Streamline code involving .= string concatenations
This was inspired by Idbbdb31. Originally, I did a regex search for
code that did string concatenations like `$str = $str . …` and replaced
them all with the .= operator. A duplicate patch was uploaded by another
author. I rebeased this patch on top of the other, which leaves all
the manual optimizations I did.

Change-Id: Iaeb73d9c63302c9409bd1051b91e0d2bd77788a7
2019-03-11 12:43:45 +01:00
jenkins-bot
dbe54b46e6 Merge "objectcache: move ERR_* constants to IExpiringStore" 2019-03-08 21:40:04 +00:00
Aaron Schulz
9fe2e8eb7e objectcache: remove BagOStuff::modifySimpleRelayEvent()
This was only there as part of EventRelayer use by WAN cache, which
has since been removed. No other usage really makes sense.

Change-Id: I201425bb88739dd8febbfbc3d24a15e6d3d88d9c
2019-03-08 13:17:26 -08:00
Aaron Schulz
74be3a0150 objectcache: move ERR_* constants to IExpiringStore
Also removed unused ERR_* constant from WANObjectCache and fixed
comment in the constructor.

Change-Id: I4377fc3f533113a459b648fad084ba097ff51762
2019-03-08 13:12:01 -08:00
jenkins-bot
db1f41172f Merge "objectcache: add $flags argument to BagOStuff::delete()" 2019-03-08 00:51:25 +00:00
jenkins-bot
a8caf02564 Merge "objectcache: fix comment typo for addBusyCallback()" 2019-03-07 23:58:51 +00:00
Aaron Schulz
5dd409cd38 objectcache: add $flags argument to BagOStuff::delete()
This makes it consistent with set() and merge(). Also, one subclass
was already using the field in this manner.

Clean up the code related to WRITE_SYNC in SqlBagOStuff.

Change-Id: I0fb84f4475311889507d3ef98afd4476fb81174f
2019-03-07 13:15:29 -08:00
Aaron Schulz
84cc936ab6 objectcache: fix comment typo for addBusyCallback()
Change-Id: I05a4d84d40b81ba25b89238b36cdb64c71e2de1e
2019-03-07 12:25:02 -08:00
Fomafix
f17c297624 Use short assignment operator in PHP
Use
  $var .= $foo
instead of
  $var = $var . $foo

Change-Id: I5dcdd7278e618c14968e5ac1fb8ea43ac2200deb
2019-03-07 09:55:49 +01:00
jenkins-bot
97b467d8ac Merge "Remove unused use in anonymous function" 2018-12-02 00:00:53 +00:00
Jakub Vrana
ff48659011 Remove unused use in anonymous function
Found by PHPStan.

Change-Id: I88247514703820a6afb7c9861dcf64326aa32d19
2018-12-01 10:17:34 +01:00
Jakub Vrana
3fc3b9e578 Change typehint callback to callable
Found by PHPStan.

Change-Id: I77877a18131bd69996bad07f2ee1c5f3ba3ba2e7
2018-12-01 10:02:48 +01:00
Fomafix
43244db9a2 Use PHP 7 '??' operator instead of if-then-else
Change-Id: If9d4be5d88c8927f63cbb84dfc8181baf62ea3eb
2018-10-21 21:46:46 +02:00
Reedy
a075271157 Update composer/spdx-licenses to 1.4.0 and mediawiki/mediawiki-codesniffer to 21.0.0
https://github.com/composer/spdx-licenses/compare/1.3.0...1.4.0

Change-Id: I39f7a1310455159866bfed5224536e800befec0d
2018-07-26 17:44:28 +00:00
jenkins-bot
d80a3827ab Merge "objectcache: improve logging and error handling in BagOStuff" 2018-07-25 04:16:18 +00:00
Aaron Schulz
c705435628 objectcache: improve logging and error handling in BagOStuff
Bug: T198239
Bug: T199383
Change-Id: I0b4110396d03b98e83a7b614caf57d7e7c284361
2018-07-24 19:11:37 +00:00