Commit graph

36 commits

Author SHA1 Message Date
Max Semenik
06a275f48e Remove more HHVM hacks
Change-Id: I6bd298ef3b887173b87004ee055be2a4f6ea5f11
2019-10-06 10:16:09 +00:00
DannyS712
4dfae3bcc7 Fix a typo (yeild -> yield)
Bug: T201491
Change-Id: I70443514d55c681e51233b6e80ac7911c5e12d5a
2019-09-13 03:38:14 +00:00
Daimona Eaytoy
e2e543f7c2 Unsuppress more phan issues (part 5)
Bug: T231636
Depends-On: I6e5fba7bd273219b1206559420b5bdb78734aa84
Change-Id: I50377746f01749b058c39fd8229f9d566224cc43
2019-09-01 09:48:31 +00:00
Aaron Schulz
94eb0c946f objectcache: improve BagOStuff arithmetic method signatures
Make the default $init value for incrWithInit() be $value.
This is far less suprising and also makes the operation
easier to replicate without conflicts.

Make decr() definitions more explicit since various cache
drivers do not handle negative incr() values (e.g. memcached).

Change-Id: I2b8d642656cc91c841abbd7a55d97eba101b027a
2019-08-24 12:50:37 -07:00
Aaron Schulz
b70e54e986 objectcache: add BagOStuff::fieldHasFlags() convenience method
Change-Id: Id2bf4ec0dd0999f988b70a895003c4b1aaae51a2
2019-08-22 15:07:14 -07:00
Timo Tijhof
460e68ab3b objectcache: Use variadic signature for makeKey()
This should help fix the following issues that various repos
are getting from Phan as of late:

> Call with 5 arg(s) to \BagOStuff::makeKey() which only takes 2 arg(s)
> defined at ../../includes/libs/objectcache/BagOStuff.php:456
> <source="PhanParamTooMany"/>

Bug: T228563
Depends-On: I5cfba063821101325a5a7359e6b8ad71a0fb1b2f
Depends-On: Ifa5b96735376f2fbe3680799f960616ba8d357ff
Change-Id: Ic9df7f3ad7f356c7cbdfe1edfbe35821b931dda6
2019-07-23 20:44:06 +02:00
Aaron Schulz
bebe30333d objectcache: add MediumSpecificBagOStuff base class for non-proxy subclasses
This make it much clearer what needs to be overridden

Change-Id: I3073f8a0605f557c6a3a93d0d8401cddd0fb8dbe
2019-07-19 13:01:46 -07:00
Aaron Schulz
383460afa3 objectcache: normalize BagOStuff method overriding pattern for *Multi() methods
Change-Id: I1bebb60307b1a166461cb5f9a55a79194cc0e363
2019-07-11 04:32:35 -07:00
Aaron Schulz
20a9c8d24e bagostuff: optimize SqlBagOStuff and fix failing segmentation tests
In SqlBagOStuff:
* Add modifyMulti() helper method to reduce code duplication
* Improve atomicity of add(), cas(), and changeTTL() queries
* Avoid integer serialization and improve atomicity of incr()
* Optimize new BagOStuff::changeTTLMulti() method

In BagOStuff:
* Add changeTTLMulti() method for subclasses to optimize
* Make set() ignore WRITE_ALLOW_SEGMENTS for integers so incr() works
* Strip WRITE_ALLOW_SEGMENTS flag from the setMulti() call in set() to
  avoid triggering bogus sanity check exceptions
* Fix BagOStuffTest::testSetSegmentable failures via the above changes
* Enforce WRITE_ALLOW_SEGMENTS sanity check in setMulti() for all the
  subclasses by using a final wrapper method
* Add WRITE_ALLOW_SEGMENTS sanity check to deleteMulti()

Bug: T113916
Change-Id: I25d1790fa9b0d1837643efccfa94a12043cfbf42
2019-07-09 22:53:18 -07:00
Aaron Schulz
4245a2c200 objectcache: deleteObjectsExpiringBefore() signature and code improvements
Make the method use a callable type hint for $progressCallback.

Also make the SQLBagOStuff subclass shuffle() the array of server and table
indexes in case the limit keeps getting applied. Only garbage collect on DBs
that were going to be written to anyway. Also use DeferredUpdates if possible.

Change-Id: I723e6377c26750ff98e33f7ab103c6162ae65f43
2019-07-09 02:32:06 -07:00
Aaron Schulz
c860f99a6e objectcache: tweak SqlBagOStuff purging to happen on write only
Also adjust the frequency default and put in a 100 row limit for
randomized purges. This adds a $limit parameter to the BagOStuff
method deleteObjectsExpiringBefore().

Change-Id: I66bfcc67e8e118f8c659dbbee13d54bf2cd937f5
2019-07-06 00:04:13 -07:00
Umherirrender
24f03d3276 Fix type hints in objectcache related classes
handleException is only used for RedisException
doWrite is also used with single array

Change-Id: I3010457ce2f143f41c4748ec957dfc3333af9761
2019-07-03 22:11:58 +02:00
Aaron Schulz
b09b3980f9 objectcache: add object segmentation support to BagOStuff
Use it for ApiStashEdit so that large PaserOutput can be stored.

Add flag to allow for value segmentation on set() in BagOStuff.
Also add a flag for immediate deletion of segments on delete().

BagOStuff now has base serialize()/unserialize() methods.

Bug: T204742
Change-Id: I0667a02612526d8ddfd91d5de48b6faa78bd1ab5
2019-06-11 16:14:17 +01:00
Reedy
9f2ffdfbd4 Remove "Squiz.WhiteSpace.FunctionSpacing" from phpcs exclusions
Change-Id: I78b3315f26ab91b6b443f5b028a635552f82f5a3
2019-05-11 02:44:26 +01:00
Aaron Schulz
962495b17a objectcache: merge getWithToken() into doGet() for simplicity
Change-Id: I581f866521e1086ca350973d9cdeff6656f48fe8
2019-03-27 13:03:25 -07:00
Aaron Schulz
25fd7dbc7b objectcache: defined some missing methods in MultiWriteBagOStuff
Also, reorder some method definitions to match the base class.
This makes it easier to compare the two. In addition, refactor
the doWrite() method to make the calls clearer.

Change-Id: I795b395b9c54645f62962461a28067ae38b291ed
2019-03-27 17:26:43 +00:00
Aaron Schulz
e626d014ac objectcache: remove BagOStuff::mergeViaLock() and update RESTBagOStuff
All subclasses are now using the CAS variant, which respects $attempts.

Change-Id: Ia7ec6a7f3337cabe95c54c1142c3c5464c1794e7
2019-03-24 00:35:23 -07:00
Aaron Schulz
547f207fa6 objectcache: make the $flags argument appear more consistently in BagOStuff
Change-Id: I08879ede5e9f0ab227497bb1dab89ea61b65abce
2019-03-12 22:55:18 +00:00
Aaron Schulz
5dd409cd38 objectcache: add $flags argument to BagOStuff::delete()
This makes it consistent with set() and merge(). Also, one subclass
was already using the field in this manner.

Clean up the code related to WRITE_SYNC in SqlBagOStuff.

Change-Id: I0fb84f4475311889507d3ef98afd4476fb81174f
2019-03-07 13:15:29 -08:00
Aaron Schulz
12200a2332 Make MultiWriteBagOStuff use the native merge() of each backend
This lets backends that support CAS take advantage of it.

Also fix race condition with merge() in asyncWrites mode where two
threads might add to a list/set but the first request finishes after
the second, which would erase an item during the set().

Bug: T198239
Change-Id: Ibd2539429746f12b4de74d0f031089cb80aa8b4b
2018-07-18 22:30:56 +01:00
jenkins-bot
2252838b4a Merge "objectcache: minor fix to MultiWriteBagOStuff::doWrite()" 2018-07-17 01:00:06 +00:00
Aaron Schulz
32d8dd04f0 objectcache: minor fix to MultiWriteBagOStuff::doWrite()
For the case of add(), any non-zero index can have async writes

Bug: T198239
Change-Id: I95f63143dcb894b9d12859c221f7d8e72fdf076f
2018-07-12 02:50:35 +00:00
Aaron Schulz
8664804e38 Implement makeKeyInternal() for ReplicatedBagOStuff/MultiWriteBagOStuff
Make these proxy to the "write"/"main" cache backend like the other
key methods.

Change-Id: Ie176577d9b52807493d6da9eea963ba440b86330
2018-07-10 13:25:04 +01:00
Aaron Schulz
6612407f9c objectcache: make MultiWriteBagOStuff handle duplicate add() operations
Bug: T198280
Change-Id: Ib1bcde2b3fbfb452f80d8d840c494be2eb70eb87
2018-06-28 16:04:35 +00:00
Max Semenik
1e680456b4 Get rid of call_user_func(_array)(), part 3
Also cleaned up nearby code in a couple places.

Change-Id: Ibf44ee7c0ceb739d7e79406e4ff39303c316e285
2018-06-10 02:21:24 +00:00
Kunal Mehta
79de8fd02f Use wikimedia/object-factory 1.0.0
Deprecate the unnamespaced version and move it to includes/compat.

Bug: T147167
Depends-On: I39c805bfb98b32f32f3d0dc1eee9e823afe1c21a
Change-Id: I3780c7adf51683f3f7adb35a88f9a25a0a2e2530
2018-02-04 12:52:44 -08:00
Aaron Schulz
93120703f6 objectcache: fix some makeKey/makeGlobalKey IDEA warnings
Change-Id: I5fefda9676a685167326c63c34b2b7df8be2e7cd
2017-11-18 19:57:26 -08:00
Umherirrender
3f1a52805e Use short type bool/int in param documentation
Enable the phpcs sniffs for this and used phpcbf

Change-Id: Iaa36687154ddd2bf663b9dd519f5c99409d37925
2017-08-20 13:20:59 +02:00
Umherirrender
ace44e2064 Use correct variable name in @param documentation
For some varargs a variable name is added with suffix ,... as seen for
many other varargs

Some @param are swapped, because there are in the wrong order

Enable Sniff MediaWiki.Commenting.FunctionComment.ParamNameNoMatch

Change-Id: I60fec6025bce824d5c67563ab7b65ad6cd628ad8
2017-08-11 19:27:19 +02:00
Brad Jorsch
edf69e62d8 MultiWriteBagOStuff: Fix async writes of mutable objects
If someone writes an object into a BagOStuff, they typically expect that
later changes to the object will not affect the value stored.
MultiWriteBagOStuff's async write handling was violating this
expectation, which is potentially causing T168040.

Bug: T168040
Change-Id: Ie897b900befdc8998614af06f9339cd07665703e
2017-07-25 02:08:00 +00:00
Timo Tijhof
dd629564df objectcache: Forward MultiWriteBagOStuff::makeKey to primary backend
Similar to what WANObjectCache and CachedBagOStuff are already doing.

Also add missing tests for WANObjectCache (similar to those for CachedBagOStuff).

Bug: T167465
Change-Id: I1a0c9324726aa6a1b221def985773b1b819181fd
2017-06-14 18:11:55 +01:00
Aaron Schulz
5f921702d2 Avoid stack overflow in LoadBalancer with CACHE_DB WAN/server cache
* Add the ability to expose key BagOStuff attributes like whether the
  emulation SQL cache is being used. Several callers use hacks to detect
  this and can be updated later.
* Fallback to a safe empty WAN cache in the CACHE_DB case. This fixes
  a regression from f4bf52e843.
* Also add this protection to the server cache, which could break
  similarly before too.
* Also fix CACHE_ANYTHING by avoid the recursion risk from
  fc1d4d7960 by just checking the disabled service map.

Bug: T123829
Bug: T141804
Change-Id: I17ee26138f69e01ec1aaddb55ab27caa4d542193
2016-08-02 13:06:49 +00:00
Bryan Davis
450bb6723d BagOStuff: Optionally log duplicate key lookups
Add new 'reportDupes' parameter to BagOStuff. This parameter enables scheduling
of a callback after the current web request ends. This callback will emit warning
log messages for all keys that were requested more than once.

The default ObjectCache factory for MediaWiki enables this option by default.
Not by default for plain BagOStuff instances, however. E.g. `new HashBagOStuff()`.

It also set 'asyncHandler' for all classes now (not just MultiWriteBagOStuff).

Bug: T128125
Co-Authored-By: Timo Tijhof <krinklemail@gmail.com>
Change-Id: I8a2b06cf54d2acf5950eed71756ecdf50e224be1
2016-04-22 04:21:20 +00:00
Timo Tijhof
bfb94f6f58 objectcache: Fix MultiWriteBagOStuff docs ('asyncHandler', not 'asyncCallback')
Follows-up 1171cc00cd.

Change-Id: I97189e916d0aba507273216e3e9ab28ce0d5ffec
2016-03-02 02:49:53 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Aaron Schulz
cce813a922 Move MultiWriteBagOStuff to /libs
Also moved related tests files to /libs.

Change-Id: I806eeaa30205733d497adde933baf0c4157f7aae
2015-10-24 12:15:26 -07:00
Renamed from includes/objectcache/MultiWriteBagOStuff.php (Browse further)