Commit graph

580 commits

Author SHA1 Message Date
Timo Tijhof
26c9849330 docs: Fix 'dependant' typos
The intended word in all these cases was the adjective "dependent".

Whilst the "dependant" does exist, it is a noun and generally
refers to a person. The word is rare used in general, but
especially so in a technology context.

Change-Id: Ic7e2d2ea6a566f4139ff1fdb77f38b0e962ccd9c
2021-02-18 16:59:20 +00:00
jenkins-bot
4deaf420d0 Merge "objectcache: make MediumSpecificBagOStuff::updateOpStats more robust" 2021-02-18 07:19:58 +00:00
Aaron Schulz
2757a31674 objectcache: make MediumSpecificBagOStuff::updateOpStats more robust
Make the method accept either an integer-indexed list of keys or a map
of keys to the send/receive payload size information tuple.

Make sure that no BagOStuff subclasses are passing in $keys directly
from the call to the public BagOStuff methods; these allow keys in the
form of a list or a map of arbitrary strings to keys.

Change-Id: I9687d25a4dd1c7b4b304f9fd543cc0a26a595962
2021-02-18 04:20:24 +00:00
jenkins-bot
58367a2934 Merge "Fix a typo in ExpirationAwareness (Idom => Idiom)" 2021-02-18 02:34:45 +00:00
Martin Urbanec
411cea4d60 Fix a typo in ExpirationAwareness (Idom => Idiom)
Change-Id: Ie181ec5d6dcd917f509af74e3af4717142a66e8d
2021-02-18 03:10:21 +01:00
jenkins-bot
ed50ed149a Merge "objectcache: factor out WANObjectCache::fetchKeys() method" 2021-02-18 00:05:10 +00:00
jenkins-bot
5ea464b234 Merge "Use static closures where safe to use" 2021-02-14 23:05:48 +00:00
Aaron Schulz
f89c3622b5 objectcache: factor out WANObjectCache::fetchKeys() method
Avoid internal calls to methods with messy legacy signatures
like get() and getMulti(). This avoids phan warnings about
suspicious array access.

Make WANObjectCache::PASS_BY_REF an empty array and tweak
the default value for $info reference parameters.

Tweak the parameter and usage documentation for get() and
getMulti() to focus on the non-legacy case.

Change-Id: Ia5b3df6d3df0c1e87821e78384c6e02991792575
2021-02-14 06:05:10 +00:00
Umherirrender
8de3b7d324 Use static closures where safe to use
This is micro-optimization of closure code to avoid binding the closure
to $this where it is not needed.

Created by I25a17fb22b6b669e817317a0f45051ae9c608208

Change-Id: I0ffc6200f6c6693d78a3151cb8cea7dce7c21653
2021-02-11 00:13:52 +00:00
Aaron Schulz
f55bc50d51 objectcache: tweak MediumSpecificBagOStuff::updateOpStats call rate metric name
This makes it easier to use Grafana dropdowns

Change-Id: Icd3066ede584a87c4add8ed1a17ce2dfcc332bb1
2021-02-09 21:43:28 -08:00
jenkins-bot
2d2fe36d3c Merge "objectache: Revert "throw on Closure" in WANObjectCache" 2021-02-09 16:58:36 +00:00
Krinkle
0ec2643767 objectache: Revert "throw on Closure" in WANObjectCache
This reverts commit 4e94fa8812.

The extra serialization call here is not needed since storing
a value naturally results in serialization and thus exception
throwing already (hence the fatal error at T273242).
Performing serialization twice comes with a significant cost,
which has in past led to notable regressions.

Adding a try-catch makes sense, but this can be placed at a
higher level instead, as is already done by Ibfcbf0eceb4b7a36
which was already merged and back-ported, making this obsolete.

Other use cases:

- Continuous integration where HashBag is used, which doesn't
  need serialize and thus doesn't cover the Closure issue.
  Cost of serialize was removed there for performance reasons,
  but if we want to add it back, let's do so there in the natural
  place where it once was (or in MediumBag, which might be preferable
  so that we can use that same code path to gradually switch to JSON
  encoding and warn/throw for non-jsonic values).

- Regular getWithSet calls (rather than the async refresh from which
  the fatal is currently seen). For regular getWithSet calls we already
  get a useful stack trace that points directly to the line of code
  where the value and cache key are computed, so additional logging
  does not seem that valuable. But for consistency we could move the
  try-catch from Ibfcbf0eceb4b7a36e33a8 a little lower to cover all
  fetchOrRegenerate() calls, not just those from scheduleAsyncRefresh.

Bug: T273242
Change-Id: Ic649016dde7b58323a028ce6d4fe617ef62a1ff8
2021-02-09 16:27:48 +00:00
jenkins-bot
726f87c0c6 Merge "WANObjectCache: throw on Closure" 2021-02-09 13:35:31 +00:00
daniel
4e94fa8812 WANObjectCache: throw on Closure
Alternative to Ia91d9f8533d4cc58564473ee9a375ba6be90eb64

Bug: T273242
Change-Id: I76a6667a1fcdc35ff972ebcdf1ad0b95ef07e015
2021-02-09 11:10:41 +00:00
jenkins-bot
963a6d4ed6 Merge "objectcache: Set 'int' phan-param type for version in WANObjectCache::set" 2021-02-09 00:33:35 +00:00
Moritz Schubotz (physikerwelt)
d54101ebb2 objectcache: Set 'int' phan-param type for version in WANObjectCache::set
It is documented that the version is an int.

Change-Id: Ic779e671ada0d0306e0935e578b47c431abd520e
2021-02-09 00:09:49 +00:00
Petr Pchelko
55120ec2c4 objectcache: Log more info when WANObjectCache async refresh fails
Bug: T264391
Change-Id: Ibfcbf0eceb4b7a36e33a8e43c40aefa17ad20f1b
2021-02-04 05:08:15 +00:00
jenkins-bot
e1e4f7f563 Merge "objectcache: Fix non-numeric string check in HashBagOStuff for PHP 8" 2021-02-01 19:43:37 +00:00
Umherirrender
26b7558462 objectcache: Fix non-numeric string check in HashBagOStuff for PHP 8
PHP 8 changes the handling for string and int comparison.
These non-numeric strings are no longer silently converted to int 0,
which previously allowed for the comparison to become false without
actually checking against non-integer values.
https://wiki.php.net/rfc/saner-numeric-strings

This fixes a unit test failure on PHP 8:
> HashBagOStuffTest::testConstructBadType
> Failed asserting that "InvalidArgumentException" is thrown.

This fails because the following new warning is encountered
instead, which PHPUnit turned into an exception:

> PHP Warning: A non-numeric value encountered

Bug: T248925
Change-Id: I6f75080ac8d549b815932527accb2fd5129d5775
2021-01-31 00:07:54 +00:00
Tim Starling
ca10d04f1a objectcache: Always enable debug mode in BagOStuff
BagOStuff has a debug mode because MemcachedClient has a debug mode,
because Ryan T. Dean put one in there in 2003. It's not strictly
necessary now that we have log levels.

This means that a few existing $this->logger->debug() calls are now on
the same effective level as $this->debug() calls, they can't be enabled
separately. But that seems inconsequential.

Change-Id: I4d1cb9153559b29468b6ca530bbc56e2b35d1721
2021-01-29 05:33:41 +00:00
Aaron Schulz
8f1e557906 objectcache: return false during more error cases in RedisBagOStuff::*Multi() methods
Change-Id: I39c4c6e7d05fcd24175ef45714923136d040ec8a
2021-01-26 22:02:40 -08:00
Aaron Schulz
b998f62302 objectcache: fix broken for loop in RedisBagOStuff::doSetMulti()
Follow-up to 57325ba3bd

Bug: T273006
Change-Id: I2f8207905de730a35ec1c9baf70c1a3c783cf980
2021-01-26 21:59:31 -08:00
Aaron Schulz
57325ba3bd objectcache: add statsd key metrics to BagOStuff classes
Update SQL, REST, and redis subclasses to emit call count and
payload size metrics for cache key operations. These metrics
are bucketed by cache key collection (similar to WANCache).

Bug: T235705
Change-Id: Icaa3fa1ae9c8b0f664c26ce70b7e1c4fc5f92767
2021-01-25 14:36:29 -08:00
Aaron Schulz
a8dbce1efe objectcache: improve "lockTSE" comments in WANObjectCache
Also marked some related constants as being public

Bug: T270155
Change-Id: I68692a7d7a5a24c0bf40f22bf1de70b02e971c53
2021-01-14 15:01:13 -08:00
Reedy
5eab9653e0 MemcachedClient: Cast Resource to integer
Bug: T269178
Change-Id: I9227528ceb0235b394c8f42621869cffe15ccba8
2020-12-15 12:43:16 +00:00
Aaron Schulz
5c335f9d77 objectcache: make BagOStuff key encoding more consistent
Add "generic" key methods for quickly deriving keys from
key component lists in a bijective manor. This is useful
for BagOStuff classes that wrap other BagOStuff instances
or for parsing keys to get stats.

Make the proxy BagOStuff classes (ReplicatedBagOStuff,
MultiWriteBagOStuff, CachedBagOStuff) use "generic" keys
so that they can convert to appropriate keys when making
backing cache instance method calls.

Make EmptyBagOStuff, HashBagOStuff, APCUBagOStuff,
RedisBagOStuff, and RESTBagOStuff use "generic" keys rather
than those of MediumSpecificBagOStuff::makeKeyInternal().
This lets proxy BagOStuff classes bypass key conversions
when used with instances of these classes as backing stores.

Also:
* Fix missing incr(), incrWithInit(), and decr() return
  values in MultiWriteBagOStuff.
* Make MultiWriteBagOfStuff, ReplicatedBagOStuff, and
  CachedBagOStuff use similar backend method forwarding
  styles by using a new BagOStuff method.
* Improved various related bits of documentation.

Bug: T250239
Bug: T235705
Change-Id: I1eb897c2cea3f5b756dd1e3c457b7cbd817599f5
2020-12-02 20:20:42 -08:00
Aaron Schulz
da03e5e765 objectcache: rename BagOStuff::makeKeyInternal() arguments and update documentation
Change-Id: I9e7119d7822eae945f0d627d1abcf491989aa882
2020-11-23 23:25:12 +00:00
James D. Forrester
04e46a3988 Upgrade wikimedia/object-factory from 2.1.0 to 3.0.0
Bug: T267074
Change-Id: Ic9cf63249744cd61bbe238c005630fc82db41d41
Depends-On: I2eac3e96eafdb540824bbc3ddc0b6cc1aed76bf2
2020-11-17 02:29:54 +00:00
Thiemo Kreuz
20b2c5000d Make use of array deconstruction directly in foreach, if possible
Deconstructing non-sparse, numerically indexed arrays directly in
foreach (a.k.a. using the list() syntax in foreach) is possible since
PHP 5.5.

The possibility to use string array keys as well as non-sequential
numeric keys in array deconstruction was added in PHP 7.1.

Change-Id: I56a48552a45f61cedc291b306cad8548fc70d485
2020-11-12 18:38:06 +00:00
Aaron Schulz
3f64eca5a4 objectcache: fix WANCache getWithSetCallback() when TTL < "lowTTL"
Previously, the refresh probability ramp-up logic assumed that
keys were always assigned logical TTLs >= "lowTTL". Thus, if a
key was saved with a TTL of 3 seconds and "lowTTL" was 10 seconds,
then the probability of a refresh at t=0 seconds would incorrectly
equal the value it should have at t=7.

Also:
* Use class constants for key metadata array keys. This makes it
  easy to track the usage of options and is less prone to typos.
* Improve some documentation slightly.

Bug: T264787
Change-Id: I7267e8639617fb8dc0850530465ff9d1b899b592
2020-11-10 17:49:32 -08:00
Aaron Schulz
b2408deee4 objectcache: convert WANCache TYPE_*/PURGE_VAL_PREFIX variables to constants
Change-Id: Ie01d7c83b12e70ad0bd03ae7c7d24d4908581c85
2020-10-15 15:33:13 -07:00
jenkins-bot
4c3256d64b Merge "objectcache: replace count metric with timing metric for WAN cache gets" 2020-09-28 19:49:08 +00:00
Umherirrender
d790580fda Fix typos related to repeated words
Change-Id: Ibc187d95b003017255bc87adf56afae7a59bd3db
2020-09-27 10:25:36 +00:00
Aaron Schulz
df48b9686c objectcache: replace count metric with timing metric for WAN cache gets
Bug: T255479
Change-Id: Ie355fc50023bd6ece1bf42cbf0b6ff9dd8b0cf88
2020-09-22 11:39:54 -07:00
daniel
d75b45ee2f objectcache: Add QoS flags to HashBagOStuff
Change-Id: I571c860ef9dd32a1eb374d59738f6e9e74b19fc2
2020-09-17 01:25:29 +00:00
Matěj Suchánek
51642e0aae Replace tabs with spaces
Change-Id: Id034c34a294585f58dba58c26db466e0e39cb557
2020-09-04 18:04:07 +02:00
daniel
f50240fe76 User: enforce pingLimiter() expiry time
This makes User::pingLimiter() include the expiry time in the payload of
the cache key that holds the current count. This allows us to ignore
stale counts.

Until now, we have been relying on the cache implementation to expire
the relevant keys in time. This however seems to fail sometimes.

Bug: T246991
Change-Id: Ifa3c558b4449f1ca133d0064781f26ac1bf59425
2020-09-03 15:57:01 +00:00
Aaron Schulz
c2edab83ee objectcache: support static routing prefixes for MemcachedBagOStuff
Bug: T257460
Change-Id: Id46d441125da16e5ddd4542d2492adf37acb470d
2020-08-21 10:55:11 -07:00
Umherirrender
6eef7e6039 Remove duplicate call to setMockTime in MultiWriteBagOStuff
Change-Id: I5770b80519502e3d4cb7553c3583a64b32769904
2020-07-31 22:07:28 +02:00
Aaron Schulz
d51f78e122 objectcache: cleanup bogus config field from MemcachedBagOStuff
Change-Id: I1c35010951c8d98d016f34b3c133f54a134f98e0
2020-07-23 06:48:20 +00:00
Aaron Schulz
7523716ebd objectcache: only use memcached "gets" when tokens are needed
This saves a few bytes in the response size and make it easy
for memcached proxies to distinguish key fetches that are part
of check-and-set cycles from those that are not.

MediumSpecificBagOStuff now requires PASS_BY_REF to fetch CAS
tokens. BagOStuff::merge() and WinCacheBagOStuff::doCas() are
the only callers that need this mode.

Bug: T257003
Change-Id: If91963f58adc4cda94f6d634ee0252a479a0fc5e
2020-07-15 00:16:46 +00:00
Bill Pirkle
8e9cba8bf0 Remove deprecated "legacy" serialization type from RESTBagOStuff
T233963 added serizalization type to RESTBagOStuff. Although
there were no known callers that were not refactored to be
aware of this, T233963 also added a deprecated "legacy" type,
marked for removal in MW 1.35, just in case. There are still no
known callers using this, so remove "legacy" as planned.

Bug: T234779
Change-Id: I0c7707692aa1d0c75e262c914e064bddc10897c7
2020-07-13 10:29:47 -05:00
jenkins-bot
39705eb311 Merge "Replace "@stable for subclassing" with "@stable to extend"" 2020-07-13 09:31:38 +00:00
daniel
3c50afa46b Replace "@stable for subclassing" with "@stable to extend"
For compliance with the new version of the table interface policy
(T255803).

This patch was created by an automated search & replace operation
on the includes/ directory.

Bug: T257789
Change-Id: Ie32c1b11b3d16ddfc0c83a757327d449ff80b2e4
2020-07-13 11:00:30 +02:00
daniel
272db6afde Replace "@stable for calling" by "@stable to call"
For compliance with the new version of the table interface policy
(T255803).

This patch was created by an automated search & replace operation
on the includes/ directory.

Bug: T257789
Change-Id: If560596f5e1e0a3da91afc36e656e7c27f040968
2020-07-13 08:55:28 +00:00
daniel
7572a152fc Annotate extensible classes.
This marks some base classes as stable for subclassing.

Bug: T247862
Change-Id: I7b77075849aa40960f2486b463e498a3ea8058e9
2020-07-09 11:07:38 +02:00
daniel
128565505c Mark additional classes as newable for now.
The following glasses are marked as newable per the
Stable Interface Policy, even though logically, they should not be
newable. This is done for classes that are currently instantiated
by extensions, and lack an alternative.

A better way for obtaining an instance of these classes should be
created in the future. At that point, direct instantiation should
be deprecated and replaced.

- includes/ApiMain.php - needs factory
- includes/media/BitmapMetadataHandler.php - should become a stateless service or use handler pattern
- includes/GitInfo.php - should become a stateless service
- includes/logging/LogPage.php - should become a stateless service or use command pattern
- includes/logging/ManualLogEntry.php - should become a stateless service or use command pattern
- includes/poolcounter/PoolCounterWorkViaCallback.php - needs a factory
- includes/context/RequestContext.php - needs to be narrowed down, and should use a factory
- includes/search/SearchHighlighter.php - should have a factory
- includes/TitleArrayFromResult.php - should perhaps be part of TitleFactory
- includes/user/User.php - should at least get a factory method for anons
- includes/diff/Diff.php: needs a TextDiffGenerator service or a factory
- includes/EditPage.php: needs a factory

Bug: T247862
Change-Id: I033158e693c98630ee167d9528fc8c9936f978d4
2020-07-08 18:06:12 +02:00
daniel
bd5c3d06a2 Annotate newable classes
This annotates classes that can safely be instantiated by
extensions, per the Stable Interface Policy.

Bug: T247862
Change-Id: Ia280f559874fc0750265ddeb7f831e65fd7d7d6a
2020-07-07 15:38:57 +02:00
Umherirrender
bc5cb7ae64 phan: Enable redundant_condition_detection
Remove duplicate casts
Suppress false positives

Bug: T248438
Change-Id: I2f89664a4bcd3b39b15e7cf850adda2f0c90ae6f
2020-07-01 20:13:07 +00:00
Aaron Schulz
55d147e4bb objectcache: Split out BagOStuffTestBase class for easier subclassing
Create a simple APCUBagOStuff subclass and also fix
APCUBagOStuff::incrWithInit() default $init value.

Change-Id: If84963fe7dcfedd6edebfb8785235263e0868ece
2020-06-13 20:15:23 +00:00