For some varargs a variable name is added with suffix ,... as seen for
many other varargs
Some @param are swapped, because there are in the wrong order
Enable Sniff MediaWiki.Commenting.FunctionComment.ParamNameNoMatch
Change-Id: I60fec6025bce824d5c67563ab7b65ad6cd628ad8
If someone writes an object into a BagOStuff, they typically expect that
later changes to the object will not affect the value stored.
MultiWriteBagOStuff's async write handling was violating this
expectation, which is potentially causing T168040.
Bug: T168040
Change-Id: Ie897b900befdc8998614af06f9339cd07665703e
Having such comments is worse than not having them. They add zero
information. But you must read the text to understand there is
nothing you don't already know from the class and the method name.
This is similar to I994d11e. Even more trivial, because this here is
about comments that don't say anything but "constructor".
Change-Id: I474dcdb5997bea3aafd11c0760ee072dfaff124c
The used phpcs has a bug, so the version 0.9.0 could not be enforced at the moment.
Will be fixed in next version, see T167168
Changed:
- Remove duplicate newline at end of file
- Add space between function and ( for closures
- and -> &&, or -> ||
Change-Id: I4172fb08861729bccd55aecbd07e029e2638d311
Similar to what WANObjectCache and CachedBagOStuff are already doing.
Also add missing tests for WANObjectCache (similar to those for CachedBagOStuff).
Bug: T167465
Change-Id: I1a0c9324726aa6a1b221def985773b1b819181fd
Backend cache queries are now avoided if they are fresh in memory.
Also factor out some code into private methods for clarity.
Change-Id: Ib343fde3dbf63f39e8cb09eca6278811a7d9738b
The warmup cache was not properly prefixed and was also using the entity
IDs instead of the cache keys. Thus, it effectively just wasted a
getMulti() query and resulted in the usual separate GETs anyway.
Added some unit tests for this.
Change-Id: I75b7a31214b515511856f9d95db32e8881d80ccc
Per https://github.com/facebook/mcrouter/wiki/List-of-Route-Handles
a NullRoute response is always given for DELETE and SET. The former
already is already handled by MediaWiki treating NOT_FOUND as success.
Change-Id: I79c26bcd6b8ffe7eea73e0d45badcc4ed63f05e6
I was bored. What? Don't look at me that way.
I mostly targetted mixed tabs and spaces, but others were not spared.
Note that some of the whitespace changes are inside HTML output,
extended regexps or SQL snippets.
Change-Id: Ie206cc946459f6befcfc2d520e35ad3ea3c0f1e0
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.
Skipping jsminplus.php as those bug numbers aren't Wikimedia's, nor obviously
someone else's.
Change-Id: I9a2210e17852ee56f11282b980ac66d8c7a95671
* This fixes keys based on some sort of change log.
Updates are wrapped in a mutex and keep track of the
last known good position.
* Make WANObjectReapUpdate class that cleans up title
related keys using the recentchanges table. This triggers
as a deferred updates on RC view.
Change-Id: I7f14b9ca2533032147e62b1a3cc004a23da86579
The method call was misnamed (should be apcu_store) and not needed anyway,
since the other BagOStuff classes fail if no key is present.
Change-Id: I726965e665c2d0c5843d8dd33504810d130b9e98
Causes HHVM 3.12.7 to segfault and or:
Fatal error: Stack overflow in includes/libs/objectcache/BagOStuff.php on line 754
This reverts commit b47ce21cec.
Bug: T150833
Change-Id: Ia52fdf65f5e386c0eaa1046328680fb5c8a9081a
This works by setting a callback to return the cache set
options. The callback will watch DB reads and create a
merged result from said usage.
This handles callers that are missing getCacheSetOptions().
Change-Id: Ia264f011e45e8cf105480955dad7e2c4c2357b73
Because the process cache can be lagged by virtue of blind TTL,
the HOLDOFF_TTL might not be enough to account for it, so avoid
using it when already inside a callback.
Also split of the tests from the MediaWiki test class, so this
does not require DB access anymore.
Change-Id: I743a1233a5efc7f036fad140a9ff8a30b32f8f27
* Renamed mDoneWrites to be clearer at what type it is.
* Also cleaned up a few callers of this method
Change-Id: I45856b210c289c2e2f193cc4328a208e20b4e0a8
This does what it says on the tin, e.g. a way to fetch
multiple keys at once. Callbacks are still called in a
serial manner when needed.
Change-Id: I8e24a6de7f46499a53ec41636c5a4f106b9b3d09
Since the WaitConditionLoop class was first introduced in 1.28 (current
master), no back-compat alias is added.
Bug: T146256
Depends-On: Ia84774d83da79fea1e167fe065c69549981f753b
Change-Id: Ibd4f15c87105b8caccbd1f661b74b6efa012b77f
* Remove wf* function and ObjectCache dependencies.
* Use the base class session field.
* Lower physical lock structure TTL and move it to the base
class as a constant.
* Resolve TODO about acquiring mixed lock types in one pass.
* Only mark servers down for 30 seconds in case of long-running
scripts.
Change-Id: Icd4be407e599524cc620975d27e85666d2532b95
This is the only usage of wikimedia/assert in the planned objectcache
library, so it doesn't make much sense to have a dependency upon the
entire library just for one usage that is trivially replaced.
Change-Id: I8f967fb2b861852c7fc74f7d42e2cbab1df05bc4
* Use OutputPage::output() as the method to mask latency, since it
takes a good while to run. By the time it runs, cache replication
should have caught up, so the reap call will likely not block.
* For redirects emitted after changes in POST, instead of masking
with OutputPage, add a parameter to the redirect and block on
the positions appearing. This uses the redirection RTT to mask
the replication latency.
Change-Id: Ib23690c302e8033610fef9a0ef451dafe8a5803e
This lets callers use adaptive TTLs on the near-expiration
preemptive refreshes if the new and current values match,
using the as-of time as $mtime.
Change-Id: Ie541c35f890c9f789d1accf9f2a43506daaf31f0
This lets callers have their own process cache but keep it
managed in a central location, making it easier to reset
when DB transaction snapshot are flushed or when unit tests
want to clear caches between runs.
Use this to replace the Revision text process cache.
Change-Id: Ic61ee9140d4ce9836cc4650adb5bb75a291fea18
Callers using wfTimestamp( TS_UNIX, ... ) where getting $minTTL
due to the output being a string number.
Change-Id: I6b67a941940f40ef9a543f11d0dbccacafaaa53b
These control worthRefreshPopular(), which will try to make sure that
heavily used keys are kept more consistent than others.
Change-Id: I2d351b50462a96d0fa8ca8fa6d28d51d3a5c94ff
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.
The old constant is an alias now.
Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
This is used as the main stash by default, which is not a
performance cache but a stash for ephemeral values typically.
Change-Id: Ie3740c0387f36a3f9b2d8d7d3b4e04ee9238ab1a
* This better handles delayed/lost cache purges by
having lower TTLs for entries that often changes.
* Use this for foreign upload description page caches,
we purges are never received from the source wiki.
* Also use this for User and LocalFile cache TTLs.
* Also move the Database::getCacheSetOptions() call in
User *before* doing the queries, which is preferred.
* Fixed some IDEA errors too, like the undeclared
mApiBase field.
Change-Id: I70f8ebb29ac853c2a530d9eedb9e7facc1b7b710
* Update documentation about relay methods.
* Change interim key set() to add() to avoid broadcasting it.
* Remove the behavior of doing purges synchronously in the
local DC first before relay. In both the event relayer and
mcrouter case, they will be asynchronous. It was hardly
even possible to use such behavior since loads come from
slave DBs, which do not see changes right after COMMIT.
Bug: T97562
Change-Id: I7759c82ae6e1b72fc227882a99c9a712a46374f6
This can change the TTL without fetching the object
so that keys can be renewed or set to expire soon.
Change-Id: Id1c2c9f89b3445cfa34263057dc5029cbe170833
* Add the ability to expose key BagOStuff attributes like whether the
emulation SQL cache is being used. Several callers use hacks to detect
this and can be updated later.
* Fallback to a safe empty WAN cache in the CACHE_DB case. This fixes
a regression from f4bf52e843.
* Also add this protection to the server cache, which could break
similarly before too.
* Also fix CACHE_ANYTHING by avoid the recursion risk from
fc1d4d7960 by just checking the disabled service map.
Bug: T123829
Bug: T141804
Change-Id: I17ee26138f69e01ec1aaddb55ab27caa4d542193
This is useful for avoiding stampedes in the one case that lockTSE
does not alone cover, which is when the key does not exist or is
tombstoned.
Also avoid saving interim values unless the key is tombstoned
since there is no point in doing that otherwise.
Change-Id: I70997e90217a0979e0589afa7a5107b0e623c7cf
Currently renders slightly broken since a single line break
is ignored in Markdown (Doxygen parses these as Markdown).
Change-Id: Ibdff93e103f98b7bd8d3ef9118f246fedab288cf
Mostly places which immediately had a:
if ( !is_callable( $callback ) ) {
throw new Exception(...);
}
check at the beginning of the function.
Change-Id: Ia78663b2231629010816bd1cda8814b996968d1d
* getWithSetCallback() takes a 'version' parameter.
* If the value at a key has a different version, then
getWithSetCallback() will automatically use a separate
key. Which value "wins" the main key does not matter.
* Purges are handled by using the main key as a sort of
check key (with no hold-off). Note that this key is always
purged on delete().
* Changed stash keys to track the same info as other keys
both for consistency and because this change needs the
generation timestamp. Renamed the stash prefix to avoid
corrupt results with Het Deploy.
* This is useful for things like the User class that use
versioning and have cross-wiki key access and purges.
Currently, bumps to version must be deployed to all wikis
at once, which this aims to avoid.
Change-Id: I26ae62f116e32b48bcf06bc13f8b9e79ae976745
MemcachedPhpBagOStuff::getMulti() wraps around this method directly,
so get_multi() returning false was causing it to violate
BagOStuff's requirement that getMulti() always return an array.
Change-Id: Ia8f076323aed5b4a12de82d519eb530ed29d7060
This makes the channels more explicit and defined in a less
ad-hoc way. Systems like Kafka would prefer explicit channel
definitions anyway, so the channel prefix just obscures things.
Change-Id: I5631eb1b1382083396a0f08904d9273cc92601e8
Add new 'reportDupes' parameter to BagOStuff. This parameter enables scheduling
of a callback after the current web request ends. This callback will emit warning
log messages for all keys that were requested more than once.
The default ObjectCache factory for MediaWiki enables this option by default.
Not by default for plain BagOStuff instances, however. E.g. `new HashBagOStuff()`.
It also set 'asyncHandler' for all classes now (not just MultiWriteBagOStuff).
Bug: T128125
Co-Authored-By: Timo Tijhof <krinklemail@gmail.com>
Change-Id: I8a2b06cf54d2acf5950eed71756ecdf50e224be1
LocalRepo::checkRedirect() cache image redirect checks in memory to avoid
unnecessary trips to memcached. Same idea as Idbd11637.
Change-Id: Idaa12376382aaa0bd8cb988240ec1da0b688359d
I searched for /\$(\S+) = (.+?\(.*?\);)\n.*?\$\1\[/, ignored
everything involving isset(), unset() or array assigments, then
skimmed through the remaining results and changed things where they
made sense. These changes were not automated, so please review them.
Change-Id: Ib37b4c66fc57648470f151ad412210b3629c2538
Two things were wrong here:
1. SessionManager shouldn't override the configured logger of the
underlying store.
2. CachedBagOStuff shouldn't override the configured logger of the
backend store on construction.
Change-Id: I24ed5a81d5b238a42934a7890dd8d0a9f9bb3b6f
Ie161e0f was done in a hurry, and so didn't do things in the best ways.
This introduces a new "CachedBagOStuff" that transparently handles all
the logic that had been copy-pasted all over in Ie161e0f.
The differences between CachedBagOStuff and MultiWriteBagOStuff are:
* CachedBagOStuff supports only one "backend".
* There's a flag for writes to only go to the in-memory cache.
* The in-memory cache is always updated.
* Locks go to the backend cache (with MultiWriteBagOStuff, it would wind
up going to the HashBagOStuff used for the in-memory cache).
Change-Id: Iea494729bd2e8c6c5ab8facf4c241232e31e8215
* This can happen due to incr/add races. Use incrWithInit()
instead to handle such cases.
* Also made BagOStuff:incrWithInit() return the new value like incr().
Change-Id: I0e3b02a4cff7c20544a9db2eaabd3f61e5a470b1
APCu's object serialization causes memory corruption when apc.serializer is set
to 'default' (see https://github.com/krakjoe/apcu/issues/38). We can work
around this bug by falling back to the pre-I4b2cf17155 behavior of
APCBagOStuff, which is not to trust apc_store() with anything other than
strings and integers, and instead serialize everything in user-space PHP code.
Bug: T120267
Change-Id: If34a1d959e2700792514b372af52919940222d83
MessageBlobStore class:
* Make logger aware.
* Log an error if json encoding fails.
* Stop using the DB table. WANObjectCache supports everything we need:
- Batch retrieval.
- Invalidate keys with wildcard selects or cascading check keys.
* Update tests slightly since the actual update now happens on-demand as
part of get() instead of within updateMessage().
ResourceLoader class:
* Remove all interaction with the msg_resource table. Remove db table later.
* Refactor code to use a hash of the blob instead of a timestamp.
Timestamps are unreliable and roll over too frequently for message blob store
because there is no authoritative source. The timestamps were inferred based on
when a change is observed. Message overrides from the local wiki have an
explicit update event when the page is edited. All other messages, such as
from MediaWiki core and extensions using LocalisationCache, have a single
timestamp for all messages which rolls over every time the cache is rebuilt.
A hash is deterministic, and won't cause needless invalidation (T102578).
* Remove redundant pre-fetching in makeModuleResponse().
This is already done by preloadModuleInfo() in respond().
* Don't bother storing and retreiving empty "{}" objects.
Instead, detect whether a module's message list is empty at runtime.
ResourceLoaderModule class:
* Make logger aware.
* Log if a module's message blob was not preloaded.
cleanupRemovedModules:
* Now that blobs have a TTL, there's no need to prune old entries.
Bug: T113092
Bug: T92357
Change-Id: Id8c26f41a82597e34013f95294cdc3971a4f52ae
It was already used everywhere as non-static via $this.
This is needed in order to allow MessageBlobStore unit tests
to disable the holdoff via a mock (mocks can't override static methods).
Change-Id: I3aad5b6e780addf1b6ce9de56c81b91f5ab358b2
If a set() happened around the exact same time as check key was
initialized via get(), the curTTL in future get() calls could
sometimes be 0 instead of a negative value, even though hold-off
should apply.
Change-Id: Ide1fd65112aff425a4798e2ec406d71f2a8e84a7
Move the holdoff period into the purge value instead of deciding
it at runtime. This opens the way for touchCheckKey() to support
a custom $holdoff parameter, which will allow callers to invalidate
keys without a holdoff period. Similar to what we already support
in delete().
Right now the holdoff period is decided at run time.
Change-Id: Id10c036272e92ae4429effc823b75e08fb11a48b
To allow batch queries for multiple keys that themselves have different check
keys. Previously check keys always applied to all keys being retrieved.
Change-Id: I9e5ba198d79020ce05a802a510762e29fcfb2f1b
Follows-up f36b73e96c, which moved these classes to libs/objectcache.
* Fix wrong @throws in MemcachedPeclBagOStuff.
* Fix wrong @returns in MemcachedBagOStuff::getClient().
* Rename MWMemcached to MemcachedClient.
* Remove mention of 'memcached.php', which doesn't exist anymore.
Change-Id: I34dbc859be4778cea489fd2344f233f30452605c
* Match behaviour of MapCacheLRU and ProcessCacheLRU.
* Add missing unit tests for TTL and maxCacheKeys eviction behaviour.
Change-Id: I559eae1cd336274b21728e86775cfbad7e2f2c6d
Especially useful when interacting with getWithSetCallback of
HashBagOStuff where a $exptime is required before the callback.
In set() it is typically left out and thus not as counter-intuitive.
Change-Id: Ic2f7adda3c00cefe701d77bea91a7e8e77ef1439
* Keys that take a long time to generate would run into
the MAX_SNAPSHOT_LAG check and have set() no-op. This
would be fine except that leaves no key there to figure
out the time since expiry and therefore whether to use
the mutex, so it didn't. This now saves the keys but with
a FLG_STALE bit set, making the next caller that sees it
perform a regeneration (unless it can't get the mutex).
* Bumped LOCK_TTL so that keys can stay locked much longer.
* This is easy to test via sleep(5) in tagUsageStatistics()
and two Special:Tags browser tabs.
Bug: T91535
Change-Id: I549e70ace3d2e7da40d3c4346ebacc36024cd522
Also consistently use self:: instead of BagOStuff:: for constants
referenced within the BagOStuff class.
Change-Id: I20fde9fa5cddcc9e92fa6a02b05dc7effa846742
* In the common ~0 second lag case, transactions up to ~7 seconds
long will have normal set() behavior (instead of just ~5 seconds).
* Like wise with ~0 second transactions tolerating ~7 seconds of
lag (instead of just ~5).
* The lower hold-off time lets caching resume 3 seconds sooner.
Change-Id: I21e2a0a6915492cec422b6a6af5541937745c15b
This generally only effects wikis with no slave DBs,
but also matters if the master has non-zero LB load.
If the master ends up being used for DB_SLAVE, care
should be shown for cache-aside writes
Interesting WAN cache events are now logged.
Change-Id: I2cd8e84138263c13ea23beb9ab3d7562340e1fd3
Also removed a few MW dependencies from MemcachedPeclBagOStuff.
It still uses an IP class method, so it has to stay for now.
Change-Id: I8c5c83046c58fb58091d6ce11b2385208262460f
For the sake of safety and correctness, the following BagOStuff::makeKey()
invocations should return distinct keys:
$cache->makeKey( 'ab:', 'cd' );
$cache->makeKey( 'ab', ':cd' );
That is not currently the case, because while we use ':' as a key path
separator, we don't escape ':' in the input supplied to makeKey(). So, make
BagOStuff::makeKeyInternal() URL-encode colons.
To prevent this from messing up the max. key length calculations, reproduce
this logic in MemcachedBagOStuff::makeKeyInternal(), in lieu of having the
method call its parent.
Change-Id: I83ea7e7336a1c9e64aa42284c2517089a736efe5
* This blocks on writing to all replicas
and returns false if any failed.
* This is useful if ChronologyProtector is to work across
domains by having the writes go everywhere so that later
reads will see them (and be local at the same time).
* Redundant doc comments were also removed.
Change-Id: I9ed098d563c64dba605e7809bc96731da3b3e79d
