MessageBlobStore class:
* Make logger aware.
* Log an error if json encoding fails.
* Stop using the DB table. WANObjectCache supports everything we need:
- Batch retrieval.
- Invalidate keys with wildcard selects or cascading check keys.
* Update tests slightly since the actual update now happens on-demand as
part of get() instead of within updateMessage().
ResourceLoader class:
* Remove all interaction with the msg_resource table. Remove db table later.
* Refactor code to use a hash of the blob instead of a timestamp.
Timestamps are unreliable and roll over too frequently for message blob store
because there is no authoritative source. The timestamps were inferred based on
when a change is observed. Message overrides from the local wiki have an
explicit update event when the page is edited. All other messages, such as
from MediaWiki core and extensions using LocalisationCache, have a single
timestamp for all messages which rolls over every time the cache is rebuilt.
A hash is deterministic, and won't cause needless invalidation (T102578).
* Remove redundant pre-fetching in makeModuleResponse().
This is already done by preloadModuleInfo() in respond().
* Don't bother storing and retreiving empty "{}" objects.
Instead, detect whether a module's message list is empty at runtime.
ResourceLoaderModule class:
* Make logger aware.
* Log if a module's message blob was not preloaded.
cleanupRemovedModules:
* Now that blobs have a TTL, there's no need to prune old entries.
Bug: T113092
Bug: T92357
Change-Id: Id8c26f41a82597e34013f95294cdc3971a4f52ae
It was already used everywhere as non-static via $this.
This is needed in order to allow MessageBlobStore unit tests
to disable the holdoff via a mock (mocks can't override static methods).
Change-Id: I3aad5b6e780addf1b6ce9de56c81b91f5ab358b2
If a set() happened around the exact same time as check key was
initialized via get(), the curTTL in future get() calls could
sometimes be 0 instead of a negative value, even though hold-off
should apply.
Change-Id: Ide1fd65112aff425a4798e2ec406d71f2a8e84a7
Move the holdoff period into the purge value instead of deciding
it at runtime. This opens the way for touchCheckKey() to support
a custom $holdoff parameter, which will allow callers to invalidate
keys without a holdoff period. Similar to what we already support
in delete().
Right now the holdoff period is decided at run time.
Change-Id: Id10c036272e92ae4429effc823b75e08fb11a48b
To allow batch queries for multiple keys that themselves have different check
keys. Previously check keys always applied to all keys being retrieved.
Change-Id: I9e5ba198d79020ce05a802a510762e29fcfb2f1b
Especially useful when interacting with getWithSetCallback of
HashBagOStuff where a $exptime is required before the callback.
In set() it is typically left out and thus not as counter-intuitive.
Change-Id: Ic2f7adda3c00cefe701d77bea91a7e8e77ef1439
* Keys that take a long time to generate would run into
the MAX_SNAPSHOT_LAG check and have set() no-op. This
would be fine except that leaves no key there to figure
out the time since expiry and therefore whether to use
the mutex, so it didn't. This now saves the keys but with
a FLG_STALE bit set, making the next caller that sees it
perform a regeneration (unless it can't get the mutex).
* Bumped LOCK_TTL so that keys can stay locked much longer.
* This is easy to test via sleep(5) in tagUsageStatistics()
and two Special:Tags browser tabs.
Bug: T91535
Change-Id: I549e70ace3d2e7da40d3c4346ebacc36024cd522
Also consistently use self:: instead of BagOStuff:: for constants
referenced within the BagOStuff class.
Change-Id: I20fde9fa5cddcc9e92fa6a02b05dc7effa846742
* In the common ~0 second lag case, transactions up to ~7 seconds
long will have normal set() behavior (instead of just ~5 seconds).
* Like wise with ~0 second transactions tolerating ~7 seconds of
lag (instead of just ~5).
* The lower hold-off time lets caching resume 3 seconds sooner.
Change-Id: I21e2a0a6915492cec422b6a6af5541937745c15b
This generally only effects wikis with no slave DBs,
but also matters if the master has non-zero LB load.
If the master ends up being used for DB_SLAVE, care
should be shown for cache-aside writes
Interesting WAN cache events are now logged.
Change-Id: I2cd8e84138263c13ea23beb9ab3d7562340e1fd3
* Make getWithSetCallback() accept a TTL field for this
* Make ChangeTag callers use this flag to avoid hundreds of
duplicate queries at Special:Tags
Change-Id: Ic1ed28294f5d557e02f39a7f20d36766244b9ded
This fixes the default $oldOpts value for the case when
$opts is given check keys and no $oldOpts value is passed
in. In that case, the getWithSetCallback() method
variable $opts would be null instead of array().
Change-Id: I825062533478b37766511aae1d06dd4b81f8dba1
* Put 'checkKeys' param in opts array instead of as a separate parameter.
It's neat to not have to skip unnamed/positional arguments that are optional.
* Move TTL to be before callback instead of after. This avoids dangling integers
toward the bottom of a large code block that have no obvious meaning.
Matches BagOStuff::getWithSetCallback.
Add unit test for lockTSE to confirm
Change-Id: I60d6b64fc5dff14108741bafe801fd1fde16d1df
This better handles values that take a long time to
generate. For example if it took 10 sec, then the old
value would not regenerate in time. 30 is a good, high,
value that is the default PHP timeout.
Change-Id: I2e4372bf4c6d28d09347012926d8c0a9deda4183
* This gets lag information that is useful when
the calling code is about to run queries that
will have their results cached.
* This is now used in place of trxTimestamp() for
WANObjectCache set() and getWithSetCallback().
* The WAN cache will use a low TTL if the lag is
too high to avoid caching stale data for weeks.
* Bumped MAX_COMMIT_DELAY as nothing enforces it.
Bug: T113204
Change-Id: I2a95b4088cff42d6e980351555f81a4b13519e24
* Doxygen needs this or the paragraph ends up in the
last bullet point
* Also removed some redundant comments
Change-Id: Ie4bf2b67fff27ecad5f24f55688161fe484c300d
* This can avoid some stale write race conditions
* Made use of this option in a few key places
* HOLDOFF_TTL was also bumped
Change-Id: I83505a59cce0119e456191c3100f7d97bc86bdbf
* They now issue the delete() write before COMMIT of
the relevant DB (or immediately if no trx is active)
* This can avoid some stale write race conditions
* Updated the WAN cache delete() docs
Change-Id: Id54887976051120b76528070d5f2ceb357d57897
* If lockTSE is set, make sure it applies during during
the entire duration of a key being tombstoned. This lets
regenerations happen during that whole time, which lowers
the chance of seeing stale data if the DBs are lagged.
* If lockTSE is not set, do not apply tempTTL (for tomstoned keys).
If traffic is high, a stale value would usually be "stashed" and
used for 5 seconds. If the lag was only 1 second, then this
is suboptimal.
* Determine tempTTL from lockTSE as they no longer make sense
being separate. This makes things easier to understand and
also makes the lockTSE value account for the last regeneration
time (via stash key TTL). Since LOCK_TSE << HOLDOFF_TTL, this
helps avoid stale reads without adding stampede risk.
Change-Id: I3b01f0ec67935a238b30e02e42004fd3b2dcfb9d
