These are static methods that have to do with processing language names
and codes. I didn't include fallback behavior, because that would mean a
circular dependency with LocalisationCache.
In the new class, I renamed AS_AUTONYMS to AUTONYMS, and added a class
constant DEFINED for 'mw' to match the existing SUPPORTED and ALL. I
also renamed fetchLanguageName(s) to getLanguageName(s).
There is 100% test coverage for the code in the new class.
This was previously committed as 2e52f48c2e and reverted because it
depended on e4468a1d6b, which had to be reverted for performance
issues. There should be no changes other than rebasing.
Bug: T201405
Change-Id: Ifa346c8a92bf1eb57dc5e79458b32b7b26f1ee8a
This removes Language::$dataCache without deprecation, because 1) I
don't know of a way to properly simulate it in the new paradigm, and 2)
I found no direct access to the member outside of the Language and
LanguageTest classes.
An earlier version of this patch (e4468a1d6b) had to be reverted
because of a massive slowdown on test runs. Based on some local testing,
this should fix the problem. Running all tests in languages is slowed
down by only around 20% instead of a factor of five, and memory usage is
actually reduced greatly (~350 MB -> ~200 MB). The slowdown is still not
great, but I assume it's par for the course for converting things to
services and is acceptable. If not, I can try to optimize further.
Bug: T231220
Bug: T231198
Bug: T231200
Bug: T201405
Change-Id: Ieadbd820379a006d8ad2d2e4a1e96241e172ec5a
Before c962b48056, the 'loadedLanguages' array was used to track
which languages were loaded and in the cache, with 'cache' being a
simple array. In that commit, the 'cache' array also started being used
for incomplete datasets, which didn't affect 'loadedLanguages'.
Then in 97e86d934b, the 'loadedLanguages' array was removed in favour
of checking keys on 'cache' directly, and 'cache' was converted to
MapCacheLRU.
This led to problem where partially loaded data was mistaken for being
full datasets (fatal error, T208897). This was fixed in a5c984cc59,
by bringing back the 'loadedLanguages' array, which fixed the issue from
the POV of partially loaded data.
However, this then exposed a new problem. The 'cache' data can be evicted
by MapCacheLRU, whereas 'loadedLanguages' is not aware of that. Thus it
claims languages are loaded that sometimes aren't. (This only affects web
requests where more than 5 language codes are involved, per MapCacheLRU.)
Fix this by re-removing the 'loadedLanguages' array, this time
strengthening the 'cache' key check to not just check that the root key
exists, but that it is in fact holding the full dataset as generated by
MessageCache::load(). The 'VERSION' key appears to be a good proxy for
that.
Bug: T230690
Change-Id: I1162a3857376aa37e5894ae3c8be84a2295782a3
The `apc.cache_by_default` setting is a PHP5-era setting relating
to the part of php5-apc now known as opcache (as opposed to the
part now known as apcu).
This setting doesn't exist in PHP 7, and trying to set it doesn't
do anything useful.
Bug: T206986
Change-Id: I46a91897b2b33b5ce6505beb74d404982cb0641c
These callers just need to load some data from DB_MASTER.
Subsequent code needing that latest title data should also use the
required flags, rather than relying on flakey global cache state.
Change-Id: I53248ea4b5bf1cd953f956c41b8244831ec5ef04
This removes most of the pre-actor user and user_text columns, and the
$wgActorTableSchemaMigrationStage setting that used to determine
whether the columns were used.
rev_user and rev_user_text remain in the code, as on Wikimedia wikis the
revision table is too large to alter at this time. A future change will
combine that with the removal of rev_comment, rev_content_model, and
rev_content_format (and the addition of rev_comment_id and rev_actor).
ActorMigration's constructor continues to take a $stage parameter, and
continues to have the logic for handling it, for the benefit of
extensions that might need their own migration process. Code using
ActorMigration for accessing the core fields should be updated to use
the new actor fields directly. That will be done for in a followup.
Bug: T188327
Change-Id: Id35544b879af1cd708f3efd303fce8d9a1b9eb02
With this change, the memory behaviour of LCStoreStaticArray
matches the other LCStore implementations. Specifically, that when
mass-rebuilding LocalisationCache entries for all language codes,
the computed data should be released from memory after
calling LCStore::finishWrite().
This doesn't affect user-facing web requests, even in the case
of stock MW where every once in a while a user request can lazy-
regenerate the LCStore, there is a process-cache in front of LCStore
in the LocalisationCache class.
The rebuildLocalisationCache.php clears that via
LocalisationCacheBulkLoad::unload(), but due to LCStoreStaticArray
internally holding on to the data, it was still leaking.
The leak was found by @Nikerabbit as part of testing for T218207.
To test this, amend rebuildLocalisationCache.php and add the
following on line 161, as the first line of the doRebuild/foreach/if
block:
echo "[$code-start-mem] " . round(memory_get_usage(true)/1024/1024, 2) . " MB\n";
If you then have LocalSettings.php configured like so:
$wgCacheDirectory = $wgTmpDirectory;
$wgLocalisationCacheConf['store'] = 'array';
Then before this patch, running rebuildLocalisationCache.php,
shows memory starting at 12 MB and growing 2-3 MB for every language
until the very end, closing with 970 MB memory use.
After this patch, it starts at 12 MB and stops growing at 32 MB.
When configuring as `['store'] = 'files'`, which uses LCStoreCDB,
the memory starts at 12 MB and stops growing at 44 MB, both before
and after this patch.
Bug: T218207
Change-Id: I0d215efee5b31766776a068b16811d52f9879312
* Remove checks in HTMLFileCache.php and Article.php.
These haven't been needed since the same check was added to Setup.php,
many years ago. When FileCache is enabled, The Setup.php code disables
MWDebug. There is no reason for FileCache to then also disable itself
based on unused config. That means both of them lose.
We now handle this logic in one place: MWDebug::setup().
* In rebuildFileCache.php, turn it off explicitly, just in case.
The previous code there didn't work because finalSetup()
is called after doMaintenance.php includes Setup.php, which
is what checked this config var to decide on MWDebug::init.
On the other hand, it's also always off in CLI mode.
But, let's not depend on that, maybe we decide to enable it on
CLI one day! Just keep it off explicitly here.
Bug: T189966
Change-Id: I45a8f77092249751dc6f276aa5bb67ebf5b4f64c
And also update approximated counts, which for the most part are lower
than reported (hooray!)
Bug: T231636
Depends-On: Ica50297ec7c71a81ba2204f9763499da925067bd
Change-Id: I78354bf5f0c831108c8f606e50c87cf6bc00d8bd
When reporting exceptions that occur during initialization, wgUser may
be null. Don't die when that happens.
Change-Id: I65d5a17d80f9021e28a218c7a5a17e399bc7ce98
This variable has never been set to anything other than the default value of
24 hours as introduced in 2003 (r2203, r2204; or 036ff960ce, edf6b38626).
The variable has never changed in core, it's not overridden at WMF,
and MessageCache is not constructed anywhere other than ServiceWiring.php
anywhere in repos on Wikimedia Gerrit, indexed by MediaWiki Codesearch,
or any GitHub-hosted repository (incl Wikia repos and WikiHow mirrors).
I've also checked all GitHub-hosted repos for boilerplates and/or public
settings files from devs or prod, and couldn't find any example of
this being overridden (after filtering out copies of the core files
themselves). Rather than having to support potentially hard-to-predict
interactions betweeen caching layers by checking its state, make it
a constant so we can code reason about it more easily.
Change-Id: Ie2e139001aae3ac54b509d94a3d917bb408eaca0
For some unknown reason, when the `actor` table has few enough NS8 rows
compared to `page` MariaDB 10.1.37 decides it makes more sense to fetch
everything from `actor` then join `revision` then `page` rather than
fetching the rows from `page` in the first place.
We can work around it by telling it to not reorder the query, but then
we also have to reorder it ourselves to put `page` first instead of
`revision`.
Bug: T231196
Change-Id: I2b2fb209e648d1e407c5c2d32d3ac9e574e361d5
These are static methods that have to do with processing language names
and codes. I didn't include fallback behavior, because that would mean a
circular dependency with LocalisationCache.
In the new class, I renamed AS_AUTONYMS to AUTONYMS, and added a class
constant DEFINED for 'mw' to match the existing SUPPORTED and ALL. I
also renamed fetchLanguageName(s) to getLanguageName(s).
There is 100% test coverage for the code in the new class.
Change-Id: I245ae94bfc1f62b6af75ea57525139adf2539fe6
This removes Language::$dataCache without deprecation, because 1) I
don't know of a way to properly simulate it in the new paradigm, and 2)
I found no direct access to the member outside of the Language and
LanguageTest classes.
Change-Id: Iaa86c48e7434f081a53f5bae8723c37c5a34f503
This avoids global function wfGetDB
Let the GenderCache work without a database connection as it is used
by the installer
Change-Id: I8a203c50de5841bc33693dadb8439a23a8c60910
The purge() method handles purging of both file cache and CDN, using
a PRESEND deferred update. This avoids code duplication and missing
file cache purge calls.
Also:
* Migrate HTMLCacheUpdate callers to just directly using HTMLCacheUpdateJob
* Add HtmlFileCacheUpdate class and defer such updates just like with CDN
* Simplify HTMLCacheUpdate constructor parameters
* Remove BacklinkCache::clear() calls which do nothing since the backlink
query does not actually happen until the job runs
Change-Id: Ic453b189a40109a73a9426538608eea87a76befa
This was removed in 97e86d934b in 2018 in favour of using
`$this->cache->has($code)`. This is a problem because there
are cases where only a narrow subset of that structure is
populated (by MessageCache->replace) without things like
$this->overridable (or anything else that MessageCache->load does)
having ocurred yet.
The assumption that keys are only added to $this->cache by
MessageCache->load (or after that method has been called) was
actually true at some point. But, this changed in 2017 when
commit c962b48056 optimised MessageCache->replace to not call
MessageCache->load.
Bug: T208897
Change-Id: Ie8bb4a4793675e5f1454e65c427f3100035c8b4d
316d205115 migrated LocalisationCache from wfDebugLog() to the Logger
interface which no more requires the messages to be terminated with a
new line.
Change-Id: Ibe2f750e7ab16cf91801988cb726adb67f7cd9d2
The way isMainCacheable() was used, it always returned false in
non-content languages, because it would try to find strings like
'hidetoc/fr' in the array of message keys (which contains strings like
'hidetoc').
The consequence of this was that MessageCache would check the database
for a MediaWiki:hidetoc/fr page even if it already knew that that page
didn't exist. This is a substantial performance hit when requesting lots
of messages, like when building version hashes for ResourceLoader's
startup module.
Follows-up 4fc5ba8bf8.
Bug: T228555
Change-Id: I20433175ca919acc1c995f4a9cd50ca53afcdd02
No longer allow hook functions to prevent message blobs from being
purged. Pass in an always-true variable for backwards compatibility,
which is then ignored.
Change-Id: I27ac9599711f2f0df2514a3934270af0ce03da7f
Using logger interface, this would fix some unit tests, plus it
avoids using global functions and ease the way for proper dependency
injection
Change-Id: I9a545c70ace08e2fd0b832d75d6a2362a83af21e
This seems to be still used internally in core but in no extension. Also,
this function really doesn't do anything so hard deprecating in preparation
for removal.
Bug: T62260
Change-Id: I568789483084a97e5b3b462235f3d00c3cb87cf9
