Commit graph

238 commits

Author SHA1 Message Date
jenkins-bot
be029c5fa4 Merge "MessageCache: Avoid Undefined offset in normalizeKey()" 2022-02-04 17:52:15 +00:00
Timo Tijhof
067b07f4fc MessageCache: Avoid Undefined offset in normalizeKey()
It is quite rare that a notice like this is best fixed so close to
the caller. More often than not, low-level code like this doesn't
have a reasonable way to handle it and it's a genuine issue but one
that a higher-level should prevent instead.

In this case though it seems fair to handle here because an empty
string is actually, according to most l10n-related classes, a valid
message key. It can't exist, but it is a valid key.

See also MessageCache::get() which does not consider it an error and
similarly returns false. That method never calls normalizeKey with
an empty string though. The error came from ApiQueryAllMessages.

I don't think it should be ApiQueryAllMessages's concern to normalize
this key or otherwise hardcode its non-existence over there.

Bug: T300792
Change-Id: I8a8eed66704862bb3a645ef5941a5264e65bde4c
2022-02-04 03:37:36 +00:00
Tim Starling
ca71e69fc6 Try not to discard Excimer timeout exceptions
Don't catch and discard exceptions from the RequestTimeout library,
except when the exception is properly handled and the code seems to be
trying to wrap things up.

In most cases the exception is rethrown. Ideally it should instead be
done by narrowing the catch, and this was feasible in a few cases. But
sometimes the exception being caught is an instance of the base class
(notably DateTime::__construct()). Often Exception is the root of the
hierarchy of exceptions being thrown and so is the obvious catch-all.

Notes on specific callers:

* In the case of ResourceLoader::respond(), exceptions were caught for API
  correctness, but processing continued. I added an outer try block for
  timeout handling so that termination would be more prompt.
* In LCStoreCDB the Exception being caught was Cdb\Exception not
  \Exception. I added an alias to avoid confusion.
* In ImageGallery I added a special exception class.
* In Message::__toString() the rationale for catching disappears
  in PHP 7.4.0+, so I added a PHP version check.
* In PoolCounterRedis, let the shutdown function do its thing, but
  rethrow the exception for logging.

Change-Id: I4c3770b9efc76a1ce42ed9f59329c36de04d657c
2022-02-02 16:27:44 +11:00
TChin
47adb6d65a Refactor global variables to use MediaWikiServices instead
Automatically refactors wg prefixed globals to use MediaWikiServices config using Rector. Doesn't include files that set globals or files that fail CI.

Rector Gist: https://gist.github.com/tchin25/7cc54f6d23aedef010b22e4dfbead228

* This patch uses a modified source code rector library for our specific use case and the rector will have different effects without it.

A writeup for future reference is here: https://meta.wikimedia.org/wiki/User:TChin_(WMF)/Using_Rector_On_MediaWiki

Change-Id: I1a691f01cd82e60bf41207d32501edb4b9835e37
2022-01-10 13:55:53 -05:00
Siddharth VP
b77dd0640c Fix typos in comments (M)
Change-Id: I5ab88a01ba3e5ea2aae853bb6f06492fbc84ceb5
2022-01-09 23:00:20 +05:30
Alexander Vorwerk
3aa5b6643b Use LanguageNameUtils constants instead of strings
Change-Id: I8521d00675008da0d7c8c768ab75352212653c66
2021-12-05 22:13:34 +01:00
Reedy
7bf779524a Remove or replace usages of "sanity"
Bug: T254646
Change-Id: I2b120f0b9c9e1dc1a6c216bfefa3f2463efe1001
2021-11-19 23:19:42 +00:00
Umherirrender
43a1a61c1e Use ParserOptions::newFromContext in MessageCache
The context is already called, just reuse it.
This avoids unstub of $wgLang in ParserOptions constructor

Change-Id: I3ac6abf6237db7c1c09740c1fec1224779e86888
2021-10-27 00:10:36 +02:00
Reedy
6c62b2164c s/necesssary/necessary/
Change-Id: I38e5ff8b5b84b0d96b9bbc3e168efe416016184b
2021-10-06 01:08:37 +01:00
Alexander Vorwerk
d028567e4c Remove MessageCache::singleton()
deprecated since 1.34, hard deprecated since 1.37 and unused.

Bug: T249031
Change-Id: Iab7e0dda610b2e65624ede2a0bb419b0b9cc03d1
2021-09-15 18:13:37 +02:00
DannyS712
04202c17bf MessageCache: stop reading from $wgUser
Use RequestContext::getMain()->getUser() instead.
Given that there is an explicit check if the user is safe
to load, this should be fine - if the RequestContext
doesn't already have a user, it'll create one via
User::newFromSession() and then the isSafeToLoad()
call will fail.

Bug: T243708
Change-Id: I217f169b2f5c5a0a337011e383b5171f7c7a2975
2021-07-14 04:55:07 +00:00
Tim Starling
9c3c0b704b Use array_fill_keys() instead of array_flip() if that reflects the developer's intention
array_fill_keys() was introduced in PHP 5.2.0 and works like
array_flip() except that it does only one thing (copying keys) instead
of two things (copying keys and values). That makes it faster and more
obvious.

When array_flip() calls were paired, I left them as is, because that
pattern is too cute. I couldn't kill something so cute.

Sometimes it was hard to figure out whether the values in array_flip()
result were used. That's the point of this change. If you use
array_fill_keys(), the intention is obvious.

Change-Id: If8d340a8bc816a15afec37e64f00106ae45e10ed
2021-06-15 00:11:10 +00:00
Petr Pchelko
bdecf516b9 Replace Title in Message/MessageCache
Change-Id: Ib0e36d767788edcdd8fa9ebc5de6bbde4cf50f12
2021-06-09 06:16:56 -07:00
jenkins-bot
707e435a39 Merge "In MessageCache respect injected content language" 2021-05-15 02:55:53 +00:00
jenkins-bot
dba72b74c3 Merge "Optimise MessageCache::isMainCacheable() for the single-message case" 2021-05-15 02:52:30 +00:00
James D. Forrester
f2f9345e39 Replace uses of DB_MASTER with DB_PRIMARY in documentation and local variables
This is just a start.

Bug: T254646
Change-Id: I9213aad4660e27afe7ff9e5d2e730cbf03911068
2021-05-14 12:40:34 -07:00
Tim Starling
2ae37fa62e In MessageCache respect injected content language
Replace the many references to $wgLanguageCode with a cached copy of the
code of the content language passed to the constructor. The references
to $wgLanguageCode in this class were just meant to be a shortcut for
$wgContLang->getCode().

Change-Id: I60c61aaef0abe6df79ab39f123206d8aae044113
2021-05-14 09:32:38 +10:00
Tim Starling
8a010024be Optimise MessageCache::isMainCacheable() for the single-message case
Loading all messages with getSubitemList() takes about 10ms per
language and loads an array with ~20k elements. When messages in many
languages are requested, this causes an OOM.

So, use getSubitemList() only when isMainCacheable() is called from
loadFromDB(). Remove the second parameter in that case, since it was
always the same.

In the getMsgFromNamespace() case, use getSubitem() to check the
specific message for existence. Have the caller specify the language, in
order to share a subitem cache entry with usual previous
getMessageForLang() call.

Bug: T247223
Change-Id: I6369f307b6bf74bd4aeb1d6e4c41d6e59e403703
2021-05-12 14:19:59 +10:00
Alexander Vorwerk
596344db7a Hard deprecate MessageCache::singleton()
deprecated since 1.34 and unused.

Bug: T249031
Change-Id: I6003c56f07469ffe796ccb415bb4d1e5ff8993a9
2021-05-03 14:11:21 +00:00
James D. Forrester
df5eb22f83 Replace uses of DB_MASTER with DB_PRIMARY
Just an auto-replace from codesniffer for now.

Change-Id: I5240dc9ac5929d291b0ef1c743ea2bfd3f428266
2021-04-29 09:24:31 -07:00
Reedy
cce3fb49d0 Use more neutral or alternative language
Bug: T277987
Change-Id: Iafc4b3e3137936046487119b7e17635f4e560277
2021-03-20 19:47:18 +00:00
C. Scott Ananian
3f990d5b4c Inline Parser::firstCallInit() into ::__construct()
This has effectively been the case since 1.35; this just cleans up the
remaining code which assumed it still needed to explicitly call
Parser::firstCallInit() on a newly-constructed Parser.

Bug: T250444
Change-Id: I340947c721172f12ff413322b4283627c0b0b3a4
2021-03-16 19:41:56 +00:00
Umherirrender
ca1f2b967b Improve docs in MessageCache class
Change-Id: I6c79531691be0f80bdcfe9141ff6cc31ee9b91f9
2020-11-30 17:59:06 +01:00
Thiemo Kreuz
b0130ca649 Update a lot of unspecific "array" types in PHPDocs
This includes fixing some mistakes, as well as removing
redundant text that doesn't add new information, either because
it literally repeats what the code already says, or is actually
duplicated.

Change-Id: I3a8dd8ce57192deda8916cc444c87d7ab1a36515
2020-10-28 11:01:33 +01:00
Umherirrender
f6404ebf38 Create revision records in batch in MessageCache::loadFromDb
This saves two queries per local overridden message when filling the
cache
(one from RevisionStore::loadSlotRecords, one from
SqlBlobStore::fetchBlobs)

Remove 'user' join, as nothing is getting user information from this
records. Join against comment and actor also not needed, but there is no
option to avoid the join.

Change-Id: Ieb5086f81310092f573ef95d997bf0bc99c30952
2020-08-18 10:34:23 +02:00
Tim Starling
68c433bd23 Hooks::run() call site migration
Migrate all callers of Hooks::run() to use the new
HookContainer/HookRunner system.

General principles:
* Use DI if it is already used. We're not changing the way state is
  managed in this patch.
* HookContainer is always injected, not HookRunner. HookContainer
  is a service, it's a more generic interface, it is the only
  thing that provides isRegistered() which is needed in some cases,
  and a HookRunner can be efficiently constructed from it
  (confirmed by benchmark). Because HookContainer is needed
  for object construction, it is also needed by all factories.
* "Ask your friendly local base class". Big hierarchies like
  SpecialPage and ApiBase have getHookContainer() and getHookRunner()
  methods in the base class, and classes that extend that base class
  are not expected to know or care where the base class gets its
  HookContainer from.
* ProtectedHookAccessorTrait provides protected getHookContainer() and
  getHookRunner() methods, getting them from the global service
  container. The point of this is to ease migration to DI by ensuring
  that call sites ask their local friendly base class rather than
  getting a HookRunner from the service container directly.
* Private $this->hookRunner. In some smaller classes where accessor
  methods did not seem warranted, there is a private HookRunner property
  which is accessed directly. Very rarely (two cases), there is a
  protected property, for consistency with code that conventionally
  assumes protected=private, but in cases where the class might actually
  be overridden, a protected accessor is preferred over a protected
  property.
* The last resort: Hooks::runner(). Mostly for static, file-scope and
  global code. In a few cases it was used for objects with broken
  construction schemes, out of horror or laziness.

Constructors with new required arguments:
* AuthManager
* BadFileLookup
* BlockManager
* ClassicInterwikiLookup
* ContentHandlerFactory
* ContentSecurityPolicy
* DefaultOptionsManager
* DerivedPageDataUpdater
* FullSearchResultWidget
* HtmlCacheUpdater
* LanguageFactory
* LanguageNameUtils
* LinkRenderer
* LinkRendererFactory
* LocalisationCache
* MagicWordFactory
* MessageCache
* NamespaceInfo
* PageEditStash
* PageHandlerFactory
* PageUpdater
* ParserFactory
* PermissionManager
* RevisionStore
* RevisionStoreFactory
* SearchEngineConfig
* SearchEngineFactory
* SearchFormWidget
* SearchNearMatcher
* SessionBackend
* SpecialPageFactory
* UserNameUtils
* UserOptionsManager
* WatchedItemQueryService
* WatchedItemStore

Constructors with new optional arguments:
* DefaultPreferencesFactory
* Language
* LinkHolderArray
* MovePage
* Parser
* ParserCache
* PasswordReset
* Router

setHookContainer() now required after construction:
* AuthenticationProvider
* ResourceLoaderModule
* SearchEngine

Change-Id: Id442b0dbe43aba84bd5cf801d86dedc768b082c7
2020-05-30 14:23:28 +00:00
Reedy
b1e515e36e Fix even more PSR12.Properties.ConstantVisibility.NotFound
Change-Id: I4a30a44bc33fc98479799438d65342f6529e14f9
2020-05-16 00:51:46 +01:00
C. Scott Ananian
85e1525862 Deprecate ParserOptions::getTidy() and ParserOptions::setTidy()
These options no longer have any effect.

Bug: T198214
Change-Id: Icc3eaed7ab8a3070c4339b272d580328ba40912d
2020-04-19 22:53:39 -07:00
C. Scott Ananian
37dc40c5df Remove ParserDiffTest
This class was last used in ~2008 as @tstarling was developing the
original wikitext Parser.  It has since code-rotted and wouldn't work
as a drop in for the Parser class any more anyway.  We'll probably
(re)invent something similar when we eventually switch Message
rendering from the legacy parser to Parsoid, but this existing code
isn't a good starting point for that; we'll need to tackle T236812
(splitting Parser into a base class) first.

Last meaningful change to ParserDiffTest:
350b498b9f

Code search:
https://codesearch.wmflabs.org/search/?q=ParserDiffTest&i=nope&files=&repos=

Bug: T236811
Change-Id: I98f1ef8ad296791a810bd8b10343f8640fd23c5e
2020-04-16 14:40:05 -04:00
Petr Pchelko
1e4beb3107 Hard deprecate Revision::newKnownCurrent
Bug: T246284
Change-Id: Iba93fdeb0a88e6fa4068f03367b23a1f670257c6
2020-03-18 11:52:44 -07:00
DannyS712
6191976e50 Make MessageCache::getParserOptions private
Bug: T247189
Change-Id: I3f1eda919f9b236c3215f66231c52ce658706bfe
2020-03-13 03:09:32 +00:00
DannyS712
6b5e8cb461 MessageCache: Pass a user when creating a new ParserOptions
Bug: T246861
Change-Id: I8b0548bfcb8e290aae7d2c0c3cb16e525b14d372
2020-03-07 02:44:07 +00:00
Petr Pchelko
204fa7e509 Remove usages of deprecated Language methods
Change-Id: Iad3375b141b1d87c890baec6ecd16ed92f93e699
2020-02-16 00:45:48 +00:00
ArtBaltai
3bf4b42490 Remove LanguageConverter dependencies on Title and use LinkTarget
Replace usage of Title in LanguageConverter with LinkTarget which
is more light weighted and provides just the props needed in Language.

Bug: T226834
Change-Id: I02a386bd9898e83c773cbd3d738d347d08f52c11
2020-02-12 18:37:11 +03:00
Peter Ovchyn
61e0908fa2 languages: Introduce LanguageConverterFactory
Done:
* Replace LanguageConverter::newConverter by LanguageConverterFactory::getLanguageConverter
* Remove LanguageConverter::newConverter from all subclasses
* Add LanguageConverterFactory integration tests which covers all languages by their code.
* Caching of LanguageConverters in factory
* Make all tests running (hope that's would be enough)
* Uncomment  the deprecated functions.
* Rename FakeConverter to TrivialLanguageConverter
* Create ILanguageConverter to have shared ancestor
* Make the LanguageConverter class abstract.
* Create table with mapping between lang code and converter instead of using name convention
* ILanguageConverter @internal
* Clean up code

Change-Id: I0e4d77de0f44e18c19956a1ffd69d30e63cf51bf
Bug: T226833, T243332
2020-02-03 11:38:03 +02:00
Umherirrender
9187f97aef Pass Title to RevisionStore::newRevisionFromRow in MessageCache
The query for this result already contains a join to page table,
reuse that information and avoid a Title::newFromId in the RevisionStore

Change-Id: Ie02c238292778c3048cc950e37f51f04fc238fea
2020-01-17 16:54:30 +00:00
James D. Forrester
4f2d1efdda Coding style: Auto-fix MediaWiki.Classes.UnsortedUseStatements.UnsortedUse
Change-Id: I94a0ae83c65e8ee419bbd1ae1e86ab21ed4d8210
2020-01-10 09:32:25 -08:00
Daimona Eaytoy
f16d87a8cc Provide a full trace to GlobalTitleFail debug entries
None of these is currently useful: see for instance [1]. Having a whole
trace can help a lot in finding where the faulty call is. Also avoid the
overhead of calling wfDebugLog.

[1] - https://logstash.wikimedia.org/app/kibana#/doc/logstash-*/logstash-mediawiki-2019.12.19/mediawiki?id=AW8gPL5bKWrIH1QRtQa3&_g=h@1251ff0

Bug: T159284
Change-Id: I25e0f0e2ef4899b4eaa644c74fdeaba21d79aefe
2019-12-20 14:12:59 +01:00
Adam Wight
3746d3dacb Cleanup: Explicit public access
Change-Id: If004457382105dc67b9f5471ddf74261a2374d04
2019-12-13 12:03:05 +01:00
Thiemo Kreuz
78ca9eff4a Remove duplicate variable name from class property PHPDocs
Repeating the variable name doesn't do anything. Documentation
generators don't need it. It's more stuff to read that doesn't add new
information. And it can become outdated.

Note there are two types of @var docs. When used inline (and not on a
class property) the variable name is needed.

Change-Id: If5a520405efacd8cefd90b878c999b842b91ac61
2019-12-02 12:58:29 +00:00
Umherirrender
c7ad21c25f Improve param docs
Change-Id: I746a69f6ed01c3ff000da125457df62b02d13b34
2019-11-28 19:08:59 +01:00
Aryeh Gregor
537bdc2d7d Deprecate Language::getMessage*For()
These call the deprecated Language::getLocalisationCache() method and
thereby indirectly access MediaWikiServices. Callers should be changed
to inject a LocalisationCache and use that directly.

There are only a few callers in code search, so it didn't seem worth
adding convenience methods to LocalisationCache. The one caller that was
already using DI was MessageCache, and I injected LocalisationCache
there.

Bug: T201405
Change-Id: I01919fba5685fc5e0a31f739714f125a22de8939
2019-10-29 11:52:07 +02:00
Aryeh Gregor
0de9c47b50 Remove Language::factory and getParentLanguage use
Change-Id: I11f8801ef47ec1a1f63d840116e69667e6f3ae3c
2019-10-27 12:34:28 +02:00
Timo Tijhof
156e0aed63 localisation: Convert MessageCache to PSR-3 logging
Change-Id: I9eaf8e419cf2895733fce1bff83aa81a3d21c39c
2019-10-12 17:38:59 +01:00
Timo Tijhof
67f3df57f9 MessageCache: Replace internal loadedLanguages array with special cache key
Before c962b48056, the 'loadedLanguages' array was used to track
which languages were loaded and in the cache, with 'cache' being a
simple array. In that commit, the 'cache' array also started being used
for incomplete datasets, which didn't affect 'loadedLanguages'.

Then in 97e86d934b, the 'loadedLanguages' array was removed in favour
of checking keys on 'cache' directly, and 'cache' was converted to
MapCacheLRU.

This led to problem where partially loaded data was mistaken for being
full datasets (fatal error, T208897). This was fixed in a5c984cc59,
by bringing back the 'loadedLanguages' array, which fixed the issue from
the POV of partially loaded data.

However, this then exposed a new problem. The 'cache' data can be evicted
by MapCacheLRU, whereas 'loadedLanguages' is not aware of that. Thus it
claims languages are loaded that sometimes aren't. (This only affects web
requests where more than 5 language codes are involved, per MapCacheLRU.)

Fix this by re-removing the 'loadedLanguages' array, this time
strengthening the 'cache' key check to not just check that the root key
exists, but that it is in fact holding the full dataset as generated by
MessageCache::load(). The 'VERSION' key appears to be a good proxy for
that.

Bug: T230690
Change-Id: I1162a3857376aa37e5894ae3c8be84a2295782a3
2019-10-02 22:47:00 +00:00
Daimona Eaytoy
e70b5b3309 Unsuppress other phan issues (part 4)
Bug: T231636
Depends-On: I58e67c2b38389df874438deada4239510d21654f
Change-Id: I6e5fba7bd273219b1206559420b5bdb78734aa84
2019-08-31 17:13:39 +00:00
daniel
b860ef0d13 Avoid fatal errors when reporting exceptions.
When reporting exceptions that occur during initialization, wgUser may
be null. Don't die when that happens.

Change-Id: I65d5a17d80f9021e28a218c7a5a17e399bc7ce98
2019-08-29 13:07:46 +02:00
jenkins-bot
bf7284d975 Merge "MessageCache: Add STRAIGHT_JOIN to avoid planner oddness" 2019-08-28 04:29:31 +00:00
Timo Tijhof
1d7f793108 MessageCache: Increase APC 'messages-big' expiry from 1min to 1h
Bug: T218207
Change-Id: Ic5d2a556912e2a16ee899eec3a0670f00dec9a8c
2019-08-27 22:58:59 +00:00
Timo Tijhof
178d312eb8 MessageCache: Remove $wgMsgCacheExpiry configuration var
This variable has never been set to anything other than the default value of
24 hours as introduced in 2003 (r2203, r2204; or 036ff960ce, edf6b38626).

The variable has never changed in core, it's not overridden at WMF,
and MessageCache is not constructed anywhere other than ServiceWiring.php
anywhere in repos on Wikimedia Gerrit, indexed by MediaWiki Codesearch,
or any GitHub-hosted repository (incl Wikia repos and WikiHow mirrors).

I've also checked all GitHub-hosted repos for boilerplates and/or public
settings files from devs or prod, and couldn't find any example of
this being overridden (after filtering out copies of the core files
themselves). Rather than having to support potentially hard-to-predict
interactions betweeen caching layers by checking its state, make it
a constant so we can code reason about it more easily.

Change-Id: Ie2e139001aae3ac54b509d94a3d917bb408eaca0
2019-08-27 17:33:11 +00:00