Commit graph

100 commits

Author SHA1 Message Date
Umherirrender
975d7e9e4a historyblob: Improve param and property documentation
Also cast scalar types when needed

Change-Id: I88b17fe03d398df1ce2686aa0e223fc58e97e037
2022-03-08 23:45:44 +00:00
Reedy
11cc2d6993 Consistently use @deprecated since rather @deprecated
Change-Id: I301786f35cd6eb9c63fb72e0b64cffa9efd7b86b
2022-03-06 02:55:54 +00:00
Umherirrender
9efd9ca45e Add explicit casts between scalar types
* Some functions accept only string, cast ints and floats to string
* After preg_matches or explode() casts numbers to int to do maths
* Cast unix timestamps to int to do maths
* Cast return values from timestamp format function to int
* Cast bitwise operator to bool when needed as bool

* php internal functions like floor/round/ceil documented to return
  float, most cases the result is used as int, added casts

Found by phan strict checks

Change-Id: Icb2de32107f43817acc45fe296fb77acf65c1786
2022-03-01 18:19:33 +01:00
Tim Starling
ca71e69fc6 Try not to discard Excimer timeout exceptions
Don't catch and discard exceptions from the RequestTimeout library,
except when the exception is properly handled and the code seems to be
trying to wrap things up.

In most cases the exception is rethrown. Ideally it should instead be
done by narrowing the catch, and this was feasible in a few cases. But
sometimes the exception being caught is an instance of the base class
(notably DateTime::__construct()). Often Exception is the root of the
hierarchy of exceptions being thrown and so is the obvious catch-all.

Notes on specific callers:

* In the case of ResourceLoader::respond(), exceptions were caught for API
  correctness, but processing continued. I added an outer try block for
  timeout handling so that termination would be more prompt.
* In LCStoreCDB the Exception being caught was Cdb\Exception not
  \Exception. I added an alias to avoid confusion.
* In ImageGallery I added a special exception class.
* In Message::__toString() the rationale for catching disappears
  in PHP 7.4.0+, so I added a PHP version check.
* In PoolCounterRedis, let the shutdown function do its thing, but
  rethrow the exception for logging.

Change-Id: I4c3770b9efc76a1ce42ed9f59329c36de04d657c
2022-02-02 16:27:44 +11:00
Siddharth VP
061ee25f7e Fix typos in comments (E-H)
Change-Id: I0748e0d5962fa909fdd6b7fcae4ab259bde4cdf1
2021-12-30 18:14:43 +05:30
Timo Tijhof
f21caa41d3 externalstore: Improve overall documentation
* Consolidate various duplicate docs to the main entry point.
  Especially the duplication that followed after ExternalStoreFactory
  and ExternalStoreAccess were introduced with ExternalStore
  left as deprecated wrapper.

* Officially mark ExternalStoreFactory as internal, pointing
  to ExternalStoreAccess instead. And document why the latter exists,
  since it seems all its methods could trivially move to the factory
  class. Follows-up I40c3b553, where it was explained that the factory
  is meant to be an internal implementation detail, with the access
  class meant to resemble ExternalStoreMedium.

* Use consistent terms for "protocol", "location" and "object"
  (reduce use of words like medium, backend, type, blob, revision,
  etc. within this code).

* Document that ExternalStore is fundamentally designed as an
  append-only store (per T247383).

Bug: T247383
Change-Id: I1094af7d35a14f9e47b8791f3e4cc888e8cbfc7f
2021-12-15 02:03:47 +00:00
jenkins-bot
f6f6bef180 Merge "ExternalStoreDB::getMaster: Hard deprecate now it's unused in prod" 2021-09-08 20:52:48 +00:00
James D. Forrester
be0d735666 ExternalStoreDB::getMaster: Hard deprecate now it's unused in prod
Depends-On: I99cc76f75928eff30e8bba2137c0a4af5819cd16
Change-Id: I9e59ba7ee6bb122abb996572e390e1f8b5260d6a
2021-09-07 18:49:44 -07:00
Umherirrender
44fd53fee3 Using @return never documentation on always-throw-function
This helps phan to detect unreachable code and also impossible types
after the functions.
It helps phan to avoid false positives for array keys
when the keys are checked before

Bug: T240141
Change-Id: I895f70e82b3053a46cd44135b15437e6f82a07b2
2021-09-07 17:29:03 +02:00
James D. Forrester
eeaec4ce6b ExternalStoreDB: Rename getMaster to getPrimary
Bug: T282894
Change-Id: I6d6361a0b84f7130a27df6a6c75a983c73e5dd95
2021-09-03 13:44:04 -07:00
Umherirrender
4d42b5e86e docs: Change wording master to primary in comments and log text
Bug: T254646
Change-Id: I5379dc79be60c99f0a30f74e5d624f81fe6f921b
2021-09-01 23:04:40 +02:00
Umherirrender
b4bd3d4ee8 Remove dead variables and functions and unneeded parenthesis
Found by sonarqube

Change-Id: I3464d25e94d98aaca55d9209d5175542b335f994
2021-08-13 21:41:59 +02:00
TChin
077b9af960 Replace Assert::parameterType with typehints
Bug: T287530
Change-Id: I6060e194339614b53e3a9c036ff3a3ac2e68f8df
2021-08-03 10:03:48 -04:00
James D. Forrester
719cf161f2 More master -> primary documentation and internal var renaming
Bug: T254646
Change-Id: I63cc8895033714bdfbf09aee933a8f0a43b387f3
2021-07-15 11:20:20 +01:00
Gergő Tisza
31408b7fef Silence transaction profiler warnings in ExternalStoreDB::fetchBlob()
ExternalStoreDB falls back to a primary connection if the blobs with
the specified ID weren't found with the normal connection. Since
the ExternalStore abstraction layer hides the fact that there are
database connections involved, the caller has no way to control
this behavior, so there is no point in emitting warnings when it
happens on GET.

Bug: T282145
Change-Id: I7f8113230e4b6493b8bdb96e639781ab9c116397
2021-05-16 18:25:57 +02:00
James D. Forrester
f2f9345e39 Replace uses of DB_MASTER with DB_PRIMARY in documentation and local variables
This is just a start.

Bug: T254646
Change-Id: I9213aad4660e27afe7ff9e5d2e730cbf03911068
2021-05-14 12:40:34 -07:00
James D. Forrester
df5eb22f83 Replace uses of DB_MASTER with DB_PRIMARY
Just an auto-replace from codesniffer for now.

Change-Id: I5240dc9ac5929d291b0ef1c743ea2bfd3f428266
2021-04-29 09:24:31 -07:00
Aaron Schulz
84d8de5331 externalstore: convert some log messages to WARNING
Bug: T281048
Change-Id: Ib1fa867b14af6acfcf55cfd642aeabec35d8d189
2021-04-27 00:20:31 +00:00
jenkins-bot
e8ce41cfc4 Merge "ExternalStoreMwstore: Remove unnecessary check" 2021-03-13 20:57:50 +00:00
Ammarpad
7ef28de237 ExternalStoreMwstore: Remove unnecessary check
FileBackendGroup::get() always returns FileBackend object
(or throws exception, which makes no difference here).

Change-Id: I6216a3925d623ae5da9407ae149a8a6828696933
2021-03-09 22:36:41 +01:00
Ammarpad
a2e7a75392 ExternalStoreMwstore: Improve method docs
Change-Id: I8c471f9052f33608ec19d7a0d4d205fe1dc2f00d
2021-03-09 22:19:38 +01:00
Thiemo Kreuz
b0130ca649 Update a lot of unspecific "array" types in PHPDocs
This includes fixing some mistakes, as well as removing
redundant text that doesn't add new information, either because
it literally repeats what the code already says, or is actually
duplicated.

Change-Id: I3a8dd8ce57192deda8916cc444c87d7ab1a36515
2020-10-28 11:01:33 +01:00
DannyS712
7e08ccf821 Fix order of wfDeprecated parameters in ExternalStoreDB::getSlave
Change-Id: Ie0fc3e4cc3233d64353a3e48ffff4e572ba34625
2020-10-06 18:33:25 +00:00
Reedy
a78a84fbe7 Hard deprecate ExternalStoreDB::getSlave()
Change-Id: Id9695f59d6347064347c2b34b0803c1ab9c262eb
2020-07-29 13:05:10 +00:00
daniel
102c9573b3 Mark exceptions as @newable
Exceptions classes are nearly always value objects, and should in most
cases by newable.

Bug: T247862
Change-Id: I4faa8ec6ea8bc44086cfc8075b32d10eea61e9df
2020-07-08 17:51:38 +00:00
Tim Starling
47a1619027 Remove terminating line breaks from debug messages
A terminating line break has not been required in wfDebug() since 2014,
however no migration was done. Some of these line breaks found their way
into LoggerInterface::debug() calls, where they mess up the formatting
of the debug log.

So, remove terminating line breaks from wfDebug() and
LoggerInterface::debug() calls.

Also:
* Fix the stripping of leading line breaks from the log header emitted
  by Setup.php. This feature, accidentally broken in 2014, allows
  requests to be distinguished in the log file.
* Avoid using the global variable $self.
* Move the logging of the client IP back to Setup.php. It was moved to
  WebRequest in the hopes that it would not always be needed, however
  $wgRequest->getIP() is now called unconditionally a few lines up in
  Setup.php. This means that it is put in its proper place after the
  "start request" message.
* Wrap the log header code in a closure so that variables like $name do
  not leak into global scope.
* In Linker.php, remove a few instances of an unnecessary second
  parameter to wfDebug().

Change-Id: I96651d3044a95b9d210b51cb8368edc76bebbb9e
2020-06-03 12:01:16 +10:00
daniel
74aba39e34 ExternalStore: report cause of non-exception failure
This should help with investigating the cause of some errors we have
been seeing sporadically.

Bug: T247429
Bug: T228088
Change-Id: Ibdb48cac447315ed6f37a9cd0e7c05deefc76a28
2020-04-17 18:00:28 +02:00
Daimona Eaytoy
41ee2f2c61 Upgrade phan to 0.9.1
Released just now.

Many old suppressions can now be removed. Enabling the issue for
undeclared variables is left to do later, given that there are
roughly 200 warning.

Change-Id: I99462a1e9232d6e75022912e2df82bc2038476ef
2020-01-25 10:53:26 +00:00
James D. Forrester
4f2d1efdda Coding style: Auto-fix MediaWiki.Classes.UnsortedUseStatements.UnsortedUse
Change-Id: I94a0ae83c65e8ee419bbd1ae1e86ab21ed4d8210
2020-01-10 09:32:25 -08:00
Aryeh Gregor
0d3d6be18d Convert FileBackendGroup to service
Bug: T234228
Change-Id: I25575f565eba122cdf971a5945572811d17fa3e1
2019-10-25 10:46:20 +03:00
Umherirrender
ba4adc3fc9 phpcs: Enable PSR12.Files.ImportStatement.LeadingSlash and make pass
Change-Id: Ifaa7bef925cc4e1b3f1751bb66abdda7fe6763d8
2019-10-11 20:01:32 +02:00
Aaron Schulz
176e08484f externalstore: deprecate getLBInfo() dependency in ExternalStoreDB
Use the cluster and master server info array instead, when possible.

Also add initializeTable() method based on the one in addWiki.php.
This lets some internals be hidden from that script a bit more.

Bug: T231162
Change-Id: I694f12cc800684bcc13351021d9abc9a01bddfe9
2019-09-10 11:34:52 +00:00
Aaron Schulz
92412c166b externalstore: use CONN_TRX_AUTOCOMMIT instead of IDatabase::clearFlag()
Change-Id: I8984623d3ff67d305597f05c596c2777c2c117e0
2019-07-27 05:08:06 -04:00
Aaron Schulz
4f76b9543a externalstore: unbreak writes to non-default storage clusters due to isReadOnly()
Instead of telling ExternalStoreMedium classes the default write stores and using
that to make them read-only, let them be configured via other means. For example,
ExternalStoreMwstore already respects FileBackend::isReadOnly() for each location
(e.g. file backends) and ExternalStoreDB checks LoadBalancer::getReadOnlyMode()
for each location (e.g. DB cluster).

Make ExternalStoreAccess::isReadOnly() take a list of base URLs, default to the
default write stores if not specified.

Bug: T227156
Change-Id: I3161890fb2ccb46d6206628f0cd88f8af9f1688c
Follows-Up: I40c3b5534fc8a31116c4c5eb64ee6e4903a6197a
2019-07-16 22:08:00 -07:00
Aaron Schulz
baafb5adb4 Make ExternalStore wrap ExternalStoreFactory and create access class
* Inject settings and global instances as dependencies to the
  ExternalStoreMedium instances. This includes the local wiki
  domain, so that wfWikiId() calls are not scattered around.
* Create ExternalStoreAccess service for read/write logic.
* Deprecate the ExternalStore wrapper methods.
* Add some exception cases for bogus store URLs are used instead
  of just giving PHP warnings and failing later.
* Make moveToExternal.php require the type/protocol to decide
  which ExternalStoreMedium to use instead of assuming "DB".
* Convert logging calls to use LoggerInterface.

Change-Id: I40c3b5534fc8a31116c4c5eb64ee6e4903a6197a
2019-06-28 14:31:44 -07:00
Daimona Eaytoy
f5eaf6470b externalstore: Fix insert*() return docs and remove redundant checks
ExternalStorage::insertWithFallback is reported to return false on
failure, but it doesn't. It has a single exit point, and return value is
checked with strlen(), so actually it can only return the URL or throw.
Thus, update any related doc and remove a redundant check from code
calling insertToDefault.

Change-Id: Ic95c3aed19118b987aef105f8077d55558f39127
2019-06-13 18:23:23 +00:00
Jesús Martínez Novo
df6100e0cc ExternalStore: Pass external domain to getReadOnlyReason
Be consistent with other paths of code that supply the external domain
id to LoadBalancer instead of defaulting to the wiki's domain.

This was causing the getReadOnlyReason to fail because it tries to
connect to the local wiki database using external storage credentials.

Bug: T200471
Change-Id: Ie73ea931d5c93f967624ee67717f87c2d9a31559
2019-05-22 19:09:00 +00:00
Aaron Schulz
bbf7fee3ac externalstore: make ExternalStoreDB::getDomainId treat false the same as null
Callers like SqlBlobStore sometimes pass in false for the current wiki

Bug: T200471
Change-Id: I3025c869df07de312471d00a0ab7107c1fa14a90
2019-05-21 21:05:23 +00:00
Aryeh Gregor
9018579681 Deprecate the Http class
All methods got moved to HttpRequestFactory or MWHttpRequest or dropped.

I made the return value of the new HttpRequestFactory::request/get/post
methods null on error instead of false, so that when we drop PHP 7
support, we can use a "?string" return value. This could theoretically
change behavior of code that was switched from the old Http methods, but
probably won't. I kept the old behavior for the deprecated methods.

I changed the default value of $wgHTTPProxy from false to ''. This way
it should be usable directly without a trivial wrapper method. For the
benefit of anyone who might have set it to false in LocalSettings.php, I
also recommend casting to string just in case.

Http::$httpEngine is deprecated. Eventually it will be removed along
with the curl and PHP engines, leaving only the Guzlle engine.

I also added deprecation of MWHttpRequest::factory, which occurred in
1.31, to the release notes for 1.34. Now hopefully we can hard-deprecate
it in another couple of versions.

Bug: T214390
Change-Id: I2a316a758d793857f248bd251b90f5e9a6440e3a
2019-05-06 12:07:26 +03:00
Thiemo Kreuz
666e357282 Replace misplaced strlen() checks with strict false comparisons
As far as I can tell the intention of this code is *not* to detect
empty strings, but `false`.

Change-Id: I406d065a54a1437fdfd46827fec9493e5fad05f9
2019-03-27 11:10:39 +01:00
Fomafix
f17c297624 Use short assignment operator in PHP
Use
  $var .= $foo
instead of
  $var = $var . $foo

Change-Id: I5dcdd7278e618c14968e5ac1fb8ea43ac2200deb
2019-03-07 09:55:49 +01:00
Aaron Schulz
dcd0a3d534 Add isCurrentWikiId()/isCurrentWikiDomain()/getCurrentWikiDomain() to WikiMap
Use these in place of various wfWikiID() calls.

Also cleanup UserRightsProxy wiki ID variable names and removed unused
and poorly named getDBname() method.

Change-Id: Ib28889663989382d845511f8d34712b08317f60e
2018-10-29 14:53:37 -07:00
Aaron Schulz
f7009eeb95 Make ExternalStoreDB "wiki" context override the server "dbname" field
This is all that is needed for b/c and going beyond that can break
foreign wiki (domain) external store access if matching per-wiki DB
names *are* used for external store.

Follow up to 92e4ace7ea.

Bug: T200471
Change-Id: I877139ff659f542da04d4d8f5ef3297dbfcfd734
2018-08-15 03:02:56 +00:00
Aaron Schulz
92e4ace7ea Use LB server configuration to force DB domains in ExternalStorageDB
This is for backwards-compatibility for pre 14ee3f2107 external store
configuration that relied on not using the main wiki DB name(s).

Bug: T200471
Change-Id: Ie60cae64e32ff2532565cbd79c8e084634a61cce
2018-08-13 18:58:02 +00:00
daniel
53fd8295ff Use consistent caching strategy in Revision storage classes
DEPLOYMENT: This changes the cache key for revision
content blobs. Expect a brief rise in ExternalStore hits.

Bug: T198704
Change-Id: Icc2d16bc5a1e27ba4caea49a784ba7aeac15042a
2018-07-05 17:44:24 +02:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
Jayprakash12345
011711e79f Fix typo accessable --> accessible
Change-Id: I239a15e1bc3a311d9284285a04094afae331111d
2018-03-14 05:25:34 +00:00
Umherirrender
554f9c857c Replace wfGetLBFactory
@deprecated since 1.27

Change-Id: I11a7253cebe525948a55cebee183e6de128fdc39
2018-02-27 20:02:48 +00:00
Aaron Schulz
d9ba7cd005 Make LocalFile check early if the revision store is available
This reduces the odds of having files without corresponding
wiki pages, given that the later is done in a deferred update.

Also made some documentation cleanups.

Bug: T187942
Change-Id: Iff516669f535713d37e0011e2d7ed285c667f1c5
2018-02-22 22:07:30 +00:00
Aaron Schulz
656f60c154 Add ExternalStoreMedium::isReadOnly() method
Use this to abort out of store() calls early

Bug: T187942
Change-Id: I9334d36e8bc3e4589775471eee03be4f4a3119a3
2018-02-22 08:15:20 -08:00