Commit graph

86 commits

Author SHA1 Message Date
Dreamy Jazz
e7393b3cc7 Exclude boilerplate maintenance code from code coverage reports
Why:
* Maintenance scripts in core have bolierplate code that is
  added before and after the class to allow directly running
  the maintenance script.
* Running the maintenance script directly has been deprecated
  since 1.40, so this boilerplate code is only to support a now
  deprecated method of running maintenance scripts.
* This code cannot also be marked as covered, due to PHPUnit
  not recognising code coverage for files.
* Therefore, it is best to ignore this boilerplate code in code
  coverage reports as it cannot be marked as covered and also
  is for deprecated code.

What:
* Wrap the boilerplate code (requiring Maintenance.php and then
  later defining the maintenance script class and running if the
  maintenance script was called directly) with @codeCoverageIgnore
  comments.
* Some files use a different boilerplate code, however, these
  should also be marked as ignored for coverage for the same
  reason that coverage is not properly reported for files.

Bug: T371167
Change-Id: I32f5c6362dfb354149a48ce9c28da9a7fc494f7c
2024-08-27 13:22:29 +01:00
Amir Sarabadani
d9370003fb maintenance: Introduce getReplicaDB() and getPrimaryDB()
And start using them instead of wfGetDB(), LB/LBF connection methods or
worse, $this->getDB().

$this->getDB() reuses the database object regardless of whether you're
calling a replica or primary, leading to returning a replica on a
primary and other way around.

Bug: T330641
Change-Id: I9e2cf85ca277022284fc26b9f37db57bd12aaa81
2024-01-18 15:12:04 +01:00
Amir Sarabadani
2659334cd5 updateSpecialPages: Call ::waitForReplication() if a host isn't reachable
That trigger an update of config and in the next call, it shouldn't show
up anymore. The whole thing needs to be revamped but oh well.

Bug: T348428
Bug: T348419
Change-Id: I4a8631d821bed97aa36d0371de3504fe5b3b3d5e
2023-10-09 12:31:38 +02:00
Umherirrender
59da02253e updateSpecialPages.php: Avoid implicit float conversion on modulo
Shown on php8.1:
Deprecated: Implicit conversion from float 0.05353713035583496 to int
loses precision in maintenance\updateSpecialPages.php on line 169

Change-Id: I0b70e1f5db182ef17018eeac099252694c83eea4
2023-09-20 22:04:46 +02:00
James D. Forrester
459cbb0494 Namespace remaining 'specialpage' files under \MediaWiki\SpecialPage
SpecialPageFactory is already here, but none of the others were yet.

Bug: T166010
Change-Id: I9689bf0a1ab329625e23669b99f019b96295fffd
2023-09-18 18:23:13 +01:00
Derick Alangi
74033c50cd maintenance: Begin using Maintenance::getServiceContainer()
Maintenance class provides a method for getting a fresh reference
of the MW services container instance. Let's make use of these in
maintenance scripts now that we have it.

NOTE: There are still some static methods like in refreshLinks.php
that makes use of services that we can't use this method for now.

Change-Id: Idba744057577896fc97c9ecf4724db27542bf01c
2023-09-04 10:39:58 +00:00
Amir Sarabadani
b525884e11 maintenance: Use $this->waitForReplication()
This adds reconfiguring db pools in case a replica gets depooled

Bug: T298485
Change-Id: Id052ce8ed45c51e51b071778858d27b48605bf93
2022-10-24 21:11:53 +02:00
Tim Starling
0077c5da15 Use short array destructuring instead of list()
Introduced in PHP 7.1. Because it's shorter and looks nice.

I used regex replacement.

Change-Id: I0555e199d126cd44501f859cb4589f8bd49694da
2022-10-21 15:33:37 +11:00
Aryeh Gregor
79fc95d39c Use MainConfigNames instead of string literals, #5
This should be the last of the usages in core, although I'm sure a few
are hiding somehow.

Change-Id: I7bf0b24bf23d3efb4c56a891830bbfe67945e899
2022-04-27 18:46:29 +03:00
Umherirrender
f11bc74471 maintenance: Improve updateSpecialPages.php and related documentation
Fix MediaStatistics if $wgQueryCacheLimit is very small to still show
all entries as documented in the class.

Found by phan strict checks

Change-Id: Ic1a8a66063f6604db4a7e92c46df94a3f6d97711
2022-03-14 20:19:53 +00:00
Umherirrender
94792e97be maintenance: Cleanup use of exit in Maintenance::execute implementation
Returning void from execute() is success.
Returning true is success, false is failure with exit(1)
Using fatalError also using exit(1)

Change-Id: I1d40430ad6226e4aab8f0810b03ee1213282d123
2021-08-30 19:48:27 +00:00
James D. Forrester
df5eb22f83 Replace uses of DB_MASTER with DB_PRIMARY
Just an auto-replace from codesniffer for now.

Change-Id: I5240dc9ac5929d291b0ef1c743ea2bfd3f428266
2021-04-29 09:24:31 -07:00
Umherirrender
c11b87822f Delete cached data of non-cached querypages with updateSpecialPages.php
If a special page is no longer expensive (by code or by config like
DisableQueryPages) the data should not longer stored in the database,
because always fresh data are shown and never the cached data

This does not delete on disabled query pages, because the data still
visible on the page.
This only affectes query pages and not very special pages like
ActiveUsers

Bug: T174513
Change-Id: Ia9015cb7bc0ab7b1a3d96d1b481c31b3e6c39657
2021-03-10 20:28:09 +01:00
Umherirrender
9a8a33bd0f Add a run mode to $wgDisableQueryPageUpdate
The run mode is used to show a different message on the special page,
instead of the current one "Updates for this page are currently
disabled. Data here will not presently be refreshed." even the data gets
updated with a different cron job.

Bug: T78711
Change-Id: Ib63d16bfea477dec43323b39671cc068530e2f0b
2019-09-26 22:33:53 +00:00
Thiemo Kreuz
b7cd670cb7 maintenance: Remove unused code from several maintenance scripts
The most notable removal is done in the orphans script. This code was
really never used. Brion introduced it in 2005, already disabled.

I have all the respect for what Brion did. I just think it does not make
much sense to keep code around for so long if it does not work anyway,
and must be rewritten from scratch anyway now that we have multi-content
revisions and such.

Change-Id: I4e8050929f90e44a6e6051bf938993a8b0cdf649
2019-03-03 16:57:19 +00:00
jenkins-bot
2f86f8dbb9 Merge "Don't throw an exception when waiting for replication times out" 2018-09-04 02:20:12 +00:00
Tim Starling
e8df0fbab1 Don't throw an exception when waiting for replication times out
For maintenance scripts it is usually harmful to throw an exception.
For jobs the exception was already caught and handled appropriately,
so this can continue as before. For DeferredUpdates it was extremely
harmful to throw an exception. So in the web case, reduce the timeout to
1s and continue as normal if the 1s timeout is reached. This allows the
DeferredUpdate to be throttled without being killed.

In the updater, increase the replication wait timeout to 5 minutes.
ALTER TABLE could indeed cause replication lag, but exiting the update
script with an exception will probably ruin your day. Update actions are
not necessarily efficiently restartable.

Do not call JobQueue::waitForBackups() when jobs are popped. Maybe it
makes sense to call a queue-specific replication wait function for
bulk inserts, like copyJobQueue.php, but doing it when jobs are popped
just makes no sense. Surely the worst that could happen is that the
queue would become locally empty? Removing this waitForBackups() call
avoids waiting for replication twice when JobQueueDB is used.

Bug: T201482
Change-Id: Ia820196caccf9c95007aea12175faf809800f084
2018-09-03 12:29:35 +10:00
Aryeh Gregor
e68fdb4065 Mass conversion to SpecialPageFactory service
Change-Id: Ia6e1e819ec6cbe8bf75b820109f51d47863e31fc
2018-08-17 12:03:12 -07:00
Fomafix
6866cfec37 Simplify PHP by using ?? and ?:
Also remove not necessary surrounding parentheses.

Change-Id: I0eb5c9c1bdfb09a800258379cdcefb5fd4d3d21c
2018-07-10 20:03:17 +00:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
Umherirrender
e1a203603c Replace wfGetLB
@deprecated since 1.27

Change-Id: Ibdd49fdfc0d1511503e1ed2173a592c612996c53
2018-05-02 22:30:24 +02:00
Umherirrender
ad776c7d5f Use ::class to resolve class names in maintenance scripts
This helps to find renamed or misspelled classes earlier.
Phan will check the class names

Change-Id: I1d4567f47f93eb1436cb98558388e48d35258666
2018-01-23 17:40:16 +00:00
Bryan Davis
9e34eeff23 Maintenance: add fatalError() method
Deprecate the second argument to Maintenance::error() in favor of a new
Maintenance::fatalError() method. This is intended to make it easier to
review flow control in maintenance scripts.

Change-Id: I75699008638f7e99b11210c7bb9e2e131fca7c9e
2017-11-21 21:34:16 -07:00
Brian Wolff
5b2bed35f3 Follow-up 67f04373d fix stupid typo
Fot should be For

Change-Id: I3cb87a94feea32f1c98345ceaa19f9a75fde266d
2017-09-28 21:37:19 +00:00
Brian Wolff
67f04373d6 Reopen db connection after doing SpecialPageCacheUpdates
Special page updates on commons have been broken recently due
to Mysql "going away" after doing the ActiveUsers special page
cache update. Presumably this is because we were only repinging
the server after QueryPage updates, and not the callback updates
(Combined with a spike in wikidata recentchanges entries making
the ActiveUsers callback now take about an hour to complete)

Bug: T176055
Change-Id: Ic344940460c353544d7c08bbe2d8c900f6cc73da
2017-09-27 22:48:31 +00:00
Aaron Schulz
c0a9ab0f6d Change "slave" => "replica DB" in /maintenance
Change-Id: Ibd3d617901130378a935402326cd4eefbb382c9e
2016-09-06 00:13:08 +00:00
Reedy
44cebea941 Update wfGetDB calls in Maintenance scripts to use getDB()
Change-Id: I9ad6745d84506b736dae94747256caac89715899
2016-01-02 16:58:23 +00:00
Kunal Mehta
f98db63ff1 updateSpecialPages: Get rid of dead SpecialPage::getFile() code
That method doesn't exist any more.

Change-Id: I35eb8e2d7e77a6fcb17187356bafd92cfede6b40
2015-10-27 09:37:05 -07:00
Alexandre Emsenhuber
9a99072827 Get the value of microtime() directly as float in updateSpecialPages.php
It has a first parameter to directly get the value as float, so use it
instead of doing string manipulation.

Change-Id: Id2dff4486ea4f308ce03fc3d5546660c4e3c26b6
2014-10-20 19:29:31 +02:00
Kevin Israel
ffa3518ed5 Move QueryPage and abstract subclasses to specialpage folder
... now that f508561f7b ("Moved $wgQueryPages stuff out of the global
scope and into a function") has been merged, which eliminated hardcoding
of the pathname except in the AutoLoader.

WantedQueryPage is moved to a separate file.

Bug: 60445
Change-Id: If2019492722083819b82afa6302670b94b181098
2014-06-24 20:03:21 -04:00
Siebrand Mazeland
606c680b21 Update formatting in maintenance/ (4/4)
Change-Id: I6b58d014a4bfd6600e4e6f80188fdcfce18482ca
2014-04-23 20:09:26 +02:00
Chad Horohoe
f508561f7b Moved $wgQueryPages stuff out of the global scope and into a function
Change-Id: Ica034bdad89133bfce0b4238d62ed00865936644
2014-03-18 08:36:32 -07:00
Aaron Schulz
1baaa740ba Made UpdateSpecialPages support --list/--only with the callback updates
* Fixed some timing output calls

Change-Id: I7bbb9123def51c703c1b77bbd946316c6a264d0f
2014-02-05 05:53:45 +00:00
Tim Starling
2be2abf30f Fix QueryPage transaction plan
The COMMIT in updateSpecialPages.php was unmatched and just logged an
error. We would like the stale results to still be presented to the user
during the hours of query execution time, so doing a DELETE in
autocommit mode before the main query starts does not seem appropriate.
And holding a master transaction open for hours, with a lock on
querycache, is certainly not the right way to do it.

So, move the DELETE to after the completion of the main query, and wrap
a transaction around the updates to querycache and querycache_info so
that the user always sees a consistent populated UI. Remove the
unmatched COMMIT from updateSpecialPages.php.

Change-Id: I27c22b96f43a1064eb17a0c6a1c56d1f4a2dff9a
2013-12-11 11:57:00 +11:00
Reedy
33de342acf If --only and we've had a hit, no need to continue iteration over $wgQueryPages
--only only accepts one "page"

Change-Id: I561a099cd6ec8a19e8931f6aa5304fc38a474e58
2013-08-22 17:20:00 +00:00
Reedy
28595b9e82 Don't do special page cache updates if --list or --only parameters passed
Change-Id: I06522ea888d2d7f5cbfd22dd70e58011d65fd2c1
2013-08-22 17:10:33 +00:00
Nemo bis
012c44fd15 Fix help for --only: override doesn't work with CamelCase
See I725d07397cfa6e61e0644a9a40635d608f6be98d

Change-Id: I51320f5b2ae4335271bda53ce523c79328ff8d83
2013-08-17 13:35:30 +02:00
Timo Tijhof
beb1c4a0ec phpcs: More require/include is not a function
Follows-up I1343872de7, Ia533aedf63 and I2df2f80b81.

Also updated usage in text in documentation and the
installer LocalSettingsGenerator.

Most of them were handled by this regex:
- find: (require|include|require_once|include_once)\s*\(\s*(.+?)\s*\)\s*;$
- replace: $1 $2;

Change-Id: I6b38aad9a5149c9c43ce18bd8edbab14b8ce43fa
2013-05-21 23:26:28 +02:00
Timo Tijhof
50e7985d4d phpcs: Fix WhiteSpace.LanguageConstructSpacing warnings
Squiz.WhiteSpace.LanguageConstructSpacing:
   Language constructs must be followed by a single space;
   expected "require_once expression" but found
   "require_once(expression)"

It is a keyword (e.g. like `new`, `return` and `print`). As
such the parentheses don't make sense.

Per our code conventions, we use a space after keywords like
these. We appeared to have an unwritten exception for `require`
that doesn't make sense. About 60% of require/include usage
was missing the space and/or had superfluous parentheses.

It is as silly as print("foo") or return("foo"), it works
because keywords have no significance for whitespace between
it and the expression that follows, and since experessions can
be wrapped in parentheses for clarity (e.g. when doing string
concatenation or mathematical operations) the parenthesis
before and after basiclaly just ignored.

Change-Id: I2df2f80b8123714bea7e0771bf94b51ad5bb4b87
2013-05-09 05:56:26 +02:00
umherirrender
bfb75bc8e2 Fixed spacing around parenthesis in languages/tests/maintenance
Change-Id: Idd4299d17f1fcf98ab1d635484cb4e880f35ee24
2013-04-28 15:57:34 +00:00
umherirrender
b114f5e1c1 Fixed some spacing in maintenance folder
Added spaces before if, foreach
Added some braces for one line statements

Change-Id: I9657f72996358f8c1c154cea1ea97970d973723c
2013-04-18 20:48:44 +02:00
Reedy
54d6d29f88 Output special page name before attempting to run function
Related to bug 42210, we don't have a blank line, not knowing what it's actually running

QueryPage updates do this already!

Change-Id: I18eb0fb4c316744d92a81f5e64e201508b27eb27
2012-11-16 23:20:58 +00:00
Alexandre Emsenhuber
2a7478b4fb Improve documentation of maintenance scripts.
Change-Id: Id7a04ff816dc47a8cc81a4da5ab0dff26b688bd5
2012-09-03 20:10:09 +02:00
jeroendedauw
38c7f444e1 Use __DIR__ instead of dirname( __FILE__ )
We can now do this since we finally switched to PHP 5.3 for MW 1.20 and get rid of the silly dirname(__FILE__) stuff :)

Change-Id: Id9b2c9cd2e678197aa81c78adced5d1d31ff57b1
2012-08-27 21:45:00 +02:00
Alexandre Emsenhuber
087e7a2c3c Pass __METHOD__ to DatabaseBase::commit() and DatabaseBase::rollback() 2012-02-28 18:41:36 +00:00
Chad Horohoe
f1cd059859 Bunch of error suppression operator fixes (bug 2011-07-06 21:57:44 +00:00
Platonides
62053ac8fd Merge r86398 into the old wfWaitForSlaves. Update core calls.
maintenance/waitForSlave.php still calls the old prototype.
Most instances were replaced with 
sed -i 's/wfWaitForSlaves( [0-9]* );/wfWaitForSlaves();/' ./maintenance/initEditCount.php ./maintenance/updateRestrictions.php ./maintenance/updateSpecialPages.php ./maintenance/importDump.php ./maintenance/moveBatch.php ./maintenance/storage/resolveStubs.php ./maintenance/storage/trackBlobs.php ./maintenance/storage/fixBug20757.php ./maintenance/storage/moveToExternal.php ./maintenance/storage/compressOld.inc ./maintenance/populateSha1.php ./maintenance/deleteDefaultMessages.php ./maintenance/migrateUserGroup.php ./maintenance/importImages.php ./maintenance/runJobs.php ./maintenance/archives/upgradeLogging.php ./maintenance/deleteBatch.php ./maintenance/populateLogSearch.php ./maintenance/populateLogUsertext.php ./maintenance/gearman/gearmanWorker.php ./maintenance/populateRevisionLength.php ./maintenance/refreshLinks.php ./maintenance/deleteSelfExternals.php ./maintenance/upgrade1_5.php ./maintenance/rebuildFileCache.php ./includes/job/RefreshLinksJob.php ./includes/installer/MysqlUpdater.php ./maintenance/convertUserOptions.php ./maintenance/populateParentId.php ./maintenance/runBatchedQuery.php ./maintenance/upgrade1_5.php ./maintenance/waitForSlave.php ./maintenance/populateCategory.php ./maintenance/importImages.php
2011-04-20 00:12:06 +00:00
Sam Reed
517b6b6b59 Revert r86236 2011-04-18 19:00:48 +00:00
Sam Reed
9ffa645af8 Update all core calls of SpecialPage->getName(); 2011-04-18 17:39:07 +00:00
Happy-melon
2c9cfd7cce Refactor the factory/i18n/list/etc static methods from SpecialPage into their own class; there's no reason we need to be parsing them in every single SpecialPage subclass. Leave all the methods as stubs in SpecialPage.php; if we required PHP 5.3 they could be replaced by a a __callStatic() magic method, but that doesn't work on PHP 5.2.
Also make a few changes to the functions available.  SpecialPageFactory::resolveAlias() now takes an optional subpage and returns array(<name>,<subpage>).  Similarly merge getPage() and getPageByAlias().  There were many examples of (extensions particularly) making dubious assumptions about the presence or absence of subpages or canonical-ness.

I didn't deprecate SpecialPage::getTitleFor() as it's got over six hundred calls.  I'm rather undecided on the best position of getPage()/executePath().  Although the latter needs cleanup anyway.
2011-04-17 11:31:11 +00:00