Commit graph

34 commits

Author SHA1 Message Date
Dreamy Jazz
4a6de09299 Fix broken deleteSelfExternals.php and test it
Why:
* The deleteSelfExternals.php maintenance script is broken on
  my local wiki as the SQL generated by the script errors out
  when run on MariaDB.
* This script should be fixed, updated to use the query builders,
  and tested to ensure that any future breakages do not go
  unnoticed.

What:
* Fix the deleteSelfExternals.php script to use a DELETE query
  that works, by generating it using the query builder.
* Create DeleteSelfExternalsTest to test that the script now
  works.

Bug: T371167
Change-Id: Id423fa5c1bfe666f57a38e733334342ac1ad2cea
2024-09-11 18:23:40 +00:00
Dreamy Jazz
e7393b3cc7 Exclude boilerplate maintenance code from code coverage reports
Why:
* Maintenance scripts in core have bolierplate code that is
  added before and after the class to allow directly running
  the maintenance script.
* Running the maintenance script directly has been deprecated
  since 1.40, so this boilerplate code is only to support a now
  deprecated method of running maintenance scripts.
* This code cannot also be marked as covered, due to PHPUnit
  not recognising code coverage for files.
* Therefore, it is best to ignore this boilerplate code in code
  coverage reports as it cannot be marked as covered and also
  is for deprecated code.

What:
* Wrap the boilerplate code (requiring Maintenance.php and then
  later defining the maintenance script class and running if the
  maintenance script was called directly) with @codeCoverageIgnore
  comments.
* Some files use a different boilerplate code, however, these
  should also be marked as ignored for coverage for the same
  reason that coverage is not properly reported for files.

Bug: T371167
Change-Id: I32f5c6362dfb354149a48ce9c28da9a7fc494f7c
2024-08-27 13:22:29 +01:00
James D. Forrester
54a934e24d maintenance: Use getServiceContainer() when in instances of Maintenance
Change-Id: I00caa744deb805f4156eebee92068025f2aea2a4
2024-02-13 19:25:23 +00:00
Amir Sarabadani
d9370003fb maintenance: Introduce getReplicaDB() and getPrimaryDB()
And start using them instead of wfGetDB(), LB/LBF connection methods or
worse, $this->getDB().

$this->getDB() reuses the database object regardless of whether you're
calling a replica or primary, leading to returning a replica on a
primary and other way around.

Bug: T330641
Change-Id: I9e2cf85ca277022284fc26b9f37db57bd12aaa81
2024-01-18 15:12:04 +01:00
Dogu
54b1731f88 Replace deprecated wfParseUrl with UrlUtils::parse
The wfParseUrl function is deprecated as of MediaWiki 1.39 and has been
replaced with the UrlUtils::parse method provided by the UrlUtils class.

List of affected classes:
- deleteSelfExternals
- UserMailer

Change-Id: I5e36ee80e5c30e95b79bf45e7b26860cb2668d56
2024-01-07 10:24:37 +00:00
Amir Sarabadani
22cec534c5 Reorg: Move LinkFilter to ExternalLinks
It's one-class namespace and I know it's not great but:
 - I hope to add more classes with the redesign of externallinks table
 - It's not named very well either, it's a collection of URL-related
   functionalities
 - Making it clear LinkFilter is about external links, not internal or
   interwiki or templatelinks etc.

Bug: T321882
Change-Id: I0dd530237f45e4fec786178ec03ee941c6bcd982
2023-03-01 22:08:29 +01:00
Ferran Tufan
cd99df60ed Maintenance scripts: replace most globals with Config
Bug: T72673
Change-Id: Idc46e88756b1aa20f5dccbe7ab3e661b2b102964
2022-07-12 21:13:19 +02:00
Umherirrender
94792e97be maintenance: Cleanup use of exit in Maintenance::execute implementation
Returning void from execute() is success.
Returning true is success, false is failure with exit(1)
Using fatalError also using exit(1)

Change-Id: I1d40430ad6226e4aab8f0810b03ee1213282d123
2021-08-30 19:48:27 +00:00
James D. Forrester
df5eb22f83 Replace uses of DB_MASTER with DB_PRIMARY
Just an auto-replace from codesniffer for now.

Change-Id: I5240dc9ac5929d291b0ef1c743ea2bfd3f428266
2021-04-29 09:24:31 -07:00
Umherirrender
5a318bd5bf Pass function name to database functions (maintenance scripts)
Useful for logging

Change-Id: I79fe037abcd74f56c935abc118d706bef0198124
2020-06-07 17:24:10 +00:00
Brad Jorsch
d65e96b763 Use new externallinks.el_index_60 field
This adds a method to LinkFilter to build the query conditions necessary
to properly use it, and adjusts code to use it.

This also takes the opportunity to clean up the calculation of el_index:
IPs are handled more sensibly and IDNs are canonicalized.

Also weird edge cases for invalid hosts like "http://.example.com" and
corresponding searches like "http://*..example.com" are now handled more
regularly instead of being treated as if the extra dot were omitted,
while explicit specification of the DNS root like "http://example.com./"
is canonicalized to the usual implicit specification.

Note that this patch will break link searches for links where the host
is an IP or IDN until refreshExternallinksIndex.php is run.

Bug: T59176
Bug: T130482
Change-Id: I84d224ef23de22dfe179009ec3a11fd0e4b5f56d
2018-11-12 22:33:18 +00:00
Umherirrender
bf5a104146 Remove wfWaitForSlaves when using Maintenance::commitTransaction
Maintenance::commitTransaction is calling waitForReplication already.
No need to wait a second time, hopefully the lags are 0 already.

Change-Id: Id457ed2cdd6bfd9663665ba0cd5c4e3dd640b738
2018-02-26 21:05:33 +01:00
Umherirrender
ad776c7d5f Use ::class to resolve class names in maintenance scripts
This helps to find renamed or misspelled classes earlier.
Phan will check the class names

Change-Id: I1d4567f47f93eb1436cb98558388e48d35258666
2018-01-23 17:40:16 +00:00
Max Semenik
dac20d0ffa Introduce Maintenance::getBatchSize()
Just to isolate the internals. Fix most of usages in the core.

Change-Id: I8b3e9ca1f42b7c49ee57f17b88ca2fc7b404f342
2017-11-05 13:26:12 -08:00
Max Semenik
5808276de3 Use Maintenance::setBatchSize() everywhere
Change-Id: Id48b95c547ddd8ea02fb1c577203f17056f38402
2017-11-04 16:10:06 -07:00
Max Semenik
59db24e90b Use addDescription() instead of accessing mDescription directly
Change-Id: I0e2aa83024b8abf5298cfea4b21bf45722ad3103
2016-01-30 01:28:32 -08:00
Reedy
44cebea941 Update wfGetDB calls in Maintenance scripts to use getDB()
Change-Id: I9ad6745d84506b736dae94747256caac89715899
2016-01-02 16:58:23 +00:00
Aaron Schulz
fa8e1a9b00 Clean up transactions in maintenance scripts
Add transaction methods to complement getDB().
This makes it easy to grep for direct begin()/commit()
calls to IDatabase by having script use their own
wrapper. Maintenance scripts are one of the few places
that can (and need to) use begin/commit instead of the
start/end atomic methods.

Eventually, there should be almost no direct callers
and those methods can be made stricter about throwing
errors on nested calls.

Change-Id: Ibbfc7a77c0d2a55f7fc2261087f6c3a19061e0aa
2015-12-30 23:40:35 +00:00
Timo Tijhof
beb1c4a0ec phpcs: More require/include is not a function
Follows-up I1343872de7, Ia533aedf63 and I2df2f80b81.

Also updated usage in text in documentation and the
installer LocalSettingsGenerator.

Most of them were handled by this regex:
- find: (require|include|require_once|include_once)\s*\(\s*(.+?)\s*\)\s*;$
- replace: $1 $2;

Change-Id: I6b38aad9a5149c9c43ce18bd8edbab14b8ce43fa
2013-05-21 23:26:28 +02:00
Timo Tijhof
50e7985d4d phpcs: Fix WhiteSpace.LanguageConstructSpacing warnings
Squiz.WhiteSpace.LanguageConstructSpacing:
   Language constructs must be followed by a single space;
   expected "require_once expression" but found
   "require_once(expression)"

It is a keyword (e.g. like `new`, `return` and `print`). As
such the parentheses don't make sense.

Per our code conventions, we use a space after keywords like
these. We appeared to have an unwritten exception for `require`
that doesn't make sense. About 60% of require/include usage
was missing the space and/or had superfluous parentheses.

It is as silly as print("foo") or return("foo"), it works
because keywords have no significance for whitespace between
it and the expression that follows, and since experessions can
be wrapped in parentheses for clarity (e.g. when doing string
concatenation or mathematical operations) the parenthesis
before and after basiclaly just ignored.

Change-Id: I2df2f80b8123714bea7e0771bf94b51ad5bb4b87
2013-05-09 05:56:26 +02:00
umherirrender
b114f5e1c1 Fixed some spacing in maintenance folder
Added spaces before if, foreach
Added some braces for one line statements

Change-Id: I9657f72996358f8c1c154cea1ea97970d973723c
2013-04-18 20:48:44 +02:00
jeroendedauw
38c7f444e1 Use __DIR__ instead of dirname( __FILE__ )
We can now do this since we finally switched to PHP 5.3 for MW 1.20 and get rid of the silly dirname(__FILE__) stuff :)

Change-Id: Id9b2c9cd2e678197aa81c78adced5d1d31ff57b1
2012-08-27 21:45:00 +02:00
Alexandre Emsenhuber
a28e00cb4b Improve documentation of maintenance scripts.
Change-Id: I2433d23544e808e16f28805f93183b1af2409c94
2012-07-08 22:50:22 +02:00
Alexandre Emsenhuber
087e7a2c3c Pass __METHOD__ to DatabaseBase::commit() and DatabaseBase::rollback() 2012-02-28 18:41:36 +00:00
Platonides
62053ac8fd Merge r86398 into the old wfWaitForSlaves. Update core calls.
maintenance/waitForSlave.php still calls the old prototype.
Most instances were replaced with 
sed -i 's/wfWaitForSlaves( [0-9]* );/wfWaitForSlaves();/' ./maintenance/initEditCount.php ./maintenance/updateRestrictions.php ./maintenance/updateSpecialPages.php ./maintenance/importDump.php ./maintenance/moveBatch.php ./maintenance/storage/resolveStubs.php ./maintenance/storage/trackBlobs.php ./maintenance/storage/fixBug20757.php ./maintenance/storage/moveToExternal.php ./maintenance/storage/compressOld.inc ./maintenance/populateSha1.php ./maintenance/deleteDefaultMessages.php ./maintenance/migrateUserGroup.php ./maintenance/importImages.php ./maintenance/runJobs.php ./maintenance/archives/upgradeLogging.php ./maintenance/deleteBatch.php ./maintenance/populateLogSearch.php ./maintenance/populateLogUsertext.php ./maintenance/gearman/gearmanWorker.php ./maintenance/populateRevisionLength.php ./maintenance/refreshLinks.php ./maintenance/deleteSelfExternals.php ./maintenance/upgrade1_5.php ./maintenance/rebuildFileCache.php ./includes/job/RefreshLinksJob.php ./includes/installer/MysqlUpdater.php ./maintenance/convertUserOptions.php ./maintenance/populateParentId.php ./maintenance/runBatchedQuery.php ./maintenance/upgrade1_5.php ./maintenance/waitForSlave.php ./maintenance/populateCategory.php ./maintenance/importImages.php
2011-04-20 00:12:06 +00:00
Chad Horohoe
26505b170a Fix concern raised by Brion in r74108 (but has really existed since the maintenance rewrite). Right now, including a maintenance script causes it to execute. This is bad when you want to reuse the particular class but not have it start executing all by itself.
Until now, we relied on setting MW_NO_SETUP which was a) hacky, b) irreversable, and c) likely to be forgotten if you didn't use one of the wrappers like runChild().

Instead, move the freaky magic to doMaintenance and have *it* check if it's in a specific call stack that indicates this is being run from the file scope and should be executed. Rename DO_MAINTENANCE to RUN_MAINTENANCE_IF_MAIN so it's nice and clear what magic happens behind the require_once().
2011-01-13 22:58:55 +00:00
Mark A. Hershberger
617a5b1e15 Whitespace fixup under tha maint directory. 2010-12-04 03:20:14 +00:00
Alexandre Emsenhuber
20072cfe4a exit -> return 2010-10-10 09:01:03 +00:00
Alexandre Emsenhuber
66234904b8 Use full path to Maintenance.php 2010-10-10 08:35:07 +00:00
Sam Reed
659778619c Stylize maintenance folder.. 2010-05-22 16:50:39 +00:00
Chad Horohoe
acd9756a1f Change deleteSelfExternals to use Maintenance.php 2009-12-04 14:57:21 +00:00
Max Semenik
ae57ab1eec (bug 20275) Fixed LIKE queries on SQLite backend
* All manually built LIKE queries in the core are replaced with a wrapper function Database::buildLike()
* This function automatically performs all escaping, so Database::escapeLike() is now almost never used
2009-10-21 19:53:03 +00:00
Brion Vibber
deb41089e1 cleanup r56216 deleteSelfExternals a little - fix short PHP tag, use escaping for good measure (though in most cases we'll be clean) 2009-09-12 00:05:04 +00:00
Brion Vibber
7326fa6cb4 Merge deleteSelfExternals.php handy pruning script from wmf-deployment r53856 2009-09-12 00:02:57 +00:00