Commit graph

49 commits

Author SHA1 Message Date
Bryan Davis
be3c297852 initEditCount: Join from user to actor to revision
Correct the join conditions used to find the current edit count for
a batch of users. Follows up b98f33c where the incorrect join order and
conditions were introduced as part of migration to QueryBuilder syntax.

Change-Id: I1841bb074b0f2be31e60a3cce3d62eb1b5999ad0
(cherry picked from commit 834891e01afa19cc1ecd973c04a453a9abb2d484)
2025-03-03 15:32:50 +00:00
Dreamy Jazz
e7393b3cc7 Exclude boilerplate maintenance code from code coverage reports
Why:
* Maintenance scripts in core have bolierplate code that is
  added before and after the class to allow directly running
  the maintenance script.
* Running the maintenance script directly has been deprecated
  since 1.40, so this boilerplate code is only to support a now
  deprecated method of running maintenance scripts.
* This code cannot also be marked as covered, due to PHPUnit
  not recognising code coverage for files.
* Therefore, it is best to ignore this boilerplate code in code
  coverage reports as it cannot be marked as covered and also
  is for deprecated code.

What:
* Wrap the boilerplate code (requiring Maintenance.php and then
  later defining the maintenance script class and running if the
  maintenance script was called directly) with @codeCoverageIgnore
  comments.
* Some files use a different boilerplate code, however, these
  should also be marked as ignored for coverage for the same
  reason that coverage is not properly reported for files.

Bug: T371167
Change-Id: I32f5c6362dfb354149a48ce9c28da9a7fc494f7c
2024-08-27 13:22:29 +01:00
Umherirrender
81c6df6a46 maintenance: Use expression builder instead of raw sql
Bug: T361023
Change-Id: Ieb229d8088cb1ff3f03e44f7ac99eb612f48bc7b
2024-07-22 22:29:20 +02:00
Umherirrender
bcd5fbcf98 maintenance: Fix missing where in initEditCount.php
The raw query does not include a where and it was missing when converted
to the QueryBuilder. For this case the IDatabase::ALL_ROWS constant
exists.

Bug: T367659
Follow-Up: Ie7dbd639c51abe21851ed3edfcd74fa3dfc192c2
Change-Id: I02b4b6d0f00020421b45f76ca7e7dbd7375716eb
2024-06-16 20:03:25 +00:00
Bartosz Dziewoński
babf311ea1 Use RawSQLValue/RawSQLExpression for some subqueries
Change-Id: Ie7dbd639c51abe21851ed3edfcd74fa3dfc192c2
2024-05-29 19:48:23 +00:00
Amir Sarabadani
d9370003fb maintenance: Introduce getReplicaDB() and getPrimaryDB()
And start using them instead of wfGetDB(), LB/LBF connection methods or
worse, $this->getDB().

$this->getDB() reuses the database object regardless of whether you're
calling a replica or primary, leading to returning a replica on a
primary and other way around.

Bug: T330641
Change-Id: I9e2cf85ca277022284fc26b9f37db57bd12aaa81
2024-01-18 15:12:04 +01:00
Reedy
b98f33cdac Convert numerous DB queries to use QueryBuilders
Bug: T344971
Change-Id: Ia727b513a6bfcaa5a0b13977a6789aa879ad2f0b
2023-10-09 19:06:53 +02:00
Derick Alangi
74033c50cd maintenance: Begin using Maintenance::getServiceContainer()
Maintenance class provides a method for getting a fresh reference
of the MW services container instance. Let's make use of these in
maintenance scripts now that we have it.

NOTE: There are still some static methods like in refreshLinks.php
that makes use of services that we can't use this method for now.

Change-Id: Idba744057577896fc97c9ecf4724db27542bf01c
2023-09-04 10:39:58 +00:00
Amir Sarabadani
310333906a maintenance: Switch simple calls of Database::select to SQB
Done semi-automatically via a php parser written on top of ANTLR4.

Bug: T311866
Change-Id: I33f5b6703c0aa9c80c907a21c2a770e30642edd3
2023-07-19 17:42:23 +02:00
Amir Sarabadani
4bb2886562 Reorg: Migrate WikiMap to WikiMap/ out of includes
And WikiReference

Bug: T321882
Change-Id: I60cf4b9ef02b9d58118caa39172677ddfe03d787
2023-02-27 05:19:46 +01:00
Amir Sarabadani
cb18d1007e Reorg: Move ActorMigration and ActorMigrationBase to user/
This may seem a bit weird but anything else related to actor is already
under user/ including ActorCache, ActorNormalization, ActorStore and
ActorStoreFactory.

Bug: T321882
Change-Id: I7072b374bba7a0cd9d905e399c822bf30bd5c0d8
2022-12-16 13:44:33 +01:00
Amir Sarabadani
b525884e11 maintenance: Use $this->waitForReplication()
This adds reconfiguring db pools in case a replica gets depooled

Bug: T298485
Change-Id: Id052ce8ed45c51e51b071778858d27b48605bf93
2022-10-24 21:11:53 +02:00
Amir Sarabadani
6b1d6e622d rdbms: Drop deprecated functions from LoadBalancer
- ::haveIndex()
  - ::isNonZeroLoad()

Deprecated in 1.34 and unused.

Change-Id: I08f8d8cad51ab1ec3c385c06737116d7ab71f2ca
2022-09-29 17:10:59 +00:00
James D. Forrester
df5eb22f83 Replace uses of DB_MASTER with DB_PRIMARY
Just an auto-replace from codesniffer for now.

Change-Id: I5240dc9ac5929d291b0ef1c743ea2bfd3f428266
2021-04-29 09:24:31 -07:00
Reedy
8ba1c75559 Replace wfWaitForSlaves() with LBFactory::waitForReplication()
Change-Id: I337147d61e2ec686a8672d0340dff4b6783f78cd
2020-05-02 02:00:01 +00:00
Ammar Abdulhamid
3bdd39de48 initEditCount.php: Fix text flow in param description
Bug: T247911
Change-Id: Ia6d446aeb2edbe25cad8b06b3372dac858c211c3
2020-03-23 22:59:07 +00:00
Aaron Schulz
7710afe569 Migrate various callers away from wfWikiId() to WikiMap
Change-Id: Idc2980c05873c428fb5ffa6ab89f723d33217a8e
2019-07-12 01:02:00 +00:00
Brad Jorsch
993baa3493 ActorMigration: Remove possibility of read-both
When this was originally written, the plan was to read both the old and
new fields during the transition period, while stopping writes to them
midway through. It turns out that the WHERE conditions to do read-both
correctly are generally not handled well by the database and working
around that would require a lot of complicated code (see what's being
removed from ApiQueryUserContribs here, for example).

We can simplify things greatly by instead having it write both fields
during the transition period, reading from the old for the first part
and the new for the second part, as is being done for MCR.

Bug: T204669
Change-Id: I4764c1c7883dc1003cb12729455c8107319f70b1
Depends-On: I845f6ae462f2539ebd35cbb5f2ca8b5714e2c1fb
Depends-On: I88b31b977543fdbdf69f8c1158e77e448df94e11
2018-10-11 12:12:00 +11:00
Umherirrender
e1a203603c Replace wfGetLB
@deprecated since 1.27

Change-Id: Ibdd49fdfc0d1511503e1ed2173a592c612996c53
2018-05-02 22:30:24 +02:00
Brad Jorsch
27c61fb1e9 Add actor table and code to start using it
Storing the user name or IP in every row in large tables like revision
and logging takes up space and makes operations on these tables slower.
This patch begins the process of moving those into one "actor" table
which other tables can reference with a single integer field.

A subsequent patch will remove the old columns.

Bug: T167246
Depends-On: I9293fd6e0f958d87e52965de925046f1bb8f8a50
Change-Id: I8d825eb02c69cc66d90bd41325133fd3f99f0226
2018-02-23 10:06:20 -08:00
Umherirrender
ad776c7d5f Use ::class to resolve class names in maintenance scripts
This helps to find renamed or misspelled classes earlier.
Phan will check the class names

Change-Id: I1d4567f47f93eb1436cb98558388e48d35258666
2018-01-23 17:40:16 +00:00
Max Semenik
f0c8cbc6d9 Remove a few unused variables
Change-Id: Ibfc4c6cbbc08b5917f1a84d86d2d4a0855e371a1
2016-09-26 17:03:32 -07:00
Aaron Schulz
950cf6016c Rename DB_SLAVE constant to DB_REPLICA
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.

The old constant is an alias now.

Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
2016-09-05 22:55:53 -07:00
Aaron Schulz
c0a9ab0f6d Change "slave" => "replica DB" in /maintenance
Change-Id: Ibd3d617901130378a935402326cd4eefbb382c9e
2016-09-06 00:13:08 +00:00
Kevin Israel
c29dc2b65f Remove checks formerly for MySQL server version
Removed these checks instead of fixing them, for the stated reasons:

* initEditCount.php: "instanceof DatabaseMysql" is not a correct way to
  check if the database server is MySQL/MariaDB, because DatabaseMysql is
  now a subclass of DatabaseMysqlBase, only used as a wrapper around the
  deprecated mysql PHP extension (not the newer mysqli extension). The
  check was intended for pre-4.1 versions of MySQL, which do not support
  subqueries.

  The effect of this change is to use single-query mode whenever there is
  only one configured database server, even when using the mysqli PHP
  extension, which would have been the original intent.

* fixBug20757.php: If the database server is not MySQL/MariaDB, the
  $lowerLeft variable would remain undefined. The script was only intended
  for use with MySQL, so the check was only necessary in order to support
  pre-4.1 versions of MySQL, which require different SQL.

Follows-up b74f88967b.

Change-Id: I7f32aed4473e5ea39dc40449ddc0af5f9a10df16
2016-08-11 04:55:06 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Max Semenik
59db24e90b Use addDescription() instead of accessing mDescription directly
Change-Id: I0e2aa83024b8abf5298cfea4b21bf45722ad3103
2016-01-30 01:28:32 -08:00
Reedy
44cebea941 Update wfGetDB calls in Maintenance scripts to use getDB()
Change-Id: I9ad6745d84506b736dae94747256caac89715899
2016-01-02 16:58:23 +00:00
tonythomas01
b74f88967b Remove version check for mysql 4.1 from Maintenance scripts
Removed version check for mysql from initEditCount.php and
storage/fixBug20757.php. Mediawiki needs MYSQL Ver 5.0.2 or
later, so the check is unnecessary.
Removed white spaces

Bug: 59126
Change-Id: I29e2ca8764fad8bf4d15bba307e80f3aacea1db7
2013-12-31 14:53:07 +05:30
Timo Tijhof
beb1c4a0ec phpcs: More require/include is not a function
Follows-up I1343872de7, Ia533aedf63 and I2df2f80b81.

Also updated usage in text in documentation and the
installer LocalSettingsGenerator.

Most of them were handled by this regex:
- find: (require|include|require_once|include_once)\s*\(\s*(.+?)\s*\)\s*;$
- replace: $1 $2;

Change-Id: I6b38aad9a5149c9c43ce18bd8edbab14b8ce43fa
2013-05-21 23:26:28 +02:00
Timo Tijhof
50e7985d4d phpcs: Fix WhiteSpace.LanguageConstructSpacing warnings
Squiz.WhiteSpace.LanguageConstructSpacing:
   Language constructs must be followed by a single space;
   expected "require_once expression" but found
   "require_once(expression)"

It is a keyword (e.g. like `new`, `return` and `print`). As
such the parentheses don't make sense.

Per our code conventions, we use a space after keywords like
these. We appeared to have an unwritten exception for `require`
that doesn't make sense. About 60% of require/include usage
was missing the space and/or had superfluous parentheses.

It is as silly as print("foo") or return("foo"), it works
because keywords have no significance for whitespace between
it and the expression that follows, and since experessions can
be wrapped in parentheses for clarity (e.g. when doing string
concatenation or mathematical operations) the parenthesis
before and after basiclaly just ignored.

Change-Id: I2df2f80b8123714bea7e0771bf94b51ad5bb4b87
2013-05-09 05:56:26 +02:00
jeroendedauw
38c7f444e1 Use __DIR__ instead of dirname( __FILE__ )
We can now do this since we finally switched to PHP 5.3 for MW 1.20 and get rid of the silly dirname(__FILE__) stuff :)

Change-Id: Id9b2c9cd2e678197aa81c78adced5d1d31ff57b1
2012-08-27 21:45:00 +02:00
Platonides
62053ac8fd Merge r86398 into the old wfWaitForSlaves. Update core calls.
maintenance/waitForSlave.php still calls the old prototype.
Most instances were replaced with 
sed -i 's/wfWaitForSlaves( [0-9]* );/wfWaitForSlaves();/' ./maintenance/initEditCount.php ./maintenance/updateRestrictions.php ./maintenance/updateSpecialPages.php ./maintenance/importDump.php ./maintenance/moveBatch.php ./maintenance/storage/resolveStubs.php ./maintenance/storage/trackBlobs.php ./maintenance/storage/fixBug20757.php ./maintenance/storage/moveToExternal.php ./maintenance/storage/compressOld.inc ./maintenance/populateSha1.php ./maintenance/deleteDefaultMessages.php ./maintenance/migrateUserGroup.php ./maintenance/importImages.php ./maintenance/runJobs.php ./maintenance/archives/upgradeLogging.php ./maintenance/deleteBatch.php ./maintenance/populateLogSearch.php ./maintenance/populateLogUsertext.php ./maintenance/gearman/gearmanWorker.php ./maintenance/populateRevisionLength.php ./maintenance/refreshLinks.php ./maintenance/deleteSelfExternals.php ./maintenance/upgrade1_5.php ./maintenance/rebuildFileCache.php ./includes/job/RefreshLinksJob.php ./includes/installer/MysqlUpdater.php ./maintenance/convertUserOptions.php ./maintenance/populateParentId.php ./maintenance/runBatchedQuery.php ./maintenance/upgrade1_5.php ./maintenance/waitForSlave.php ./maintenance/populateCategory.php ./maintenance/importImages.php
2011-04-20 00:12:06 +00:00
Chad Horohoe
26505b170a Fix concern raised by Brion in r74108 (but has really existed since the maintenance rewrite). Right now, including a maintenance script causes it to execute. This is bad when you want to reuse the particular class but not have it start executing all by itself.
Until now, we relied on setting MW_NO_SETUP which was a) hacky, b) irreversable, and c) likely to be forgotten if you didn't use one of the wrappers like runChild().

Instead, move the freaky magic to doMaintenance and have *it* check if it's in a specific call stack that indicates this is being run from the file scope and should be executed. Rename DO_MAINTENANCE to RUN_MAINTENANCE_IF_MAIN so it's nice and clear what magic happens behind the require_once().
2011-01-13 22:58:55 +00:00
Mark A. Hershberger
617a5b1e15 Whitespace fixup under tha maint directory. 2010-12-04 03:20:14 +00:00
Alexandre Emsenhuber
540efd5e6c Use the load balancer to get the server count, not $wgServers 2010-10-10 11:39:08 +00:00
Alexandre Emsenhuber
a601380355 Added a description and @file 2010-10-10 11:27:07 +00:00
Chad Horohoe
4c74490bb8 Big commit: kill almost every freeResult() call as useless 2010-08-08 12:27:48 +00:00
Platonides
1af741ee27 Use the canonical names for classes. 2010-07-27 18:33:04 +00:00
Sam Reed
659778619c Stylize maintenance folder.. 2010-05-22 16:50:39 +00:00
Alexandre Emsenhuber
44729064cd * batch change for "while ( $row = $db->fetchObject( $res ) )" and similar to "foreach ( $res as $row )"
* identation fix in nukeNS.php
2009-08-17 21:15:31 +00:00
Alexandre Emsenhuber
2ed25cf8d3 Changed $wgDBname to wfWikiID() (used for display) to be consistent with other maintenance scripts, also avoid an E_NOTICE since it wasn't declared as global :) 2009-08-15 12:45:15 +00:00
Chad Horohoe
567f244e36 Revert r54244 which was stupid and fix this properly. Require commandLine.inc/Maintenance.php using the full path every time. 2009-08-03 21:56:41 +00:00
Chad Horohoe
5ff7ffe312 * Add way to get full input from stdin() without having to check the length
* Missing global
2009-08-02 21:42:47 +00:00
Chad Horohoe
a1c51e18af Merge maintenance-work branch (now with less errors!):
* Docs have been updated to indicate the standard on how to write maintenance scripts (MW.org docs will follow) Have ported vast majority of maintenance scripts to new format. Remaining ones (mostly FiveUpgrade-related) are a bit more tricky. commandLine.inc is untouched for now. Many have gotten code-style updates as well. Deleted .inc files were only used by their .php counterparts, and have been merged into single files.
* (bug 11867) Lock error on redirect table when running orphans.php
* (bug 16322) Allow maintenance scripts to accept DB user/pass over input or params
* (bug 18566) Maintenance script to un/protect pages
* initStats overhaul, now uses class SiteStatsInit. Also fixes bug 18930
2009-08-02 19:35:17 +00:00
Alexandre Emsenhuber
087a9f70c5 WARNING: HUGE COMMIT
Doxygen documentation update:
* Changed alls @addtogroup to @ingroup. @addtogroup adds the comment to the group description, but doesn't add the file, class, function, ... to the group like @ingroup does. See for example http://svn.wikimedia.org/doc/group__SpecialPage.html where it's impossible to see related files, classes, ... that should belong to that group.
* Added @file to file description, it seems that it should be explicitely decalred for file descriptions, otherwise doxygen will think that the comment document the first class, variabled, function, ... that is in that file.
* Removed some empty comments
* Removed some ?>

Added following groups:
* ExternalStorage
* JobQueue
* MaintenanceLanguage

One more thing: there are still a lot of warnings when generating the doc.
2008-05-20 17:13:28 +00:00
Aryeh Gregor
a15c419b3d Remove ?>'s from files. They're pointless, and just asking for people to mess with the files and add trailing whitespace. (Yes, I looked over every one and reverted those that were bogus. Slash-enter a million times in less worked well enough, although it was a bit mind-numbing.) 2007-06-29 01:19:14 +00:00
Brion Vibber
a8f204894f Add replication-friendly background batch update mode.
Rip out the ugly MySQL 4.0 mode and use the background mode for it, it should be compatible too.
Add a little help text
2006-12-15 00:10:33 +00:00
Brion Vibber
288a7eb7f1 Going ahead and adding this field while other DB updates are pending.
Interfaces to use it can be added shortly.

* Add user_editcount field to provide data for heuristics on account use.
  Incremented on edit, with lazy initialization from past revision data.
  Can batch-initialize with maintenance/initEditCount.php (not yet friendly
  to replication environments, this will do all accounts in one query).
* Allow raw SQL subsections in Database::update() SET portion as well as
  for WHERE portion. Handy for increments and such.
2006-12-14 13:22:52 +00:00