Commit graph

151 commits

Author SHA1 Message Date
James D. Forrester
9e5c1e8ac7 Add namespace to IDBAccessObject and DBAccessObjectUtils
Bug: T353458
Change-Id: I23cf7991f8792d4d000d1780463d8ce76dc0aee0
2024-09-27 16:19:10 -04:00
Pppery
d35741b403 NamespaceDupes: check if page exists from primary database
Bug: T374631
Change-Id: Ia3480d40bd7696726cf05cec3d873d2cb4e2e255
2024-09-12 11:39:39 -04:00
Dreamy Jazz
e7393b3cc7 Exclude boilerplate maintenance code from code coverage reports
Why:
* Maintenance scripts in core have bolierplate code that is
  added before and after the class to allow directly running
  the maintenance script.
* Running the maintenance script directly has been deprecated
  since 1.40, so this boilerplate code is only to support a now
  deprecated method of running maintenance scripts.
* This code cannot also be marked as covered, due to PHPUnit
  not recognising code coverage for files.
* Therefore, it is best to ignore this boilerplate code in code
  coverage reports as it cannot be marked as covered and also
  is for deprecated code.

What:
* Wrap the boilerplate code (requiring Maintenance.php and then
  later defining the maintenance script class and running if the
  maintenance script was called directly) with @codeCoverageIgnore
  comments.
* Some files use a different boilerplate code, however, these
  should also be marked as ignored for coverage for the same
  reason that coverage is not properly reported for files.

Bug: T371167
Change-Id: I32f5c6362dfb354149a48ce9c28da9a7fc494f7c
2024-08-27 13:22:29 +01:00
Pppery
4997e2e74e NamespaceDupes: treat Special and Media namespaces like interwikis
Bug: T18839
Change-Id: I4d87c77fbadd23f6bd73468dd43319ff05d1bd0b
2024-07-11 22:27:05 +00:00
Umherirrender
1364f24a5c maintenance: Check PageLinksSchemaMigrationStage in namespaceDupes.php
Additional reverts 906b7757b6

Bug: T364546
Change-Id: Ib019e9e631b374dd9584fccc2566bd24545821b2
2024-05-29 22:21:26 +00:00
Umherirrender
420e071271 Remove TemplateLinksSchemaMigrationStage config
Bug: T299417
Follow-Up: I906e069a63d1dae14924c72318b22b16244371d6
Change-Id: Ia5f730af82f904bd42552e6c6a5c4dadf2454d3a
2024-05-15 22:27:44 +02:00
Amir Sarabadani
906b7757b6 Disable namespaceDupes again
Bug: T364546
Change-Id: I09bc73f30ca4d7bb3855a2f0b8c1dcf00e102bf5
2024-05-09 16:21:53 +02:00
jenkins-bot
43876fb2fa Merge "namespaceDupes: Fix fatals caused by READ_NEW and WRITE_BOTH" 2024-02-10 21:36:07 +00:00
Brooke Vibber
dcd9c3ae26 Update name & email for bvibber
Updating name & email addresses for Brooke Vibber.

Re-ran updateCredits.php as well so there are some new entries in
there as well.

There are a couple of files in resources/libs that will have to
be changed upstream to keep tests happy, I will do patches
later. :D

Change-Id: I2f2e75d3fa42e8cf6de19a8fbb615bac28efcd54
2024-02-08 17:02:16 -08:00
Amir Sarabadani
bfc64fabf4 namespaceDupes: Fix fatals caused by READ_NEW and WRITE_BOTH
If it's WRITE_BOTH, means the old fields also needs updating but since
it's READ_NEW, the fields are set to linktarget fields. That is wrong.

Bug: T350431
Change-Id: I6512d8b119861ff7e6016202e8c36da0536a9b26
2024-02-08 13:23:58 +01:00
Alexander Vorwerk
0c7eb5393a namespaceDupes: Reduce batchsize to 100 for link update
Change-Id: I3718041a97d6ba39d8953211b80f64af06c9da77
2024-01-31 23:58:17 +01:00
Umherirrender
cf30bf6984 maintenance: Acquire new linktarget in namespaceDupes.php
The use of LinksMigration::getLinksConditions returns 1=0 when there is
no lt_id in the linktarget.
Aquire a new id instead of a fatal sql error as update to 1=0 is invalid

Bug: T341993
Follow-up: I01c3a545248b06d1f73aa99dd898f60767482c8f
Change-Id: I264d366adb4588cbcfb14d52c4a56a05edeef8c1
2024-01-25 21:11:32 +01:00
Amir Sarabadani
014bc61006 Remove more indirect calls to IDBAccessObject::READ_* constants
Found via (?<!IDBAccessObject)::READ_

We are planning to deprecate and remove implementing IDBAccessObject
interface just to use the constants.

Bug: T354194
Change-Id: I89d442fa493b8e5332ce118e5bf13f13b8dd3477
2024-01-23 15:42:38 +01:00
Amir Sarabadani
d9370003fb maintenance: Introduce getReplicaDB() and getPrimaryDB()
And start using them instead of wfGetDB(), LB/LBF connection methods or
worse, $this->getDB().

$this->getDB() reuses the database object regardless of whether you're
calling a replica or primary, leading to returning a replica on a
primary and other way around.

Bug: T330641
Change-Id: I9e2cf85ca277022284fc26b9f37db57bd12aaa81
2024-01-18 15:12:04 +01:00
jenkins-bot
108d4f399d Merge "rdbms: Change docs for IDatabase::affectedRows on UPDATE IGNORE" 2023-12-04 20:40:40 +00:00
Zoranzoki21
93e9331d90 Revert "Disable namespaceDupes.php for now"
This reverts commit 8b1532d4dd.

Reason for revert: Because of recent changes to the script,
it should be safe to use it again.

Bug: T350443
Change-Id: I56593542bb0676792251d7e966b31be69be437bb
2023-11-23 17:32:09 +00:00
James D. Forrester
67217d08df Namespace remaining files under includes/deferred
Bug: T166010
Change-Id: Ibd40734b96fd2900e3ce12239d09becfb4150059
2023-11-22 10:08:53 -05:00
Umherirrender
c18785f57d rdbms: Change docs for IDatabase::affectedRows on UPDATE IGNORE
IDatabase::affectedRows does not return the affected rows for UPDATE
IGNORE as mention in the docs, it returns the matching rows and making
IDatabase::affectedRows not usable to determine if update has update the
row or not, reverting usage in namespaceDupes.php (01a548cc)

Change-Id: Ia3b7cd034731da9c7067a11a3bfa51c81a4c5530
2023-11-21 21:50:10 +01:00
Umherirrender
baa4ceea55 maintenance: Batch delete for key conflicts in namespaceDupes.php
Limit the deletion of key conflicts for the links table to
$wgUpdateRowsPerQuery rows.

Also use IDatabase::factorConds to build the condiiton

Bug: T350443
Change-Id: I64ec5a01b457a395a1e830c28aabbc2dd3c0f539
2023-11-21 20:32:50 +01:00
jenkins-bot
54a15ffef4 Merge "maintenance: Batch update for _from_namespace in namespaceDupes.php" 2023-11-20 02:50:18 +00:00
jenkins-bot
ca9f8418d9 Merge "maintenance: Reduce delete attempts in namespaceDupes.php" 2023-11-20 02:38:46 +00:00
Umherirrender
7f3fcea319 maintenance: Batch update for rev_page in namespaceDupes.php
Limit the update to $wgUpdateRowsPerQuery rows.
The update now is over the primary key.

Bug: T350443
Change-Id: I4286ea35115aab331eaecf71fb3daf03708d3d38
2023-11-18 23:14:51 +01:00
Umherirrender
ee7aa4d7c7 maintenance: Batch update for _from_namespace in namespaceDupes.php
Limit the update to $wgUpdateRowsPerQuery rows.
The update now is over the primary key.

Also select only the rows with different _from_namespace and needs
change.
There is always a _backlinks_namespace index with the _from_namespace
field and all primary key fields to support this.

Bug: T350443
Change-Id: I503da35f71902c5ed57c5244cb3833fc4730ada3
2023-11-18 22:45:41 +01:00
Umherirrender
01a548ccb5 maintenance: Reduce delete attempts in namespaceDupes.php
Bug: T350443
Change-Id: I9b233d348e7b01358f123bd0d6128cb801ac8627
2023-11-17 22:48:46 +01:00
Umherirrender
c31b09176d maintenance: Use QueryBuilder in namespaceDupes.php
Remove unused $db class property

Change-Id: I2f149485dcc4dd40b1d4fb0cf588b14bec4eecbb
2023-11-17 16:35:03 +01:00
Amir Sarabadani
ad118dbb75 maintenance: Migrate $db->buildLike() to expression builder
Bug: T210206
Change-Id: Ie7bf3701fa9d51a43167ce7ec0c1f30bc090296b
2023-11-06 14:27:03 +01:00
Tyler Cipriani
8b1532d4dd Disable namespaceDupes.php for now
namespaceDupes.php has lacks limits on delete and update queries. For
large updates and deletes this causes replication lag.

Disable this script until this issue is resolved.

Bug: T350443
Change-Id: I2b578535ff77f3080b4672ce098c24775f08a1e2
2023-11-02 18:32:06 -06:00
Bartosz Dziewoński
e89be77475 Remove allowances for nullable rd_interwiki and rd_fragment
After the other changes in T346290 these fields can never be null.
The only place that needs to handle null values is the migration
script fixInconsistentRedirects.php.

Bug: T346290
Change-Id: I0235c4be93b203f369b29522b54c0110ee8d61e9
2023-10-03 19:29:55 +02:00
Amir Sarabadani
eaedb7da16 maintenance: Migrate another batch to SelectQueryBuilder
Around fifty-ish. Found becuase of fixed MigrateSelect.

Bug: T344971
Change-Id: If85428d5a033822bfd8ee1f6ab730863bfad55bd
2023-09-21 14:15:42 +02:00
James D. Forrester
94ece673b2 Namespace TitleValue under \MediaWiki\Title
One of the big ones, so doing this alone.

Bug: T166010
Change-Id: I4c901d5c32696d8334ec30cede7d9b6f3d8d645e
2023-09-18 18:24:39 +01:00
Derick Alangi
74033c50cd maintenance: Begin using Maintenance::getServiceContainer()
Maintenance class provides a method for getting a fresh reference
of the MW services container instance. Let's make use of these in
maintenance scripts now that we have it.

NOTE: There are still some static methods like in refreshLinks.php
that makes use of services that we can't use this method for now.

Change-Id: Idba744057577896fc97c9ecf4724db27542bf01c
2023-09-04 10:39:58 +00:00
Taavi Väänänen
6b10489e35
namespaceDupes: Remove extra addQuotes() calls
buildComparison() already quotes arguments given to it, so doing it
twice makes the query return wrong results.

Bug: T333166
Follows-Up: Ic368a87fb5ce4c13608b03206cd68518ec9732d4
Change-Id: I008b74623e6e5d7f7c6b5b5ceb690c2aa7e5b6cb
2023-03-27 16:41:41 +03:00
James D. Forrester
ad06527fb4 Reorg: Namespace the Title class
This is moderately messy.

Process was principally:

* xargs rg --files-with-matches '^use Title;' | grep 'php$' | \
  xargs -P 1 -n 1 sed -i -z 's/use Title;/use MediaWiki\\Title\\Title;/1'
* rg --files-without-match 'MediaWiki\\Title\\Title;' . | grep 'php$' | \
  xargs rg --files-with-matches 'Title\b' | \
  xargs -P 1 -n 1 sed -i -z 's/\nuse /\nuse MediaWiki\\Title\\Title;\nuse /1'
* composer fix

Then manual fix-ups for a few files that don't have any use statements.

Bug: T166010
Follows-Up: Ia5d8cb759dc3bc9e9bbe217d0fb109e2f8c4101a
Change-Id: If8fc9d0d95fc1a114021e282a706fc3e7da3524b
2023-03-02 08:46:53 -05:00
Umherirrender
1b342a8893 Various doc fixes about false and null on method arguments/return types
Doc-only changes

Change-Id: Ice974b3ba41708859dfe646e94b31c5ebbf26410
2022-11-03 18:55:47 +01:00
Amir Sarabadani
b525884e11 maintenance: Use $this->waitForReplication()
This adds reconfiguring db pools in case a replica gets depooled

Bug: T298485
Change-Id: Id052ce8ed45c51e51b071778858d27b48605bf93
2022-10-24 21:11:53 +02:00
Tim Starling
0077c5da15 Use short array destructuring instead of list()
Introduced in PHP 7.1. Because it's shorter and looks nice.

I used regex replacement.

Change-Id: I0555e199d126cd44501f859cb4589f8bd49694da
2022-10-21 15:33:37 +11:00
Bartosz Dziewoński
ec79aa3943 SQLPlatform: Introduce buildComparison()
Builds a condition comparing multiple values, for use with indexes
that cover multiple fields, common when e.g. paging through results
or doing batch operations. Can also be to generate a simple comparison
without writing raw SQL (see T210206).

Update a few manually constructed conditions to use this method.
There are more maintenance scripts and API classes that use the
same patterns, but this is a start.

As you can see by the code I'm replacing, there are many ways to do
this. I picked the one used by maintenance/TableCleanup.php, since
I found it the easiest to understand.

Change-Id: Ic368a87fb5ce4c13608b03206cd68518ec9732d4
2022-09-10 04:22:19 +02:00
Amir Sarabadani
b9c3564759 maintenance: Add support for links migration to namespaceDupes.php
Bug: T314711
Change-Id: I01c3a545248b06d1f73aa99dd898f60767482c8f
2022-08-09 20:11:41 +02:00
Umherirrender
99e2581d26 Use WikiPageFactory to instance WikiPage in namespaceDupes.php
Bug: T259948
Change-Id: Ibf097ecb9562a730102d85496a8b49b566e2c05e
2022-06-25 02:23:08 +02:00
Aryeh Gregor
79fc95d39c Use MainConfigNames instead of string literals, #5
This should be the last of the usages in core, although I'm sure a few
are hiding somehow.

Change-Id: I7bf0b24bf23d3efb4c56a891830bbfe67945e899
2022-04-27 18:46:29 +03:00
Umherirrender
6caf78c2c8 phan: Remove PhanPossiblyUndeclaredVariable suppression
Make phan stricter about conditional variable declaration
Remaining false positive issues are suppressed.
The suppression and the setting change can only be done together

Bug: T259172
Change-Id: I1f200ac37df7448453688bf464a8250c97313e5d
2022-03-30 19:47:15 +00:00
Umherirrender
6dd8a2bb32 phan: Disable scalar_implicit_cast setting
Make phan stricter about scalar types by setting scalar_implicit_cast to
false (the default in mediawiki-phan-config)

Bug: T242536
Bug: T301991
Change-Id: Ia2fe30b17804186571722e728578121c8b75d455
2022-03-18 18:52:24 +00:00
Tim Starling
682aad7557 Move LinksUpdate and LinksDeletionUpdate into the new namespace
Change-Id: I5cf7a08324d08aa89c23540222ba8eddc1ae2647
2022-01-04 15:35:57 +11:00
Thiemo Kreuz
b4c63c64ae Remove some more comments that literally repeat the code
Nothing to learn from these.

You can find a longer explanation in the comments in I93751e6.

Change-Id: I195aae70fc282b58be5b18160783f27d38605d15
2021-12-09 19:01:36 +01:00
Gergő Tisza
551e7448ca namespaceDupes.php: Handle invalid titles
Apply prefix and suffix to invalid titles just like it's applied
to conflicting titles. (Typically this will be needed when the
Foo namespace is created and there is a Foo: article already.)

Also remove a broken for loop. Presumably the intent was to
repeatedly add prefix/suffix on conflict, but actually it would
just get into an infinite loop. The behavior doesn't seem that
useful anyway.

Bug: T293407
Change-Id: Ieef6698b461e7db003dc326e5cd82accd2909540
2021-10-16 12:26:24 +00:00
Umherirrender
0e4bb6d611 Change 'fromdbmaster' to WikiPage::READ_LATEST
Constants are harder to misspell

Change-Id: Ie1afcc8580cccf815df1cc97f05a5be81676d9f7
2021-09-26 19:27:33 +00:00
James D. Forrester
df5eb22f83 Replace uses of DB_MASTER with DB_PRIMARY
Just an auto-replace from codesniffer for now.

Change-Id: I5240dc9ac5929d291b0ef1c743ea2bfd3f428266
2021-04-29 09:24:31 -07:00
Reedy
354848b0cd maintenance: Mark some closures as static
Bug: T274036
Change-Id: Ic959dfddcf2867e4cf26970f375b347f4b41584d
2021-02-07 02:18:30 +00:00
Thiemo Kreuz
20b2c5000d Make use of array deconstruction directly in foreach, if possible
Deconstructing non-sparse, numerically indexed arrays directly in
foreach (a.k.a. using the list() syntax in foreach) is possible since
PHP 5.5.

The possibility to use string array keys as well as non-sequential
numeric keys in array deconstruction was added in PHP 7.1.

Change-Id: I56a48552a45f61cedc291b306cad8548fc70d485
2020-11-12 18:38:06 +00:00
Thiemo Kreuz
6f901ef17c Make use of native array_column() function where possible
Change-Id: I78108e7dc5a10d892e97b0101c1b72cb8d363e97
2020-11-11 22:48:53 +00:00