Commit graph

360 commits

Author SHA1 Message Date
jenkins-bot
aedb1cc864 Merge "DatabaseUpdater: Don't require script path in 'runMaintenance'" 2024-06-19 00:06:29 +00:00
Bartosz Dziewoński
c5ee9dc5d0 DatabaseUpdater: Don't require script path in 'runMaintenance'
Bug: T367918
Change-Id: If9453e0be0cca8cd5c90f0294f0978f672aa76b9
2024-06-18 22:12:26 +02:00
Amir Sarabadani
bb6c6e4174 schema: Drop old pagelinks columns
It has been dropped in production already.

Bug: T299947
Change-Id: I8ec1e7d9224c81d6494c39c78df9e4bdac38d377
2024-06-18 21:13:38 +02:00
Tim Starling
2998d9bd47 block: Migrate to the new block schema on non-WMF wikis
Migrate from ipblocks to block/block_target and drop the ipblocks
table. Update tests.

In PostgresUpdater, change some schema update functions to skip field
updates if the table doesn't exist, by analogy with
DatabaseUpdater::modifyField.

Bug: T346293
Change-Id: Icf91b35f7f729cead7c800429653eb30731762a1
2024-05-09 10:14:43 -04:00
jenkins-bot
f99f308bf4 Merge "Drop 1.35 db updates" 2024-05-05 06:50:24 +00:00
Alexander Vorwerk
28bdd790ee Make rc_id a bigint
Wikidata is at 51.3% of its maximum value (see [1]), so we still have
comfortable time to make this change, but it will inevitably be
necessary and there is no point in postponing this change into the
future. The autoincrement value will not get smaller. ¯\_(ツ)_/¯

Also since rc only stores stuff for 30 days, the table is not that
big.

[1] - https://grafana.wikimedia.org/d/79S1Hq9Mz/wikidata-reliability-metrics?viewPanel=29&orgId=1

Bug: T63111
Change-Id: Icf3dc9815814ef73aa6a39f1c221a349e6b76872
2024-05-04 21:14:51 +02:00
Alexander Vorwerk
b2d44a162c Cleanup revision table schema
* Drop default value from rev_actor and rev_comment_id
* Make rev_id a bigint

Bug: T215466
Depends-On: I88318d7bcc063bc86a56eeb5f00048ea6e81964b
Change-Id: Id0a3d920e8b2dc8643fa3c0341b34ab3ed5761dc
2024-05-04 20:22:36 +02:00
Alexander Vorwerk
bda3118e56 Drop 1.35 db updates
Per official policy, updates from versions older than two LTS releases
are not supported.

Change-Id: I8fb08e226c1a37580f95ee622a0703eee312d9ba
2024-05-03 12:53:12 +00:00
Amir Sarabadani
cf77897613 Run pagelinks migration for third parties too
Production is read new almost everywhere, mediawiki core should reflect
that as well.

Bug: T299947
Change-Id: I8f05c4d7026394205bfc54b68dc725392852ae2a
2024-04-27 16:57:37 -04:00
Amir Sarabadani
f0bfc3d433 Schema: Drop iwl_prefix_from_title from iwlinks
After exhuastive research, we concluded that iwl_prefix_from_title is
not used and in case it's actually used, other indexes provide enough
cardinality.

This table is about to grow quite large in Commons, let's avoid making
it bigger than it needs to be.

Bug: T343131
Change-Id: I89e40dff384291968d2465e4109a3d212ae2f8c7
2024-02-08 17:01:26 +01:00
Amir Sarabadani
616744db1d Schema: Drop unused and useless indexes of sites table
This table has eight indexes plus PK. It has around 1000 rows only. Even
if it needs these indexes (which it doesn't), they are still useless.

Looking at the code, the only potential useful index is the one on
site_global_key, they are showing up in the report of unused indexes in
the db and I checked with Fandom (which might benefit from an index on
this table) and they said they don't use sites table.

Bug: T342856
Change-Id: I06b3db0f33bd35bfa68f4b418d8c2f4b9b988409
2024-02-01 15:32:05 +01:00
Dreamy Jazz
326d3eede8 Add populateUserIsTemp.php to update.php
Why:
* The populateUserIsTemp.php maintenance script is used to populate
  the user_is_temp column in the user table that was added in
  f283c0e990 but was not set to 1 for
  temporary users until 6e68107b3a.
* This script therefore needs to be added to update.php so that wikis
  that enabled temporary account autocreation have this column
  populated.

What:
* Add to MysqlUpdater, PostgresUpdater, and SqliteUpdater code to
  run the populateUserIsTemp.php maintenance script.

Bug: T355181
Change-Id: Ifea9f9c5740d01585c9f832c912ec98d28312e80
2024-01-29 20:21:41 +00:00
Amir Sarabadani
f056a26242 Schema: Drop cl_collation_ext index
Unused since Ie4dd91ee29308c980e

Bug: T342854
Change-Id: I3acf563c64ff176ade3e0c6745839a168e92473b
2024-01-26 16:16:41 +01:00
Reedy
1c0692d19e installer: Namespace remaining classes
Change-Id: I11a8df30a2117ef29a4ade25f1082f25eb18490a
2024-01-13 23:45:14 +00:00
Tim Starling
fd9c7c2d3e Add the new block and block_target tables
Bug: T346293
Change-Id: I3822ad03227405a608dea1d788bcdb8321b95bb3
2024-01-08 14:29:59 +11:00
Thalia
caf9912323 Use year in temporary user names and restart index each year
Why:

* Part of a temporary user name is generated from an index that
  increments, which is stored in the database.
* As specified in T345855, the index will be restarted each year.
* Also specified in T345855, the year will be included in
  generated temporary user names.

What:
* Since the year must be included in the name in order to avoid
  naming conflicts if the index is restarted each year, both are
  implemented together and controlled by a single config.
* Add a new config option that, when true, restarts the name
  generation index at the start of each year and add the year into
  the user name: $wgAutoCreateTempUser['serialProvider']['useYear']
* Add a uas_year column to the user_autocreate_serial table, which
  is unique in combination with uas_shard, so the index can be
  stored for each shard/year combination.
* The year is added into the username just after the prefix, as
  specified in T345855. This is based on research that having the
  year near the start of the name aids understanding that the
  names are not IP addresses. The position of the year within the
  name is therefore not configurable (though whether to include
  it is). See T345855 for the research.

Bug: T349494
Bug: T349501
Depends-On: I6b3c640a4e74f52fd4a4f46de5a2cbe80fe3b665
Change-Id: If51acb3f4efa361ce36d919c862a52501a5a7d24
2024-01-05 17:14:19 +00:00
Amir Sarabadani
ee4c449056 installer: Migrate to SQB in several places
And drop PostgresUpdater::addPgEnumValue() to remove another call to
::select() as it is unused and use of ENUM is discouraged.

Bug: T344971
Change-Id: I7832049e334451b7427f38e45684df9de993030e
2023-11-16 20:03:49 +01:00
Daimona Eaytoy
48a1c0bb2f Autofix spacing around commas
This was done automatically using the
`Universal.WhiteSpace.CommaSpacing` sniff, which will be included in the
next release of the MW PHPCS config.

Some of these have been adjusted manually where the autofix broke
vertical alignment.

Change-Id: I54a4668d8a2759b9d7de47742c943a535a04e211
2023-10-25 01:08:44 +02:00
Brian Wolff
86608dfd4f Store image sizes as 64-bit bigint instead of 32-bit integers
This is meant in preparation for MediaWiki supporting files
larger than 4gb.

Bug: T191805
Change-Id: Ie67dd01aa0a8b28d9afc1805243e711fcadbc0f8
2023-10-04 08:01:25 -07:00
Bartosz Dziewoński
35596d980a installer: Add database updater for 2008/2011 redirect schema changes
In 2008, the `redirect` table was added, and in 2011, it gained the
fields `rd_interwiki` and `rd_fragment`. We have never performed
proper maintenance for those changes, instead relying on code in
WikiPage to update it when the page was visited, or on an optional
run of refreshLinks.php.

I would like to remove the code in WikiPage, so we probably need to
perform this maintenance in the database updater. You know, for the
millions of people who have been dutifully upgrading their MediaWiki
installations since 2008, but never visited the pages there.

The script is a trimmed-down version of refreshLinks.php, without all
the weird stuff, and using a better index for the queries.

Bug: T346290
Change-Id: Iea251d2737b2fb472c4efb060ad2b97735b4ac53
2023-09-21 20:28:13 -07:00
Daimona Eaytoy
3f35734ac0 Remove redundant empty() constructs
empty() only makes sense when the expression it checks is possibly
undefined, otherwise it's equivalent to a truthiness check with the
additional downside of suppressing errors when it's not wanted.

Replace it with simple truthiness checks, using strict comparison when
that seems to help with polymorphic variables.

These were caught by a bespoke phan plugin.

Change-Id: I52999e5286a0d9ad70b0da40b41b9f998ecb990e
2023-09-08 23:18:11 +02:00
Amir Sarabadani
e5eda1c358 Schema: Drop old externallinks columns and indexes
Already dropped from production

Also dropping FixExtLinksProtocolRelative as it's not useful anymore and
it has been run in previous releases so it's not worth fixing.

Bug: T312666
Change-Id: I1dd6e704b34e685ada6e316da11243d10827d769
2023-09-05 15:32:23 +02:00
Amir Sarabadani
dae881296d Schema: Add pl_target_id column to pagelinks
Similar to templatelinks

Bug: T342689
Change-Id: I2ed692d7d0cdf756d29618363bec7fc761ff3df1
2023-07-25 21:03:45 +02:00
Amir Sarabadani
e33818cd08 Fix space after comma in single-line array declaration
We are introducing a new phpcs sniff to make sure this doesn't happen

That sniff found this so far.

Bug: T342297
Change-Id: Ibce3f3d28e7d2cb5b0ff7230f584e76446965ddc
2023-07-24 19:04:59 +02:00
Amir Sarabadani
080883da09 Schema: Set default or nullable to three columns of externallinks
We need to drop these columns and we need to make them take default
values so we can issue write queries without these columns.

Also noting that MySQL prior to 8.0 can't set default values to blob
columns making this way more complicated than it should be but MariaDB
can set them (https://mariadb.com/kb/en/blob/). We also made these
columns nullable to make this work in MySQL.

Bug: T341828
Change-Id: I0d60742b6ce7adf642393ee00b66aa539b76dfc1
2023-07-18 11:59:09 +02:00
Amir Sarabadani
c7056e8fd8 Set default migration stage of externallinks to read new and write both
Bug: T341828
Depends-On: I2d4ef8fc3bffa43bc611af3eab0494f4900df557
Change-Id: Ia55cb6cbdb28484e820f9cf3d6aacac00a86ffab
2023-07-13 22:43:43 +02:00
Alexander Vorwerk
b3611755d7 Drop revision_comment_temp
Bug: T299954
Change-Id: I85d21b1eff70a7d70e8ce14f25d66f7e7c76e5fe
2023-06-07 15:34:57 +00:00
Alexander Vorwerk
ef8d45fd12 Migrate revision_comment_temp to rev_comment_id
Bug: T299954
Change-Id: If3224c73645d5ce5542228a6cee2848e98a9efee
2023-05-27 21:59:26 +00:00
Thalia
f283c0e990 schema: Add user_is_temp column to the user table
This allows temporary users to be identified from applications
external to MediaWiki, by the user table alone (without referring
to the $AutoCreateTempUser['matchPattern'] config).

Bug: T333223
Change-Id: I83c5ff42654164590fb0361c84e65a5315ddbda8
2023-05-10 19:18:39 +01:00
Alexander Vorwerk
599b5a0918 Drop 1.34 db updates
Per official policy, updates from versions older than two TLS releases
are not supported.

Change-Id: I3dc98fe61c8a508aa3ade694f8c0a6a00115031b
2023-02-09 23:28:10 +00:00
Func
18ebc3cbcc PostgresUpdater: Move setDefault ahead of changeNullableField
A user named "Anonymous" existed in setups with Postgres before 1.37,
default values should be set before changing the fields to not nullable.

Bug: T297031
Bug: T326039
Change-Id: Ib0e00a0c608ccca7f40524c135c8f58daded7d69
2022-12-31 21:54:16 +08:00
jenkins-bot
c903124b77 Merge "Make use of ?:, ?? and ??= operators in mostly trivial cases" 2022-12-16 02:51:26 +00:00
Umherirrender
fd516a98e1 Fix whitespaces after comma
Change-Id: Ide6de0a53661e6f650099d7b1f274a02699441df
2022-12-15 01:24:14 +01:00
Umherirrender
a26d8f6f48 postgres: Fix upgrade for templatelinks primary key
Use the default name of the constraint (templatelinks_pkey),
there is no name listed on install

Bug: T324516
Follow-Up: I2b8a29043e952060e7a79b6a7a3d647d48cd16fb
Change-Id: Ib5d186a09b29d04ce6b961b8a3e30d97eca9e761
2022-12-05 23:21:22 +01:00
thiemowmde
70aa9c8e35 Make use of ?:, ?? and ??= operators in mostly trivial cases
The motivation is to make the code less confusing. I hope this is the
case.

?? is an older PHP 7.0 feature.
??= was added in PHP 7.4, which we can finally use.

Change-Id: Id807affa52bd1151a74c064623b41d950a389560
2022-12-05 21:37:13 +01:00
Reedy
1bd4b85fce PostgresUpdater: Improve some indenting
It was hard to see visually that these SQL commands were part of the command above

Change-Id: I20c7c240881b79021cc543f64fa7b78561ca720b
2022-12-01 14:13:43 +00:00
Reedy
f157cc848c PostgresUpdater: Remove trailing space from 'user_id ' column
Change-Id: Ibb8566dccfb055db130a80739ea960dd5fc3db08
Follows-Up: Idc3ded012890d4ace805a646d75cd805b49b250c
2022-12-01 12:33:51 +00:00
Amir Sarabadani
080c70879a schema: Add new fields for externallinks so we can reduce duplication
Bug: T318604
Change-Id: I217817bc518eaa86c9952187c6f1a861f480ccaf
2022-10-18 16:18:54 +00:00
Amir Sarabadani
4bde067722 Drop 1.32 and 1.33 db updates
Per official policy, updates from versions older than two TLS releases
are not supported.

Finding the implicit marker is a bit tricky. The user might try to
upgrade from a really old version that doesn't have tag_summary table
and mistakenly think it's new and allow upgrade. So turning the check
logic into a boolean AND where it must be new enough to have
change_tag_def table AND new enough not to have tag_summary table.

Change-Id: I5ff9630a6539a587a47930847e108ac53757106f
2022-09-26 18:34:31 +02:00
Amir Sarabadani
310e819750 Drop 1.31 db updates
Per official policy, updates from versions older than two TLS releases
are not supported.

We can drop more versions, given that 1.39 is LTS but to ease the
review, let's drop one release per patch

Change-Id: I84bd1d3dbe777909e30710fff5bcb655af9a4261
2022-09-26 17:49:02 +02:00
Amir Sarabadani
6c4194e23e schema: Drop tl_title and tl_namespace fields from templatelinks
The day has gone. Still keeping the code as the schema changes are not
done in production but the data migration has been finished.

Bug: T299417
Change-Id: I906e069a63d1dae14924c72318b22b16244371d6
2022-09-06 19:53:15 +02:00
Amir Sarabadani
692dde00df Add support for write new for templatelinks migration
- schema change to allow tl_namespace and tl_title being empty
   This is done by removing them from primary key. They don't need to be
   nullable as they have default value.
 - Make sure with WRITE_NEW, updater avoids writing to the old columns

Bug: T306674
Change-Id: I2b8a29043e952060e7a79b6a7a3d647d48cd16fb
2022-07-12 14:46:54 +02:00
Amir Sarabadani
ce0b6f40fa schema: Drop legacy page_restrictions in page table
Bug: T35334
Change-Id: I17e73d5ef165481a5dd4c210da933b99c65ff79c
2022-05-26 13:59:57 +02:00
Amir Sarabadani
24115a8f4c Start clean up of revision_actor_temp table
It is being dropped in production

Bug: T215466
Change-Id: I66b2cb8653252e720c897351065978119f040ba7
2022-05-23 15:37:42 +00:00
Tim Starling
e8dbf5f80c TempUser infrastructure and services
Add services and utilities for automatic creation of temporary user
accounts on page save, in order to avoid exposing the user's IP
address.

* Add $wgAutoCreateTempUser, for configuring the system
* Add TempUserConfig service, which interprets the config.
* Add TempUserCreator service, which creates users during page save as
  requested by EditPage. With proxy methods to TempUserConfig for
  convenience.
* Add table user_autocreate_serial. Table creation is necessary before
  the feature is enabled but is not necessary before deployment of this
  commit.

Bug: T300263
Change-Id: Ib14a352490fc42039106523118e8d021844e3dfb
2022-04-14 09:23:55 +10:00
Alexander Vorwerk
d66fe8b7c7 Bump minimum required version for upgrade to 1.31
The policy allows this and since 1.39 is going to be the next LTS
release, I think it is fine to do this now.

Change-Id: If426e0ee349252ccc0ba9c4222c7d6865ab57fa2
2022-03-15 22:12:59 +01:00
Amir Sarabadani
9137db1d8a Add tl_target_id to templatelinks
Part of normalizing it, note that the field must be nullable now and
that will change later.

Bug: T299418
Change-Id: Id543dfa20a153312f66d2f45a64ac23e7272dabe
2022-01-28 17:41:01 +01:00
Amir Sarabadani
203d422dc4 Drop rev_page_id index on revision
This is already applied in production and known to be safe.

Bug: T163532
Change-Id: Ief29372f13b2d7cdb19395dcda6eb15e9a53efca
2022-01-27 09:43:27 +01:00
jenkins-bot
ed940242bb Merge "Add linktarget table" 2022-01-27 00:23:03 +00:00
Amir Sarabadani
4a6e986141 Add linktarget table
Bug: T299416
Change-Id: Icae4513dd99635335857100d8a0c7102986933e5
2022-01-21 15:52:58 +01:00