Commit graph

480 commits

Author SHA1 Message Date
Derk-Jan Hartman
b00f7237ee Searchindex primary key, title length and utf8mb4
- Make si_page primary key
- Allow si_title to be larger
- Switch from utf8 to utf8mb4
- Remove default of empty string for titles

Sqlite is not migrated, as it has it's own overrides
Postgres is migrated, but is not in actual use

This is mostly from I273e3a7715abf97d2889904642c7c375e76de4f6

Bug: T249976
Bug: T231827
Change-Id: I12adff3e6ca6a9986ff207bef16272195c3a6a48
(cherry picked from commit a2ba7ee14d6b38a5e89bcc63c3bd0ca8b5107702)
2024-11-02 11:14:37 +00:00
Tim Starling
7b909408fc Move update-keys.sql to a PHP array
Improve maintainability of update-keys.sql by moving the list from an
SQL file to a PHP array in the same file as the updates it is disabling.

It was apparently an SQL file for the convenience of third parties
wishing to install MediaWiki by manually sourcing tables.sql. Support
this use case by adding an option to update.php which inserts the update
keys. Users should source tables-generated.sql and then run
update.php --initial

The array is the same for each DB type, but it's correct for it to have
the same scope as getCoreUpdateList(). Factoring out common parts of
getCoreUpdateList() and this new array is a separate project.

Bug: T167924
Change-Id: I404ee29aadcc2f3f24f78d1111090395129cd021
2024-09-13 12:02:07 +10:00
Amir Sarabadani
bb6c6e4174 schema: Drop old pagelinks columns
It has been dropped in production already.

Bug: T299947
Change-Id: I8ec1e7d9224c81d6494c39c78df9e4bdac38d377
2024-06-18 21:13:38 +02:00
Tim Starling
2998d9bd47 block: Migrate to the new block schema on non-WMF wikis
Migrate from ipblocks to block/block_target and drop the ipblocks
table. Update tests.

In PostgresUpdater, change some schema update functions to skip field
updates if the table doesn't exist, by analogy with
DatabaseUpdater::modifyField.

Bug: T346293
Change-Id: Icf91b35f7f729cead7c800429653eb30731762a1
2024-05-09 10:14:43 -04:00
jenkins-bot
f99f308bf4 Merge "Drop 1.35 db updates" 2024-05-05 06:50:24 +00:00
Alexander Vorwerk
28bdd790ee Make rc_id a bigint
Wikidata is at 51.3% of its maximum value (see [1]), so we still have
comfortable time to make this change, but it will inevitably be
necessary and there is no point in postponing this change into the
future. The autoincrement value will not get smaller. ¯\_(ツ)_/¯

Also since rc only stores stuff for 30 days, the table is not that
big.

[1] - https://grafana.wikimedia.org/d/79S1Hq9Mz/wikidata-reliability-metrics?viewPanel=29&orgId=1

Bug: T63111
Change-Id: Icf3dc9815814ef73aa6a39f1c221a349e6b76872
2024-05-04 21:14:51 +02:00
Alexander Vorwerk
b2d44a162c Cleanup revision table schema
* Drop default value from rev_actor and rev_comment_id
* Make rev_id a bigint

Bug: T215466
Depends-On: I88318d7bcc063bc86a56eeb5f00048ea6e81964b
Change-Id: Id0a3d920e8b2dc8643fa3c0341b34ab3ed5761dc
2024-05-04 20:22:36 +02:00
Alexander Vorwerk
bda3118e56 Drop 1.35 db updates
Per official policy, updates from versions older than two LTS releases
are not supported.

Change-Id: I8fb08e226c1a37580f95ee622a0703eee312d9ba
2024-05-03 12:53:12 +00:00
Amir Sarabadani
f0bfc3d433 Schema: Drop iwl_prefix_from_title from iwlinks
After exhuastive research, we concluded that iwl_prefix_from_title is
not used and in case it's actually used, other indexes provide enough
cardinality.

This table is about to grow quite large in Commons, let's avoid making
it bigger than it needs to be.

Bug: T343131
Change-Id: I89e40dff384291968d2465e4109a3d212ae2f8c7
2024-02-08 17:01:26 +01:00
Amir Sarabadani
616744db1d Schema: Drop unused and useless indexes of sites table
This table has eight indexes plus PK. It has around 1000 rows only. Even
if it needs these indexes (which it doesn't), they are still useless.

Looking at the code, the only potential useful index is the one on
site_global_key, they are showing up in the report of unused indexes in
the db and I checked with Fandom (which might benefit from an index on
this table) and they said they don't use sites table.

Bug: T342856
Change-Id: I06b3db0f33bd35bfa68f4b418d8c2f4b9b988409
2024-02-01 15:32:05 +01:00
Amir Sarabadani
f056a26242 Schema: Drop cl_collation_ext index
Unused since Ie4dd91ee29308c980e

Bug: T342854
Change-Id: I3acf563c64ff176ade3e0c6745839a168e92473b
2024-01-26 16:16:41 +01:00
Tim Starling
fd9c7c2d3e Add the new block and block_target tables
Bug: T346293
Change-Id: I3822ad03227405a608dea1d788bcdb8321b95bb3
2024-01-08 14:29:59 +11:00
Thalia
caf9912323 Use year in temporary user names and restart index each year
Why:

* Part of a temporary user name is generated from an index that
  increments, which is stored in the database.
* As specified in T345855, the index will be restarted each year.
* Also specified in T345855, the year will be included in
  generated temporary user names.

What:
* Since the year must be included in the name in order to avoid
  naming conflicts if the index is restarted each year, both are
  implemented together and controlled by a single config.
* Add a new config option that, when true, restarts the name
  generation index at the start of each year and add the year into
  the user name: $wgAutoCreateTempUser['serialProvider']['useYear']
* Add a uas_year column to the user_autocreate_serial table, which
  is unique in combination with uas_shard, so the index can be
  stored for each shard/year combination.
* The year is added into the username just after the prefix, as
  specified in T345855. This is based on research that having the
  year near the start of the name aids understanding that the
  names are not IP addresses. The position of the year within the
  name is therefore not configurable (though whether to include
  it is). See T345855 for the research.

Bug: T349494
Bug: T349501
Depends-On: I6b3c640a4e74f52fd4a4f46de5a2cbe80fe3b665
Change-Id: If51acb3f4efa361ce36d919c862a52501a5a7d24
2024-01-05 17:14:19 +00:00
Umherirrender
daa6526ce8 installer: Fix location of patch-pagelinks-target_id.sql for postgres
Bug: T351619
Follow-Up: I2ed692d7d0cdf756d29618363bec7fc761ff3df1
Change-Id: Ie2273c6803a83831b09f176057d6a9afccdfbdf2
2023-11-20 19:11:46 +01:00
Brian Wolff
86608dfd4f Store image sizes as 64-bit bigint instead of 32-bit integers
This is meant in preparation for MediaWiki supporting files
larger than 4gb.

Bug: T191805
Change-Id: Ie67dd01aa0a8b28d9afc1805243e711fcadbc0f8
2023-10-04 08:01:25 -07:00
Amir Sarabadani
e5eda1c358 Schema: Drop old externallinks columns and indexes
Already dropped from production

Also dropping FixExtLinksProtocolRelative as it's not useful anymore and
it has been run in previous releases so it's not worth fixing.

Bug: T312666
Change-Id: I1dd6e704b34e685ada6e316da11243d10827d769
2023-09-05 15:32:23 +02:00
Amir Sarabadani
dae881296d Schema: Add pl_target_id column to pagelinks
Similar to templatelinks

Bug: T342689
Change-Id: I2ed692d7d0cdf756d29618363bec7fc761ff3df1
2023-07-25 21:03:45 +02:00
Amir Sarabadani
080883da09 Schema: Set default or nullable to three columns of externallinks
We need to drop these columns and we need to make them take default
values so we can issue write queries without these columns.

Also noting that MySQL prior to 8.0 can't set default values to blob
columns making this way more complicated than it should be but MariaDB
can set them (https://mariadb.com/kb/en/blob/). We also made these
columns nullable to make this work in MySQL.

Bug: T341828
Change-Id: I0d60742b6ce7adf642393ee00b66aa539b76dfc1
2023-07-18 11:59:09 +02:00
Alexander Vorwerk
b3611755d7 Drop revision_comment_temp
Bug: T299954
Change-Id: I85d21b1eff70a7d70e8ce14f25d66f7e7c76e5fe
2023-06-07 15:34:57 +00:00
Marks Polakovs
c210066e6d Improve idempotency of postgres index upgrades
Change DROP INDEX statements to DROP INDEX IF EXISTS, meaning the
upgrade does not fail if they do not already exist (which is the desired
end state anyway).

Bug: T336868
Change-Id: Id073beda84065b067994a7405a78d484346b550d
2023-05-17 14:58:41 +01:00
Thalia
f283c0e990 schema: Add user_is_temp column to the user table
This allows temporary users to be identified from applications
external to MediaWiki, by the user table alone (without referring
to the $AutoCreateTempUser['matchPattern'] config).

Bug: T333223
Change-Id: I83c5ff42654164590fb0361c84e65a5315ddbda8
2023-05-10 19:18:39 +01:00
Amir Sarabadani
76656251b5 postgres: Make the upgrade ignore dropping indexes that might not exist
Due to drifts, 1.35 didn't have tl_namespace nor tl_backlinks_namespace
but that was fixed in 1.36 which the drop of tl_title tries to drop
leading to errors.

Bug: T330382
Change-Id: I555e855b3c0e97e5df1f0fab1de104301d40eae4
2023-02-27 22:14:40 +01:00
Amir Sarabadani
080c70879a schema: Add new fields for externallinks so we can reduce duplication
Bug: T318604
Change-Id: I217817bc518eaa86c9952187c6f1a861f480ccaf
2022-10-18 16:18:54 +00:00
Amir Sarabadani
c910301518 Drop useless maint scripts and sql files after 1.31-1.33 clean up
These files are not needed anymore. Either because they are orphan now
and no updater will run these sql files.

Or they are maint scripts to migrate the data and that has been removed
in I5ff9630a65 and I84bd1d3dbe7

Change-Id: I00c49d8d649e3fa7641f3c57bf3ea4440ae542df
2022-09-28 16:22:14 +02:00
Amir Sarabadani
4bde067722 Drop 1.32 and 1.33 db updates
Per official policy, updates from versions older than two TLS releases
are not supported.

Finding the implicit marker is a bit tricky. The user might try to
upgrade from a really old version that doesn't have tag_summary table
and mistakenly think it's new and allow upgrade. So turning the check
logic into a boolean AND where it must be new enough to have
change_tag_def table AND new enough not to have tag_summary table.

Change-Id: I5ff9630a6539a587a47930847e108ac53757106f
2022-09-26 18:34:31 +02:00
Amir Sarabadani
310e819750 Drop 1.31 db updates
Per official policy, updates from versions older than two TLS releases
are not supported.

We can drop more versions, given that 1.39 is LTS but to ease the
review, let's drop one release per patch

Change-Id: I84bd1d3dbe777909e30710fff5bcb655af9a4261
2022-09-26 17:49:02 +02:00
Amir Sarabadani
6c4194e23e schema: Drop tl_title and tl_namespace fields from templatelinks
The day has gone. Still keeping the code as the schema changes are not
done in production but the data migration has been finished.

Bug: T299417
Change-Id: I906e069a63d1dae14924c72318b22b16244371d6
2022-09-06 19:53:15 +02:00
Amir Sarabadani
692dde00df Add support for write new for templatelinks migration
- schema change to allow tl_namespace and tl_title being empty
   This is done by removing them from primary key. They don't need to be
   nullable as they have default value.
 - Make sure with WRITE_NEW, updater avoids writing to the old columns

Bug: T306674
Change-Id: I2b8a29043e952060e7a79b6a7a3d647d48cd16fb
2022-07-12 14:46:54 +02:00
Amir Sarabadani
ce0b6f40fa schema: Drop legacy page_restrictions in page table
Bug: T35334
Change-Id: I17e73d5ef165481a5dd4c210da933b99c65ff79c
2022-05-26 13:59:57 +02:00
Amir Sarabadani
24115a8f4c Start clean up of revision_actor_temp table
It is being dropped in production

Bug: T215466
Change-Id: I66b2cb8653252e720c897351065978119f040ba7
2022-05-23 15:37:42 +00:00
Tim Starling
e8dbf5f80c TempUser infrastructure and services
Add services and utilities for automatic creation of temporary user
accounts on page save, in order to avoid exposing the user's IP
address.

* Add $wgAutoCreateTempUser, for configuring the system
* Add TempUserConfig service, which interprets the config.
* Add TempUserCreator service, which creates users during page save as
  requested by EditPage. With proxy methods to TempUserConfig for
  convenience.
* Add table user_autocreate_serial. Table creation is necessary before
  the feature is enabled but is not necessary before deployment of this
  commit.

Bug: T300263
Change-Id: Ib14a352490fc42039106523118e8d021844e3dfb
2022-04-14 09:23:55 +10:00
Alexander Vorwerk
d66fe8b7c7 Bump minimum required version for upgrade to 1.31
The policy allows this and since 1.39 is going to be the next LTS
release, I think it is fine to do this now.

Change-Id: If426e0ee349252ccc0ba9c4222c7d6865ab57fa2
2022-03-15 22:12:59 +01:00
Amir Sarabadani
9137db1d8a Add tl_target_id to templatelinks
Part of normalizing it, note that the field must be nullable now and
that will change later.

Bug: T299418
Change-Id: Id543dfa20a153312f66d2f45a64ac23e7272dabe
2022-01-28 17:41:01 +01:00
Amir Sarabadani
203d422dc4 Drop rev_page_id index on revision
This is already applied in production and known to be safe.

Bug: T163532
Change-Id: Ief29372f13b2d7cdb19395dcda6eb15e9a53efca
2022-01-27 09:43:27 +01:00
Amir Sarabadani
4a6e986141 Add linktarget table
Bug: T299416
Change-Id: Icae4513dd99635335857100d8a0c7102986933e5
2022-01-21 15:52:58 +01:00
Amir Sarabadani
ede308ed55 Drop pr_user from page_restrictions
It's not used for 13 years, safe to say it won't be used anytime soon

Bug: T199377
Change-Id: Iecf4fb6046699a1758ad2d1dc55a3ee8eb4b0389
2021-12-06 14:15:39 +01:00
Ammar Abdulhamid
1adaca51c3 Rename change_tag indexes to have ct_ prefix
Bug: T270033
Change-Id: I8a429726c99f6cadea0d671fd871f66b5611c856
2021-06-08 17:57:15 +01:00
Ammarpad
a8c01d7726 Rename name_title index to have page_ prefix
Bug: T270033
Change-Id: Id70d0e0a37dd0d000079820d51cef2791f5ec42e
2021-06-05 20:21:07 +02:00
Amir Sarabadani
7fdca7ce51 Add index on oldimage.oi_timestamp
Mirroring what happens on image table.

Bug: T279982
Change-Id: Ib67c32b10d3b88e09514d8fbb5dcb4f977108ba2
2021-05-29 20:44:32 +02:00
Amir Sarabadani
acd5076099 Migrate searchindex to abstract schema
For MySQL/Sqlite:
 - Nothing

For Postgres:
 - Introduce the table. It's not used but it'll be used in the future (T220450)

Bug: T164898
Bug: T230428
Bug: T220450
Change-Id: I9f33132676344f8cd813f7a438b3a6a078fd281c
2021-05-27 15:01:24 +02:00
Ammarpad
5954b380fc Migrate revision table to abstract schema
Postgres:
 - Drop foreign key from rev_page
 - Make rev_page not nullable
 - Change rev_comment_id from int to bigint
 - Change rev_actor from int to bigint
 - Sync rev_page_id index with MySQL

MySQL/SQlite:
 - Drop default from rev_timestamp

Additional changes in the generator script to handle more
formatting issues due to use of additional custom table options

Bug: T230428
Bug: T164898
Change-Id: Ia07dd52e43123473a1728523a3f863280537db8e
2021-05-20 14:52:15 -07:00
Aaron Schulz
12923c1cd5 objectcache: add last-modified token field to objectcache table
Also added token and flags fields. The token field can
be used as a tie-breaker for modtime and also for faster
cas() operations. The flags field makes serialization and
compression format changes easier in the future.

Bug: T274174
Change-Id: I45731a877b21835652993c2d285165a76eeae3e9
2021-05-18 20:06:37 -07:00
Ammarpad
88bfadf5ed Migrate user table to abstract schema
Postgres:
 - Rename the table from `mwuser` to `user`
 - Transition user_id from int to serial
 - Make user_token not nullable and add default
 - Make user_real_name not nullable and add default
 - Make user_email not nullable
 - Make user_newpassword not nullable
 - Make user_password not nullable
 - Drop UNIQUE contraint on user_name and add default

MySQL/SQLite:
 * No changes

Bug: T164898
Bug: T230428
Change-Id: I746714f7b3ae16f9625f97bcca84280fcd8b61a0
2021-04-24 21:44:26 +01:00
jenkins-bot
e614e8653e Merge "Drop unused patch-rc_cur_id-not-null.sql" 2021-04-02 23:01:34 +00:00
jenkins-bot
fc8944012a Merge "Drop unused patch-update_sequences.sql" 2021-04-02 22:54:59 +00:00
Amir Sarabadani
ac7391c6ea Drop unused patch-rc_cur_id-not-null.sql
Unused since Iebb6855e8f6f44470bb (2012)

Change-Id: I82d16d2c0d28902a0c3aedf8fece69c02a56c95c
2021-04-02 15:57:54 +02:00
Amir Sarabadani
1980d47f7f Drop unused patch-update_sequences.sql
Unused since r71040 (2010)

Change-Id: Ib0d841a974eef95898e4627d0b93ab3dd4d64f93
2021-04-02 15:55:58 +02:00
Amir Sarabadani
18b33350d8 Drop now unused patch-kill-iwl_prefix.sql
Leftover from I0ff387e057c1fb

Bug: T272199
Change-Id: If05e8613b6c87d8267088e6d082a8324359469b1
2021-04-02 02:12:07 +02:00
Amir Sarabadani
b4a236b9d9 Drop now unused patch-categorylinks-better-collation.sql
Leftover from I0ff387e057c1fb

Bug: T272199
Change-Id: I6a00353eeb16496ce3ed2cd5f39906fa3b35e204
2021-04-02 00:02:19 +02:00
Amir Sarabadani
4213630c2a Drop unused patch-uploadstash_sequence.sql
This was introduced in r103367 as a follow up to r103365 while none of
the commits wire this to update.php and since then it has been unused.

And even if it was wired dynamically somehow that I missed, it's not
needed anymore as we don't support direct upgrades from these versions
(1.18 and 1.19) anymore.

Change-Id: I1e16de398bbe61adcb55787117378ea96a2b3e9f
2021-03-30 20:42:16 +02:00