Commit graph

155 commits

Author SHA1 Message Date
Derk-Jan Hartman
b00f7237ee Searchindex primary key, title length and utf8mb4
- Make si_page primary key
- Allow si_title to be larger
- Switch from utf8 to utf8mb4
- Remove default of empty string for titles

Sqlite is not migrated, as it has it's own overrides
Postgres is migrated, but is not in actual use

This is mostly from I273e3a7715abf97d2889904642c7c375e76de4f6

Bug: T249976
Bug: T231827
Change-Id: I12adff3e6ca6a9986ff207bef16272195c3a6a48
(cherry picked from commit a2ba7ee14d6b38a5e89bcc63c3bd0ca8b5107702)
2024-11-02 11:14:37 +00:00
Amir Sarabadani
bb6c6e4174 schema: Drop old pagelinks columns
It has been dropped in production already.

Bug: T299947
Change-Id: I8ec1e7d9224c81d6494c39c78df9e4bdac38d377
2024-06-18 21:13:38 +02:00
jenkins-bot
f99f308bf4 Merge "Drop 1.35 db updates" 2024-05-05 06:50:24 +00:00
Alexander Vorwerk
28bdd790ee Make rc_id a bigint
Wikidata is at 51.3% of its maximum value (see [1]), so we still have
comfortable time to make this change, but it will inevitably be
necessary and there is no point in postponing this change into the
future. The autoincrement value will not get smaller. ¯\_(ツ)_/¯

Also since rc only stores stuff for 30 days, the table is not that
big.

[1] - https://grafana.wikimedia.org/d/79S1Hq9Mz/wikidata-reliability-metrics?viewPanel=29&orgId=1

Bug: T63111
Change-Id: Icf3dc9815814ef73aa6a39f1c221a349e6b76872
2024-05-04 21:14:51 +02:00
Alexander Vorwerk
b2d44a162c Cleanup revision table schema
* Drop default value from rev_actor and rev_comment_id
* Make rev_id a bigint

Bug: T215466
Depends-On: I88318d7bcc063bc86a56eeb5f00048ea6e81964b
Change-Id: Id0a3d920e8b2dc8643fa3c0341b34ab3ed5761dc
2024-05-04 20:22:36 +02:00
Alexander Vorwerk
bda3118e56 Drop 1.35 db updates
Per official policy, updates from versions older than two LTS releases
are not supported.

Change-Id: I8fb08e226c1a37580f95ee622a0703eee312d9ba
2024-05-03 12:53:12 +00:00
Amir Sarabadani
f0bfc3d433 Schema: Drop iwl_prefix_from_title from iwlinks
After exhuastive research, we concluded that iwl_prefix_from_title is
not used and in case it's actually used, other indexes provide enough
cardinality.

This table is about to grow quite large in Commons, let's avoid making
it bigger than it needs to be.

Bug: T343131
Change-Id: I89e40dff384291968d2465e4109a3d212ae2f8c7
2024-02-08 17:01:26 +01:00
Amir Sarabadani
616744db1d Schema: Drop unused and useless indexes of sites table
This table has eight indexes plus PK. It has around 1000 rows only. Even
if it needs these indexes (which it doesn't), they are still useless.

Looking at the code, the only potential useful index is the one on
site_global_key, they are showing up in the report of unused indexes in
the db and I checked with Fandom (which might benefit from an index on
this table) and they said they don't use sites table.

Bug: T342856
Change-Id: I06b3db0f33bd35bfa68f4b418d8c2f4b9b988409
2024-02-01 15:32:05 +01:00
Amir Sarabadani
f056a26242 Schema: Drop cl_collation_ext index
Unused since Ie4dd91ee29308c980e

Bug: T342854
Change-Id: I3acf563c64ff176ade3e0c6745839a168e92473b
2024-01-26 16:16:41 +01:00
Tim Starling
fd9c7c2d3e Add the new block and block_target tables
Bug: T346293
Change-Id: I3822ad03227405a608dea1d788bcdb8321b95bb3
2024-01-08 14:29:59 +11:00
Thalia
caf9912323 Use year in temporary user names and restart index each year
Why:

* Part of a temporary user name is generated from an index that
  increments, which is stored in the database.
* As specified in T345855, the index will be restarted each year.
* Also specified in T345855, the year will be included in
  generated temporary user names.

What:
* Since the year must be included in the name in order to avoid
  naming conflicts if the index is restarted each year, both are
  implemented together and controlled by a single config.
* Add a new config option that, when true, restarts the name
  generation index at the start of each year and add the year into
  the user name: $wgAutoCreateTempUser['serialProvider']['useYear']
* Add a uas_year column to the user_autocreate_serial table, which
  is unique in combination with uas_shard, so the index can be
  stored for each shard/year combination.
* The year is added into the username just after the prefix, as
  specified in T345855. This is based on research that having the
  year near the start of the name aids understanding that the
  names are not IP addresses. The position of the year within the
  name is therefore not configurable (though whether to include
  it is). See T345855 for the research.

Bug: T349494
Bug: T349501
Depends-On: I6b3c640a4e74f52fd4a4f46de5a2cbe80fe3b665
Change-Id: If51acb3f4efa361ce36d919c862a52501a5a7d24
2024-01-05 17:14:19 +00:00
Umherirrender
daa6526ce8 installer: Fix location of patch-pagelinks-target_id.sql for postgres
Bug: T351619
Follow-Up: I2ed692d7d0cdf756d29618363bec7fc761ff3df1
Change-Id: Ie2273c6803a83831b09f176057d6a9afccdfbdf2
2023-11-20 19:11:46 +01:00
Brian Wolff
86608dfd4f Store image sizes as 64-bit bigint instead of 32-bit integers
This is meant in preparation for MediaWiki supporting files
larger than 4gb.

Bug: T191805
Change-Id: Ie67dd01aa0a8b28d9afc1805243e711fcadbc0f8
2023-10-04 08:01:25 -07:00
Amir Sarabadani
e5eda1c358 Schema: Drop old externallinks columns and indexes
Already dropped from production

Also dropping FixExtLinksProtocolRelative as it's not useful anymore and
it has been run in previous releases so it's not worth fixing.

Bug: T312666
Change-Id: I1dd6e704b34e685ada6e316da11243d10827d769
2023-09-05 15:32:23 +02:00
Amir Sarabadani
dae881296d Schema: Add pl_target_id column to pagelinks
Similar to templatelinks

Bug: T342689
Change-Id: I2ed692d7d0cdf756d29618363bec7fc761ff3df1
2023-07-25 21:03:45 +02:00
Amir Sarabadani
080883da09 Schema: Set default or nullable to three columns of externallinks
We need to drop these columns and we need to make them take default
values so we can issue write queries without these columns.

Also noting that MySQL prior to 8.0 can't set default values to blob
columns making this way more complicated than it should be but MariaDB
can set them (https://mariadb.com/kb/en/blob/). We also made these
columns nullable to make this work in MySQL.

Bug: T341828
Change-Id: I0d60742b6ce7adf642393ee00b66aa539b76dfc1
2023-07-18 11:59:09 +02:00
Marks Polakovs
c210066e6d Improve idempotency of postgres index upgrades
Change DROP INDEX statements to DROP INDEX IF EXISTS, meaning the
upgrade does not fail if they do not already exist (which is the desired
end state anyway).

Bug: T336868
Change-Id: Id073beda84065b067994a7405a78d484346b550d
2023-05-17 14:58:41 +01:00
Thalia
f283c0e990 schema: Add user_is_temp column to the user table
This allows temporary users to be identified from applications
external to MediaWiki, by the user table alone (without referring
to the $AutoCreateTempUser['matchPattern'] config).

Bug: T333223
Change-Id: I83c5ff42654164590fb0361c84e65a5315ddbda8
2023-05-10 19:18:39 +01:00
Amir Sarabadani
76656251b5 postgres: Make the upgrade ignore dropping indexes that might not exist
Due to drifts, 1.35 didn't have tl_namespace nor tl_backlinks_namespace
but that was fixed in 1.36 which the drop of tl_title tries to drop
leading to errors.

Bug: T330382
Change-Id: I555e855b3c0e97e5df1f0fab1de104301d40eae4
2023-02-27 22:14:40 +01:00
Amir Sarabadani
080c70879a schema: Add new fields for externallinks so we can reduce duplication
Bug: T318604
Change-Id: I217817bc518eaa86c9952187c6f1a861f480ccaf
2022-10-18 16:18:54 +00:00
Amir Sarabadani
c910301518 Drop useless maint scripts and sql files after 1.31-1.33 clean up
These files are not needed anymore. Either because they are orphan now
and no updater will run these sql files.

Or they are maint scripts to migrate the data and that has been removed
in I5ff9630a65 and I84bd1d3dbe7

Change-Id: I00c49d8d649e3fa7641f3c57bf3ea4440ae542df
2022-09-28 16:22:14 +02:00
Amir Sarabadani
4bde067722 Drop 1.32 and 1.33 db updates
Per official policy, updates from versions older than two TLS releases
are not supported.

Finding the implicit marker is a bit tricky. The user might try to
upgrade from a really old version that doesn't have tag_summary table
and mistakenly think it's new and allow upgrade. So turning the check
logic into a boolean AND where it must be new enough to have
change_tag_def table AND new enough not to have tag_summary table.

Change-Id: I5ff9630a6539a587a47930847e108ac53757106f
2022-09-26 18:34:31 +02:00
Amir Sarabadani
310e819750 Drop 1.31 db updates
Per official policy, updates from versions older than two TLS releases
are not supported.

We can drop more versions, given that 1.39 is LTS but to ease the
review, let's drop one release per patch

Change-Id: I84bd1d3dbe777909e30710fff5bcb655af9a4261
2022-09-26 17:49:02 +02:00
Amir Sarabadani
6c4194e23e schema: Drop tl_title and tl_namespace fields from templatelinks
The day has gone. Still keeping the code as the schema changes are not
done in production but the data migration has been finished.

Bug: T299417
Change-Id: I906e069a63d1dae14924c72318b22b16244371d6
2022-09-06 19:53:15 +02:00
Tim Starling
e8dbf5f80c TempUser infrastructure and services
Add services and utilities for automatic creation of temporary user
accounts on page save, in order to avoid exposing the user's IP
address.

* Add $wgAutoCreateTempUser, for configuring the system
* Add TempUserConfig service, which interprets the config.
* Add TempUserCreator service, which creates users during page save as
  requested by EditPage. With proxy methods to TempUserConfig for
  convenience.
* Add table user_autocreate_serial. Table creation is necessary before
  the feature is enabled but is not necessary before deployment of this
  commit.

Bug: T300263
Change-Id: Ib14a352490fc42039106523118e8d021844e3dfb
2022-04-14 09:23:55 +10:00
Alexander Vorwerk
d66fe8b7c7 Bump minimum required version for upgrade to 1.31
The policy allows this and since 1.39 is going to be the next LTS
release, I think it is fine to do this now.

Change-Id: If426e0ee349252ccc0ba9c4222c7d6865ab57fa2
2022-03-15 22:12:59 +01:00
Amir Sarabadani
9137db1d8a Add tl_target_id to templatelinks
Part of normalizing it, note that the field must be nullable now and
that will change later.

Bug: T299418
Change-Id: Id543dfa20a153312f66d2f45a64ac23e7272dabe
2022-01-28 17:41:01 +01:00
Amir Sarabadani
203d422dc4 Drop rev_page_id index on revision
This is already applied in production and known to be safe.

Bug: T163532
Change-Id: Ief29372f13b2d7cdb19395dcda6eb15e9a53efca
2022-01-27 09:43:27 +01:00
Amir Sarabadani
4a6e986141 Add linktarget table
Bug: T299416
Change-Id: Icae4513dd99635335857100d8a0c7102986933e5
2022-01-21 15:52:58 +01:00
Amir Sarabadani
acd5076099 Migrate searchindex to abstract schema
For MySQL/Sqlite:
 - Nothing

For Postgres:
 - Introduce the table. It's not used but it'll be used in the future (T220450)

Bug: T164898
Bug: T230428
Bug: T220450
Change-Id: I9f33132676344f8cd813f7a438b3a6a078fd281c
2021-05-27 15:01:24 +02:00
Aaron Schulz
12923c1cd5 objectcache: add last-modified token field to objectcache table
Also added token and flags fields. The token field can
be used as a tie-breaker for modtime and also for faster
cas() operations. The flags field makes serialization and
compression format changes easier in the future.

Bug: T274174
Change-Id: I45731a877b21835652993c2d285165a76eeae3e9
2021-05-18 20:06:37 -07:00
jenkins-bot
e614e8653e Merge "Drop unused patch-rc_cur_id-not-null.sql" 2021-04-02 23:01:34 +00:00
jenkins-bot
fc8944012a Merge "Drop unused patch-update_sequences.sql" 2021-04-02 22:54:59 +00:00
Amir Sarabadani
ac7391c6ea Drop unused patch-rc_cur_id-not-null.sql
Unused since Iebb6855e8f6f44470bb (2012)

Change-Id: I82d16d2c0d28902a0c3aedf8fece69c02a56c95c
2021-04-02 15:57:54 +02:00
Amir Sarabadani
1980d47f7f Drop unused patch-update_sequences.sql
Unused since r71040 (2010)

Change-Id: Ib0d841a974eef95898e4627d0b93ab3dd4d64f93
2021-04-02 15:55:58 +02:00
Amir Sarabadani
18b33350d8 Drop now unused patch-kill-iwl_prefix.sql
Leftover from I0ff387e057c1fb

Bug: T272199
Change-Id: If05e8613b6c87d8267088e6d082a8324359469b1
2021-04-02 02:12:07 +02:00
Amir Sarabadani
b4a236b9d9 Drop now unused patch-categorylinks-better-collation.sql
Leftover from I0ff387e057c1fb

Bug: T272199
Change-Id: I6a00353eeb16496ce3ed2cd5f39906fa3b35e204
2021-04-02 00:02:19 +02:00
Amir Sarabadani
4213630c2a Drop unused patch-uploadstash_sequence.sql
This was introduced in r103367 as a follow up to r103365 while none of
the commits wire this to update.php and since then it has been unused.

And even if it was wired dynamically somehow that I missed, it's not
needed anymore as we don't support direct upgrades from these versions
(1.18 and 1.19) anymore.

Change-Id: I1e16de398bbe61adcb55787117378ea96a2b3e9f
2021-03-30 20:42:16 +02:00
Ammarpad
eece54b831 Remove unused schema change.
There's no reason to keep schema change that's no longer useful.
History is already tracked by git.

Change-Id: Ia8aa0d21ae5f2d799d9a641ebe7b924f56e07ab1
2021-03-27 09:32:29 +01:00
Amir Sarabadani
6ba7c4fa09 Drop rest of unsupported upgrade paths and patches in Postgres
These are from versions we don't support direct upgrade anymore and they
are already removed from update.php wiring.

Bug: T272199
Change-Id: I863d91d7d620943571cb8bb22f663c2efbbd0821
2021-03-22 07:43:34 +01:00
Amir Sarabadani
447db80038 Drop tsearch fixes in postgres
These changes to tsearch functions were introduced in 2010 in a version
that we definitely don't support direct upgrades from.

Bug: T272199
Change-Id: I47c4054552daad55c0e13eec36865706ee6d1903
2021-03-22 07:29:06 +01:00
Amir Sarabadani
e2c005f91f Drop unsupported 1.23 upgrade code and patches from Postgres
We don't support upgrading from 1.23 anymore.

Bug: T272199
Change-Id: Iac9652095927c86c47f45a2aa131c28efef5c73a
2021-03-22 07:26:53 +01:00
Amir Sarabadani
11f091db7d Drop add_interwiki updater in Postgres
This function was removed in abstracting interwiki table (Id652cb544)
and it's not needed anymore.

Change-Id: Ifc956ae91f8362e9872cd449fcab5cfb4870ec7f
2021-03-20 18:25:02 +00:00
Ammarpad
2ca437dca9 Drop 1.27 upgrade patch files
6 files for Mysql/Sqlite/PG

Change-Id: Idef4906c5a7fdac2667c35bac7e8cc54629647f8
2021-03-09 07:15:37 +01:00
Amir Sarabadani
1053405664 Rename new_name_timestamp on recentchanges to rc_new_name_timestamp
To make it have a uniform prefix for index and column names

Bug: T270033
Change-Id: I8eb600416913092bd5aeb70389bba6e8a54d1d57
2021-03-01 19:00:52 +00:00
Amir Sarabadani
fa7b284d70 Remove all old index checks in PG
I checked all of them with https://github.com/wikimedia/mediawiki/blob/REL1_26/includes/installer/PostgresUpdater.php
using a script and all of them except one are old and have
not been added after 1.26. The only left is actually added but changed
in the mean time and is needed.

Also cleaning up checkIwlPrefix that was introduced in 1.17.

Bug: T272199
Change-Id: Ifc58c8d860ebc0e6d154203cafc6a64654bced92
2021-02-13 20:24:14 +01:00
Ammarpad
4fe97dfd8b Postgres: Drop some pre-1.25 upgrade schema updates (part 1)
These are not clearly marked per release as they're in MysqlUpdater
until we reach 1.23 release, and I am not sure whether they're truly
in chronological order. Some of the files dropped here exists for Mysql
and here is there corresponding versions there, the rest do not exist
there.

* patch-page_restrictions.sql --1.10
* patch-protected_titles.sql --1.12

* patch-page_props.sql --1.13
* patch-category.sql --1.13
* patch-updatelog.sql --1.13

* patch-change_tag.sql --1.15

* patch-user_properties.sql --1.16
* patch-log_search.sql --1.16
* patch-l10n_cache.sql -- 1.16

* patch-iwlinks.sql -- 1.17
* patch-module_deps.sql -- 1.17

* patch-uploadstash.sql -1.18
* patch-user_former_groups.sql --1.18

* patch-sites.sql --1.21

These have no corresponding files
* patch-querycachetwo.sql
* patch-redirect.sql

Bug: T272199
Change-Id: I42830ffe8eb3ccad9b1f55d9f2629b388806335b
2021-02-03 06:57:12 +01:00
Ammar Abdulhamid
20d1849b53 Migrate objectcache to abstract schema
For Postgres:
 - Drop Unique constraint on `keyname` and make primary key
 - Change type of `value` from BYTEA to TEXT and drop its default
 - Make `value` nullable to sync with MySQL/SQLite

MySQL:
 - Change exptime from DATETIME to TIMESTAMP

MySQL/SQLite:
 - Make 'exptime' not nullable

Bug: T230428
Bug: T164898
Change-Id: Iab9de8a1bb2cb01b6e3e69e66f1bbe089d53d0a7
2021-01-20 13:50:28 +01:00
Amir Sarabadani
c33876f834 Migrate text table to abstract schema
For MySQL/Sqlite, drop the table options, they are for MySQL < 5.0 which
was released in 2003 and not any use today.

For Postgres, rename pagecontent table to text, approved as part of the
RFC. Swap order of creating auto-generated tables and manual tables.

Bug: T230428
Bug: T164898
Change-Id: Id7510b80beed7a7297353094a57d5e2d4f12fc64
2021-01-09 16:07:54 +01:00
Amir Sarabadani
1476e3b5b5 Migrate categorylinks to abstract schema
To get MySQL/SQLite migrated, changing type of cl_to and
cl_sortkey_prefix from "varchar() binary" to varbinary

For Postgres:
 - Dropping foreign key on cl_from
 - Setting proper default for cl_from, cl_to, cl_sortkey, cl_collation
 - Set cl_sortkey to not nullable
 - Dropping unique index cl_from which in turn should become PK
 - Adding several missing indexes
 - Fixing columns of cl_sortkey

Bug: T230428
Bug: T164898
Change-Id: I83043ef41f9c753734f1e4b51237d979031dd71a
2020-12-03 07:43:02 +00:00