Commit graph

85 commits

Author SHA1 Message Date
Amir Sarabadani
080883da09 Schema: Set default or nullable to three columns of externallinks
We need to drop these columns and we need to make them take default
values so we can issue write queries without these columns.

Also noting that MySQL prior to 8.0 can't set default values to blob
columns making this way more complicated than it should be but MariaDB
can set them (https://mariadb.com/kb/en/blob/). We also made these
columns nullable to make this work in MySQL.

Bug: T341828
Change-Id: I0d60742b6ce7adf642393ee00b66aa539b76dfc1
2023-07-18 11:59:09 +02:00
Alexander Vorwerk
b3611755d7 Drop revision_comment_temp
Bug: T299954
Change-Id: I85d21b1eff70a7d70e8ce14f25d66f7e7c76e5fe
2023-06-07 15:34:57 +00:00
Thalia
f283c0e990 schema: Add user_is_temp column to the user table
This allows temporary users to be identified from applications
external to MediaWiki, by the user table alone (without referring
to the $AutoCreateTempUser['matchPattern'] config).

Bug: T333223
Change-Id: I83c5ff42654164590fb0361c84e65a5315ddbda8
2023-05-10 19:18:39 +01:00
Amir Sarabadani
080c70879a schema: Add new fields for externallinks so we can reduce duplication
Bug: T318604
Change-Id: I217817bc518eaa86c9952187c6f1a861f480ccaf
2022-10-18 16:18:54 +00:00
Amir Sarabadani
6c4194e23e schema: Drop tl_title and tl_namespace fields from templatelinks
The day has gone. Still keeping the code as the schema changes are not
done in production but the data migration has been finished.

Bug: T299417
Change-Id: I906e069a63d1dae14924c72318b22b16244371d6
2022-09-06 19:53:15 +02:00
Amir Sarabadani
692dde00df Add support for write new for templatelinks migration
- schema change to allow tl_namespace and tl_title being empty
   This is done by removing them from primary key. They don't need to be
   nullable as they have default value.
 - Make sure with WRITE_NEW, updater avoids writing to the old columns

Bug: T306674
Change-Id: I2b8a29043e952060e7a79b6a7a3d647d48cd16fb
2022-07-12 14:46:54 +02:00
Amir Sarabadani
ce0b6f40fa schema: Drop legacy page_restrictions in page table
Bug: T35334
Change-Id: I17e73d5ef165481a5dd4c210da933b99c65ff79c
2022-05-26 13:59:57 +02:00
Amir Sarabadani
24115a8f4c Start clean up of revision_actor_temp table
It is being dropped in production

Bug: T215466
Change-Id: I66b2cb8653252e720c897351065978119f040ba7
2022-05-23 15:37:42 +00:00
Tim Starling
e8dbf5f80c TempUser infrastructure and services
Add services and utilities for automatic creation of temporary user
accounts on page save, in order to avoid exposing the user's IP
address.

* Add $wgAutoCreateTempUser, for configuring the system
* Add TempUserConfig service, which interprets the config.
* Add TempUserCreator service, which creates users during page save as
  requested by EditPage. With proxy methods to TempUserConfig for
  convenience.
* Add table user_autocreate_serial. Table creation is necessary before
  the feature is enabled but is not necessary before deployment of this
  commit.

Bug: T300263
Change-Id: Ib14a352490fc42039106523118e8d021844e3dfb
2022-04-14 09:23:55 +10:00
Amir Sarabadani
9137db1d8a Add tl_target_id to templatelinks
Part of normalizing it, note that the field must be nullable now and
that will change later.

Bug: T299418
Change-Id: Id543dfa20a153312f66d2f45a64ac23e7272dabe
2022-01-28 17:41:01 +01:00
Amir Sarabadani
203d422dc4 Drop rev_page_id index on revision
This is already applied in production and known to be safe.

Bug: T163532
Change-Id: Ief29372f13b2d7cdb19395dcda6eb15e9a53efca
2022-01-27 09:43:27 +01:00
Amir Sarabadani
4a6e986141 Add linktarget table
Bug: T299416
Change-Id: Icae4513dd99635335857100d8a0c7102986933e5
2022-01-21 15:52:58 +01:00
Amir Sarabadani
ede308ed55 Drop pr_user from page_restrictions
It's not used for 13 years, safe to say it won't be used anytime soon

Bug: T199377
Change-Id: Iecf4fb6046699a1758ad2d1dc55a3ee8eb4b0389
2021-12-06 14:15:39 +01:00
Ammar Abdulhamid
1adaca51c3 Rename change_tag indexes to have ct_ prefix
Bug: T270033
Change-Id: I8a429726c99f6cadea0d671fd871f66b5611c856
2021-06-08 17:57:15 +01:00
Ammarpad
a8c01d7726 Rename name_title index to have page_ prefix
Bug: T270033
Change-Id: Id70d0e0a37dd0d000079820d51cef2791f5ec42e
2021-06-05 20:21:07 +02:00
Amir Sarabadani
7fdca7ce51 Add index on oldimage.oi_timestamp
Mirroring what happens on image table.

Bug: T279982
Change-Id: Ib67c32b10d3b88e09514d8fbb5dcb4f977108ba2
2021-05-29 20:44:32 +02:00
Amir Sarabadani
acd5076099 Migrate searchindex to abstract schema
For MySQL/Sqlite:
 - Nothing

For Postgres:
 - Introduce the table. It's not used but it'll be used in the future (T220450)

Bug: T164898
Bug: T230428
Bug: T220450
Change-Id: I9f33132676344f8cd813f7a438b3a6a078fd281c
2021-05-27 15:01:24 +02:00
Ammarpad
5954b380fc Migrate revision table to abstract schema
Postgres:
 - Drop foreign key from rev_page
 - Make rev_page not nullable
 - Change rev_comment_id from int to bigint
 - Change rev_actor from int to bigint
 - Sync rev_page_id index with MySQL

MySQL/SQlite:
 - Drop default from rev_timestamp

Additional changes in the generator script to handle more
formatting issues due to use of additional custom table options

Bug: T230428
Bug: T164898
Change-Id: Ia07dd52e43123473a1728523a3f863280537db8e
2021-05-20 14:52:15 -07:00
Aaron Schulz
12923c1cd5 objectcache: add last-modified token field to objectcache table
Also added token and flags fields. The token field can
be used as a tie-breaker for modtime and also for faster
cas() operations. The flags field makes serialization and
compression format changes easier in the future.

Bug: T274174
Change-Id: I45731a877b21835652993c2d285165a76eeae3e9
2021-05-18 20:06:37 -07:00
Ammarpad
88bfadf5ed Migrate user table to abstract schema
Postgres:
 - Rename the table from `mwuser` to `user`
 - Transition user_id from int to serial
 - Make user_token not nullable and add default
 - Make user_real_name not nullable and add default
 - Make user_email not nullable
 - Make user_newpassword not nullable
 - Make user_password not nullable
 - Drop UNIQUE contraint on user_name and add default

MySQL/SQLite:
 * No changes

Bug: T164898
Bug: T230428
Change-Id: I746714f7b3ae16f9625f97bcca84280fcd8b61a0
2021-04-24 21:44:26 +01:00
Ammar Abdulhamid
6a3aa5b5a2 Migrate page to abstract schema
Postgres:
 - Change page_namespace from smallint to int
 - Change page_random from numeric with arbitrary precision to float
 - Make page_touched not nullable

MySQL/SQLite:
 - Change datatype of page_title from varchar (with binary collation)
   to varbinary(255)
 - Drop default empty string from timestamp field of page_touched

Bug: T230428
Bug: T164898
Change-Id: Ibdaf332ea1da309d31d35a6ebbc1b8fefced335e
2021-03-21 12:07:12 +01:00
Amir Sarabadani
2cc79854e8 Migrate archive table to abstract schema
One of the last ones left.

For MySQL/Sqlite:
 - Dropping default of ar_timestamp, empty string is not a valid
   timestamp.
 - Changing ar_title from "varchar() binary" to varbinary

for Postgres:
 - Set default for ar_namespace and ar_title
 - Change datatype of ar_comment_id, ar_actor, ar_namespace
The indexes were fixed separately.

Bug: T230428
Bug: T164898
Bug: T42626
Depends-On: I83cf1cd51ac9cf933c9175cefd6e38a6914f3494
Change-Id: Ic1d13a82b27f7fa39a0f0ea9c5b7b193b007e4ab
2021-03-13 21:51:16 +01:00
Amir Sarabadani
1053405664 Rename new_name_timestamp on recentchanges to rc_new_name_timestamp
To make it have a uniform prefix for index and column names

Bug: T270033
Change-Id: I8eb600416913092bd5aeb70389bba6e8a54d1d57
2021-03-01 19:00:52 +00:00
Amir Sarabadani
4a6d4baaed Migrate recentchanges table to abstract schema
This table is massive but thankfully we fixed most of its complexities
in previous patches.

For MySQL/Sqlite:
 - Change type of rc_title and rc_source from "varchar binary" to
   "varbinary"
 - Drop default of rc_timestamp
 - Change rc_timestamp from varbinary(14) to binary(14) to standardize
   timestamp datatypes

One index doesn't follow the uniform prefix rule but since it's in a
maintenance script, will fix that in a follow up.

Bug: T230428
Bug: T42626
Change-Id: I13994e02ad3a2293148346ef7be96746578ad854
2021-02-26 13:56:56 +01:00
Ammar Abdulhamid
dbd4dd19f8 Rename site_identifiers indexes to have si_ prefix
Bug: T270033
Change-Id: I6751f0fd992054b61222ece55c83d05d24af9000
2021-01-30 12:19:55 +01:00
Amir Sarabadani
fe6ff91134 Migrate image table to abstract schema
For MySQL/Sqlite:
 - Change datatype of img_name from "varchar() binary" to varbinary.
 - Drop default of img_timestamp

For Postgres:
 - Adding two missing indexes.
 - Renaming two indexes
 - Setting default value for five fields
 - Fix data type of four fields
 - Drop default of img_metadata
 - Make three fields not nullable

Bug: T230428
Bug: T164898
Change-Id: I237af3558b0e1c1fecd874c3c90ba6780e50aaa4
2021-01-28 20:13:03 +01:00
Amir Sarabadani
6e53acb7cc Migrate ipblocks to abstract schema
For MySQL/Sqlite:
 - Drop default values of ipb_timestamp and ipb_expiry as part of
   standardizing timestamp columns

For Postgres:
 - Drop foreign key on two columns as approved by the RFC
 - Set default of ipb_user
 - Make three columns not nullable to sync with MySQL
 - Change data type of two columns to BIGINT to be the same as MySQL

Bug: T230428
Bug: T164898
Bug: T42626
Change-Id: I2c5303d76c6ce059d7fef324a4521c6336c5b1f3
2021-01-23 18:21:24 +00:00
Ammar Abdulhamid
20d1849b53 Migrate objectcache to abstract schema
For Postgres:
 - Drop Unique constraint on `keyname` and make primary key
 - Change type of `value` from BYTEA to TEXT and drop its default
 - Make `value` nullable to sync with MySQL/SQLite

MySQL:
 - Change exptime from DATETIME to TIMESTAMP

MySQL/SQLite:
 - Make 'exptime' not nullable

Bug: T230428
Bug: T164898
Change-Id: Iab9de8a1bb2cb01b6e3e69e66f1bbe089d53d0a7
2021-01-20 13:50:28 +01:00
Amir Sarabadani
dba59bc223 Migrate oldimage to abstract schema
For MySQL/Sqlite:
 - Changing oi_name and oi_archive_name to varbinary instead of "varchar
   binary"
 - Dropping default from oi_timestamp

For Postgres:
 - Adding the correct default to seven fields
 - Removing incorrect default from oi_metadata
 - Add missing index oi_actor_timestamp
 - Change four fields to have correct data type
 - Make oi_major_mime and oi_minor_mime not nullable
 - Drop foreign key on oi_name

Bug: T230428
Bug: T164898
Change-Id: I4dff0e48081a5797a29513f9c8b87f9ffcaadb5d
2021-01-16 21:18:25 +01:00
Amir Sarabadani
c33876f834 Migrate text table to abstract schema
For MySQL/Sqlite, drop the table options, they are for MySQL < 5.0 which
was released in 2003 and not any use today.

For Postgres, rename pagecontent table to text, approved as part of the
RFC. Swap order of creating auto-generated tables and manual tables.

Bug: T230428
Bug: T164898
Change-Id: Id7510b80beed7a7297353094a57d5e2d4f12fc64
2021-01-09 16:07:54 +01:00
Ammarpad
bbda7591e1 Use TIMESTAMPTZ consistenly in Postgres SQL schema
'TIMESTAMPTZ' is an alias for the canonical name of the data type
'TIMESTAMP(0) WITH TIME ZONE' which though descriptive, looks rather
convoluted. Let's use the alias for consistency throughout.

Bug: T230428
Follow-up: Iabbe8f085812acd90536a06fee2ecd4db9b047d5
Change-Id: I277658b1ebd851466022e7874579cf507684d40a
2021-01-03 02:10:20 +01:00
Amir Sarabadani
f9ccfa7cef Bring back timestamp time of cl_timestamp instead of binary(14)
Fixing this is going to be complicated. We have to first turn this into
varbinary(20), write a maintenance script to fix the values, then make sure
mediawiki work properly with two value types at the same time and then
turning it to binary(14).

For now, let's unblock 1.36 release, we will follow up on fixing this
for good later.

Bug: T270032
Change-Id: Iabbe8f085812acd90536a06fee2ecd4db9b047d5
2021-01-01 13:23:59 +00:00
Amir Sarabadani
37fc2d1ba1 Migrate filearchive to abstract schema
For MySQL:
 - Drop default from fa_deleted_timestamp and fa_timestamp
 - Change fa_name and fa_storage_group from varchar() binary to
   varbinary()

For Postgres:
 - Set default for three fields
 - Drop foreign key on fa_deleted_user
 - Change field type for five fields
 - Rename indexes to make it make in sync with MySQL/Postgres
 - Rebuild an index to make it in sync MySQL/Postgres

Bug: T230428
Bug: T164898
Bug: T42626
Change-Id: Ia2be151c46c73f15592db03540834a6456460cac
2020-12-23 04:04:27 +01:00
Amir Sarabadani
443c94de3f Rename four logging indexes to have log_ prefix
To follow database coding convention and avoid clashes in RDBMS engines
where indexes are global

Bug: T270033
Change-Id: I17028bba489d04b01d22c39dc8935a4f0f771c7e
2020-12-21 07:39:50 +00:00
Ammar Abdulhamid
9907b56c9b Rename all sites indexes
Use uniform prefix with the column names

Bug: T270033
Change-Id: I771ffc67a1363d8877c57dce1d8b2acdca8fe5bb
2020-12-16 03:17:06 +01:00
Ammar Abdulhamid
58dee4465c Rename user_properties index
Use uniform prefix with the column names

Bug: T270033
Change-Id: I9ee1f8ac9a8d0e575e16344910495f341c2714db
2020-12-15 04:37:08 +01:00
Amir Sarabadani
5f374fa3f8 Migrate uploadstash to abstract schema
For MySQL:
 - Migrating us_timestamp from varbinary(14) to binary(14) to
   standardize timestamp fields

For postgres:
 - Fixing datatype of 8 fields of the table
 - Set "NOT NULL" for 7 fields
 - Rename indexes to drop _idx suffix

Bug: T230428
Bug: T164898
Bug: T42626
Change-Id: I5391416ae463856cf5e9b05f542dec9b389b2f7c
2020-12-13 19:12:36 +01:00
Ammar Abdulhamid
b9094b0248 Migrate logging to abstract schema
Some changes needed for the migration:

Postgres:
 - Add default to five fields that have it in MySQL
 - Make log_params not nullable
 - Change log_comment_id and log_actor type to big integer
 - Change log_namespace to integer
 - Drop logging_actor_time index

MySQL/SQLite
 - Change log_title to varbinary/BLOB

Bug: T230428
Bug: T164898
Change-Id: Iad59d2a78f61d93637917ca320d3e173863ce5e1
2020-12-11 21:40:28 +00:00
Amir Sarabadani
1476e3b5b5 Migrate categorylinks to abstract schema
To get MySQL/SQLite migrated, changing type of cl_to and
cl_sortkey_prefix from "varchar() binary" to varbinary

For Postgres:
 - Dropping foreign key on cl_from
 - Setting proper default for cl_from, cl_to, cl_sortkey, cl_collation
 - Set cl_sortkey to not nullable
 - Dropping unique index cl_from which in turn should become PK
 - Adding several missing indexes
 - Fixing columns of cl_sortkey

Bug: T230428
Bug: T164898
Change-Id: I83043ef41f9c753734f1e4b51237d979031dd71a
2020-12-03 07:43:02 +00:00
Ammar Abdulhamid
b922a292c9 Fix strange formatting issues in Postgres schema sql
Mere presence of partial index's WHERE clause is disrupting anything
that comes after it leading to strange and non-uniform formatting

Change-Id: I068113b3026d6c3422348e86b3048234b905bb1d
2020-11-24 11:47:18 +01:00
Amir Sarabadani
a2a0088d6b Migrate slot_roles and content_models to abstract schema
These tables don't have drift between MySQL and Postgres but their
primary keys don't have explicit "NOT NULL" statement making them
nullable in sqlite. Fixing this drift.

Also changing the PK fields of these two tables from smallint to int,
given that PG would be complicated with smallint auto_increment PKs

Bug: T230428
Bug: T258366
Change-Id: Icf6ce044eaf0f09b1a2bdd8a1f618cef1e0415bf
2020-11-22 00:24:08 +01:00
Ammar Abdulhamid
40ebc05ce4 Migrate job to abstract schema
Postgres:
 - Drop deafault from job_id
 - Add default to job_cmd
 - Change job_namespace datatype to INTEGER
 - Drop  job_cmd_namespace_title index
 - Add job_cmd index on the same fields as MySQL
 - Rename index job_timestamp_idx

MySQL:
 - Change job_title datatype to VARBINARY
 - Change job_timestamp datatype to BINARY (mwtimestamp)
 - Change job_token_timestamp datatype to BINARY (mwtimestamp)

Bug: T230428
Bug: T164898
Bug: T42626
Change-Id: I207aefc48c7bcbb5b5362af4b63d9a1383019a6d
2020-11-21 23:38:28 +01:00
Amir Sarabadani
69d7bf0cfe Fix un_user_ip index column
This happened during the abstraction (Idc3ded012890d).
Given that the patch was merged recently,
I don't think we need database patches for it.

Bug: T230428
Change-Id: Ia8c529bc502ed1674476f71f67c80003a2ae83cf
2020-11-21 23:07:26 +01:00
Ammar Abdulhamid
b9717c8bf8 Migrate page_props to abstract schema
For Postges:
 - Drop foreign key from 'pp_page'
 - Move pk addition from the SQL file to the Updater class
 - Override default SQL value for float to return FLOAT which is in use.

For MySQL:
 - Override default SQL value for float to return FLOAT which is in use
 Hack added in generateSchemaSql.php

Bug: T230428
Bug: T164898
Change-Id: I3c591c9b8f43647b0208690bca79f2d0dd206922
2020-11-18 06:20:32 +01:00
Amir Sarabadani
db4315b942 Rename namespace_title index on watchlist to wl_namespace_title
Indexes need to have prefixes, to avoid clashes in several RDBMSes where
indexes are global

Bug: T266228
Change-Id: I96a04903f6123d1fb8081fcf15cdd1f29708d320
2020-11-16 15:24:24 +01:00
Ammar Abdulhamid
16ebaa8ce4 Migrate revision_actor_temp to abstract schema
For Postgres:
 - Drop foreign key from 'revactor_page'
 - Change datatype of 'revactor_actor' to BigInt as in MySQL/SQLite
 - Make 'revactor_page' non-nullable as in MySQL/SQLite

For MySQL/SQLite:
 - Drop empty string default constraint from 'revactor_timestamp' as
 this is not allowed in the corresponding PG's TIMESTAMPTZ field

Bug: T230428
Bug: T164898
Change-Id: I8c5c4b338a3000b1e4c2ab82fdae4a9819925868
2020-11-10 12:27:40 +01:00
Ammar Abdulhamid
01cd9e25d7 Migrate revision_comment_temp to abstract schema
For Postgres:
 - Change 'revcomment_comment_id' to BigInt as it's in MySQL/SQLite

Bug: T230428
Bug: T164898
Change-Id: I4b5af210b63d932e2eb0120075a5fa5250924ace
2020-11-10 07:35:10 +01:00
Amir Sarabadani
b14f9ead33 Migrate ip_changes to abstract schema
In order to make Postgres work:
 - Dropped the autoincrement sequence, this is the PK without
   autoincrement
 - Change ipc_hex from BYTEA to TEXT
 - Set default for ipc_rev_id

MySQL/SQLite:
 -Drop default for ipc_rev_timestamp

Bug: T230428
Bug: T164898
Change-Id: I7f85d65f6ee3ac8b1d28e33095f37dce81e1727e
2020-11-05 22:38:50 +01:00
Ammar Abdulhamid
bafdce78b1 Migrate externallinks to abstract schema
Postgres changes:
 - Drop foreign key from `el_from`
 - Change 'el_index_60 type from BYTEA to TEXT
 - Set default for `el_from` to sync with MySQL
 - Also renamed these indexes to sync with MySQL
   - 'externallinks_from_to' -> 'el_from'
   - 'externallinks_index' -> 'el_index'
 - Add index 'el_to' on 'el_to' and 'el_from' columns. This completes
   the PG indexes to 5 to match up MySQL indexes

Bug: T230428
Bug: T164898
Change-Id: I1ab9bde19e456256db99d1244130e8ee11f9b6c8
2020-11-05 19:00:24 +01:00
Ammar Abdulhamid
7fbcee5748 Migrate protected_titles to abstract schema
For Postgres:
 - Drop foreign key from pt_title
 - Change pt_user to non-nullable to sync with MySQL
 - Change pt_expiry to non-nullable to sync with MySQL
 - Modify pt_reason_id to use BIGINT to sync with MySQL
 - Drop default from pt_create_perm field since MySQL and SQLite
don't have it and the field is not nullable.
 -

For MySQL/SQLite:
 - Modify pt_title to use varbinary
 - Drop DEFAULT constraint from pt_expiry (Postgres already does not have it)

Bug: T230428
Bug: T164898
Change-Id: Iff193754260046222ba9b7e704c46e27f21b1a6b
2020-11-02 15:05:23 +01:00