Commit graph

412 commits

Author SHA1 Message Date
Amir Sarabadani
fa7b284d70 Remove all old index checks in PG
I checked all of them with https://github.com/wikimedia/mediawiki/blob/REL1_26/includes/installer/PostgresUpdater.php
using a script and all of them except one are old and have
not been added after 1.26. The only left is actually added but changed
in the mean time and is needed.

Also cleaning up checkIwlPrefix that was introduced in 1.17.

Bug: T272199
Change-Id: Ifc58c8d860ebc0e6d154203cafc6a64654bced92
2021-02-13 20:24:14 +01:00
Amir Sarabadani
6809067ef3 Fix drifts in columns of recentchanges in PG
To make abstracting this table easier. Reducing the schema drift between
schema of this table in PG and MySQL:
 - Changing datatype of four fields
 - Set default for seven fields
 - Change nullability of two fields.

Bug: T230428
Bug: T164898
Change-Id: Id6fa9fd1ebf840f6354fcd8b8ebf0e393ba9e624
2021-02-06 01:19:00 +01:00
Ammarpad
4fe97dfd8b Postgres: Drop some pre-1.25 upgrade schema updates (part 1)
These are not clearly marked per release as they're in MysqlUpdater
until we reach 1.23 release, and I am not sure whether they're truly
in chronological order. Some of the files dropped here exists for Mysql
and here is there corresponding versions there, the rest do not exist
there.

* patch-page_restrictions.sql --1.10
* patch-protected_titles.sql --1.12

* patch-page_props.sql --1.13
* patch-category.sql --1.13
* patch-updatelog.sql --1.13

* patch-change_tag.sql --1.15

* patch-user_properties.sql --1.16
* patch-log_search.sql --1.16
* patch-l10n_cache.sql -- 1.16

* patch-iwlinks.sql -- 1.17
* patch-module_deps.sql -- 1.17

* patch-uploadstash.sql -1.18
* patch-user_former_groups.sql --1.18

* patch-sites.sql --1.21

These have no corresponding files
* patch-querycachetwo.sql
* patch-redirect.sql

Bug: T272199
Change-Id: I42830ffe8eb3ccad9b1f55d9f2629b388806335b
2021-02-03 06:57:12 +01:00
Ammar Abdulhamid
dbd4dd19f8 Rename site_identifiers indexes to have si_ prefix
Bug: T270033
Change-Id: I6751f0fd992054b61222ece55c83d05d24af9000
2021-01-30 12:19:55 +01:00
Amir Sarabadani
fe6ff91134 Migrate image table to abstract schema
For MySQL/Sqlite:
 - Change datatype of img_name from "varchar() binary" to varbinary.
 - Drop default of img_timestamp

For Postgres:
 - Adding two missing indexes.
 - Renaming two indexes
 - Setting default value for five fields
 - Fix data type of four fields
 - Drop default of img_metadata
 - Make three fields not nullable

Bug: T230428
Bug: T164898
Change-Id: I237af3558b0e1c1fecd874c3c90ba6780e50aaa4
2021-01-28 20:13:03 +01:00
Amir Sarabadani
7f6c51617b Standardize archive indexes
For Sqlite and MySQL, renaming name_title_timestamp to
ar_name_title_timestamp so it follows the database convention.

For Postgres:
 - Rename archive_name_title_timestamp to ar_name_title_timestamp
 - Drop archive_actor (that has only ar_actor)
 - Add ar_actor_timestamp (that has ar_actor, ar_timestamp)

This fixes drifts of PG and MySQL and makes work for abstracting the
table easier.

Bug: T230428
Bug: T164898
Change-Id: I123a4c47602bdd415f0fe5823f96159dcd0d84b8
2021-01-28 11:32:49 +01:00
Amir Sarabadani
6e53acb7cc Migrate ipblocks to abstract schema
For MySQL/Sqlite:
 - Drop default values of ipb_timestamp and ipb_expiry as part of
   standardizing timestamp columns

For Postgres:
 - Drop foreign key on two columns as approved by the RFC
 - Set default of ipb_user
 - Make three columns not nullable to sync with MySQL
 - Change data type of two columns to BIGINT to be the same as MySQL

Bug: T230428
Bug: T164898
Bug: T42626
Change-Id: I2c5303d76c6ce059d7fef324a4521c6336c5b1f3
2021-01-23 18:21:24 +00:00
Ammar Abdulhamid
20d1849b53 Migrate objectcache to abstract schema
For Postgres:
 - Drop Unique constraint on `keyname` and make primary key
 - Change type of `value` from BYTEA to TEXT and drop its default
 - Make `value` nullable to sync with MySQL/SQLite

MySQL:
 - Change exptime from DATETIME to TIMESTAMP

MySQL/SQLite:
 - Make 'exptime' not nullable

Bug: T230428
Bug: T164898
Change-Id: Iab9de8a1bb2cb01b6e3e69e66f1bbe089d53d0a7
2021-01-20 13:50:28 +01:00
Amir Sarabadani
31513848ef Fix Postgres index drifts on recentchanges table
To make migrating recentchanges table to abstract schema easier

Bug: T230428
Bug: T164898
Change-Id: I727dfc6b644701bd99d49628dfeeaa0ca46f168c
2021-01-16 23:15:54 +01:00
Amir Sarabadani
dba59bc223 Migrate oldimage to abstract schema
For MySQL/Sqlite:
 - Changing oi_name and oi_archive_name to varbinary instead of "varchar
   binary"
 - Dropping default from oi_timestamp

For Postgres:
 - Adding the correct default to seven fields
 - Removing incorrect default from oi_metadata
 - Add missing index oi_actor_timestamp
 - Change four fields to have correct data type
 - Make oi_major_mime and oi_minor_mime not nullable
 - Drop foreign key on oi_name

Bug: T230428
Bug: T164898
Change-Id: I4dff0e48081a5797a29513f9c8b87f9ffcaadb5d
2021-01-16 21:18:25 +01:00
Amir Sarabadani
c33876f834 Migrate text table to abstract schema
For MySQL/Sqlite, drop the table options, they are for MySQL < 5.0 which
was released in 2003 and not any use today.

For Postgres, rename pagecontent table to text, approved as part of the
RFC. Swap order of creating auto-generated tables and manual tables.

Bug: T230428
Bug: T164898
Change-Id: Id7510b80beed7a7297353094a57d5e2d4f12fc64
2021-01-09 16:07:54 +01:00
Amir Sarabadani
5cfdb13c0f Add missing PG indexes of ipblocks
To make the work of abstracting this table easier.

Bug: T230428
Bug: T164898
Change-Id: I259c99aec7b6df27269fe02172ae4905ecfd29c2
2021-01-09 15:01:51 +01:00
Ammarpad
bbda7591e1 Use TIMESTAMPTZ consistenly in Postgres SQL schema
'TIMESTAMPTZ' is an alias for the canonical name of the data type
'TIMESTAMP(0) WITH TIME ZONE' which though descriptive, looks rather
convoluted. Let's use the alias for consistency throughout.

Bug: T230428
Follow-up: Iabbe8f085812acd90536a06fee2ecd4db9b047d5
Change-Id: I277658b1ebd851466022e7874579cf507684d40a
2021-01-03 02:10:20 +01:00
Amir Sarabadani
f9ccfa7cef Bring back timestamp time of cl_timestamp instead of binary(14)
Fixing this is going to be complicated. We have to first turn this into
varbinary(20), write a maintenance script to fix the values, then make sure
mediawiki work properly with two value types at the same time and then
turning it to binary(14).

For now, let's unblock 1.36 release, we will follow up on fixing this
for good later.

Bug: T270032
Change-Id: Iabbe8f085812acd90536a06fee2ecd4db9b047d5
2021-01-01 13:23:59 +00:00
Amir Sarabadani
37fc2d1ba1 Migrate filearchive to abstract schema
For MySQL:
 - Drop default from fa_deleted_timestamp and fa_timestamp
 - Change fa_name and fa_storage_group from varchar() binary to
   varbinary()

For Postgres:
 - Set default for three fields
 - Drop foreign key on fa_deleted_user
 - Change field type for five fields
 - Rename indexes to make it make in sync with MySQL/Postgres
 - Rebuild an index to make it in sync MySQL/Postgres

Bug: T230428
Bug: T164898
Bug: T42626
Change-Id: Ia2be151c46c73f15592db03540834a6456460cac
2020-12-23 04:04:27 +01:00
Amir Sarabadani
443c94de3f Rename four logging indexes to have log_ prefix
To follow database coding convention and avoid clashes in RDBMS engines
where indexes are global

Bug: T270033
Change-Id: I17028bba489d04b01d22c39dc8935a4f0f771c7e
2020-12-21 07:39:50 +00:00
Ammar Abdulhamid
9907b56c9b Rename all sites indexes
Use uniform prefix with the column names

Bug: T270033
Change-Id: I771ffc67a1363d8877c57dce1d8b2acdca8fe5bb
2020-12-16 03:17:06 +01:00
Ammar Abdulhamid
58dee4465c Rename user_properties index
Use uniform prefix with the column names

Bug: T270033
Change-Id: I9ee1f8ac9a8d0e575e16344910495f341c2714db
2020-12-15 04:37:08 +01:00
Amir Sarabadani
5f374fa3f8 Migrate uploadstash to abstract schema
For MySQL:
 - Migrating us_timestamp from varbinary(14) to binary(14) to
   standardize timestamp fields

For postgres:
 - Fixing datatype of 8 fields of the table
 - Set "NOT NULL" for 7 fields
 - Rename indexes to drop _idx suffix

Bug: T230428
Bug: T164898
Bug: T42626
Change-Id: I5391416ae463856cf5e9b05f542dec9b389b2f7c
2020-12-13 19:12:36 +01:00
Ammar Abdulhamid
b9094b0248 Migrate logging to abstract schema
Some changes needed for the migration:

Postgres:
 - Add default to five fields that have it in MySQL
 - Make log_params not nullable
 - Change log_comment_id and log_actor type to big integer
 - Change log_namespace to integer
 - Drop logging_actor_time index

MySQL/SQLite
 - Change log_title to varbinary/BLOB

Bug: T230428
Bug: T164898
Change-Id: Iad59d2a78f61d93637917ca320d3e173863ce5e1
2020-12-11 21:40:28 +00:00
Ammar Abdulhamid
cb77a1121f Rename all PG logging indexes
with exception of logging_actor_time which does not exist
in MySQL and thus will be removed in migrating the table to
abstract schema

New names unify the indexes with MySQL/SQLite names

Bug: T164898
Change-Id: I90bec59b86cdd527d77cc6084056ab4267cc5b40
2020-12-07 08:00:47 +01:00
Amir Sarabadani
1476e3b5b5 Migrate categorylinks to abstract schema
To get MySQL/SQLite migrated, changing type of cl_to and
cl_sortkey_prefix from "varchar() binary" to varbinary

For Postgres:
 - Dropping foreign key on cl_from
 - Setting proper default for cl_from, cl_to, cl_sortkey, cl_collation
 - Set cl_sortkey to not nullable
 - Dropping unique index cl_from which in turn should become PK
 - Adding several missing indexes
 - Fixing columns of cl_sortkey

Bug: T230428
Bug: T164898
Change-Id: I83043ef41f9c753734f1e4b51237d979031dd71a
2020-12-03 07:43:02 +00:00
jenkins-bot
ea77501778 Merge "Fix strange formatting issues in Postgres schema sql" 2020-11-26 21:33:54 +00:00
Ammar Abdulhamid
b922a292c9 Fix strange formatting issues in Postgres schema sql
Mere presence of partial index's WHERE clause is disrupting anything
that comes after it leading to strange and non-uniform formatting

Change-Id: I068113b3026d6c3422348e86b3048234b905bb1d
2020-11-24 11:47:18 +01:00
Ammar Abdulhamid
b7f35505d0 Fix some PG page table indexes drift with MySQL
There are many changes in migrating page to abstract schema,
so I split this out to simplify it a bit.

 - Rename index: page_len_idx -> page_len
 - Rename index: page_random_idx -> page_random
 - Rename index: page_unique_name -> name_title
 - Add index: page_redirect_namespace_len

Bug: T164898
Change-Id: I8193860129f1134f5dad7d633d069077ee597465
2020-11-23 07:21:21 +01:00
Amir Sarabadani
a2a0088d6b Migrate slot_roles and content_models to abstract schema
These tables don't have drift between MySQL and Postgres but their
primary keys don't have explicit "NOT NULL" statement making them
nullable in sqlite. Fixing this drift.

Also changing the PK fields of these two tables from smallint to int,
given that PG would be complicated with smallint auto_increment PKs

Bug: T230428
Bug: T258366
Change-Id: Icf6ce044eaf0f09b1a2bdd8a1f618cef1e0415bf
2020-11-22 00:24:08 +01:00
Ammar Abdulhamid
40ebc05ce4 Migrate job to abstract schema
Postgres:
 - Drop deafault from job_id
 - Add default to job_cmd
 - Change job_namespace datatype to INTEGER
 - Drop  job_cmd_namespace_title index
 - Add job_cmd index on the same fields as MySQL
 - Rename index job_timestamp_idx

MySQL:
 - Change job_title datatype to VARBINARY
 - Change job_timestamp datatype to BINARY (mwtimestamp)
 - Change job_token_timestamp datatype to BINARY (mwtimestamp)

Bug: T230428
Bug: T164898
Bug: T42626
Change-Id: I207aefc48c7bcbb5b5362af4b63d9a1383019a6d
2020-11-21 23:38:28 +01:00
Amir Sarabadani
69d7bf0cfe Fix un_user_ip index column
This happened during the abstraction (Idc3ded012890d).
Given that the patch was merged recently,
I don't think we need database patches for it.

Bug: T230428
Change-Id: Ia8c529bc502ed1674476f71f67c80003a2ae83cf
2020-11-21 23:07:26 +01:00
Ammar Abdulhamid
b9717c8bf8 Migrate page_props to abstract schema
For Postges:
 - Drop foreign key from 'pp_page'
 - Move pk addition from the SQL file to the Updater class
 - Override default SQL value for float to return FLOAT which is in use.

For MySQL:
 - Override default SQL value for float to return FLOAT which is in use
 Hack added in generateSchemaSql.php

Bug: T230428
Bug: T164898
Change-Id: I3c591c9b8f43647b0208690bca79f2d0dd206922
2020-11-18 06:20:32 +01:00
Amir Sarabadani
db4315b942 Rename namespace_title index on watchlist to wl_namespace_title
Indexes need to have prefixes, to avoid clashes in several RDBMSes where
indexes are global

Bug: T266228
Change-Id: I96a04903f6123d1fb8081fcf15cdd1f29708d320
2020-11-16 15:24:24 +01:00
jenkins-bot
d009d75c73 Merge "Start generateSchemaChangeSql.php" 2020-11-16 08:40:36 +00:00
Ammar Abdulhamid
16ebaa8ce4 Migrate revision_actor_temp to abstract schema
For Postgres:
 - Drop foreign key from 'revactor_page'
 - Change datatype of 'revactor_actor' to BigInt as in MySQL/SQLite
 - Make 'revactor_page' non-nullable as in MySQL/SQLite

For MySQL/SQLite:
 - Drop empty string default constraint from 'revactor_timestamp' as
 this is not allowed in the corresponding PG's TIMESTAMPTZ field

Bug: T230428
Bug: T164898
Change-Id: I8c5c4b338a3000b1e4c2ab82fdae4a9819925868
2020-11-10 12:27:40 +01:00
Ammar Abdulhamid
01cd9e25d7 Migrate revision_comment_temp to abstract schema
For Postgres:
 - Change 'revcomment_comment_id' to BigInt as it's in MySQL/SQLite

Bug: T230428
Bug: T164898
Change-Id: I4b5af210b63d932e2eb0120075a5fa5250924ace
2020-11-10 07:35:10 +01:00
Amir Sarabadani
b14f9ead33 Migrate ip_changes to abstract schema
In order to make Postgres work:
 - Dropped the autoincrement sequence, this is the PK without
   autoincrement
 - Change ipc_hex from BYTEA to TEXT
 - Set default for ipc_rev_id

MySQL/SQLite:
 -Drop default for ipc_rev_timestamp

Bug: T230428
Bug: T164898
Change-Id: I7f85d65f6ee3ac8b1d28e33095f37dce81e1727e
2020-11-05 22:38:50 +01:00
Ammar Abdulhamid
bafdce78b1 Migrate externallinks to abstract schema
Postgres changes:
 - Drop foreign key from `el_from`
 - Change 'el_index_60 type from BYTEA to TEXT
 - Set default for `el_from` to sync with MySQL
 - Also renamed these indexes to sync with MySQL
   - 'externallinks_from_to' -> 'el_from'
   - 'externallinks_index' -> 'el_index'
 - Add index 'el_to' on 'el_to' and 'el_from' columns. This completes
   the PG indexes to 5 to match up MySQL indexes

Bug: T230428
Bug: T164898
Change-Id: I1ab9bde19e456256db99d1244130e8ee11f9b6c8
2020-11-05 19:00:24 +01:00
Ammar Abdulhamid
7fbcee5748 Migrate protected_titles to abstract schema
For Postgres:
 - Drop foreign key from pt_title
 - Change pt_user to non-nullable to sync with MySQL
 - Change pt_expiry to non-nullable to sync with MySQL
 - Modify pt_reason_id to use BIGINT to sync with MySQL
 - Drop default from pt_create_perm field since MySQL and SQLite
don't have it and the field is not nullable.
 -

For MySQL/SQLite:
 - Modify pt_title to use varbinary
 - Drop DEFAULT constraint from pt_expiry (Postgres already does not have it)

Bug: T230428
Bug: T164898
Change-Id: Iff193754260046222ba9b7e704c46e27f21b1a6b
2020-11-02 15:05:23 +01:00
Amir Sarabadani
892d308989 Migrate interwiki table to abstract schema
For MySQL, turning bool to tinyint(1) but doesn't matter because bool is
an alias of tinyint(1)

For Postgres:
 - Changing type of iw_prefix and iw_wikiid to varchar(32)
   and varchar(64) respectively to make it in sync with Mysql.
 - Dropping add_interwiki function, it doesn't serve any purpose (search
   didn't bring anything)

Bug: T230428
Bug: T164898
Change-Id: Id652cb544ac1b216f2fad1a8a959739ead8a92cd
2020-11-02 06:01:00 +00:00
Amir Sarabadani
1125167b8d Start generateSchemaChangeSql.php
Creating schema changes from abstract schema and even abstracting a
schema change.

Bug: T230420
Change-Id: If626e866642af820dd70c5f9b0fe7c6a951e0a25
2020-11-01 21:14:53 +01:00
Ammar Abdulhamid
74c0553250 Migrate new_talk to abstract schema
For MySQL
 - Modify user_last_timestamp from VARBINARY to BINARY

For Postgres
 - Drop foreign key on user_id
 - Rename indexes to match MySQL convention

Bug:  T230428
Change-Id: Idc3ded012890d4ace805a646d75cd805b49b250c
2020-10-26 14:10:24 +01:00
jenkins-bot
ae9b8c3d29 Merge "Drop all "DROP SEQUENCE" in Postgres" 2020-10-23 20:23:38 +00:00
Amir Sarabadani
a9dcbe3585 Migrate sites table to abstract schema
For MySQL/Sqlite, using mwtinyint instead of bool, in reality it doesn't
matter since bool is an alias of tinyint(1) in MySQL.

In Postgres:
 - Changing type of site_domain from TEXT to VARCHAR to make it in sync
   with MySQL/Sqlite
 - Renaming all indexes from site_* to sites_* to be in sync with
   MySQL/Sqlite

Bug: T230428
Bug: T164898
Change-Id: Id73bb372807730446901d772fcdbd2f333b397ba
2020-10-22 19:35:39 +02:00
Ammar Abdulhamid
c8e8449caf Migrate watchlist table to Abstract schema
Postgres:
 - Set empty string as default for wl_title to match MySQL
 - Drop foreign key from wl_user
 - Change wl_namespace datatype from SMALLINT to INT to match MySQL/SQLite
 - Drop PG-specific indexes and sync them with MySQL

Bug: T230428
Bug: T164898
Change-Id: Ie996b81ca59069443976b90d7bcdce29997d768a
2020-10-22 15:12:00 +02:00
Ammar Abdulhamid
6599e5706c Migrate querycache_info to abstract schema
Additional changes for Postgres:
 - Set empty string as default for qci_type
 - Set PG-equivalent of the given MySql/Sqlite timestamp as
   default timestamp for qci_timestamp
 - Make qci_type non nullable
 - Make qci_timestamp non nullable
 - Drop UNIQUE constraint on qci_type

Bug: T230428
Bug: T164898
Depends-On: If344395615087c360597a5b3d66ea03e930b7d9b
Change-Id: I741d2d079696d4b4eba09945341054d2a145bddc
2020-10-20 08:18:03 +01:00
Amir Sarabadani
709df8f234 Drop all "DROP SEQUENCE" in Postgres
tables.sql never was meant as an idempotent script or something that can
be ran on an existing database or schema. This serves no purposes and
can cause issues if tables-generated.sql gets to run first (by dropping
sequences that just got created), currently this doesn't happen and
installer, first installs manual tables but it can change in the future.

Change-Id: I299a0ae1e49f415910892981c07cfd582d198598
2020-10-19 09:21:35 +02:00
Amir Sarabadani
4200969929 Migrate user_groups to abstract schema
For Postgres:
 - Dropping foreign key on ug_user
 - Setting default on ug_user and ug_group
 - Renaming indexes so they would be the same as MySQL/Sqlite

Bug: T42626
Bug: T164898
Bug: T230428
Change-Id: If507d8b8f363016760fe34aac2a0e0a2d99b66aa
2020-10-19 09:01:23 +02:00
Amir Sarabadani
ab8932638a Migrate page_restrictions to abstract schema
Sqlite doesn't need any change as binary/varbinary both turn into BLOB.

For Postgres:
 - Dropping foreign key on pr_page, as all FKs should be dropped (See
   T164898)
 - Add all of indexes, it seems PG was missing all four indexes of the
   MySQL version and didn't have any
 - Changing PK from (pr_page,pr_type) to (pr_id).
  - Had to add changePrimaryKey function to handle such cases, similar
    to checkIndex
 - Set pr_page to not nullable

Bug: T42626
Bug: T164898
Bug: T230428
Change-Id: Ied11f9c2ee633bf5ae325a6c2ea163dcc8d8d3c5
2020-10-19 09:00:26 +02:00
Amir Sarabadani
3d31e8d2d0 Migrate querycachetwo table to abstract schema
Yes, name of that table is really querycachetwo.

For migrating MySQL/Sqlite, changed data type of qcc_title and
qcc_titletwo to varbinary.

For migrating Postgres, just renaming indexes to stay in sync with
MySQL/Sqlite was enough.

Bug: T230428
Change-Id: I43952836ce184419622a6f851cb5e9319172d656
2020-09-26 16:39:53 +02:00
Amir Sarabadani
77f11539c2 Migrate querycache to abstract schema
Turning qc_title to varbinary in MySQL to make it consistent with the
rest of fields.

For Postgres:
 - Setting 0 as default for qc_value and qc_namespace
 - Changing data type of qc_namespace from SMALLINT to INT to make it
   the same with MySQL/Sqlite
 - Renaming index to make it the same with other schemas
 - Setting empty string as default of qc_title

Bug: T230428
Bug: T164898
Change-Id: I81527121609014d29182ae735a2c1f532d7d4d2a
2020-09-26 15:52:43 +02:00
Amir Sarabadani
3db2a91569
Migrate ipblocks_restrictions to abstract schema
Also dropping the fkey on ir_ipb_id in Postgres as approved by RFC T164898

Bug: T230428
Bug: T164898
Change-Id: Iaddec3b9992eee39fe82a45d5bda59bd51cde510
2020-09-22 16:01:45 -07:00
Amir Sarabadani
c0ee12dc86 Migrate change_tag_def table to abstract schema
Also fixing one schema data type drift between MySQL and Postgres
and fixing the problem with booleans, currently DBAL turns "boolean" to
TINYINT(1) in MySQL because it doesn't support it but it leave it like
that in Postgres which in turn breaks it because mediawiki currently
doesn't cast values properly, so for now just turning them to SMALLINT
which is the status quo (See T257755#6335566)

Bug: T230428
Bug: T164898
Change-Id: Ia495a451d51722496942b720a32393f7a6728dee
2020-09-17 10:21:47 +02:00