Commit graph

267 commits

Author SHA1 Message Date
Ammar Abdulhamid
c8e8449caf Migrate watchlist table to Abstract schema
Postgres:
 - Set empty string as default for wl_title to match MySQL
 - Drop foreign key from wl_user
 - Change wl_namespace datatype from SMALLINT to INT to match MySQL/SQLite
 - Drop PG-specific indexes and sync them with MySQL

Bug: T230428
Bug: T164898
Change-Id: Ie996b81ca59069443976b90d7bcdce29997d768a
2020-10-22 15:12:00 +02:00
Ammar Abdulhamid
6599e5706c Migrate querycache_info to abstract schema
Additional changes for Postgres:
 - Set empty string as default for qci_type
 - Set PG-equivalent of the given MySql/Sqlite timestamp as
   default timestamp for qci_timestamp
 - Make qci_type non nullable
 - Make qci_timestamp non nullable
 - Drop UNIQUE constraint on qci_type

Bug: T230428
Bug: T164898
Depends-On: If344395615087c360597a5b3d66ea03e930b7d9b
Change-Id: I741d2d079696d4b4eba09945341054d2a145bddc
2020-10-20 08:18:03 +01:00
Amir Sarabadani
4200969929 Migrate user_groups to abstract schema
For Postgres:
 - Dropping foreign key on ug_user
 - Setting default on ug_user and ug_group
 - Renaming indexes so they would be the same as MySQL/Sqlite

Bug: T42626
Bug: T164898
Bug: T230428
Change-Id: If507d8b8f363016760fe34aac2a0e0a2d99b66aa
2020-10-19 09:01:23 +02:00
Amir Sarabadani
ab8932638a Migrate page_restrictions to abstract schema
Sqlite doesn't need any change as binary/varbinary both turn into BLOB.

For Postgres:
 - Dropping foreign key on pr_page, as all FKs should be dropped (See
   T164898)
 - Add all of indexes, it seems PG was missing all four indexes of the
   MySQL version and didn't have any
 - Changing PK from (pr_page,pr_type) to (pr_id).
  - Had to add changePrimaryKey function to handle such cases, similar
    to checkIndex
 - Set pr_page to not nullable

Bug: T42626
Bug: T164898
Bug: T230428
Change-Id: Ied11f9c2ee633bf5ae325a6c2ea163dcc8d8d3c5
2020-10-19 09:00:26 +02:00
Amir Sarabadani
3d31e8d2d0 Migrate querycachetwo table to abstract schema
Yes, name of that table is really querycachetwo.

For migrating MySQL/Sqlite, changed data type of qcc_title and
qcc_titletwo to varbinary.

For migrating Postgres, just renaming indexes to stay in sync with
MySQL/Sqlite was enough.

Bug: T230428
Change-Id: I43952836ce184419622a6f851cb5e9319172d656
2020-09-26 16:39:53 +02:00
Amir Sarabadani
77f11539c2 Migrate querycache to abstract schema
Turning qc_title to varbinary in MySQL to make it consistent with the
rest of fields.

For Postgres:
 - Setting 0 as default for qc_value and qc_namespace
 - Changing data type of qc_namespace from SMALLINT to INT to make it
   the same with MySQL/Sqlite
 - Renaming index to make it the same with other schemas
 - Setting empty string as default of qc_title

Bug: T230428
Bug: T164898
Change-Id: I81527121609014d29182ae735a2c1f532d7d4d2a
2020-09-26 15:52:43 +02:00
Amir Sarabadani
3db2a91569
Migrate ipblocks_restrictions to abstract schema
Also dropping the fkey on ir_ipb_id in Postgres as approved by RFC T164898

Bug: T230428
Bug: T164898
Change-Id: Iaddec3b9992eee39fe82a45d5bda59bd51cde510
2020-09-22 16:01:45 -07:00
Amir Sarabadani
c0ee12dc86 Migrate change_tag_def table to abstract schema
Also fixing one schema data type drift between MySQL and Postgres
and fixing the problem with booleans, currently DBAL turns "boolean" to
TINYINT(1) in MySQL because it doesn't support it but it leave it like
that in Postgres which in turn breaks it because mediawiki currently
doesn't cast values properly, so for now just turning them to SMALLINT
which is the status quo (See T257755#6335566)

Bug: T230428
Bug: T164898
Change-Id: Ia495a451d51722496942b720a32393f7a6728dee
2020-09-17 10:21:47 +02:00
Amir Sarabadani
be4f90c7be Migrate watchlist_expiry to abstract schema
This takes advantage of custom data type registered in the previous
commit.

Also fixing Postgres drift by dropping the sequence, PK of this table is
not autoincrement so it shouldn't have sequence in Postgres.

Bug: T230428
Bug: T164898
Change-Id: I4c10990ead1e52c979254d2ac6a25ccf62a31508
2020-09-17 10:21:46 +02:00
Amir Sarabadani
b0c18769da Migrate category to abstract schema
In order to migrate MySQL and Sqlite to abstract schema changed the
cat_title data type from varchar binary to varbinary. This wouldn't
affect production.

For migrating Postgres, renamed two indexes from category_* to cat_* to
make it in sync with MySQL/Sqlite

Bug: T164898
Bug: T230428
Change-Id: Iad11aa4f7d809465cb20ac9748bf52b0e1bcd5a4
2020-09-05 20:49:12 +02:00
Amir Sarabadani
cd7a28d30f Migrate iwlinks to abstract schema
In order to migrate MySQL and Sqlite to abstract schema changed the
iwl_title data type from varchar binary to varbinary. This wouldn't
affect production.

For migrating Postgres:
 - Turning the unique index to PK to make it in sync with MySQL

Bug: T164898
Bug: T230428
Change-Id: Iaa625b66c874023b8cf2403917fa2fa120279208
2020-09-05 20:02:43 +02:00
Amir Sarabadani
1770341562 Migrate langlinks to abstract schema
In order to migrate MySQL and Sqlite to abstract schema changed the
ll_title data type from varchar binary to varbinary. This wouldn't
affect production.

For migrating Postgres:
 - Dropping foreign key on ll_from
 - Setting default of empty string for ll_lang and ll_title
 - Make ll_lang and ll_title both non-nullable to be in sync with MySQL
 - Turning the unique index to primary key, similar to MySQL
 - Renaming an index to sync with MySQL

Bug: T164898
Bug: T230428
Change-Id: I57f22896ff67266f99bf08f6dd1b9cc4c51b1db9
2020-09-05 19:29:23 +02:00
Amir Sarabadani
61eaa48f3c Migrate imagelinks table to abstract schema
In order to migrate MySQL and Sqlite to abstract schema changed the
il_to data type from varchar binary to varbinary. This wouldn't
affect production.

For migrating Postgres:
 - Dropping foreign key on il_from
 - Setting default of empty string for il_to
 - Completely rewriting indexes to make it synced with MySQL

Bug: T164898
Bug: T230428
Change-Id: I59f0d0a56d938a168bf1c7de2a1be47f15d1add1
2020-08-22 22:59:00 +02:00
Amir Sarabadani
aecc67b83d Migrate templatelinks table to abstract schema
In order to migrate MySQL and Sqlite to abstract schema changed the
tl_title data type from varchar binary to varbinary. This wouldn't
affect production.

For migrating Postgres:
 - Dropping foreign key on tl_from
 - Changing data type of tl_namespace from SMALLINT to INT and setting
   its default
 - Setting default of empty string for tl_title
 - Completely rewriting indexes to make it synced with MySQL

Bug: T164898
Bug: T230428
Change-Id: Idbb65d870f58f7146b9c8bd860e6530bef1e0f12
2020-08-22 15:23:03 +02:00
Amir Sarabadani
31dd3a9ebe Migrate pagelinks table to abstract schema
In order to migrate MySQL and Sqlite to abstract schema changed the
pl_title data type from varchar binary to varbinary. This wouldn't
affect production.

For migrating Postgres:
 - Dropping foreign key on pl_from
 - Changing data type of pl_namespace from SMALLINT to INT and setting
   its default
 - Setting default of empty string for pl_title
 - Completely rewriting indexes to make it synced with MySQL

Bug: T164898
Bug: T230428
Change-Id: I4af8202742a1826e6b3f8ff36cf4f7b612b82690
2020-08-19 13:58:00 +02:00
Amir Sarabadani
a531f4ad0b Migrate redirect table to abstract schema
Changing "varchar() binary" to varbinary in MySQL and Sqlite for two
columns: rd_title and rd_fragment.

Fixing db drifts of schema in Postgres:
 - Setting the data type of rd_namespace from SMALLINT and setting its
   default
 - Set empty string as default of rd_title
 - Changing datatype of rd_interwiki to varchar (PG supports that)

Bug: T164898
Bug: T230428
Change-Id: I9563792b1fc0ce5f3de78c08703a768a0c2e99d0
2020-08-13 00:37:45 +02:00
Amir Sarabadani
5e69ad1aad Migrate l10n_cache and module_deps to abstract schema
Also, fixing two data type drifts in Postgres:
 - lc_key was varchar (with no binary flag) in MySQL but TEXT in
   Postgres while PG supports varchar. Changing PG to varchar.
 - lc_value was BYTEA but the rest of varbinary fields are mapped to
   TEXT, changing it to TEXT data type.

And fixing primary key of these tables in Postgres too as it was a unique index instead.

Bug: T198811
Bug: T164898
Bug: T230428
Change-Id: I50305556bd870461d05f98c5272cf1d6a65deb15
2020-08-06 22:07:22 +02:00
Amir Sarabadani
e4bc792e6e Migrate content table to abstract schema
content.content_id was BIGINT in MySQL (and Sqlite) but INT in Postgres
so making Postgres BIGINT as well.

Bug: T230428
Bug: T164898
Change-Id: Ib8492e224478dc069ba17e489f7b2bb22d5da804
2020-08-03 00:46:01 +02:00
Amir Sarabadani
e464cc22ec Migrate change_tag to abstract schema
This is the first table that doesn't have any drift between MySQL and
Postgres.

Bug: T230428
Change-Id: I8be8906fc0f998c8aec79e148487219b3d43e57c
2020-08-03 00:46:01 +02:00
Amir Sarabadani
1799b9d09e Migrate log_search table to abstract schema
Also ls_value was varchar (with no binary flag) in MySQL but TEXT in
Postgres while PG supports varchar. Changing PG to varchar.

Bug: T230428
Bug: T164898
Change-Id: I2f7fae7b1e781021cb7d3ed39c40c2fc9d20a680
2020-08-03 00:46:01 +02:00
Amir Sarabadani
b345b8f6da Migrate user_properties to abstract schema
Also, dropping foreign key constraint on user_properties.up_user as it's decided to
drop all of foreign key constraints in T164898

And fixing primary key of the table in Postgres as it was a unique index
instead.

Bug: T198811
Bug: T164898
Bug: T230428
Change-Id: I60a2b65df62ec93a363309f8a17c29b83fd6f058
2020-07-31 17:49:50 +02:00
Amir Sarabadani
671b365394 Migrate site_stats table to abstract schema
Six fields were BIGINT in MySQL (and Sqlite) but INT in Postgres
so making Postgres BIGINT as well:
 - ss_total_edits
 - ss_good_articles
 - ss_total_pages
 - ss_users
 - ss_active_users
 - ss_images

Bug: T230428
Bug: T164898
Change-Id: I527b2797af3c8ba1a7402a4428b565f8e35cc132
2020-07-31 01:46:41 +02:00
Amir Sarabadani
aa3c07964c Migrate slots table to abstract schema
Three fields were BIGINT in MySQL (and Sqlite) but INT in Postgres
so making Postgres BIGINT as well:
 - slot_revision_id
 - slot_content_id
 - slot_origin

Bug: T230428
Bug: T164898
Change-Id: I5333cc0cbdc36356bd865ae118b883bc367c31eb
2020-07-21 18:05:41 +02:00
Amir Sarabadani
ac534bd119 Migrate comment table to abstract schema
comment.comment_id was BIGINT in MySQL (and Sqlite) but INT in Postgres
so making Postgres BIGINT as well.

Bug: T230428
Bug: T164898
Change-Id: I29a1878eea2adfd6e7d8ad85a94dc83718f9671e
2020-07-13 23:53:33 +02:00
Amir Sarabadani
b36210cae2 Migrate bot_passwords table to abstract schema
bot_passwords.bp_token didn't have a default in Postgres while it has
one in MySQL (and Sqlite), so adding the default to Postgres as well

Bug: T230428
Bug: T164898
Change-Id: I0ae4dbf8f2a5382081c6211c9cad51843000e3f1
2020-07-13 22:34:37 +02:00
Amir Sarabadani
5f03e0b5af Migrate actor and user_former_groups tables to abstract schema
Also during the abstraction, fixing the schema drift between MySQL
and Postgres in these two tables by removing foreign key, sequences,
and changing data types (as outlined in T164898)

Bug: T230428
Bug: T164898
Change-Id: If737a746629511b5a53d7ae70328fd558bd58d0e
2020-07-12 19:19:54 +02:00
Amir Sarabadani
3ad43ae06e Migrate site_identifiers to abstract schema
And converge the schema drift between mysql and postgres by setting all
to binary

Bug: T230428
Change-Id: I6f82b97c6e44c76ccdc0714aa8b32e959fbe896f
2020-06-14 18:37:30 +02:00
Amir Sarabadani
c1578fa017 Migrate updatelog to abstract schema
And fixing the schema drift between mysql and postgres in this table by
changing field type of ul_key and ul_value from TEXT and TEXT to
VARCHAR(255) and BYTEA respectively.

The automatic generation is done with running generateSchemaSql.php
maintenance script. More info:
https://www.mediawiki.org/wiki/Manual:Schema_changes#Automatically_generated

Bug: T230428
Bug: T164898
Change-Id: Id785539d32546166d6f7a5c3cb1924f4841a2963
2020-06-14 16:25:10 +02:00
jenkins-bot
74778ccd17 Merge "Remove ipb_anon_only from ipb_address_unique UNIQUE INDEX" 2020-06-06 21:24:21 +00:00
daniel
fe8b5fa79c make rev_actor_timestamp index cover the rev_id field.
This adds the rev_id column to the end of the rev_actor_timestamp index
to make it unambiguous. With the id field added, the index matches the
pagination criterion for user contributions (filter by actor, sort by
timestamp, then disambiguate based on revision id). The index is not
marked UNIQUE since this would potentially degrade write performance.
Uniqueness is already guaranteed by the rev_id field being the primary
key.

Note that no provisions are made to change the definition of the
rev_actor_timestamp index for existing instances. This index was only
recently added in I18071a2fe45907a0cf1b0fefebd96a97a2dacb7b and has not
been part of any release. It has also not yet been created on the wikimedia
servers. For this reason, any existing instances are assumed to be for
testing only. Instances would also continue to function normally with
the previous index definition.

With this patch, the new index will be created correctly when updating
from 1.34 or earlier. It will however not be modified for installations
of some development version of 1.35.

Bug: T200259
Bug: T238966
Change-Id: I511bb21b1ca820d950818cc831f8e3fef43a1559
2020-06-03 09:14:42 +02:00
Dayllan Maza
39448a9cf6 Remove ipb_anon_only from ipb_address_unique UNIQUE INDEX
Bug: T251188
Change-Id: I4618f659be64134664ac1644de2b51d74f000e4c
2020-06-02 17:15:36 -04:00
Brad Jorsch
c04732b987 Alter revision for actor, comment, and MCR
We're finally to the point of making the massive alter to the `revision`
table that we've been building up to for 2.5 years now.

Changes here are:
* Drop `rev_text_id`, `rev_content_model`, and `rev_content_format` that
  MCR obsoleted.
* Drop `ar_text_id`, `ar_content_model`, and `ar_content_format` that
  MCR obsoleted.
* Replace `rev_comment` with `rev_comment_id`.
* Replace `rev_user` and `rev_user_text` with `rev_actor`, plus
  associated index changes.

Future patches will make the code changes to migrate data from
`revision_actor_temp` and `revision_comment_temp` into the new
`revision` columns.

Bug: T251343
Bug: T184615
Bug: T215466
Change-Id: I18071a2fe45907a0cf1b0fefebd96a97a2dacb7b
2020-05-08 15:26:56 +00:00
Petr Pchelko
d6084c9dcd Fix Postgres user_newtalk table
In mysql the user_id and user_ip fields are not nullable,
but include a default 0 and '', however in Postgress the default
was not set, while user_id was set to not nullable. In the
code that uses this table, we set either user_id or user_ip
depending on whether the user is anon or registered. Thus both
fields should either be nullable, or contain a default.
Given that mysql has defaults set, I've opted for bringing
Postgress closer to what mysql does.

This was always broken and was discovered by adding new tests
for this functionality.

Bug: T239640
Change-Id: I75bf469a30225687a4b0cb550a4068cb07208c01
2020-05-04 11:17:03 -07:00
MusikAnimal
865755dbe5 Create new table for watchlist expiry
This table will store expiries for watchlist items, so that pages can be
automatically unwatched at a specific time.

Bug: T240094
Change-Id: I014ffccb91969e7a222a83858c55e71067b0f2af
2020-02-07 14:59:50 -05:00
Brad Jorsch
9d67de01cb Postgres: Add PK to redirect table
Apparently when the PG schema was created, this PK was overlooked.

Change-Id: I208f03d7b3f768691905081244193bad4b789175
2019-11-27 08:24:49 +00:00
Timo Tijhof
0da1ecfd89 profiler: Remove ProfilerOutputDb and profileinfo.php entry point
Bug: T231366
Change-Id: I211c8192200d9af00914f9847608a6daf2898f91
2019-11-06 15:28:00 -05:00
Brad Jorsch
c29909e59f Mostly drop old pre-actor user schemas
This removes most of the pre-actor user and user_text columns, and the
$wgActorTableSchemaMigrationStage setting that used to determine
whether the columns were used.

rev_user and rev_user_text remain in the code, as on Wikimedia wikis the
revision table is too large to alter at this time. A future change will
combine that with the removal of rev_comment, rev_content_model, and
rev_content_format (and the addition of rev_comment_id and rev_actor).

ActorMigration's constructor continues to take a $stage parameter, and
continues to have the logic for handling it, for the benefit of
extensions that might need their own migration process. Code using
ActorMigration for accessing the core fields should be updated to use
the new actor fields directly. That will be done for in a followup.

Bug: T188327
Change-Id: Id35544b879af1cd708f3efd303fce8d9a1b9eb02
2019-09-09 11:38:36 -04:00
Brad Jorsch
0abb9338f8 Mostly drop old comment schemas
This removes most of the pre-CommentStore text columns, and the
$wgCommentTableSchemaMigrationStage setting that used to determine
whether the columns were used.

rev_comment remains in the code, as on Wikimedia wikis the revision
table is too large to alter at this time. A future change will combine
that with the removal of rev_user_text, rev_content_model, and
rev_content_format (and the addition of rev_comment_id and rev_actor).

CommentStore's constructor continues to take a $stage parameter, and
continues to have the logic for handling it, for the benefit of
extensions that might need their own migration process.

Bug: T166733
Change-Id: I1479c73774e01ead1490adf6128f820c09bce9d4
2019-02-07 16:59:27 +11:00
Amir Sarabadani
8640b66568 Clean ups of dropping tag_summary table
Change-Id: I92dc906d0dfd870e6b77671e5d7cfd3f56090325
2018-12-19 10:14:33 +00:00
Amir Sarabadani
2ce92e90a7 Drop tag_summary and valid_tag tables
Bug: T209525
Change-Id: I153607239145a772f5ea47eefd3340096a494b4b
2018-12-18 20:45:36 +00:00
Kevin Israel
ba1207d5d5 PostgreSQL: Drop three unused columns
The columns dropped in this commit are:

* category.cat_hidden
  (for MySQL, dropped in 1.20 / 0cbc85494d)
* site_stats.ss_admins
  (for MySQL, dropped in 1.21 / bd602d2083)
* recentchanges.rc_cur_time
  (for MySQL, dropped in 1.24 / 2b441eba40)

Bug: T64103
Change-Id: I7a7e9a2ac5066a9c9ab2583419d507e14fdd8c2c
2018-12-14 13:01:48 -05:00
Amir Sarabadani
1427bb1821 Drop ct_tag column from change_tag table
Bug: T194163
Change-Id: I05d43510b51a67733b0107f0a768e261fe8ddabd
2018-11-28 23:03:43 +01:00
Brad Jorsch
d099bb6f95 Drop the image_comment_temp table
It is no longer used.

Bug: T188132
Change-Id: Ic8efeddc030f48e82ba861926121b64eca37d169
2018-11-14 15:04:31 -05:00
David Barratt
c2bd4b12c7 Make Schema changes for Partial Blocks
Update the database schema so a block can be switched between a sitewide and
partial block. Also add the restrictions table for specifiying the partial
blocks.

Bug: T197144
Change-Id: I4a725312c4b980a1b14e5ca826069fa2278a5913
2018-09-07 15:25:49 -07:00
Amir Sarabadani
da9d5c2188 Schema change for reading ct_tag_id instead of ct_tag
Adding unique index on ct_tag_id and making indexes on ct_tag non unique

Bug: T193874
Change-Id: I24609c57e47308d1330a97527f4ea374d0d307ba
2018-09-04 19:29:48 +02:00
Aaron Schulz
27a2aceb25 Drop the transcache table from the schema
Bug: T189702
Change-Id: I3286a99165953392126fcff07d565738863de6a1
2018-08-31 19:50:49 -07:00
Amir Sarabadani
ddd47d2808 Add index on rc_this_oldid
Approved by TechComm

Bug: T139012
Change-Id: I0ccfd26d68a5ceef552ae3aa37a6e345b24ff84e
2018-08-02 21:46:54 +02:00
Reedy
267d99fa85 Convert numerous UNIQUE INDEX to PRIMARY KEY
MySQL, SQLite, PostgreSQL and MSSQL done with transitional patches.

One additional duplicate index removed from PostgreSQL schema.

Having a PK is essential to do maintenance, specially on large tasks.
By not having a PK it is impossible to add it in a safe way if not done
directly on the master.

Having a PK means that we can easily change the PK into another one if
needed in the future. The ones we chose might not be the best ones, but
will allow us to get them changed.

Bug: T198811
Change-Id: I6b96a427687085c6c24bcd759c9739f81288b919
2018-07-11 20:36:33 +01:00
Reedy
b2101d7202 Rename index type_action to log_type_action
Bug: T51199
Change-Id: I697fd5689e573bfa965ca75385f919c9a8403c14
2018-07-11 19:07:21 +00:00
Brad Jorsch
c84083e413 Make archive.ar_rev_id unique
To follow up I39b0825c, this change replaces the existing non-unique
index on the column with a unique index, to help avoid some of these
sort of bugs in the future.

Bug: T193180
Change-Id: I932478c9c6a13210bc9dff75286d0f08da56682c
2018-06-04 08:52:06 +00:00