Commit graph

253 commits

Author SHA1 Message Date
Brian Wolff
528042054f Fix some phan-taint-check false positives
Change-Id: Ic5ccbb3f97722476bee7188b83b80cdc652d2a64
2018-09-21 17:27:44 -07:00
David Barratt
c2bd4b12c7 Make Schema changes for Partial Blocks
Update the database schema so a block can be switched between a sitewide and
partial block. Also add the restrictions table for specifiying the partial
blocks.

Bug: T197144
Change-Id: I4a725312c4b980a1b14e5ca826069fa2278a5913
2018-09-07 15:25:49 -07:00
Amir Sarabadani
da9d5c2188 Schema change for reading ct_tag_id instead of ct_tag
Adding unique index on ct_tag_id and making indexes on ct_tag non unique

Bug: T193874
Change-Id: I24609c57e47308d1330a97527f4ea374d0d307ba
2018-09-04 19:29:48 +02:00
jenkins-bot
2f86f8dbb9 Merge "Don't throw an exception when waiting for replication times out" 2018-09-04 02:20:12 +00:00
Tim Starling
e8df0fbab1 Don't throw an exception when waiting for replication times out
For maintenance scripts it is usually harmful to throw an exception.
For jobs the exception was already caught and handled appropriately,
so this can continue as before. For DeferredUpdates it was extremely
harmful to throw an exception. So in the web case, reduce the timeout to
1s and continue as normal if the 1s timeout is reached. This allows the
DeferredUpdate to be throttled without being killed.

In the updater, increase the replication wait timeout to 5 minutes.
ALTER TABLE could indeed cause replication lag, but exiting the update
script with an exception will probably ruin your day. Update actions are
not necessarily efficiently restartable.

Do not call JobQueue::waitForBackups() when jobs are popped. Maybe it
makes sense to call a queue-specific replication wait function for
bulk inserts, like copyJobQueue.php, but doing it when jobs are popped
just makes no sense. Surely the worst that could happen is that the
queue would become locally empty? Removing this waitForBackups() call
avoids waiting for replication twice when JobQueueDB is used.

Bug: T201482
Change-Id: Ia820196caccf9c95007aea12175faf809800f084
2018-09-03 12:29:35 +10:00
Aaron Schulz
27a2aceb25 Drop the transcache table from the schema
Bug: T189702
Change-Id: I3286a99165953392126fcff07d565738863de6a1
2018-08-31 19:50:49 -07:00
jenkins-bot
4444f35d1d Merge "Add index on rc_this_oldid" 2018-08-17 20:45:43 +00:00
Aryeh Gregor
90d4f56fe4 Mass conversion of $wgContLang to service
Brought to you by vim macros.

Bug: T200246
Change-Id: I79e919f4553e3bd3eb714073fed7a43051b4fb2a
2018-08-11 22:44:29 -06:00
Amir Sarabadani
ddd47d2808 Add index on rc_this_oldid
Approved by TechComm

Bug: T139012
Change-Id: I0ccfd26d68a5ceef552ae3aa37a6e345b24ff84e
2018-08-02 21:46:54 +02:00
Timo Tijhof
9c166a8379 installer: Add missing new-line after "comment fields are up to date"
Before:
> ...comment fields are up to date...hitcounter doesn't exist.
> ...site_stats table does not contain ss_total_views field.

After:
> ...comment fields are up to date.
> ...hitcounter doesn't exist.
> ...site_stats table does not contain ss_total_views field.

Change-Id: Ia99758f9af379a84c20d99e05838802965def3b6
2018-07-12 21:16:48 -07:00
Reedy
267d99fa85 Convert numerous UNIQUE INDEX to PRIMARY KEY
MySQL, SQLite, PostgreSQL and MSSQL done with transitional patches.

One additional duplicate index removed from PostgreSQL schema.

Having a PK is essential to do maintenance, specially on large tasks.
By not having a PK it is impossible to add it in a safe way if not done
directly on the master.

Having a PK means that we can easily change the PK into another one if
needed in the future. The ones we chose might not be the best ones, but
will allow us to get them changed.

Bug: T198811
Change-Id: I6b96a427687085c6c24bcd759c9739f81288b919
2018-07-11 20:36:33 +01:00
Reedy
b2101d7202 Rename index type_action to log_type_action
Bug: T51199
Change-Id: I697fd5689e573bfa965ca75385f919c9a8403c14
2018-07-11 19:07:21 +00:00
Brad Jorsch
d5713fc655 Run populateContentTables.php from update.php
Since update.php will eventually be dropping the old columns and fields,
we should make sure the migration script gets run first.

Bug: T197818
Change-Id: I47f5ba4e21b4fe2ef0beb9fc83896cd4b0a0d505
2018-07-10 13:13:26 -04:00
Brad Jorsch
5c380739b2 Avoid recreating ar_revid index after it's replaced by ar_revid_uniq
Bug: T193180
Change-Id: I274e33de0a348c0ee42b08b349272db7e2151647
2018-06-05 11:35:03 -04:00
Brad Jorsch
c84083e413 Make archive.ar_rev_id unique
To follow up I39b0825c, this change replaces the existing non-unique
index on the column with a unique index, to help avoid some of these
sort of bugs in the future.

Bug: T193180
Change-Id: I932478c9c6a13210bc9dff75286d0f08da56682c
2018-06-04 08:52:06 +00:00
Amir Sarabadani
f45314cd4f Add ct_tag_id field to change_tag table
Part of normalizing the table

Bug: T193867
Change-Id: I8c38e673aab14c5b87e5501da1a752c5bd4709c8
2018-05-18 16:37:04 +02:00
Brad Jorsch
1c7bf1baf0
Deduplicate archive.ar_rev_id
Old bugs and such may have left the archive table with multiple rows
using the same ar_rev_id, or rows that also exist in the revision table.
These need to be cleaned up for MCR.

The maintenance script added here will delete rows that appear to be
duplicates of the same change, and will assign new IDs to rows that do
not appear to be duplicates.

Bug: T193180
Change-Id: I39b0825c9469e074ded3df33a4f06a1ef0edb494
2018-05-16 14:34:11 +02:00
Brad Jorsch
aef72c5cf6 Populate externallinks.el_index_60 and drop default
Adds a maintenance script to populate the field, has that be
automatically run during update.php, and drops the no-longer-needed
default value on the column (where possible: mssql has some sort of
constraint thing going on that I have no idea how it works).

Bug: T59176
Change-Id: I971edf013a1a39466aca3b6e34c915cb24fd3aa7
2018-05-15 12:08:35 -04:00
Amir Sarabadani
6039593649 Introduce change_tag_def table
Table defining tag names for IDs. Also stores hit counts to avoid expensive queries on change_tag
See T185355 for more information

Bug: T193867
Bug: T185355
Change-Id: I4fd943589b3ed304471304c8beda15327a8edbcf
2018-05-08 18:48:52 +02:00
Amir Sarabadani
f303a13a72 Change index on rc_namespace, rc_title to rc_namespace, rc_title, rc_timestamp
Bug: T190444
Change-Id: I5c59f1ee9306710505654a8db965f85ebf855873
2018-04-04 09:08:28 +02:00
jenkins-bot
1a9ba829a5 Merge "Populate ar_rev_id and make it non-nullable" 2018-04-03 15:37:05 +00:00
jenkins-bot
dd942172f8 Merge "DB: Add img_description_id column" 2018-04-03 01:18:42 +00:00
Brad Jorsch
f78faf4566 Populate ar_rev_id and make it non-nullable
Revisions deleted before MediaWiki 1.5 do not have a value in this
field. This is going to be a problem for migration to the MCR schema, so
provide a maintenance script to clean this up.

Then, for good measure, change the schema to make the field
non-nullable.

Bug: T182678
Change-Id: Ie2e11f12a30f379db32c3e074658012c6f93adb0
2018-04-02 13:08:55 -04:00
Eddie Greiner-Petter
27c76fa4ae Overhaul site_stats table
The site stats table holds a bunch of metric fields, two of which are of
data type "bigint unsigned", 3 are "bigint" (signed) and one is int
(signed).  Also the default values differ widely: It is 0 on the
"unsigned" fields and the "int" field, but -1 on the three others. This
patch makes all of this more consistent:

Set all fields (except the ss_row_id, which isn't changed) data type to
"bigint unsigned". Also set NULL as the default value for all those
fields. Obviously -1 isn't a possible default value any more. Also, 0
can easily be mistaken for a real value (e.g. ss_active_users=0 -->
"there is nobody active on this wiki"). NULL, by it's definition, is the
value of choice for a value to insert into fields of which we don't know
a correct value.

The respective patch files were tested locally against MySql, Sqlite,
Postgres and SQL Server 2016. Neither oracle nor the upgrade with
update.php was tested.

Bug: T56888
Change-Id: I7d42aae434852a56b6f8dd559d8a5f3bce416021
2018-03-26 23:51:40 +00:00
jenkins-bot
9ced9ebca2 Merge "MCR: replace slot_inherited with slot_origin" 2018-03-20 12:06:32 +00:00
daniel
b0bdfd7e9f MCR: replace slot_inherited with slot_origin
Bug: T189004
Change-Id: Ie9dbda3296a71f584c82a5f275098adc225a53d5
2018-03-19 21:51:56 +01:00
addshore
8d15ade672 Add default for revision.rev_text_id where missing
This adds a default of 0 for:
 - Mssql
 - Mysql
 - Sqlite

The field is already NULLable in:
 - Oracle
 - Postgres

Bug: T188741
Change-Id: Ibe1cc363d21fb96f38c263181bf30a429a1c7b45
2018-03-19 18:20:27 +00:00
Brad Jorsch
267af09d32 DB: Add img_description_id column
This begins the process of merging image_comment_temp into the image
table by adding the needed column. Iab5f5215 will adjust the code
to use it and to add the necessary migration script.

Note this patch puts the new schema change in the 1.30 section rather
than the 1.31 section. This allows Iab5f5215 to have migrateComments.php
migrate the comments directly to the new field instead of having to
populate and then depopulate the temporary table.

Bug: T188132
Change-Id: I2485c5a758bf03bb2b4991eea920abd9d0d30bda
2018-03-13 16:54:25 -04:00
Brad Jorsch
99b65649c0 rdbms: allow callers to hint that native insertSelect() is safe
An INSERT SELECT in MySQL/MariaDB is unsafe for replication if a column
is getting values from auto-increment, statement-based replication is in
use, and the default innodb_autoinc_lock_mode is set.

I9173f655 added checks to force non-native insertSelect for the
statement-based replication and innodb_autoinc_lock_mode != 2 case, but
determining whether a column is getting values from auto-increment is
too hard to do automatically there.

Instead, let's add a flag to let the caller hint that the query isn't
getting any auto-increment values. And use it in MysqlUpdater when
appropriate.

Bug: T160993
Change-Id: If70450a64aa3bcbf763c62838bb21306d124ae3d
2018-02-28 13:58:37 -05:00
Brad Jorsch
27c61fb1e9 Add actor table and code to start using it
Storing the user name or IP in every row in large tables like revision
and logging takes up space and makes operations on these tables slower.
This patch begins the process of moving those into one "actor" table
which other tables can reference with a single integer field.

A subsequent patch will remove the old columns.

Bug: T167246
Depends-On: I9293fd6e0f958d87e52965de925046f1bb8f8a50
Change-Id: I8d825eb02c69cc66d90bd41325133fd3f99f0226
2018-02-23 10:06:20 -08:00
Umherirrender
3124a990a2 Use ::class to resolve class names in includes files
This helps to find renamed or misspelled classes earlier.
Phan will check the class names

Change-Id: I07a925c2a9404b0865e8a8703864ded9d14aa769
2018-01-27 20:34:29 +01:00
WMDE-Fisch
a7872722b7 Replace deprecated wfGetLB() calls here and there
Change-Id: Ic7cec2dcc6b8e0d500a4c37eb134976b314c33ca
2018-01-20 19:24:24 +00:00
Brad Jorsch
295b055ea3 Migrate ar_text to modern storage
This really should have been done a very long time ago.

This adds a maintenance script to migrate rows with ar_text_id null to
modern storage, either the text table or ExternalStore.

Bug: T36925
Change-Id: I5608c6b6d3ecad516b785e13d668427c1b762e41
2018-01-09 16:15:44 -05:00
daniel
943c724198 MCR database schema
Schema additions/updates for MySQL, SQLite, PostgreSQL, MSSQL
and Oracle to add new tables:

* content
* content_models
* slots
* slot_roles

See also https://www.mediawiki.org/wiki/Multi-Content_Revisions/Database_Schema

Bug: T174028
Change-Id: I30a3a9834d54d0e6957553d91908b2b73b2c802f
2017-12-23 17:43:09 -05:00
Reedy
ce65e570c5 installer: Reuse $db->makeList() rather than reinventing the wheel
Change-Id: Ib1f896e5bed17bc240fe585fd6d5f753995a12e1
2017-10-07 00:36:34 +00:00
jenkins-bot
d75a576e26 Merge "build: Updating mediawiki/mediawiki-codesniffer to 13.0.0" 2017-09-30 23:44:57 +00:00
jenkins-bot
52ce204415 Merge "Fix patchfile used to add newtalk table on mysql" 2017-09-29 05:31:59 +00:00
Reedy
c16b1e8191 Fix patchfile used to add newtalk table on mysql
Remove orghaned patch-drop-user_newtalk.sql

Change-Id: I27ad5a628deedbab2e659200f003cdeb18ea3620
2017-09-26 21:25:51 +00:00
Umherirrender
bd741b83bc build: Updating mediawiki/mediawiki-codesniffer to 13.0.0
Change-Id: Ia24dbf015f2b4781683ca980a460d0ac3e85674e
2017-09-25 17:31:56 +02:00
jenkins-bot
4c15a41b33 Merge "Remove return bool from MysqlUpdater::doInterwikiUpdate" 2017-09-19 16:41:31 +00:00
Reedy
e6a77bb2d4 Don't unconditionally run patch-editsummary-length.sql
Bug: T176041
Change-Id: I165ee4fa1c0cfadf5f8f400a0ea9db220ed7dbbb
2017-09-18 14:27:59 +00:00
Umherirrender
6b95fdecc2 Remove return bool from MysqlUpdater::doInterwikiUpdate
All return values unequal to false are treated as "update done",
so return void is okay.

Change-Id: I553f6acb450a5fe1708aec641181f9a07da1ac39
2017-09-16 20:39:18 +00:00
Reedy
f4022fe876 Making numerous fields unsigned to match their target referenced column
Bug: T157227
Change-Id: Ic11822d6c893103adcc99a3e188a592f13c968b8
2017-09-06 19:34:09 +01:00
tjlsangria
93eef8d2c5 Convert INDEX into PRIMARY KEY in l10n_cache
Bug: T146591
Change-Id: I85ab18bfa30e562e76253e3ef71653dbf46db041
2017-09-05 17:58:12 +00:00
Reedy
575a886a46 Remove patch-log_search-rename-index.sql from MW 1.16 updaters
Otherwise, the PK on log_search keeps getting dropped, ls_field_val readded and
this is then repeated on every run of update.php

Change-Id: I6b00933c29b5e24ec2833001fcf45053292d190c
Follows-up: Id635297838938c7c5dfe65d45285a4d16d65152d
2017-09-01 17:44:49 +01:00
Brad Jorsch
11cf01dd9a Add comment table and code to start using it
A subsequent patch will remove the old columns.

Bug: T166732
Change-Id: Ic3a434c061ed6e443ea072bc62dda09acbeeed7f
2017-08-30 15:05:00 +10:00
Reedy
0f13fff160 Convert UNIQUE keys into PRIMARY KEY
WMF DBAs have been doing a massive effort to convert UNIQUE KEYS into
PRIMARY KEY.

Having a PK is essential to do maintenance, specially on large tasks.
By not having a PK it is impossible to add it in a safe way if not done
directly on the master.

Having a PK means that we can easily change the PK into another one if
needed in the future. The ones we chose might not be the best ones, but
will allow us to get them changed.

Bug: T172514
Change-Id: Id635297838938c7c5dfe65d45285a4d16d65152d
2017-08-29 18:25:37 +01:00
MusikAnimal
d779a09035 Create new table to be used for querying IP ranges.
When revisions by logged out users are created, they will get copied
to this table. We can then JOIN it with revision when querying for
ranges at Special:Contributions.

DBA approval for this table can be found at T156318

Bug: T163562

Change-Id: I6593eb13701128faa782691a6b25ec01869c827d
2017-08-10 21:28:16 +00:00
Matthias Mullie
6260545fee Follow-up to '3D' file type addition
Based on feedback on Id78a504302fa664b353a4c45bdc5d99f95a0180a

Bug: T157348
Change-Id: Id6bfe8d20307f62e4673a5ef30e79f164b1ea034
2017-07-28 23:46:33 +00:00
Mark Holmquist
a2f5d05ae8 Add 3D filetype for STL files
Other filetypes use the text/plain MIME type, so we'll have to do something
else to support them.

Bug: T157348
Change-Id: Id78a504302fa664b353a4c45bdc5d99f95a0180a
2017-04-17 17:01:52 +02:00