Commit graph

88 commits

Author SHA1 Message Date
jenkins-bot
bd78869618 Merge "No yoda conditions" 2018-12-09 01:34:23 +00:00
jenkins-bot
023fec5d7b Merge "[MCR] Introduce SlotRoleHandler and SlotRoleRegistry" 2018-12-03 16:20:06 +00:00
daniel
db987c700a [MCR] Introduce SlotRoleHandler and SlotRoleRegistry
These new classes provide a mechanism for defining the
behavior of slots, like the content models it supports.
This acts as an extension point for extensions that need
to define custom slots, like the MediaInfo extension
for the SDC project.

Bug: T194046
Change-Id: Ia20c98eee819293199e541be75b5521f6413bc2f
2018-11-30 12:29:05 -08:00
Brad Jorsch
fd5a1842d2 Default $wgCommentTableSchemaMigrationStage to MIGRATION_NEW
WRITE_NEW mode seems to be working well on Wikimedia sites. Let's change
the default to MIGRATION_NEW so existing installs and Wikimedia CI will
start using the new code rather than the old.

This also fixes some unit tests that were broken with MIGRATION_NEW, and
updates some that were forcing MIGRATION_OLD to force MIGRATION_NEW
instead.

Bug: T166733
Change-Id: I7bf4ad0105dd1f6cc49eba3ddcb7a51badcd5ed3
Depends-On: I30f7cdcc3875f3f7af116c1e41e88f62ab9e91d0
2018-11-28 13:59:19 -05:00
Fomafix
3ee1560232 No yoda conditions
Replace
  if ( 42 === $foo )
by
  if ( $foo === 42 )

Change-Id: Ice320ef1ae64a59ed035c20134326b35d454f943
2018-11-21 17:54:39 +01:00
daniel
7d8a959543 Allow getRevisionText to function without the text table.
Without this patch, getRevisionText would fail silently (by
returning false) when the text table no longer gets joined,
due to the switch to the new MCR schema.

Bug: T205808
Change-Id: Iffc25c82a5d2b865c28070c76156d39d390cc675
2018-10-17 10:54:39 -04:00
Brad Jorsch
993baa3493 ActorMigration: Remove possibility of read-both
When this was originally written, the plan was to read both the old and
new fields during the transition period, while stopping writes to them
midway through. It turns out that the WHERE conditions to do read-both
correctly are generally not handled well by the database and working
around that would require a lot of complicated code (see what's being
removed from ApiQueryUserContribs here, for example).

We can simplify things greatly by instead having it write both fields
during the transition period, reading from the old for the first part
and the new for the second part, as is being done for MCR.

Bug: T204669
Change-Id: I4764c1c7883dc1003cb12729455c8107319f70b1
Depends-On: I845f6ae462f2539ebd35cbb5f2ca8b5714e2c1fb
Depends-On: I88b31b977543fdbdf69f8c1158e77e448df94e11
2018-10-11 12:12:00 +11:00
Aaron Schulz
fe0af6cad5 rdbms: Database::selectDB() update the domain and handle failure better
LoadBalancer uses Database::getDomainId() for deciding which keys to use
in the foreign connection handle arrays. This method should reflect any
changes made to the DB selection.

If the query fails, then do not change domain field. This is the sort of
approach that LoadBalancer is expects in openForeignConnection(). Also,
throw an exception when selectDB() fails.

The db/schema/prefix fields of Database no longer exist in favor of just
using the newer currentDomain field.

Also:
* Add IDatabase::selectDomain() method and made selectDB() wrap it.
* Extract the DB name from sqlite files if not explicitly provided.
* Fix inconsistent open() return values from Database subclasses.
* Make a relationSchemaQualifier() method to handle the concern of
  omitting schema names in queries. The means that getDomainId() can
  still return the right value, rather than confusingly omitt the schema.
* Make RevisionStore::checkDatabaseWikiId() account for the domain schema.
  Unlike d2a4d614fc, this does not incorrectly assume the storage is
  always for the current wiki domain. Also, LBFactorySingle sets the local
  domain so it is defined even in install.php.
* Make RevisionStoreDbTestBase actually set the LoadBalancer local domain.
* Make RevisionTest::testLoadFromTitle() account for the domain schema.

Bug: T193565
Change-Id: I6e51cd54c6da78830b38906b8c46789c79498ab5
2018-10-10 12:03:30 -07:00
Brad Jorsch
dff469a408 Re-namespace RevisionStore and RevisionRecord classes
During development a lot of classes were placed in MediaWiki\Storage\.
The precedent set would mean that every class relating to something
stored in a database table, plus all related value classes and such,
would go into that namespace.

Let's put them into MediaWiki\Revision\ instead. Then future classes
related to the 'page' table can go into MediaWiki\Page\, future classes
related to the 'user' table can go into MediaWiki\User\, and so on.

Note I didn't move DerivedPageDataUpdater, PageUpdateException,
PageUpdater, or RevisionSlotsUpdate in this patch. If these are kept
long-term, they probably belong in MediaWiki\Page\ or MediaWiki\Edit\
instead.

Bug: T204158
Change-Id: I16bea8927566a3c73c07e4f4afb3537e05aa04a5
2018-10-09 10:22:48 -04:00
Gergő Tisza
6e8d39c6e7
Add constant for the name of the 'main' slot for MCR
Bug: T202142
Change-Id: I97a74e5a029b014f3c2195188936d5c8233c1b7f
2018-09-24 16:52:12 -07:00
daniel
452c71663b Consolidate tests for getQueryInfo() and related methods.
This consolidates tests for getQueryInfo, getArchiveQueryInfo,
getSlotQueryInfo, and similar methods that help application logic
be compatible with different migration stages of different aspects
of the revision storage schema.

Bug: T198561
Change-Id: I8e4ae69d7e00721a0af125afaf9a708f7fe99b0a
2018-09-10 18:16:46 +00:00
daniel
53fd8295ff Use consistent caching strategy in Revision storage classes
DEPLOYMENT: This changes the cache key for revision
content blobs. Expect a brief rise in ExternalStore hits.

Bug: T198704
Change-Id: Icc2d16bc5a1e27ba4caea49a784ba7aeac15042a
2018-07-05 17:44:24 +02:00
jenkins-bot
284bc0b5eb Merge "When encountering bad blobs, log the text row id." 2018-06-19 12:32:44 +00:00
daniel
2f9e892c74 When encountering bad blobs, log the text row id.
Change-Id: Ie7c932f229854fd3582d507fa7cc154ffdd21f55
2018-06-19 13:59:01 +02:00
addshore
0ef66de3cf MCR RevisionStore, multi content mode..
Bug: T174024
Change-Id: Ifabf39e12ba843eb754ad0c029b7d16a311047a5
2018-06-14 17:30:33 +02:00
addshore
125bf7e44f [MCR] RevisionStore, enable insertions for new schema
Enable inserts to the new MCR db schema in single slot mode only.

TODO:
 - RELEASE NOTES

Notes:
 - When in MIGRATION_WRITE_NEW or greater, deleting and then
 restoring a page will result in different data in the revision table.
 For example, if you delete a page that has text_ids present in the
 revision table and restore it, the text_ids will be blank after.
 - When in MIGRATION_WRITE_BOTH or greater the archive table will
 start to ar_content_model entries where previously it would have been
 given NULL. This is due to the old content schema having NULL in the db
 when the default content model is used, but the new schema will always
 have a value, taken from the content_models table

Note: If259b1e1c49ce was squashed into this change.

Bug: T183488
Bug: T174024
Change-Id: Ic2221da30c8f6ac2ba42720fcd568f2d0ed70534
2018-06-14 13:36:08 +00:00
jenkins-bot
2224e31507 Merge "Drop archive.ar_text and ar_flags" 2018-04-23 20:15:26 +00:00
Kunal Mehta
28267081d8 RevisionTest: Fix getMockTitle()
There's no such function "getModel", it's probably supposed to be
"getContentModel". PHPUnit 6 emits warnings for trying to mock functions
that don't exist.

Change-Id: I0e5434e48958a8b75238b2ebffb5c1dc4e60c864
2018-04-07 17:54:52 -07:00
Brad Jorsch
21c6ae1163 Drop archive.ar_text and ar_flags
This should have been done long ago. Now it is being done.

This also changes ar_text_id to NOT NULL, since it should never be null
anymore, and DEFAULT 0 in preparation for MCR stopping writing it.

Bug: T33223
Change-Id: I18f1c740b7537c7dc3cfeba9b241d0a9f31caa34
2018-04-03 12:54:27 -04:00
Brad Jorsch
27c61fb1e9 Add actor table and code to start using it
Storing the user name or IP in every row in large tables like revision
and logging takes up space and makes operations on these tables slower.
This patch begins the process of moving those into one "actor" table
which other tables can reference with a single integer field.

A subsequent patch will remove the old columns.

Bug: T167246
Depends-On: I9293fd6e0f958d87e52965de925046f1bb8f8a50
Change-Id: I8d825eb02c69cc66d90bd41325133fd3f99f0226
2018-02-23 10:06:20 -08:00
Reedy
39f0f919c5 Update suppressWarning()/restoreWarning() calls
Bug: T182273
Change-Id: I9e1b628fe5949ca54258424c2e45b2fb6d491d0f
2018-02-10 08:50:12 +00:00
addshore
423ba71f42 DI for CommentStore in RevisionStore
Change-Id: I527388514489e79c53b6016a8bd3119ee1750c83
2018-02-08 18:09:41 +00:00
jenkins-bot
409e870719 Merge "Mock CommentStore in some RevisionTest tests" 2018-02-06 15:32:03 +00:00
jenkins-bot
bbedc24556 Merge "Pass $key into CommentStore methods and use MediawikiServices" 2018-02-06 15:31:59 +00:00
addshore
0db2e18216 Mock CommentStore in some RevisionTest tests
Change-Id: I16b00a5514547d93a2308393098e3b363505374e
2018-02-06 12:40:47 +00:00
addshore
e5879da149 Pass $key into CommentStore methods and use MediawikiServices
This allows CommentStore to be added to MediaWikiServices
without the need of an aditional Factory.

This change includes a compatability layer to allow the behaviour
from 1.30 to continue to be used while deprecated.

CommentStore::newKey has been deprecated.
Keys are now passed into the public methods of CommentStore
where needed.
The following CommentStore methods have had their signatures changed
to introduced a $key parameter, but when used in conjunction with
CommentStore::newKey behaviour will remain unchanged:
  * CommentStore::getFields
  * CommentStore::getJoin
  * CommentStore::getComment
  * CommentStore::getCommentLegacy
  * CommentStore::insert
  * CommentStore::insertWithTemplate

Change-Id: I3abb62a5cfb0dcd456da9f4eb35583476ae41cfb
2018-02-05 15:34:12 +00:00
addshore
5350782908 Force CommentMigration MIGRATION_OLD in 2 RevisionTest tests
These tests fail when the comment store migration is not
set to old, this probably indicates that we need some more
tests for other cases, or that we don't want to test the comment
store functionality here and should mock comment store.

Change-Id: I06d2171498da6e07997569d4a74a0c5cfc913f33
2018-02-01 14:40:57 +00:00
jenkins-bot
8d6cb3df87 Merge "Make Revision::__construct work with bad page ID" 2018-01-11 16:33:34 +00:00
daniel
4589f4d781 Make Revision::__construct work with bad page ID
For backwards-copatibility, we need to be able to construct a Revision
object even for bad page IDs.

Bug: T184689
Change-Id: I18c823d7b72504447982364d581b34e98924b67f
2018-01-11 17:03:48 +01:00
daniel
04bac0dee1 Handle failure to load content in Revision getSize, etc
The Revision class used to just return null if size or hsash were unknown
and could nto be determined. This patch restores this behavior by
catching any RevisionAccessExceptions raised by RevisionRecord when
failing to load content.

Bug: T184693
Bug: T184690
Change-Id: I393ea19b9fb48219583fc65ce81ea14d8d0a2357
2018-01-11 15:23:03 +01:00
Thiemo Mättig
72fa7b9dfc Fix inconsistent capitalization of different method calls
Change-Id: I9f5b9e59e8cdadf65e80077fe2d3a9822b4592fe
2017-12-27 12:35:13 +01:00
addshore
e51f95dea3 [MCR] Introduce BlobStoreFactory
This allows Revision::getRevisionText to get
a different BlobStore instance when $wiki is passed in
restoring the behaviour for $wiki before the MCR Revision
overhaul patch was merged.
Ia4c20a91e98df0b9b14b138eb4825c55e5200384

Bug: T183634
Bug: T183631
bug: T183583
Change-Id: Ib0949454e9a003c2965adc1aab38e31fcf121afe
2017-12-24 23:22:30 +00:00
Kunal Mehta
0eecfd015d RevisionTest: Fix @covers tags
Namespaced classes need to be absolutely named.

Change-Id: I48a4b356835e9edf3151a0c8d3bed08ea8cfb452
2017-12-21 23:29:38 -08:00
Kunal Mehta
ee3f8a60ef RevisionTest: Make @covers tags absolute (re-apply)
PHPUnit requires @covers tags to be absolute, they cannot depend
upon `use ...` statements.

Change-Id: I06efa94be3a814acbcb116162db162dcbf36432e
2017-12-21 20:43:54 -08:00
daniel
6af796f3e0 MCR: Deprecate and gut Revision class
This is a re-submission of I4f24e7fbb68.

As a first major step towards Multi-Content-Revisions (MCR),
this patch turns the Revision class into a legacy proxy for
the new RevisionRecord and RevisionStore classes.

Backwards compatibility is maintained for all but some
rare edge cases, like constructing a completely empty
Revision object.

For more information on MCR, see
<https://www.mediawiki.org/wiki/Requests_for_comment/Multi-Content_Revisions>.

NOTE: once this is merged, verify create/delete/restore cycle on beta,
      ideally with emulated replication lag.

Bug: T174025
Change-Id: Ia4c20a91e98df0b9b14b138eb4825c55e5200384
2017-12-21 18:08:54 +00:00
Daniel Kinzler
09bf4f5bb2 Revert "[MCR] Turn Revision into a proxy to new code."
This reverts commit 9dcc56b3c9.

With this patch applied, newly created revisions are sometimes not found
just after submitting an edit, until replicas have caught up.

Our best theory is that it somehow interfere with ChronologyProtector,
but we don't have a good idea how.

Also, as legoktm mentioned, the commit message is terrible and needs fixing.

Change-Id: Idf3404f3fa8f8d08a7fb2ab8268726e2c1edecfe
2017-12-19 12:38:48 +00:00
Kunal Mehta
1d17bbe563 RevisionTest: Make @covers tags absolute
PHPUnit requires @covers tags to be absolute, they cannot depend
upon `use ...` statements.

Change-Id: I92fadc51cc924477d884536b860272b5a230cd36
2017-12-18 22:01:02 -08:00
Kunal Mehta
7f02e0afb9 RevisionTest: Revision::constructFromRowArray is not a function
I'm guessing that RevisionStore::newMutableRevisionFromArray() is
what is supposed to be tested here, since Revision::__construct()
calls that if it gets an array.

Change-Id: Ia6c2e7eb667aa746a12ce74aad62e58d2468defd
2017-12-18 19:24:46 -08:00
daniel
9dcc56b3c9 [MCR] Turn Revision into a proxy to new code.
Change-Id: I4f24e7fbb683cb51f3fd8b250732bae9c7541ba2
2017-12-18 14:37:29 +00:00
Umherirrender
bc8734f882 RevisionTest: Use utf-8 as name in iconv
utf8 is an alias that doesn't work with all PHP installations.

Change-Id: Ief6e5d8b73c9b4a6614430b699f3399b8137689a
2017-12-15 05:18:09 +00:00
addshore
b8351fed48 More user related Revision construction test cases
I4f24e7fbb683cb51f3fd8b250732bae9c7541ba2 releaved
a failing test "PageArchiveTest::testUndeleteRevisions"
that constructs a revision from an array only passing
in 'user_text' and no 'user' field.

After investigation this is quite an odd case that
probably shouldnt happen in production code.

The test cases in this patch test the current behaviour.

Bug: T180210
Change-Id: I7040a16ac6cb29b49f8e7bed1caee72cdd4c6a61
2017-11-20 20:37:11 +00:00
addshore
181dc242ea Set wgCommentTableSchemaMigrationStage for some RevisionTests
Bug: T180922
Change-Id: Ib88b53e36a66042b3c177cae0663040b4fb14a4c
2017-11-20 09:33:28 +00:00
addshore
25e5d9b8c1 [MCR] tests for Revision::getQueryInfo
Bug: T180210
Change-Id: I5bdb5eed853e22bacd6b4c2546343e9d0f2d8c89
2017-11-18 18:22:24 +00:00
addshore
211143d7eb [MCR] tests for Revision::getArchiveQueryInfo
Bug: T180210
Change-Id: Icb016be8d69dfdfa83f44bbc4fb259b5beb30678
2017-11-18 18:08:45 +00:00
addshore
b2ff18b9ed [MCR] Readd various field & cond method tests for Revision
This is a partial revert of:
Idcfd15568489d9f03a7ba4460e96610d33bc4089
which removed these tests.

Bug: T180210
Change-Id: Ib0617ee0a7bd4391ed25415b44a8ed077a985eaa
2017-11-18 17:50:51 +00:00
addshore
962e7a4f2a Tests for Revision::getRevisionText with ExternalStore usage
Bug: T180210
Change-Id: I20e47a44a064ede4f5e674f57ec4d8e39b379ad6
2017-11-14 18:37:52 +00:00
addshore
64c973060c Initial tests for Revision::getRevisionText
Bug: T180210
Change-Id: Idd5c1af7ecc336ba3f32b15fd8d5cb5bd19f649b
2017-11-14 10:41:11 +00:00
addshore
c97caafa2e Tests for Revision::decompressRevisionText
Bug: T180210
Change-Id: I21f6bb02757bcf59ce483e7d61685d372f134048
2017-11-10 14:11:59 +00:00
addshore
aab3e3be88 Run RevisionIntegrationTest for both ContentHandlerUseDB settings
core used to have a test case that ran various Revision tests
with and without ContentHandlerUseDB set, however it did not
run due to the class / file having a bad name.

This patch reintroduces the running on Revision tests for both
ContentHandlerUseDB settings.

The global has to be set in setup / before the page used for
testing is created, else this page will be created with the
wrong values in the DB for use in the tests.

Change-Id: Ied1023e22eeac413fa86ea6946aa31619fd66319
2017-11-10 14:11:17 +00:00
addshore
9fe46fdd32 Split Revision tests into Unit & Integration classes
Change-Id: If10b102a1a0d680b5f067bf34c0fafcb59c09048
2017-10-13 14:46:36 +01:00