Commit graph

7294 commits

Author SHA1 Message Date
jenkins-bot
d72f24589b Merge "Pass in ServiceOptions to BlockManager" 2019-07-10 18:02:43 +00:00
jenkins-bot
c9db907126 Merge "Tidy up conditions for applying a block from a cookie" 2019-07-10 18:02:36 +00:00
jenkins-bot
d64677c1be Merge "Make RouterTest extend MediawikiUnitTestCase and not integration" 2019-07-10 17:28:35 +00:00
jenkins-bot
7388b7b62b Merge "Replace old URLs in SampleTest" 2019-07-10 17:02:17 +00:00
Amir Sarabadani
e88601918d Make RouterTest extend MediawikiUnitTestCase and not integration
it's in unit directory, it causes things to break

Change-Id: Ibeafafde5a3b9f3f8e6fda0ceef8575aa544e20b
2019-07-10 18:49:23 +02:00
jenkins-bot
8b698a80cf Merge "bagostuff: optimize SqlBagOStuff and fix failing segmentation tests" 2019-07-10 16:34:47 +00:00
jenkins-bot
4904807042 Merge "rdbms: make IDatabase::isReadOnly() return true for replica DBs managed by LoadBalancer" 2019-07-10 16:12:54 +00:00
Thalia
786a7a168a Pass in ServiceOptions to BlockManager
Change-Id: Ic63d7ff35a71e36c4e6157e9d472e2870f95f00d
2019-07-10 16:33:21 +01:00
Thalia
4b90befa9a Tidy up conditions for applying a block from a cookie
Change-Id: Id9dd6ae395f5bb811db4c741be9db8aa2eb6fb70
2019-07-10 16:28:55 +01:00
jenkins-bot
3163473969 Merge "Isolate globals for ContentSecurityPolicy tests" 2019-07-10 14:09:26 +00:00
Thalia
ff168c8cd7 Replace old URLs in SampleTest
Update URLs to point to documentation for version 6.5, and fix a
couple of broken URLs.

Change-Id: I25211d4da19a40837d7fb7fc6219486f1a7a7c0e
2019-07-10 15:00:12 +01:00
jenkins-bot
ad4c7e66d9 Merge "REST: add write access checks to BasicAccess" 2019-07-10 10:41:11 +00:00
jenkins-bot
51aed12d94 Merge "REST: basic read restrictions" 2019-07-10 10:26:26 +00:00
Aaron Schulz
da5f1240b0 rdbms: make IDatabase::isReadOnly() return true for replica DBs managed by LoadBalancer
Change-Id: I25b70fe0ea113c6741601cf0366a0fe9dc5cc3ef
2019-07-09 23:51:24 -07:00
Aaron Schulz
20a9c8d24e bagostuff: optimize SqlBagOStuff and fix failing segmentation tests
In SqlBagOStuff:
* Add modifyMulti() helper method to reduce code duplication
* Improve atomicity of add(), cas(), and changeTTL() queries
* Avoid integer serialization and improve atomicity of incr()
* Optimize new BagOStuff::changeTTLMulti() method

In BagOStuff:
* Add changeTTLMulti() method for subclasses to optimize
* Make set() ignore WRITE_ALLOW_SEGMENTS for integers so incr() works
* Strip WRITE_ALLOW_SEGMENTS flag from the setMulti() call in set() to
  avoid triggering bogus sanity check exceptions
* Fix BagOStuffTest::testSetSegmentable failures via the above changes
* Enforce WRITE_ALLOW_SEGMENTS sanity check in setMulti() for all the
  subclasses by using a final wrapper method
* Add WRITE_ALLOW_SEGMENTS sanity check to deleteMulti()

Bug: T113916
Change-Id: I25d1790fa9b0d1837643efccfa94a12043cfbf42
2019-07-09 22:53:18 -07:00
jenkins-bot
e0d2eabecf Merge "Move unit tests, round III" 2019-07-09 20:36:37 +00:00
Amir Sarabadani
57261a926c Move unit tests, round III
Depends-On: I16691fc8ac063705ba0c2bc63b96c4534ca8660b
Bug: T87781
Change-Id: I5e1ab06e3decef6cc6090551d54dc4314ab9314a
2019-07-09 22:06:59 +02:00
jenkins-bot
467a280684 Merge "rdbms: avoid recursion in LoadBalancer when the master has non-zero load" 2019-07-09 19:48:58 +00:00
Aaron Schulz
79d1881ede rdbms: avoid recursion in LoadBalancer when the master has non-zero load
Add and use IDatabase::getServerConnection() method to avoid loops caused
caused by pickReaderIndex() calling getConnection() for the master server.
That lead to getReadOnlyReason() triggering pickReaderIndex() again.

Make getLaggedReplicaMode() apply when the master has non-zero load and
the replicas are all lagged.

Remove "allReplicasDownMode" in favor of checking getExistingReaderIndex()
instead. This reduces the amount of state to keep track of a bit.

Follow-up to 95e2c99094

Bug: T226678
Bug: T226770
Change-Id: Id932c3fcc00625e3960f76d054d38d9679d25ecc
2019-07-09 19:26:46 +00:00
jenkins-bot
da1947d5d4 Merge "Unset all globals unneeded for unit tests, assert correct directory" 2019-07-09 18:57:55 +00:00
Amir Sarabadani
d23af35764 Unset all globals unneeded for unit tests, assert correct directory
* Unset globals to avoid tests that look like unit tests but actually rely on
  globals
* move some tests out of unit directory so that the test suite will pass.
* Assert that tests which extend MediaWikiUnitTestCase are in a directory with
  "/unit/" in its path name

Depends-On: I67b37b1bde94eaa3d4298d9bd98ac57995ce93b9
Depends-On: I90921679518ee95fe393f8b1bbd9134daf0ba032
Bug: T87781
Change-Id: I16691fc8ac063705ba0c2bc63b96c4534ca8660b
2019-07-09 14:09:29 -04:00
jenkins-bot
c24ac2cba4 Merge "Run SpecialPageFatalTest with lang=qqx" 2019-07-09 17:34:22 +00:00
Antoine Musso
c40d76e23b Recognizes Open Document Database
Mediawiki does not recognizes the mime type of an OpenOffice / Libre
Office Database Frontend Document and thus uses application/zip.

Our MimeAnalyzer::detectZipType() looks at the Zip files to detect known
signatures, however it is based on a reference from 2005 which did not
have the Database type yet.

OASIS Open Document Format v1.2 specify non normative MIME types, the
reference supposedly being the ones registered with the IANA, but we
have some dispredancy: ODF recommends 'base' while the IANA got
'database' registered: https://www.iana.org/assignments/media-types/

I supposed the format got approved as is, the Database mime type being
'recommended' probably because the IANA registration was pending.

Add the 'base' type which is being used by OpenOffice 3.x and later as
well as at least Libre Office 5.x.
Document my findings in code comments.
Add a test.

Bug: T35515
Change-Id: If0210a87067358612ecb8b3edd001fb05d01653d
2019-07-09 13:42:07 +02:00
Tim Starling
f7ed8615e1 REST: add write access checks to BasicAccess
This is a stub implementation which just checks for the apiwrite
permission.

Change-Id: Ib84cd93e7f0f5e31cf620b2d30609035c4448c95
2019-07-09 15:23:57 +10:00
Tim Starling
94c0baaa2f REST: basic read restrictions
Protect private wikis by providing basic read restrictions,
closely following the example of the action API.

The BasicAccess module provides a narrow interface for this
functionality, without exposing the whole session/user concept to the
router.

Also, add RouterTest and fix a bug in Router::getRelativePath() thus
discovered.

Change-Id: I82319d56f08b2eec4a585ff6dbd348ccdbadc5b5
2019-07-09 15:23:20 +10:00
jenkins-bot
14d8b187f3 Merge "rdbms: normalize Database/LBFactory logging and add snapshot flushing warnings" 2019-07-08 18:53:38 +00:00
Máté Szabó
6c462836a7 Make MSCompoundFileReader::readFile platform-agnostic
MSCompoundFileReader::readFile uses iconv to convert information
given in UTF-16 character set with little-endian byte order to
the UTF-8 character set. The input string has no BOM and the byte order
is not explicitly given, causing iconv to try to guess the byte order
based on the host operating system. This causes the method to return
different results for the same file in different environments.

This patch explicitly provides the byte order for the input to be
converted (UTF-16LE) to ensure portability and predictability.

As part of this, move MSCompoundFileReaderTest into the unit test tree.

Bug: T225019
Change-Id: I62154897d303b28c288c3a4f2f5456bedcc81852
2019-07-08 10:45:42 -07:00
Amir Sarabadani
16b82d0ef0 Run SpecialPageFatalTest with lang=qqx
SpecialPageFatalTest:testSpecialPageDoesNotFatal is one of the slowest
tests (specially running Special:Version) due to the fact that it needs
to translate so many message keys.

3206ms to run SpecialPageFatalTest:testSpecialPageDoesNotFatal with data set "Version"

Running with lang=qqx would ensure that the special doesn't fatal but also avoid
unnecessary message translations

Change-Id: I8ff715ac539e93915c98f7209523df1b3ea3a7e8
2019-07-08 14:52:06 +02:00
jenkins-bot
8435d49442 Merge "Move unit tests FormatJsonTest.php to a dedicated file in unit tests" 2019-07-08 12:49:17 +00:00
Amir Sarabadani
7ec9745444 Split SanitizerTest to unit and integration tests
Out of 150 tests of SanitizerTest.php, 100 of them are pure unit tests
they are moved to the new file in the new structure, the rest stay

Change-Id: I366d37607abff4bcd624a56fb8b2299729fbc088
2019-07-08 09:48:07 +02:00
Amir Sarabadani
fe9863fd55 Move unit tests FormatJsonTest.php to a dedicated file in unit tests
Out of 140 tests of this file, 131 one of them are pure unit test
Let's keep the 9 in the original file and move the rest

Bug: T87781
Change-Id: I86dfe17f794c615048b3c20487b0e84d38d13b93
2019-07-07 21:35:40 -04:00
Aaron Schulz
04d591935c rdbms: normalize Database/LBFactory logging and add snapshot flushing warnings
Make flushSnapshot() logging more detailed and check for explicit transaction
rounds. Also removing periods and trailing newlines from log/exception messages.

Change-Id: I0f6520f563680ab3a65b6338ced59ba25a2ec7b5
2019-07-06 13:10:24 -07:00
jenkins-bot
d71db83e74 Merge "Don't process "all" option in Html::namespaceSelectorOptions" 2019-07-06 15:17:54 +00:00
jenkins-bot
c069752a95 Merge "rdbms: clean up $groups logic in LoadBalancer and expand comments" 2019-07-05 22:00:09 +00:00
jenkins-bot
11aa19e5f2 Merge "Bidi isolate user names in block error paramters" 2019-07-05 19:25:38 +00:00
Thalia
8cfa62d837 Bidi isolate user names in block error paramters
This fixes parameters returned by AbstractBlock::getBlockErrorParams,
but not those from ApiBlockInfoTrait.

Change-Id: I122017808766de1e9a9035f2f39a7b08607e56c1
2019-07-05 15:01:26 +01:00
jenkins-bot
77cc011b9c Merge "SlotRecord:compute sha1 if empty." 2019-07-05 06:56:31 +00:00
Aaron Schulz
ab09cf2fa5 rdbms: clean up $groups logic in LoadBalancer and expand comments
Split out private resolveGroups() method for normalizing query
groups. Move some server index validation to getConnectionIndex()
and make it stricter.

Make getConnectionIndex() group fallback logic aware of the generic
group. The main use case for custom default groups is heavy scripts
or jobs that run in the background. It is probably better to have
good DB server redundancy for the query group and rely on it rather
than possibly fallback onto all of the main servers used for normal
requests. The later behavior could spread slowness or outages.

Also make getAnyOpenConnection() more robust and readable by
splitting out a private pickAnyOpenConnection() method that
checks IDatabase::trxLevel().

In addition, remove redundant is_int() check from isOpen()
method as it will return false in that case even without it.

Remove bogus getConnection() parameter in testCopyTestData().

Change-Id: Ica619c5487c761c724791d151db7388e4b41b0aa
2019-07-04 22:26:55 +00:00
Timo Tijhof
a69880e05c resourceloader: Only output ResourceLoaderDynamicStyles when needed
In mediawiki.js, this marker has always been optional, falling back to
appending to <head>. When no stylesheets need to be after the marker
(e.g. no site styles on the wiki, and user is not logged-in), then
there is no need for the marker to exist.

In a previous refactor, I was going to do this and created an
"$append" variable in the function to do what this commit does,
but I forgot to actually use it for anything.

Test Plan:
* Local wiki, with no MediaWiki:Common/{Skinname}.css pages existing.
* When logged-out, before this change, there is a marker, now there is not.
* When creating "MediaWiki:Group-user.css" and logging in, there is still
  a marker, and it is still above the <link> for that user styles request
  in the <head>.

Bug: T219342
Change-Id: I2e9657f318088860916823efeb96ae4f1532974c
2019-07-04 19:14:39 +00:00
jenkins-bot
2b4c62b597 Merge "rdbms: make implement IResultWrapper directly instead of via inheritence" 2019-07-04 13:58:16 +00:00
Aaron Schulz
a830c14d0b rdbms: make implement IResultWrapper directly instead of via inheritence
Change-Id: If1b15c0c21d0ee336025fb99f47fc19ddf1d5435
2019-07-04 13:42:53 +00:00
Aaron Schulz
1b031cbf3f rdbms: make temp table tracking in Database more robust
Do not register table changes until query success

Change-Id: I8c0eeb510d15e2f4cc014f62b7a0f146e31b6613
2019-07-04 13:36:30 +00:00
daniel
440d9b84be SlotRecord:compute sha1 if empty.
The SHA1 should be computed automatically if empty, not just if it is
missing entirely. This is needed since rev_sha1 may contain an empty
string for old revisions.

Bug: T200653
Bug: T219816
Change-Id: Ia6870a828bc9661fb05085e36315a86483ec48c4
2019-07-04 08:57:17 +00:00
Matěj Suchánek
5fa9a2a191 Don't process "all" option in Html::namespaceSelectorOptions
The value of "all" can be completely arbitrary and in practice
it's usually an empty string. Avoid attempting to filter it out
and format anyhow.

Bug: T227202
Change-Id: I371466407bd038914faa4dc3b0cae9547cf427eb
2019-07-04 10:17:59 +02:00
jenkins-bot
5326140b30 Merge "LogFormatter::formatParameterValue: handle bad user names" 2019-07-03 18:49:45 +00:00
jenkins-bot
932daf05d2 Merge "Ensure canHaveTalkPage returns false when getTalkPage would fail." 2019-07-03 16:55:34 +00:00
daniel
20717fd031 LogFormatter::formatParameterValue: handle bad user names
This makes LogFormatter more robust against bad user names coming from
broken log records.

Bug: T224368
Change-Id: Ifce9e86bddba274182253cd25338766e8093d6ad
2019-07-03 18:49:58 +02:00
jenkins-bot
b7e7a97da6 Merge "Assert that LocalSettings file exists" 2019-07-03 12:35:00 +00:00
daniel
dbce648a15 Ensure canHaveTalkPage returns false when getTalkPage would fail.
This causes Title::getTalkPage and NamespaceInfo::getTitle() to throw
an MWException when called on a LinkTarget that is an interwiki link
or a relative section link. These methods were already throwing
MWException when called on a link to a Special page.

Bug: T224814
Change-Id: I525c186a5b8b8fc22bca195da48afead3bfbd402
2019-07-03 10:40:10 +02:00
jenkins-bot
7387f89ea0 Merge "tests: Use a fancy generator provider in ReleaseNotesTest" 2019-07-02 21:05:53 +00:00