Commit graph

5025 commits

Author SHA1 Message Date
Aaron Schulz
e0805d32e4 Disallow setting DBO_IGNORE in Database for sanity
In the off chance something called this, it would break all
sorts of code that expects that either query result functions
either succeed or throw an error.

Callers are not expected to have to check if the result of
a query is meaningful or false due to an error.

Change-Id: I0b4fe1403f55a399ffd40817ed12f857087d6f83
2017-11-29 21:31:03 +00:00
Aaron Schulz
ea22e3d1f6 objectcache: only give current format keys getWithSetCallback() callbacks
Callback code that happens to make use of $oldValue might not be able
to handle missing, new, or changed fields due to key version changes.

Overhaul testGetWithSetCallback_versions() to be cleaner and cover the
case of unversioned => versioned keys.

Change-Id: If108a73078c530c985d30bdadcbfa9ddd53dc2be
2017-11-28 21:26:06 +00:00
jenkins-bot
19d24c13d6 Merge "Use time forcing methods to avoid WANObjectCacheTest flakeiness" 2017-11-28 20:58:08 +00:00
jenkins-bot
5dad90f357 Merge "Introduce ClearUserWatchlistJob" 2017-11-28 18:31:52 +00:00
addshore
989ba87562 Introduce ClearUserWatchlistJob
Change-Id: Icea573a10078ea3f09dc2e4e9fdc737bf639935d
2017-11-28 17:11:40 +00:00
Cormac Parle
d36d987c4e Do not strip Content-Type header for POST requests to swift
libcurl adds 'Content-Type: application/x-www-form-urlencoded'
to a POST request if the 'Content-Type' header is not set
manually. Because data in swift is updated via POST, the
Content-Type header must be set explicitly to stop a run of
refreshFileHeaders.php from changing the Content-Type of all
files in swift to application/x-www-form-urlencoded

Bug: T178849
Change-Id: I43c21bc1b73e37104cf07cd5f1c1557f472b9898
2017-11-28 14:24:27 +00:00
Aaron Schulz
300c655377 Use time forcing methods to avoid WANObjectCacheTest flakeiness
Use of microtime() is now just for baselines, and it is no longer
assumed to be increasing with each call. Such an assumption is
particuliarly bad on Windows.

I've done 100X run rounds with now failures on Windows.

Change-Id: Ica2a47982495bc95b10ca507414972744ea9507e
2017-11-28 06:50:44 +00:00
jenkins-bot
fb7979f479 Merge "objectcache: add "graceTTL" option to WANObjectCache::getWithSetCallback()" 2017-11-28 02:21:50 +00:00
Aaron Schulz
0d7137099a objectcache: add "graceTTL" option to WANObjectCache::getWithSetCallback()
Also made worthRefreshExpiring() fully match the method documentation.

Change-Id: I48a4b1b9d006de100389b47c03c1185a9316abfc
2017-11-27 17:11:13 -08:00
Kunal Mehta
bdb5b592f4 shell: Optionally restrict commands' access with firejail
Introduces a FirejailCommand class, which can be used to add additional
restrictions to a command, for increased security. For now, firejail
containment needs to be enabled on a per-command basis.

The following restrictions are implemented:
* NO_ROOT - disallows any root access, including via setuid binaries
* SECCOMP - block dangerous syscalls with seccomp
* PRIVATE_DEV - create a private /dev
* NO_NETWORK - deny all network access
* NO_EXECVE - block the execve syscall

A convenient Shell::RESTRICT_DEFAULT is equivalent to NO_ROOT | SECCOMP
| PRIVATE_DEV, with the expectation that more restrictions may be added
to it in the future.

In addition, specific paths can be whitelisted with
Command::whitelistPaths(). Any file/directory that isn't whitelisted in
that top level directory (e.g. /srv) won't exist inside the firejail.

$wgShellRestrictionMethod can be set to false for no restriction system,
'firejail' to explicitly use it, or 'autodetect' to autodetect whatever
system is available. In the future the default should be changed to
autodetection once firejail is tested more.

Bug: T173370
Change-Id: Id74df0dbba40e1e7c07c4368aacffb6eb06a17c5
2017-11-28 00:06:40 +00:00
Reedy
c16af68fb6 Replace MimeMagic::singleton() calls
Change-Id: Ieed41b5d6b0f568fe2872e7754f2feae7868fe7a
2017-11-27 02:13:51 +00:00
jenkins-bot
785960660f Merge "objectcache: add "staleTTL" into WANObjectCache::getWithSetCallback()" 2017-11-26 21:55:47 +00:00
Aaron Schulz
fce6862e46 objectcache: add "staleTTL" into WANObjectCache::getWithSetCallback()
This simply involves passing it through to the set() call

Also added some related commons to adaptiveTTL() involving
usage of this option.

Change-Id: Id5833a5d4efb6cad2eb646832e5b0188e86e12fc
2017-11-26 21:49:47 +00:00
jenkins-bot
d75c150177 Merge "More user related Revision construction test cases" 2017-11-24 13:00:45 +00:00
jenkins-bot
4a9f0872a0 Merge "Remove unused WikiPageTestContentHandlerUseDB (test)" 2017-11-23 21:10:19 +00:00
addshore
e6e59bb425 Remove method level @group Database tags
MessageTest did not have the group at class level
either so that is added in this patch.

Change-Id: Idb6213282d6854044f374c9df51ea32d0a5523e1
2017-11-23 19:20:44 +00:00
addshore
d849eb16ff Clarify the use of @group Database in MediaWikiTestCase
Only the class level doc comment is checked.
Adding @group Database to a method doc comment will do
nothing.

Change-Id: Ic0fff3636d6c44e7a27df8806c9712155ce81d8c
2017-11-23 19:19:14 +00:00
addshore
f963c76dfa Remove unused WikiPageTestContentHandlerUseDB (test)
This class had the wrong name so has not been run since
it was written in 2012.

In If5b1fa39610dbe366649f79beda58f4ea6d58c8f I fixed the
name but the tests fail, so lets just remove it and rewrite
the tests.

Change-Id: Iccdcd4a4f72c8b1329d36ff2e8699ba6b23c9ae7
2017-11-23 19:11:25 +00:00
jenkins-bot
9de578da93 Merge "objectcache: add some WAN cache preemptive refresh tests" 2017-11-23 03:53:45 +00:00
Aaron Schulz
1772c4fe34 objectcache: add some WAN cache preemptive refresh tests
Added some extra sanity checks to WANObjectCache

Change-Id: Iac511b0cc1fc8d57ac98e9d7f2cacbcddc1c6db9
2017-11-22 18:38:53 -08:00
Timo Tijhof
d8cf850032 InstallDocFormatter: Add missing @covers scope
The format() method is a one-line wrapper around execute(), which
is the real method being tested here. Given the class doesn't
contain any other methods, increase the scope to the whole class.

Change-Id: I8b2c7736c6708c0915a9ce15fef0dc85c63621ea
2017-11-22 11:01:54 -08:00
WMDE-Fisch
3ad5a6d28a Add tests for DiffEngine tooltips
Change-Id: I8e6560efc35a6bd875f96edb9b59c9961b81a956
2017-11-22 15:24:20 +01:00
jenkins-bot
a88aff710b Merge "Add unit tests for Html helper methods and change messageBox visibility" 2017-11-21 21:18:40 +00:00
jdlrobson
ba45450feb Add unit tests for Html helper methods and change messageBox visibility
* Messagebox is now private to Html class to discourage unconventional
  usages
* Tests are added for all three helper methods added in
  4e7021a231

Bug: T166915
Change-Id: I1c3e4131b2439c0f4fb94ad4e616a909b52d6b78
2017-11-21 21:11:28 +00:00
jenkins-bot
9f3bc1fe59 Merge "Add string length limits" 2017-11-21 15:21:26 +00:00
Gergő Tisza
17d001b73a Add string length limits
Adds two new ApiBase::getAllowedParams() keys:
PARAM_MAX_BYTES and PARAM_MAX_CHARS, to set a length
limit for a (string-like) parameter.

This makes it easy to document and enforce database
field length limits (where relying on the database
would either result in unfriendly error messages or
silent truncation, depending on DB settings) and
also exposes them in structured form so API clients
can verify the length without doing roundtrips.

Change-Id: I2e784972d7e11cad79fdef887bbcde297dbd9ce0
2017-11-21 10:24:11 +00:00
Aaron Schulz
67072948b6 objectcache: Make MemcachedBagOStuff::makeKeyInternal always have a key class
Even if a key is too long and shortened, it should still have some key class.

Change-Id: I006b6b03ad1302e9e49362bbd051332bc6105837
2017-11-21 02:45:01 +00:00
jenkins-bot
694120c80e Merge "Crimean Tatar Transliteration" 2017-11-20 22:27:57 +00:00
tjones
a0b511319c Crimean Tatar Transliteration
This is a first pass at Latin/Cyrillic translitertion for Crimean
Tatar (crh).

Includes transliteration tables, prefix/suffix mappings, regex
mappings, and exceptions lists for words and abbreviations.

Regularize CRH language name in messages/* files.

Fix "varient" typos in qqq.json.

Add unit tests for CRH transliteration.

Bug: T23582
Change-Id: I424703f99adf837f6217872b882d1ea26bfdd068
2017-11-20 16:56:38 -05:00
addshore
b8351fed48 More user related Revision construction test cases
I4f24e7fbb683cb51f3fd8b250732bae9c7541ba2 releaved
a failing test "PageArchiveTest::testUndeleteRevisions"
that constructs a revision from an array only passing
in 'user_text' and no 'user' field.

After investigation this is quite an odd case that
probably shouldnt happen in production code.

The test cases in this patch test the current behaviour.

Bug: T180210
Change-Id: I7040a16ac6cb29b49f8e7bed1caee72cdd4c6a61
2017-11-20 20:37:11 +00:00
jenkins-bot
7bfb09bbf1 Merge "Tests for TitleValue::__toString" 2017-11-20 16:26:48 +00:00
addshore
60eaec2cc3 Tests for TitleValue::__toString
Change-Id: Ibb157b8e6e4fdc75bdb27ecaa4a552304465c4b2
2017-11-20 09:39:55 +00:00
addshore
181dc242ea Set wgCommentTableSchemaMigrationStage for some RevisionTests
Bug: T180922
Change-Id: Ib88b53e36a66042b3c177cae0663040b4fb14a4c
2017-11-20 09:33:28 +00:00
addshore
25e5d9b8c1 [MCR] tests for Revision::getQueryInfo
Bug: T180210
Change-Id: I5bdb5eed853e22bacd6b4c2546343e9d0f2d8c89
2017-11-18 18:22:24 +00:00
addshore
211143d7eb [MCR] tests for Revision::getArchiveQueryInfo
Bug: T180210
Change-Id: Icb016be8d69dfdfa83f44bbc4fb259b5beb30678
2017-11-18 18:08:45 +00:00
addshore
b2ff18b9ed [MCR] Readd various field & cond method tests for Revision
This is a partial revert of:
Idcfd15568489d9f03a7ba4460e96610d33bc4089
which removed these tests.

Bug: T180210
Change-Id: Ib0617ee0a7bd4391ed25415b44a8ed077a985eaa
2017-11-18 17:50:51 +00:00
jenkins-bot
bf3dc9c456 Merge "Fix RemexCompatMunger infinite recursion" 2017-11-18 03:54:22 +00:00
addshore
4a5da08ece Revision::userCanBitfield test fallback to $wgUser
Bug: T180210
Change-Id: Ibda256d6fdd8762e0e90748d71034979e2de106f
2017-11-17 18:24:03 +00:00
jenkins-bot
d982dff625 Merge "Expose string->bool conversion as function" 2017-11-17 15:53:53 +00:00
Tim Starling
324e4bca4f Fix RemexCompatMunger infinite recursion
When TreeBuilder requests reparenting of all child nodes of a given
element, we do this by removing the existing child nodes, and then
inserting the proposed new parent under the old parent. However, when a
p-wrap diversion is in place, the insertion of the new parent is
diverted into the p-wrap, and the p-wrap then becomes a child of the new
parent, causing a reference loop, and ultimately infinite recursion in
Serializer.

Instead, divert the entire reparent request to the p-wrap, so that the
new parent is a child of the p-wrap. This makes sense since the new
parent is always a formatting element. The only caller of
reparentChildren(), apart from proxies, is AAA step 17, which reparents
children under the formatting element cloned from the AFE list.

Left in some debug code for next time.

Bug: T178632
Change-Id: Id77d21d99748e94c064ef24c43ee0033de627b8e
2017-11-17 23:27:14 +11:00
jenkins-bot
2f15b22b91 Merge "Use Remex in Sanitizer::stripAllTags()" 2017-11-16 20:34:31 +00:00
Huji Lee
2a06bc44ca UserGroupsChanged hook should specify the performer of the change
Otherwise, there will be a unit-testing error when Echo is enabled

Bug: T180292
Change-Id: Ibc185c82ad2a03e06e5727a633e6ab6bccce3345
2017-11-16 02:56:32 +00:00
jenkins-bot
c586a099d7 Merge "Move Sanitizer.php to includes/parser/" 2017-11-16 01:33:21 +00:00
jenkins-bot
6fc89902dd Merge "SanitizerTest: Add tests for stripAllTags" 2017-11-16 01:32:47 +00:00
Roan Kattouw
ddb4913f53 Use Remex in Sanitizer::stripAllTags()
Using a real HTML tokenizer fixes bugs when < or > appear in attribute
values. The old implementation used delimiterReplace(), which didn't
handle this case:

    > print Sanitizer::stripAllTags( '<p data-foo="a&lt;b>c">Hello</p>' );
    c">Hello

We also can't use PHP's built-in strip_tags() because it doesn't handle
<?php and <? correctly:

    > print strip_tags('1<span class="<?php">2</span>3');
    1
    > print strip_tags('1<span class="<?">2</span>3');
    1

Bug: T179978
Change-Id: I53b98e6c877c00c03ff110914168b398559c9c3e
2017-11-15 17:31:31 -08:00
Roan Kattouw
7980e38a84 Move Sanitizer.php to includes/parser/
Change-Id: Id08d91c747ec77d715459b89b03eee247ccd4e1b
2017-11-15 15:16:41 -08:00
Roan Kattouw
ce538a60a7 SanitizerTest: Add tests for stripAllTags
Bug: T179978
Change-Id: I9776cfd51b1b3ec772d4216168fbe466f48f5892
2017-11-15 15:16:36 -08:00
Bartosz Dziewoński
eed3cf7747 Preferences: Remove unwise caching of Preferences::getPreferences()
The result of this function depends on the $user and $context
parameters (e.g. it includes the username from the user, and
localisation messages the language from the context). However,
both of them would be ignored if the result was cached, even
if calling with a different $user or $context.

Rather than make this more complicated just remove the caching.
This is not a hot code path: this function is not called at all
on normal page views, it's called just once when viewing
preferences, and at most twice when saving them.

Change-Id: I92390120a16448383a25e9ba2dd35a434a2f21bf
2017-11-15 11:04:52 +00:00
Bartosz Dziewoński
f1ca6b8ca9 ApiOptionsTest: Do not use ->at()
Quoting PHPUnit docs:

  The $index parameter for the at() matcher refers to the index,
  starting at zero, in all method invocations for a given mock object.
  Exercise caution when using this matcher as it can lead to brittle
  tests which are too closely tied to specific implementation details.

Indeed these test cases would break horribly with unintuitive error
messages ("Mocked method does not exist") if anything in preferences
or API code called any additional methods on the mocked user. For
example, it relied on the caching in Preferences::getPreferences(),
which is being removed in I92390120a16448383a25e9ba2dd35a434a2f21bf.

I'm pretty sure all that matters here is that all the setOption()
calls with different arguments happen, so let's test just that.

Change-Id: I30a814151a006e5f147eebb918344049807b2b97
2017-11-15 11:04:09 +00:00
jenkins-bot
716814a5d5 Merge "Treat langtags in SVG switch case-insensitively" 2017-11-15 10:17:32 +00:00