Commit graph

77 commits

Author SHA1 Message Date
Aryeh Gregor
8c4f59db64 New LanguageFallback service
This replaces the static Language methods getFallbackFor(),
getFallbacksFor(), and getFallbacksIncludingSiteLanguage(). There is
100% unit and integration test coverage for the new class.

One deliberate functional change: I changed one place where we threw
MWException to InvalidArgumentException.

Bug: T201405
Depends-On: Ie7a89f6ed7d52a0bc01672019ff92e7ee105a1f3
Change-Id: I49222eb55f1feec5b1dcd40f364cffe0c8801855
2019-10-08 15:11:39 -07:00
jenkins-bot
52b44696ba Merge "Split some Language methods to LanguageNameUtils" 2019-10-08 21:10:07 +00:00
jenkins-bot
8685746269 Merge "Registration: Allow relative paths in config arrays, not just strings" 2019-10-08 18:04:46 +00:00
Matthias Mullie
095a8e5d8a Registration: Allow relative paths in config arrays, not just strings
The previous code to allow relative paths (via "path": true) was
limited to only string values, which led to needing a non-static
registration callback function.

Bug: T231855
Change-Id: I4e82f38627f2f0d50d8b05a8c02b5a35342f6526
2019-10-08 10:20:04 -07:00
jenkins-bot
05ce3b7740 Merge "Title: make newFromText, isValid, and canExist behave consistently." 2019-10-08 13:54:57 +00:00
daniel
9171cbe4ca Title: make newFromText, isValid, and canExist behave consistently.
This patch fixes the documentation of newFromText and makeTitleSafe to
no longer state incorrectly that any Title they return is guaranteed to
be valid. That has never been true.

It also makes canExist() checks a lot stricter, to match the assumptions
of current callers, namely that the title can exist as a wiki page.

Finally, it replaces several existing calls to isValid() with calls to
canExist().

Bug: T229705
Change-Id: I2a483136ec6acca49afb5eb32cb94616672b8fb2
2019-10-08 11:16:11 +02:00
Aryeh Gregor
6d80b6c082 Split some Language methods to LanguageNameUtils
These are static methods that have to do with processing language names
and codes. I didn't include fallback behavior, because that would mean a
circular dependency with LocalisationCache.

In the new class, I renamed AS_AUTONYMS to AUTONYMS, and added a class
constant DEFINED for 'mw' to match the existing SUPPORTED and ALL. I
also renamed fetchLanguageName(s) to getLanguageName(s).

There is 100% test coverage for the code in the new class.

This was previously committed as 2e52f48c2e and reverted because it
depended on e4468a1d6b, which had to be reverted for performance
issues. There should be no changes other than rebasing.

Bug: T201405
Change-Id: Ifa346c8a92bf1eb57dc5e79458b32b7b26f1ee8a
2019-10-07 15:20:52 -07:00
jenkins-bot
42f58ae012 Merge "Setup: Move wgArticlePath validation to its main consumer (PathRouter)" 2019-10-07 21:58:16 +00:00
Aryeh Gregor
043d88f680 Make LocalisationCache a service
This removes Language::$dataCache without deprecation, because 1) I
don't know of a way to properly simulate it in the new paradigm, and 2)
I found no direct access to the member outside of the Language and
LanguageTest classes.

An earlier version of this patch (e4468a1d6b) had to be reverted
because of a massive slowdown on test runs. Based on some local testing,
this should fix the problem. Running all tests in languages is slowed
down by only around 20% instead of a factor of five, and memory usage is
actually reduced greatly (~350 MB -> ~200 MB). The slowdown is still not
great, but I assume it's par for the course for converting things to
services and is acceptable. If not, I can try to optimize further.

Bug: T231220
Bug: T231198
Bug: T231200
Bug: T201405
Change-Id: Ieadbd820379a006d8ad2d2e4a1e96241e172ec5a
2019-10-07 13:18:47 -07:00
Bill Pirkle
2ed69af15c Add optional serialization_type and hmac_key param values to RESTBagOStuff
T233537 made RESTBagOStuff work with the Kask external session
storage service, but broke backward compatibility. Add optional
values to the RESTBagOStuff $params constructor parameter to
support communicating with Kask, and to allow using HMAC. The
new values are:
  serialization_type: legacy (default), PHP, or JSON
  hmac_key: HMAC key to use for protecting the serialized blob
If these new values are not specified, behavior remains
unchanged (PHP serialization with no HMAC protection).

Bug: T233963
Change-Id: Ia2625c04e08cfe9616569500f1d613be73c170a2
2019-10-06 15:52:01 -05:00
Max Semenik
fb06e4cd85 Update tests to use PHPUnit 6 class names
Bug: T192167
Change-Id: I42b0c8908b4968b95b08f861a40af18dc79fa0a1
2019-10-06 01:01:28 -07:00
Max Semenik
b0a8b0d275 Cleanup of old PHPUnit usage, part 3
Bug: T192167
Bug: T234597
Change-Id: I88a1ced51d7b57bc683de4a58bbeaa1537762113
2019-10-05 00:00:45 -07:00
Max Semenik
0f9cc78aba Cleanup of old PHPUnit usage, part 2
Bug: T192167
Bug: T234597
Change-Id: Ifec7f02f64d6613dbfa1878007f68c2e411a83dc
2019-10-04 21:47:46 -07:00
jenkins-bot
9f5f21ac29 Merge "Begin cleaning up PHPUnit 4 code from tests" 2019-10-05 02:34:55 +00:00
Max Semenik
bc3878e33a Begin cleaning up PHPUnit 4 code from tests
This process will be broken up into several parts for reviewability.

Bug: T192167
Change-Id: Ie415fd3308384a5ca2b3de24ba037785f8a3a714
2019-10-04 14:19:05 -07:00
Umherirrender
fb3914739e Type hint against IResultWrapper
IDatabase::select returns the interface IResultWrapper,
not the implementation ResultWrapper

Change-Id: I8904eff2db48ff00a4a9486be842266c638bc249
2019-10-04 19:58:19 +02:00
Thiemo Kreuz
e4272518f7 tests: Replace PHPUnit's loose assertEquals(false) with assertFalse()
assertEquals( false, … ) still succeeds when the actual value is 0, null,
an empty string, even an empty array. All these should be reported as a
failure, I would argue.

Note this patch previously also touched assertSame( false ). I reverted
these. The only benefit would have been consistency within this codebase,
but there is no strict reason to prefer one over the other. assertFalse()
and assertSame( false ) are functionally identical.

Change-Id: Ic5f1c7d504e7249002d3184520012e03313137b4
2019-10-04 00:30:36 +00:00
Daimona Eaytoy
c12b32c9a3 HHVM removal: Kill HHVM_VERSION checks in PHPUnit tests
Change-Id: I1d18b2d6956d326e5f9e50cd01ce2bb683a5b653
2019-10-03 15:35:21 +00:00
jenkins-bot
565925035a Merge "resourceloader: Add version to ResourceLoaderImage urls for long-cache" 2019-09-28 20:20:32 +00:00
Timo Tijhof
172de0b957 resourceloader: Add version to ResourceLoaderImage urls for long-cache
The code previously here did not work well as it merely forwarded the
hash from the current web request. This had numerous issues:

1. It was often null because requests for stylesheets do not cary
   a version hash.
2. When requested by JavaScript, the version hash would be a
   combination-hash of many unrelated modules, thus when requested as
   part of different batches, it would produce different urls which
   is not ideal.

The impact of this is minimal currently because we basically never use
these urls, as SVGs are almost always embedded instead of ref'ed by url.
PNG urls are only generated for non-JS modules and then only used in older
browsers not supporting SVG. And, even after all that, for the edge case
of an SVG being ref'ed by url, they would be stored in LocalStorage by
mw.loader with the name+version of the module the image belonged to, not
the version hash of the batch request it came with.

Which means that, yes, localstorage key for "somemodule+someversion" would
have different values for different users, based on which batch the value
came with, because the image urls were using the version hash of the batch
request from ResourceLoaderContext. This is weird, but didn't cause bugs
or inefficiencies because the user would never be exposed to the other
possible urls for that image because we always check LocalStorage first.
It did cause fragmentation server-side in Varnish, though.

This is all fixed now by always including a version, and setting it to
the version of the module. This means there is no more Varnish fragmentation
for these. And it means that browsers are now allowed to cache the images
served from these urls for 30+ days (immutable) instead of only 5min,
which is what happened when they didn't have a version parameter (or set to
null).

Bug: T233343
Change-Id: I4af7fda03698ed4c288d154e7787fb2f3cbbe6c5
2019-09-28 02:06:32 +01:00
Thiemo Kreuz
c2211946f7 tests: Replace PHPUnit's loose assertEquals(null) with assertNull()
assertEquals( null, … ) still succeeds when the actual value is 0, false,
an empty string, even an empty array. All these should be reported as a
failure, I would argue.

Note this patch previously also touched assertSame( null ). I reverted
these. The only benefit would have been consistency within this codebase,
but there is no strict reason to prefer one over the other. assertNull()
and assertSame( null ) are functionally identical.

Change-Id: I92102e833a8bc6af90b9516826abf111e2b79aac
2019-09-27 19:15:38 +00:00
Timo Tijhof
480b7341f8 Setup: Move wgArticlePath validation to its main consumer (PathRouter)
The variable is also read in a few other places, such as to
export the value from api.php (siteinfo) and load.php (mw.config)
but those requests don't need to be held back by this extra
logic.

Alternatively, if we really want to require this for all consumption,
we should probably let PathRouter provide the value and require
consumers to use it. E.g. services->getPathRouter->getArticlePath,
or something like that.

As easy first step, I'm moving it to PathRouter, called from
WebRequest::getPathInfo which is still called on all index.php
requests for any wiki page action in any namespace (incl Special)
when the wiki uses anything other than the default 'index.php?title='
article path.

Test Plan:
* Set '$wgArticlePath = 'bla';`
* View /mediawiki/index.php/Main_Page, and observe the fatal
  error message (same as before this change).

Bug: T189966
Change-Id: Id06c2557e2addb58faeef0b6f7767a65b8de55a5
2019-09-25 23:06:52 +00:00
Timo Tijhof
d85ea2b030 resourceloader: Convert ImageModule test to stricter unit test
Bug: T233343
Change-Id: I1f9d0b6b735c9192bb32baf249eefea76c65929e
2019-09-24 23:17:31 +00:00
jenkins-bot
0fc1dee35e Merge "tests: Prefer PHPUnit's assertSame() when comparing empty strings" 2019-09-20 15:55:08 +00:00
jenkins-bot
095c50ade1 Merge "tests: Prefer assertSame() when comparing the integer 0" 2019-09-20 15:33:02 +00:00
Thiemo Kreuz
e06ce9f467 tests: Prefer PHPUnit's assertSame() when comparing empty strings
assertSame() is guaranteed to never do any magic type conversion.
This can be critical when accidentially comparing empty strings (a
value PHP considers to be "falsy") to false, 0, 0.0, null, and such.

Change-Id: I2e2685c5992cae252f629a68ffe1a049f2e5ed1b
2019-09-20 15:27:58 +00:00
Thiemo Kreuz
32a429e8c4 tests: Prefer assertSame() when comparing the integer 0
assertSame() is guaranteed to not do any type conversion. This can be
critical when acciden tially comparing, for example, 0 to 0.0.

Change-Id: Iffcc9bda69573623ba14af655dcd697d0fcce525
2019-09-19 15:35:23 +00:00
Petr Pchelko
0df763f71d Use UserIdentity instead of User in REST
Change-Id: Ia6a517c6a64664be2363492108f9497fc949f299
2019-09-18 14:45:01 +10:00
Tim Starling
9911a36050 REST: Properly handle HEAD requests
Just handle a HEAD request as a GET request, if no specific HEAD handler
exists. This is simple and similar to what the rest of MediaWiki does in
response to a HEAD request.

Bug: T226043
Change-Id: I7b2bd657c20b56844459874131a3d85fabe7db3d
2019-09-17 16:03:14 +10:00
Tim Starling
8b1a6cc58a Use TextFormatter in the REST API
* Add ResponseFactory::createLocalizedHttpError(), which generates a
  JSON response body from a MessageValue
* ResponseFactory::__construct() accepts an array of TextFormatter
  objects. For ease of testing, the array may be empty. The integrated
  ResponseFactory has a TextFormatter for English, and one for
  $wgContLang if that is different.
* Use createLocalizedHttpError() to show helpful error messages for
  errors generated by Router.

Change-Id: I897a0aee42227916c568333ab384966f1b87f599
2019-09-17 16:03:14 +10:00
jenkins-bot
02dc5d8685 Merge "Fix installation failure due to unexpected dbpath under CLI installation" 2019-09-06 17:38:36 +00:00
Brad Jorsch
ebfbd2d42a rest: Use ParamValidator library, add BodyValidator
Parameter validation is based on parameter definitions like those in the
Action API, using the new ParamValidator library. Handlers should use
the provided Handler methods to access parameters rather than fetching
them directly from the RequestInterface.

Body validation allows the handler to have the (non-form-data) body of a
request parsed and validated. The only validator included in this patch
ignores the body entirely; future patches may implement validation for
JSON bodies based on JSON schemas, or the like.

Bug: T223239
Change-Id: I3c37ea2b432840514b6bff90007c8403989225d5
2019-09-04 10:12:35 -04:00
jenkins-bot
a9fb2b3b06 Merge "FileJournal tests" 2019-09-03 11:22:00 +00:00
Aryeh Gregor
5bbcaef231 FileJournal tests
100% unit test coverage for FileJournal and NullFileJournal. 100%
integration test coverage for DBFileJournal. Unit tests for
DBFileJournal once it supports injection.

I removed FileJournal and NullFileJournal from the list of classes that
FileBackendTest tests. It doesn't actually test them, it just happens to
run code from them without checking its correctness at all.

Depends-On: Ic22075bb5e81b7c2c4c1b8647547aa55306a10a7
Change-Id: I46d10ab7b87c23937aa04d7ec1922abfcf3bd611
2019-09-03 10:12:41 +00:00
Timo Tijhof
29e0183a56 phpunit: Repair GLOBALS reset in MediaWikiUnitTestCase
This code didn't work because the $GLOBALS array is exposed by reference.
Once this reference was broken by unset(), the rest just manipulated a
local array that happens to be called "GLOBALS". It must not be unset or
re-assigned. It can only be changed in-place.

Before this, the execution of a MediaWikiUnitTestCase test stored a
copy of GLOBALS in unitGlobals, then lost the GLOBALS pointer and
created a new variable called "GLOBALS". As such, the tearDown() function
didn't do what it meant to do, either – which then results in odd
failures like T230023

Rewrite it as follows:

* In setup, store the current GLOBALS keys and values, then reduce
  GLOBALS to only the whitelisted keys and values.

* In teardown, restore the original state.

* As optimisation, do this from setUpBeforeClass as well, so that
  there are relatively few globals to reset between tests.
  (Thanks @Simetrical!)

The following tests were previously passing by accident under
MediaWikiUnitTestCase but actually did depend on global config.

* MainSlotRoleHandlerTest (…, ContentHandler, $wgContentHandlers)
* SlotRecordTest (…, ContentHandler, $wgContentHandlers)
* WikiReferenceTest (wfParseUrl, $wgUrlProtocols)
* DifferenceEngineSlotDiffRendererTest (DifferenceEngine, wfDebug, …)
* SlotDiffRendererTest (…, ContentHandler, $wgContentHandlers)
* FileBackendDBRepoWrapperTest (wfWikiID, "Backend domain ID not provided")
* JpegMetadataExtractorTest (…, wfDebug, …, LoggerFactory, …)
* ParserFactoryTest (…, wfDebug, …, LoggerFactory, InvalidArgumentException)
* MediaWikiPageNameNormalizerTest (…, wfDebug, …, LoggerFactory, …)
* SiteExporterTest (SiteImporter, wfLogWarning, …)
* SiteImporterTest (Site::newForType, $wgSiteTypes)
* ZipDirectoryReaderTest (…, wfDebug, …, LoggerFactory, …)

Bug: T230023
Change-Id: Ic22075bb5e81b7c2c4c1b8647547aa55306a10a7
2019-09-02 20:58:34 +01:00
Aryeh Gregor
97b9dbc6ce Integration tests for Language fallback methods
100% coverage.  Next: reimplementation as a service, and tests for the
new service.

Change-Id: Icdd5b84eb91fc83905209ce08d499011f7288e33
2019-08-29 18:25:12 +03:00
Aryeh Gregor
709773ab57 createNoOpMock() method for PHPUnit tests
This is just a shortcut for a commonly-used pattern, when you want to
create a mock that's never supposed to have methods called.

Change-Id: Ia7267e3d3108c1ff94485f7e44bf409808a762be
2019-08-29 18:00:37 +03:00
Aryeh Gregor
bd2a439502 Integration tests for FileBackendGroup
100% coverage except for one bit of the code that I didn't understand.
Unit tests to come, together with rewrite as a service.

Change-Id: Ib01758d994a9e5587a4fcb5edc3d80010ef05615
2019-08-28 11:21:44 +03:00
Aryeh Gregor
5a6c18a086 LockManagerGroupFactory to replace singletons
100% test coverage of code that appears to be working and used, in both
LockManagerGroupFactory and also LockManagerGroup. Where possible I
wrote it as unit tests. One preexisting code path seems to be broken and
I marked the test as skipped. Two methods look unused and perhaps not
especially helpful, so I didn't write tests for them yet in case we want
to just get rid of them instead.

Change-Id: Iaa7354f31c451b87773468609c674a3bf1d4382f
2019-08-27 13:19:15 +03:00
jenkins-bot
d75e186b73 Merge "REST: call MediaWiki::preOutputCommit and doPostOutputShutdown" 2019-08-26 23:36:36 +00:00
Amir Sarabadani
308e6427ae Revert "Make LocalisationCache a service"
This reverts commits:
 - 76a940350d
 - b78b8804d0
 - 2e52f48c2e
 - e4468a1d6b

Bug: T231200
Bug: T231198
Change-Id: I1a7e46a979ae5c9c8130dd3927f6663a216ba753
2019-08-26 18:28:26 +02:00
Tim Starling
86693df247 REST: call MediaWiki::preOutputCommit and doPostOutputShutdown
As in api.php. Among other things, this enables profiling.

Move EntryPoint test out of unit/ so that it passes.

Use ob_start()/ob_end_clean() instead of assuming an output buffer is
open, so that EntryPoint::execute() can be run from CLI mode.

Change-Id: I38162a9eac6fd5acfed2035b87cac4a97ffd50d6
2019-08-26 14:19:43 +10:00
Aryeh Gregor
2e52f48c2e Split some Language methods to LanguageNameUtils
These are static methods that have to do with processing language names
and codes. I didn't include fallback behavior, because that would mean a
circular dependency with LocalisationCache.

In the new class, I renamed AS_AUTONYMS to AUTONYMS, and added a class
constant DEFINED for 'mw' to match the existing SUPPORTED and ALL. I
also renamed fetchLanguageName(s) to getLanguageName(s).

There is 100% test coverage for the code in the new class.

Change-Id: I245ae94bfc1f62b6af75ea57525139adf2539fe6
2019-08-23 12:52:35 +03:00
Aryeh Gregor
5e2199c5b0 BadFileLookup to replace wfIsBadImage
I think this probably shouldn't be directly in the MediaWiki namespace,
but I don't know where is a better place to put it.

In order to avoid gratuitous use of TitleFormatter, I changed the cache
format -- the old implementation used getPrefixedDBkey() and I switched
to an ns/dbkey pair. I also changed the cache keys to use SHA1 instead
of MD5, by Daniel's request.

The previous implementation cached the parsed blacklist for one minute
without invalidation, so it could return slightly stale results, but it
didn't retrieve the bad image list message on a cache hit. The new
implementation unconditionally retrieves the bad image list message, but
uses a hash of it in the cache key and caches for one day. The new
behavior happens to be more cleanly implementable in a service.

Bug: T200882
Bug: T139216
Change-Id: I69fed1b1f3cfc1aa149e0739780e67f6de01609d
2019-08-21 20:45:37 +03:00
Aryeh Gregor
a83b33582a TempFSFileFactory service
This replaces TempFSFile::factory(), which is now deprecated.

Change-Id: I9e65c3867e26c16687560dccc7d9f3e195a8bdd6
2019-08-21 16:26:05 +03:00
Aryeh Gregor
d7d5fb81b5 Tests for TempFSFile
These are in preparation for making a TempFSFileFactory service, thus
the odd break-up into two files. I split it into a separate commit so
that we could verify that the same tests pass before and after the
conversion to service.

Tests cover everything except getUsableTempDirectory() (which I don't
see how to test), and register_shutdown_function()-related stuff (which
seems actually impossible to test without starting a new PHP process).

Change-Id: If61b7ea3e332adc2bceefc8e6879a9e9443c99dd
2019-08-20 11:25:02 +03:00
Aryeh Gregor
8ec24b607a Introduce MovePageFactory
This will help make MovePage more testable.

In the course of abstracting the logic out of ParserFactoryTest to
FactoryArgTestTrait so it could be used in MovePageFactoryTest, I made
them all unit tests instead of integration. This required some
modification to the Parser constructor so that it didn't access
MediaWikiServices unnecessarily.

Change-Id: Idaa1633f32dfedfa37516bb9180cfcfbe7ca31aa
2019-08-19 20:25:31 +03:00
jenkins-bot
22a0bde74b Merge "Add recursion check to createService()" 2019-08-19 15:36:40 +00:00
Aryeh Gregor
a995d9be1d Add recursion check to createService()
This will throw when trying to create a service while already in the
process of creating that same service, i.e., if there's a circular
service dependency. This would have saved me a whole bunch of debugging
time. :)

Change-Id: Id148d4f221f35f4069f3e0ab0069d13ca271df3d
2019-08-18 10:51:24 +03:00
Aryeh Gregor
c503f129c3 Introduce TitleParser::makeTitleValueSafe()
This works exactly the same as Title::makeTitleSafe(), but for
TitleValues. 100% test coverage of splitTitleString(). Also added
identical parallel tests for Title::makeTitleSafe() to verify that
behavior is the same.

Along the way, I discovered that TitleValue doesn't allow an empty page
name even if there's an interwiki prefix, which is wrong, so I fixed it.

Bug: T220966
Change-Id: I4b915244ceee4c1857178dd68dcdf57f1ee32200
2019-08-14 21:34:39 +03:00