Commit graph

74 commits

Author SHA1 Message Date
Aryeh Gregor
c250d07bac Unit tests for FileBackend
Currently 62.79% coverage, 108/172 lines.

One oddity discovered during testing was that the "quick" variants of
most methods don't have an $opts parameter. It seems like just an
oversight, so I added it.

Bug: T234227
Change-Id: If2978065392cd6dcf693a588bb1ce6b5d43828f2
2019-10-30 09:35:13 +02:00
Aryeh Gregor
0d3d6be18d Convert FileBackendGroup to service
Bug: T234228
Change-Id: I25575f565eba122cdf971a5945572811d17fa3e1
2019-10-25 10:46:20 +03:00
Umherirrender
5bd311b1a2 Add public as visibility in tests folder
Add public, protected or private to function missing a visibility
Enable the tests folder for the phpcs sniff

Change-Id: Ibefce76ea9984c47e08c94889ea2eafca7565e2c
2019-10-10 21:55:37 +02:00
Thiemo Kreuz
e4272518f7 tests: Replace PHPUnit's loose assertEquals(false) with assertFalse()
assertEquals( false, … ) still succeeds when the actual value is 0, null,
an empty string, even an empty array. All these should be reported as a
failure, I would argue.

Note this patch previously also touched assertSame( false ). I reverted
these. The only benefit would have been consistency within this codebase,
but there is no strict reason to prefer one over the other. assertFalse()
and assertSame( false ) are functionally identical.

Change-Id: Ic5f1c7d504e7249002d3184520012e03313137b4
2019-10-04 00:30:36 +00:00
Aaron Schulz
63b05b8490 Expand testDoQuickOperations() tests for FileBackend
Change-Id: I0633f13f07b5a3234b9bbae4840b064972f68fd9
2019-09-29 20:46:17 +00:00
Thiemo Kreuz
c2211946f7 tests: Replace PHPUnit's loose assertEquals(null) with assertNull()
assertEquals( null, … ) still succeeds when the actual value is 0, false,
an empty string, even an empty array. All these should be reported as a
failure, I would argue.

Note this patch previously also touched assertSame( null ). I reverted
these. The only benefit would have been consistency within this codebase,
but there is no strict reason to prefer one over the other. assertNull()
and assertSame( null ) are functionally identical.

Change-Id: I92102e833a8bc6af90b9516826abf111e2b79aac
2019-09-27 19:15:38 +00:00
Thiemo Kreuz
32a429e8c4 tests: Prefer assertSame() when comparing the integer 0
assertSame() is guaranteed to not do any type conversion. This can be
critical when acciden tially comparing, for example, 0 to 0.0.

Change-Id: Iffcc9bda69573623ba14af655dcd697d0fcce525
2019-09-19 15:35:23 +00:00
Aaron Schulz
20c645945c Make doTestLockCalls() faster
This was mostly doing many runs just to test for random lock errors in
versions of Windows from a few years back. Just do three quick rounds.

Change-Id: I2ec2a8ce66a5c14bfc9338b60f8a57b2de8eaece
2019-09-13 22:41:13 -07:00
Aaron Schulz
417dea8e95 filebackend: optimize 'move' in FSFileBackend to avoid is_file() calls
Also, remove clearstatcache() call that is redundant with moveInternal()

Change-Id: I56e6c3d91427e7d0b49011f884b77daa5eb0b61c
2019-09-07 15:18:43 -07:00
Aaron Schulz
00e4420020 filebackend: optimize 'delete' for FSFileBackend to avoid is_file() calls
This also should reduce the chance of warnings due to race conditions

Change-Id: I06b6bcc5e0f009bb3d5135591d13ff098710a5b3
2019-09-07 15:06:28 -07:00
Aaron Schulz
903f1810c8 filebackend: improve internal use of FileBackend constants
Add more result constants and split up FileBackend::UNKNOWN for
clarity. This follows up 5719815f3b, which added that constant.

Make internal FileBackendStore::doGet* methods distinguish I/O errors
from missing files; the return types of public FileBackend methods are
unchanged. Avoid process caching any mtime/size/sha1 values in the
case of I/O errors. Use error constants consistently for stat methods
when given invalid paths.

Also:
* Factor out FileBackendStore::processCacheAndPersistStatEntries() method
  to reduce significant code duplication.
* Consolidate duplicated isPathUsable() checks in FileOp subclasses to
  FileOp::precheck().
* Remove null process cache value check from FileBackend::getFileStat()
  as null values are never stored in the process cache to begin with.
* Reformat some oddly wrapped lines to look cleaner.

Change-Id: Id0e4b0da0bb2ed3184847b35142d587c7f3d953d
2019-09-03 19:31:17 -07:00
Aryeh Gregor
5bbcaef231 FileJournal tests
100% unit test coverage for FileJournal and NullFileJournal. 100%
integration test coverage for DBFileJournal. Unit tests for
DBFileJournal once it supports injection.

I removed FileJournal and NullFileJournal from the list of classes that
FileBackendTest tests. It doesn't actually test them, it just happens to
run code from them without checking its correctness at all.

Depends-On: Ic22075bb5e81b7c2c4c1b8647547aa55306a10a7
Change-Id: I46d10ab7b87c23937aa04d7ec1922abfcf3bd611
2019-09-03 10:12:41 +00:00
Timo Tijhof
4938038175 tests: Remove use of wfRandomString() for test fixtures
This reduces confidence in the test. There is no guruantee that
it won't return the same value twice during the duration of a full
PHPUnit run of all test suites, whether twice in a row or 20 minutes
apart.

For a test that needs a string of any kind, use an explicit, consinstent
and cheap literal value.

For a test that specifically needs some kind of uniqueness compared to
something else within the same test case, do so explicitly.

Tests that require something globally unique (for some undefined/vague
definition of "global") were not found, and should not exist anyway.

Also, in libs/objectcache tests, fix order of parameters in some
assertions (expected first, then actual), and use assertFalse/assertSame
instead of assertEqual for cases where false is expected to remove
tolerance of other loosely equal values.

Change-Id: Ifc60e88178da471330b94bfbf12e2731d2efc77d
2019-05-06 17:22:03 +00:00
Aryeh Gregor
9018579681 Deprecate the Http class
All methods got moved to HttpRequestFactory or MWHttpRequest or dropped.

I made the return value of the new HttpRequestFactory::request/get/post
methods null on error instead of false, so that when we drop PHP 7
support, we can use a "?string" return value. This could theoretically
change behavior of code that was switched from the old Http methods, but
probably won't. I kept the old behavior for the deprecated methods.

I changed the default value of $wgHTTPProxy from false to ''. This way
it should be usable directly without a trivial wrapper method. For the
benefit of anyone who might have set it to false in LocalSettings.php, I
also recommend casting to string just in case.

Http::$httpEngine is deprecated. Eventually it will be removed along
with the curl and PHP engines, leaving only the Guzlle engine.

I also added deprecation of MWHttpRequest::factory, which occurred in
1.31, to the release notes for 1.34. Now hopefully we can hard-deprecate
it in another couple of versions.

Bug: T214390
Change-Id: I2a316a758d793857f248bd251b90f5e9a6440e3a
2019-05-06 12:07:26 +03:00
Thiemo Kreuz
4b71077646 Remove a few obscure "done" and "empty" comments
These don't add any knowledge to what is already obvious from the
code, I find.

Change-Id: Ia613b6a059f78dbeefdfd020899bd1a6e239a731
2019-01-30 20:35:14 +00:00
Aaron Schulz
ea10f22dff Disallow empty paths in LockManager
Fix broken FileBackend test that had paths normalizing to null.

Change-Id: I8a7e88ae44118cc9578e1b306a5a008ccefc23af
2018-07-26 11:02:43 -07:00
Reedy
39f0f919c5 Update suppressWarning()/restoreWarning() calls
Bug: T182273
Change-Id: I9e1b628fe5949ca54258424c2e45b2fb6d491d0f
2018-02-10 08:50:12 +00:00
Umherirrender
45da581551 Use ::class to resolve class names in tests
This helps to find renamed or misspelled classes earlier.
Phan will check the class names

Change-Id: Ie541a7baae10ab6f5c13f95ac2ff6598b8f8950c
2018-01-26 22:49:13 +01:00
Gergő Tisza
525bfbc8df Switch to librarized version of TestingAccessWrapper
Replaces \TestingAccessWrapper (defined in core) with
\Wikimedia\TestingAccessWrapper (defined in the composer package
wikimedia/testing-access-wrapper).

See https://gerrit.wikimedia.org/r/#/q/topic:librarize-testing-access-wrapper
for downstream patches.

The core version of the class is kept around for a while to avoid
circular dependency problems.

Bug: T163434
Change-Id: I52cc257e593da3d6c3b01a909e554a950225aec8
2017-04-20 14:15:57 +00:00
Tim Starling
f3690d48bc Fix @covers for FileBackend
It's up to the developer to choose the level of granularity they are
aspiring for in their test coverage. But that granularity level should be
reflected in @covers. When you have hundreds of lines of code testing a
single-line function (FileBackend::doOperation), that's a good hint that
something went wrong.

FileBackendTest is 2700 lines of code and appears to be aiming to test
the whole filebackend module. So I adjusted @covers to reflect that.

Change-Id: Iacd8cf475d1761c3b32c739983343619a9509d6b
2017-02-27 19:12:22 +11:00
jenkins-bot
8afc7161ed Merge "FSFile and TempFSFile cleanups" 2016-09-20 23:21:30 +00:00
Aaron Schulz
f7e3ac3f95 FSFile and TempFSFile cleanups
* Remove wf* function dependencies. This includes wfTempDir().
  Callers now should specify the directory, though it will try to do
  most of the wfTempDir() logic anyway if they do not.
* Update callers to inject wfTempDir() so $wgTmpDirectory is used by
  TempFSFile instead of it probing to find a valid directory itself.
* Move most of the wfTempDir() logic to TempFSFile::getUsableTempDirectory().
* Remove unused getMimeType() method.

Change-Id: Idd55936b07f9448a6c90577708722b7b52b8fe66
2016-09-19 19:55:09 +00:00
Aaron Schulz
8a573aa1f5 Move some FileBackend related classes to /libs
Change-Id: I0c3a84c2ed8b869519f4d38475a77539c24a88a8
2016-09-19 07:31:25 +00:00
Aaron Schulz
d42a05475a Added Range support to FileBackend::streamFile()
* Added HTTP options headers parameter to streamFile().
* Refactored doStreamFile() to either call StreamFile::stream()
  or delagate that to the subclass. SwiftFileBackend now relays
  the full Swift response rather than manually making the headers.
  This also makes Range headers easy to support.
* Made use of this in img_auth.php for performance on private wikis.
* Elimate stat call in streamFile() for Swift if "headers" is empty.
* Refactored StreamFile a bit to inject request headers instead
  of using the $_SERVER global. A header options parameter is used
  instead, which also supports Range.
* Removed now unused prepareForStream().
* Cleaned up streamFile() unit tests.

Change-Id: I2ccbcbca6caabb8cf65bd6b3084cede2e6ea628a
2016-06-04 05:43:43 -07:00
Reedy
b5656b6953 Many more function case mismatches
Change-Id: I5d3a5eb8adea1ecbf136415bb9fd7a162633ccca
2016-03-19 00:20:58 +00:00
Reedy
1834ee3d8e Fix numerous class/function casing
Change-Id: I23982bfa0548c9ea3bdb432be7982f1563930715
2016-03-18 23:14:49 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
jenkins-bot
26760fc856 Merge "tests: Clean up use of mt_rand()" 2015-12-06 22:32:43 +00:00
Aaron Schulz
d2c33200f7 Remove DeferredUpdates::forceDeferral() testing hack
Change-Id: I7f2ad8190e1a746b6c39190b86fa9c83c5b921fe
2015-12-04 21:18:29 +00:00
Timo Tijhof
fbab4fecef tests: Clean up use of mt_rand()
* ApiQueryTest: One random is enough.

* FileBackendTest: More consistent and idiomatic via wfRandomString()

* MigrateFileRepoLayoutTest: Use getNewTempDirectory(). Similar to
  what FileBackendTest used already.
* UploadFromUrlTestSuite: Use getNewTempDirectory().

Change-Id: I772de2134be41506d8ed08367be8c18f354bfc72
2015-12-03 13:45:55 +00:00
Reedy
00c426e3c2 Replace wfBaseConvert with Wikimedia\base_convert
Change-Id: Iadab3d018c3559daf79be90edb23d131729bdb68
2015-11-24 22:51:42 +00:00
jenkins-bot
b85aeec5e2 Merge "Improve MIME detection in FileBackend" 2015-11-03 11:10:41 +00:00
Aaron Schulz
de290cd02d Improve MIME detection in FileBackend
The content type detector will now inspect the file contents
to better handle extensionless files.

Also dependency inject the callback and make the default one
use FileInfo.

Change-Id: Iad59bf6c6a416b706f976a4c425763fd30e2debb
2015-11-02 00:36:54 +00:00
Aaron Schulz
642ca3862b Normalize header case for FileBackend operations
Normalize all headers to lower case at the start of the
FileBackend operation methods. This makes it easy for
subclasses to check for certain headers, e.g. content-type.

Change-Id: Ia69976326d17a51bcaa61f2781aa669ae7bd9c28
2015-10-27 12:34:35 -07:00
Aaron Schulz
169b7b98b5 Added replication=async option to FileBackendMultiWrite
* This will defer writes to non-master backends till the
  end up the web request. This is useful for multi-DC setups.

Bug: T112708
Change-Id: I118c07764dd4a4f4f2590d4548238df12860e750
2015-10-01 23:30:41 +00:00
Vivek Ghaisas
c54766586a Fix issues identified by SpaceBeforeSingleLineComment sniff
Change-Id: I048ccb1fa260e4b7152ca5f09b053defdd72d8f9
2015-09-26 23:06:52 +00:00
Aaron Schulz
3e8294900a Added read affinity tests for FileBackendMultiWrite
Change-Id: I1407273175396d4d3631531ff0ca0afa3009913e
2015-09-21 11:21:00 -07:00
aude
851552a5c4 Replace use of assertType with assertInternalType and assertInstanceOf
These are available in phpunit since 3.5.0, which I think
is reasonable to expect people to have at this point,
especially when we actually require 3.7.0 or higher in phpunit.php:

Use assertSame instead of assertArrayEquals in UIDGeneratorTest.
assertSame (and assertEqual) care about sortorder of the array,
and is perfectly sufficient in this case to check they array is correct.

if ( $puVersion !== '@package_version@' && version_compare( $puVersion, '3.7.0', '<' ) ) {
        die( "PHPUnit 3.7.0 or later required; you have {$puVersion}.\n" );
}

Change-Id: Ic32ab45110e4c4304ef046ae8d0e98c741255559
2015-04-09 01:15:44 -04:00
Chad Horohoe
c33f4de066 Profile all external HTTP requests from MW
Change-Id: Ie980b080da2ef21ec7d9fc32f1accc55710de140
2015-03-03 20:54:30 -08:00
Timo Tijhof
48d715148b Use MediaWikiTestCase methods for tempdir in unit tests
* Use MediaWikiTestCase::getNewTempFile and getNewTempDirectory
  instead of wfTempDir().

  The upload api tests wrote a tempnam() file directly (where
  wfTempDir() is typically shared with other systems and concurrent
  runs). Use MediaWikiTestCase::getNewTempFile and
  getNewTempDirectory instead.

  This also ensures its removal by the teardown handler without
  needing manual unlink() calls. And it doesn't rely on the test
  passing. (Many unlink calls where at the bottom of tests,
  which wouldn't be reached in case of failure).

* For the upload test, the presistent storing of
  'Oberaargletscher_from_Oberaar.jpg' (downloaded from Commons)
  was removed. Note that this didn't work for Jenkins builds anyway
  as Jenkins builds set $wgTmpDirectory to a unique directory
  in tmpfs associated with an individual build.

* For filebackend tests, moved directory creation from the dataProvider
  to the main test.

  Implemented addTmpFiles() to allow subclasses to register
  additional files (created by other means) to be cleaned up also.

  Removed unused $tmpName and $toPath parameters in data
  provider for FileBackendTest::testStore. And fixed weird double
  $op2 variable name to be called $op3.

* Skipped parserTest.inc, MockFileBackend.php, and
  UploadFromUrlTestSuite.php as those don't use MediaWikiTestCase.

Change-Id: Ic7feb06ef0c1006eb99485470a1a59419f972545
2015-02-11 03:49:02 +00:00
addshore
892a992ab8 Remove MediaWikiPHPUnitCommand
All functionality has been moved to other places

Change-Id: I6b6b0ef846bc63108c4dff9e17098432fd9d1697
2014-08-02 21:35:34 +00:00
Siebrand Mazeland
896bda7c85 Pass phpcs-strict on some test files (7/11)
Change-Id: Ia0ceea60e7ef43959f868378f3fcd463a3adf782
2014-04-24 18:27:12 +00:00
Aaron Schulz
7f367aa9d4 Fixed broken filebackend test for "describe"
Change-Id: I30e3cfc29a1db92e7f1b85af54db778ee639e016
2014-04-15 17:38:50 -07:00
Aaron Schulz
f46cd2a593 Fixed wrong usage of $tmpFile in Swift backend
Change-Id: I21293be38c53b4dca3030c26a201f70d8ed243f3
2014-01-15 13:05:01 -08:00
jenkins-bot
00e5290c56 Merge "filebackend: Added supported for retrieving file metadata/headers" 2014-01-09 18:36:43 +00:00
Aaron Schulz
5b6ff97519 Fixed minor unit test dir leakage
Change-Id: I78cf483dde5078d4fad4e1738d8bb8a8f634e3b1
2014-01-08 15:59:39 -08:00
Aaron Schulz
ac14c1c8a6 filebackend: Added supported for retrieving file metadata/headers
* This can be useful for carrying over metadata when copying files around
* Also fixed a bug in sanitizeHdrs() for Swift (broken content-disposition)

Change-Id: I4534e9acac2b306086797b3677f85c05b98e39fc
2014-01-08 13:22:11 -08:00
Aaron Schulz
2af7ad827d Removed cloudfiles dependency in SwiftFileBackend
* Added a new curl multi wrapper under libs/.
* Used the new MultiHttpClient class instead of CloudFiles
* Removed CDN related config and code as it does not work.
  well due to mutating files and tiny purge rate limits by CDNs.
* Also removed the unused and esoteric 'swiftAnonUser' parameter.
* Support concurrent 'describe' operations.
* Set ACLs on containers on intial PUT instead of in two steps.
  Previously if the second step failed, prepare() would never
  set the ACLs correctly since it aborts if the container exists.
* Improve token deletion to only trigger on 401s rather than other
  random errors like 50X timeouts.
* Removed duplicative getResponse* code by just using one closure
  called for both the concurrent and serial cases. Both cases are
  now funneled through doExecuteOpHandlesInternal().
* Add the MD5 to stat info, as it might be useful for speeding up
  the copy scripts in some cases (when both backends have md5 set).
* Avoid use of trigger_error() since there is a swift log group.
* A few other small code cleanups.

Change-Id: I1adb3e2df6df8cf01d3ad74158de96ea9a79da2e
2014-01-07 23:56:34 -08:00
Aaron Schulz
039e6863a5 Fixed undefined variable errors that made tests fail
Change-Id: Ic004e0d8d8efe3e4b313b737479b8098aa0048bc
2013-12-18 10:36:22 -08:00
Aaron Schulz
9760847672 Added streamFile() unit tests
Change-Id: I2ef5680829048ab4fcc0a9f7ff9064d016800217
2013-12-17 13:35:33 -08:00