Commit graph

30 commits

Author SHA1 Message Date
jenkins-bot
00e5290c56 Merge "filebackend: Added supported for retrieving file metadata/headers" 2014-01-09 18:36:43 +00:00
Aaron Schulz
5b6ff97519 Fixed minor unit test dir leakage
Change-Id: I78cf483dde5078d4fad4e1738d8bb8a8f634e3b1
2014-01-08 15:59:39 -08:00
Aaron Schulz
ac14c1c8a6 filebackend: Added supported for retrieving file metadata/headers
* This can be useful for carrying over metadata when copying files around
* Also fixed a bug in sanitizeHdrs() for Swift (broken content-disposition)

Change-Id: I4534e9acac2b306086797b3677f85c05b98e39fc
2014-01-08 13:22:11 -08:00
Aaron Schulz
2af7ad827d Removed cloudfiles dependency in SwiftFileBackend
* Added a new curl multi wrapper under libs/.
* Used the new MultiHttpClient class instead of CloudFiles
* Removed CDN related config and code as it does not work.
  well due to mutating files and tiny purge rate limits by CDNs.
* Also removed the unused and esoteric 'swiftAnonUser' parameter.
* Support concurrent 'describe' operations.
* Set ACLs on containers on intial PUT instead of in two steps.
  Previously if the second step failed, prepare() would never
  set the ACLs correctly since it aborts if the container exists.
* Improve token deletion to only trigger on 401s rather than other
  random errors like 50X timeouts.
* Removed duplicative getResponse* code by just using one closure
  called for both the concurrent and serial cases. Both cases are
  now funneled through doExecuteOpHandlesInternal().
* Add the MD5 to stat info, as it might be useful for speeding up
  the copy scripts in some cases (when both backends have md5 set).
* Avoid use of trigger_error() since there is a swift log group.
* A few other small code cleanups.

Change-Id: I1adb3e2df6df8cf01d3ad74158de96ea9a79da2e
2014-01-07 23:56:34 -08:00
Aaron Schulz
039e6863a5 Fixed undefined variable errors that made tests fail
Change-Id: Ic004e0d8d8efe3e4b313b737479b8098aa0048bc
2013-12-18 10:36:22 -08:00
Aaron Schulz
9760847672 Added streamFile() unit tests
Change-Id: I2ef5680829048ab4fcc0a9f7ff9064d016800217
2013-12-17 13:35:33 -08:00
Aaron Schulz
2aa84651ca filebackend: cleaned up the FileBackend constructor
* Added some b/c code with deprecation warnings

Change-Id: Ifceffbc0a37a223bcd7cd3dc60181fc85765bc46
2013-12-03 13:57:01 -08:00
Aaron Schulz
2f29c7551b Revert "filebackend: cleaned up the FileBackend constructor"
Actually this messes with the implicit backend made for things like Math (when unconfigured), which uses the "new" operator.

This reverts commit 1f129a22cb.

Change-Id: I4c72c4f7c8b82e38df5496cf2b90fc9e19c40334
2013-12-03 20:40:24 +00:00
Aaron Schulz
1f129a22cb filebackend: cleaned up the FileBackend constructor
* Moved some of the graph construction work to FileBackendGroup.
  This helps the code in not depending on the rest of MW so much.
* Updated tests and FileBackendMultiwrite, which are the only things
  directly constructing FileBackend objects.

Change-Id: I188a053c70ce088ce34613d5db40e6708e3ea9b7
2013-11-23 20:21:53 -08:00
addshore
4e941cf4ca Add @covers tags for more tests
Change-Id: Iff3af78e9b41c445b7f066b6c0d0f4a87d2d6c4e
2013-10-21 11:28:39 +02:00
Timo Tijhof
4bd5471ca3 docs: Remove odd colons after @todo
Most were this way already:
https://doc.wikimedia.org/mediawiki-core/master/php/html/todo.html

Ran a find/replace on the odd ones. Also made them all
lower case.

Change-Id: I70c6a69344ddebc603e9a1c1d87e3cc4f4f4c560
2013-05-15 06:23:40 +00:00
Aaron Schulz
d1940caa1f [FileBackend] Added "adviseStat" option for the "listing followed by stat" case.
* Used this parameter to speed up copyFileBackend.php.
* Also added mtime checks to copyFileBackend.php and a few cleanups.
* Also fixed some incorrect getDirListPageInternal/getFileListPageInternal docs.

Change-Id: I424ef238f7adf4cf1f33b74e3a4e187dcb328a99
2013-05-09 16:19:40 +00:00
Siebrand Mazeland
791d0b2a98 Update code formatting
Change-Id: I16a9b42651f1cfb1a70dffbb67b7b83dfeb90d03
2013-04-26 14:21:20 +00:00
Aaron Schulz
e2f2e69724 Cleaned up test cleanup for FileBackend and avoid use of @.
Change-Id: Ie9f0090f626384a90b0139b9f8c2d94bf0bb8f23
2013-04-18 05:43:51 +00:00
Brian Wolff
2ca33d1ce7 Make concurrent runs of FileBackendTest independent
Make each run of the FileBackendTest be for a different "wiki"
so you could have multiple runs at the same time without them
interfering with each other.

There's been intermittent failures on jenkins for this test. I'm
not sure if its some race condition (which wouldn't really make
sense given each run should be independent I thought), but this
can't hurt.

Change-Id: I3b639f1c783efc140028c3acdce35ef3f58bf66f
2013-04-17 16:20:48 -03:00
Siebrand Mazeland
6da93fc6f6 Update code formatting
Also update some previous inconsistencies pointed out by Krinkle in change IDs:
* Ide20743a2e84ff68549286120e6cff9d9f396f54
* I811ca957b6588085d67606ebc0cd4033a1e53839

Change-Id: Ife33b931870d0d7e04fcb40974997436d27f528f
2013-03-27 14:15:11 +01:00
ASchulz
c328cf6401 [FileBackend] Cleanup behavior for coping/moving a file over itself.
* Previously, for doQuickOperations(), the default move function
  would just delete the file if it was moved over itself. In fact,
  the php-cloudfiles bindings have this same bug in its move function.
* For both copy and move, when the source and destination are the
  same path, make sure that the headers get updated as specified.
  This only applies for the 'overwrite' case (not 'overwriteSame').
* Fixed bad status for doQuickOperations() when copying a file over
  itself using FSFileBackend.
* Clarified the documentation for 'overwriteSame' option.
* Renamed destSameAsSource to overwriteSameCase in FileOp for clarity.

Change-Id: I3f609d9413795c654de27958b1e807b1fc785f31
2013-03-25 17:06:30 -07:00
Timo Tijhof
b36d883017 Tests: Make phpunit providers "public static".
Follows-up I9d2b148e57 (including phpunit/languages this time).

Bug: 46434
Change-Id: I30e5efcd88c516121c454676bd7a18f9b7c8fca6
2013-03-22 03:12:37 +01:00
Aaron Schulz
1045622590 [FileBackend] Fixed "ignoreMissingSource" bug in multiwrite backend.
* Beefed up unit tests for this case and other "ignoreMissingSource" cases
  such as when the source container or parent directory do not exist.

Change-Id: Iea6dae2424edfd0c4367e5cff606c09a4e8a865b
2013-03-12 13:39:40 -07:00
Aaron Schulz
2135b80fcd [LockManager] Various fixes to lock managers.
* Improved handling of corrupt values in cache for MemcLockManager.
  Also improved the use of Status warnings a bit.
* Removed broken special-case handling for SH->EX lock escalation.
  Updated MysqLockManager to compensate.
* Made FSLockManager only use one handle per file, which is more
  efficient and avoids errors when escalating locks (SH->EX).
* Made lock unit tests have more useful output on failure.

Change-Id: Ib304712fa2b6b3fd02bfc1b08b6f238c771960c2
2013-03-12 04:14:47 +00:00
Aaron Schulz
98dc0349b2 [LockManager] Added ScopedLock::release function.
* Added more lock unit tests.

Change-Id: Ic359307cf26cfb621ae8d2db867801735a375dde
2013-03-05 13:33:51 -08:00
Antoine Musso
593d873a83 style
tests/phpunit/includes/StringUtilsTest.php:42:3
 - avoid function calls in a FOR loop test part

tests/phpunit/includes/PathRouterTest.php:155 to 165
 - fix space/tabs indenting
 - convert double quotes to single for consistency

tests/phpunit/data/xmp/7.result.php
 - reindent to use tabs instead of spaces
 - removed trailing whitespaces

tests/phpunit/includes/filebackend/FileBackendTest.php
 - spaces in indentation

Change-Id: I9c15803f961fc88b798ada3dd6c2b292c1de2143
2013-02-20 08:34:03 +00:00
Siebrand Mazeland
e509e2c3a1 Fix parentheses
Change-Id: I6e7d99796635ed8f4ec462ae8661069b8518e5c5
2013-02-14 14:08:51 +01:00
Antoine Musso
0fd05285d7 pass codesniffer on tests/
Fix almost all occurences of the following sniffs:

Generic.CodeAnalysis.UselessOverridingMethod.Found
Generic.Formatting.NoSpaceAfterCast.SpaceFound
Generic.Functions.FunctionCallArgumentSpacing.SpaceBeforeComma
Generic.Functions.OpeningFunctionBraceKernighanRitchie.BraceOnNewLine
Generic.PHP.LowerCaseConstant.Found
PSR2.Classes.PropertyDeclaration.ScopeMissing
PSR2.Files.EndFileNewline.TooMany
PSR2.Methods.MethodDeclaration.StaticBeforeVisibility

Change-Id: I96aacef5bafe5a2bca659744fba1380999cfc37d
2013-01-28 12:14:26 +01:00
Aaron Schulz
1063008826 [FileBackend] Fixed bug where "false" could be included in getDirectoryList().
Change-Id: Ib14192d4f5c5143b07af2191b62ea3919483445a
2013-01-23 12:50:21 -08:00
Aaron Schulz
2e0e6a0013 [FileBackend] Added back ability to delete file HTTP headers.
* This partially reverts b80bd6159b.
* Clarified 'headers' parameter documentation a bit.
* Added the "describe" function wrappers that the other ops have.

Change-Id: I7142fd4d973f5f3407d244a4cae793934083bc26
2012-11-21 05:30:27 +00:00
Jan Gerber
b80bd6159b [FileBackend] Added support for changing headers on existing objects.
* Added a 'describe' file operation type to doOperations()/doQuickOperations().
  This can be used by scripts to fill in headers like X-Content-Duration for
  files that already exists.
* Removed wrong comments about removing headers (they don't get removed with null).
* Added some quick unit tests.

Change-Id: I43c5907b59421beaa9487eefac0cdbf8bc6c6d85
2012-11-20 16:07:28 -08:00
umherirrender
9e504dd8e6 Unset DirectoryIterator in the FileBackend
This is necessory to close all handlers on the directory and than it is
possible to rmdir the directory without a 'permission denied' on a
windows machine.

Corresponding unit test failure:
1) FileBackendTest::testRecursiveClean
Dir mwstore://localtesting/unittest-cont1/e/a no longer exists
(FSFileBackend).
Failed asserting that true matches expected false.

Change-Id: Ide58c6d0ec101bf0ddcd6b7414c51075fc28825b
2012-11-16 12:55:22 -08:00
Aaron Schulz
9f95bddda7 [FileBackend] Added getFileHttpUrl() function.
* This can speed up certain video file operations for scripts that support
  specifying source files via URLs and support HTTP Range headers.
* Updated unit tests.

Change-Id: I60cb95c2e3dd9f7df1f740e9182be7c79af69d6e
2012-11-08 23:01:19 +00:00
Aaron Schulz
963c4a265f Moved FileBackend tests to /filebackend.
Change-Id: Ieb789ac1c9803f74bad3a4fef3a57587d6dd6260
2012-11-02 12:12:36 -07:00
Renamed from tests/phpunit/includes/filerepo/FileBackendTest.php (Browse further)