Commit graph

142 commits

Author SHA1 Message Date
Aaron Schulz
c11e38e3db Dependency inject $wgTmpDirectory into FileBackend classes
Change-Id: I219daffa67ff672bf5bd38921f4b1ca058d96b0f
2016-09-21 04:00:49 +00:00
Aaron Schulz
c64c3ef9a8 Make Status extend StatusValue and start FileBackend update
* This avoids code duplication.
* Callers can safely start type-hinting StatusValue as well.
* Also moved the wrap() logic out of Status::__construct(), which
  only wrap() was (and should have been) using. Use "static" as
  well, so subclass behave properly.
* The docs and type-hints in /filebackend are updated as an example.
* A migration pattern is to inject a StatusValue wrapper
  into the backend and use it on all returned statuses, so MediaWiki
  will still get Status for message methods.

Change-Id: Iff9255f34870ea6b0c4b91f6ddc69eea95186aba
2016-09-17 15:45:40 -07:00
Aaron Schulz
d42a05475a Added Range support to FileBackend::streamFile()
* Added HTTP options headers parameter to streamFile().
* Refactored doStreamFile() to either call StreamFile::stream()
  or delagate that to the subclass. SwiftFileBackend now relays
  the full Swift response rather than manually making the headers.
  This also makes Range headers easy to support.
* Made use of this in img_auth.php for performance on private wikis.
* Elimate stat call in streamFile() for Swift if "headers" is empty.
* Refactored StreamFile a bit to inject request headers instead
  of using the $_SERVER global. A header options parameter is used
  instead, which also supports Range.
* Removed now unused prepareForStream().
* Cleaned up streamFile() unit tests.

Change-Id: I2ccbcbca6caabb8cf65bd6b3084cede2e6ea628a
2016-06-04 05:43:43 -07:00
Max Semenik
943563062f Kill mbstring fallbacks
In the age when we require PHP 5.5, pretending that mbstring emulation
is not slow and silly is silly.

Bug: T129435
Change-Id: Ic8235c9da9a926df63ec7388900c44eab454eebe
2016-04-06 22:14:48 +00:00
Reedy
b5656b6953 Many more function case mismatches
Change-Id: I5d3a5eb8adea1ecbf136415bb9fd7a162633ccca
2016-03-19 00:20:58 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Ricordisamoa
b982fbbd07 Stop doing $that = $this in includes/filebackend
Closures support $this as of PHP 5.4

Change-Id: Ib11139ec81336710c22146f9ff714e8afd3aa2cf
2016-02-10 18:33:52 +01:00
Reedy
00c426e3c2 Replace wfBaseConvert with Wikimedia\base_convert
Change-Id: Iadab3d018c3559daf79be90edb23d131729bdb68
2015-11-24 22:51:42 +00:00
Aaron Schulz
ddd2454967 Make Swift iterators throw errors on failure
This is in line with FSFileBackend and is provides the calling
loop of a way to know the listing failed, rather than just
giving incomplete information.

Change-Id: I34c66d9dbf84e5f33982d17e61364c1595d3e9f7
2015-11-06 16:10:05 -08:00
Aaron Schulz
17c91ad610 Replace newAccelerator() with getLocalServerInstance()
The name is clearer and more consistent, with simpler arguments.

Change-Id: I7205a99ce033e8b086a52cd02c8a721e99c84b1e
2015-11-02 21:39:08 +00:00
Aaron Schulz
0e3bfab533 Make Swift backend respect Content-Type in create/store
Change-Id: I479d1fc4f261bdcdc380404db0cb2f2397877759
2015-10-26 11:16:18 -07:00
Aaron Schulz
72734cfffa Replace some calls to wfGetMainCache()
Change-Id: I9ba8fd5918877af6312b0854634bd36211438f00
2015-10-20 23:52:07 -07:00
jenkins-bot
6c4601cfbd Merge "Fixed headers sent in addMissingMetadata()" 2015-08-27 08:08:08 +00:00
Timo Tijhof
f1223f90fd objectcache: Use newAccelerator() fallback instead of try/catch
Also remove confusing use of $wgMemc in LoadMonitorMySQL which
should always be the same as wfGetMainCache().

Change-Id: I4fb9d075a37d3d45af71a5026ccf2eb17f24d7b0
2015-08-25 01:03:10 +02:00
Timo Tijhof
79ce51d1c9 objectcache: Make first parameter of newAccelerator optional
Makes it more convenient to use.

Change-Id: I1e11f7a759bd2816e47d1c2453cbe39b8f44b2f0
2015-08-24 23:43:19 +02:00
Aaron Schulz
84e389ec68 Fixed headers sent in addMissingMetadata()
* This used to relay back *all* of the GET headers, which includes
  things that are not changeable (Content-Length) or make no sense
  (x-trans-id). Only send the actual HTTP/metadata headers. Oddly
  enough sending garbage headers gives a 2XX response, but does
  not actually change anything in Swift.

Bug: T105810
Change-Id: Id4829cf998a6d8763fd26d49cd8d9c16cf5527dd
2015-08-05 01:46:52 -07:00
Aaron Schulz
44b1fa8edb Cleaned up and split up Swift header parsing methods a bit
* Added tests for the metadata headers portion of the code

Change-Id: I8ac65e31212b4cca4592f963e0ca5ad30e1349f7
2015-08-05 01:12:50 -07:00
Aaron Schulz
6cef0373c7 Added sanitizeHdrs() tests for Swift
Change-Id: I2e3c3225c729e5220ca16f6ef4518da49e7f721c
2015-07-29 18:57:23 -07:00
jenkins-bot
3eb00083e9 Merge "Clear the stat cache in addMissingMetadata() to avoid more POSTs" 2015-07-28 13:45:00 +00:00
Aaron Schulz
6e7c034901 Improved addMissingMetadata() on POST failure
* If the POST failed but the sha1 was computed, then use
  and cache that value rather than "false".

Change-Id: I42b53c823013ecd9b281406e3d533a21e0de7cfb
2015-07-22 13:19:12 -07:00
Aaron Schulz
54eb877a52 Clear the stat cache in addMissingMetadata() to avoid more POSTs
Change-Id: Icc075e424bdbed6868692ec734dff1e7d2003dd6
2015-07-22 12:04:47 -07:00
Aaron Schulz
83a9ac3ee8 Avoid using trigger_error() in addMissingMetadata()
* Just use the normal swift log instead

Bug: T105810
Change-Id: Ifa463e6298a1b57ab6a8d92f9defc744619d9fb6
2015-07-17 20:29:10 +00:00
Kunal Mehta
f6e5079a69 Use mediawiki/at-ease library for suppressing warnings
wfSuppressWarnings() and wfRestoreWarnings() were split out into a
separate library. All usages in core were replaced with the new
functions, and the wf* global functions are marked as deprecated.

Additionally, some uses of @ were replaced due to composer's autoloader
being loaded even earlier.

Ie1234f8c12693408de9b94bf6f84480a90bd4f8e adds the library to
mediawiki/vendor.

Bug: T100923
Change-Id: I5c35079a0a656180852be0ae6b1262d40f6534c4
2015-06-11 18:49:29 +00:00
jenkins-bot
5161541d87 Merge "Converted FileBackendStore to using the WAN cache" 2015-05-18 08:34:10 +00:00
Timo Tijhof
19bff2537e Consistently name the $this callback variable "$that"
Most are already, but some had differnet names. In PHP 5.4+ this is
redundant as they inherit automatically, but we need these for PHP 5.3
compatibility. Settle on "$that" to make these easier to find.

Change-Id: I0b68b2c550fde9c2da53d844421e34b8df0c42ed
2015-05-13 23:05:07 +01:00
Aaron Schulz
48cffdff82 Converted FileBackendStore to using the WAN cache
Bug: T93141
Change-Id: I76efb0dc45a697c5fdfc50932e1198a6d663a445
2015-04-30 01:30:38 +00:00
jenkins-bot
b7ec2c9b33 Merge "Fix and make PHPDoc tags in FileBackend more specific" 2015-03-18 16:45:20 +00:00
Thiemo Mättig
d71f05290e Fix and make PHPDoc tags in FileBackend more specific
A pure documentation patch. Should almost be a no-brainer. ;-)

Change-Id: I9bf4d6056f7cde193fe5882a04a7973af9b5a7f4
2015-03-18 17:22:52 +01:00
jenkins-bot
29790fbc04 Merge "Set stat cache when getting local copy files in Swift" 2015-01-14 21:26:11 +00:00
Aaron Schulz
6921770414 Updated some try-catch statements: MWException -> Exception
Change-Id: I76601a86e30f4984e3b1a8c8ec5ef5a0f652433a
2015-01-09 17:20:22 -08:00
Aaron Schulz
d81ad30744 Set stat cache when getting local copy files in Swift
Change-Id: I672051b8d04a5db4da8236deba48b266ec7dd9ab
2015-01-09 23:18:26 +00:00
Aaron Schulz
4ff8136807 Removed remaining profile calls
Change-Id: I31c81c78715048004fc8fca0f27d09c1fa71c118
2015-01-08 02:49:33 -08:00
Aaron Schulz
3226337f17 Converted some section profiling to scopedProfileIn
Change-Id: If69de34a046f4f33fc57b93beaf264f522ce7386
2015-01-07 14:19:06 -08:00
Gilles Dubuc
a6f2310cc7 filebackend: Log more details about backend-fail-internal errors
Bug: 73094
Change-Id: I9e45284e3cfd55bc34d87009387a2c1766e2db4a
Mingle: https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/991
2014-11-17 23:36:47 +00:00
Aaron Schulz
2565f65ff0 Bumped the default Swift token expiry a bit
Change-Id: I59f698a024180410ffd0c02290f6d61db5999f12
2014-09-24 17:04:01 -07:00
umherirrender
21e0c1c533 Correct variable names in @param to match method declarations
Some @param have a typo in the variable name,
some @param's were in wrong order.

Change-Id: Ie25806831027112b398f6f4a909c59147ac3a5fa
2014-08-13 21:48:28 +02:00
umherirrender
b883e8c7c0 Cleanup some docs (includes/[e-l])
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling

Change-Id: I41a84e8e1dec39170aa655250325ffc485eaeaef
2014-07-24 19:43:03 +02:00
umherirrender
4ee680a8b3 Fixed spacing
- Removed spaces after not operator (!)
- Removed spaces inside array index
- use tab as indent instead of spaces
- Add newline at end of file
- Removed spaces after casts

Change-Id: I9ba17c4385fcb43d38998d45f89cf42952bc791b
2014-07-24 11:53:04 +02:00
Aaron Schulz
c699000ae0 Fixed fatal in SwiftFileBackend when a bogus HTTP status is given
* Error was "Fatal error: Call to a member function getSize() on a non-object"

Change-Id: Id1a1b1add9daee4402b2ed017b95c9a9f0e08170
2014-05-30 11:48:14 -07:00
Aaron Schulz
0d54e6e0bf Added unicode encoding support flags to FileBackend
* Fixed bug where even using Swift/Azure on Windows
  would disallow non-ASCII file names.

bug: 1780
Change-Id: I19ed72da0b099d35cae74fb08eeb22c113da1065
2014-05-13 14:11:36 -07:00
Aaron Schulz
45c1d73bdb Made partial file download errors in Swift more useful
Change-Id: I97a5cd1ed38508468122e33968b88adf2570a9a3
2014-05-01 22:44:24 +00:00
umherirrender
957adbef22 Fixed some @params documentation (includes/[file...|upload])
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.

Change-Id: I8804ebe0922d3a414863b162a2110e0b9e49b80f
2014-04-19 17:19:17 +02:00
umherirrender
2000672ac3 Fixed spacing
- Added spaces after if/foreach/catch
- Added new line before end of file
- Added or removed spaces before/after parenthesis, comma
- Added spaces around string concat

Change-Id: I0590070f1b3542108e242730e8d9a3ba9831e94f
2014-03-20 20:37:30 +00:00
Aaron Schulz
6112acac29 Make use of strong consistency with Ceph RGW
Change-Id: Ic3b764363e91c3279d3dc017ddce630dae26bec5
2014-03-19 17:27:48 +00:00
Aaron Schulz
4f7a786fbe Remove useless "src" param when wrapping doGetFileStatMulti()
Change-Id: I6008384ea872d71dcd2d183186a4aee5048b4dd4
2014-03-03 10:02:31 -08:00
Aaron Schulz
96ae9c07e3 Removed pointless cache bypass in Swift backend
* This was added by mistake in 2af7ad8 apparently

Change-Id: Ic4004428095953ce07e1e97da43748eba04f6639
2014-02-25 17:44:24 -08:00
Aaron Schulz
6b1a60a1ca Added concurrent HEAD request support for Swift
* This adds a new preloadFileStat() method to FileBackend.
* FileBackend::doOperations() makes use of this instead of primeFileCache().
  The later was mostly useless due to the 'latest' flag, which stat entries
  rarely had (and since they use add(), 'latest' ones do not override the
  non-'latest' ones).
* Simplified primeContainerCache/primeFileCache to no longer accept
  FileOp objects and made sure nothing passes those in anymore.

Change-Id: Iee3878cadd69c3a594c701c0aff81aab5f473de5
2014-02-20 08:50:08 +00:00
Aaron Schulz
6d377ae0e3 Added some profiling calls to Swift backend
Change-Id: Ic692c897493fe4388568930c2f4f861b89f2ff78
2014-02-18 11:32:51 -08:00
Aaron Schulz
0259fccfe3 Actually write-back the container cache in SwiftFileBackend
Change-Id: Ia072dd09c1fa5b4dec20a80e69425f7364488362
2014-02-16 07:33:13 +00:00
Aaron Schulz
3d670f1719 Reduce the amount of Swift auth cache keys used on wiki farms
Change-Id: Ic0a2f744d5d94bfb22286a19be3de0c37ecdf519
2014-02-12 07:07:20 +00:00