Commit graph

298 commits

Author SHA1 Message Date
Aaron Schulz
0fa9a391e3 filebackend: Avoid using popen() when "parallelize" is disabled
Bug: 66467
Change-Id: Iaf8eb2ecfad166e26e265a98933399dc4bc255a3
2014-06-18 20:12:03 +00:00
Aaron Schulz
4e72a7dfa4 Fixed FSFileOpHandle comment
Change-Id: Ie89d18f173f4fb172c35317cee9403e5f5f43d47
2014-06-13 10:17:33 -07:00
Aaron Schulz
c36eb818f8 Made FSFileBackend use closures for doExecuteOpHandlesInternal()
Change-Id: I8401a92384cb454eb79a4607b350c50d432a9510
2014-06-12 13:40:03 -07:00
Aaron Schulz
c699000ae0 Fixed fatal in SwiftFileBackend when a bogus HTTP status is given
* Error was "Fatal error: Call to a member function getSize() on a non-object"

Change-Id: Id1a1b1add9daee4402b2ed017b95c9a9f0e08170
2014-05-30 11:48:14 -07:00
Aaron Schulz
1d59d6152e Improved FileBackend failure logging a bit
* This partly follows up on 2cfce9f

Change-Id: I9d15f44d8000f52fd7b307d358c895452ee96a8d
2014-05-22 17:37:33 -07:00
Aaron Schulz
399198c28e Replace FOR UPDATE with LockManager use in LocalFile::lock()
* This avoids excess contention where inserts of rows for
  similarly named files get blocked. This often would effect
  users of UploadWizard or any bot doing mass uploads.

Change-Id: Ie7a328f7d4f03aa249770804417347a50356ea42
2014-05-20 14:22:52 -07:00
Aaron Schulz
0d54e6e0bf Added unicode encoding support flags to FileBackend
* Fixed bug where even using Swift/Azure on Windows
  would disallow non-ASCII file names.

bug: 1780
Change-Id: I19ed72da0b099d35cae74fb08eeb22c113da1065
2014-05-13 14:11:36 -07:00
Aaron Schulz
45c1d73bdb Made partial file download errors in Swift more useful
Change-Id: I97a5cd1ed38508468122e33968b88adf2570a9a3
2014-05-01 22:44:24 +00:00
jenkins-bot
f0d1e12ffa Merge "Avoid bogus IE extension check errors in img_auth.php" 2014-04-22 20:59:04 +00:00
Aaron Schulz
a650998aa7 Avoid bogus IE extension check errors in img_auth.php
Change-Id: I57083d3fe0517b94d3b786970b412e6ec51cf5f0
2014-04-21 19:30:48 -07:00
Aaron Schulz
603d8970c3 Made TempFSFile try to purge files on fatals too
Change-Id: I3e80bea0d869be989c951dfb35a218bb66c9de17
2014-04-21 11:18:39 -07:00
jenkins-bot
a21d59cc6e Merge "Make room for preloadFileStat() call in FileBackend::doOperationsInternal" 2014-04-21 17:13:35 +00:00
jenkins-bot
e40b19ed4e Merge "Bail out on FileBackend operations if the initial stat calls failed" 2014-04-21 17:13:33 +00:00
umherirrender
957adbef22 Fixed some @params documentation (includes/[file...|upload])
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.

Change-Id: I8804ebe0922d3a414863b162a2110e0b9e49b80f
2014-04-19 17:19:17 +02:00
umherirrender
b9cd789fce docs: closure -> Closure; callback -> callable
Changed closure to capital word Closure in doc and type hint,
also changed callback in docs to callable

Change-Id: I52c8e8f13d38a837052101c38b9986be780ca057
2014-04-19 08:43:31 +02:00
Aaron Schulz
d52a559fba Make room for preloadFileStat() call in FileBackend::doOperationsInternal
Change-Id: I60f71155abcf7f69423639b10ec301ac192df728
2014-04-18 17:02:19 -07:00
Aaron Schulz
2cfce9fd8d Bail out on FileBackend operations if the initial stat calls failed
* The preloadFileStat() call acts as a good canary for whether all
  affected servers are up. If anything failed in it, then it's not
  worth bothering to send the actual write requests.
* Also made FileBackend::preloadFileStat abstract while at it since
  the two subclasses implemented it and so should any others.
* Fixed a silly comment typo.

Change-Id: I5bd427f654aa4a9d6bfe4ed7566276e8ac520b30
2014-04-18 12:41:23 -07:00
Aaron Schulz
62505ef38d Made runParallelBatches() fully set the status array on abort
Change-Id: Ie35b28b85e734db6910a42ed384374ba3d594a8e
2014-04-18 18:04:04 +00:00
Aaron Schulz
8aaab01e1d Made preloadFileStat/preloadCache actually work for FileBackendMultiWrite
* Previously the path substitution did not take place

Change-Id: I58ad3e2aa44a2b08f4c6bad1a783fb92ea79dbf4
2014-04-12 15:42:39 -07:00
umherirrender
23fab68274 Fix spacing after @param and friends in comments
Searched for:
\@(param|return|throws|since|deprecated|access|todo|var)[ \t]{2,}

Change-Id: Icce22ba9fe0635455691ca58d9872d618151f346
2014-04-05 20:02:29 +00:00
Aaron Schulz
02e0d115a1 Tweaked "latest" handling of filebackend stat entries
* Let "latest" stat entries override non "latest" so
  that future getFileStat() calls with the "latest" flag
  can actually have a cache hit.

Change-Id: I1e9391039537d608b89773b4d51575e3b364a751
2014-03-24 06:55:01 +00:00
umherirrender
2000672ac3 Fixed spacing
- Added spaces after if/foreach/catch
- Added new line before end of file
- Added or removed spaces before/after parenthesis, comma
- Added spaces around string concat

Change-Id: I0590070f1b3542108e242730e8d9a3ba9831e94f
2014-03-20 20:37:30 +00:00
Aaron Schulz
6112acac29 Make use of strong consistency with Ceph RGW
Change-Id: Ic3b764363e91c3279d3dc017ddce630dae26bec5
2014-03-19 17:27:48 +00:00
Aaron Schulz
4f7a786fbe Remove useless "src" param when wrapping doGetFileStatMulti()
Change-Id: I6008384ea872d71dcd2d183186a4aee5048b4dd4
2014-03-03 10:02:31 -08:00
Aaron Schulz
96ae9c07e3 Removed pointless cache bypass in Swift backend
* This was added by mistake in 2af7ad8 apparently

Change-Id: Ic4004428095953ce07e1e97da43748eba04f6639
2014-02-25 17:44:24 -08:00
Aaron Schulz
f5f2233daa Avoid key fragmentation in FileBackend
* Make sure that global backends do not prefix the cache keys
  with local wiki IDs for container and file stat entries.
  This causes fragmentation and breaks invalidation.

Change-Id: Ic02d6e41f828dc82cc07c7d24c8af29c46392a9c
2014-02-24 13:22:15 -08:00
Aaron Schulz
6b1a60a1ca Added concurrent HEAD request support for Swift
* This adds a new preloadFileStat() method to FileBackend.
* FileBackend::doOperations() makes use of this instead of primeFileCache().
  The later was mostly useless due to the 'latest' flag, which stat entries
  rarely had (and since they use add(), 'latest' ones do not override the
  non-'latest' ones).
* Simplified primeContainerCache/primeFileCache to no longer accept
  FileOp objects and made sure nothing passes those in anymore.

Change-Id: Iee3878cadd69c3a594c701c0aff81aab5f473de5
2014-02-20 08:50:08 +00:00
Aaron Schulz
6d377ae0e3 Added some profiling calls to Swift backend
Change-Id: Ic692c897493fe4388568930c2f4f861b89f2ff78
2014-02-18 11:32:51 -08:00
Aaron Schulz
0259fccfe3 Actually write-back the container cache in SwiftFileBackend
Change-Id: Ia072dd09c1fa5b4dec20a80e69425f7364488362
2014-02-16 07:33:13 +00:00
Aaron Schulz
4265d8f23f Added simpler handleError() method in RedisConnectionPool
* Callers should not need to give $server since it is stored in RedisConnRef

Change-Id: I902d984d6a7f19dd0d8c71ee374cbed359de378e
2014-02-12 07:29:32 +00:00
Aaron Schulz
3d670f1719 Reduce the amount of Swift auth cache keys used on wiki farms
Change-Id: Ic0a2f744d5d94bfb22286a19be3de0c37ecdf519
2014-02-12 07:07:20 +00:00
Aaron Schulz
288941aa53 Fixed broken auth caching for Swift
* A set() call was missing since 2af7ad8

Change-Id: I62b277e886c7ce6142a4d20f705b8337c3e4be58
2014-02-12 06:29:44 +00:00
Aaron Schulz
914fb04afa Simplified Swift backend to use maxConns flag in MultiHttpClient
Change-Id: Iee3756a490243511be86676c01d0a47940529e0b
2014-02-12 00:08:48 +00:00
Siebrand Mazeland
f077c4b1d6 Update formatting
Change-Id: I18aff576262479c9bb1c56eb8e1d1aaae200e4b1
2014-02-06 09:27:05 +01:00
Siebrand Mazeland
0cef781886 Fix CodeSniffer errors and warnings
Change-Id: Idc74e34634d88625773fb8f73315f61edfa67e28
2014-02-05 11:20:17 +01:00
Aaron Schulz
da802de2f2 Use "rb" instead of "rb+" for Swift store operation handles
bug: 60318
Change-Id: I411b2505287d4e4addff3231f9374775353b347b
2014-02-03 23:17:19 +00:00
Aaron Schulz
4db00dc184 Fixes and cleanups to FileOpBatch
* Fixed bug where operations that failed after precheck() would
  not properly update the Status. This could cause failed ops to
  be treated as successful.
* Removed special casing for 1-sized batches, this was mostly
  for Swift and is irrelevant after it was rewritten.
* Removed unused return value and fixed the docs

bug: 60318
Change-Id: I7f12ebf711bc196313745f943070f8bdb6335964
2014-02-03 23:11:44 +00:00
Aaron Schulz
a9424c3c3e Use ProfileSection in FileOpBatch
Change-Id: I5453130f89a2b46a18dc6ecfb23b2f75c6c2f16f
2014-02-03 12:59:04 -08:00
jenkins-bot
0db4c8d514 Merge "Always return an array from SwiftFileBackend::addMissingMetadata" 2014-01-27 17:59:48 +00:00
Bryan Davis
03efc39055 Always return an array from SwiftFileBackend::addMissingMetadata
Correct the behavior of SwiftFileBackend::addMissingMetadata to match
the documentation and calling code expectations by always returning an
array.

Bug: 60477
Change-Id: I8d12888a6ebfa23502f8cf3a28866dcb00a18ec8
2014-01-27 09:54:25 -08:00
addshore
6e57736499 Fix phpdoc in SwiftFileBackend
Docs for $status and $code were swapped.

Change-Id: I88bd81b671f0e9a4176483fb744e45f2477672b5
2014-01-27 16:07:55 +00:00
Aaron Schulz
a0eef4ef0d Removed expiremental LSLockManager class
* This is pretty much obsoleted by RedisLockManager

Change-Id: I0feff44b5833072c8eb5a598169e1bb271d49de2
2014-01-17 15:16:01 -08:00
Aaron Schulz
688200c607 Fixed use of X-Newest in Swift backend
Change-Id: I9c9572269cd5366272070ccba954c3bd417aa696
2014-01-15 16:48:59 -08:00
Aaron Schulz
f46cd2a593 Fixed wrong usage of $tmpFile in Swift backend
Change-Id: I21293be38c53b4dca3030c26a201f70d8ed243f3
2014-01-15 13:05:01 -08:00
Aaron Schulz
c00ab2af44 Fixed temp URL breakage in Swift
* The API behavior is not well-documented in that it wants the HMAC
  to use the unencoded object name. CloudFiles had this same issue.

bug: 59894
Change-Id: I3f0c80eed55991bb85d0c12f98927efa19af6760
2014-01-13 19:50:41 +00:00
Aaron Schulz
12d88861c5 Fixed silly doc typos
Change-Id: I53792affed74aa5985f82994b886b13603e23c65
2014-01-09 20:25:25 -08:00
jenkins-bot
159cdf0d94 Merge "Fixed file props test failure for MemoryFileBackend" 2014-01-09 22:22:26 +00:00
Aaron Schulz
c745ff9a58 Fixed bogus var usage in SwiftFileBackend
Change-Id: Ia0dec914208cb3100b7f5bbd2291985eac05dc02
2014-01-09 12:39:13 -08:00
Aaron Schulz
83cc94cd81 Fixed file props test failure for MemoryFileBackend
Change-Id: Ia593023569e3874e11598c38961a68a0da75883f
2014-01-09 11:04:20 -08:00
jenkins-bot
00e5290c56 Merge "filebackend: Added supported for retrieving file metadata/headers" 2014-01-09 18:36:43 +00:00