Commit graph

386 commits

Author SHA1 Message Date
jenkins-bot
7e6a9ce4ad Merge "filebackend: Clean up TempFSFile and fix IDEA errors" 2016-03-03 01:38:05 +00:00
Aaron Schulz
b3c80a0ff5 filebackend: Clean up TempFSFile and fix IDEA errors
Change-Id: I4e25e3bf906fa3a918f4462fac1a6be5e85696aa
2016-03-03 01:25:37 +00:00
Aaron Schulz
a5d903860a Allow FSFile objects for src in FileBackend::do*Operations
Convenience aside, this lets multiwrite backends do async replication for
"store" operations safely, buy keeping a handle to the source file that
prevents it from getting prematurely deleted before the post-send writes
to the secondary backends can happen.

Bug: T91869
Change-Id: I1254de527c47835c35fed6e526b42953c1b2b2ca
2016-03-02 18:49:56 +00:00
Bartosz Dziewoński
c161c46d26 Improve code suffering from PHP 5.3's lack of support for foo()[]
I searched for /\$(\S+) = (.+?\(.*?\);)\n.*?\$\1\[/, ignored
everything involving isset(), unset() or array assigments, then
skimmed through the remaining results and changed things where they
made sense. These changes were not automated, so please review them.

Change-Id: Ib37b4c66fc57648470f151ad412210b3629c2538
2016-02-28 22:49:20 +01:00
Aaron Schulz
c0446571d5 Use sync writes for replication of 'store' operations
The local temporary file might not live long enough to be
copied into storage during post-send.

Bug: T128124
Change-Id: Ifb5260958db008ec8b9f4db90f95a42e5ecaeadc
2016-02-27 00:20:13 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Ricordisamoa
b982fbbd07 Stop doing $that = $this in includes/filebackend
Closures support $this as of PHP 5.4

Change-Id: Ib11139ec81336710c22146f9ff714e8afd3aa2cf
2016-02-10 18:33:52 +01:00
jenkins-bot
3639bb8736 Merge "Fix some FSFileBackend IDEA errors" 2015-12-31 01:18:46 +00:00
Aaron Schulz
1e03c50891 Fix some FSFileBackend IDEA errors
Change-Id: I372ad7af21f223b670498ac17ecac90918ceb2a8
2015-12-29 01:46:05 -08:00
Aaron Schulz
d1f9b125f9 Fix some FileOpBatch IDEA errors
Change-Id: Idd670054b2e257255da18bb8e7048ea9041be269
2015-12-29 01:34:22 -08:00
Reedy
00c426e3c2 Replace wfBaseConvert with Wikimedia\base_convert
Change-Id: Iadab3d018c3559daf79be90edb23d131729bdb68
2015-11-24 22:51:42 +00:00
umherirrender
78328dee73 Use Database::fetchRow instead of array cast
Change-Id: I71435955de604a9b3c2b2570e144b55b35421e4c
2015-11-20 20:44:56 +01:00
Aaron Schulz
ddd2454967 Make Swift iterators throw errors on failure
This is in line with FSFileBackend and is provides the calling
loop of a way to know the listing failed, rather than just
giving incomplete information.

Change-Id: I34c66d9dbf84e5f33982d17e61364c1595d3e9f7
2015-11-06 16:10:05 -08:00
jenkins-bot
b85aeec5e2 Merge "Improve MIME detection in FileBackend" 2015-11-03 11:10:41 +00:00
Timo Tijhof
8a821011ce Remove getLocalServerInstance try/catch in UIDGenerator and DBLockManager
Follows-up 17c91ad610 in which the exception was removed.

Change-Id: If0c8465d0e04f69f29f3f2602dac7ae4538f7542
2015-11-02 22:16:05 +00:00
Aaron Schulz
17c91ad610 Replace newAccelerator() with getLocalServerInstance()
The name is clearer and more consistent, with simpler arguments.

Change-Id: I7205a99ce033e8b086a52cd02c8a721e99c84b1e
2015-11-02 21:39:08 +00:00
Aaron Schulz
de290cd02d Improve MIME detection in FileBackend
The content type detector will now inspect the file contents
to better handle extensionless files.

Also dependency inject the callback and make the default one
use FileInfo.

Change-Id: Iad59bf6c6a416b706f976a4c425763fd30e2debb
2015-11-02 00:36:54 +00:00
Aaron Schulz
642ca3862b Normalize header case for FileBackend operations
Normalize all headers to lower case at the start of the
FileBackend operation methods. This makes it easy for
subclasses to check for certain headers, e.g. content-type.

Change-Id: Ia69976326d17a51bcaa61f2781aa669ae7bd9c28
2015-10-27 12:34:35 -07:00
jenkins-bot
8ab6c38d4f Merge "Make Swift backend respect Content-Type in create/store" 2015-10-27 10:40:11 +00:00
Aaron Schulz
8c4e6beaf3 filebackend: Remove some old Content-Disposition b/c code
Change-Id: I6af4a84bd549e2bfb1b110fa790f499d144f236b
2015-10-26 18:16:27 +00:00
Aaron Schulz
0e3bfab533 Make Swift backend respect Content-Type in create/store
Change-Id: I479d1fc4f261bdcdc380404db0cb2f2397877759
2015-10-26 11:16:18 -07:00
Aaron Schulz
5262d41a6c Minor fix to primeFileCache() comment
Change-Id: I7d040c5a914574df72fa31612df0eb28e15fd46c
2015-10-23 17:18:56 -07:00
Aaron Schulz
72734cfffa Replace some calls to wfGetMainCache()
Change-Id: I9ba8fd5918877af6312b0854634bd36211438f00
2015-10-20 23:52:07 -07:00
Aaron Schulz
7c07943eab Update more docs and type hints to use IDatabase
Change-Id: I8c8d85b32a8aba21e14d2a2dde4c25eb509186c1
2015-10-06 18:49:52 -07:00
Aaron Schulz
28194278ea Fix duplicate automatic file backend bug
Follow-up 8a3816529a

Bug: T114810
Change-Id: I2706c90077627b3df35fe530e0a919cfd0a75d78
2015-10-06 14:50:12 -07:00
Aaron Schulz
8a3816529a Apply $wgReadOnly to all file backends
* Also added a wfConfiguredReadOnly() method
  to avoid DB_SLAVE connections

Change-Id: I9e7ec95c4b2f763505166d2345d27abaef6257a3
2015-10-03 00:15:28 +00:00
Aaron Schulz
169b7b98b5 Added replication=async option to FileBackendMultiWrite
* This will defer writes to non-master backends till the
  end up the web request. This is useful for multi-DC setups.

Bug: T112708
Change-Id: I118c07764dd4a4f4f2590d4548238df12860e750
2015-10-01 23:30:41 +00:00
Aaron Schulz
64c78c5e1e Set getFeatures() for MemoryFileBackend
Change-Id: I565e2e2c8a4a89c6706349a4ef9826f87445d1d2
2015-09-21 15:04:12 -07:00
jenkins-bot
b3fdef18a1 Merge "Added readAffinity flag to FileBackendMultiWrite" 2015-09-21 12:20:45 +00:00
jenkins-bot
d9859b32cd Merge "Removed old "noPush" config hacks from FileBackendMultiWrite" 2015-09-21 09:40:43 +00:00
jenkins-bot
de82ab7c24 Merge "Prevent IDEA error in doOperationsInternal()" 2015-09-21 09:31:01 +00:00
jenkins-bot
e5454da763 Merge "Removed unused LockManager var" 2015-09-21 09:29:47 +00:00
jenkins-bot
8d13ffae12 Merge "Made some FileBackendMultiWrite docs more accurate" 2015-09-21 09:29:24 +00:00
Aaron Schulz
fb0994d5c5 Prevent IDEA error in doOperationsInternal()
Change-Id: I2f0a1885803fff9dc0eb8ca4a595a83736ce421b
2015-09-20 23:04:29 -07:00
Aaron Schulz
639388755c Removed unused LockManager var
Change-Id: I02c1fef624e6f2fe04e3c6d525aaed881eacabd7
2015-09-20 13:57:46 -07:00
Aaron Schulz
bd6ec3bc78 Added readAffinity flag to FileBackendMultiWrite
* A backend with this set (normally 1) will be used for
  non-latest reads. This can be used to prefer a local,
  replicated, backend instead of one farther away (for
  multi-DC setups)
* Note that listings still come from the master always.

Bug: T112708
Change-Id: Ic4bf4ba5a2c9ef78abd11dbd0d4b48c73cad6923
2015-09-20 19:25:43 +00:00
Aaron Schulz
49e9910d0f Made some FileBackendMultiWrite docs more accurate
Change-Id: I14c152601fef3d3c75e7ea7aac72ff87c9f5fd52
2015-09-19 21:11:53 -07:00
Aaron Schulz
92b9ee98a1 Removed old "noPush" config hacks from FileBackendMultiWrite
Change-Id: I80e688cc61c0a656524a0db6aeabb3d3f6e667bd
2015-09-18 17:36:09 -07:00
Aaron Schulz
5cea7769d3 Cleaned up getScopedFileLocks() return value
* Just return the ScopedLock itself without wrappig it in an array.
  This also makes getScopedLocksForOps() actually return the correct
  type according to the docs

Change-Id: Ic2d01d56c3fcb22af3adb5e5d4d1586d62316552
2015-09-18 17:58:11 +00:00
Aaron Schulz
b2ba479eca Fixed some IDEA doc warnings
Change-Id: I9ccc6599d5fbcc143ef33a5619b3c1d1c52e192f
2015-09-11 12:38:37 -07:00
jenkins-bot
6c4601cfbd Merge "Fixed headers sent in addMissingMetadata()" 2015-08-27 08:08:08 +00:00
Timo Tijhof
f1223f90fd objectcache: Use newAccelerator() fallback instead of try/catch
Also remove confusing use of $wgMemc in LoadMonitorMySQL which
should always be the same as wfGetMainCache().

Change-Id: I4fb9d075a37d3d45af71a5026ccf2eb17f24d7b0
2015-08-25 01:03:10 +02:00
Timo Tijhof
79ce51d1c9 objectcache: Make first parameter of newAccelerator optional
Makes it more convenient to use.

Change-Id: I1e11f7a759bd2816e47d1c2453cbe39b8f44b2f0
2015-08-24 23:43:19 +02:00
Aaron Schulz
84e389ec68 Fixed headers sent in addMissingMetadata()
* This used to relay back *all* of the GET headers, which includes
  things that are not changeable (Content-Length) or make no sense
  (x-trans-id). Only send the actual HTTP/metadata headers. Oddly
  enough sending garbage headers gives a 2XX response, but does
  not actually change anything in Swift.

Bug: T105810
Change-Id: Id4829cf998a6d8763fd26d49cd8d9c16cf5527dd
2015-08-05 01:46:52 -07:00
Aaron Schulz
44b1fa8edb Cleaned up and split up Swift header parsing methods a bit
* Added tests for the metadata headers portion of the code

Change-Id: I8ac65e31212b4cca4592f963e0ca5ad30e1349f7
2015-08-05 01:12:50 -07:00
Aaron Schulz
6cef0373c7 Added sanitizeHdrs() tests for Swift
Change-Id: I2e3c3225c729e5220ca16f6ef4518da49e7f721c
2015-07-29 18:57:23 -07:00
jenkins-bot
3eb00083e9 Merge "Clear the stat cache in addMissingMetadata() to avoid more POSTs" 2015-07-28 13:45:00 +00:00
Aaron Schulz
6e7c034901 Improved addMissingMetadata() on POST failure
* If the POST failed but the sha1 was computed, then use
  and cache that value rather than "false".

Change-Id: I42b53c823013ecd9b281406e3d533a21e0de7cfb
2015-07-22 13:19:12 -07:00
Aaron Schulz
54eb877a52 Clear the stat cache in addMissingMetadata() to avoid more POSTs
Change-Id: Icc075e424bdbed6868692ec734dff1e7d2003dd6
2015-07-22 12:04:47 -07:00
Aaron Schulz
83a9ac3ee8 Avoid using trigger_error() in addMissingMetadata()
* Just use the normal swift log instead

Bug: T105810
Change-Id: Ifa463e6298a1b57ab6a8d92f9defc744619d9fb6
2015-07-17 20:29:10 +00:00