Commit graph

80 commits

Author SHA1 Message Date
Aaron Schulz
8d2f7b9178 Never treat persistent stat cache as "latest" in FileBackend
bug: T89184
Change-Id: I035c0f85a641efa0e4e3209b2bcf9847c08ab0d2
2015-02-10 23:30:55 +00:00
Aaron Schulz
a611ebf1ff Changed FileBackend exceptions to subclass Exception
Change-Id: Ic7d4d6cf0dde3e93ef78758b1a6b03f78c9bcdba
2015-01-17 14:48:00 -08:00
Aaron Schulz
d1bc7d2cd9 Removed some b/c code from file backend
Change-Id: Ie8dbff588b067a202b24dfeaf792301c1077f08b
2014-09-12 00:54:24 -07:00
umherirrender
3b2b6a2773 Add missing @param to function docs
Change-Id: I47fa96a976f55a1a93cb75397285edb8c7f4cd8a
2014-08-14 20:22:52 +02:00
Aaron Schulz
399198c28e Replace FOR UPDATE with LockManager use in LocalFile::lock()
* This avoids excess contention where inserts of rows for
  similarly named files get blocked. This often would effect
  users of UploadWizard or any bot doing mass uploads.

Change-Id: Ie7a328f7d4f03aa249770804417347a50356ea42
2014-05-20 14:22:52 -07:00
Aaron Schulz
0d54e6e0bf Added unicode encoding support flags to FileBackend
* Fixed bug where even using Swift/Azure on Windows
  would disallow non-ASCII file names.

bug: 1780
Change-Id: I19ed72da0b099d35cae74fb08eeb22c113da1065
2014-05-13 14:11:36 -07:00
Aaron Schulz
a650998aa7 Avoid bogus IE extension check errors in img_auth.php
Change-Id: I57083d3fe0517b94d3b786970b412e6ec51cf5f0
2014-04-21 19:30:48 -07:00
jenkins-bot
e40b19ed4e Merge "Bail out on FileBackend operations if the initial stat calls failed" 2014-04-21 17:13:33 +00:00
umherirrender
957adbef22 Fixed some @params documentation (includes/[file...|upload])
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.

Change-Id: I8804ebe0922d3a414863b162a2110e0b9e49b80f
2014-04-19 17:19:17 +02:00
Aaron Schulz
2cfce9fd8d Bail out on FileBackend operations if the initial stat calls failed
* The preloadFileStat() call acts as a good canary for whether all
  affected servers are up. If anything failed in it, then it's not
  worth bothering to send the actual write requests.
* Also made FileBackend::preloadFileStat abstract while at it since
  the two subclasses implemented it and so should any others.
* Fixed a silly comment typo.

Change-Id: I5bd427f654aa4a9d6bfe4ed7566276e8ac520b30
2014-04-18 12:41:23 -07:00
Aaron Schulz
8aaab01e1d Made preloadFileStat/preloadCache actually work for FileBackendMultiWrite
* Previously the path substitution did not take place

Change-Id: I58ad3e2aa44a2b08f4c6bad1a783fb92ea79dbf4
2014-04-12 15:42:39 -07:00
Aaron Schulz
6b1a60a1ca Added concurrent HEAD request support for Swift
* This adds a new preloadFileStat() method to FileBackend.
* FileBackend::doOperations() makes use of this instead of primeFileCache().
  The later was mostly useless due to the 'latest' flag, which stat entries
  rarely had (and since they use add(), 'latest' ones do not override the
  non-'latest' ones).
* Simplified primeContainerCache/primeFileCache to no longer accept
  FileOp objects and made sure nothing passes those in anymore.

Change-Id: Iee3878cadd69c3a594c701c0aff81aab5f473de5
2014-02-20 08:50:08 +00:00
Siebrand Mazeland
f077c4b1d6 Update formatting
Change-Id: I18aff576262479c9bb1c56eb8e1d1aaae200e4b1
2014-02-06 09:27:05 +01:00
Aaron Schulz
ac14c1c8a6 filebackend: Added supported for retrieving file metadata/headers
* This can be useful for carrying over metadata when copying files around
* Also fixed a bug in sanitizeHdrs() for Swift (broken content-disposition)

Change-Id: I4534e9acac2b306086797b3677f85c05b98e39fc
2014-01-08 13:22:11 -08:00
jenkins-bot
76dc198651 Merge "filebackend: cleaned up the FileBackend constructor" 2013-12-10 22:09:21 +00:00
Aaron Schulz
a0878fe2b4 filebackend: exception handling cleanups
* Split out FileBackendException class and reduced direct use of MWException

Change-Id: I325c1798b6d90972c12a5dccc37989af34d857f3
2013-12-07 23:22:07 -07:00
Aaron Schulz
2aa84651ca filebackend: cleaned up the FileBackend constructor
* Added some b/c code with deprecation warnings

Change-Id: Ifceffbc0a37a223bcd7cd3dc60181fc85765bc46
2013-12-03 13:57:01 -08:00
jenkins-bot
b0b37fd95c Merge "Revert "filebackend: cleaned up the FileBackend constructor"" 2013-12-03 20:46:05 +00:00
Aaron Schulz
2f29c7551b Revert "filebackend: cleaned up the FileBackend constructor"
Actually this messes with the implicit backend made for things like Math (when unconfigured), which uses the "new" operator.

This reverts commit 1f129a22cb.

Change-Id: I4c72c4f7c8b82e38df5496cf2b90fc9e19c40334
2013-12-03 20:40:24 +00:00
jenkins-bot
dfe52313df Merge "filebackend: cleaned up the FileBackend constructor" 2013-12-03 20:22:10 +00:00
Aaron Schulz
a277bb9bed More file backend docs
Change-Id: Ic81e298baa7cb21d0e494e6e2ed749e7f401f2b5
2013-11-24 15:49:17 -08:00
Aaron Schulz
1f129a22cb filebackend: cleaned up the FileBackend constructor
* Moved some of the graph construction work to FileBackendGroup.
  This helps the code in not depending on the rest of MW so much.
* Updated tests and FileBackendMultiwrite, which are the only things
  directly constructing FileBackend objects.

Change-Id: I188a053c70ce088ce34613d5db40e6708e3ea9b7
2013-11-23 20:21:53 -08:00
Siebrand Mazeland
34a4bcbd48 Update documentation of file backend classes
Change-Id: I2113aecc53f1b361d9c276113bc0a9f7076cd21c
2013-11-23 19:24:43 +01:00
Siebrand Mazeland
a5c35215e2 Update formatting of file backend classes
Change-Id: Ie69c324e2f1ea0b8a654adee06ebda494cebd10c
2013-11-22 22:17:15 +01:00
Reedy
08e84030d3 Replace php_sapi_name() with PHP_SAPI
Change-Id: I222789cacaa2653c4dac62708ce228674c90af4c
2013-10-25 13:57:26 +01:00
Aaron Schulz
ef2f2610ab filebackend: try to combine SH and EX lock acquisition
* getPathsToLockForOpsInternal() now returns an array in the format LockManager can use
* Also fixed getScopedLocksForOps() for multiwrite backend (it should accept proxy backend paths)
* Updated getScopedFileLocks() docs

Change-Id: Id0dc50c159c5fbc8fca10a9e46c5be23afcb3e9a
2013-10-18 22:57:38 +00:00
Aaron Schulz
5749bd7352 filebackend: normalize paths used in locking
Change-Id: I0d823e4d20c89a22ad89a6af8739075f7a10c235
2013-07-06 15:06:38 -07:00
Aaron Schulz
754b8babca filebackend: throw exceptions during file iteration.
* This lets calling code be far more robust rather than just
  silently ignoring entries due to some temporary problem.

Change-Id: I3ce2ae34f6cff5e40a80b8da5688503a387ce2a6
2013-06-15 23:01:47 -07:00
Aaron Schulz
7242d938f3 Cleaned up variable/type order in docs.
* A few other minor doc cleanups and obvious type hints.

Change-Id: I454fad0c1942068b4f3632ebd1b605bfbb9dfe7c
2013-06-13 11:18:52 -07:00
Aaron Schulz
b14212992d [FileBackend] Moved short-circuit check from 6b1194b9 up to FileBackend.
Change-Id: I57566a1779f353c18d96c59c47112c753aea6b90
2013-05-10 13:55:39 -07:00
Aaron Schulz
d1940caa1f [FileBackend] Added "adviseStat" option for the "listing followed by stat" case.
* Used this parameter to speed up copyFileBackend.php.
* Also added mtime checks to copyFileBackend.php and a few cleanups.
* Also fixed some incorrect getDirListPageInternal/getFileListPageInternal docs.

Change-Id: I424ef238f7adf4cf1f33b74e3a4e187dcb328a99
2013-05-09 16:19:40 +00:00
umherirrender
1bfc8feb25 Fixed spacing in actions/cache/filebackend/filerepo/job folder
Added spaces before if, foreach
Added some braces for one line statements

Change-Id: Idb93d34e314e5f314223b79208968d6bcd30c40e
2013-04-20 19:18:13 +02:00
Aaron Schulz
2d7fe59935 [FileBackend] Clarified the noAccess/noListing docs a bit.
Change-Id: I3f008e3d1931423fc3c73af587d657040fdffe83
2013-04-01 16:39:36 -07:00
Aaron Schulz
3131dbc93b Clarified FileBackend::doOperations() docs a bit more.
Change-Id: Ie0b131c1a09bc75be39bbb7e9c87b8d3221a0fc5
2013-03-25 19:00:37 -07:00
ASchulz
c328cf6401 [FileBackend] Cleanup behavior for coping/moving a file over itself.
* Previously, for doQuickOperations(), the default move function
  would just delete the file if it was moved over itself. In fact,
  the php-cloudfiles bindings have this same bug in its move function.
* For both copy and move, when the source and destination are the
  same path, make sure that the headers get updated as specified.
  This only applies for the 'overwrite' case (not 'overwriteSame').
* Fixed bad status for doQuickOperations() when copying a file over
  itself using FSFileBackend.
* Clarified the documentation for 'overwriteSame' option.
* Renamed destSameAsSource to overwriteSameCase in FileOp for clarity.

Change-Id: I3f609d9413795c654de27958b1e807b1fc785f31
2013-03-25 17:06:30 -07:00
Aaron Schulz
73b5fc4e0a A few minor doc group tweaks.
Change-Id: I18a8d733b3dbaa925a676c881225a63e90ff2f05
2013-03-18 15:14:55 -07:00
Aaron Schulz
139292dadd [FileBackend] Replaced redundant "disposition" param with "headers".
* Deprecated the "disposition" parameter.
* This also adds "headers" to to copy/move operations.

Change-Id: I63faa1dbbadb42c401b6ed4ac58776dc336ba3ab
2013-03-12 14:16:36 -07:00
Aaron Schulz
797e52dd05 Reformatted some array parameter docs.
Change-Id: Ie9eefd3693e82f483a9ef0da3f972452ae7bf49e
2013-03-11 11:00:35 -07:00
Tyler Anthony Romeo
4dcc7961df Fixed @param tags to conform with Doxygen format.
Doxygen expects parameter types to come before the
parameter name in @param tags. Used a quick regex
to switch everything around where possible. This
only fixes cases where a primitve variable (or a
primitive followed by other types) is the variable
type. Other cases will need to be fixed manually.

Change-Id: Ic59fd20856eb0489d70f3469a56ebce0efb3db13
2013-03-11 13:15:01 -04:00
Reedy
c3e4057e06 Kill off numerous unused variables
Change-Id: I7039f1328f37ee669b694f73ee282602186bffd1
2013-03-08 02:36:24 +00:00
Aaron Schulz
66e73f8583 [FileBackend] Added a "ttl" option to getFileHttpUrl().
Change-Id: Ifc7b9c012276ed1f1e436ffafbd5615689680054
2013-03-06 20:55:43 -08:00
Aaron Schulz
13610ff519 Fixed documentation typo.
Change-Id: I02c3944efc63508c7aa798b3079167b85c760454
2013-02-20 13:30:58 -08:00
Aaron Schulz
aa7fbbb0cd [FileBackend] Documentation tweaks.
Change-Id: I63b03207a18c7523675a4929031dc560a2d8b5f8
2013-02-04 23:20:18 -08:00
Aaron Schulz
dce693efdc [FileBackend] Set ignore_user_abort() in file operations.
* This reduces the change of partial operations. WMF sites already
  set ignore_user_abort() in configuration, but this makes sure that
  it always happens during file changes.

Change-Id: I702c27fc3c19aca0cdd39b793a3250ead40bfe71
2013-02-01 12:30:47 -08:00
Aaron Schulz
a4f27a198f [LockManager] Generallized wiki field to a domain field.
* In some cases (like certain qualified file backends), a domain
  may be desired that is not simple a wiki ID.
* Also added a LockManangerGroup::config() function to export
  the configuration of a lock manager.

Change-Id: I5a75c55be3cc9017ed8a717d6d30e466fd00340f
2013-02-01 10:40:45 -08:00
Aaron Schulz
2cec654eea [FileBackend] Added more documentation.
Change-Id: I6a0f0dc253fbe82a58345d4179248071062286ff
2013-01-31 22:06:57 -08:00
Aaron Schulz
c1067b5c3a [FileBackend] Improved some header related docs.
Change-Id: I823af49b39afedda51f8192cd6a1a7467a8b78a7
2012-12-20 11:50:35 -08:00
Aaron Schulz
69b24f9a63 [FileBackend] Removed obsolete "allowStale" parameter.
* This was obsoleted by doQuickOperations().

Change-Id: I9e1d9c1e1c910e4ce6de4317299424305b31469c
2012-11-27 13:55:58 -08:00
Aaron Schulz
ac56107ba3 Merge "Revert "[FileBackend] Added optional callback parameter to concatenate()."" 2012-11-25 01:05:56 +00:00
Aaron Schulz
65612b4290 Revert "[FileBackend] Added optional callback parameter to concatenate()."
This gives updates for the faster part (local FS file changes) and not
the slow GETs that really need this. This should be re-done to use a
byte based read progress callback (that handles the parallel GETs).

This reverts commit 6c9f13b547

Change-Id: Id8d739e1d5048e7f38c68eda4f9a008682707cba
2012-11-24 17:01:12 -08:00