Commit graph

22 commits

Author SHA1 Message Date
Aaron Schulz
38d6b20148 r109469: protect FileOpScopedPHPTimeout from abuse by infinite loops. Also commented out the invocation of it for now. 2012-02-07 00:35:55 +00:00
Aaron Schulz
92408acbfb Renamed FileBackend to FileBackendStore and ugly FileBackendBase to FileBackend. Callers doing things like FileBackend::extensionFromPath() are now more proper. 2012-01-29 22:22:28 +00:00
Aaron Schulz
d70c7e6b28 * Split up process cache in FileBackend into separate arrays for expensive and inexpensive entries. Put the local reference FSFile object cache in the former since it takes up disk space on /tmp.
* Removed FileBackendBase::resolveWikiId(); doesn't really work well with FileBackendMultiWrite and the functionality is best handled in resolveContainerName(). Follows-up r108303.
* Gave FileOp::doAttempt() a default implementation (a no-op) to be more libertarian.
* Some documentation tweaks.
2012-01-29 19:23:26 +00:00
Aaron Schulz
d1b159ef63 In FileBackend:
* Use 'b' param in some fopen() calls as needed for Windows and newline handling.
* Removed some useless padding code in FileBackend::getContainerShard(). Initialized $m to make IDE happy.
* Updated some code comments.
In SwiftFileBackend:
* Manually set the ETag when using php-cloudfiles for creating files to avoid https://github.com/rackspace/php-cloudfiles/issues/59.
* Manually set the content type based on how StreamFile::getType(). This makes it safe to read files directly out of the proxy to end-users. The streamFile() backend functions already uses a similar content-type check.
2012-01-25 01:57:28 +00:00
Aaron Schulz
859982819d r109904: worked around PHP < 5.3 suckage to keep this compatible 2012-01-24 18:49:53 +00:00
Aaron Schulz
73d4d6edbf Made FileOp classes enforce required params. Also reverts r109823. 2012-01-24 05:54:47 +00:00
Antoine Musso
ed90f5548f FileOp: allow creation of an empty file
When creating a file but ommitting the 'content' key, it would throw
a notice. Using getParams let us forget about the 'content' key and
hence create files with no content.
2012-01-23 14:48:49 +00:00
Aaron Schulz
e2e0c0d9b7 * Follow-up r109009: Check that paths are usable in FileOp::doPrecheck(). Also lock parent directories to avoid prepare()/clean() race conditions for FS backends.
* Fixed bogus $params var in logException() call in SwiftFileBackend.
* Added 'latest' param to FileBackendMultliWrite::consistencyCheck().
* Dummy-proof FileBackend::getFileStat() w.r.t the 'latest' param and removed related FileOp::allowStaleReads() comment.
* Tweaked backend-fail-batchsize message from r109469.
2012-01-19 23:18:03 +00:00
Aaron Schulz
d0fc7d5d89 Renamed 'overwriteDest' FileBackend operation parameter to just 'overwrite', which is shorter and more consistent with 'overwriteSame' 2012-01-19 02:24:49 +00:00
Aaron Schulz
95f31706ee In FileBackend/FileOp:
* Added a sane default max file size to FileBackend. Operation batches need to check this before trying anything.
* Temporarily adjust the PHP execution time limit in attemptBatch() to reduce the chance of dying in the middle of it. Also added a maximum batch size limit.
* Added some code comments.
2012-01-19 02:07:48 +00:00
Aaron Schulz
f1a06be61c * Fixed type check in StreamFile::prepareForStream() for 404s
* Code comment tweaks and fixes
2012-01-17 05:34:01 +00:00
Aaron Schulz
c4a01a7afd In FileOp/FileBackend:
* Added SHA-1 to FileOp::attemptBatch $predicates (since concatenate was removed from FileOp). This lets us get rid of temp file backups, as the remaining failure case is just the backend medium going down (the ops would break in that case anyway). Doing so reduces RTTs and backup file I/O overhead. This also simplifies expiry object support by avoiding having to stash the expiry values of temp backup objects somewhere.
* Improved precheck() and attempt() status logic in FileOp::attemptBatch() a bit. Use separate $subStatus var to check if each op is OK.
* A few minor code cleanups and comment tweaks.
* Fixed MoveFileOp bug found in unit tests when the source/dest are the same and an overwriteDest/overwriteSame param is given (the file would just get deleted). Improved CopyFileOp in this case too.
Other:
* Added more unit tests.
2012-01-15 22:45:14 +00:00
Aaron Schulz
36c143110a In FileBackend:
* Made secure() call doPrepare() for caller dummy proofing (especially those that don't check the status).
In FSFileBackend:
* Removed redundant wfMkdirParents() calls in FSFileBackend, we already have prepare() for this purpose. This also keeps it's behavior more consistent with the other backends.
* Made use of 'backend-fail-store' message in doStoreInternal().
Other:
* Updated unit tests and renamed $src => $source in some functions for consistency.
* Added some documentation comments and @since tags.
2012-01-13 23:30:46 +00:00
Aaron Schulz
5a6559d8b1 Reverted r108743 per CR comment. This should at least be discussed first. 2012-01-12 19:41:18 +00:00
Sam Reed
a5c200a25b Kill "* @return void" 2012-01-12 19:12:24 +00:00
Aaron Schulz
230c2ccab7 * Fixed FileOp::attemptBatch() bug that had status->ok set wrong if 'force' option was used.
* Added missing prepare() call to File::maybeDoTransform().
* Improved documentation.
2012-01-09 21:04:48 +00:00
Aaron Schulz
94c3039106 * Follow-up r107170: Moved FileBackend::concatenate() outside of doOperations() as it's own separate operation. It does not mutate storage files like the others and having it in doOperations() broke FileBackendMultiWrite. This change also makes overriding doOperationsInternal() easier (suching as using a custom batch operation storage API).
* Added sanity check to FileBackendMultiWrite constructor.
* Moved FileBackend::create() function up a bit.
2012-01-08 09:25:15 +00:00
Aaron Schulz
360e3376bf In FileBackend:
* Added getFileSize()/getFileStat() functions. Refactored some functions to use the stat function for better reuse and caching/consistency.
* Refactored streamFile() to allow for subclasses to avoid local file copying with less duplication. Also make last-modified check actually work since we always get the timestamp of the original file.
* Renamed 'ignoreErrors' parameter to 'force'.
In FileBackendMultiWrite:
* Simplified how read ops are done (use 'master' backend for consistency).
* Added consistency check to doOperationsInternal() to check if the files are synced.
* Various fixes after testing.
In StreamFile:
* Split out prepareForStream() function from stream() in StreamFile for code reuse.
In FileBackendTest:
* Properly cover FileBackendMultiWrite in tests.
* Various test improvements.
2012-01-08 08:40:00 +00:00
Aaron Schulz
ae4a810b1e * Made use of FileBackend function 'latest' param in FileOp.
* Added FileBackend process cache for fileExists(), getFileTimestamp(), and getLocalReference().
* Refactored getFileSha1Base36() into parent class and subclass functions.
* Removed some FileBackendMultiWrite comment duplication.
2012-01-05 06:18:36 +00:00
Aaron Schulz
875f8a28e9 In FileBackendBase/FileBackend:
* Changed concatenate to store to a specified temp FS file rather than a final storage destination. This makes it better fit the use case (chunked upload), so we can avoid extra copying around. Subclasses no longer have to implement this function now as well.
* Added extensionFromPath() helper function.
* Moved createInternal() up a bit and fixed @see comments pointing to the wrong functions.
In FSFileBackend:
* Use parent implementation of doConcatenateInternal().
In FileRepo/File:
* Added FileRepo::ALLOW_STALE and made thumbnail transforms use it.
2011-12-23 18:59:39 +00:00
Aaron Schulz
2c4ef137b4 * Renamed FileBackend functions internal to FileBackend/FileOp, making their usage clearer.
* Added convenience functions to FileBackendBase for basic file ops. Previously, doOperation() was the only convenience function...give it some friends :)
* More documentation comments.
2011-12-21 09:16:28 +00:00
Aaron Schulz
5275f9b097 Merged FileBackend branch. Manually avoiding merging the many prop-only changes SVN likes to sprinkle in (easy to spot from the change list). Did not add SwiftFileBackend.php as it still is in development. 2011-12-20 03:52:06 +00:00