Commit graph

91 commits

Author SHA1 Message Date
Aaron Schulz
8d1eecab49 Allow "." in filebackend container prefixes
* This character in $wgDBname seems to be common enough.
  The wiki ID is the default container prefix, and if it
  is not accepted, then confusing errors occur that block
  all file uploads. The case of dots is handled now.
* Improved variable naming in resolveStoragePath().

Bug: T46066
Change-Id: I68458a876855894d01e8cf880bd6cfcfdae17bd0
2015-06-26 06:49:20 +00:00
Kunal Mehta
f6e5079a69 Use mediawiki/at-ease library for suppressing warnings
wfSuppressWarnings() and wfRestoreWarnings() were split out into a
separate library. All usages in core were replaced with the new
functions, and the wf* global functions are marked as deprecated.

Additionally, some uses of @ were replaced due to composer's autoloader
being loaded even earlier.

Ie1234f8c12693408de9b94bf6f84480a90bd4f8e adds the library to
mediawiki/vendor.

Bug: T100923
Change-Id: I5c35079a0a656180852be0ae6b1262d40f6534c4
2015-06-11 18:49:29 +00:00
Aaron Schulz
48cffdff82 Converted FileBackendStore to using the WAN cache
Bug: T93141
Change-Id: I76efb0dc45a697c5fdfc50932e1198a6d663a445
2015-04-30 01:30:38 +00:00
jenkins-bot
b7ec2c9b33 Merge "Fix and make PHPDoc tags in FileBackend more specific" 2015-03-18 16:45:20 +00:00
Thiemo Mättig
d71f05290e Fix and make PHPDoc tags in FileBackend more specific
A pure documentation patch. Should almost be a no-brainer. ;-)

Change-Id: I9bf4d6056f7cde193fe5882a04a7973af9b5a7f4
2015-03-18 17:22:52 +01:00
Aaron Schulz
8d2f7b9178 Never treat persistent stat cache as "latest" in FileBackend
bug: T89184
Change-Id: I035c0f85a641efa0e4e3209b2bcf9847c08ab0d2
2015-02-10 23:30:55 +00:00
Aaron Schulz
4ff8136807 Removed remaining profile calls
Change-Id: I31c81c78715048004fc8fca0f27d09c1fa71c118
2015-01-08 02:49:33 -08:00
Aaron Schulz
3226337f17 Converted some section profiling to scopedProfileIn
Change-Id: If69de34a046f4f33fc57b93beaf264f522ce7386
2015-01-07 14:19:06 -08:00
Ricordisamoa
12dec5d85d Fix some stuttering in comments and documentation
Change-Id: I9c0088b9aab37335203cad45a1d6fa8ac3f43321
2014-12-17 19:44:10 +00:00
umherirrender
b883e8c7c0 Cleanup some docs (includes/[e-l])
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling

Change-Id: I41a84e8e1dec39170aa655250325ffc485eaeaef
2014-07-24 19:43:03 +02:00
umherirrender
abfa0d7854 Standardize @todo
Always use lowercase @todo
Changed some "@FIXME" to "@todo FIXME: "

Change-Id: I016d1f9511e6f3bbd22427f0fffeccd77ab5e0db
2014-07-23 22:27:20 +02:00
umherirrender
53c420e278 Fixed spacing
- use tab as indent instead of spaces
- Added space after closures "function"
- Added spaces around string_concat
- Added newline inside empty blocks
- Removed four spaces after comma

Change-Id: I4425b0c6a69b36f40acfea6511b8950cf09ce2b2
2014-07-20 21:41:41 +02:00
Aaron Schulz
0fa9a391e3 filebackend: Avoid using popen() when "parallelize" is disabled
Bug: 66467
Change-Id: Iaf8eb2ecfad166e26e265a98933399dc4bc255a3
2014-06-18 20:12:03 +00:00
Aaron Schulz
1d59d6152e Improved FileBackend failure logging a bit
* This partly follows up on 2cfce9f

Change-Id: I9d15f44d8000f52fd7b307d358c895452ee96a8d
2014-05-22 17:37:33 -07:00
jenkins-bot
a21d59cc6e Merge "Make room for preloadFileStat() call in FileBackend::doOperationsInternal" 2014-04-21 17:13:35 +00:00
jenkins-bot
e40b19ed4e Merge "Bail out on FileBackend operations if the initial stat calls failed" 2014-04-21 17:13:33 +00:00
umherirrender
957adbef22 Fixed some @params documentation (includes/[file...|upload])
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.

Change-Id: I8804ebe0922d3a414863b162a2110e0b9e49b80f
2014-04-19 17:19:17 +02:00
umherirrender
b9cd789fce docs: closure -> Closure; callback -> callable
Changed closure to capital word Closure in doc and type hint,
also changed callback in docs to callable

Change-Id: I52c8e8f13d38a837052101c38b9986be780ca057
2014-04-19 08:43:31 +02:00
Aaron Schulz
d52a559fba Make room for preloadFileStat() call in FileBackend::doOperationsInternal
Change-Id: I60f71155abcf7f69423639b10ec301ac192df728
2014-04-18 17:02:19 -07:00
Aaron Schulz
2cfce9fd8d Bail out on FileBackend operations if the initial stat calls failed
* The preloadFileStat() call acts as a good canary for whether all
  affected servers are up. If anything failed in it, then it's not
  worth bothering to send the actual write requests.
* Also made FileBackend::preloadFileStat abstract while at it since
  the two subclasses implemented it and so should any others.
* Fixed a silly comment typo.

Change-Id: I5bd427f654aa4a9d6bfe4ed7566276e8ac520b30
2014-04-18 12:41:23 -07:00
Aaron Schulz
02e0d115a1 Tweaked "latest" handling of filebackend stat entries
* Let "latest" stat entries override non "latest" so
  that future getFileStat() calls with the "latest" flag
  can actually have a cache hit.

Change-Id: I1e9391039537d608b89773b4d51575e3b364a751
2014-03-24 06:55:01 +00:00
Aaron Schulz
6112acac29 Make use of strong consistency with Ceph RGW
Change-Id: Ic3b764363e91c3279d3dc017ddce630dae26bec5
2014-03-19 17:27:48 +00:00
Aaron Schulz
f5f2233daa Avoid key fragmentation in FileBackend
* Make sure that global backends do not prefix the cache keys
  with local wiki IDs for container and file stat entries.
  This causes fragmentation and breaks invalidation.

Change-Id: Ic02d6e41f828dc82cc07c7d24c8af29c46392a9c
2014-02-24 13:22:15 -08:00
Aaron Schulz
6b1a60a1ca Added concurrent HEAD request support for Swift
* This adds a new preloadFileStat() method to FileBackend.
* FileBackend::doOperations() makes use of this instead of primeFileCache().
  The later was mostly useless due to the 'latest' flag, which stat entries
  rarely had (and since they use add(), 'latest' ones do not override the
  non-'latest' ones).
* Simplified primeContainerCache/primeFileCache to no longer accept
  FileOp objects and made sure nothing passes those in anymore.

Change-Id: Iee3878cadd69c3a594c701c0aff81aab5f473de5
2014-02-20 08:50:08 +00:00
Siebrand Mazeland
f077c4b1d6 Update formatting
Change-Id: I18aff576262479c9bb1c56eb8e1d1aaae200e4b1
2014-02-06 09:27:05 +01:00
Aaron Schulz
c745ff9a58 Fixed bogus var usage in SwiftFileBackend
Change-Id: Ia0dec914208cb3100b7f5bbd2291985eac05dc02
2014-01-09 12:39:13 -08:00
Aaron Schulz
ac14c1c8a6 filebackend: Added supported for retrieving file metadata/headers
* This can be useful for carrying over metadata when copying files around
* Also fixed a bug in sanitizeHdrs() for Swift (broken content-disposition)

Change-Id: I4534e9acac2b306086797b3677f85c05b98e39fc
2014-01-08 13:22:11 -08:00
Aaron Schulz
a0878fe2b4 filebackend: exception handling cleanups
* Split out FileBackendException class and reduced direct use of MWException

Change-Id: I325c1798b6d90972c12a5dccc37989af34d857f3
2013-12-07 23:22:07 -07:00
Siebrand Mazeland
8c0cbe6f83 Update ternary notation
Change-Id: I553dd1d42477326848e6e40b824058b6790e2eb5
2013-12-04 09:52:51 +01:00
Siebrand Mazeland
c232652303 Remove unused local variable
Just count if there are array members

Change-Id: I4f9cc8ba554404796822317fdb285e2f560d01dd
2013-11-25 06:48:23 +01:00
Siebrand Mazeland
34a4bcbd48 Update documentation of file backend classes
Change-Id: I2113aecc53f1b361d9c276113bc0a9f7076cd21c
2013-11-23 19:24:43 +01:00
Siebrand Mazeland
a5c35215e2 Update formatting of file backend classes
Change-Id: Ie69c324e2f1ea0b8a654adee06ebda494cebd10c
2013-11-22 22:17:15 +01:00
Aaron Schulz
beb7359243 Removed excess profile calls
Change-Id: Idd57e8d7fff56c62a66c7fcaf4627b9096f0c363
2013-11-16 23:44:41 -08:00
Aaron Schulz
2afdc066f5 filebackend: improved "adviseStat" performance
* Use the normal page size instead of limiting it way down to the cache
  size. Track the stat information in the pages and load into into the
  stat cache as entries are accessed. This should also be less prone to
  evictions causing HEAD requests (or memcached hits).
* Also bumped CACHE_CHEAP_SIZE up to 500.
* Fix a few doc bits

Change-Id: I8d44a072e7bcc56c83d8d9c8c9ac9864530bccf8
2013-11-08 04:35:57 +00:00
Aaron Schulz
584de11eb1 Fixed broken quickDescribe() method
Change-Id: Ic68f483c4008ff6ae8c4accef2f9d1f180a5d377
2013-10-24 16:34:07 -07:00
Aaron Schulz
ef2f2610ab filebackend: try to combine SH and EX lock acquisition
* getPathsToLockForOpsInternal() now returns an array in the format LockManager can use
* Also fixed getScopedLocksForOps() for multiwrite backend (it should accept proxy backend paths)
* Updated getScopedFileLocks() docs

Change-Id: Id0dc50c159c5fbc8fca10a9e46c5be23afcb3e9a
2013-10-18 22:57:38 +00:00
Aaron Schulz
91cab51cb5 Factored out a small filebackend getContentType() function
Change-Id: I221483ab4a0ed36ea34c19db1740dfaecf223677
2013-09-25 09:57:07 -07:00
Aaron Schulz
ad372af149 Switched FileBackendStore to use ProfileSection when possible.
* Also removed outer profiling layer.

Change-Id: Ice3b921fe0c9981ac16f005d66a78bf4f4a5c601
2013-06-19 00:43:54 +00:00
Aaron Schulz
7242d938f3 Cleaned up variable/type order in docs.
* A few other minor doc cleanups and obvious type hints.

Change-Id: I454fad0c1942068b4f3632ebd1b605bfbb9dfe7c
2013-06-13 11:18:52 -07:00
Aaron Schulz
a379dbf9db Removed redundant documentation for some methods.
Change-Id: Iad422716f6796cc4b69405182f4a74daa0c93663
2013-06-12 19:58:15 +00:00
Aaron Schulz
40078d1c69 [FileBackend] Avoid concurrency for the 1 op case in doQuickOperationsInternal().
Change-Id: I6526d3b77fc078d80a0fb537d504a7fabcbb4c41
2013-05-15 10:35:51 -07:00
Aaron Schulz
b14212992d [FileBackend] Moved short-circuit check from 6b1194b9 up to FileBackend.
Change-Id: I57566a1779f353c18d96c59c47112c753aea6b90
2013-05-10 13:55:39 -07:00
Aaron Schulz
6b1194b9da [FileBackend] Optimize the case were no operations are provided.
Change-Id: Iaf858ce2e8b21e031d27b984029f0424518201af
2013-05-10 13:45:34 -07:00
Aaron Schulz
d1940caa1f [FileBackend] Added "adviseStat" option for the "listing followed by stat" case.
* Used this parameter to speed up copyFileBackend.php.
* Also added mtime checks to copyFileBackend.php and a few cleanups.
* Also fixed some incorrect getDirListPageInternal/getFileListPageInternal docs.

Change-Id: I424ef238f7adf4cf1f33b74e3a4e187dcb328a99
2013-05-09 16:19:40 +00:00
Aaron Schulz
dfd2277cd1 [FileBackend] Process cache negatives for SHA1 on file stat.
Change-Id: Idc9f82309d52818acf57adc9c4eb04370a7b422a
2013-05-07 14:15:11 -07:00
umherirrender
1bfc8feb25 Fixed spacing in actions/cache/filebackend/filerepo/job folder
Added spaces before if, foreach
Added some braces for one line statements

Change-Id: Idb93d34e314e5f314223b79208968d6bcd30c40e
2013-04-20 19:18:13 +02:00
Aaron Schulz
79910f36d4 [FileBackend] Made stat cache factor in last-modified time.
* This should lesson the chance of consistency problems
  if there is any memcached flapping.
* Also cleaned up some documentation.

Change-Id: I89401892c7e848fcff0b093eaa861dfe02387a5c
2013-04-19 03:32:57 +00:00
Aaron Schulz
14fc5f5a93 [FileBackend] Disallow files ending in / and clarified some docs.
Change-Id: I162ca34fe9f92d862345004bd184013531da9e8d
2013-04-19 03:30:14 +00:00
Aaron Schulz
dcc69671e0 [FileBackend] Simplified the shard list iterator to use AppendIterator and FilterIterator.
Change-Id: I22f79447b1edec4fa6d7fc06d67f3f831a484f16
2013-04-17 17:06:52 +00:00
umherirrender
15abcf71ca Added/Removed spaces around string concatenation
And added/removed spaces around some other tokens,
like +, -, *, /, <, >, =, !

Fixed windows newline style

Change-Id: I0b9c8c408f3f6bfc0d685a074d7ec468fb848fc8
2013-04-13 13:36:24 +02:00