Commit graph

66 commits

Author SHA1 Message Date
Umherirrender
ee73e6ac1b Remove unused local variable assignment
Dead code found by phan

Change-Id: I9fc404d546a4fb1c61394cb6359eb774fd94383a
2023-02-04 22:16:31 +01:00
Umherirrender
20c3cc6ef1 Remove unused variable from unpacking arrays
Change-Id: Iac27cb4aa936cb494b3e491ce22c88c7fad375a1
2022-11-22 23:21:08 +00:00
jenkins-bot
a85966542d Merge "Use short array destructuring instead of list()" 2022-10-21 11:11:06 +00:00
Tim Starling
0077c5da15 Use short array destructuring instead of list()
Introduced in PHP 7.1. Because it's shorter and looks nice.

I used regex replacement.

Change-Id: I0555e199d126cd44501f859cb4589f8bd49694da
2022-10-21 15:33:37 +11:00
Tim Starling
43a93d9782 Use the null coalescing assignment operator
Available since PHP 7.4.

Automated search, manual replacement.

Change-Id: Ibb163141526e799bff08cfeb4037b52144bb39fa
2022-10-21 13:26:49 +11:00
Matěj Suchánek
e47c441078 Fix many typos in comments
Found using IntelliJ's "Typo" code inspection.

Change-Id: I746220ebe6e1e39f6cb503390ec9053e6518cf16
2022-05-10 12:46:11 +00:00
Umherirrender
9efd9ca45e Add explicit casts between scalar types
* Some functions accept only string, cast ints and floats to string
* After preg_matches or explode() casts numbers to int to do maths
* Cast unix timestamps to int to do maths
* Cast return values from timestamp format function to int
* Cast bitwise operator to bool when needed as bool

* php internal functions like floor/round/ceil documented to return
  float, most cases the result is used as int, added casts

Found by phan strict checks

Change-Id: Icb2de32107f43817acc45fe296fb77acf65c1786
2022-03-01 18:19:33 +01:00
Umherirrender
b126dbe3f2 Fix various documentation related to null types
The functions returning null or the class property is set explict null

Found by phan strict checks

Change-Id: I4a271093fb6526564d8083a08249c64cb21f2453
2022-02-26 10:31:24 +01:00
Tim Starling
ca71e69fc6 Try not to discard Excimer timeout exceptions
Don't catch and discard exceptions from the RequestTimeout library,
except when the exception is properly handled and the code seems to be
trying to wrap things up.

In most cases the exception is rethrown. Ideally it should instead be
done by narrowing the catch, and this was feasible in a few cases. But
sometimes the exception being caught is an instance of the base class
(notably DateTime::__construct()). Often Exception is the root of the
hierarchy of exceptions being thrown and so is the obvious catch-all.

Notes on specific callers:

* In the case of ResourceLoader::respond(), exceptions were caught for API
  correctness, but processing continued. I added an outer try block for
  timeout handling so that termination would be more prompt.
* In LCStoreCDB the Exception being caught was Cdb\Exception not
  \Exception. I added an alias to avoid confusion.
* In ImageGallery I added a special exception class.
* In Message::__toString() the rationale for catching disappears
  in PHP 7.4.0+, so I added a PHP version check.
* In PoolCounterRedis, let the shutdown function do its thing, but
  rethrow the exception for logging.

Change-Id: I4c3770b9efc76a1ce42ed9f59329c36de04d657c
2022-02-02 16:27:44 +11:00
Siddharth VP
804dcac8a3 Fix typos in comments (S)
Change-Id: Id86769ca81ff65707edde520585aa9ea117123e1
2022-01-09 23:28:53 +05:30
jenkins-bot
3690fb0f5d Merge "Use DeletePage in FileDeleteForm and fix output of ApiDelete" 2021-11-23 09:55:55 +00:00
Reedy
2a2bb1e9bd Remove or replace usages of "sane"
Bug: T254646
Change-Id: I096b2cf738a1395a14f1d47bcbed0c2c686c2581
2021-11-22 13:35:17 +00:00
Reedy
7bf779524a Remove or replace usages of "sanity"
Bug: T254646
Change-Id: I2b120f0b9c9e1dc1a6c216bfefa3f2463efe1001
2021-11-19 23:19:42 +00:00
Daimona Eaytoy
b77ae63e48 Use DeletePage in FileDeleteForm and fix output of ApiDelete
- Use DeletePage in FileDeleteForm instead of
  WikiPage::doDeleteArticleReal
- Properly handle scheduled deletions in FileDeleteForm: previously, a
  null status value could indicate a missing page OR a scheduled
  deletion, but the code always assumed the first, and it would generate
  a duplicated log entry. The API response would also not contain the
  "delete-scheduled" message. This has been broken since the introduction
  of scheduled deletion.
- In ApiDelete, for file deletions, check whether the status is OK not
  good. The two might be equivalent, but this way it's more consistent.
- Add some documentation for the Status objects returned by file-related
  methods. This is still incomplete, as there are many methods using
  Status and none of them says what the status could be. In particular,
  this means that for now we keep checking whether the status is OK
  instead of good, even though it's unclear what could produce a
  non-fatal error.
- In LocalFileDeleteBatch, avoid using a class property for the returned
  status, as that's hard to follow. Instead, use a local variable and
  pass it around when needed.

Bug: T288758
Change-Id: I22d60c05bdd4a3ea531e63dbb9e49efc36935137
2021-11-18 01:41:51 +01:00
Alexandros Kosiaris
5a62ea7d77 Force using HTTP 1.1 for SwiftFileBackend
We want to force the use of HTTP/1.1 instead of leaving it up to the host
default which might be HTTP/2, introducing a performance regression for
large uploads.

Since we have to pass this to every run()/runMulti(), set it as a private
constant that can be passed directlry or added to existing option arrays.

Bug: T275752
Change-Id: Ifbae4d585daee28fa92b07f53c5854320a18a311
2021-10-29 10:54:26 -07:00
kalle
a646366e7c Allow x-delete-at and x-delete-after expiry headers
From:
https://docs.openstack.org/swift/latest/overview_expiring_objects.html

The swift-object-expirer offers scheduled deletion of objects.
The Swift client would use the X-Delete-At or X-Delete-After
headers during an object PUT or POST and the cluster would
automatically quit serving that object at the specified time
and would shortly thereafter remove the object from the system.

Bug: T280496
Change-Id: I886f3cea51f80d820841f8548941ca02a0a514c3
2021-04-26 17:41:30 +02:00
Tim Starling
8bde51490a Swift HTTP client request log including timings
* Send Swift's MultiHttpClient logs to the FileOperation channel
* Add more detailed debugging information to those logs, including
  request execution time.

Bug: T276017
Change-Id: I221be1f452e03d19b7cd81430a9349a9db14f81f
2021-03-10 16:29:21 +11:00
Umherirrender
8de3b7d324 Use static closures where safe to use
This is micro-optimization of closure code to avoid binding the closure
to $this where it is not needed.

Created by I25a17fb22b6b669e817317a0f45051ae9c608208

Change-Id: I0ffc6200f6c6693d78a3151cb8cea7dce7c21653
2021-02-11 00:13:52 +00:00
Thiemo Kreuz
81362c930f filebackend: Fix index error in SwiftFileBackend
This fixes a regression introduced by If3d2f18.

Bug: T259023
Change-Id: I654b1bcf0ff65f68ed7d92d51a6b39325da731f3
2020-07-28 15:30:53 +02:00
Thiemo Kreuz
9c6473dc1d Reduce nesting by turning big if-else into guard clauses
Change-Id: If3d2f18af0cc7d0f8fe9f764bd4d0e4bae70e440
2020-07-24 11:55:31 +02:00
Tim Starling
1996c78545 In SwiftFileBackend allow HTTP timeouts to be set in the constructor
The default is not explained in the doc comment since it will change
after T245170.

Bug: T226979
Change-Id: I07b83efc09bed4fd18944c13e80465d0ea08743c
2020-05-15 13:19:11 +10:00
Daimona Eaytoy
598c4d7fcb build: Upgrade phan to 0.9.0
Scalar casts are still allowed (for now), because there's a huge amount
of false positives. Ditto for invalid array offsets.

Thoughts about the rest: luckily, many false positives with array offsets
have gone. Moreover, since *Internal issues are suppressed in the base
config, we can remove inline suppressions.

Unfortunately, there are a couple of new issues about array additions
with only false positives, because apparently they don't take
branches into account.

Change-Id: I5a3913c6e762f77bfdae55051a395fae95d1f841
2019-12-07 20:16:19 +00:00
Aaron Schulz
5b3ce5dc8c filebackend: reduce hashing I/O in SwiftFileBackend::doStoreInternal()
Compute both hashes with one read of the local source file

Change-Id: I9679b50d99a46747a6606ca059ef0005ee1aaaa6
2019-09-15 21:30:50 -07:00
jenkins-bot
e80ff96c37 Merge "filebackend: rename and simplify header sanitizing methods in SwiftFileBackend" 2019-09-14 00:12:05 +00:00
Derick Alangi
bf25b795ea filebackend: Use HTTPFileStreamer::send404Message() in caller instead
Replaced usage of StreamFile::STEAM_HEADLESS (now making it unused) by
HTTPFileStreamer::STREAM_HEADLESS. Also, deprecated these unused constants.
Searched around and found nothing and CS also says so too;

https://codesearch.wmflabs.org/search/?q=StreamFile%3A%3A(STREAM_ALLOW_OB%7CSTREAM_HEADLESS)&i=nope&files=&repos=

Change-Id: Ia3fc2cf072fa2fddb910c64a547d12cb243290d5
2019-09-12 14:09:39 +01:00
Aaron Schulz
8d1e7c00c6 filebackend: rename and simplify header sanitizing methods in SwiftFileBackend
In addition:
* Remove restriction that Content-Type was ignored in 'copy' and 'move'.
* Fix failing tests due to copy/move not respecting 'ignoreMissingSource'
  when using doQuickOperations().

Change-Id: Id1befeefeaaf75f0a6bf0e68b1e62271d432f953
2019-09-08 12:42:25 -07:00
Daimona Eaytoy
b3e4a6f61f Add docblocks to a bunch of methods
Copied from I014ea7f048862cc8cc387a68bf0f1cc5acbd9c6e.

Change-Id: Ic3216386af173cfb9cbffda0826879829e52ffbb
2019-09-08 18:33:31 +00:00
Aaron Schulz
e390198c4e filebackend: use self:: instead of FileBackend:: for some constant uses
Change-Id: Iae82a074ef9da3c36b34713d31ec65407d2a90b7
2019-09-08 14:15:19 +00:00
Aaron Schulz
903f1810c8 filebackend: improve internal use of FileBackend constants
Add more result constants and split up FileBackend::UNKNOWN for
clarity. This follows up 5719815f3b, which added that constant.

Make internal FileBackendStore::doGet* methods distinguish I/O errors
from missing files; the return types of public FileBackend methods are
unchanged. Avoid process caching any mtime/size/sha1 values in the
case of I/O errors. Use error constants consistently for stat methods
when given invalid paths.

Also:
* Factor out FileBackendStore::processCacheAndPersistStatEntries() method
  to reduce significant code duplication.
* Consolidate duplicated isPathUsable() checks in FileOp subclasses to
  FileOp::precheck().
* Remove null process cache value check from FileBackend::getFileStat()
  as null values are never stored in the process cache to begin with.
* Reformat some oddly wrapped lines to look cleaner.

Change-Id: Id0e4b0da0bb2ed3184847b35142d587c7f3d953d
2019-09-03 19:31:17 -07:00
Daimona Eaytoy
e70b5b3309 Unsuppress other phan issues (part 4)
Bug: T231636
Depends-On: I58e67c2b38389df874438deada4239510d21654f
Change-Id: I6e5fba7bd273219b1206559420b5bdb78734aa84
2019-08-31 17:13:39 +00:00
Daimona Eaytoy
7f7efbe026 Globally unsuppress phan issues with low count
All of these suppression prevent the detection of many common mistakes,
and could easily prevent things like T231488. Especially if there are
few issues of a given type, it's way better to suppress them inline,
instead of disabling them for the whole core.
This patch only touches the one with a lower count (although those
counts may be out of date).

Bug: T231636
Change-Id: Ica50297ec7c71a81ba2204f9763499da925067bd
2019-08-30 09:40:47 +00:00
Aaron Schulz
5719815f3b filebackend: add idiom constant to FileBackend for null results
Change-Id: I65f043e87d82192c13d3627fb45ef222f0290bf8
2019-08-29 01:19:33 -07:00
Aaron Schulz
d01d98012b filebackend: use AtEase in FileBackend related classes
Change-Id: I819e0e923dfd3ffa7299acd1d6103e3cdf2baa93
2019-08-23 13:46:30 +00:00
Aryeh Gregor
a83b33582a TempFSFileFactory service
This replaces TempFSFile::factory(), which is now deprecated.

Change-Id: I9e65c3867e26c16687560dccc7d9f3e195a8bdd6
2019-08-21 16:26:05 +03:00
Aaron Schulz
bcfba52cd1 filebackend: avoid bogus SwiftFileBackend log warnings due to HTTP 202 responses
Change-Id: I12897bd2e29f18bbdce9a54a18415e83acc31365
2019-06-28 13:28:49 -07:00
Umherirrender
4b489fa2a7 Fix type hint for properties holding MapCacheLRU
Change-Id: Ia22b1dbcde3b96b3f437a213da95b3556fa42d70
2019-06-05 20:32:16 +02:00
James D. Forrester
460bcf81e7 build: Upgrade mediawiki/mediawiki-phan-config from 0.5.0 to 0.6.0 and make pass
Fix five instances of PhanPluginDuplicateConditionalNullCoalescing;
escape the rest for now.

Bug: T219114
Change-Id: Ic4bb30c43c5315ce6b878b37b432c6e219414f8b
2019-05-13 14:57:07 +00:00
Reedy
7f04a47d68 Split filebackend files to class per file
Change-Id: Idf9f4177fb9a523ce41227bda2af923bf418396b
2019-04-14 23:02:13 +01:00
Aaron Schulz
e1497e3593 filebackend: avoiding computing file SHA-1 hashes unless needed
FileBackendStore already supports stat info not returning SHA-1.
Build on that logic with a "requireSHA1" parameter to getFileStat()
to move some logic from SwiftFileBackend to the parent class and
avoid computing missing SHA-1's for Swift when nothing actually
requested the SHA-1. Only getFileSha1Base36() needs to trigger this
lazy-population logic.

Note that thumbnails only use doQuickOperations(), which does not
need to examine SHA-1s, it only does regular getFileStat() calls.

Also renamed addMissingMetadata() to addMissingHashMetadata().

Bug: T204174
Change-Id: I2a378cb2a34608a6da2f8abe604861ff391ffaa7
2018-12-10 22:51:26 +00:00
Aaron Schulz
ec7dfb4a34 filebackend: switch callers from ProcessCacheLRU to MapCacheLRU
Change-Id: I6efe7122ff30053a85461d31be7b5ddad0a5d2af
2018-07-14 11:27:09 +00:00
Umherirrender
130ec2523d Fix PhanTypeMismatchDeclaredParam
Auto fix MediaWiki.Commenting.FunctionComment.DefaultNullTypeParam sniff

Change-Id: I865323fd0295aabd06f3e3c75e0e5043fb31069e
2018-07-07 00:34:30 +00:00
Max Semenik
1e680456b4 Get rid of call_user_func(_array)(), part 3
Also cleaned up nearby code in a couple places.

Change-Id: Ibf44ee7c0ceb739d7e79406e4ff39303c316e285
2018-06-10 02:21:24 +00:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
Timo Tijhof
a89095bd52 filebackend: Fix undefined 'req_params' context in FileOperation log
Follows-up 9d49075b38. Noticed various log messages on Beta Cluster
from this code path that used "{params}" even in the real messages
(not normalised). Turned out, it isn't being defined due to a typo.

Change-Id: Ib9cf644fdf413035504d9d7486696cffa7737d4a
2018-05-20 01:39:36 +02:00
Gilles Dubuc
cd233f1165 Separate additional swift users for public and private
For better security seggregation it can be desirable
to set different additional swift users when dealing
with private containers.

Bug: T187822
Change-Id: I66d26ec81b3a3577d274c0d28cf86db7b505e082
2018-02-26 10:25:59 +01:00
Reedy
39f0f919c5 Update suppressWarning()/restoreWarning() calls
Bug: T182273
Change-Id: I9e1b628fe5949ca54258424c2e45b2fb6d491d0f
2018-02-10 08:50:12 +00:00
Chad Horohoe
5c93fabafd Drop XCache support
It was never super popular anyway, APC was always the best option.
The project has no plans to move to PHP7, so it's quickly reaching
its end of life. Oh, and Fedora dropped it from their repos 2 years
ago.

Change-Id: Ia3257e86a6323d8943f04a5c05c72c0bd4c4b0a9
2018-02-07 13:45:40 -08:00
Aaron Schulz
795412ccaf Clean up formatting of SwiftFileBackend::sanitizeHdrsStrict()
Change-Id: I5e411f4bf50836d22db01220853776494a9e69d5
2017-11-28 09:07:37 -08:00
Cormac Parle
d36d987c4e Do not strip Content-Type header for POST requests to swift
libcurl adds 'Content-Type: application/x-www-form-urlencoded'
to a POST request if the 'Content-Type' header is not set
manually. Because data in swift is updated via POST, the
Content-Type header must be set explicitly to stop a run of
refreshFileHeaders.php from changing the Content-Type of all
files in swift to application/x-www-form-urlencoded

Bug: T178849
Change-Id: I43c21bc1b73e37104cf07cd5f1c1557f472b9898
2017-11-28 14:24:27 +00:00
Aaron Schulz
205f4606af SwiftFileBackend::resolveContainerPath() check the proper length
The length sanity check should use this instead of urlencode()
see it is rawurlencode() that is actually used.

Change-Id: I5632e30c14c8ab27c8324c3e31311ca8bff7c162
2017-11-17 23:37:04 +00:00