Commit graph

47 commits

Author SHA1 Message Date
Aryeh Gregor
0d3d6be18d Convert FileBackendGroup to service
Bug: T234228
Change-Id: I25575f565eba122cdf971a5945572811d17fa3e1
2019-10-25 10:46:20 +03:00
Umherirrender
1eb5d8e44d Do not output wikitext in maintenance script
Also remove print_r with a string
Always use english for maintenance scripts

Bug: T229843
Change-Id: I539a1ac1f6a201dd8ee9ce89599cc34bebfba79f
2019-09-20 18:28:34 +02:00
Daimona Eaytoy
e70b5b3309 Unsuppress other phan issues (part 4)
Bug: T231636
Depends-On: I58e67c2b38389df874438deada4239510d21654f
Change-Id: I6e5fba7bd273219b1206559420b5bdb78734aa84
2019-08-31 17:13:39 +00:00
Aaron Schulz
a7ef8e430a Fix various fatals and IDEA warnings in CopyFileBackend
Change-Id: I4a65a247dbb19055f9bc481b0d83bbe9737b17a3
2019-07-04 18:18:43 -07:00
Thiemo Kreuz
cc329a5b06 Update PHPDoc types in several maintenance scripts and related
There is like a billion of these in our core codebase. I would love to
update them all. But I don't have the time, at least not to do them all
at once.

TL;DR: It's an improvement. :-)

Change-Id: I0a9c51bdcf0f261f971586bb8703f418324b0f98
2019-03-07 17:54:26 +00:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
Umherirrender
ad776c7d5f Use ::class to resolve class names in maintenance scripts
This helps to find renamed or misspelled classes earlier.
Phan will check the class names

Change-Id: I1d4567f47f93eb1436cb98558388e48d35258666
2018-01-23 17:40:16 +00:00
Bryan Davis
9e34eeff23 Maintenance: add fatalError() method
Deprecate the second argument to Maintenance::error() in favor of a new
Maintenance::fatalError() method. This is intended to make it easier to
review flow control in maintenance scripts.

Change-Id: I75699008638f7e99b11210c7bb9e2e131fca7c9e
2017-11-21 21:34:16 -07:00
Max Semenik
dac20d0ffa Introduce Maintenance::getBatchSize()
Just to isolate the internals. Fix most of usages in the core.

Change-Id: I8b3e9ca1f42b7c49ee57f17b88ca2fc7b404f342
2017-11-05 13:26:12 -08:00
Max Semenik
5808276de3 Use Maintenance::setBatchSize() everywhere
Change-Id: Id48b95c547ddd8ea02fb1c577203f17056f38402
2017-11-04 16:10:06 -07:00
Ori Livneh
e5a0d6a183 Drop check for mbstring in copyFileBackend maintenance script
Already checked in PHPVersionCheck.php as of 41d1fa1c3a.

Change-Id: I35e55b63378f1768076be0c45392fdc5d48168bf
2016-06-30 11:58:20 -07:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Max Semenik
59db24e90b Use addDescription() instead of accessing mDescription directly
Change-Id: I0e2aa83024b8abf5298cfea4b21bf45722ad3103
2016-01-30 01:28:32 -08:00
Ori Livneh
c099155a17 ellapsed => elapsed
Also fix some files that don't end with a newline.

Change-Id: Id0672d685b929a5832b42f733dad49683536180a
2015-06-23 03:32:33 +00:00
Siebrand Mazeland
f2f31fadc7 Update formatting in maintenance/ (2/4)
Change-Id: I2b791d3bff0de464b6bdaaeae0622c065389c31c
2014-04-23 20:08:42 +02:00
umherirrender
e78776373e Fixed some @params documentation (maintenance)
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.

Change-Id: I727deec35a712de0f0c676cc87dfa661f1ee965b
2014-04-17 22:48:32 +02:00
Aaron Schulz
b3c1283cc6 Optimized copyFileBackend to use MD5 from listing if given (e.g. Swift)
Change-Id: Ic495d6c8c6dffb0b34e6b5eefb8de4d057a51069
2014-01-10 01:48:10 +00:00
umherirrender
0bc583af2c Move closing parenthesis from multi line if and function to own line
The Line continuation Coding conventions prefers the closing parenthesis
on the same line than the beginning curly braces. This is done for ifs
and functions.
Also move some boolean operator from the end of a line to the beginning
and changed some indentation to make the condition hopefully better
readable.

Change-Id: Id0437b06bde86eb5a75bc59eefa19e7edb624426
2013-12-01 21:39:00 +01:00
Aaron Schulz
30d115c4ae Avoid HEAD requests for files missing from the destination
Change-Id: I8a3d2ddeb650c800d1e4af144d5cd2cb23cd8744
2013-11-12 23:26:32 +00:00
Aaron Schulz
1b726ce0da Added a "syncviadelete" param to copyFileBackend script
* Added a bit of tabbing to clarify the output

Change-Id: I4b0095595d742452b151e6ed6765a9dc805a55ae
2013-10-16 23:27:08 +00:00
Aaron Schulz
f8ba647478 Fixed trailing slash removal code.
Change-Id: I152dcec5cc8f4c417e3503f4b8fd3fdd8bc4fda5
2013-05-31 10:56:12 -07:00
Timo Tijhof
beb1c4a0ec phpcs: More require/include is not a function
Follows-up I1343872de7, Ia533aedf63 and I2df2f80b81.

Also updated usage in text in documentation and the
installer LocalSettingsGenerator.

Most of them were handled by this regex:
- find: (require|include|require_once|include_once)\s*\(\s*(.+?)\s*\)\s*;$
- replace: $1 $2;

Change-Id: I6b38aad9a5149c9c43ce18bd8edbab14b8ce43fa
2013-05-21 23:26:28 +02:00
Aaron Schulz
95f0a167e0 Clear stat cache before determining local copy error message.
* This makes it clearer if the listings are just wonky.

Change-Id: I78078fb9fac88b17e010b0eb990dbbeb83ea48f1
2013-05-13 13:14:22 -07:00
Aaron Schulz
e87bb70a67 Added wiki ID to backend copy script errors.
Change-Id: Ibe1fc6037d20f6669578295b6db62a4e414ee6b7
2013-05-13 10:51:01 -07:00
Aaron Schulz
d5cf69f8fa Added missing continue to avoid fatal error.
Change-Id: Ibb87c9e59e424fa84d662bb42349085d186dcc99
2013-05-12 17:58:37 +00:00
Aaron Schulz
7cb1672c9d Made copyFileBackend.php skip files that cannot be copied instead of stopping.
Change-Id: Ib15efdd35f38adefe3db500b4b6e5a71ea1858c7
2013-05-12 01:14:42 +00:00
umherirrender
c6894adefc Fixed spacing
Added spaces around some parenthesis

Change-Id: If1e50e2a7b0046e91c0bbce8bf6641d8b2446a1d
2013-05-09 19:48:10 +02:00
Aaron Schulz
d1940caa1f [FileBackend] Added "adviseStat" option for the "listing followed by stat" case.
* Used this parameter to speed up copyFileBackend.php.
* Also added mtime checks to copyFileBackend.php and a few cleanups.
* Also fixed some incorrect getDirListPageInternal/getFileListPageInternal docs.

Change-Id: I424ef238f7adf4cf1f33b74e3a4e187dcb328a99
2013-05-09 16:19:40 +00:00
Timo Tijhof
50e7985d4d phpcs: Fix WhiteSpace.LanguageConstructSpacing warnings
Squiz.WhiteSpace.LanguageConstructSpacing:
   Language constructs must be followed by a single space;
   expected "require_once expression" but found
   "require_once(expression)"

It is a keyword (e.g. like `new`, `return` and `print`). As
such the parentheses don't make sense.

Per our code conventions, we use a space after keywords like
these. We appeared to have an unwritten exception for `require`
that doesn't make sense. About 60% of require/include usage
was missing the space and/or had superfluous parentheses.

It is as silly as print("foo") or return("foo"), it works
because keywords have no significance for whitespace between
it and the expression that follows, and since experessions can
be wrapped in parentheses for clarity (e.g. when doing string
concatenation or mathematical operations) the parenthesis
before and after basiclaly just ignored.

Change-Id: I2df2f80b8123714bea7e0771bf94b51ad5bb4b87
2013-05-09 05:56:26 +02:00
Aaron Schulz
f8af77dd9d Reduced memory usage a bit for copyFileBackend.php with --missingonly.
Change-Id: Iae650b5c3a5f99c5b2e31e29c276957ea6b8bb50
2013-04-30 18:42:03 -07:00
Aaron Schulz
e712a37ea9 Mention the batch-download time in copyFileBackend.php.
Change-Id: Idc46f9ed276d116b5b7eda0fbf354659243b646f
2013-02-26 08:46:35 +00:00
Aaron Schulz
58a67086c6 Improved concurrency of copyFileBackend.php with pipelined downloads.
Change-Id: I0bde212cbba527d89b0c43f5174a77788cd07b71
2013-01-22 13:54:07 -08:00
jeroendedauw
38c7f444e1 Use __DIR__ instead of dirname( __FILE__ )
We can now do this since we finally switched to PHP 5.3 for MW 1.20 and get rid of the silly dirname(__FILE__) stuff :)

Change-Id: Id9b2c9cd2e678197aa81c78adced5d1d31ff57b1
2012-08-27 21:45:00 +02:00
Aaron Schulz
eb65b839f8 [FileBackend] Added option for copy script to skip non-UTF8 filenames.
Change-Id: I9b48012d890c2c4589984576a31d84dde535ad5b
2012-08-10 10:17:31 -07:00
Aaron Schulz
a649bfb419 [FileBackend] Skip over some illegal paths and output the error.
Change-Id: Ib32aadc6398168e4a63279d7da61a0dd1e1a6483
2012-08-01 16:48:33 -07:00
Aaron Schulz
a3e0cd0886 Free up some memory by unsetting large arrays in backend copy script.
Change-Id: I55b6a6f8228370d948821dcd4de22fda51539fd0
2012-07-17 23:25:30 -07:00
Aaron Schulz
6455336013 [FileBackend] Added "bypassReadOnly" flag.
* Added flag to bypass read-only checks to the 6 write functions.
* Made backend copy and sync scripts pass in this new flag.
* Also made sync script use doQuickOperations() since its faster.

Change-Id: Iee47f79ed7ab002cfc2d0adb5321c3a8520f971d
2012-07-17 20:55:49 -07:00
Aaron Schulz
0126fd93a1 [FileBackend] Made copy script retry on batch copy failure.
Change-Id: I5db35371a41fa72d449cba1fb7e15f19f0cefa21
2012-07-12 16:33:27 -07:00
Aaron Schulz
bd006888c9 [FileBackend] Tweaks to speed up backend copy script.
* Added 'missingonly' option to copy based only on a listing comparison.
* Made copy operation batches use doQuickOperations().

Change-Id: If5657a4c4d895980e27776357bb408b350727382
2012-07-12 11:11:45 -07:00
Aaron
46992c8bb4 [FileBackend] Added hash skipping option to speed up script.
Change-Id: If810bbdeffdbfa37a3609992c3e8038ee4deebe6
2012-07-11 16:34:43 -07:00
Aaron
a5821cf8be [FileBackend] Only show "copied file(s)" for files the script had to copy.
Change-Id: I22741303e05d6ea50d33875d9f7438d2b3905b18
2012-07-10 12:21:11 -07:00
Aaron
082cde1f9d [FileBackend] Improved copy/sync script output.
Change-Id: I5f76fdb579cf17959093e7a8659839e8b8829b1c
2012-07-10 11:33:24 -07:00
Aaron
4a6a0632fe [FileBackend] Added "ratefile" option for backend copy script.
* This can be used to control the concurrency dynamically.

Change-Id: I7ebdbffa144928f559c34c4a4684b03597f616ce
2012-07-10 10:24:37 -07:00
Tim Starling
49df232e1e Merge "[FileBackend] Changed copy script to use batches for concurrency." 2012-06-26 05:45:33 +00:00
Alexandre Emsenhuber
9eee3153ae Improve documentation of maintenance scripts.
Change-Id: I328a3d42a09f2b082ebc153c2f399793e23942d7
2012-06-16 22:59:39 +02:00
Aaron Schulz
ce0677a1a2 [FileBackend] Changed copy script to use batches for concurrency.
* Also added a 'subdir' option so multiple shards can be done at once.

Change-Id: Id8c3a89a4cb30978f66db3bf95ebfebc0a9c01b4
2012-05-06 18:07:29 -07:00
Aaron
4845f5c01b [FileBackend] Added a script to copy files from one backend to another. Useful for re-sharding.
Change-Id: Id942e83420fc5214ddcc8a5b483fd72166a76ff2
2012-04-26 10:21:58 -07:00