Commit graph

42 commits

Author SHA1 Message Date
Bill Pirkle
0584339f5e Added non-parallel fallback to MultiHttpClient when curl is unavailable
If the curl extension is not available, fall back to the existing
HttpRequestFactory and associated classes. Also added related phpunit tests.

Bug: T139169
Change-Id: I2f9d4acbb491bce28d7105e124c5cee7e16e86d7
2018-07-23 00:19:59 +00:00
Tim Starling
792dc3c74a Improve timeouts in MultiHttpClient
* PHP 7 only checks for request/connection timeout after the timeout
  passed to curl_multi_select() expires. So it's necessary to use a
  select timeout which is much shorter than 10s. I filed
  https://bugs.php.net/bug.php?id=76480 for this.
* Use millisecond resolution timeout parameters. These have been
  available since libcurl 7.16.2, released Jan 2008, available in
  Debian 7, Ubuntu 14.04 and CentOS 6.

Change-Id: Ia07b824dde179b33e14b81a76d580ce547bca315
2018-06-15 21:04:05 +10:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
James D. Forrester
9d875febd5 Bump PHP version requirement to 7.0.0+
Bug: T172165
Change-Id: I740f32ac859d9bb3787fdf8414f82ae6f410492f
2018-05-24 16:15:56 +01:00
Kunal Mehta
230958d97c Autofix MediaWiki.Commenting.FunctionComment.SpacingDoc* errors
Change-Id: I63761ebce04c03b9b13237919c27cc10180f198f
2018-05-19 14:07:03 -07:00
Fomafix
73a514b574 Remove superfluous spaces and semicolons in comments
Change-Id: Ib4b452f1843ec250c8c1fcc2a738d80726b6135d
2018-02-05 19:05:57 +00:00
Moritz Schubotz (physikerwelt)
708875ab32
Throw exception if php-curl is missing
Currently MediaWiki just dies silently, if the php
curl extension is missing.

Throw an exception instead that redirects the admin to the appropriate help page.

Change-Id: I25edfcb34ee5451a742cac6ae099019a16f6c417
2018-02-02 17:21:43 +01:00
Umherirrender
3f1a52805e Use short type bool/int in param documentation
Enable the phpcs sniffs for this and used phpcbf

Change-Id: Iaa36687154ddd2bf663b9dd519f5c99409d37925
2017-08-20 13:20:59 +02:00
Umherirrender
ace44e2064 Use correct variable name in @param documentation
For some varargs a variable name is added with suffix ,... as seen for
many other varargs

Some @param are swapped, because there are in the wrong order

Enable Sniff MediaWiki.Commenting.FunctionComment.ParamNameNoMatch

Change-Id: I60fec6025bce824d5c67563ab7b65ad6cd628ad8
2017-08-11 19:27:19 +02:00
Umherirrender
a9007e8baf Add missing & to @param documentation to match functon call
Change-Id: I81e68310abcbc59964b22e0e74842d509f6b1fb9
2017-08-11 18:47:46 +02:00
Aaron Schulz
82e2c924e4 Remove "@author Aaron Schulz" annotations
Bug: T139301
Change-Id: Ib5248e8e27d60611c7373bce4b29dd5e85aa3489
2017-06-27 15:24:14 -07:00
Tim Starling
cde44feba3 Improve HTTP logging
* Log HTTP debug lines to the "http" channel instead of wfDebug()
* Add the ability to do debug logging to MultiHttpClient
* Add a static method Http::createMultiClient() which provides a
  MultiHttpClient configured similarly to the way individual requests
  are configured, respecting the wiki's $wgHTTPTimeout and writing debug
  logs.
* In EtcdConfig, pass the logger instance through to the MultiHttpClient
  backend.

Change-Id: Ic5bdcb0cae95d7b3715ab5261758be082751c3ff
2017-05-19 06:49:26 +02:00
Fomafix
202f695f67 Update weblinks in comments from HTTP to HTTPS
Use HTTPS instead of HTTP where the HTTP link is a redirect to the HTTPS link.

Also update some defect links.

Change-Id: Ic3a5eac910d098ed5c2a21e9f47c9b6ee06b2643
2016-11-07 15:24:46 +01:00
Kunal Mehta
1651a2b485 Remove pre-5.5 compat from MultiHttpClient
Change-Id: I5f9f1a94c0207a14c0d5cc77292aa04af4ac6470
2016-10-01 22:25:06 -07:00
Kunal Mehta
24c0261daf MultiHttpClient: Don't use wfArrayToCgi()
http_build_query() should be functionally identical.

Change-Id: Ia7d909e97a89d66dbe5187e9a0423e2ecf8561e8
2016-09-20 23:10:54 -07:00
Brion VIBBER
425090d4eb Merge "Added Range support to FileBackend::streamFile()" 2016-07-19 19:20:12 +00:00
Stanislav Malyshev
7ade0a7c67 Create BagOStuff implementation to talk to RestBase
Or any other HTTP REST server.

Bug: T137272
Change-Id: Iefef24ffa831ba59d7725da8d20d5addb544b3ab
2016-07-06 17:52:07 +00:00
Aaron Schulz
d42a05475a Added Range support to FileBackend::streamFile()
* Added HTTP options headers parameter to streamFile().
* Refactored doStreamFile() to either call StreamFile::stream()
  or delagate that to the subclass. SwiftFileBackend now relays
  the full Swift response rather than manually making the headers.
  This also makes Range headers easy to support.
* Made use of this in img_auth.php for performance on private wikis.
* Elimate stat call in streamFile() for Swift if "headers" is empty.
* Refactored StreamFile a bit to inject request headers instead
  of using the $_SERVER global. A header options parameter is used
  instead, which also supports Range.
* Removed now unused prepareForStream().
* Cleaned up streamFile() unit tests.

Change-Id: I2ccbcbca6caabb8cf65bd6b3084cede2e6ea628a
2016-06-04 05:43:43 -07:00
Bartosz Dziewoński
c161c46d26 Improve code suffering from PHP 5.3's lack of support for foo()[]
I searched for /\$(\S+) = (.+?\(.*?\);)\n.*?\$\1\[/, ignored
everything involving isset(), unset() or array assigments, then
skimmed through the remaining results and changed things where they
made sense. These changes were not automated, so please review them.

Change-Id: Ib37b4c66fc57648470f151ad412210b3629c2538
2016-02-28 22:49:20 +01:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Roan Kattouw
e95721aae1 SECURITY: Work around CURL insanity breaking POST parameters that start with '@'
CURL has a "feature" where passing array( 'foo' => '@bar' )
in CURLOPT_POSTFIELDS results in the contents of the file named "bar"
being POSTed. This makes it impossible to POST the literal string "@bar",
because array( 'foo' => '%40bar' ) gets double-encoded to foo=%2540bar.

Disable this "feature" by setting CURLOPT_SAFE_UPLOAD to true,
if available. According to the PHP manual, this option became
available in 5.5 and started defaulting to true in 5.6.
However, we support versions as low as 5.3, and this option
doesn't exist at all in 5.6.99-hhvm, which we run in production.

For versions where this option is not available (pre-5.5 versions
and HHVM), serialize POSTFIELDS arrays to strings. This works
around the issue because the '@' "feature" only works
for arrays, not strings, as of PHP 5.2. (We don't support pre-5.2
versions, and I've verified 5.6.99-hhvm behaves this way as well.)

Bug: T118032
Signed-off-by: Chad Horohoe <chadh@wikimedia.org>
Change-Id: I3f996e2eb87c7bd3b94ca9d3cc14a3e12f34f241
2015-12-18 01:22:30 -08:00
Aaron Schulz
ce16b7ccbf Restore "userAgent" MultiHttpClient constructor override
This is more convenient than needing to set it on each
HTTP request. Also, it is still documented as being there.

Change-Id: I246c9c3861d72701c844200b82d1a38a0c301795
2015-10-06 22:29:42 +00:00
Timo Tijhof
1340a7717c MultiHttpClient: Don't use "MW" in User-Agent
Follows-up 6f51037d68.

Change-Id: I5e404c3a6ab9c3d1126676ee5fc22149422aa51f
2015-10-06 13:02:21 -07:00
Tim Starling
6f51037d68 Add User-Agent support to MultiHttpClient requests
Because it is polite to send a User-Agent (and as part of T114558
investigation).

Change-Id: I5096896d0891c0c9674ff8b4284aaa3ac7293c67
2015-10-06 15:23:54 +11:00
Aaron Schulz
795c895884 Fixed occasional MultiHttpClient "undefined index" errors
* These show sometimes in Swift FileBackend tests

Change-Id: I38c1817812282c94cab2011da7c322e92eec96ca
2015-08-06 20:56:34 +00:00
Niklas Laxström
4c9a819b21 Use feature test rather than version check
Not sure why I didn't do this in the first place.

Bug: T106579
Change-Id: I238aedacd4f960ad4ca7549c232fd3a7eec3f02f
2015-07-22 21:43:17 +02:00
jenkins-bot
e19a52f2f5 Merge "Fix MultiHttpClient to return correct errors" 2015-07-14 21:38:06 +00:00
Niklas Laxström
db2d4b6709 Fix MultiHttpClient to return correct errors
Bug: T89758
Change-Id: Ia1f1986d150df1a9f463d48f1a914a04608550f8
2015-07-14 23:21:50 +02:00
Vivek Ghaisas
ec02aba4c0 Change spaces to tabs
Bug: T103243
Change-Id: I145c056ef1b058c9c24b4a402ac9e7778b23e260
2015-06-21 10:00:58 +03:00
Aaron Schulz
a2ea6116af Fixed some <code> tags for doxygen
Change-Id: I550b8760556e4916ee9b63ee244bbbcb4d926142
2015-04-29 06:01:59 +01:00
Aaron Schulz
ef4119cb9e Document units in MultiHttpClient
Change-Id: I0350e4e558f881e3accd01818a56e11465d92d00
2015-04-20 21:22:55 -07:00
jenkins-bot
b5cf0db562 Merge "Make a VirtualRESTService class for Parsoid" 2015-01-07 01:29:44 +00:00
Alex Monk
24409664f1 Make a VirtualRESTService class for Parsoid
So that code to deal with Parsoid can be shared across different extensions.

Relies on I1d57ff24

Bug: T1218
Change-Id: Id658d925b722b885fd425c27f7c4fd2cd02ad0b4
2015-01-07 01:19:45 +00:00
Reedy
4d9143c7f5 Add lots of @throws
Change-Id: I09d0c13070f966fcf23d2638d8fc1328279a5995
2014-12-24 13:49:20 +00:00
Alex Monk
901b28fb51 Fix error in docs of VirtualRESTServiceClient and MultiHttpClient
err -> error

Change-Id: I758c026599367b282150a513b09d6f6c837a3a80
2014-12-19 21:51:45 +00:00
Aaron Schulz
0702671f2e Fixed doc comment
Change-Id: I906f3f34ac27207dd6b87b07fa6e791bc7b66691
2014-08-14 15:35:30 -07:00
Aaron Schulz
4d6d11d64b Added VirtualRESTServiceClient/VirtualRESTService classes
* Reference handling cleanup for loops in MultiHttpClient
* Support a slightly more convenient request array format for MultiHttpClient
* Added missing license header to MultiHttpClient

Change-Id: Icc86ed9e4c2ac78363aa7b6f3147253bb59e2afb
2014-04-29 17:01:25 -07:00
Aaron Schulz
c6d766fb0e Set the "error" field in the right array
Change-Id: Id822ceb53c57daac0000bef3b304bb0260efe4ce
2014-04-07 13:15:31 -07:00
Aaron Schulz
bc7f296240 MultiHttpClient tweaks
* Reference handling cleanup for loops in MultiHttpClient
* Support a slightly more convenient request array format for MultiHttpClient
* Added missing license header to MultiHttpClient

Change-Id: I0a2eef02c06d9c900c4a1ae1b0eeb292755d6cd9
2014-02-11 16:53:10 -08:00
Aaron Schulz
086d13957c Added pipelining and max connection support to MultiHttpClient
* Also added timeout options to the run()/runMulti() methods

Change-Id: I2b080567e17fa29804c36e3b259b1ce214e6f465
2014-02-10 17:56:21 -08:00
Aaron Schulz
616df59b32 Fixed http_build_query warnings on PHP < 5.4
* bug: 59880

Change-Id: I2eae4bed17a72b58ac33f3538dd7d16f4356a3a0
2014-01-09 13:46:29 -08:00
Aaron Schulz
2af7ad827d Removed cloudfiles dependency in SwiftFileBackend
* Added a new curl multi wrapper under libs/.
* Used the new MultiHttpClient class instead of CloudFiles
* Removed CDN related config and code as it does not work.
  well due to mutating files and tiny purge rate limits by CDNs.
* Also removed the unused and esoteric 'swiftAnonUser' parameter.
* Support concurrent 'describe' operations.
* Set ACLs on containers on intial PUT instead of in two steps.
  Previously if the second step failed, prepare() would never
  set the ACLs correctly since it aborts if the container exists.
* Improve token deletion to only trigger on 401s rather than other
  random errors like 50X timeouts.
* Removed duplicative getResponse* code by just using one closure
  called for both the concurrent and serial cases. Both cases are
  now funneled through doExecuteOpHandlesInternal().
* Add the MD5 to stat info, as it might be useful for speeding up
  the copy scripts in some cases (when both backends have md5 set).
* Avoid use of trigger_error() since there is a swift log group.
* A few other small code cleanups.

Change-Id: I1adb3e2df6df8cf01d3ad74158de96ea9a79da2e
2014-01-07 23:56:34 -08:00