Commit graph

75 commits

Author SHA1 Message Date
Chad Horohoe
c33f4de066 Profile all external HTTP requests from MW
Change-Id: Ie980b080da2ef21ec7d9fc32f1accc55710de140
2015-03-03 20:54:30 -08:00
jenkins-bot
44aec8a00a Merge "More debug diagnostics for upload by URL" 2015-02-23 21:39:41 +00:00
saper
27bdde86c7 More debug diagnostics for upload by URL
Increase debug log verbosity to
troubleshoot file upload by URL issues.

Bug: 73200
Change-Id: I530bd38b85911e0c153280b7d9a9da023f799b02
2015-02-23 13:16:23 -08:00
Aaron Schulz
e369f66d00 Replace wfRunHooks calls with direct Hooks::run calls
* This avoids the overhead of an extra function call

Change-Id: I8ee996f237fd111873ab51965bded3d91e61e4dd
2014-12-10 12:26:59 -08:00
Siebrand Mazeland
69a2ecfe3e Update formatting of includes/upload/
Change-Id: I8cf59cd3bb6dd8de2ed6509b7bc2ef9ff7c5caf1
2014-05-09 16:53:19 +02:00
umherirrender
957adbef22 Fixed some @params documentation (includes/[file...|upload])
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.

Change-Id: I8804ebe0922d3a414863b162a2110e0b9e49b80f
2014-04-19 17:19:17 +02:00
Bryan Davis
f9c6af781c Fix TempFSFile creation
A new TempFSFile should be instantiated via the factory rather than
using the object constructor.

Change-Id: Icd059566087c6a644a0d0ed69abfe13b88b38ee4
2013-12-13 12:49:45 -07:00
Bryan Davis
e4009c7367 Use TempFSFile in UploadFromUrl
Make it a little harder for the developer to leak files in the temp
directory by managing the temporary file with a TempFSFile instance.

Change-Id: I8bafc3a6e6212781d9e2e4f8f370ce7c446f1fd5
2013-12-12 16:10:14 -07:00
Brian Wolff
7a0ed924ef Have separate timeouts for upload by url ($wgCopyUpload[Async]Timeout)
Add $wgCopyUploadTimeout and $wgCopyUploadAsyncTimeout to control
the timeout for the http request used to fetch the file during upload
by url. People reasonably may want to make this higher than $wgHTTPTimeout
for the case where very large files are being downloaded.

Also add the ability for callers of UploadFromUrl::fetchFile
to override the timeout as they please (and set any other HTTP opts).

This patch was inspired by conversation with dan-nl about gwtoolset.

Change-Id: Ia85a97434c14adcdaafc0802cbe0530bfa57a435
2013-08-07 21:05:07 +02:00
nischayn22
50ca5a7d17 hook to intercept upload_by_url
This is planned to be used for intercepting by UW for Flickr blacklisting.

Bug: 42307
Change-Id: Ia3daaeba1ce5e69e751ffc2ae5afd5e449cf4aa7
2013-07-24 19:11:33 +02:00
umherirrender
ee31d37821 Fixed spacing in context/installer/media/templates/upload folder
Added spaces before if, foreach
Added some braces for one line statements

Change-Id: I9761be9fa47adc3554852a97b19792b4648466ad
2013-04-21 06:22:06 +00:00
Timo Tijhof
acb292d733 phpcs: Fix Squiz.WhiteSpace.ScopeClosingBrace.ContentBefore violations
ERROR: Closing brace must be on a line by itself
Squiz.WhiteSpace.ScopeClosingBrace.ContentBefore

- For non-empty scopes it means the closing brace must be on a separate
  line. This is already the case in most classes in some cases the "lazy
  closing" is still used.
  array(
   'x' ); // Moved } to next line
  function () { return 'x'; } // Moved } to next line
  case 1:
    stuff; break; // Moved break to next line
- For empty function it serves as a visual distinction between there not
  being a scope block / function body and there being an empty function
  body.
  function foo(); // No body
  function foo() {} // Empty body - violation
  function bar() { // Empty body corrected
  }

Change-Id: I0310ec379c6d41cc7d483671994d027a49f32164
2013-04-11 07:34:41 +00:00
Platonides
a1e7329d8f Use JobQueueGroup::singleton()
Replaces usages of the deprecated pop() and insert()/insertBatch()
methods of Job class.

Change-Id: I5572c9e31a14bd660d7e3b9aa7792d0be9a26103
2012-12-07 20:30:45 +01:00
Kaldari
ba3fa1aca5 Merge "Adding wildcard support to $wgCopyUploadsDomains" 2012-09-28 23:20:52 +00:00
Reedy
7476057925 Add $wgCopyUploadProxy to be used for copy uploads only.
Means we don't break stuff on the cluster such as search by setting $wgHTTPProxy

Change-Id: I094b2bfd097e7e5c1c45b6332357d959ed95ac99
2012-09-28 01:33:48 +01:00
Kaldari
e3b95ca5eb Adding wildcard support to $wgCopyUploadsDomains
Change-Id: I06bff1c626d847e4e28dca0772a931f6d6925736
2012-09-24 17:24:16 -07:00
Tim Starling
ad03f2f753 Revert Uri class due to bug 39710
Revert 94f623363b. Apparently all
ReverseChronologicalPager subclasses were broken by the unexplained
interface change in wfArrayToCgi(): changing the interpretation of
null array values from "omit this key" to "include this key with no
value". The function has always been called with the former
interpretation, that's why the special case was in there.

Also reverted dependent changes 4b517fa and part of 3d97704.

Change-Id: I41dadbde5d0e0e4c54cc76bfc13c263c15c73828
2012-08-28 10:53:09 +10:00
Siebrand Mazeland
3d97704f91 Replace some deprecated method calls.
Change-Id: Id4beca7b6821139fcc319c5694917e68668835ee
2012-08-19 23:27:43 +02:00
Reedy
a69152d732 Minor bits of documentation updates to upload code
Including hint for bug 39195

Change-Id: I4cdab97011330c12a943112f6c05134362d26a82
2012-08-09 16:20:09 +01:00
Kaldari
7bf1303c10 More accurate function description for isAllowed()
Change-Id: Idf95efc20714bfb2595259bcce6ba9eaf9a63fae
2012-07-23 11:17:25 -07:00
awjrichards
c29fd59775 Big oops - merged to wrong branch.
Revert "Revert to arbitrarily old point before initial remote branch creation to help clean up"

This reverts commit ee0d3d330f
2012-06-05 22:58:54 +00:00
awjrichards
ee0d3d330f Revert to arbitrarily old point before initial remote branch creation to help clean up
Change-Id: I41a3d1e55d3ea9dffa42451237fe065f9334361d
2012-06-02 08:43:04 -07:00
Alexandre Emsenhuber
58bb669812 Added missing GPLv2 headers in some places.
Also made file/class documentation more consistent.

Change-Id: Ia1a7230adc92079b505362317d2e03b36130cc8b
2012-05-29 13:01:11 +02:00
Reedy
f8d2958da9 Can't return values from functions that don't return anything
Change-Id: If964ce2f39904f1e9e3101e7cafaa6a96d58adfc
2012-04-19 14:07:40 +01:00
Sam Reed
4ee96a4a4b Followup r11120, wfParseUrl() can return false, if it does, the host isn't going to be valid 2012-02-11 15:13:01 +00:00
Sam Reed
b834075972 * (bug 32341) Add upload by URL domain limitation.
Essentially reverts r109741  live again

Which was r109562, r109564, r109570
2012-02-09 23:22:38 +00:00
Sam Reed
09a78c1368 More return documentation 2012-02-09 21:36:14 +00:00
Antoine Musso
2bf8d0db90 Fix doc for includes/upload 2012-02-08 17:03:43 +00:00
Sam Reed
af198296ef Revert feature out of r109562, r109564, r109570
Leaving typo/brace fixes and other minor code improvements
2012-01-22 17:33:44 +00:00
Sam Reed
9553784f82 Followup r109562
Don't use such a naieve host check
2012-01-19 20:26:10 +00:00
Sam Reed
f619564720 * (bug 32341) Add upload by URL domain limitation. 2012-01-19 19:16:06 +00:00
Sam Reed
68480e2ed3 Improve/add to documentation
Add braces

Explicitly define member variable
2011-12-24 23:11:25 +00:00
Brion Vibber
a16a028057 * (bug 29154) Allow upload-by-URL to follow HTTP redirects 2011-09-22 01:15:06 +00:00
Sam Reed
896f05efde Trim trailing whitespace
Add documentation

Remove unused variables

Remove an extract from ImageFunctionsTest
2011-07-24 21:36:04 +00:00
Brion Vibber
c07ed42491 * (bug 29174) Fix regression in upload-by-URL: files larger than PHP memory limit work again
r65152 switched upload-by-URL ($wgAllowCopyUploads) to use Http / MwHttpRequest class instead of CURL directly.
While this is mostly nice, it switched from saving large files directly to a temp file to buffering them in memory, causing large files to fail when they hit the PHP memory limit.

Fix uses MwHttpRequest's callback capability to override the read handler; now appending it to the temporary file as we go, and can confirm that largish files work again; was able to upload a 64mb .ogv that previously didn't work for me: http://prototype.wikimedia.org/tmh/images/b/b2/File-Arborophila_brunneopectus_pair_feeding_-_Kaeng_Krachan.ogv

Also expanded the documentation on MwHttpRequest::setCallback to clarify the function parameters and return value for the callback (which currently matches the low-level CURL handler's callback directly).
Note that the non-CURL implementation doesn't abort the read if the callback doesn't return the expected number of bytes, but this is an immediate fatal end of request on the Curl backend. May want further cleanup.
2011-05-27 22:31:48 +00:00
Neil Kandalgaonkar
43cf8e6d6e change the name of UploadBase::verifyPermissions() to verifyTitlePermissions() for clarity. A wrapper function is left for extensions that rely on the old method 2011-03-25 20:37:26 +00:00
Brian Wolff
6f7958499a trim the upload_by_url field from upload form, since if you have an extra space at the beggining like I
did when copying and pasting a url, gives a really misleading error message.
2011-03-04 22:21:48 +00:00
Sam Reed
baf83f74b8 More explicit variable definitions, function documentation 2011-02-20 13:33:42 +00:00
Bryan Tong Minh
b20a0d3327 Follow-up r81612, disable $wgAllowAsyncCopyUploads 2011-02-06 22:53:07 +00:00
Sam Reed
bdb325cb4e Fixup some trailing newlines
Fixup indentation of a few methods

Add a couple of return type hints
2011-02-02 10:42:17 +00:00
Bryan Tong Minh
db544af120 $wgMaxUploadSize may now be set to an array to specify the upload size limit per upload type.
Backwards compatible, if not set to an array, applies to all uploads. If set to an array, per upload type maximums can be set, using the file and url keys. If the * key is set this value will be used as maximum for non-specified types.
2011-01-06 19:42:55 +00:00
Sam Reed
03e4248def Properly qualify usage of class constants 2010-12-22 00:25:16 +00:00
Mark A. Hershberger
d0b46f4bbe re r75906 — replace the removed getSessionKey() with stashSession() 2010-12-08 18:59:43 +00:00
Brian Wolff
8adc7befe5 (Bug 25872) Rename HttpRequest class to MWHttpRequest to avoid conflict with php extension.
This also keeps the old HttpRequest class name around, so it should not break backwards compatability.
2010-11-12 07:32:09 +00:00
Bryan Tong Minh
3e1eff2a01 Follow-up r70137: Made asynchronous upload working a bit more. It now fully works from the API; works still needs to be done for the normal UI. PHPUnit tests are updated and should cover most code paths that can be called from the API.
ApiUpload:
* Added "statuskey" parameter; this is the key that is returned by an async upload
* Refactored warnings transformation into its own function
* filename is no longer required on all uploads

UploadFromUrlJob:
* Moved upload results to its own entry in $_SESSION, instead of using the one from upload
* Fix storing in session by calling wfSetupSession and session_write_close where needed

Tests:
* Set $wgUser in ApiSetup, so that individual tests don't have to do this for themselves
* Added tests to cover most code paths from the API
* Fixed UploadFromUrlTestSuite so that its tests are included in a regular phpunit invocation (something strange with the AutoLoader; not sure what)

Other files:
* Allow passing session id to wfSetupSession
* Explicitly close the session before doing jobs, so that jobs can't manipulate the current session
2010-09-06 10:18:53 +00:00
Alexandre Emsenhuber
2d078552de * Standardised file description headers
* Added some descriptions
* Added @file where needed
2010-08-20 20:39:04 +00:00
Bryan Tong Minh
d7e6870b8f Made asynchronous upload by URL working, partly. Hid it behind $wgAllowAsyncCopyUploads. If there are no errors then everything works expected; the same if there are unrecoverable errors. User intervention to solve warnings is not yet possible, because $_SESSION is not available in runJobs. This also means that async with leavemessage = false is broken.
Other changes:
* Moved verifyPermissions check in ApiUpload down pending r70135 implementation in the API.
* In User::leaveMessage: append message to end of talk page; add a newline before the heading
2010-07-29 13:53:51 +00:00
Bryan Tong Minh
796933d11f Restructured upload-by-url:
* In ApiUpload: moved stuff that is checking instead of actual uploading out of performUpload method
* Made UploadFromUrl conform to standards: 
** In initialize* do only initialization, no actual work
** Moved file fetching to fetchFile
** Consistent use of tempnam()
** Perform the uploading in performUpload, don't define our own doUpload method
* Moved almost all job magic to the UploadFromUrlJob class. This way the job is almost a regular client, and we don't need many special cases to deal with async uploading. 
* Made leaving a message optional; results will be stored in the session otherwise

I did not actually test the async uploading, because I first wanted to commit a properly working synchronous upload-by-url system.
2010-07-28 17:14:51 +00:00
Bryan Tong Minh
b2c4d114e9 Style fixes on UploadFromUrl.php 2010-07-28 13:43:38 +00:00
Sam Reed
04f68827d0 Removal of unused globals
Removal of one setting of a variable to '', then not using further
2010-07-24 19:11:52 +00:00