r65152 switched upload-by-URL ($wgAllowCopyUploads) to use Http / MwHttpRequest class instead of CURL directly.
While this is mostly nice, it switched from saving large files directly to a temp file to buffering them in memory, causing large files to fail when they hit the PHP memory limit.
Fix uses MwHttpRequest's callback capability to override the read handler; now appending it to the temporary file as we go, and can confirm that largish files work again; was able to upload a 64mb .ogv that previously didn't work for me: http://prototype.wikimedia.org/tmh/images/b/b2/File-Arborophila_brunneopectus_pair_feeding_-_Kaeng_Krachan.ogv
Also expanded the documentation on MwHttpRequest::setCallback to clarify the function parameters and return value for the callback (which currently matches the low-level CURL handler's callback directly).
Note that the non-CURL implementation doesn't abort the read if the callback doesn't return the expected number of bytes, but this is an immediate fatal end of request on the Curl backend. May want further cleanup.
Backwards compatible, if not set to an array, applies to all uploads. If set to an array, per upload type maximums can be set, using the file and url keys. If the * key is set this value will be used as maximum for non-specified types.
ApiUpload:
* Added "statuskey" parameter; this is the key that is returned by an async upload
* Refactored warnings transformation into its own function
* filename is no longer required on all uploads
UploadFromUrlJob:
* Moved upload results to its own entry in $_SESSION, instead of using the one from upload
* Fix storing in session by calling wfSetupSession and session_write_close where needed
Tests:
* Set $wgUser in ApiSetup, so that individual tests don't have to do this for themselves
* Added tests to cover most code paths from the API
* Fixed UploadFromUrlTestSuite so that its tests are included in a regular phpunit invocation (something strange with the AutoLoader; not sure what)
Other files:
* Allow passing session id to wfSetupSession
* Explicitly close the session before doing jobs, so that jobs can't manipulate the current session
Other changes:
* Moved verifyPermissions check in ApiUpload down pending r70135 implementation in the API.
* In User::leaveMessage: append message to end of talk page; add a newline before the heading
* In ApiUpload: moved stuff that is checking instead of actual uploading out of performUpload method
* Made UploadFromUrl conform to standards:
** In initialize* do only initialization, no actual work
** Moved file fetching to fetchFile
** Consistent use of tempnam()
** Perform the uploading in performUpload, don't define our own doUpload method
* Moved almost all job magic to the UploadFromUrlJob class. This way the job is almost a regular client, and we don't need many special cases to deal with async uploading.
* Made leaving a message optional; results will be stored in the session otherwise
I did not actually test the async uploading, because I first wanted to commit a properly working synchronous upload-by-url system.
<siebrand> PHP Warning: Missing argument 4 for UploadFromUrl::initialize(), called in /www/w/includes/upload/UploadFromUrl.php on line 99 and defined in /www/w/includes/upload/UploadFromUrl.php on line 38
* Define variable for ApiUserrights.php that wasn't defined before.
* Add convertVerifyErrorToStatus and getVerificationErrorCode to
UploadBase to translate error consts since UploadFromUrl will
need a message to display to end-users.
* refactor mime-checking out of UploadBase::verifyFile into
UploadBase::verifyMimeType
* Make UploadBase::verifyFile always return arrays for errors
* Use HttpFunctions instead of custom curl handler for async downloading
* TODO: Need a way to feed errors back to the requestor
* TODO: Need to add watchlist param handling and warnings checks.
rename UploadBase::initialize to UploadBase::initializeFileInfo
Each sub-class implements initialize() with a different prototype. Only two sub-classes actually use the parent initialize().
* Reverted HttpFunctions.php to r45549 and renamed wgSyncHTTPTimeout back to wgHTTPTimeout
* Edited out the asynchronous features from UploadFromUrl. Made fetchFile() use the curlCopy() function from new-upload r47811 instead of Http::doDownload(). Wrote my own URL validity check to avoid having to use either of the two buggy precedents.
* Removed UploadFromChunk
* Removed chunk upload and background status from ApiUpload.php
* Reverted r54669, use of addScriptClass()
* Left getHeadScripts() in its current location (OutputPage) instead of moving it back to SkinTemplate, just added wikibits.js to it to replace the removed addCoreScripts2Top()
