Commit graph

236 commits

Author SHA1 Message Date
Matthias Mullie
4e50971b32 Add API warnings when upload is same as older versions
Bug: T141822
Change-Id: I115d84d865c59200dbb60bd962c093185c9afafe
2016-09-06 18:43:00 +00:00
Bartosz Dziewoński
13d2035cbb ApiUpload: Better handle unreasonably large metadata in 'imageinfo'
Bug: T143993
Change-Id: I1fcdbca9981dd034572eeb32070d574cf97a132f
2016-08-26 16:53:52 +02:00
Bartosz Dziewoński
0afc3cf016 ApiUpload: Fix fatal in dieStatusWithCode()
If $extraData was null, but $moreExtraData was given, the following
fatal would occur:

  Fatal error: Unsupported operand types in
  /var/www/html/w/includes/api/ApiUpload.php on line 408

Follow-up to c9b5b3e988.

Change-Id: I613eed1f7429247fe46afa454d36f518f6a81ebe
2016-08-22 21:16:21 +02:00
Bartosz Dziewoński
959daa2ca4 Check for warnings for assembled file after a chunked upload
Bug: T130564
Change-Id: Iebc84f030c45c634dc29b02cbd720f33abf50f2b
2016-08-19 01:33:10 +00:00
Bartosz Dziewoński
ec41a9c559 UploadBase: Stop mLocalFile doubling as stashed file
"I've a great idea", they said. "You know what would be cool? If I
made this boring getter, getLocalFile(), return something completely
different after the file was stashed. This will be a nice surprise for
someone in the future to discover", they added gleefully.

I am pretty sure everything still works, but I never could get async
upload publishing to work locally, so I'd appreciate some testing.

Change-Id: I11dcf2ed89e4f1dd8ddf081af521da005efdbf39
2016-08-19 03:00:30 +02:00
Bartosz Dziewoński
c9b5b3e988 Run 'UploadStashFile' hook for chunked uploads too
In UploadFromChunks, doStashFile() only stashes the current chunk,
and stashing the whole file after all chunks are uploaded is done
in concatenateChunks().

Also more custom code on top of custom code to handle ApiMessage
errors, because action=upload is special.

Follow-up to 2ee27d9a4d.

Change-Id: I968280353f15bbca9b1059631fa2cfd7c3cd2f1d
2016-08-17 17:53:08 +02:00
Bartosz Dziewoński
09f2cba006 Do not call the 'UploadStashFile' hook for partially uploaded files
Consumers of this hook don't expect partial files, and even if they did,
they have no way to tell if they've been given a partial file.

Also document a pre-existing restriction that verifyUpload() must be
called first (for non-partial files). ApiUpload previously called this
function for partial files too, causing T143161.

Follow-up to 2ee27d9a4d.

Bug: T143161
Change-Id: I107cb957e046545ea72834d72233cc1ed2867e42
2016-08-17 01:34:52 +02:00
Bartosz Dziewoński
2ee27d9a4d Introduce UploadStashFile hook, improve API handling of stash errors
UploadBase:
* Introduce a new method, tryStashFile(), as a replacement for the
  now-soft-deprecated stashFile(). The method runs the new hook and
  returns a Status object, with an error (if the hook returned an
  error) or a value (if it didn't).
* Introduce a new hook, UploadStashFile, allowing extensions to
  prevent a file from being stashed. Note that code in extensions
  which has not been updated for MediaWiki 1.28 may still call
  stashFile() directly, and therefore not call this hook. For
  important checks (not just for UI), extension authors should use
  UploadVerifyFile or UploadVerifyUpload hooks.
* Extract common code of tryStashFile() and stashFile() to
  a new protected method doStashFile().

SpecialUpload:
* Use tryStashFile() when stashing a file after a warning or
  "recoverable error" was encountered.

ApiUpload:
* Refactor stashing code so that error handling only happens in one
  place, not four different ones. Use Status objects rather than
  exception throwing/catching for control flow.
* Simplify the error messages slightly (error codes are unchanged).
  Produce better ones by always using handleStashException().
  'stashfailed' is now always at root (not nested inside 'warnings'),
  behaving the same as 'filekey' does on success.
* Use tryStashFile() when stashing. Handle errors so as to allow
  custom API results passed via ApiMessage to be preserved.

Some API result changes for different requests are shown below.

  api.php?action=upload&format=json&filename=good.png&file=...&stash=1

    Before:
      {
        "error": {
          "code": "stashfilestorage",
          "info": "Could not store upload in the stash: Stashing temporary file failed: UploadStashFileException Error storing file in '/tmp/phpB32SRT': Could not create directory \"mwstore://local-backend/local-temp/3/3a\".",
          "*": "See http://localhost:3080/w/api.php for API usage"
        }
      }

    After:
      {
        "error": {
          "code": "stashfilestorage",
          "info": "Could not store upload in the stash: Error storing file in '/tmp/phpB32SRT': Could not create directory \"mwstore://local-backend/local-temp/3/3a\".",
          "*": "See http://localhost:3080/w/api.php for API usage"
        }
      }

  api.php?action=upload&format=json&filename=[bad].png&file=...

    Before:
      {
        "upload": {
          "result": "Warning",
          "warnings": {
            "badfilename": "-bad-.png",
            "stashfailed": "Stashing temporary file failed: UploadStashFileException Error storing file in '/tmp/phpB32SRT': Could not create directory \"mwstore://local-backend/local-temp/3/3a\"."
          }
        }
      }

    After:
      {
        "upload": {
          "result": "Warning",
          "stashfailed": "Could not store upload in the stash: Error storing file in '/tmp/phpB32SRT': Could not create directory \"mwstore://local-backend/local-temp/3/3a\"."
          "warnings": {
            "badfilename": "-bad-.png",
          }
        }
      }

Bug: T140521
Change-Id: I2f574b355cd33b2e9fa7ff8e1793503b257cce65
2016-08-09 18:10:18 +02:00
jenkins-bot
b3307c8a5a Merge "ApiUpload: Better handle ApiMessage errors from UploadVerifyFile hook" 2016-07-11 19:34:49 +00:00
jenkins-bot
96d9fb9731 Merge "Introduce new hook UploadVerifyUpload to allow preventing file uploads" 2016-07-07 02:27:44 +00:00
Brad Jorsch
5f01cbb3ad Allow Message::newFromSpecifier to handle ApiMessages
Instead of constructing a new Message from the Message as
a MessageSpecifier, just clone the existing Message which will preserve
subclass data.

Also, make use of this to simplify the logic in ApiBase::parseMsg().

Change-Id: I9545acb8da752c0c21e16d8b1d37d8802fcb329d
2016-06-28 12:15:27 -04:00
Bartosz Dziewoński
931796f88b ApiUpload: Better handle ApiMessage errors from UploadVerifyFile hook
The extra data is actually understood and output now.

Bug: T137961
Change-Id: Ifac8995a4d16d11840cee814177fc2808bc2072c
2016-06-23 04:39:02 +00:00
Bartosz Dziewoński
8f2acfcde4 Introduce new hook UploadVerifyUpload to allow preventing file uploads
The new hook runs at the beginning of UploadBase::performUpload().
Unlike the existing UploadVerifyFile hook, the new one provides the
information about the file description page being created, and the
user who is performing the upload. It also does not run for uploads
to stash.

The action=upload API and Special:Upload have been changed to treat
errors from UploadBase::performUpload() as recoverable, which means
that they will attempt to stash the file (previously they would require
the user to upload it again). This is mostly intended for errors from
the new hook, but I think it makes sense for the existing ones, which
are mostly transient filesystem erorrs.

Bug: T89302
Change-Id: Ie68801b307de8456e1753ba54a29c34c8063bc36
2016-06-22 23:38:50 +02:00
Matthias Mullie
ed6a2d5df1 Check if user is blocked during upload process
Further down, this was already somewhat being checked.
On L112, `verifyTitlePermissions` is called, which will fail
if the user is blocked. However:

* This was not being checked during stashed uploads
* Block just "happens" to be part of that check: the intent is
  actually to verify the file title. The error is treated as
  recoverable (the title can be changed), but it isn't (the
  user can't unblock himself in this process)

Bug: T111228
Change-Id: I9cbf250a0b92c3daa3a0843f2257cc049abd3923
2016-05-03 13:23:44 +00:00
jenkins-bot
8cb8af442a Merge "Use english messages for background use of Status::getWikiText" 2016-04-13 14:03:56 +00:00
umherirrender
932c37e3cb Use english messages for background use of Status::getWikiText
Status::getWikiText is used for internal logging, api error messages and
maintenance scripts. All this places are usually in english, so pass an
english language to getWikiText.

Change-Id: I3010fca8eb5740a3a851c55a8b12e171714c78f7
2016-04-12 20:01:44 +02:00
Mark Holmquist
785d472559 Add preference for watching uploaded files
Adds a preference in the Watchlist section for watching uploaded files
for an account. Also works from API-based upload methods, so
UploadWizard and other tools should work fine.

Bug: T33313
Change-Id: If962e667de12b35904b2d1b2d9e99c26b588ec2a
2016-04-08 15:21:17 -05:00
Siebrand Mazeland
5b119a0e44 Replace uses of join() by implode()
All of core uses implode() consistently now.

Change-Id: Iba50898c64c43f356d1caf8869f484e90d9ff651
2016-03-08 18:24:16 +00:00
Siebrand Mazeland
bc0ae710e3 Use single quotes in API where possible
Change-Id: I972e296f4820f78f5dfcecc27bc4912ca84a3178
2016-03-08 17:27:00 +01:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Bartosz Dziewoński
05c2c65ab9 Allow users to tag file uploads
Using either action=upload API or Special:Upload. (No user interface
is provided for the latter, this is meant to be used by on-wiki
scripts/gadgets enhancing the upload process.)

Modelled after how ae3ab9eef0
implemented tagging of regular edits.

Bug: T121876
Change-Id: Ia3e0dbd895b2f8bc66985b24db35f112b6f9a22d
2016-02-16 03:51:29 +01:00
Bartosz Dziewoński
1ebc9128d0 Remove putrid remains of $wgAllowAsyncCopyUploads
None of this works and it's been long begging for a mercy kill.
All it does is waste contributor time on updating deprecations
in the dead code. I imagine we wouldn't reuse much of this
code if we're ever going to reimplement it.

Bug: T119336
Change-Id: Ibd26a4bea621857aac77823017e9be9b7dc52cca
2016-01-22 00:18:25 +00:00
Reedy
32ff1f4c47 Remove various unused variables
Change-Id: I4b1b20b4126735cb32a80e473fe48d523bcb24d1
2015-11-07 21:22:17 +00:00
jenkins-bot
0183ae1453 Merge "Allow passing detailed permission errors data to API" 2015-11-04 22:38:47 +00:00
Brad Jorsch
05ff357a42 API: Log all deprecated parameter uses to api-feature-usage.log
Some were being logged, and some weren't. Let's log them all
automatically when PARAM_DEPRECATED is processed, instead of requiring
each module to manually log them.

Bug: T117569
Change-Id: Ia38aeeccd0b9857b12b28914f509284483fbcca8
2015-11-03 12:23:00 -05:00
Bartosz Dziewoński
92c29b8891 Allow passing detailed permission errors data to API
Using the new system introduced in
1c57794e37 (see T47843).

This change allows Title::getUserPermissionsErrors() to include
MessageSpecifiers instead of string message keys in its return value.
This doesn't seem to have any bad effects, and should work seamlessly as
long as callers aren't trying to do anything stupid and just pass the
value to PermissionsError or OutputPage::showPermissionsErrorPage()
or wfMessage() or some such.

If the callers *are* trying something stupid, nothing worse than
duplicated or otherwise less-than-perfect error messages (in code
which tries to handle some message keys specially) should happen.
(I fixed wfMergeErrorArrays(), but who knows what else lurks in all
this code.) Any problems should only affect new-style errors using
MessageSpecifier, though.

Since MessageSpecifiers tend to be stringable, we probably won't get
fatals, but might get incorrect checks. Should we try to log this
happening somehow?

Goes with I42a0c5b0ea7e61088dd609b764dd7d1396c60cd5 in TitleBlacklist.

Bug: T115258
Change-Id: I1334ba21a2862973a9d8ff5be2c9bec06a82698b
2015-11-02 17:11:50 +01:00
jenkins-bot
376879ee06 Merge "ApiBase::PARAM_DFLT => null is the default anyway" 2015-10-26 19:02:33 +00:00
csteipp
59b627b0b7 SECURITY: API: Improve validation in chunked uploading
This fixes a few shortcomings in the chunked uploader:
* Raises an error if offset + chunksize > filesize.
* Enforces a minimum chunk size for non-final chunks.
* Refuses additional chunks after seeing a final chunk.
* Status of a chunked upload in progress is now available with
  'checkstatus'.

Bug: T91203
Bug: T91205
Change-Id: I2262db1bc8460616b069c564475d2e4148001768
2015-10-16 14:10:44 -07:00
csteipp
c804391572 SECURITY: Throttle uploads
Add throttle check in ApiUpload and SpecialUpload.

Bug: T91850
Change-Id: If33cc99f304aab2486507c7500b4abb06b6b5d70
2015-10-16 11:23:18 -07:00
Thiemo Mättig
3ecd418501 ApiBase::PARAM_DFLT => null is the default anyway
Change-Id: Ic3deeb6b3d7cacbdb85da9ba3cb19051c1182b8f
2015-10-05 10:56:32 +02:00
Siebrand Mazeland
0fdb0ce284 Fix most PHP CodeSniffer warnings in includes/api
Change-Id: I01bb3e4c96d6034a5b6c18728bb0574c710ea9db
2015-09-28 14:24:52 +02:00
Brad Jorsch
11893e4761 API: Improve upload error reporting
* Include the detailed message text in the error for verification-error and
  hookaborted
* Actually return the raw "details" for hookaborted and unknown-error
  (previously it was colliding with the standard "error" and "code"
  elements).

Bug: T105224
Change-Id: I13b7b6ad02fbbf46bf3d6b4c683493b2fecf8c58
2015-07-10 09:31:04 -04:00
Brad Jorsch
df80f1ead5 API: Add more parameter types and improve info
New types 'text' and 'password' for where a <textarea> or
<input type="password"> would be preferred over <input type="text">.

Some timestamp parameters get actually tagged as 'timestamp'.

'submodule' types change the 'submodules' output property from a boolean
to an object indicating the mapping from values to module paths. And
they get an indication of the submodule parameter prefix (e.g.
generator's "g"), if applicable. "generator" actually gets reported as a
submodule type, using this new mechanism.

action=paraminfo will now indicate ApiBase::PARAM_RANGE_ENFORCE status,
and return better-formatted defaults for timestamps and booleans.

Change-Id: Ic862d6f8fe13f7eb6b4298683514d33af5823e47
2015-05-29 19:26:44 +00:00
Brad Jorsch
1c57794e37 API: Overhaul ApiResult, make format=xml not throw, and add json formatversion
ApiResult was a mess: some methods could only be used with an array
reference instead of manipulating the stored data, methods that had both
array-ref and internal-data versions had names that didn't at all
correspond, some methods that worked on an array reference were
annoyingly non-static, and then the whole mess with setIndexedTagName.

ApiFormatXml is also entirely annoying to deal with, as it liked to
throw exceptions if certain metadata wasn't provided that no other
formatter required. Its legacy also means we have this silly convention
of using empty-string rather than boolean true, annoying restrictions on
keys (leading to things that should be hashes being arrays of key-value
object instead), '*' used as a key all over the place, and so on.

So, changes here:
* ApiResult is no longer an ApiBase or a ContextSource.
* Wherever sensible, ApiResult provides a static method working on an
  arrayref and a non-static method working on internal data.
* Metadata is now always added to ApiResult's internal data structure.
  Formatters are responsible for stripping it if necessary. "raw mode"
  is deprecated.
* New metadata to replace the '*' key, solve the array() => '[]' vs '{}'
  question, and so on.
* New class for formatting warnings and errors using i18n messages, and
  support for multiple errors and a more machine-readable format for
  warnings. For the moment, though, the actual output will not be changing
  yet (see T47843 for future plans).
* New formatversion parameter for format=json and format=php, to select
  between BC mode and the modern output.
* In BC mode, booleans will be converted to empty-string presence style;
  modules currently returning booleans will need to use
  ApiResult::META_BC_BOOLS to preserve their current output.

Actual changes to the API modules' output (e.g. actually returning
booleans for the new formatversion) beyond the use of
ApiResult::setContentValue() are left for a future change.

Bug: T76728
Bug: T57371
Bug: T33629
Change-Id: I7b37295e8862b188d1f3b0cd07f66ac34629678f
2015-04-10 16:57:15 -04:00
rillke
9853803750 Chunked upload: Return expected offset on offset error
Bug: T87535
Change-Id: If68e0075e73a78c1dd8d95839f7ee9374a995201
2015-02-12 17:04:41 +01:00
umherirrender
40a21ab2b8 Pass user to FileRepo::getUploadStash
This avoids use of $wgUser in UploadStash

Change-Id: I82ca69818317508109b4d5f4823a20de47f29b01
2015-01-31 21:46:05 +01:00
Aaron Schulz
6921770414 Updated some try-catch statements: MWException -> Exception
Change-Id: I76601a86e30f4984e3b1a8c8ec5ef5a0f652433a
2015-01-09 17:20:22 -08:00
Mark Holmquist
8e96834ce4 Remove stash handling when other errors are the problem
This interferes with getting the correct error message, which makes it
nearly impossible to help debug problems...

Change-Id: Ib04b897bc912065aaa4900f7904fe724ffec5385
2014-11-24 13:24:11 -06:00
jenkins-bot
79168e0f8b Merge "Add more detailed upload stash error messages" 2014-10-29 17:40:53 +00:00
Brad Jorsch
f62bc7536e API: Fix access on getExamplesMessages
ApiBase declares it protected, but for some reason I had made it public
in all subclasses.

Change-Id: I8a50d4f47e66c7f09137968d3941dc5cdc1d28e4
2014-10-29 11:15:27 -04:00
Mark Holmquist
7585e214d2 Add more detailed upload stash error messages
There are seven (used) error types in the stash class, and we umbrella'd
them all into one error message, which is mighty silly. This should give
us more information.

Also added to the mw.Api.errors list so UploadWizard can handle them.

Change-Id: I79bf0c29a4cef19363d111cc1128e35256ae572a
2014-10-27 19:41:25 +00:00
Brad Jorsch
ad225f501c API: Internationalize all remaining core API modules
This also adds some new ApiBase::PARAM_* constants to generate more
helpful help, and a method to override the default description message
for the use of ApiDisabled and ApiQueryDisabled.

Bug: 71638
Change-Id: Ic0c3d232e0498d58a043037e2e0c6f0b1c3edad3
2014-10-20 16:56:35 -04:00
Aaron Schulz
c7844017c0 Made upload jobs avoid using the user session
* This causes problems with some session handlers and it is
  also trickier to deal with in non CLI script without leaking
  cookie headers.

Change-Id: Iaf2a57f9299e42a5f68bf85115e62e88fa0f8ed6
2014-09-29 16:35:59 -07:00
Gilles Dubuc
650b76518b Chunked upload result should consistently have a "stage" value for "Poll"
Change-Id: If9c2b215c43eef002482695701182c471f8bb450
Mingle: https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/855
2014-09-08 08:05:44 +02:00
Brad Jorsch
fdddf94570 API: Overhaul token handling
The current token handling is a mess. This simplifies things greatly:
* *All* tokens are obtained from action=query&meta=tokens, rather than
  being spread over action=tokens, action=query&prop=info,
  action=query&prop=revisions, action=query&prop=recentchanges, and
  action=query&prop=users. All these old methods are deprecated.
* Similarly, there is only one hook to register new token types. All old
  hooks are deprecated.
* All tokens are cacheable.
* Most token types are dropped in favor of a 'csrf' token. They already
  were returning the same token anyway.
* All token-using modules will document the required token type in a
  standard manner in action=help and are documented in machine-readable
  fashion in action=paraminfo.

Note this will require updates to all extensions using tokens.

Change-Id: I2793a3f2dd64a4bebb0b4d065e09af1e9f63fb89
2014-08-26 14:53:45 -04:00
Brad Jorsch
b5cd9e2f6b API: Log usage of various deprecated features
This will let us know how aggressively we can finally remove these.

Change-Id: I03fab36e921807e74fbabfa878756af254d89a1b
2014-08-14 16:51:14 -07:00
umherirrender
3b2b6a2773 Add missing @param to function docs
Change-Id: I47fa96a976f55a1a93cb75397285edb8c7f4cd8a
2014-08-14 20:22:52 +02:00
Yuri Astrakhan
703464a88c Cleanup - let's make IDEs more useful
http://phpdoc.org/docs/latest/references/phpdoc/types.html

If IDEs have many warnings, we don't look at them.
Let's minimize the number of warnings, and make them useful again.

* Some function docs fixes
* Removed unused $iwprefixes var in ApiQuerySearch.php
* declared private $blockStatusByUid in SpecialActiveusers
* declared private $repo in UploadFromChunks

Change-Id: Ifd20f78b168b9a913fdb8d89dc26a76a173b1c29
2014-08-13 16:02:59 -04:00
Brad Jorsch
f0a6435f3b API: Remove action=paraminfo 'props' and 'errors' result properties
The format for 'props' was never specified and the list for 'errors' is
impossible to keep updated when considering that many errors come from
MediaWiki backend code and extension hook functions. And since there
doesn't seem to be any real use case for either of these, let's just
kill both of them instead of wasting effort on trying to fix them.

Note that neither getResultProperties nor getPossibleErrors are called
from any extensions in gerrit, and none of the other deprecated methods
are called outside of the implementations of those two methods. Removing
the obsolete methods is left to the maintainers of the extensions, as
keeping them hurts nothing and is needed to maintain compatibility with
earlier versions of MediaWiki.

Change-Id: Ie11a401d60c834059fbf1b5625ca8ea093b3337c
2014-08-07 16:51:19 +01:00
Reedy
e48ecbc524 Switch API to use Config classes
Only done where globals are config (so not $wgParser, $wgContLang etc)

Change-Id: Ic39cdd858cfb9096a2bc09618f97e64270d76f13
2014-06-15 23:56:38 +01:00