Commit graph

385 commits

Author SHA1 Message Date
Aaron Schulz
e369f66d00 Replace wfRunHooks calls with direct Hooks::run calls
* This avoids the overhead of an extra function call

Change-Id: I8ee996f237fd111873ab51965bded3d91e61e4dd
2014-12-10 12:26:59 -08:00
jenkins-bot
cad7a1125d Merge "Don't try to verify XML well-formedness for partial SVG uploads" 2014-11-06 18:50:29 +00:00
Mark Holmquist
7585e214d2 Add more detailed upload stash error messages
There are seven (used) error types in the stash class, and we umbrella'd
them all into one error message, which is mighty silly. This should give
us more information.

Also added to the mw.Api.errors list so UploadWizard can handle them.

Change-Id: I79bf0c29a4cef19363d111cc1128e35256ae572a
2014-10-27 19:41:25 +00:00
jenkins-bot
cc8cfcde3d Merge "Avoid attempting to prerender thumbnails that will fail" 2014-10-03 17:45:15 +00:00
Gilles Dubuc
0ce5bafb4b Avoid attempting to prerender thumbnails that will fail
For non-vectorial content, requesting a thumbnail larger than
the original results in a 500. Prerendering in its current form
introduces an increase in 500s that dilutes the real problematic
500s, making troubleshooting harder than it needs to be.

Change-Id: I9418dee7653ad7954c3788ecdd350fc8772edd32
Mingle: https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/301
2014-10-03 18:09:07 +02:00
Brad Jorsch
d1bbbe61c4 Don't try to verify XML well-formedness for partial SVG uploads
Chunked uploads of SVGs are currently failing with "invalid XML" errors
because UploadBase::detectScriptInSvg() requires the full file but is
being called from UploadBase::verifyPartialFile().

So let's do the check twice: once in UploadBase::verifyPartialFile()
where it will pass if non-well-formed, and once in
UploadBase::verifyFile() where it will fail if non-well-formed.

Bug: 65724
Change-Id: I6126e185eb4b183c31946f13c576521f1ed19c16
2014-10-03 11:49:45 -04:00
jenkins-bot
31f73c4213 Merge "Made upload jobs avoid using the user session" 2014-10-03 15:41:11 +00:00
Gilles Dubuc
478332cfbe Fix version comments in thumbnail prerendering code
Change-Id: I74b52d2ecc382dfc0b55615ce512e8c64d02e9cb
Mingle: https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/301
2014-09-30 14:20:11 +02:00
Aaron Schulz
c7844017c0 Made upload jobs avoid using the user session
* This causes problems with some session handlers and it is
  also trickier to deal with in non CLI script without leaking
  cookie headers.

Change-Id: Iaf2a57f9299e42a5f68bf85115e62e88fa0f8ed6
2014-09-29 16:35:59 -07:00
umherirrender
df24b7209d Fixed spacing
- Added newline at end of file
- Removed double spaces/newlines
- Added space after if/function and parentheses/brackets
- Removed space before comma/cast
- Fixed indent of some lines

Change-Id: I29867ffdffdfb7d2b56997e9393497c7dc12f7d3
2014-09-29 20:46:19 +02:00
jenkins-bot
e6e6861881 Merge "Add ability to pre-render thumbnails at upload time" 2014-09-25 15:40:07 +00:00
csteipp
6aa3befeb0 SECURITY: Enhance CSS filtering in SVG files
* Filter <style> elements
* Normalize style elements and attributes before filtering
* Add checks for attributes that contain css
* Add unit tests for html5sec and reported bugs

Bug:69008
Change-Id: I732eece710f1bfaaeea1e5de541fcd4cfb375de7
2014-09-24 21:06:54 +02:00
Gilles Dubuc
623d778902 Add ability to pre-render thumbnails at upload time
This introduces an option that allows one to define a list of
thumbnail sizes to be rendered by async jobs at upload time.

Change-Id: Ida193699738c79aca333fa78b8b536d14a410841
Mingle: https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/301
2014-09-12 09:20:41 +02:00
umherirrender
3a85ac488f Do not return anything on __construct
Change-Id: I47f67ed17112af832b19f7e7538cae3e0b8354ef
2014-08-24 11:29:45 +02:00
jenkins-bot
496e14b0b2 Merge "Correct variable names in @param to match method declarations" 2014-08-13 20:13:37 +00:00
Yuri Astrakhan
703464a88c Cleanup - let's make IDEs more useful
http://phpdoc.org/docs/latest/references/phpdoc/types.html

If IDEs have many warnings, we don't look at them.
Let's minimize the number of warnings, and make them useful again.

* Some function docs fixes
* Removed unused $iwprefixes var in ApiQuerySearch.php
* declared private $blockStatusByUid in SpecialActiveusers
* declared private $repo in UploadFromChunks

Change-Id: Ifd20f78b168b9a913fdb8d89dc26a76a173b1c29
2014-08-13 16:02:59 -04:00
umherirrender
21e0c1c533 Correct variable names in @param to match method declarations
Some @param have a typo in the variable name,
some @param's were in wrong order.

Change-Id: Ie25806831027112b398f6f4a909c59147ac3a5fa
2014-08-13 21:48:28 +02:00
rillke
c31fbf073e Unify the spelling of MIME in documentation
Writing MIME as written in Wikipedia and some documentation clean up.

Change-Id: I9dfc36d2bf55d72d9374c4075bd6d45eef0415a4
2014-08-07 23:38:45 +02:00
Matthew Flaschen
731a7a8ebd Clarify comment in UploadBase
Change-Id: Ied29298d312ca19ecfdd2196fafa89130321987c
2014-07-28 22:48:42 -04:00
Antoine Musso
dcdf45010d Remove some duplicate newlines in call to wfDebug()
Spotted on beta:

--------
commonswiki-375c0c3e:

0.0752   1.9M  mime: <image/png> extension: <png>

--------

With lot of requests, that causes them to be interlayed and ungreppable.

Change-Id: Ib99bc9547bdecb9efa59b7d292ddd7d10a552652
2014-07-27 14:31:26 +00:00
umherirrender
768ac15c70 Cleanup some docs (includes/[s-z])
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling

Change-Id: Ie419638e909a47aa72a274043604247830ee1a81
2014-07-24 19:43:44 +02:00
jenkins-bot
79d908ef74 Merge "Fix mime detection of easily-confused-with text/plain formats" 2014-07-04 07:43:42 +00:00
Brian Wolff
69905ce9c7 Fix mime detection of easily-confused-with text/plain formats
json, csv, and tsv are often detected as text/plain. However that's
not right. This patch causes MediaWiki to look at the file extension
of files detected as text/plain, and if the file extension is
for a "textual" type, use the mime type associated with that extension.

This change also changes the "does mime type match uploaded file
extension" check to use the mime based on the file contents
plus extension, as opposed to just the file contents. Various
documentation suggests this is more appropriate (e.g. line 807
of MimeMagic.php). In my opinion we should use just the file
contents when verifying file is not on blacklist, but use ext
when verifying file type matches extension, and for decided
what handler specific checks to run. Not the detect mime type
with extension doesn't override the detected mime type with
the extension, but only uses the extension if content based
detection is ambigious or not specific enough.

This patch should be reviewed by csteipp before merge for
any potential security implications.

Note: This is partially fixing a regression from 3846d10487,
where previously csv and json files were allowed to be uploaded,
and that change prevented them

Bug: 66036
Bug: 45424
Change-Id: Ib637fe6850a81b26f84dc8c00ab4772f3d3a1f34
2014-07-04 04:03:31 -03:00
umherirrender
fa5c5f2837 Fix return doc for UploadBase::isAllowed
Method can return bool or string

Change-Id: Ib290843ce97926bfc529d510b1898f35c7e3f6c6
2014-06-29 14:49:24 +02:00
csteipp
7114801e78 SECURITY: Prevent external resources in SVG files
On bug 65724, it was discovered that a user could upload SVG images
with embedded <image> elements that pulled in the resource via http.
This could allow an attacker to track all viewers of an SVG by having
the image embed another image hosted on their own server.

While testing the patch, I also identified 3 more element namespaces
that have been used on commons and seem harmless, so I added those to
the whitelist.

Change-Id: Iaaabc3a60c0ec4e6e426a8680d7a2cef5d469d29
2014-06-25 22:25:45 +02:00
csteipp
02e55fa26b Add some SVG element namespaces
Noticed that we have several old images on commons that use these
namespaces. They seem harmless, so I think we should add them.

Change-Id: Ib80298256c5d4717867eb857d5404a9487fed784
2014-06-02 12:18:48 -07:00
jenkins-bot
7fa77c2d52 Merge "Added unicode encoding support flags to FileBackend" 2014-05-15 15:06:19 +00:00
Aaron Schulz
0d54e6e0bf Added unicode encoding support flags to FileBackend
* Fixed bug where even using Swift/Azure on Windows
  would disallow non-ASCII file names.

bug: 1780
Change-Id: I19ed72da0b099d35cae74fb08eeb22c113da1065
2014-05-13 14:11:36 -07:00
Siebrand Mazeland
d185ec734a Pass phpcs-strict on includes/upload
Change-Id: I80eaa22398227a4edb7160d560b625102c7436ac
2014-05-10 10:25:54 +02:00
Siebrand Mazeland
69a2ecfe3e Update formatting of includes/upload/
Change-Id: I8cf59cd3bb6dd8de2ed6509b7bc2ef9ff7c5caf1
2014-05-09 16:53:19 +02:00
umherirrender
23bb3d1cb4 Follow-Ups to "Fixed some @params documentation"
Fix of inline comments of the following patch sets:
Follow-Up: I0056b4a8df243cfc0c5f25378de48f7a35170aca
Follow-Up: I7f605aa9e117b5fd80d9b1440864fe526d2b14a5
Follow-Up: I3622f216a2ca8ac1b5e51892be9f98665f65bc36
Follow-Up: I6627ba0e76d3577c40bf2473e0f78a5ad7368634
Follow-Up: Id75b5ecf648ca50f955b3bde3307c82c4366b102
Follow-Up: I4ca5231119f33039d91da3b57a41cd40719a576b

Change-Id: Id9bbe84b2820e9db44af5783411e955f55f643d4
2014-04-23 13:39:49 +02:00
umherirrender
957adbef22 Fixed some @params documentation (includes/[file...|upload])
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.

Change-Id: I8804ebe0922d3a414863b162a2110e0b9e49b80f
2014-04-19 17:19:17 +02:00
umherirrender
725d9d125d Removed unneeded spaces and colons in @param and friends
Also swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.

Change-Id: Ic36c8c7820a6c2d603f1138130670c6bf6a1ca59
2014-04-08 16:02:49 +00:00
kaldari
0fb104497c Adding debugging for bug 62241
In the event of a bad filename warning record the relavent variables
so that we can figure out what is causing the failure.

See also change I532f8ee7c7455e43862b0204a7fff23d6792412e

Bug: 62241
Change-Id: If5d8b9e7f34c0f9bd09684d7dada8fafd7f5f7c2
2014-03-31 21:04:43 +00:00
umherirrender
2000672ac3 Fixed spacing
- Added spaces after if/foreach/catch
- Added new line before end of file
- Added or removed spaces before/after parenthesis, comma
- Added spaces around string concat

Change-Id: I0590070f1b3542108e242730e8d9a3ba9831e94f
2014-03-20 20:37:30 +00:00
Brian Wolff
3090ace527 When checking whitelist of extensions, only count last extension.
When we are doing blacklisted extensions, we count all extensions
as some programs (like apache sometimes) consider extensions that
aren't the final extension. However when doing whitelists we need
to only count the last extension, otherwise people can name files
foo.goodExt.BadExt. For example [[commons:File:Deamado ko.png.bmp]]

I do not believe this represents a security risk as bad files are
still filtered out. However it does allow unwanted files to be
uploaded.

Bug: 62451
Change-Id: Ie27c15f749812710571f432bc5915e498f8017e3
2014-03-09 06:12:04 -03:00
csteipp
7d923a6b53 SECURITY: Disallow non-whitelisted namespaces
Disallow uploading non-whitelisted namespaces. Also disallow iframe
elements.

User will get an error including the namespace name if they use a non-
whitelisted namespace.

Bug: 60771
Change-Id: Id5c022543184b19b77ad32d9a8a0c2dbbc5e9038
2014-02-27 15:06:13 +01:00
jenkins-bot
806df0771e Merge "UploadStash us_props fix for Postgres" 2014-02-21 14:40:14 +00:00
Aaron Schulz
0ef8100a25 UploadStash us_props fix for Postgres
* Follows up c7f4586fe1

Change-Id: I5a8568efaf0817ca743de722d5486457ff1a8d87
2014-02-19 13:28:26 -08:00
Alexandre Emsenhuber
8d701eeeea Remove trailing line breaks from wfDebugLog() messages
This is useless since the message is passed through trim() and a
line break is added afterwards.

Change-Id: I1a26b30a07f7c9c749fce5bb6b2b4b3d79901b7c
2014-02-04 22:16:13 +01:00
jenkins-bot
7af7ba2f05 Merge "Add missing uploadstash.us_props for PostgreSQL" 2014-01-29 18:19:48 +00:00
csteipp
b4260e2daa SECURITY: Return error on invalid XML for SVGs
Return an error from UploadBase::detectScriptInSvg when the svg has
XML that cannot be parsed. Usually the XML is invalid, or the parser has
run out of memory trying to parse the file.

This patch is rebased on top of bug 57550.

Bug: 58553
Change-Id: I32661a27d7417cc2c69b844c805b190d6486d17a
2014-01-13 22:15:27 -08:00
mglaser
5205b03f62 SECURITY: Disallow stylesheets in svg
Bug: 57550
Change-Id: I73d148519c077e628d82a89280faa088bac9bdf5
2014-01-14 02:00:12 +01:00
Brad Jorsch
81486e15ee Do not reveal RevDeled file title
When an file is being uploaded, we tell the user if it has been
uploaded and deleted in the past. But we shouldn't reveal the title
under which that was done if the file was suppressed via RevDel.

Note this introduces a breaking API change: clients may now receive a
"deleted-archive" warning with an empty value if the title was
suppressed.

Bug: 59167
Change-Id: I01a3ac2ba0cbac6b28de26a7f1a1d09b6b02097e
2014-01-03 12:40:26 -05:00
Bartosz Dziewoński
c817809a11 Remove reference to deleted file
It didn't even point to the right location…

Follow-up to I70d20bf2.

Change-Id: I7f909873035b9176ba9b7c2bf41b05d9eede3cc4
2014-01-03 13:01:28 +01:00
umherirrender
073abe3e12 No variable assignment on return statement
Split the variable assignment and the return statement in two lines for
better readability.

When there was two return statements in one method the logic was swapped
to have only one return statement.

Change-Id: Id7a01b4a2df96036435f9e1a9be5678dd124b0af
2014-01-02 09:43:35 +00:00
Bryan Davis
f9c6af781c Fix TempFSFile creation
A new TempFSFile should be instantiated via the factory rather than
using the object constructor.

Change-Id: Icd059566087c6a644a0d0ed69abfe13b88b38ee4
2013-12-13 12:49:45 -07:00
Bryan Davis
e4009c7367 Use TempFSFile in UploadFromUrl
Make it a little harder for the developer to leak files in the temp
directory by managing the temporary file with a TempFSFile instance.

Change-Id: I8bafc3a6e6212781d9e2e4f8f370ce7c446f1fd5
2013-12-12 16:10:14 -07:00
saper
c7f4586fe1 Add missing uploadstash.us_props for PostgreSQL
This field was introduced in e5d41cca90
but only for MySQL.

Change-Id: I057b1dd845a5ab7ae05f5597454981a6e2c12698
2013-12-09 20:46:17 +01:00
jenkins-bot
bdc5103819 Merge "Move closing parenthesis from multi line if and function to own line" 2013-12-02 06:33:48 +00:00