Commit graph

239 commits

Author SHA1 Message Date
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Bartosz Dziewoński
b40bfd30c5 Document how LocalFile::upload() and UploadBase::performUpload() ignore user permissions
Follow-up to 61c7852049.

Change-Id: I6e69b1fa12f336dc4dd12e845cb8bf42abd599b2
2016-01-26 18:03:56 +01:00
Bartosz Dziewoński
61c7852049 Make it possible to tag new file uploads without messy queries
UploadBase::performUpload() now takes a $tags parameter and passes it
to LocalFile::upload() and LocalFile::recordUpload2(), which
eventually adds the requested tags to the log_id, rev_id and rc_id
that are created for the file upload.

Previously you'd have to query the database for the latest rev_id and
log_id for the page title under which the title is being uploaded, as
performUpload() is unable to return them to you because it's all
deferred in funny ways.

Bug: T121874
Change-Id: I99a8fd67c84219d2715d3d88cc21500614431179
2016-01-25 20:06:53 +01:00
Bartosz Dziewoński
ed6648427a Work around broken HHVM ini_get() for 'upload_max_filesize' and 'post_max_size'
In HHVM, the settings 'upload_max_filesize' and 'post_max_size' are
not available via ini_get() due to some long-standing bug
(https://github.com/facebook/hhvm/issues/4993). Instead, one can use
'hhvm.server.upload.upload_max_file_size' and 'hhvm.server.max_post_size'
(in a typical PHP fashion, their names are subtly different than the
originals as to increase the potential for confusion).

Added a new method UploadBase::getMaxPhpUploadSize() to handle this.

Additionally:
* 'post_max_size' can be set to 0, which is equivalent to no limit.
  Handle this correctly.
* $wgMaxUploadSize can be an array structure, instead of just a number.
  Handle this correctly by using UploadBase::getMaxUploadSize().
* When no maximum is set, use PHP_INT_MAX rather than 1e100. It should
  be big enough, and the latter is a float, results in 0 when cast to
  int, and doesn't look as pretty when formatted in GB in the interface.

Bug: T116347
Change-Id: Idf707253eeae1b90792a7e26d2ab66d1317e67ae
2015-11-08 19:48:23 +00:00
Timo Tijhof
e8275758fe objectcache: Introduce IExpiringStore for convenient TTL constants
Also consistently use self:: instead of BagOStuff:: for constants
referenced within the BagOStuff class.

Change-Id: I20fde9fa5cddcc9e92fa6a02b05dc7effa846742
2015-10-28 04:07:25 +00:00
csteipp
c804391572 SECURITY: Throttle uploads
Add throttle check in ApiUpload and SpecialUpload.

Bug: T91850
Change-Id: If33cc99f304aab2486507c7500b4abb06b6b5d70
2015-10-16 11:23:18 -07:00
umherirrender
977c810302 Remove empty line comments
Remove empty line comments as found by the
MediaWiki.WhiteSpace.SpaceBeforeSingleLineComment.EmptyComment sniff

Change-Id: I5d694f7a7d3bc97e16300ba03c60ad17f3c912a5
2015-10-14 09:46:44 +02:00
Bartosz Dziewoński
f90c0659a9 UploadBase: Return 'was-deleted' warning in addition to 'exists-normalized', not instead of
The 'was-deleted' warning was generated by getExistsWarning(), which
was returning immediately if this was found to be the case. A bunch of
later checks were incorrectly skipped, in particular 'exists-normalized',
which was resulting in UploadWizard incorrectly ignoring that problem.

I'm not sure why that was part of getExistsWarning() at all, it
doesn't seem very relevant. For that matter, neither do the 'thumb',
'thumb-name' and 'bad-prefix' warnings that it also generates, but
this should not be a problem in practice and so I'm leaving them alone.

Other than by allowing some more warning types to appear together or
in different order, this should not affect action=upload API output or
Special:Upload (which was updated appropriately). It does affect
'action=query&prop=imageinfo' output's 'html' property (used for AJAX
checks on Special:Upload), which no longer includes the 'was-deleted'
warning; this was never specified anywhere and just a side-effect.

Bug: T48741
Change-Id: I3686ee8ffd635f5f06f51971b6f16e3e66f33a9e
2015-10-12 14:22:11 +02:00
Amir E. Aharoni
c37d6549fd Fix Generic.Files.LineLength phpcs failure in 11 files under includes/
Bug: T102614
Change-Id: I0d759be6ef568c2c6f28606d3002484ad77a1830
2015-10-03 17:08:26 +00:00
Prateek Saxena
183d45c988 UploadBase: Remove UPLOAD_VERIFICATION_ERROR
It was replaced by HOOK_ABORTED five years ago and isn't being used
anywhere now.

Change-Id: I20feb33c108ae56f25a0cd01da1a326b290106c2
2015-10-01 20:09:28 +05:30
Vivek Ghaisas
c54766586a Fix issues identified by SpaceBeforeSingleLineComment sniff
Change-Id: I048ccb1fa260e4b7152ca5f09b053defdd72d8f9
2015-09-26 23:06:52 +00:00
Aaron Schulz
592a6e7d2e Avoid master queries on image history view
* The path that needs READ_LATEST already calls load() as needed first

Bug: T92357
Change-Id: Ia06bba6c2853823add2e527bb1b013b64d3f020a
2015-09-16 10:41:12 -07:00
Alex Monk
d4a0af097e Avoid passing anything but a string to Title::newFromText in UploadBase::getTitle
Bug: T109974
Change-Id: I54d42bcdcb95cb5b6a09c1c3bdea8891a14a7fdb
2015-08-23 19:27:40 +01:00
Aaron Schulz
3e7d509699 Switched upload chunk status store to the main object stash
Bug: T88493
Change-Id: I91993943ebc39846b78956e71c83358868198c2a
2015-06-30 11:04:33 -07:00
umherirrender
70f3afd548 Remove unneeded empty lines at begin of if/else/foreach body
An if body must not begin with an empty line

Change-Id: I62b058be337fcc85a120fcd3dadce564db59a271
2015-06-19 20:05:45 +02:00
Kunal Mehta
f6e5079a69 Use mediawiki/at-ease library for suppressing warnings
wfSuppressWarnings() and wfRestoreWarnings() were split out into a
separate library. All usages in core were replaced with the new
functions, and the wf* global functions are marked as deprecated.

Additionally, some uses of @ were replaced due to composer's autoloader
being loaded even earlier.

Ie1234f8c12693408de9b94bf6f84480a90bd4f8e adds the library to
mediawiki/vendor.

Bug: T100923
Change-Id: I5c35079a0a656180852be0ae6b1262d40f6534c4
2015-06-11 18:49:29 +00:00
Aaron Schulz
9632223e4c Fixed Job constructor IDE notices about variable types
Change-Id: I4b4e4e38e8d416c3445c52ced311f5fbfcde868a
2015-05-30 08:09:30 +00:00
Ubuntu
bf6e57d32e SVG upload with specific error (warning) message when blocking
This patch is to generate specific error (warning) message when
blocking an svg file.

The checkSvgScriptCallback function has been updated, and it's
return type is changed from boolean to array.

A new variable is added to XmlTypeCheck class that contains the
type of error when svg file is uploaded, which is used to generate
concrete error messages later on.

I have added concrete error messages to i18n/en.json and their description
to qqq.json file. Please review the error messages and their description.

Bug: T85924
Change-Id: I3f687bf5b86ce66b703591b85fd03f073aacff4f
2015-05-25 20:47:45 +00:00
csteipp
b5f491fb6d SECURITY: Fix animate blacklist
The blacklist should prevent animating any element's xlink:href to a
javascript url.

Bug: T86711
Change-Id: Ia9e9192165fdfe1701f22605eee0b0e5c9137d5a
2015-04-01 09:55:22 -07:00
csteipp
bf5f708dc5 SECURITY: Don't allow embedded application/xml in SVG's
Fix for iSEC-WMF1214-11 and issue reported by Cure 53, which got
around our blacklist on embedded href targets. Use a whitelist instead.

Bug: T85850
Change-Id: I17b7ed65935b818695a83fd901fcaf90fffecf28
2015-04-01 09:54:59 -07:00
Tyler Romeo
b813539d6d SECURITY: Make SVG @import checking case insensitive
@import in embedded CSS is case-insensitive, meaning
an attacker can put "@iMpOrT" and it should still
work.

This uses stripos instead of strpos to make the check
case insensitive.

Bug: T85349
Change-Id: I31db9d81f46460af2d8d3f161ba46c2ab7a170d1
2015-04-01 09:54:44 -07:00
Aaron Schulz
5085a4b5cf Made wfFindFile/wfLocalFile callers use explicit "latest" flags
* Callers that should not use caches won't
* Aliased the old "bypassCache" param to "latest"

bug: T89184
Change-Id: I9f79e5942ced4ae13ba4de0b4c62908cc746e777
2015-03-06 04:18:50 +00:00
rillke
46ebee74a6 Detect duplicate archived files by SHA1 search on upload
- As of now, we detect duplicate files by a combination of sha1
  and file extension.
- Since multiple file extensions for the same MIME are permitted
  this method does not reliably work.

Bug: T74070
Change-Id: If13059441097799227f23ece36a96c8375f17aab
2015-02-18 19:52:03 +00:00
Ricordisamoa
2ae155da52 Fix phpcs errors in includes/
Mostly Squiz.WhiteSpace.SuperfluousWhitespace.EmptyLines

Change-Id: I678b2f0902f11cd1dfa1611b9da24e7237df9122
2015-01-08 20:15:07 +01:00
Chad Horohoe
aa21e125a3 Remove obvious function-level profiling
Xhprof generates this data now. Custom profiling of various
sub-function units are kept.

Calls to profiler represented about 3% of page execution
time on Special:BlankPage (1.5% in/out); after this change
it's down to about 0.98% of page execution time.

Change-Id: Id9a1dc9d8f80bbd52e42226b724a1e1213d07af7
2015-01-07 11:14:24 -08:00
Aaron Schulz
e369f66d00 Replace wfRunHooks calls with direct Hooks::run calls
* This avoids the overhead of an extra function call

Change-Id: I8ee996f237fd111873ab51965bded3d91e61e4dd
2014-12-10 12:26:59 -08:00
jenkins-bot
cad7a1125d Merge "Don't try to verify XML well-formedness for partial SVG uploads" 2014-11-06 18:50:29 +00:00
jenkins-bot
cc8cfcde3d Merge "Avoid attempting to prerender thumbnails that will fail" 2014-10-03 17:45:15 +00:00
Gilles Dubuc
0ce5bafb4b Avoid attempting to prerender thumbnails that will fail
For non-vectorial content, requesting a thumbnail larger than
the original results in a 500. Prerendering in its current form
introduces an increase in 500s that dilutes the real problematic
500s, making troubleshooting harder than it needs to be.

Change-Id: I9418dee7653ad7954c3788ecdd350fc8772edd32
Mingle: https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/301
2014-10-03 18:09:07 +02:00
Brad Jorsch
d1bbbe61c4 Don't try to verify XML well-formedness for partial SVG uploads
Chunked uploads of SVGs are currently failing with "invalid XML" errors
because UploadBase::detectScriptInSvg() requires the full file but is
being called from UploadBase::verifyPartialFile().

So let's do the check twice: once in UploadBase::verifyPartialFile()
where it will pass if non-well-formed, and once in
UploadBase::verifyFile() where it will fail if non-well-formed.

Bug: 65724
Change-Id: I6126e185eb4b183c31946f13c576521f1ed19c16
2014-10-03 11:49:45 -04:00
jenkins-bot
31f73c4213 Merge "Made upload jobs avoid using the user session" 2014-10-03 15:41:11 +00:00
Gilles Dubuc
478332cfbe Fix version comments in thumbnail prerendering code
Change-Id: I74b52d2ecc382dfc0b55615ce512e8c64d02e9cb
Mingle: https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/301
2014-09-30 14:20:11 +02:00
Aaron Schulz
c7844017c0 Made upload jobs avoid using the user session
* This causes problems with some session handlers and it is
  also trickier to deal with in non CLI script without leaking
  cookie headers.

Change-Id: Iaf2a57f9299e42a5f68bf85115e62e88fa0f8ed6
2014-09-29 16:35:59 -07:00
umherirrender
df24b7209d Fixed spacing
- Added newline at end of file
- Removed double spaces/newlines
- Added space after if/function and parentheses/brackets
- Removed space before comma/cast
- Fixed indent of some lines

Change-Id: I29867ffdffdfb7d2b56997e9393497c7dc12f7d3
2014-09-29 20:46:19 +02:00
jenkins-bot
e6e6861881 Merge "Add ability to pre-render thumbnails at upload time" 2014-09-25 15:40:07 +00:00
csteipp
6aa3befeb0 SECURITY: Enhance CSS filtering in SVG files
* Filter <style> elements
* Normalize style elements and attributes before filtering
* Add checks for attributes that contain css
* Add unit tests for html5sec and reported bugs

Bug:69008
Change-Id: I732eece710f1bfaaeea1e5de541fcd4cfb375de7
2014-09-24 21:06:54 +02:00
Gilles Dubuc
623d778902 Add ability to pre-render thumbnails at upload time
This introduces an option that allows one to define a list of
thumbnail sizes to be rendered by async jobs at upload time.

Change-Id: Ida193699738c79aca333fa78b8b536d14a410841
Mingle: https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/301
2014-09-12 09:20:41 +02:00
jenkins-bot
496e14b0b2 Merge "Correct variable names in @param to match method declarations" 2014-08-13 20:13:37 +00:00
Yuri Astrakhan
703464a88c Cleanup - let's make IDEs more useful
http://phpdoc.org/docs/latest/references/phpdoc/types.html

If IDEs have many warnings, we don't look at them.
Let's minimize the number of warnings, and make them useful again.

* Some function docs fixes
* Removed unused $iwprefixes var in ApiQuerySearch.php
* declared private $blockStatusByUid in SpecialActiveusers
* declared private $repo in UploadFromChunks

Change-Id: Ifd20f78b168b9a913fdb8d89dc26a76a173b1c29
2014-08-13 16:02:59 -04:00
umherirrender
21e0c1c533 Correct variable names in @param to match method declarations
Some @param have a typo in the variable name,
some @param's were in wrong order.

Change-Id: Ie25806831027112b398f6f4a909c59147ac3a5fa
2014-08-13 21:48:28 +02:00
rillke
c31fbf073e Unify the spelling of MIME in documentation
Writing MIME as written in Wikipedia and some documentation clean up.

Change-Id: I9dfc36d2bf55d72d9374c4075bd6d45eef0415a4
2014-08-07 23:38:45 +02:00
Matthew Flaschen
731a7a8ebd Clarify comment in UploadBase
Change-Id: Ied29298d312ca19ecfdd2196fafa89130321987c
2014-07-28 22:48:42 -04:00
Antoine Musso
dcdf45010d Remove some duplicate newlines in call to wfDebug()
Spotted on beta:

--------
commonswiki-375c0c3e:

0.0752   1.9M  mime: <image/png> extension: <png>

--------

With lot of requests, that causes them to be interlayed and ungreppable.

Change-Id: Ib99bc9547bdecb9efa59b7d292ddd7d10a552652
2014-07-27 14:31:26 +00:00
umherirrender
768ac15c70 Cleanup some docs (includes/[s-z])
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling

Change-Id: Ie419638e909a47aa72a274043604247830ee1a81
2014-07-24 19:43:44 +02:00
jenkins-bot
79d908ef74 Merge "Fix mime detection of easily-confused-with text/plain formats" 2014-07-04 07:43:42 +00:00
Brian Wolff
69905ce9c7 Fix mime detection of easily-confused-with text/plain formats
json, csv, and tsv are often detected as text/plain. However that's
not right. This patch causes MediaWiki to look at the file extension
of files detected as text/plain, and if the file extension is
for a "textual" type, use the mime type associated with that extension.

This change also changes the "does mime type match uploaded file
extension" check to use the mime based on the file contents
plus extension, as opposed to just the file contents. Various
documentation suggests this is more appropriate (e.g. line 807
of MimeMagic.php). In my opinion we should use just the file
contents when verifying file is not on blacklist, but use ext
when verifying file type matches extension, and for decided
what handler specific checks to run. Not the detect mime type
with extension doesn't override the detected mime type with
the extension, but only uses the extension if content based
detection is ambigious or not specific enough.

This patch should be reviewed by csteipp before merge for
any potential security implications.

Note: This is partially fixing a regression from 3846d10487,
where previously csv and json files were allowed to be uploaded,
and that change prevented them

Bug: 66036
Bug: 45424
Change-Id: Ib637fe6850a81b26f84dc8c00ab4772f3d3a1f34
2014-07-04 04:03:31 -03:00
umherirrender
fa5c5f2837 Fix return doc for UploadBase::isAllowed
Method can return bool or string

Change-Id: Ib290843ce97926bfc529d510b1898f35c7e3f6c6
2014-06-29 14:49:24 +02:00
csteipp
7114801e78 SECURITY: Prevent external resources in SVG files
On bug 65724, it was discovered that a user could upload SVG images
with embedded <image> elements that pulled in the resource via http.
This could allow an attacker to track all viewers of an SVG by having
the image embed another image hosted on their own server.

While testing the patch, I also identified 3 more element namespaces
that have been used on commons and seem harmless, so I added those to
the whitelist.

Change-Id: Iaaabc3a60c0ec4e6e426a8680d7a2cef5d469d29
2014-06-25 22:25:45 +02:00
csteipp
02e55fa26b Add some SVG element namespaces
Noticed that we have several old images on commons that use these
namespaces. They seem harmless, so I think we should add them.

Change-Id: Ib80298256c5d4717867eb857d5404a9487fed784
2014-06-02 12:18:48 -07:00
jenkins-bot
7fa77c2d52 Merge "Added unicode encoding support flags to FileBackend" 2014-05-15 15:06:19 +00:00