Fix mime detection of easily-confused-with text/plain formats
json, csv, and tsv are often detected as text/plain. However that's
not right. This patch causes MediaWiki to look at the file extension
of files detected as text/plain, and if the file extension is
for a "textual" type, use the mime type associated with that extension.
This change also changes the "does mime type match uploaded file
extension" check to use the mime based on the file contents
plus extension, as opposed to just the file contents. Various
documentation suggests this is more appropriate (e.g. line 807
of MimeMagic.php). In my opinion we should use just the file
contents when verifying file is not on blacklist, but use ext
when verifying file type matches extension, and for decided
what handler specific checks to run. Not the detect mime type
with extension doesn't override the detected mime type with
the extension, but only uses the extension if content based
detection is ambigious or not specific enough.
This patch should be reviewed by csteipp before merge for
any potential security implications.
Note: This is partially fixing a regression from 3846d1048766a7,
where previously csv and json files were allowed to be uploaded,
and that change prevented them
Bug: 66036
Bug: 45424
Change-Id: Ib637fe6850a81b26f84dc8c00ab4772f3d3a1f34
2014-06-24 19:15:32 +00:00
|
|
|
<?php
|
2015-06-09 17:02:48 +00:00
|
|
|
class MimeMagicTest extends PHPUnit_Framework_TestCase {
|
2016-09-22 04:57:13 +00:00
|
|
|
/** @var MimeAnalyzer */
|
|
|
|
|
private $mimeAnalyzer;
|
Fix mime detection of easily-confused-with text/plain formats
json, csv, and tsv are often detected as text/plain. However that's
not right. This patch causes MediaWiki to look at the file extension
of files detected as text/plain, and if the file extension is
for a "textual" type, use the mime type associated with that extension.
This change also changes the "does mime type match uploaded file
extension" check to use the mime based on the file contents
plus extension, as opposed to just the file contents. Various
documentation suggests this is more appropriate (e.g. line 807
of MimeMagic.php). In my opinion we should use just the file
contents when verifying file is not on blacklist, but use ext
when verifying file type matches extension, and for decided
what handler specific checks to run. Not the detect mime type
with extension doesn't override the detected mime type with
the extension, but only uses the extension if content based
detection is ambigious or not specific enough.
This patch should be reviewed by csteipp before merge for
any potential security implications.
Note: This is partially fixing a regression from 3846d1048766a7,
where previously csv and json files were allowed to be uploaded,
and that change prevented them
Bug: 66036
Bug: 45424
Change-Id: Ib637fe6850a81b26f84dc8c00ab4772f3d3a1f34
2014-06-24 19:15:32 +00:00
|
|
|
|
|
|
|
|
function setUp() {
|
2016-09-22 04:57:13 +00:00
|
|
|
global $IP;
|
|
|
|
|
|
|
|
|
|
$this->mimeAnalyzer = new MimeAnalyzer( [
|
|
|
|
|
'infoFile' => $IP . "/includes/libs/mime/mime.info",
|
|
|
|
|
'typeFile' => $IP . "/includes/libs/mime/mime.types",
|
|
|
|
|
'xmlTypes' => [
|
|
|
|
|
'http://www.w3.org/2000/svg:svg' => 'image/svg+xml',
|
|
|
|
|
'svg' => 'image/svg+xml',
|
|
|
|
|
'http://www.lysator.liu.se/~alla/dia/:diagram' => 'application/x-dia-diagram',
|
|
|
|
|
'http://www.w3.org/1999/xhtml:html' => 'text/html', // application/xhtml+xml?
|
|
|
|
|
'html' => 'text/html', // application/xhtml+xml?
|
|
|
|
|
]
|
|
|
|
|
] );
|
Fix mime detection of easily-confused-with text/plain formats
json, csv, and tsv are often detected as text/plain. However that's
not right. This patch causes MediaWiki to look at the file extension
of files detected as text/plain, and if the file extension is
for a "textual" type, use the mime type associated with that extension.
This change also changes the "does mime type match uploaded file
extension" check to use the mime based on the file contents
plus extension, as opposed to just the file contents. Various
documentation suggests this is more appropriate (e.g. line 807
of MimeMagic.php). In my opinion we should use just the file
contents when verifying file is not on blacklist, but use ext
when verifying file type matches extension, and for decided
what handler specific checks to run. Not the detect mime type
with extension doesn't override the detected mime type with
the extension, but only uses the extension if content based
detection is ambigious or not specific enough.
This patch should be reviewed by csteipp before merge for
any potential security implications.
Note: This is partially fixing a regression from 3846d1048766a7,
where previously csv and json files were allowed to be uploaded,
and that change prevented them
Bug: 66036
Bug: 45424
Change-Id: Ib637fe6850a81b26f84dc8c00ab4772f3d3a1f34
2014-06-24 19:15:32 +00:00
|
|
|
parent::setUp();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* @dataProvider providerImproveTypeFromExtension
|
2014-07-24 12:55:43 +00:00
|
|
|
* @param string $ext File extension (no leading dot)
|
|
|
|
|
* @param string $oldMime Initially detected MIME
|
|
|
|
|
* @param string $expectedMime MIME type after taking extension into account
|
Fix mime detection of easily-confused-with text/plain formats
json, csv, and tsv are often detected as text/plain. However that's
not right. This patch causes MediaWiki to look at the file extension
of files detected as text/plain, and if the file extension is
for a "textual" type, use the mime type associated with that extension.
This change also changes the "does mime type match uploaded file
extension" check to use the mime based on the file contents
plus extension, as opposed to just the file contents. Various
documentation suggests this is more appropriate (e.g. line 807
of MimeMagic.php). In my opinion we should use just the file
contents when verifying file is not on blacklist, but use ext
when verifying file type matches extension, and for decided
what handler specific checks to run. Not the detect mime type
with extension doesn't override the detected mime type with
the extension, but only uses the extension if content based
detection is ambigious or not specific enough.
This patch should be reviewed by csteipp before merge for
any potential security implications.
Note: This is partially fixing a regression from 3846d1048766a7,
where previously csv and json files were allowed to be uploaded,
and that change prevented them
Bug: 66036
Bug: 45424
Change-Id: Ib637fe6850a81b26f84dc8c00ab4772f3d3a1f34
2014-06-24 19:15:32 +00:00
|
|
|
*/
|
|
|
|
|
function testImproveTypeFromExtension( $ext, $oldMime, $expectedMime ) {
|
2016-09-22 04:57:13 +00:00
|
|
|
$actualMime = $this->mimeAnalyzer->improveTypeFromExtension( $oldMime, $ext );
|
Fix mime detection of easily-confused-with text/plain formats
json, csv, and tsv are often detected as text/plain. However that's
not right. This patch causes MediaWiki to look at the file extension
of files detected as text/plain, and if the file extension is
for a "textual" type, use the mime type associated with that extension.
This change also changes the "does mime type match uploaded file
extension" check to use the mime based on the file contents
plus extension, as opposed to just the file contents. Various
documentation suggests this is more appropriate (e.g. line 807
of MimeMagic.php). In my opinion we should use just the file
contents when verifying file is not on blacklist, but use ext
when verifying file type matches extension, and for decided
what handler specific checks to run. Not the detect mime type
with extension doesn't override the detected mime type with
the extension, but only uses the extension if content based
detection is ambigious or not specific enough.
This patch should be reviewed by csteipp before merge for
any potential security implications.
Note: This is partially fixing a regression from 3846d1048766a7,
where previously csv and json files were allowed to be uploaded,
and that change prevented them
Bug: 66036
Bug: 45424
Change-Id: Ib637fe6850a81b26f84dc8c00ab4772f3d3a1f34
2014-06-24 19:15:32 +00:00
|
|
|
$this->assertEquals( $expectedMime, $actualMime );
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
function providerImproveTypeFromExtension() {
|
2016-02-17 09:09:32 +00:00
|
|
|
return [
|
|
|
|
|
[ 'gif', 'image/gif', 'image/gif' ],
|
|
|
|
|
[ 'gif', 'unknown/unknown', 'unknown/unknown' ],
|
|
|
|
|
[ 'wrl', 'unknown/unknown', 'model/vrml' ],
|
|
|
|
|
[ 'txt', 'text/plain', 'text/plain' ],
|
|
|
|
|
[ 'csv', 'text/plain', 'text/csv' ],
|
|
|
|
|
[ 'tsv', 'text/plain', 'text/tab-separated-values' ],
|
|
|
|
|
[ 'js', 'text/javascript', 'application/javascript' ],
|
|
|
|
|
[ 'js', 'application/x-javascript', 'application/javascript' ],
|
|
|
|
|
[ 'json', 'text/plain', 'application/json' ],
|
|
|
|
|
[ 'foo', 'application/x-opc+zip', 'application/zip' ],
|
|
|
|
|
[ 'docx', 'application/x-opc+zip',
|
|
|
|
|
'application/vnd.openxmlformats-officedocument.wordprocessingml.document' ],
|
|
|
|
|
[ 'djvu', 'image/x-djvu', 'image/vnd.djvu' ],
|
|
|
|
|
[ 'wav', 'audio/wav', 'audio/wav' ],
|
|
|
|
|
];
|
Fix mime detection of easily-confused-with text/plain formats
json, csv, and tsv are often detected as text/plain. However that's
not right. This patch causes MediaWiki to look at the file extension
of files detected as text/plain, and if the file extension is
for a "textual" type, use the mime type associated with that extension.
This change also changes the "does mime type match uploaded file
extension" check to use the mime based on the file contents
plus extension, as opposed to just the file contents. Various
documentation suggests this is more appropriate (e.g. line 807
of MimeMagic.php). In my opinion we should use just the file
contents when verifying file is not on blacklist, but use ext
when verifying file type matches extension, and for decided
what handler specific checks to run. Not the detect mime type
with extension doesn't override the detected mime type with
the extension, but only uses the extension if content based
detection is ambigious or not specific enough.
This patch should be reviewed by csteipp before merge for
any potential security implications.
Note: This is partially fixing a regression from 3846d1048766a7,
where previously csv and json files were allowed to be uploaded,
and that change prevented them
Bug: 66036
Bug: 45424
Change-Id: Ib637fe6850a81b26f84dc8c00ab4772f3d3a1f34
2014-06-24 19:15:32 +00:00
|
|
|
}
|
|
|
|
|
|
2014-06-29 23:46:40 +00:00
|
|
|
/**
|
|
|
|
|
* Test to make sure that encoder=ffmpeg2theora doesn't trigger
|
|
|
|
|
* MEDIATYPE_VIDEO (bug 63584)
|
|
|
|
|
*/
|
|
|
|
|
function testOggRecognize() {
|
2016-09-22 04:57:13 +00:00
|
|
|
$oggFile = __DIR__ . '/../../../data/media/say-test.ogg';
|
|
|
|
|
$actualType = $this->mimeAnalyzer->getMediaType( $oggFile, 'application/ogg' );
|
2014-06-29 23:46:40 +00:00
|
|
|
$this->assertEquals( $actualType, MEDIATYPE_AUDIO );
|
|
|
|
|
}
|
2017-01-17 21:23:02 +00:00
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Test to make sure that Opus audio files don't trigger
|
|
|
|
|
* MEDIATYPE_MULTIMEDIA (bug T151352)
|
|
|
|
|
*/
|
|
|
|
|
function testOpusRecognize() {
|
|
|
|
|
$oggFile = __DIR__ . '/../../../data/media/say-test.opus';
|
|
|
|
|
$actualType = $this->mimeAnalyzer->getMediaType( $oggFile, 'application/ogg' );
|
|
|
|
|
$this->assertEquals( $actualType, MEDIATYPE_AUDIO );
|
|
|
|
|
}
|
Fix mime detection of easily-confused-with text/plain formats
json, csv, and tsv are often detected as text/plain. However that's
not right. This patch causes MediaWiki to look at the file extension
of files detected as text/plain, and if the file extension is
for a "textual" type, use the mime type associated with that extension.
This change also changes the "does mime type match uploaded file
extension" check to use the mime based on the file contents
plus extension, as opposed to just the file contents. Various
documentation suggests this is more appropriate (e.g. line 807
of MimeMagic.php). In my opinion we should use just the file
contents when verifying file is not on blacklist, but use ext
when verifying file type matches extension, and for decided
what handler specific checks to run. Not the detect mime type
with extension doesn't override the detected mime type with
the extension, but only uses the extension if content based
detection is ambigious or not specific enough.
This patch should be reviewed by csteipp before merge for
any potential security implications.
Note: This is partially fixing a regression from 3846d1048766a7,
where previously csv and json files were allowed to be uploaded,
and that change prevented them
Bug: 66036
Bug: 45424
Change-Id: Ib637fe6850a81b26f84dc8c00ab4772f3d3a1f34
2014-06-24 19:15:32 +00:00
|
|
|
}
|
