wiki.techinc.nl/includes/media/PNGMetadataExtractor.php

420 lines
12 KiB
PHP
Raw Normal View History

<?php
/**
* PNG frame counter and metadata extractor.
*
* Slightly derived from GIFMetadataExtractor.php
* Deliberately not using MWExceptions to avoid external dependencies, encouraging
* redistribution.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
* http://www.gnu.org/copyleft/gpl.html
*
* @file
* @ingroup Media
*/
/**
* PNG frame counter.
*
* @ingroup Media
*/
class PNGMetadataExtractor {
/** @var string */
private static $pngSig;
/** @var int */
private static $crcSize;
/** @var array */
private static $textChunks;
public const VERSION = 1;
private const MAX_CHUNK_SIZE = 3145728; // 3 mebibytes
public static function getMetadata( $filename ) {
self::$pngSig = pack( "C8", 137, 80, 78, 71, 13, 10, 26, 10 );
self::$crcSize = 4;
/* based on list at http://owl.phy.queensu.ca/~phil/exiftool/TagNames/PNG.html#TextualData
* and https://www.w3.org/TR/PNG/#11keywords
*/
self::$textChunks = [
'xml:com.adobe.xmp' => 'xmp',
# Artist is unofficial. Author is the recommended
# keyword in the PNG spec. However some people output
# Artist so support both.
'artist' => 'Artist',
'model' => 'Model',
'make' => 'Make',
'author' => 'Artist',
'comment' => 'PNGFileComment',
'description' => 'ImageDescription',
'title' => 'ObjectName',
'copyright' => 'Copyright',
# Source as in original device used to make image
# not as in who gave you the image
'source' => 'Model',
'software' => 'Software',
'disclaimer' => 'Disclaimer',
'warning' => 'ContentWarning',
'url' => 'Identifier', # Not sure if this is best mapping. Maybe WebStatement.
'label' => 'Label',
'creation time' => 'DateTimeDigitized',
/* Other potentially useful things - Document */
];
$frameCount = 0;
$loopCount = 1;
$text = [];
$duration = 0.0;
Use the unserialized form of image metadata internally Image metadata is usually a serialized string representing an array. Passing the string around internally and having everything unserialize it is an awkward convention. Also, many image handlers were reading the file twice: once for getMetadata() and again for getImageSize(). Often getMetadata() would actually read the width and height and then throw it away. So, in filerepo: * Add File::getMetadataItem(), which promises to allow partial loading of metadata per my proposal on T275268 in a future commit. * Add File::getMetadataArray(), which returns the unserialized array. Some file handlers were returning non-serializable strings from getMetadata(), so I gave them a legacy array form ['_error' => ...] * Changed MWFileProps to return the array form of metadata. * Deprecate the weird File::getImageSize(). It was apparently not called by anything, but was overridden by UnregisteredLocalFile. * Wrap serialize/unserialize with File::getMetadataForDb() and File::loadMetadataFromDb() in preparation for T275268. In MediaHandler: * Merged MediaHandler::getImageSize() and MediaHandler::getMetadata() into getSizeAndMetadata(). Deprecated the old methods. * Instead of isMetadataValid() we now have isFileMetadataValid(), which only gets a File object, so it can decide what data it needs to load. * Simplified getPageDimensions() by having it return false for non-paged media. It was not called in that case, but was implemented anyway. In specific handlers: * Rename DjVuHandler::getUnserializedMetadata() and extractTreesFromMetadata() for clarity. "Metadata" in these function names meant an XML string. * Updated DjVuImage::getImageSize() to provide image sizes in the new style. * In ExifBitmapHandler, getRotationForExif() now takes just the Orientation tag, rather than a serialized string. Also renamed for clarity. * In GIFMetadataExtractor, return the width, height and bits per channel instead of throwing them away. There was some conflation in decodeBPP() which I picked apart. Refer to GIF89a section 18. * In JpegMetadataExtractor, process the SOF0/SOF2 segment to extract bits per channel, width, height and components (channel count). This is essentially a port of PHP's getimagesize(), so should be bugwards compatible. * In PNGMetadataExtractor, return the width and height, which were previously assigned to unused local variables. I verified the implementation by referring to the specification. * In SvgHandler, retain the version validation from unpackMetadata(), but rename the function since it now takes an array as input. In tests: * In ExifBitmapTest, refactored some tests by using a provider. * In GIFHandlerTest and PNGHandlerTest, I removed the tests in which getMetadata() returns null, since it doesn't make sense when ported to getMetadataArray(). I added tests for empty arrays instead. * In tests, I retained serialization of input data since I figure it's useful to confirm that existing database rows will continue to be read correctly. I removed serialization of expected values, replacing them with plain data. * In tests, I replaced access to private class constants like BROKEN_FILE with string literals, since stability is essential. If the class constant changes, the test should fail. Elsewhere: * In maintenance/refreshImageMetadata.php, I removed the check for shrinking image metadata, since it's not easy to implement and is not future compatible. Image metadata is expected to shrink in future. Bug: T275268 Change-Id: I039785d5b6439d71dcc21dcb972177dba5c3a67d
2021-05-19 00:24:32 +00:00
$width = 0;
$height = 0;
$bitDepth = 0;
$colorType = 'unknown';
if ( !$filename ) {
2010-07-28 19:38:59 +00:00
throw new Exception( __METHOD__ . ": No file name specified" );
} elseif ( !file_exists( $filename ) || is_dir( $filename ) ) {
2010-07-28 19:38:59 +00:00
throw new Exception( __METHOD__ . ": File $filename does not exist" );
}
$fh = fopen( $filename, 'rb' );
if ( !$fh ) {
2010-07-28 19:38:59 +00:00
throw new Exception( __METHOD__ . ": Unable to open file $filename" );
}
// Check for the PNG header
$buf = self::read( $fh, 8 );
if ( $buf != self::$pngSig ) {
2010-07-28 19:38:59 +00:00
throw new Exception( __METHOD__ . ": Not a valid PNG file; header: $buf" );
}
// Read chunks
while ( !feof( $fh ) ) {
$buf = self::read( $fh, 4 );
$chunk_size = unpack( "N", $buf )[1];
if ( $chunk_size < 0 ) {
throw new Exception( __METHOD__ . ": Chunk size too big for unpack" );
}
$chunk_type = self::read( $fh, 4 );
$buf = self::read( $fh, $chunk_size );
$crc = self::read( $fh, self::$crcSize );
$computed = crc32( $chunk_type . $buf );
if ( pack( 'N', $computed ) !== $crc ) {
wfDebug( __METHOD__ . ': chunk has invalid CRC, skipping' );
continue;
}
if ( $chunk_type == "IHDR" ) {
$width = unpack( 'N', substr( $buf, 0, 4 ) )[1];
$height = unpack( 'N', substr( $buf, 4, 4 ) )[1];
$bitDepth = ord( substr( $buf, 8, 1 ) );
// Detect the color type in British English as per the spec
// https://www.w3.org/TR/PNG/#11IHDR
switch ( ord( substr( $buf, 9, 1 ) ) ) {
case 0:
$colorType = 'greyscale';
break;
case 2:
$colorType = 'truecolour';
break;
case 3:
$colorType = 'index-coloured';
break;
case 4:
$colorType = 'greyscale-alpha';
break;
case 6:
$colorType = 'truecolour-alpha';
break;
default:
$colorType = 'unknown';
break;
}
} elseif ( $chunk_type == "acTL" ) {
if ( $chunk_size < 4 ) {
wfDebug( __METHOD__ . ": acTL chunk too small" );
continue;
}
$actl = unpack( "Nframes/Nplays", $buf );
$frameCount = $actl['frames'];
$loopCount = $actl['plays'];
} elseif ( $chunk_type == "fcTL" ) {
$buf = substr( $buf, 20 );
if ( strlen( $buf ) < 4 ) {
wfDebug( __METHOD__ . ": fcTL chunk too small" );
continue;
}
$fctldur = unpack( "ndelay_num/ndelay_den", $buf );
if ( $fctldur['delay_den'] == 0 ) {
$fctldur['delay_den'] = 100;
}
if ( $fctldur['delay_num'] ) {
$duration += $fctldur['delay_num'] / $fctldur['delay_den'];
}
} elseif ( $chunk_type == "iTXt" ) {
// Extracts iTXt chunks, uncompressing if necessary.
$items = [];
if ( preg_match(
'/^([^\x00]{1,79})\x00(\x00|\x01)\x00([^\x00]*)(.)[^\x00]*\x00(.*)$/Ds',
$buf, $items )
) {
/* $items[1] = text chunk name, $items[2] = compressed flag,
* $items[3] = lang code (or ""), $items[4]= compression type.
* $items[5] = content
*/
// Theoretically should be case-sensitive, but in practise...
$items[1] = strtolower( $items[1] );
if ( !isset( self::$textChunks[$items[1]] ) ) {
// Only extract textual chunks on our list.
continue;
}
$items[3] = strtolower( $items[3] );
if ( $items[3] == '' ) {
// if no lang specified use x-default like in xmp.
$items[3] = 'x-default';
}
// if compressed
if ( $items[2] == "\x01" ) {
if ( function_exists( 'gzuncompress' ) && $items[4] === "\x00" ) {
Wikimedia\suppressWarnings();
$items[5] = gzuncompress( $items[5] );
Wikimedia\restoreWarnings();
if ( $items[5] === false ) {
// decompression failed
wfDebug( __METHOD__ . ' Error decompressing iTxt chunk - ' . $items[1] );
continue;
}
} else {
wfDebug( __METHOD__ . ' Skipping compressed png iTXt chunk due to lack of zlib,'
. " or potentially invalid compression method" );
continue;
}
}
$finalKeyword = self::$textChunks[$items[1]];
$text[$finalKeyword][$items[3]] = $items[5];
$text[$finalKeyword]['_type'] = 'lang';
} else {
// Error reading iTXt chunk
wfDebug( __METHOD__ . ": Invalid iTXt chunk" );
}
} elseif ( $chunk_type == 'tEXt' ) {
// In case there is no \x00 which will make explode fail.
if ( strpos( $buf, "\x00" ) === false ) {
wfDebug( __METHOD__ . ": Invalid tEXt chunk: no null byte" );
continue;
}
list( $keyword, $content ) = explode( "\x00", $buf, 2 );
if ( $keyword === '' ) {
wfDebug( __METHOD__ . ": Empty tEXt keyword" );
continue;
}
// Theoretically should be case-sensitive, but in practise...
$keyword = strtolower( $keyword );
if ( !isset( self::$textChunks[$keyword] ) ) {
// Don't recognize chunk, so skip.
continue;
}
Wikimedia\suppressWarnings();
$content = iconv( 'ISO-8859-1', 'UTF-8', $content );
Wikimedia\restoreWarnings();
if ( $content === false ) {
wfDebug( __METHOD__ . ": Read error (error with iconv)" );
continue;
}
$finalKeyword = self::$textChunks[$keyword];
$text[$finalKeyword]['x-default'] = $content;
$text[$finalKeyword]['_type'] = 'lang';
} elseif ( $chunk_type == 'zTXt' ) {
if ( function_exists( 'gzuncompress' ) ) {
// In case there is no \x00 which will make explode fail.
if ( strpos( $buf, "\x00" ) === false ) {
wfDebug( __METHOD__ . ": No null byte in zTXt chunk" );
continue;
}
list( $keyword, $postKeyword ) = explode( "\x00", $buf, 2 );
if ( $keyword === '' || $postKeyword === '' ) {
wfDebug( __METHOD__ . ": Empty zTXt chunk" );
continue;
}
// Theoretically should be case-sensitive, but in practise...
$keyword = strtolower( $keyword );
if ( !isset( self::$textChunks[$keyword] ) ) {
// Don't recognize chunk, so skip.
continue;
}
$compression = substr( $postKeyword, 0, 1 );
$content = substr( $postKeyword, 1 );
if ( $compression !== "\x00" ) {
wfDebug( __METHOD__ . " Unrecognized compression method in zTXt ($keyword). Skipping." );
continue;
}
Wikimedia\suppressWarnings();
$content = gzuncompress( $content );
Wikimedia\restoreWarnings();
if ( $content === false ) {
// decompression failed
wfDebug( __METHOD__ . ' Error decompressing zTXt chunk - ' . $keyword );
continue;
}
Wikimedia\suppressWarnings();
$content = iconv( 'ISO-8859-1', 'UTF-8', $content );
Wikimedia\restoreWarnings();
if ( $content === false ) {
wfDebug( __METHOD__ . ": iconv error in zTXt chunk" );
continue;
}
$finalKeyword = self::$textChunks[$keyword];
$text[$finalKeyword]['x-default'] = $content;
$text[$finalKeyword]['_type'] = 'lang';
} else {
wfDebug( __METHOD__ . " Cannot decompress zTXt chunk due to lack of zlib. Skipping." );
}
} elseif ( $chunk_type == 'tIME' ) {
// last mod timestamp.
if ( $chunk_size !== 7 ) {
wfDebug( __METHOD__ . ": tIME wrong size" );
continue;
}
// Note: spec says this should be UTC.
$t = unpack( "ny/Cm/Cd/Ch/Cmin/Cs", $buf );
$strTime = sprintf( "%04d%02d%02d%02d%02d%02d",
$t['y'], $t['m'], $t['d'], $t['h'],
$t['min'], $t['s'] );
$exifTime = wfTimestamp( TS_EXIF, $strTime );
if ( $exifTime ) {
$text['DateTime'] = $exifTime;
}
} elseif ( $chunk_type == 'pHYs' ) {
// how big pixels are (dots per meter).
if ( $chunk_size !== 9 ) {
wfDebug( __METHOD__ . ": pHYs wrong size" );
continue;
}
$dim = unpack( "Nwidth/Nheight/Cunit", $buf );
if ( $dim['unit'] == 1 ) {
// Need to check for negative because php
// doesn't deal with super-large unsigned 32-bit ints well
if ( $dim['width'] > 0 && $dim['height'] > 0 ) {
// unit is meters
// (as opposed to 0 = undefined )
$text['XResolution'] = $dim['width']
. '/100';
$text['YResolution'] = $dim['height']
. '/100';
$text['ResolutionUnit'] = 3;
// 3 = dots per cm (from Exif).
}
}
} elseif ( $chunk_type == "IEND" ) {
break;
}
}
fclose( $fh );
if ( $loopCount > 1 ) {
$duration *= $loopCount;
}
if ( isset( $text['DateTimeDigitized'] ) ) {
// Convert date format from rfc2822 to exif.
foreach ( $text['DateTimeDigitized'] as $name => &$value ) {
if ( $name === '_type' ) {
continue;
}
// @todo FIXME: Currently timezones are ignored.
// possibly should be wfTimestamp's
// responsibility. (at least for numeric TZ)
$formatted = wfTimestamp( TS_EXIF, $value );
if ( $formatted ) {
// Only change if we could convert the
// date.
// The png standard says it should be
// in rfc2822 format, but not required.
// In general for the exif stuff we
// prettify the date if we can, but we
// display as-is if we cannot or if
// it is invalid.
// So do the same here.
$value = $formatted;
}
}
}
return [
Use the unserialized form of image metadata internally Image metadata is usually a serialized string representing an array. Passing the string around internally and having everything unserialize it is an awkward convention. Also, many image handlers were reading the file twice: once for getMetadata() and again for getImageSize(). Often getMetadata() would actually read the width and height and then throw it away. So, in filerepo: * Add File::getMetadataItem(), which promises to allow partial loading of metadata per my proposal on T275268 in a future commit. * Add File::getMetadataArray(), which returns the unserialized array. Some file handlers were returning non-serializable strings from getMetadata(), so I gave them a legacy array form ['_error' => ...] * Changed MWFileProps to return the array form of metadata. * Deprecate the weird File::getImageSize(). It was apparently not called by anything, but was overridden by UnregisteredLocalFile. * Wrap serialize/unserialize with File::getMetadataForDb() and File::loadMetadataFromDb() in preparation for T275268. In MediaHandler: * Merged MediaHandler::getImageSize() and MediaHandler::getMetadata() into getSizeAndMetadata(). Deprecated the old methods. * Instead of isMetadataValid() we now have isFileMetadataValid(), which only gets a File object, so it can decide what data it needs to load. * Simplified getPageDimensions() by having it return false for non-paged media. It was not called in that case, but was implemented anyway. In specific handlers: * Rename DjVuHandler::getUnserializedMetadata() and extractTreesFromMetadata() for clarity. "Metadata" in these function names meant an XML string. * Updated DjVuImage::getImageSize() to provide image sizes in the new style. * In ExifBitmapHandler, getRotationForExif() now takes just the Orientation tag, rather than a serialized string. Also renamed for clarity. * In GIFMetadataExtractor, return the width, height and bits per channel instead of throwing them away. There was some conflation in decodeBPP() which I picked apart. Refer to GIF89a section 18. * In JpegMetadataExtractor, process the SOF0/SOF2 segment to extract bits per channel, width, height and components (channel count). This is essentially a port of PHP's getimagesize(), so should be bugwards compatible. * In PNGMetadataExtractor, return the width and height, which were previously assigned to unused local variables. I verified the implementation by referring to the specification. * In SvgHandler, retain the version validation from unpackMetadata(), but rename the function since it now takes an array as input. In tests: * In ExifBitmapTest, refactored some tests by using a provider. * In GIFHandlerTest and PNGHandlerTest, I removed the tests in which getMetadata() returns null, since it doesn't make sense when ported to getMetadataArray(). I added tests for empty arrays instead. * In tests, I retained serialization of input data since I figure it's useful to confirm that existing database rows will continue to be read correctly. I removed serialization of expected values, replacing them with plain data. * In tests, I replaced access to private class constants like BROKEN_FILE with string literals, since stability is essential. If the class constant changes, the test should fail. Elsewhere: * In maintenance/refreshImageMetadata.php, I removed the check for shrinking image metadata, since it's not easy to implement and is not future compatible. Image metadata is expected to shrink in future. Bug: T275268 Change-Id: I039785d5b6439d71dcc21dcb972177dba5c3a67d
2021-05-19 00:24:32 +00:00
'width' => $width,
'height' => $height,
'frameCount' => $frameCount,
'loopCount' => $loopCount,
'duration' => $duration,
'text' => $text,
'bitDepth' => $bitDepth,
'colorType' => $colorType,
];
}
/**
* Read a chunk, checking to make sure its not too big.
*
* @param resource $fh The file handle
* @param int $size Size in bytes.
* @throws Exception If too big
* @return string The chunk.
*/
private static function read( $fh, $size ) {
if ( $size > self::MAX_CHUNK_SIZE ) {
throw new Exception( __METHOD__ . ': Chunk size of ' . $size .
' too big. Max size is: ' . self::MAX_CHUNK_SIZE );
}
if ( $size === 0 ) {
return '';
}
$result = fread( $fh, $size );
if ( $result === false ) {
throw new Exception( __METHOD__ . ': read error' );
}
if ( strlen( $result ) < $size ) {
throw new Exception( __METHOD__ . ': unexpected end of file' );
}
return $result;
}
}