Use the unserialized form of image metadata internally

Image metadata is usually a serialized string representing an array.
Passing the string around internally and having everything unserialize
it is an awkward convention.

Also, many image handlers were reading the file twice: once for
getMetadata() and again for getImageSize(). Often getMetadata()
would actually read the width and height and then throw it away.

So, in filerepo:

* Add File::getMetadataItem(), which promises to allow partial
  loading of metadata per my proposal on T275268 in a future commit.
* Add File::getMetadataArray(), which returns the unserialized array.
  Some file handlers were returning non-serializable strings from
  getMetadata(), so I gave them a legacy array form ['_error' => ...]
* Changed MWFileProps to return the array form of metadata.
* Deprecate the weird File::getImageSize(). It was apparently not
  called by anything, but was overridden by UnregisteredLocalFile.
* Wrap serialize/unserialize with File::getMetadataForDb() and
  File::loadMetadataFromDb() in preparation for T275268.

In MediaHandler:

* Merged MediaHandler::getImageSize() and MediaHandler::getMetadata()
  into getSizeAndMetadata(). Deprecated the old methods.
* Instead of isMetadataValid() we now have isFileMetadataValid(), which
  only gets a File object, so it can decide what data it needs to load.
* Simplified getPageDimensions() by having it return false for non-paged
  media. It was not called in that case, but was implemented anyway.

In specific handlers:

* Rename DjVuHandler::getUnserializedMetadata() and
  extractTreesFromMetadata() for clarity. "Metadata" in these function
  names meant an XML string.
* Updated DjVuImage::getImageSize() to provide image sizes in the new
  style.
* In ExifBitmapHandler, getRotationForExif() now takes just the
  Orientation tag, rather than a serialized string. Also renamed for
  clarity.
* In GIFMetadataExtractor, return the width, height and bits per channel
  instead of throwing them away. There was some conflation in
  decodeBPP() which I picked apart. Refer to GIF89a section 18.
* In JpegMetadataExtractor, process the SOF0/SOF2 segment to extract
  bits per channel, width, height and components (channel count). This
  is essentially a port of PHP's getimagesize(), so should be bugwards
  compatible.
* In PNGMetadataExtractor, return the width and height, which were
  previously assigned to unused local variables. I verified the
  implementation by referring to the specification.
* In SvgHandler, retain the version validation from unpackMetadata(),
  but rename the function since it now takes an array as input.

In tests:

* In ExifBitmapTest, refactored some tests by using a provider.
* In GIFHandlerTest and PNGHandlerTest, I removed the tests in which
  getMetadata() returns null, since it doesn't make sense when ported to
  getMetadataArray(). I added tests for empty arrays instead.
* In tests, I retained serialization of input data since I figure it's
  useful to confirm that existing database rows will continue to be read
  correctly. I removed serialization of expected values, replacing them
  with plain data.
* In tests, I replaced access to private class constants like
  BROKEN_FILE with string literals, since stability is essential. If
  the class constant changes, the test should fail.

Elsewhere:

* In maintenance/refreshImageMetadata.php, I removed the check for
  shrinking image metadata, since it's not easy to implement and is
  not future compatible. Image metadata is expected to shrink in
  future.

Bug: T275268
Change-Id: I039785d5b6439d71dcc21dcb972177dba5c3a67d
This commit is contained in:
Tim Starling 2021-05-19 10:24:32 +10:00
parent f3dfcd73c7
commit b4849e03b7
46 changed files with 1130 additions and 819 deletions

View file

@ -265,6 +265,7 @@ because of Phabricator reports.
* The PatchFileLocation trait was removed without deprecation.
* ActorMigrationBase::getExistingActorId and ::getNewActorId, hard deprecated
since 1.36, were removed.
* The protected property LocalFile::$metadata was removed without deprecation.
* …
=== Deprecations in 1.37 ===
@ -388,6 +389,13 @@ because of Phabricator reports.
* JobSpecification::getTitle was deprecated without providing a replacement.
It wasn't used and job given the purpose of JobSpecification class it's
not needed.
* The protected method File::getImageSize() is deprecated.
* MediaHandler::getImageSize(), ::getMetadata() and ::isMetadataValid were
deprecated and should no longer be overridden. Instead, subclasses should
override getSizeAndMetadata().
* Deprecated File::getMetadata(). Instead use ::getMetadataArray(),
::getMetadataItem() and ::getMetadataItems().
* …
=== Other changes in 1.37 ===

View file

@ -559,9 +559,7 @@ class ApiQueryImageInfo extends ApiQueryBase {
}
if ( $meta && $exists ) {
Wikimedia\suppressWarnings();
$metadata = unserialize( $file->getMetadata() );
Wikimedia\restoreWarnings();
$metadata = $file->getMetadataArray();
if ( $metadata && $version !== 'latest' ) {
$metadata = $file->convertMetadataVersion( $metadata, $version );
}

View file

@ -11,7 +11,6 @@ use MediaWiki\MediaWikiServices;
use MediaWiki\Page\PageIdentity;
use MediaWiki\Permissions\Authority;
use MediaWiki\User\UserIdentity;
use Wikimedia\AtEase\AtEase;
/**
* Base code for files.
@ -741,7 +740,7 @@ abstract class File implements IDBAccessObject, MediaHandlerState {
* Get handler-specific metadata
* Overridden by LocalFile, UnregisteredLocalFile
* STUB
* @stable to override
* @deprecated since 1.37 use getMetadataArray() or getMetadataItem()
* @return string|false
*/
public function getMetadata() {
@ -756,6 +755,41 @@ abstract class File implements IDBAccessObject, MediaHandlerState {
$this->handlerState[$key] = $value;
}
/**
* Get the unserialized handler-specific metadata
* STUB
* @since 1.37
* @return array
*/
public function getMetadataArray(): array {
return [];
}
/**
* Get a specific element of the unserialized handler-specific metadata.
*
* @since 1.37
* @param string $itemName
* @return mixed
*/
public function getMetadataItem( string $itemName ) {
$items = $this->getMetadataItems( [ $itemName ] );
return $items[$itemName] ?? null;
}
/**
* Get multiple elements of the unserialized handler-specific metadata.
*
* @since 1.37
* @param string[] $itemNames
* @return array
*/
public function getMetadataItems( array $itemNames ): array {
return array_intersect_key(
$this->getMetadataArray(),
array_fill_keys( $itemNames, true ) );
}
/**
* Like getMetadata but returns a handler independent array of common values.
* @see MediaHandler::getCommonMetaArray()
@ -775,17 +809,12 @@ abstract class File implements IDBAccessObject, MediaHandlerState {
/**
* get versioned metadata
*
* @param array|string $metadata Array or string of (serialized) metadata
* @param array $metadata Array of unserialized metadata
* @param int $version Version number.
* @return array Array containing metadata, or what was passed to it on fail
* (unserializing if not array)
*/
public function convertMetadataVersion( $metadata, $version ) {
$handler = $this->getHandler();
if ( !is_array( $metadata ) ) {
// Just to make the return type consistent
$metadata = unserialize( $metadata );
}
if ( $handler ) {
return $handler->convertMetadataVersion( $metadata, $version );
} else {
@ -2161,11 +2190,13 @@ abstract class File implements IDBAccessObject, MediaHandlerState {
* @note Use getWidth()/getHeight() instead of this method unless you have a
* a good reason. This method skips all caches.
*
* @stable to override
* @deprecated since 1.37
*
* @param string $filePath The path to the file (e.g. From getLocalRefPath() )
* @return array|false The width, followed by height, with optionally more things after
*/
protected function getImageSize( $filePath ) {
wfDeprecated( __METHOD__, '1.37' );
if ( !$this->getHandler() ) {
return false;
}
@ -2342,17 +2373,7 @@ abstract class File implements IDBAccessObject, MediaHandlerState {
public function getContentHeaders() {
$handler = $this->getHandler();
if ( $handler ) {
$metadata = $this->getMetadata();
if ( is_string( $metadata ) ) {
$metadata = AtEase::quietCall( 'unserialize', $metadata );
}
if ( !is_array( $metadata ) ) {
$metadata = [];
}
return $handler->getContentHeaders( $metadata );
return $handler->getContentHeaders( $this->getMetadataArray() );
}
return [];

View file

@ -187,6 +187,17 @@ class ForeignAPIFile extends File {
return false;
}
/**
* @return array
*/
public function getMetadataArray(): array {
if ( isset( $this->mInfo['metadata'] ) ) {
return self::parseMetadata( $this->mInfo['metadata'] );
}
return [];
}
/**
* @return array|null Extended metadata (see imageinfo API for format) or
* null on error
@ -197,11 +208,11 @@ class ForeignAPIFile extends File {
/**
* @param mixed $metadata
* @return mixed
* @return array
*/
public static function parseMetadata( $metadata ) {
if ( !is_array( $metadata ) ) {
return $metadata;
return [ '_error' => $metadata ];
}
'@phan-var array[] $metadata';
$ret = [];

View file

@ -28,7 +28,7 @@ use MediaWiki\Revision\RevisionRecord;
use MediaWiki\Revision\RevisionStore;
use MediaWiki\User\UserIdentity;
use MediaWiki\User\UserIdentityValue;
use Wikimedia\AtEase\AtEase;
use Wikimedia\Rdbms\Blob;
use Wikimedia\Rdbms\Database;
use Wikimedia\Rdbms\IDatabase;
use Wikimedia\Rdbms\IResultWrapper;
@ -60,7 +60,7 @@ use Wikimedia\Rdbms\IResultWrapper;
* @ingroup FileAbstraction
*/
class LocalFile extends File {
private const VERSION = 12; // cache version
private const VERSION = 13; // cache version
private const CACHE_FIELD_MAX_LEN = 1000;
@ -85,8 +85,8 @@ class LocalFile extends File {
/** @var int Size in bytes (loadFromXxx) */
protected $size;
/** @var string Handler-specific metadata */
protected $metadata;
/** @var array Unserialized metadata */
protected $metadataArray = [];
/** @var string SHA-1 base 36 content hash */
protected $sha1;
@ -283,7 +283,6 @@ class LocalFile extends File {
public function __construct( $title, $repo ) {
parent::__construct( $title, $repo );
$this->metadata = '';
$this->historyLine = 0;
$this->historyRes = null;
$this->dataLoaded = false;
@ -354,13 +353,14 @@ class LocalFile extends File {
$cacheVal['user'] = $this->user->getId();
$cacheVal['user_text'] = $this->user->getName();
}
$cacheVal['metadata'] = $this->metadataArray;
// Strip off excessive entries from the subset of fields that can become large.
// If the cache value gets to large it will not fit in memcached and nothing will
// get cached at all, causing master queries for any file access.
foreach ( $this->getLazyCacheFields( '' ) as $field ) {
if ( isset( $cacheVal[$field] )
&& strlen( $cacheVal[$field] ) > 100 * 1024
&& strlen( serialize( $cacheVal[$field] ) ) > 100 * 1024
) {
unset( $cacheVal[$field] ); // don't let the value get too big
}
@ -408,9 +408,13 @@ class LocalFile extends File {
/**
* Load metadata from the file itself
*
* @internal
* @param string|null $path The path or virtual URL to load from, or null
* to use the previously stored file.
*/
protected function loadFromFile() {
$props = $this->repo->getFileProps( $this->getVirtualUrl() );
public function loadFromFile( $path = null ) {
$props = $this->repo->getFileProps( $path ?? $this->getVirtualUrl() );
$this->setProps( $props );
}
@ -433,7 +437,7 @@ class LocalFile extends File {
// and self::loadFromCache() for the caching, and self::setProps() for
// populating the object from an array of data.
return [ 'size', 'width', 'height', 'bits', 'media_type',
'major_mime', 'minor_mime', 'metadata', 'timestamp', 'sha1', 'description' ];
'major_mime', 'minor_mime', 'timestamp', 'sha1', 'description' ];
}
/**
@ -502,14 +506,16 @@ class LocalFile extends File {
# Unconditionally set loaded=true, we don't want the accessors constantly rechecking
$this->extraDataLoaded = true;
$fieldMap = $this->loadExtraFieldsWithTimestamp( $this->repo->getReplicaDB(), $fname );
$db = $this->repo->getReplicaDB();
$fieldMap = $this->loadExtraFieldsWithTimestamp( $db, $fname );
if ( !$fieldMap ) {
$fieldMap = $this->loadExtraFieldsWithTimestamp( $this->repo->getMasterDB(), $fname );
$db = $this->repo->getMasterDB();
$fieldMap = $this->loadExtraFieldsWithTimestamp( $db, $fname );
}
if ( $fieldMap ) {
if ( isset( $fieldMap['metadata'] ) ) {
$this->metadata = $fieldMap['metadata'];
$this->loadMetadataFromDbFieldValue( $db, $fieldMap['metadata'] );
}
} else {
throw new MWException( "Could not find data for image '{$this->getName()}'." );
@ -557,10 +563,6 @@ class LocalFile extends File {
}
}
if ( isset( $fieldMap['metadata'] ) ) {
$fieldMap['metadata'] = $this->repo->getReplicaDB()->decodeBlob( $fieldMap['metadata'] );
}
return $fieldMap;
}
@ -604,7 +606,6 @@ class LocalFile extends File {
*/
public function loadFromRow( $row, $prefix = 'img_' ) {
$this->dataLoaded = true;
$this->extraDataLoaded = true;
$unprefixed = $this->unprefixRow( $row, $prefix );
@ -622,7 +623,8 @@ class LocalFile extends File {
$this->timestamp = wfTimestamp( TS_MW, $unprefixed['timestamp'] );
$this->metadata = $this->repo->getReplicaDB()->decodeBlob( $unprefixed['metadata'] );
$this->loadMetadataFromDbFieldValue(
$this->repo->getReplicaDB(), $unprefixed['metadata'] );
if ( empty( $unprefixed['major_mime'] ) ) {
$this->major_mime = 'unknown';
@ -710,7 +712,7 @@ class LocalFile extends File {
} else {
$handler = $this->getHandler();
if ( $handler ) {
$validity = $handler->isMetadataValid( $this, $this->getMetadata() );
$validity = $handler->isFileMetadataValid( $this );
if ( $validity === MediaHandler::METADATA_BAD ) {
$upgrade = true;
} elseif ( $validity === MediaHandler::METADATA_COMPATIBLE ) {
@ -776,7 +778,7 @@ class LocalFile extends File {
'img_media_type' => $this->media_type,
'img_major_mime' => $major,
'img_minor_mime' => $minor,
'img_metadata' => $dbw->encodeBlob( $this->metadata ),
'img_metadata' => $this->getMetadataForDb( $dbw ),
'img_sha1' => $this->sha1,
],
[ 'img_name' => $this->getName() ],
@ -826,6 +828,20 @@ class LocalFile extends File {
$this->mime = $info['mime'];
list( $this->major_mime, $this->minor_mime ) = self::splitMime( $this->mime );
}
if ( isset( $info['metadata'] ) ) {
if ( is_string( $info['metadata'] ) ) {
$this->loadMetadataFromString( $info['metadata'] );
} elseif ( is_array( $info['metadata'] ) ) {
$this->metadataArray = $info['metadata'];
} else {
$logger = LoggerFactory::getInstance( 'LocalFile' );
$logger->warning( __METHOD__ . ' given invalid metadata of type ' .
gettype( $info['metadata'] ) );
$this->metadataArray = [];
}
$this->extraDataLoaded = true;
}
}
/** splitMime inherited */
@ -942,13 +958,89 @@ class LocalFile extends File {
}
/**
* Get handler-specific metadata
* @stable to override
* Get handler-specific metadata as a serialized string
*
* @deprecated since 1.37 use getMetadataArray() or getMetadataItem()
* @return string
*/
public function getMetadata() {
$this->load( self::LOAD_ALL ); // large metadata is loaded in another step
return $this->metadata;
$data = $this->getMetadataArray();
if ( !$data ) {
return '';
} elseif ( array_keys( $data ) === [ '_error' ] ) {
// Legacy error encoding
return $data['_error'];
} else {
return serialize( $this->getMetadataArray() );
}
}
/**
* Get unserialized handler-specific metadata
*
* @since 1.37
* @return array
*/
public function getMetadataArray(): array {
$this->load( self::LOAD_ALL );
return $this->metadataArray;
}
/**
* Serialize the metadata array for insertion into img_metadata, oi_metadata
* or fa_metadata
*
* @internal
* @param IDatabase $db
* @return string|Blob
*/
public function getMetadataForDb( IDatabase $db ) {
$this->load( self::LOAD_ALL );
if ( !$this->metadataArray ) {
$s = '';
} else {
$s = serialize( $this->metadataArray );
}
if ( !is_string( $s ) ) {
throw new MWException( 'Could not serialize image metadata value for DB' );
}
return $db->encodeBlob( $s );
}
/**
* Unserialize a metadata blob which came from the database and store it
* in $this.
*
* @since 1.37
* @param IDatabase $db
* @param string|Blob $metadataBlob
*/
protected function loadMetadataFromDbFieldValue( IDatabase $db, $metadataBlob ) {
$this->loadMetadataFromString( $db->decodeBlob( $metadataBlob ) );
}
/**
* Unserialize a metadata string which came from some non-DB source, or is
* the return value of IDatabase::decodeBlob().
*
* @since 1.37
* @param string $metadataString
*/
protected function loadMetadataFromString( $metadataString ) {
$this->extraDataLoaded = true;
$metadataString = (string)$metadataString;
if ( $metadataString === '' ) {
$this->metadataArray = [];
} else {
// phpcs:ignore Generic.PHP.NoSilencedErrors.Discouraged
$data = @unserialize( $metadataString );
if ( !is_array( $data ) ) {
// Legacy error encoding
$this->metadataArray = [ '_error' => $metadataString ];
} else {
$this->metadataArray = $data;
}
}
}
/**
@ -1399,13 +1491,19 @@ class LocalFile extends File {
$options = [];
$handler = MediaHandler::getHandler( $props['mime'] );
if ( $handler ) {
$metadata = AtEase::quietCall( 'unserialize', $props['metadata'] );
if ( !is_array( $metadata ) ) {
$metadata = [];
if ( is_string( $props['metadata'] ) ) {
// This supports callers directly fabricating a metadata
// property using serialize(). Normally the metadata property
// comes from MWFileProps, in which case it won't be a string.
// phpcs:ignore Generic.PHP.NoSilencedErrors.Discouraged
$metadata = @unserialize( $props['metadata'] );
} else {
$metadata = $props['metadata'];
}
$options['headers'] = $handler->getContentHeaders( $metadata );
if ( is_array( $metadata ) ) {
$options['headers'] = $handler->getContentHeaders( $metadata );
}
} else {
$options['headers'] = [];
}
@ -1560,7 +1658,7 @@ class LocalFile extends File {
'img_major_mime' => $this->major_mime,
'img_minor_mime' => $this->minor_mime,
'img_timestamp' => $timestamp,
'img_metadata' => $dbw->encodeBlob( $this->metadata ),
'img_metadata' => $this->getMetadataForDb( $dbw ),
'img_sha1' => $this->sha1
] + $commentFields + $actorFields,
__METHOD__,
@ -1635,7 +1733,7 @@ class LocalFile extends File {
'img_major_mime' => $this->major_mime,
'img_minor_mime' => $this->minor_mime,
'img_timestamp' => $timestamp,
'img_metadata' => $dbw->encodeBlob( $this->metadata ),
'img_metadata' => $this->getMetadataForDb( $dbw ),
'img_sha1' => $this->sha1
] + $commentFields + $actorFields,
[ 'img_name' => $this->getName() ],
@ -2309,7 +2407,7 @@ class LocalFile extends File {
// If extra data (metadata) was not loaded then it must have been large
return $this->extraDataLoaded
&& strlen( serialize( $this->metadata ) ) <= self::CACHE_FIELD_MAX_LEN;
&& strlen( serialize( $this->metadataArray ) ) <= self::CACHE_FIELD_MAX_LEN;
}
/**

View file

@ -177,10 +177,17 @@ class LocalFileRestoreBatch {
) {
// Refresh our metadata
// Required for a new current revision; nice for older ones too. :)
// @phan-suppress-next-line SecurityCheck-PathTraversal False positive T268920
$props = MediaWikiServices::getInstance()->getRepoGroup()->getFileProps( $deletedUrl );
$this->file->loadFromFile( $deletedUrl );
$mime = $this->file->getMimeType();
list( $majorMime, $minorMime ) = File::splitMime( $mime );
$mediaInfo = [
'minor_mime' => $minorMime,
'major_mime' => $majorMime,
'media_type' => $this->file->getMediaType(),
'metadata' => $this->file->getMetadataForDb( $dbw )
];
} else {
$props = [
$mediaInfo = [
'minor_mime' => $row->fa_minor_mime,
'major_mime' => $row->fa_major_mime,
'media_type' => $row->fa_media_type,
@ -198,11 +205,11 @@ class LocalFileRestoreBatch {
'img_size' => $row->fa_size,
'img_width' => $row->fa_width,
'img_height' => $row->fa_height,
'img_metadata' => $props['metadata'],
'img_metadata' => $mediaInfo['metadata'],
'img_bits' => $row->fa_bits,
'img_media_type' => $props['media_type'],
'img_major_mime' => $props['major_mime'],
'img_minor_mime' => $props['minor_mime'],
'img_media_type' => $mediaInfo['media_type'],
'img_major_mime' => $mediaInfo['major_mime'],
'img_minor_mime' => $mediaInfo['minor_mime'],
'img_actor' => $row->fa_actor,
'img_timestamp' => $row->fa_timestamp,
'img_sha1' => $sha1
@ -240,10 +247,10 @@ class LocalFileRestoreBatch {
'oi_bits' => $row->fa_bits,
'oi_actor' => $row->fa_actor,
'oi_timestamp' => $row->fa_timestamp,
'oi_metadata' => $props['metadata'],
'oi_media_type' => $props['media_type'],
'oi_major_mime' => $props['major_mime'],
'oi_minor_mime' => $props['minor_mime'],
'oi_metadata' => $mediaInfo['metadata'],
'oi_media_type' => $mediaInfo['media_type'],
'oi_major_mime' => $mediaInfo['major_mime'],
'oi_minor_mime' => $mediaInfo['minor_mime'],
'oi_deleted' => $this->unsuppress ? 0 : $row->fa_deleted,
'oi_sha1' => $sha1
] + $commentStore->insert( $dbw, 'oi_description', $comment );

View file

@ -368,7 +368,7 @@ class OldLocalFile extends LocalFile {
'oi_media_type' => $this->media_type,
'oi_major_mime' => $major,
'oi_minor_mime' => $minor,
'oi_metadata' => $this->metadata,
'oi_metadata' => $this->getMetadataForDb( $dbw ),
'oi_sha1' => $this->sha1,
], [
'oi_name' => $this->getName(),
@ -462,6 +462,7 @@ class OldLocalFile extends LocalFile {
if ( !$props['fileExists'] ) {
return false;
}
$this->setProps( $props );
$commentFields = MediaWikiServices::getInstance()->getCommentStore()
->insert( $dbw, 'oi_description', $comment );
@ -477,7 +478,7 @@ class OldLocalFile extends LocalFile {
'oi_bits' => $props['bits'],
'oi_actor' => $actorId,
'oi_timestamp' => $dbw->timestamp( $timestamp ),
'oi_metadata' => $props['metadata'],
'oi_metadata' => $this->getMetadataForDb( $dbw ),
'oi_media_type' => $props['media_type'],
'oi_major_mime' => $props['major_mime'],
'oi_minor_mime' => $props['minor_mime'],

View file

@ -44,10 +44,10 @@ class UnregisteredLocalFile extends File {
protected $mime;
/** @var array[]|bool[] Dimension data */
protected $dims;
protected $pageDims;
/** @var bool|string Handler-specific metadata which will be saved in the img_metadata field */
protected $metadata;
/** @var array|null */
protected $sizeAndMetadata;
/** @var MediaHandler */
public $handler;
@ -103,7 +103,7 @@ class UnregisteredLocalFile extends File {
if ( $mime ) {
$this->mime = $mime;
}
$this->dims = [];
$this->pageDims = [];
}
/**
@ -116,14 +116,22 @@ class UnregisteredLocalFile extends File {
$page = 1;
}
if ( !isset( $this->dims[$page] ) ) {
if ( !isset( $this->pageDims[$page] ) ) {
if ( !$this->getHandler() ) {
return false;
}
$this->dims[$page] = $this->handler->getPageDimensions( $this, $page );
if ( $this->getHandler()->isMultiPage( $this ) ) {
$this->pageDims[$page] = $this->handler->getPageDimensions( $this, $page );
} else {
$info = $this->getSizeAndMetadata();
return [
'width' => $info['width'],
'height' => $info['height']
];
}
}
return $this->dims[$page];
return $this->pageDims[$page];
}
/**
@ -158,43 +166,38 @@ class UnregisteredLocalFile extends File {
return $this->mime;
}
/**
* @param string $filename
* @return array|bool
*/
protected function getImageSize( $filename ) {
if ( !$this->getHandler() ) {
return false;
}
return $this->handler->getImageSize( $this, $this->getLocalRefPath() );
}
/**
* @return int
*/
public function getBitDepth() {
$gis = $this->getImageSize( $this->getLocalRefPath() );
if ( !$gis || !isset( $gis['bits'] ) ) {
return 0;
}
return $gis['bits'];
$info = $this->getSizeAndMetadata();
return $info['bits'] ?? 0;
}
/**
* @return string|false
*/
public function getMetadata() {
if ( !isset( $this->metadata ) ) {
$info = $this->getSizeAndMetadata();
return $info['metadata'] ? serialize( $info['metadata'] ) : false;
}
public function getMetadataArray(): array {
$info = $this->getSizeAndMetadata();
return $info['metadata'];
}
private function getSizeAndMetadata() {
if ( $this->sizeAndMetadata === null ) {
if ( !$this->getHandler() ) {
$this->metadata = false;
$this->sizeAndMetadata = [ 'width' => 0, 'height' => 0, 'metadata' => [] ];
} else {
$this->metadata = $this->handler->getMetadata( $this, $this->getLocalRefPath() );
$this->sizeAndMetadata = $this->getHandler()->getSizeAndMetadataWithFallback(
$this, $this->getLocalRefPath() );
}
}
return $this->metadata;
return $this->sizeAndMetadata;
}
/**

View file

@ -29,16 +29,13 @@ use Wikimedia\Timestamp\ConvertibleTimestamp;
*
* @ingroup FileBackend
*/
class FSFile implements MediaHandlerState {
class FSFile {
/** @var string Path to file */
protected $path;
/** @var string File SHA-1 in base 36 */
protected $sha1Base36;
/** @var array */
private $handlerState = [];
/**
* Sets up the file object
*
@ -230,22 +227,4 @@ class FSFile implements MediaHandlerState {
return $fsFile->getSha1Base36();
}
/**
* @unstable This will soon be removed.
* @param string $key
* @return mixed|null
*/
public function getHandlerState( string $key ) {
return $this->handlerState[$key] ?? null;
}
/**
* @unstable This will soon be removed.
* @param string $key
* @param mixed $value
*/
public function setHandlerState( string $key, $value ) {
$this->handlerState[$key] = $value;
}
}

View file

@ -166,6 +166,9 @@ class BitmapMetadataHandler {
$seg = JpegMetadataExtractor::segmentSplitter( $filename );
if ( isset( $seg['SOF'] ) ) {
$meta->addMetadata( [ 'SOF' => $seg['SOF'] ] );
}
if ( isset( $seg['COM'] ) && isset( $seg['COM'][0] ) ) {
$meta->addMetadata( [ 'JPEGFileComment' => $seg['COM'] ], 'native' );
}

View file

@ -51,14 +51,14 @@ class BmpHandler extends BitmapHandler {
/**
* Get width and height from the bmp header.
*
* @param File|FSFile $image
* @param MediaHandlerState $state
* @param string $filename
* @return array|false
* @return array
*/
public function getImageSize( $image, $filename ) {
public function getSizeAndMetadata( $state, $filename ) {
$f = fopen( $filename, 'rb' );
if ( !$f ) {
return false;
return [];
}
$header = fread( $f, 54 );
fclose( $f );
@ -72,9 +72,12 @@ class BmpHandler extends BitmapHandler {
$w = wfUnpack( 'V', $w, 4 );
$h = wfUnpack( 'V', $h, 4 );
} catch ( Exception $e ) {
return false;
return [];
}
return [ $w[1], $h[1] ];
return [
'width' => $w[1],
'height' => $h[1]
];
}
}

View file

@ -263,29 +263,21 @@ class DjVuHandler extends ImageHandler {
* @return string XML metadata as a string.
* @throws MWException
*/
private function getUnserializedMetadata( File $file ) {
$metadata = $file->getMetadata();
if ( substr( $metadata, 0, 3 ) === '<?xml' ) {
private function getXMLMetadata( File $file ) {
$unser = $file->getMetadataArray();
if ( isset( $unser['error'] ) ) {
return false;
} elseif ( isset( $unser['xml'] ) ) {
return $unser['xml'];
} elseif ( isset( $unser['_error'] )
&& is_string( $unser['_error'] )
&& substr( $unser['_error'], 0, 3 ) === '<?xml'
) {
// Old style. Not serialized but instead just a raw string of XML.
return $metadata;
return $unser['_error'];
} else {
return false;
}
Wikimedia\suppressWarnings();
$unser = unserialize( $metadata );
Wikimedia\restoreWarnings();
if ( is_array( $unser ) ) {
if ( isset( $unser['error'] ) ) {
return false;
} elseif ( isset( $unser['xml'] ) ) {
return $unser['xml'];
} else {
// Should never ever reach here.
throw new MWException( "Error unserializing DjVu metadata." );
}
}
// unserialize failed. Guess it wasn't really serialized after all,
return $metadata;
}
/**
@ -302,14 +294,14 @@ class DjVuHandler extends ImageHandler {
return $image->getHandlerState( self::STATE_META_TREE );
}
$metadata = $this->getUnserializedMetadata( $image );
if ( !$this->isMetadataValid( $image, $metadata ) ) {
$xml = $this->getXMLMetadata( $image );
if ( !$xml ) {
wfDebug( "DjVu XML metadata is invalid or missing, should have been fixed in upgradeRow" );
return false;
}
$trees = $this->extractTreesFromMetadata( $metadata );
$trees = $this->extractTreesFromXML( $xml );
$image->setHandlerState( self::STATE_TEXT_TREE, $trees['TextTree'] );
$image->setHandlerState( self::STATE_META_TREE, $trees['MetaTree'] );
@ -322,16 +314,16 @@ class DjVuHandler extends ImageHandler {
/**
* Extracts metadata and text trees from metadata XML in string form
* @param string $metadata XML metadata as a string
* @param string $xml XML metadata as a string
* @return array
*/
protected function extractTreesFromMetadata( $metadata ) {
protected function extractTreesFromXML( $xml ) {
Wikimedia\suppressWarnings();
try {
// Set to false rather than null to avoid further attempts
$metaTree = false;
$textTree = false;
$tree = new SimpleXMLElement( $metadata, LIBXML_PARSEHUGE );
$tree = new SimpleXMLElement( $xml, LIBXML_PARSEHUGE );
if ( $tree->getName() == 'mw-djvu' ) {
/** @var SimpleXMLElement $b */
foreach ( $tree->children() as $b ) {
@ -354,11 +346,6 @@ class DjVuHandler extends ImageHandler {
return [ 'MetaTree' => $metaTree, 'TextTree' => $textTree ];
}
public function getImageSize( $image, $path ) {
$state = $image instanceof MediaHandlerState ? $image : new TrivialMediaHandlerState;
return $this->getDjVuImage( $state, $path )->getImageSize();
}
public function getThumbType( $ext, $mime, $params = null ) {
global $wgDjvuOutputExtension;
static $mime;
@ -370,25 +357,26 @@ class DjVuHandler extends ImageHandler {
return [ $wgDjvuOutputExtension, $mime ];
}
public function getMetadata( $image, $path ) {
public function getSizeAndMetadata( $state, $path ) {
wfDebug( "Getting DjVu metadata for $path" );
$state = $image instanceof MediaHandlerState ? $image : new TrivialMediaHandlerState;
$xml = $this->getDjVuImage( $state, $path )->retrieveMetaData();
$djvuImage = $this->getDjVuImage( $state, $path );
$xml = $djvuImage->retrieveMetaData();
if ( $xml === false ) {
// Special value so that we don't repetitively try and decode a broken file.
return serialize( [ 'error' => 'Error extracting metadata' ] );
$metadata = [ 'error' => 'Error extracting metadata' ];
} else {
return serialize( [ 'xml' => $xml ] );
$metadata = [ 'xml' => $xml ];
}
return [ 'metadata' => $metadata ] + $djvuImage->getImageSize();
}
public function getMetadataType( $image ) {
return 'djvuxml';
}
public function isMetadataValid( $image, $metadata ) {
return !empty( $metadata ) && $metadata != serialize( [] );
public function isFileMetadataValid( $image ) {
return $image->getMetadataArray() ? self::METADATA_GOOD : self::METADATA_BAD;
}
public function pageCount( File $image ) {

View file

@ -63,21 +63,20 @@ class DjVuImage {
}
/**
* Return data in the style of getimagesize()
* @return array|false Array or false on failure
* Return width and height
* @return array An array with "width" and "height" keys, or an empty array on failure.
*/
public function getImageSize() {
$data = $this->getInfo();
if ( $data !== false ) {
$width = $data['width'];
$height = $data['height'];
return [ $width, $height, 'DjVu',
"width=\"$width\" height=\"$height\"" ];
return [
'width' => $data['width'],
'height' => $data['height']
];
} else {
return [];
}
return false;
}
// ---------

View file

@ -42,9 +42,6 @@ class ExifBitmapHandler extends BitmapHandler {
return $metadata;
}
if ( !is_array( $metadata ) ) {
$metadata = unserialize( $metadata );
}
if ( !isset( $metadata['MEDIAWIKI_EXIF_VERSION'] ) || $metadata['MEDIAWIKI_EXIF_VERSION'] != 2 ) {
return $metadata;
}
@ -83,28 +80,30 @@ class ExifBitmapHandler extends BitmapHandler {
/**
* @param File $image
* @param string $metadata
* @return bool|int
*/
public function isMetadataValid( $image, $metadata ) {
public function isFileMetadataValid( $image ) {
global $wgShowEXIF;
if ( !$wgShowEXIF ) {
# Metadata disabled and so an empty field is expected
return self::METADATA_GOOD;
}
if ( $metadata === self::OLD_BROKEN_FILE ) {
$exif = $image->getMetadataArray();
if ( !$exif ) {
wfDebug( __METHOD__ . ': error unserializing?' );
return self::METADATA_BAD;
}
if ( $exif === [ '_error' => self::OLD_BROKEN_FILE ] ) {
# Old special value indicating that there is no Exif data in the file.
# or that there was an error well extracting the metadata.
wfDebug( __METHOD__ . ": back-compat version" );
return self::METADATA_COMPATIBLE;
}
if ( $metadata === self::BROKEN_FILE ) {
if ( $exif === [ '_error' => self::BROKEN_FILE ] ) {
return self::METADATA_GOOD;
}
Wikimedia\suppressWarnings();
$exif = unserialize( $metadata );
Wikimedia\restoreWarnings();
if ( !isset( $exif['MEDIAWIKI_EXIF_VERSION'] )
|| $exif['MEDIAWIKI_EXIF_VERSION'] != Exif::version()
) {
@ -140,17 +139,7 @@ class ExifBitmapHandler extends BitmapHandler {
}
public function getCommonMetaArray( File $file ) {
$metadata = $file->getMetadata();
if ( $metadata === self::OLD_BROKEN_FILE
|| $metadata === self::BROKEN_FILE
|| $this->isMetadataValid( $file, $metadata ) === self::METADATA_BAD
) {
// So we don't try and display metadata from PagedTiffHandler
// for example when using InstantCommons.
return [];
}
$exif = unserialize( $metadata );
$exif = $file->getMetadataArray();
if ( !$exif ) {
return [];
}
@ -163,33 +152,19 @@ class ExifBitmapHandler extends BitmapHandler {
return 'exif';
}
/**
* Wrapper for base classes ImageHandler::getImageSize() that checks for
* rotation reported from metadata and swaps the sizes to match.
*
* @param File|FSFile $image
* @param string $path
* @return array|false
*/
public function getImageSize( $image, $path ) {
$gis = parent::getImageSize( $image, $path );
// Don't just call $image->getMetadata(); FSFile::getPropsFromPath() calls us with a bogus object.
// This may mean we read EXIF data twice on initial upload.
protected function applyExifRotation( $info, $metadata ) {
if ( $this->autoRotateEnabled() ) {
$meta = $this->getMetadata( $image, $path );
$rotation = $this->getRotationForExif( $meta );
$rotation = $this->getRotationForExifFromOrientation( $metadata['Orientation'] ?? null );
} else {
$rotation = 0;
}
if ( $rotation == 90 || $rotation == 270 ) {
$width = $gis[0];
$gis[0] = $gis[1];
$gis[1] = $width;
$width = $info['width'];
$info['width'] = $info['height'];
$info['height'] = $width;
}
return $gis;
return $info;
}
/**
@ -209,40 +184,32 @@ class ExifBitmapHandler extends BitmapHandler {
return 0;
}
$data = $file->getMetadata();
return $this->getRotationForExif( $data );
$orientation = $file->getMetadataItem( 'Orientation' );
return $this->getRotationForExifFromOrientation( $orientation );
}
/**
* Given a chunk of serialized Exif metadata, return the orientation as
* degrees of rotation.
*
* @param string|false $data
* @param int|null $orientation
* @return int 0, 90, 180 or 270
* @todo FIXME: Orientation can include flipping as well; see if this is an issue!
*/
protected function getRotationForExif( $data ) {
if ( !$data ) {
protected function getRotationForExifFromOrientation( $orientation ) {
if ( $orientation === null ) {
return 0;
}
Wikimedia\suppressWarnings();
$data = unserialize( $data );
Wikimedia\restoreWarnings();
if ( isset( $data['Orientation'] ) ) {
# See http://sylvana.net/jpegcrop/exif_orientation.html
switch ( $data['Orientation'] ) {
case 8:
return 90;
case 3:
return 180;
case 6:
return 270;
default:
return 0;
}
# See http://sylvana.net/jpegcrop/exif_orientation.html
switch ( $orientation ) {
case 8:
return 90;
case 3:
return 180;
case 6:
return 270;
default:
return 0;
}
return 0;
}
}

View file

@ -21,6 +21,8 @@
* @ingroup Media
*/
use Wikimedia\RequestTimeout\TimeoutException;
/**
* Handler for GIF images.
*
@ -32,17 +34,27 @@ class GIFHandler extends BitmapHandler {
*/
private const BROKEN_FILE = '0';
public function getMetadata( $image, $filename ) {
public function getSizeAndMetadata( $state, $filename ) {
try {
$parsedGIFMetadata = BitmapMetadataHandler::GIF( $filename );
} catch ( TimeoutException $e ) {
throw $e;
} catch ( Exception $e ) {
// Broken file?
wfDebug( __METHOD__ . ': ' . $e->getMessage() );
return self::BROKEN_FILE;
return [ 'metadata' => [ '_error' => self::BROKEN_FILE ] ];
}
return serialize( $parsedGIFMetadata );
return [
'width' => $parsedGIFMetadata['width'],
'height' => $parsedGIFMetadata['height'],
'bits' => $parsedGIFMetadata['bits'],
'metadata' => array_diff_key(
$parsedGIFMetadata,
[ 'width' => true, 'height' => true, 'bits' => true ]
)
];
}
/**
@ -65,12 +77,7 @@ class GIFHandler extends BitmapHandler {
* @return array|bool
*/
public function getCommonMetaArray( File $image ) {
$meta = $image->getMetadata();
if ( !$meta ) {
return [];
}
$meta = unserialize( $meta );
$meta = $image->getMetadataArray();
if ( !isset( $meta['metadata'] ) ) {
return [];
}
@ -86,14 +93,9 @@ class GIFHandler extends BitmapHandler {
* @return bool
*/
public function getImageArea( $image ) {
$ser = $image->getMetadata();
if ( $ser ) {
$metadata = unserialize( $ser );
if ( isset( $metadata['frameCount'] ) && $metadata['frameCount'] > 0 ) {
return $image->getWidth() * $image->getHeight() * $metadata['frameCount'];
} else {
return $image->getWidth() * $image->getHeight();
}
$metadata = $image->getMetadataArray();
if ( isset( $metadata['frameCount'] ) && $metadata['frameCount'] > 0 ) {
return $image->getWidth() * $image->getHeight() * $metadata['frameCount'];
} else {
return $image->getWidth() * $image->getHeight();
}
@ -104,12 +106,9 @@ class GIFHandler extends BitmapHandler {
* @return bool
*/
public function isAnimatedImage( $image ) {
$ser = $image->getMetadata();
if ( $ser ) {
$metadata = unserialize( $ser );
if ( isset( $metadata['frameCount'] ) && $metadata['frameCount'] > 1 ) {
return true;
}
$metadata = $image->getMetadataArray();
if ( isset( $metadata['frameCount'] ) && $metadata['frameCount'] > 1 ) {
return true;
}
return false;
@ -130,17 +129,14 @@ class GIFHandler extends BitmapHandler {
return 'parsed-gif';
}
public function isMetadataValid( $image, $metadata ) {
if ( $metadata === self::BROKEN_FILE ) {
// Do not repetitivly regenerate metadata on broken file.
public function isFileMetadataValid( $image ) {
$data = $image->getMetadataArray();
if ( $data === [ '_error' => self::BROKEN_FILE ] ) {
// Do not repetitively regenerate metadata on broken file.
return self::METADATA_GOOD;
}
Wikimedia\suppressWarnings();
$data = unserialize( $metadata );
Wikimedia\restoreWarnings();
if ( !$data || !is_array( $data ) ) {
if ( !$data || isset( $data['_error'] ) ) {
wfDebug( __METHOD__ . " invalid GIF metadata" );
return self::METADATA_BAD;
@ -166,9 +162,7 @@ class GIFHandler extends BitmapHandler {
$original = parent::getLongDesc( $image );
Wikimedia\suppressWarnings();
$metadata = unserialize( $image->getMetadata() );
Wikimedia\restoreWarnings();
$metadata = $image->getMetadataArray();
if ( !$metadata || $metadata['frameCount'] <= 1 ) {
return $original;
@ -202,10 +196,7 @@ class GIFHandler extends BitmapHandler {
* @return float The duration of the file.
*/
public function getLength( $file ) {
$serMeta = $file->getMetadata();
Wikimedia\suppressWarnings();
$metadata = unserialize( $serMeta );
Wikimedia\restoreWarnings();
$metadata = $file->getMetadataArray();
if ( !$metadata || !isset( $metadata['duration'] ) || !$metadata['duration'] ) {
return 0.0;

View file

@ -90,13 +90,15 @@ class GIFMetadataExtractor {
// Read BPP
$buf = fread( $fh, 1 );
$bpp = self::decodeBPP( $buf );
list( $bpp, $have_map ) = self::decodeBPP( $buf );
// Skip over background and aspect ratio
fread( $fh, 2 );
// Skip over the GCT
self::readGCT( $fh, $bpp );
if ( $have_map ) {
self::readGCT( $fh, $bpp );
}
while ( !feof( $fh ) ) {
$buf = fread( $fh, 1 );
@ -110,10 +112,12 @@ class GIFMetadataExtractor {
# # Read BPP
$buf = fread( $fh, 1 );
$bpp = self::decodeBPP( $buf );
list( $bpp, $have_map ) = self::decodeBPP( $buf );
# # Read GCT
self::readGCT( $fh, $bpp );
if ( $have_map ) {
self::readGCT( $fh, $bpp );
}
fread( $fh, 1 );
self::skipBlock( $fh );
} elseif ( $buf == self::$gifExtensionSep ) {
@ -256,6 +260,9 @@ class GIFMetadataExtractor {
'duration' => $duration,
'xmp' => $xmp,
'comment' => $comment,
'width' => $width,
'height' => $height,
'bits' => $bpp,
];
}
@ -265,18 +272,16 @@ class GIFMetadataExtractor {
* @return void
*/
private static function readGCT( $fh, $bpp ) {
if ( $bpp > 0 ) {
$max = 2 ** $bpp;
for ( $i = 1; $i <= $max; ++$i ) {
fread( $fh, 3 );
}
$max = 2 ** $bpp;
for ( $i = 1; $i <= $max; ++$i ) {
fread( $fh, 3 );
}
}
/**
* @param string $data
* @throws Exception
* @return int
* @return array [ int bits per channel, bool have GCT ]
*/
private static function decodeBPP( $data ) {
if ( strlen( $data ) < 1 ) {
@ -288,7 +293,7 @@ class GIFMetadataExtractor {
$have_map = $buf & 1;
return $have_map ? $bpp : 0;
return [ $bpp, $have_map ];
}
/**

View file

@ -229,10 +229,6 @@ abstract class ImageHandler extends MediaHandler {
}
}
/**
* @inheritDoc
* @stable to override
*/
public function getImageSize( $image, $path ) {
Wikimedia\suppressWarnings();
$gis = getimagesize( $path );
@ -241,6 +237,23 @@ abstract class ImageHandler extends MediaHandler {
return $gis;
}
public function getSizeAndMetadata( $state, $path ) {
// phpcs:ignore Generic.PHP.NoSilencedErrors.Discouraged
$gis = @getimagesize( $path );
if ( $gis ) {
$info = [
'width' => $gis[0],
'height' => $gis[1],
];
if ( isset( $gis['bits'] ) ) {
$info['bits'] = $gis['bits'];
}
} else {
$info = [];
}
return $info;
}
/**
* Function that returns the number of pixels to be thumbnailed.
* Intended for animated GIFs to multiply by the number of frames.

View file

@ -100,7 +100,7 @@ class JpegHandler extends ExifBitmapHandler {
return $res;
}
public function getMetadata( $image, $filename ) {
public function getSizeAndMetadata( $state, $filename ) {
try {
$meta = BitmapMetadataHandler::Jpeg( $filename );
if ( !is_array( $meta ) ) {
@ -109,23 +109,27 @@ class JpegHandler extends ExifBitmapHandler {
}
$meta['MEDIAWIKI_EXIF_VERSION'] = Exif::version();
return serialize( $meta );
} catch ( Exception $e ) {
$info = [
'width' => $meta['SOF']['width'] ?? 0,
'height' => $meta['SOF']['height'] ?? 0,
];
if ( isset( $meta['SOF']['bits'] ) ) {
$info['bits'] = $meta['SOF']['bits'];
}
$info = $this->applyExifRotation( $info, $meta );
unset( $meta['SOF'] );
$info['metadata'] = $meta;
return $info;
} catch ( MWException $e ) {
// BitmapMetadataHandler throws an exception in certain exceptional
// cases like if file does not exist.
wfDebug( __METHOD__ . ': ' . $e->getMessage() );
/* This used to use 0 (ExifBitmapHandler::OLD_BROKEN_FILE) for the cases
* * No metadata in the file
* * Something is broken in the file.
* However, if the metadata support gets expanded then you can't tell if the 0 is from
* a broken file, or just no props found. A broken file is likely to stay broken, but
* a file which had no props could have props once the metadata support is improved.
* Thus switch to using -1 to denote only a broken file, and use an array with only
* MEDIAWIKI_EXIF_VERSION to denote no props.
*/
return ExifBitmapHandler::BROKEN_FILE;
// This used to return an integer-like string from getMetadata(),
// producing a value which could not be unserialized in
// img_metadata. The "_error" array key matches the legacy
// unserialization for such image rows.
return [ 'metadata' => [ '_error' => ExifBitmapHandler::BROKEN_FILE ] ];
}
}

View file

@ -157,6 +157,13 @@ class JpegMetadataExtractor {
} elseif ( $buffer === "\xD9" || $buffer === "\xDA" ) {
// EOI - end of image or SOS - start of scan. either way we're past any interesting segments
return $segments;
} elseif ( in_array( $buffer, [
"\xC0", "\xC1", "\xC2", "\xC3", "\xC5", "\xC6", "\xC7",
"\xC9", "\xCA", "\xCB", "\xCD", "\xCE", "\xCF" ] )
) {
// SOF0, SOF1, SOF2, ... (same list as getimagesize)
$temp = self::jpegExtractMarker( $fh );
$segments["SOF"] = wfUnpack( 'Cbits/nheight/nwidth/Ccomponents', $temp );
} else {
// segment we don't care about, so skip
$size = wfUnpack( "nint", fread( $fh, 2 ), 2 );

View file

@ -107,6 +107,8 @@ abstract class MediaHandler {
* @note If this is a multipage file, return the width and height of the
* first page.
*
* @deprecated since 1.37, override getSizeAndMetadata instead
*
* @param File|FSFile $image The image object, or false if there isn't one.
* Warning, FSFile::getPropsFromPath might pass an FSFile instead of File (!)
* @param string $path The filename
@ -116,11 +118,42 @@ abstract class MediaHandler {
* key. All other array keys are ignored. Returning a 'bits' key is optional
* as not all formats have a notion of "bitdepth". Returns false on failure.
*/
abstract public function getImageSize( $image, $path );
public function getImageSize( $image, $path ) {
return false;
}
/**
* Get image size information and metadata array.
*
* If this returns null, the caller will fall back to getImageSize() and
* getMetadata().
*
* If getImageSize() or getMetadata() are implemented in the most derived
* class, they will be used instead of this function. To override this
* behaviour, override useLegacyMetadata().
*
* @stable to override
* @since 1.37
*
* @param MediaHandlerState $state An object for saving process-local state.
* This is normally a File object which will be passed back to other
* MediaHandler methods like pageCount(), if they are called in the same
* request. The handler can use this object to save its state.
* @param string $path The filename
* @return array|null Null to fall back to getImageSize(), or an array with
* the following keys. All keys are optional.
* - width: The width. If multipage, return the first page width. (optional)
* - height: The height. If multipage, return the first page height. (optional)
* - bits: The number of bits for each color (optional)
* - metadata: A JSON-serializable array of metadata (optional)
*/
public function getSizeAndMetadata( $state, $path ) {
return null;
}
/**
* Get handler-specific metadata which will be saved in the img_metadata field.
* @stable to override
* @deprecated since 1.37 override getSizeAndMetadata() instead
*
* @param File|FSFile $image The image object, or false if there isn't one.
* Warning, FSFile::getPropsFromPath might pass an FSFile instead of File (!)
@ -131,6 +164,115 @@ abstract class MediaHandler {
return '';
}
/**
* If this returns true, the new method getSizeAndMetadata() will not be
* called. The legacy methods getMetadata() and getImageSize() will be used
* instead.
*
* @since 1.37
* @stable to override
* @return bool
*/
protected function useLegacyMetadata() {
return $this->hasMostDerivedMethod( 'getMetadata' )
|| $this->hasMostDerivedMethod( 'getImageSize' );
}
/**
* Check whether a method is implemented in the most derived class.
*
* @since 1.37
* @param string $name
* @return bool
*/
protected function hasMostDerivedMethod( $name ) {
$rc = new ReflectionClass( $this );
$rm = new ReflectionMethod( $this, $name );
if ( $rm->getDeclaringClass()->getName() === $rc->getName() ) {
return true;
} else {
return false;
}
}
/**
* Get the metadata array and the image size, with b/c fallback.
*
* The legacy methods will be used if useLegacyMetadata() returns true or
* if getSizeAndMetadata() returns null.
*
* Absent metadata will be normalized to an empty array. Absent width and
* height will be normalized to zero.
*
* @param File|FSFile $file This must be a File or FSFile to support the
* legacy methods. When the legacy methods are removed, this will be
* narrowed to MediaHandlerState.
* @param string $path
* @return array|false|null False on failure, or an array with the following keys:
* - width: The width. If multipage, return the first page width.
* - height: The height. If multipage, return the first page height.
* - bits: The number of bits for each color (optional)
* - metadata: A JSON-serializable array of metadata
* @since 1.37
*/
final public function getSizeAndMetadataWithFallback( $file, $path ) {
if ( !$this->useLegacyMetadata() ) {
if ( $file instanceof MediaHandlerState ) {
$state = $file;
} else {
$state = new TrivialMediaHandlerState;
}
$info = $this->getSizeAndMetadata( $state, $path );
if ( $info === false ) {
return false;
}
if ( $info !== null ) {
if ( !is_array( $info['metadata'] ) ) {
throw new InvalidArgumentException( 'Media handler ' .
static::class . ' returned ' . gettype( $info['metadata'] ) .
' for metadata, should be array' );
}
return $info + [ 'width' => 0, 'height' => 0, 'metadata' => [] ];
}
}
$blob = $this->getMetadata( $file, $path );
// @phan-suppress-next-line PhanParamTooMany
$size = $this->getImageSize(
$file,
$path,
$blob // Secret TimedMediaHandler parameter
);
if ( $blob === false && $size === false ) {
return false;
}
if ( $size ) {
$info = [
'width' => $size[0] ?? 0,
'height' => $size[1] ?? 0
];
if ( isset( $size['bits'] ) ) {
$info['bits'] = $size['bits'];
}
} else {
$info = [ 'width' => 0, 'height' => 0 ];
}
if ( $blob !== false ) {
// phpcs:ignore Generic.PHP.NoSilencedErrors.Discouraged
$metadata = @unserialize( $blob );
if ( $metadata === false ) {
// Unserialize error
$metadata = [ '_error' => $blob ];
} elseif ( !is_array( $metadata ) ) {
$metadata = [];
}
$info['metadata'] = $metadata;
} else {
$info['metadata'] = [];
}
return $info;
}
/**
* Get metadata version.
*
@ -162,20 +304,11 @@ abstract class MediaHandler {
* media handlers to convert between metadata versions.
* @stable to override
*
* @param string|array $metadata Metadata array (serialized if string)
* @param array $metadata Metadata array
* @param int $version Target version
* @return array Serialized metadata in specified version, or $metadata on fail.
*/
public function convertMetadataVersion( $metadata, $version = 1 ) {
if ( !is_array( $metadata ) ) {
// unserialize to keep return parameter consistent.
Wikimedia\suppressWarnings();
$ret = unserialize( $metadata );
Wikimedia\restoreWarnings();
return $ret;
}
return $metadata;
}
@ -204,7 +337,7 @@ abstract class MediaHandler {
* triggering as bad metadata for a large number of files can cause
* performance problems.
*
* @stable to override
* @deprecated since 1.37 use isFileMetadataValid
* @param File $image
* @param string $metadata The metadata in serialized form
* @return bool|int
@ -213,6 +346,35 @@ abstract class MediaHandler {
return self::METADATA_GOOD;
}
/**
* Check if the metadata is valid for this handler.
* If it returns MediaHandler::METADATA_BAD (or false), Image
* will reload the metadata from the file and update the database.
* MediaHandler::METADATA_GOOD for if the metadata is a-ok,
* MediaHandler::METADATA_COMPATIBLE if metadata is old but backwards
* compatible (which may or may not trigger a metadata reload).
*
* @note Returning self::METADATA_BAD will trigger a metadata reload from
* file on page view. Always returning this from a broken file, or suddenly
* triggering as bad metadata for a large number of files can cause
* performance problems.
*
* This was introduced in 1.37 to replace isMetadataValid(), which took a
* serialized string as a parameter. Handlers overriding this method are
* expected to use accessors to get the metadata out of the File. The
* reasons for the change were to get rid of serialization, and to allow
* handlers to partially load metadata with getMetadataItem(). For example
* a handler could just validate a version number.
*
* @stable to override
* @since 1.37
* @param File $image
* @return bool|int
*/
public function isFileMetadataValid( $image ) {
return self::METADATA_GOOD;
}
/**
* Get an array of standard (FormatMetadata type) metadata values.
*
@ -432,9 +594,8 @@ abstract class MediaHandler {
* expanded in the future.
* Returns false if unknown.
*
* It is expected that handlers for paged media (e.g. DjVuHandler)
* will override this method so that it gives the correct results
* for each specific page of the file, using the $page argument.
* For a single page document format (!isMultipage()), this should return
* false.
*
* @note For non-paged media, use getImageSize.
*
@ -445,15 +606,7 @@ abstract class MediaHandler {
* @return array|bool
*/
public function getPageDimensions( File $image, $page ) {
$gis = $this->getImageSize( $image, $image->getLocalRefPath() );
if ( $gis ) {
return [
'width' => $gis[0],
'height' => $gis[1]
];
} else {
return false;
}
return false;
}
/**

View file

@ -21,6 +21,8 @@
* @ingroup Media
*/
use Wikimedia\RequestTimeout\TimeoutException;
/**
* Handler for PNG images.
*
@ -30,21 +32,31 @@ class PNGHandler extends BitmapHandler {
private const BROKEN_FILE = '0';
/**
* @param File|FSFile $image
* @param MediaHandlerState $state
* @param string $filename
* @return string
* @return array
*/
public function getMetadata( $image, $filename ) {
public function getSizeAndMetadata( $state, $filename ) {
try {
$metadata = BitmapMetadataHandler::PNG( $filename );
} catch ( TimeoutException $e ) {
throw $e;
} catch ( Exception $e ) {
// Broken file?
wfDebug( __METHOD__ . ': ' . $e->getMessage() );
return self::BROKEN_FILE;
return [ 'metadata' => [ '_error' => self::BROKEN_FILE ] ];
}
return serialize( $metadata );
return [
'width' => $metadata['width'],
'height' => $metadata['height'],
'bits' => $metadata['bitDepth'],
'metadata' => array_diff_key(
$metadata,
[ 'width' => true, 'height' => true, 'bits' => true ]
)
];
}
/**
@ -68,12 +80,8 @@ class PNGHandler extends BitmapHandler {
* @return array The metadata array
*/
public function getCommonMetaArray( File $image ) {
$meta = $image->getMetadata();
$meta = $image->getMetadataArray();
if ( !$meta ) {
return [];
}
$meta = unserialize( $meta );
if ( !isset( $meta['metadata'] ) ) {
return [];
}
@ -87,12 +95,9 @@ class PNGHandler extends BitmapHandler {
* @return bool
*/
public function isAnimatedImage( $image ) {
$ser = $image->getMetadata();
if ( $ser ) {
$metadata = unserialize( $ser );
if ( $metadata['frameCount'] > 1 ) {
return true;
}
$metadata = $image->getMetadataArray();
if ( isset( $metadata['frameCount'] ) && $metadata['frameCount'] > 1 ) {
return true;
}
return false;
@ -111,17 +116,14 @@ class PNGHandler extends BitmapHandler {
return 'parsed-png';
}
public function isMetadataValid( $image, $metadata ) {
if ( $metadata === self::BROKEN_FILE ) {
// Do not repetitivly regenerate metadata on broken file.
public function isFileMetadataValid( $image ) {
$data = $image->getMetadataArray();
if ( $data === [ '_error' => self::BROKEN_FILE ] ) {
// Do not repetitively regenerate metadata on broken file.
return self::METADATA_GOOD;
}
Wikimedia\suppressWarnings();
$data = unserialize( $metadata );
Wikimedia\restoreWarnings();
if ( !$data || !is_array( $data ) ) {
if ( !$data || isset( $data['_error'] ) ) {
wfDebug( __METHOD__ . " invalid png metadata" );
return self::METADATA_BAD;
@ -146,9 +148,7 @@ class PNGHandler extends BitmapHandler {
global $wgLang;
$original = parent::getLongDesc( $image );
Wikimedia\suppressWarnings();
$metadata = unserialize( $image->getMetadata() );
Wikimedia\restoreWarnings();
$metadata = $image->getMetadataArray();
if ( !$metadata || $metadata['frameCount'] <= 0 ) {
return $original;
@ -183,10 +183,7 @@ class PNGHandler extends BitmapHandler {
* @return float The duration of the file.
*/
public function getLength( $file ) {
$serMeta = $file->getMetadata();
Wikimedia\suppressWarnings();
$metadata = unserialize( $serMeta );
Wikimedia\restoreWarnings();
$metadata = $file->getMetadataArray();
if ( !$metadata || !isset( $metadata['duration'] ) || !$metadata['duration'] ) {
return 0.0;

View file

@ -78,6 +78,8 @@ class PNGMetadataExtractor {
$loopCount = 1;
$text = [];
$duration = 0.0;
$width = 0;
$height = 0;
$bitDepth = 0;
$colorType = 'unknown';
@ -400,6 +402,8 @@ class PNGMetadataExtractor {
}
return [
'width' => $width,
'height' => $height,
'frameCount' => $frameCount,
'loopCount' => $loopCount,
'duration' => $duration,

View file

@ -69,12 +69,9 @@ class SvgHandler extends ImageHandler {
*/
public function isAnimatedImage( $file ) {
# @todo Detect animated SVGs
$metadata = $file->getMetadata();
if ( $metadata ) {
$metadata = $this->unpackMetadata( $metadata );
if ( isset( $metadata['animated'] ) ) {
return $metadata['animated'];
}
$metadata = $this->validateMetadata( $file->getMetadataArray() );
if ( isset( $metadata['animated'] ) ) {
return $metadata['animated'];
}
return false;
@ -93,15 +90,12 @@ class SvgHandler extends ImageHandler {
* @return string[] Array of language codes, or empty if no language switching supported.
*/
public function getAvailableLanguages( File $file ) {
$metadata = $file->getMetadata();
$langList = [];
if ( $metadata ) {
$metadata = $this->unpackMetadata( $metadata );
if ( isset( $metadata['translations'] ) ) {
foreach ( $metadata['translations'] as $lang => $langType ) {
if ( $langType === SVGReader::LANG_FULL_MATCH ) {
$langList[] = strtolower( $lang );
}
$metadata = $this->validateMetadata( $file->getMetadataArray() );
if ( isset( $metadata['translations'] ) ) {
foreach ( $metadata['translations'] as $lang => $langType ) {
if ( $langType === SVGReader::LANG_FULL_MATCH ) {
$langList[] = strtolower( $lang );
}
}
}
@ -239,7 +233,7 @@ class SvgHandler extends ImageHandler {
return new ThumbnailImage( $image, $dstUrl, $dstPath, $params );
}
$metadata = $this->unpackMetadata( $image->getMetadata() );
$metadata = $this->validateMetadata( $image->getMetadataArray() );
if ( isset( $metadata['error'] ) ) { // sanity check
$err = wfMessage( 'svg-long-error', $metadata['error']['message'] );
@ -378,26 +372,6 @@ class SvgHandler extends ImageHandler {
}
}
/**
* @param File|FSFile $file
* @param string $path Unused
* @param bool|array $metadata
* @return array|false
*/
public function getImageSize( $file, $path, $metadata = false ) {
if ( $metadata === false && $file instanceof File ) {
$metadata = $file->getMetadata();
}
$metadata = $this->unpackMetadata( $metadata );
if ( isset( $metadata['width'] ) && isset( $metadata['height'] ) ) {
return [ $metadata['width'], $metadata['height'], 'SVG',
"width=\"{$metadata['width']}\" height=\"{$metadata['height']}\"" ];
} else { // error
return [ 0, 0, 'SVG', "width=\"0\" height=\"0\"" ];
}
}
public function getThumbType( $ext, $mime, $params = null ) {
return [ 'png', 'image/png' ];
}
@ -414,7 +388,7 @@ class SvgHandler extends ImageHandler {
public function getLongDesc( $file ) {
global $wgLang;
$metadata = $this->unpackMetadata( $file->getMetadata() );
$metadata = $this->validateMetadata( $file->getMetadataArray() );
if ( isset( $metadata['error'] ) ) {
return wfMessage( 'svg-long-error', $metadata['error']['message'] )->text();
}
@ -433,11 +407,11 @@ class SvgHandler extends ImageHandler {
}
/**
* @param File|FSFile $file
* @param MediaHandlerState $state
* @param string $filename
* @return string Serialised metadata
* @return array
*/
public function getMetadata( $file, $filename ) {
public function getSizeAndMetadata( $state, $filename ) {
$metadata = [ 'version' => self::SVG_METADATA_VERSION ];
try {
@ -452,17 +426,18 @@ class SvgHandler extends ImageHandler {
wfDebug( __METHOD__ . ': ' . $e->getMessage() );
}
return serialize( $metadata );
return [
'width' => $metadata['width'] ?? 0,
'height' => $metadata['height'] ?? 0,
'metadata' => $metadata
];
}
protected function unpackMetadata( $metadata ) {
Wikimedia\suppressWarnings();
$unser = unserialize( $metadata );
Wikimedia\restoreWarnings();
protected function validateMetadata( $unser ) {
if ( isset( $unser['version'] ) && $unser['version'] == self::SVG_METADATA_VERSION ) {
return $unser;
} else {
return false;
return null;
}
}
@ -470,9 +445,9 @@ class SvgHandler extends ImageHandler {
return 'parsed-svg';
}
public function isMetadataValid( $image, $metadata ) {
$meta = $this->unpackMetadata( $metadata );
if ( $meta === false ) {
public function isFileMetadataValid( $image ) {
$meta = $this->validateMetadata( $image->getMetadataArray() );
if ( !$meta ) {
return self::METADATA_BAD;
}
if ( !isset( $meta['originalWidth'] ) ) {
@ -499,11 +474,7 @@ class SvgHandler extends ImageHandler {
'visible' => [],
'collapsed' => []
];
$metadata = $file->getMetadata();
if ( !$metadata ) {
return false;
}
$metadata = $this->unpackMetadata( $metadata );
$metadata = $this->validateMetadata( $file->getMetadataArray() );
if ( !$metadata || isset( $metadata['error'] ) ) {
return false;
}
@ -608,11 +579,7 @@ class SvgHandler extends ImageHandler {
}
public function getCommonMetaArray( File $file ) {
$metadata = $file->getMetadata();
if ( !$metadata ) {
return [];
}
$metadata = $this->unpackMetadata( $metadata );
$metadata = $this->validateMetadata( $file->getMetadataArray() );
if ( !$metadata || isset( $metadata['error'] ) ) {
return [];
}

View file

@ -21,6 +21,8 @@
* @ingroup Media
*/
use Wikimedia\RequestTimeout\TimeoutException;
/**
* Handler for Tiff images.
*
@ -73,34 +75,33 @@ class TiffHandler extends ExifBitmapHandler {
return $wgTiffThumbnailType;
}
/**
* @param File|FSFile $image
* @param string $filename
* @throws MWException
* @return string
*/
public function getMetadata( $image, $filename ) {
public function getSizeAndMetadata( $state, $filename ) {
global $wgShowEXIF;
if ( $wgShowEXIF ) {
try {
$meta = BitmapMetadataHandler::Tiff( $filename );
if ( !is_array( $meta ) ) {
// This should never happen, but doesn't hurt to be paranoid.
throw new MWException( 'Metadata array is not an array' );
}
$meta['MEDIAWIKI_EXIF_VERSION'] = Exif::version();
return serialize( $meta );
} catch ( Exception $e ) {
// BitmapMetadataHandler throws an exception in certain exceptional
// cases like if file does not exist.
wfDebug( __METHOD__ . ': ' . $e->getMessage() );
return ExifBitmapHandler::BROKEN_FILE;
try {
$meta = BitmapMetadataHandler::Tiff( $filename );
if ( !is_array( $meta ) ) {
// This should never happen, but doesn't hurt to be paranoid.
throw new MWException( 'Metadata array is not an array' );
}
} else {
return '';
$info = [
'width' => $meta['ImageWidth'] ?? 0,
'height' => $meta['ImageLength'] ?? 0,
];
$info = $this->applyExifRotation( $info, $meta );
if ( $wgShowEXIF ) {
$meta['MEDIAWIKI_EXIF_VERSION'] = Exif::version();
$info['metadata'] = $meta;
}
return $info;
} catch ( TimeoutException $e ) {
throw $e;
} catch ( Exception $e ) {
// BitmapMetadataHandler throws an exception in certain exceptional
// cases like if file does not exist.
wfDebug( __METHOD__ . ': ' . $e->getMessage() );
return [ 'metadata' => [ '_error' => ExifBitmapHandler::BROKEN_FILE ] ];
}
}

View file

@ -46,31 +46,33 @@ class WebPHandler extends BitmapHandler {
private const VP8X_XMP = 4;
private const VP8X_ANIM = 2;
public function getMetadata( $image, $filename ) {
public function getSizeAndMetadata( $state, $filename ) {
$parsedWebPData = self::extractMetadata( $filename );
if ( !$parsedWebPData ) {
return self::BROKEN_FILE;
return [ 'metadata' => [ '_error' => self::BROKEN_FILE ] ];
}
$parsedWebPData['metadata']['_MW_WEBP_VERSION'] = self::_MW_WEBP_VERSION;
return serialize( $parsedWebPData );
$info = [
'width' => $parsedWebPData['width'],
'height' => $parsedWebPData['height'],
'metadata' => $parsedWebPData
];
return $info;
}
public function getMetadataType( $image ) {
return 'parsed-webp';
}
public function isMetadataValid( $image, $metadata ) {
if ( $metadata === self::BROKEN_FILE ) {
public function isFileMetadataValid( $image ) {
$data = $image->getMetadataArray();
if ( $data === [ '_error' => self::BROKEN_FILE ] ) {
// Do not repetitivly regenerate metadata on broken file.
return self::METADATA_GOOD;
}
Wikimedia\suppressWarnings();
$data = unserialize( $metadata );
Wikimedia\restoreWarnings();
if ( !$data || !is_array( $data ) ) {
if ( !$data || !isset( $data['_error'] ) ) {
wfDebug( __METHOD__ . " invalid WebP metadata" );
return self::METADATA_BAD;
@ -230,24 +232,6 @@ class WebPHandler extends BitmapHandler {
];
}
public function getImageSize( $file, $path, $metadata = false ) {
if ( $file === null ) {
$metadata = self::getMetadata( $file, $path );
}
if ( $metadata === false && $file instanceof File ) {
$metadata = $file->getMetadata();
}
Wikimedia\suppressWarnings();
$metadata = unserialize( $metadata );
Wikimedia\restoreWarnings();
if ( $metadata == false ) {
return false;
}
return [ $metadata['width'], $metadata['height'] ];
}
/**
* @param File $file
* @return bool True, not all browsers support WebP
@ -272,12 +256,9 @@ class WebPHandler extends BitmapHandler {
* @return bool
*/
public function isAnimatedImage( $image ) {
$ser = $image->getMetadata();
if ( $ser ) {
$metadata = unserialize( $ser );
if ( isset( $metadata['animated'] ) && $metadata['animated'] === true ) {
return true;
}
$metadata = $image->getMetadataArray();
if ( isset( $metadata['animated'] ) && $metadata['animated'] === true ) {
return true;
}
return false;

View file

@ -53,32 +53,6 @@ class XCFHandler extends BitmapHandler {
return [ 'png', 'image/png' ];
}
/**
* Get width and height from the XCF header.
*
* @param File|FSFile $image
* @param string $filename
* @return array|false
*/
public function getImageSize( $image, $filename ) {
$header = self::getXCFMetaData( $filename );
if ( !$header ) {
return false;
}
# Forge a return array containing metadata information just like getimagesize()
# See PHP documentation at: https://www.php.net/getimagesize
return [
0 => $header['width'],
1 => $header['height'],
2 => null, # IMAGETYPE constant, none exist for XCF.
3 => "height=\"{$header['height']}\" width=\"{$header['width']}\"",
'mime' => 'image/x-xcf',
'channels' => null,
'bits' => 8, # Always 8-bits per color
];
}
/**
* Metadata for a given XCF file
*
@ -87,13 +61,13 @@ class XCFHandler extends BitmapHandler {
* @author Hashar
*
* @param string $filename Full path to a XCF file
* @return bool|array Metadata Array just like PHP getimagesize()
* @return array|null Metadata Array just like PHP getimagesize()
*/
private static function getXCFMetaData( $filename ) {
# Decode master structure
$f = fopen( $filename, 'rb' );
if ( !$f ) {
return false;
return null;
}
# The image structure always starts at offset 0 in the XCF file.
# So we just read it :-)
@ -126,15 +100,15 @@ class XCFHandler extends BitmapHandler {
"/Nbase_type", # /
$binaryHeader
);
} catch ( Exception $mwe ) {
return false;
} catch ( MWException $mwe ) {
return null;
}
# Check values
if ( $header['magic'] !== 'gimp xcf' ) {
wfDebug( __METHOD__ . " '$filename' has invalid magic signature." );
return false;
return null;
}
# TODO: we might want to check for sane values of width and height
@ -144,17 +118,7 @@ class XCFHandler extends BitmapHandler {
return $header;
}
/**
* Store the channel type
*
* Greyscale files need different command line options.
*
* @param File|FSFile $file The image object, or false if there isn't one.
* Warning, FSFile::getPropsFromPath might pass an (object)array() instead (!)
* @param string $filename
* @return string
*/
public function getMetadata( $file, $filename ) {
public function getSizeAndMetadata( $state, $filename ) {
$header = self::getXCFMetaData( $filename );
$metadata = [];
if ( $header ) {
@ -178,18 +142,22 @@ class XCFHandler extends BitmapHandler {
// Marker to prevent repeated attempted extraction
$metadata['error'] = true;
}
return serialize( $metadata );
return [
'width' => $header['width'] ?? 0,
'height' => $header['height'] ?? 0,
'bits' => 8,
'metadata' => $metadata
];
}
/**
* Should we refresh the metadata
*
* @param File $file The file object for the file in question
* @param string $metadata Serialized metadata
* @return bool|int One of the self::METADATA_(BAD|GOOD|COMPATIBLE) constants
*/
public function isMetadataValid( $file, $metadata ) {
if ( !$metadata ) {
public function isFileMetadataValid( $file ) {
if ( !$file->getMetadataArray() ) {
// Old metadata when we just put an empty string in there
return self::METADATA_BAD;
} else {
@ -217,9 +185,7 @@ class XCFHandler extends BitmapHandler {
* @return bool
*/
public function canRender( $file ) {
Wikimedia\suppressWarnings();
$xcfMeta = unserialize( $file->getMetadata() );
Wikimedia\restoreWarnings();
$xcfMeta = $file->getMetadataArray();
if ( isset( $xcfMeta['colorType'] ) && $xcfMeta['colorType'] === 'index-coloured' ) {
return false;
}

View file

@ -286,7 +286,7 @@ class UploadStash {
// Database is going to truncate this and make the field invalid.
// Prioritize important metadata over file handler metadata.
// File handler should be prepared to regenerate invalid metadata if needed.
$fileProps['metadata'] = false;
$fileProps['metadata'] = [];
$serializedFileProps = serialize( $fileProps );
}

View file

@ -85,11 +85,9 @@ class MWFileProps {
# Height, width and metadata
$handler = MediaHandler::getHandler( $info['mime'] );
if ( $handler ) {
$info['metadata'] = $handler->getMetadata( $fsFile, $path );
// @phan-suppress-next-line PhanParamTooMany
$gis = $handler->getImageSize( $fsFile, $path, $info['metadata'] );
if ( is_array( $gis ) ) {
$info = $this->extractImageSizeInfo( $gis ) + $info;
$sizeAndMetadata = $handler->getSizeAndMetadataWithFallback( $fsFile, $path );
if ( $sizeAndMetadata ) {
$info = $sizeAndMetadata + $info;
}
}
}
@ -97,22 +95,6 @@ class MWFileProps {
return $info;
}
/**
* Extract image size information
*
* @param array $gis
* @return array
*/
private function extractImageSizeInfo( array $gis ) {
$info = [];
# NOTE: $gis[2] contains a code for the image type. This is no longer used.
$info['width'] = $gis[0];
$info['height'] = $gis[1];
$info['bits'] = $gis['bits'] ?? 0;
return $info;
}
/**
* Empty place holder props for non-existing files
*
@ -135,7 +117,7 @@ class MWFileProps {
*/
public function newPlaceholderProps() {
return FSFile::placeholderProps() + [
'metadata' => '',
'metadata' => [],
'width' => 0,
'height' => 0,
'bits' => 0,

View file

@ -325,9 +325,7 @@ class ImportImages extends Maintenance {
$publishOptions = [];
$handler = MediaHandler::getHandler( $props['mime'] );
if ( $handler ) {
$metadata = \Wikimedia\AtEase\AtEase::quietCall( 'unserialize', $props['metadata'] );
$publishOptions['headers'] = $handler->getContentHeaders( $metadata );
$publishOptions['headers'] = $handler->getContentHeaders( $props['metadata'] );
} else {
$publishOptions['headers'] = [];
}

View file

@ -152,36 +152,11 @@ class RefreshImageMetadata extends Maintenance {
if ( $file->getUpgraded() ) {
// File was upgraded.
$upgraded++;
$newLength = strlen( $file->getMetadata() );
$oldLength = strlen( $row->img_metadata );
if ( $newLength < $oldLength - 5 ) {
// If after updating, the metadata is smaller then
// what it was before, that's probably not a good thing
// because we extract more data with time, not less.
// Thus this probably indicates an error of some sort,
// or at the very least is suspicious. Have the - 5 just
// to weed out any inconsequential changes.
$error++;
$this->output(
"Warning: File:{$row->img_name} used to have " .
"$oldLength bytes of metadata but now has $newLength bytes.\n"
);
} elseif ( $verbose ) {
$this->output( "Refreshed File:{$row->img_name}.\n" );
}
$this->output( "Refreshed File:{$row->img_name}.\n" );
} else {
$leftAlone++;
if ( $force ) {
$file->upgradeRow();
$newLength = strlen( $file->getMetadata() );
$oldLength = strlen( $row->img_metadata );
if ( $newLength < $oldLength - 5 ) {
$error++;
$this->output(
"Warning: File:{$row->img_name} used to have " .
"$oldLength bytes of metadata but now has $newLength bytes. (forced)\n"
);
}
if ( $verbose ) {
$this->output( "Forcibly refreshed File:{$row->img_name}.\n" );
}

View file

@ -1421,7 +1421,7 @@ class ParserTestRunner {
'bits' => 8,
'media_type' => MEDIATYPE_BITMAP,
'mime' => 'image/jpeg',
'metadata' => serialize( [] ),
'metadata' => [],
'sha1' => Wikimedia\base_convert( '1', 16, 36, 31 ),
'fileExists' => true
],
@ -1442,7 +1442,7 @@ class ParserTestRunner {
'bits' => 8,
'media_type' => MEDIATYPE_BITMAP,
'mime' => 'image/png',
'metadata' => serialize( [] ),
'metadata' => [],
'sha1' => Wikimedia\base_convert( '2', 16, 36, 31 ),
'fileExists' => true
],
@ -1462,7 +1462,7 @@ class ParserTestRunner {
'bits' => 0,
'media_type' => MEDIATYPE_DRAWING,
'mime' => 'image/svg+xml',
'metadata' => serialize( [
'metadata' => [
'version' => SvgHandler::SVG_METADATA_VERSION,
'width' => 240,
'height' => 180,
@ -1472,7 +1472,7 @@ class ParserTestRunner {
'en' => SVGReader::LANG_FULL_MATCH,
'ru' => SVGReader::LANG_FULL_MATCH,
],
] ),
],
'sha1' => Wikimedia\base_convert( '', 16, 36, 31 ),
'fileExists' => true
],
@ -1493,7 +1493,7 @@ class ParserTestRunner {
'bits' => 24,
'media_type' => MEDIATYPE_BITMAP,
'mime' => 'image/jpeg',
'metadata' => serialize( [] ),
'metadata' => [],
'sha1' => Wikimedia\base_convert( '3', 16, 36, 31 ),
'fileExists' => true
],
@ -1513,7 +1513,7 @@ class ParserTestRunner {
'bits' => 0,
'media_type' => MEDIATYPE_VIDEO,
'mime' => 'application/ogg',
'metadata' => serialize( [] ),
'metadata' => [],
'sha1' => Wikimedia\base_convert( '', 16, 36, 31 ),
'fileExists' => true
],
@ -1533,7 +1533,7 @@ class ParserTestRunner {
'bits' => 0,
'media_type' => MEDIATYPE_AUDIO,
'mime' => 'application/ogg',
'metadata' => serialize( [] ),
'metadata' => [],
'sha1' => Wikimedia\base_convert( '', 16, 36, 31 ),
'fileExists' => true
],
@ -1554,7 +1554,7 @@ class ParserTestRunner {
'bits' => 0,
'media_type' => MEDIATYPE_OFFICE,
'mime' => 'image/vnd.djvu',
'metadata' => '<?xml version="1.0" ?>
'metadata' => [ 'xml' => '<?xml version="1.0" ?>
<!DOCTYPE DjVuXML PUBLIC "-//W3C//DTD DjVuXML 1.1//EN" "pubtext/DjVuXML-s.dtd">
<DjVuXML>
<HEAD></HEAD>
@ -1579,7 +1579,7 @@ class ParserTestRunner {
<PARAM name="GAMMA" value="2.2" />
</OBJECT>
</BODY>
</DjVuXML>',
</DjVuXML>' ],
'sha1' => Wikimedia\base_convert( '', 16, 36, 31 ),
'fileExists' => true
],

View file

@ -561,7 +561,12 @@ class LocalFileTest extends MediaWikiIntegrationTestCase {
$this->assertSame( 10816824, $file->getSize() );
$this->assertSame( 1000, $file->getWidth() );
$this->assertSame( 1800, $file->getHeight() );
$this->assertSame( $meta, $file->getMetadata() );
$this->assertSame( unserialize( $meta ), $file->getMetadataArray() );
$this->assertSame( 'truecolour', $file->getMetadataItem( 'colorType' ) );
$this->assertSame(
[ 'loopCount' => 1, 'bitDepth' => 16 ],
$file->getMetadataItems( [ 'loopCount', 'bitDepth', 'nonexistent' ] )
);
$this->assertSame( 16, $file->getBitDepth() );
$this->assertSame( 'BITMAP', $file->getMediaType() );
$this->assertSame( 'image/png', $file->getMimeType() );
@ -578,7 +583,12 @@ class LocalFileTest extends MediaWikiIntegrationTestCase {
$this->assertSame( 10816824, $file->getSize() );
$this->assertSame( 1000, $file->getWidth() );
$this->assertSame( 1800, $file->getHeight() );
$this->assertSame( $meta, $file->getMetadata() );
$this->assertSame( unserialize( $meta ), $file->getMetadataArray() );
$this->assertSame( 'truecolour', $file->getMetadataItem( 'colorType' ) );
$this->assertSame(
[ 'loopCount' => 1, 'bitDepth' => 16 ],
$file->getMetadataItems( [ 'loopCount', 'bitDepth', 'nonexistent' ] )
);
$this->assertSame( 16, $file->getBitDepth() );
$this->assertSame( 'BITMAP', $file->getMediaType() );
$this->assertSame( 'image/png', $file->getMimeType() );
@ -592,4 +602,31 @@ class LocalFileTest extends MediaWikiIntegrationTestCase {
$file = $repo->findFile( $title );
$this->assertSame( false, $file );
}
public function provideLegacyMetadataRoundTrip() {
return [
[ '0' ],
[ '-1' ],
[ '' ]
];
}
/**
* Test the legacy function LocalFile::getMetadata()
* @dataProvider provideLegacyMetadataRoundTrip
* @covers LocalFile
*/
public function testLegacyMetadataRoundTrip( $meta ) {
$file = new class( $meta ) extends LocalFile {
public function __construct( $meta ) {
$repo = MediaWikiServices::getInstance()->getRepoGroup()->getLocalRepo();
parent::__construct(
Title::newFromText( 'File:TestLegacyMetadataRoundTrip' ),
$repo );
$this->loadMetadataFromString( $meta );
$this->dataLoaded = true;
}
};
$this->assertSame( $meta, $file->getMetadata() );
}
}

View file

@ -128,6 +128,8 @@ class BitmapMetadataHandlerTest extends MediaWikiIntegrationTestCase {
$result = BitmapMetadataHandler::PNG( $this->filePath . 'xmp.png' );
$expected = [
'width' => 50,
'height' => 50,
'frameCount' => 0,
'loopCount' => 1,
'duration' => 0,

View file

@ -24,19 +24,20 @@ class DjVuTest extends MediaWikiMediaTestCase {
$this->handler = new DjVuHandler();
}
public function testGetImageSize() {
$this->assertSame(
[ 2480, 3508, 'DjVu', 'width="2480" height="3508"' ],
$this->handler->getImageSize( null, $this->filePath . '/LoremIpsum.djvu' ),
'Test file LoremIpsum.djvu should have a size of 2480 * 3508'
);
public function testGetSizeAndMetadata() {
$info = $this->handler->getSizeAndMetadata(
new TrivialMediaHandlerState, $this->filePath . '/LoremIpsum.djvu' );
$this->assertSame( 2480, $info['width'] );
$this->assertSame( 3508, $info['height'] );
$this->assertIsString( $info['metadata']['xml'] );
}
public function testInvalidFile() {
$this->assertEquals(
'a:1:{s:5:"error";s:25:"Error extracting metadata";}',
$this->handler->getMetadata( null, $this->filePath . '/some-nonexistent-file' ),
'Getting metadata for an inexistent file should return false'
[ 'metadata' => [ 'error' => 'Error extracting metadata' ] ],
$this->handler->getSizeAndMetadata(
new TrivialMediaHandlerState, $this->filePath . '/some-nonexistent-file' ),
'Getting metadata for a nonexistent file should return false'
);
}

View file

@ -2,6 +2,7 @@
/**
* @group Media
* @covers ExifBitmapHandler
*/
class ExifBitmapTest extends MediaWikiMediaTestCase {
@ -19,64 +20,47 @@ class ExifBitmapTest extends MediaWikiMediaTestCase {
$this->handler = new ExifBitmapHandler;
}
/**
* @covers ExifBitmapHandler::isMetadataValid
*/
public function testIsOldBroken() {
$res = $this->handler->isMetadataValid( null, ExifBitmapHandler::OLD_BROKEN_FILE );
$this->assertEquals( ExifBitmapHandler::METADATA_COMPATIBLE, $res );
public function provideIsFileMetadataValid() {
return [
'old broken' => [
ExifBitmapHandler::OLD_BROKEN_FILE,
ExifBitmapHandler::METADATA_COMPATIBLE
],
'broken' => [
ExifBitmapHandler::BROKEN_FILE,
ExifBitmapHandler::METADATA_GOOD
],
'invalid' => [
'Something Invalid Here.',
ExifBitmapHandler::METADATA_BAD
],
'good' => [
// phpcs:ignore Generic.Files.LineLength
'a:16:{s:10:"ImageWidth";i:20;s:11:"ImageLength";i:20;s:13:"BitsPerSample";a:3:{i:0;i:8;i:1;i:8;i:2;i:8;}s:11:"Compression";i:5;s:25:"PhotometricInterpretation";i:2;s:16:"ImageDescription";s:17:"Created with GIMP";s:12:"StripOffsets";i:8;s:11:"Orientation";i:1;s:15:"SamplesPerPixel";i:3;s:12:"RowsPerStrip";i:64;s:15:"StripByteCounts";i:238;s:11:"XResolution";s:19:"1207959552/16777216";s:11:"YResolution";s:19:"1207959552/16777216";s:19:"PlanarConfiguration";i:1;s:14:"ResolutionUnit";i:2;s:22:"MEDIAWIKI_EXIF_VERSION";i:2;}',
ExifBitmapHandler::METADATA_GOOD
],
'old good' => [
// phpcs:ignore Generic.Files.LineLength
'a:16:{s:10:"ImageWidth";i:20;s:11:"ImageLength";i:20;s:13:"BitsPerSample";a:3:{i:0;i:8;i:1;i:8;i:2;i:8;}s:11:"Compression";i:5;s:25:"PhotometricInterpretation";i:2;s:16:"ImageDescription";s:17:"Created with GIMP";s:12:"StripOffsets";i:8;s:11:"Orientation";i:1;s:15:"SamplesPerPixel";i:3;s:12:"RowsPerStrip";i:64;s:15:"StripByteCounts";i:238;s:11:"XResolution";s:19:"1207959552/16777216";s:11:"YResolution";s:19:"1207959552/16777216";s:19:"PlanarConfiguration";i:1;s:14:"ResolutionUnit";i:2;s:22:"MEDIAWIKI_EXIF_VERSION";i:1;}',
ExifBitmapHandler::METADATA_COMPATIBLE
],
// Handle metadata from paged tiff handler (gotten via instant commons) gracefully.
'paged tiff' => [
// phpcs:ignore Generic.Files.LineLength
'a:6:{s:9:"page_data";a:1:{i:1;a:5:{s:5:"width";i:643;s:6:"height";i:448;s:5:"alpha";s:4:"true";s:4:"page";i:1;s:6:"pixels";i:288064;}}s:10:"page_count";i:1;s:10:"first_page";i:1;s:9:"last_page";i:1;s:4:"exif";a:9:{s:10:"ImageWidth";i:643;s:11:"ImageLength";i:448;s:11:"Compression";i:5;s:25:"PhotometricInterpretation";i:2;s:11:"Orientation";i:1;s:15:"SamplesPerPixel";i:4;s:12:"RowsPerStrip";i:50;s:19:"PlanarConfiguration";i:1;s:22:"MEDIAWIKI_EXIF_VERSION";i:1;}s:21:"TIFF_METADATA_VERSION";s:3:"1.4";}',
ExifBitmapHandler::METADATA_BAD
],
];
}
/**
* @covers ExifBitmapHandler::isMetadataValid
*/
public function testIsBrokenFile() {
$res = $this->handler->isMetadataValid( null, ExifBitmapHandler::BROKEN_FILE );
$this->assertEquals( ExifBitmapHandler::METADATA_GOOD, $res );
/** @dataProvider provideIsFileMetadataValid */
public function testIsFileMetadataValid( $serializedMetadata, $expected ) {
$file = $this->getMockFileWithMetadata( $serializedMetadata );
$res = $this->handler->isFileMetadataValid( $file );
$this->assertEquals( $expected, $res );
}
/**
* @covers ExifBitmapHandler::isMetadataValid
*/
public function testIsInvalid() {
$res = $this->handler->isMetadataValid( null, 'Something Invalid Here.' );
$this->assertEquals( ExifBitmapHandler::METADATA_BAD, $res );
}
/**
* @covers ExifBitmapHandler::isMetadataValid
*/
public function testGoodMetadata() {
// phpcs:ignore Generic.Files.LineLength
$meta = 'a:16:{s:10:"ImageWidth";i:20;s:11:"ImageLength";i:20;s:13:"BitsPerSample";a:3:{i:0;i:8;i:1;i:8;i:2;i:8;}s:11:"Compression";i:5;s:25:"PhotometricInterpretation";i:2;s:16:"ImageDescription";s:17:"Created with GIMP";s:12:"StripOffsets";i:8;s:11:"Orientation";i:1;s:15:"SamplesPerPixel";i:3;s:12:"RowsPerStrip";i:64;s:15:"StripByteCounts";i:238;s:11:"XResolution";s:19:"1207959552/16777216";s:11:"YResolution";s:19:"1207959552/16777216";s:19:"PlanarConfiguration";i:1;s:14:"ResolutionUnit";i:2;s:22:"MEDIAWIKI_EXIF_VERSION";i:2;}';
$res = $this->handler->isMetadataValid( null, $meta );
$this->assertEquals( ExifBitmapHandler::METADATA_GOOD, $res );
}
/**
* @covers ExifBitmapHandler::isMetadataValid
*/
public function testIsOldGood() {
// phpcs:ignore Generic.Files.LineLength
$meta = 'a:16:{s:10:"ImageWidth";i:20;s:11:"ImageLength";i:20;s:13:"BitsPerSample";a:3:{i:0;i:8;i:1;i:8;i:2;i:8;}s:11:"Compression";i:5;s:25:"PhotometricInterpretation";i:2;s:16:"ImageDescription";s:17:"Created with GIMP";s:12:"StripOffsets";i:8;s:11:"Orientation";i:1;s:15:"SamplesPerPixel";i:3;s:12:"RowsPerStrip";i:64;s:15:"StripByteCounts";i:238;s:11:"XResolution";s:19:"1207959552/16777216";s:11:"YResolution";s:19:"1207959552/16777216";s:19:"PlanarConfiguration";i:1;s:14:"ResolutionUnit";i:2;s:22:"MEDIAWIKI_EXIF_VERSION";i:1;}';
$res = $this->handler->isMetadataValid( null, $meta );
$this->assertEquals( ExifBitmapHandler::METADATA_COMPATIBLE, $res );
}
/**
* Handle metadata from paged tiff handler (gotten via instant commons) gracefully.
* @covers ExifBitmapHandler::isMetadataValid
*/
public function testPagedTiffHandledGracefully() {
// phpcs:ignore Generic.Files.LineLength
$meta = 'a:6:{s:9:"page_data";a:1:{i:1;a:5:{s:5:"width";i:643;s:6:"height";i:448;s:5:"alpha";s:4:"true";s:4:"page";i:1;s:6:"pixels";i:288064;}}s:10:"page_count";i:1;s:10:"first_page";i:1;s:9:"last_page";i:1;s:4:"exif";a:9:{s:10:"ImageWidth";i:643;s:11:"ImageLength";i:448;s:11:"Compression";i:5;s:25:"PhotometricInterpretation";i:2;s:11:"Orientation";i:1;s:15:"SamplesPerPixel";i:4;s:12:"RowsPerStrip";i:50;s:19:"PlanarConfiguration";i:1;s:22:"MEDIAWIKI_EXIF_VERSION";i:1;}s:21:"TIFF_METADATA_VERSION";s:3:"1.4";}';
$res = $this->handler->isMetadataValid( null, $meta );
$this->assertEquals( ExifBitmapHandler::METADATA_BAD, $res );
}
/**
* @covers ExifBitmapHandler::convertMetadataVersion
*/
public function testConvertMetadataLatest() {
$metadata = [
'foo' => [ 'First', 'Second', '_type' => 'ol' ],
@ -86,9 +70,6 @@ class ExifBitmapTest extends MediaWikiMediaTestCase {
$this->assertEquals( $metadata, $res );
}
/**
* @covers ExifBitmapHandler::convertMetadataVersion
*/
public function testConvertMetadataToOld() {
$metadata = [
'foo' => [ 'First', 'Second', '_type' => 'ol' ],
@ -108,9 +89,6 @@ class ExifBitmapTest extends MediaWikiMediaTestCase {
$this->assertEquals( $expected, $res );
}
/**
* @covers ExifBitmapHandler::convertMetadataVersion
*/
public function testConvertMetadataSoftware() {
$metadata = [
'Software' => [ [ 'GIMP', '1.1' ] ],
@ -124,9 +102,6 @@ class ExifBitmapTest extends MediaWikiMediaTestCase {
$this->assertEquals( $expected, $res );
}
/**
* @covers ExifBitmapHandler::convertMetadataVersion
*/
public function testConvertMetadataSoftwareNormal() {
$metadata = [
'Software' => [ "GIMP 1.2", "vim" ],

View file

@ -15,19 +15,18 @@ class GIFHandlerTest extends MediaWikiMediaTestCase {
}
/**
* @return string Value of GIFHandler::BROKEN_FILE
* @return array Unserialized metadata array for GIFHandler::BROKEN_FILE
*/
private function brokenFile() : string {
$const = new ReflectionClassConstant( GIFHandler::class, 'BROKEN_FILE' );
return $const->getValue();
private function brokenFile() : array {
return [ '_error' => 0 ];
}
/**
* @covers GIFHandler::getMetadata
* @covers GIFHandler::getSizeAndMetadata
*/
public function testInvalidFile() {
$res = $this->handler->getMetadata( null, $this->filePath . '/README' );
$this->assertEquals( $this->brokenFile(), $res );
$res = $this->handler->getSizeAndMetadata( null, $this->filePath . '/README' );
$this->assertEquals( $this->brokenFile(), $res['metadata'] );
}
/**
@ -36,7 +35,7 @@ class GIFHandlerTest extends MediaWikiMediaTestCase {
* @dataProvider provideIsAnimated
* @covers GIFHandler::isAnimatedImage
*/
public function testIsAnimanted( $filename, $expected ) {
public function testIsAnimated( $filename, $expected ) {
$file = $this->dataFile( $filename, 'image/gif' );
$actual = $this->handler->isAnimatedImage( $file );
$this->assertEquals( $expected, $actual );
@ -71,20 +70,21 @@ class GIFHandlerTest extends MediaWikiMediaTestCase {
/**
* @param string $metadata Serialized metadata
* @param int $expected One of the class constants of GIFHandler
* @dataProvider provideIsMetadataValid
* @covers GIFHandler::isMetadataValid
* @dataProvider provideIsFileMetadataValid
* @covers GIFHandler::isFileMetadataValid
*/
public function testIsMetadataValid( $metadata, $expected ) {
$actual = $this->handler->isMetadataValid( null, $metadata );
public function testIsFileMetadataValid( $metadata, $expected ) {
$file = $this->getMockFileWithMetadata( $metadata );
$actual = $this->handler->isFileMetadataValid( $file );
$this->assertEquals( $expected, $actual );
}
public function provideIsMetadataValid() {
public function provideIsFileMetadataValid() {
// phpcs:disable Generic.Files.LineLength
return [
[ $this->brokenFile(), GIFHandler::METADATA_GOOD ],
[ '0', GIFHandler::METADATA_GOOD ],
[ '', GIFHandler::METADATA_BAD ],
[ null, GIFHandler::METADATA_BAD ],
[ 'a:0:{}', GIFHandler::METADATA_BAD ],
[ 'Something invalid!', GIFHandler::METADATA_BAD ],
[
'a:4:{s:10:"frameCount";i:1;s:6:"looped";b:0;s:8:"duration";d:0.1000000000000000055511151231257827021181583404541015625;s:8:"metadata";a:2:{s:14:"GIFFileComment";a:1:{i:0;s:35:"GIF test file ⁕ Created with GIMP";}s:15:"_MW_GIF_VERSION";i:1;}}',
@ -96,26 +96,61 @@ class GIFHandlerTest extends MediaWikiMediaTestCase {
/**
* @param string $filename
* @param string $expected Serialized array
* @dataProvider provideGetMetadata
* @covers GIFHandler::getMetadata
* @param array $expected Unserialized metadata
* @dataProvider provideGetSizeAndMetadata
* @covers GIFHandler::getSizeAndMetadata
*/
public function testGetMetadata( $filename, $expected ) {
public function testGetSizeAndMetadata( $filename, $expected ) {
$file = $this->dataFile( $filename, 'image/gif' );
$actual = $this->handler->getMetadata( $file, "$this->filePath/$filename" );
$this->assertEquals( unserialize( $expected ), unserialize( $actual ) );
$actual = $this->handler->getSizeAndMetadata( $file, "$this->filePath/$filename" );
$this->assertEquals( $expected, $actual );
}
public static function provideGetMetadata() {
public static function provideGetSizeAndMetadata() {
// phpcs:disable Generic.Files.LineLength
return [
[
'nonanimated.gif',
'a:4:{s:10:"frameCount";i:1;s:6:"looped";b:0;s:8:"duration";d:0.1000000000000000055511151231257827021181583404541015625;s:8:"metadata";a:2:{s:14:"GIFFileComment";a:1:{i:0;s:35:"GIF test file ⁕ Created with GIMP";}s:15:"_MW_GIF_VERSION";i:1;}}'
[
'width' => 45,
'height' => 30,
'bits' => 1,
'metadata' => [
'frameCount' => 1,
'looped' => false,
'duration' => 0.1,
'metadata' => [
'GIFFileComment' => [ 'GIF test file ⁕ Created with GIMP' ],
'_MW_GIF_VERSION' => 1,
],
],
]
],
[
'animated-xmp.gif',
'a:4:{s:10:"frameCount";i:4;s:6:"looped";b:1;s:8:"duration";d:2.399999999999999911182158029987476766109466552734375;s:8:"metadata";a:5:{s:6:"Artist";s:7:"Bawolff";s:16:"ImageDescription";a:2:{s:9:"x-default";s:18:"A file to test GIF";s:5:"_type";s:4:"lang";}s:15:"SublocationDest";s:13:"The interwebs";s:14:"GIFFileComment";a:1:{i:0;s:16:"GIƒ·test·file";}s:15:"_MW_GIF_VERSION";i:1;}}'
[
'width' => 45,
'height' => 30,
'bits' => 1,
'metadata' => [
'frameCount' => 4,
'looped' => true,
'duration' => 2.4,
'metadata' => [
'Artist' => 'Bawolff',
'ImageDescription' => [
'x-default' => 'A file to test GIF',
'_type' => 'lang',
],
'SublocationDest' => 'The interwebs',
'GIFFileComment' => [
0 => 'GIƒ·test·file',
],
'_MW_GIF_VERSION' => 1,
],
],
],
],
];
// phpcs:enable

View file

@ -16,54 +16,39 @@ class Jpeg2000HandlerTest extends MediaWikiIntegrationTestCase {
}
/**
* @dataProvider provideTestGetImageSize
* @dataProvider provideTestGetSizeAndMetadata
*/
public function testGetImageSize( $path, $expectedResult ) {
public function testGetSizeAndMetadata( $path, $expectedResult ) {
$handler = new Jpeg2000Handler();
$this->assertEquals( $expectedResult, $handler->getImageSize( null, $path ) );
$this->assertEquals( $expectedResult, $handler->getSizeAndMetadata(
new TrivialMediaHandlerState, $path ) );
}
public function provideTestGetImageSize() {
public function provideTestGetSizeAndMetadata() {
return [
[ __DIR__ . '/../../data/media/jpeg2000-lossless.jp2', [
0 => 100,
1 => 100,
2 => 10,
3 => 'width="100" height="100"',
'width' => 100,
'height' => 100,
'bits' => 8,
'channels' => 3,
'mime' => 'image/jp2'
] ],
[ __DIR__ . '/../../data/media/jpeg2000-lossy.jp2', [
0 => 100,
1 => 100,
2 => 10,
3 => 'width="100" height="100"',
'width' => 100,
'height' => 100,
'bits' => 8,
'channels' => 3,
'mime' => 'image/jp2'
] ],
[ __DIR__ . '/../../data/media/jpeg2000-alpha.jp2', [
0 => 100,
1 => 100,
2 => 10,
3 => 'width="100" height="100"',
'width' => 100,
'height' => 100,
'bits' => 8,
'channels' => 4,
'mime' => 'image/jp2'
] ],
[ __DIR__ . '/../../data/media/jpeg2000-profile.jpf', [
0 => 100,
1 => 100,
2 => 10,
3 => 'width="100" height="100"',
'width' => 100,
'height' => 100,
'bits' => 8,
'channels' => 4,
'mime' => 'image/jp2'
] ],
// Error cases
[ __FILE__, false ],
[ __FILE__, [] ],
];
}
}

View file

@ -5,6 +5,8 @@
* @covers JpegHandler
*/
class JpegTest extends MediaWikiMediaTestCase {
/** @var JpegHandler */
private $handler;
protected function setUp() : void {
parent::setUp();
@ -17,18 +19,27 @@ class JpegTest extends MediaWikiMediaTestCase {
public function testInvalidFile() {
$file = $this->dataFile( 'README', 'image/jpeg' );
$res = $this->handler->getMetadata( $file, $this->filePath . 'README' );
$this->assertEquals( ExifBitmapHandler::BROKEN_FILE, $res );
$res = $this->handler->getSizeAndMetadataWithFallback( $file, $this->filePath . 'README' );
$this->assertEquals( [ '_error' => ExifBitmapHandler::BROKEN_FILE ], $res['metadata'] );
}
public function testJpegMetadataExtraction() {
$file = $this->dataFile( 'test.jpg', 'image/jpeg' );
$res = $this->handler->getMetadata( $file, $this->filePath . 'test.jpg' );
// phpcs:ignore Generic.Files.LineLength
$expected = 'a:7:{s:16:"ImageDescription";s:9:"Test file";s:11:"XResolution";s:4:"72/1";s:11:"YResolution";s:4:"72/1";s:14:"ResolutionUnit";i:2;s:16:"YCbCrPositioning";i:1;s:15:"JPEGFileComment";a:1:{i:0;s:17:"Created with GIMP";}s:22:"MEDIAWIKI_EXIF_VERSION";i:2;}';
$res = $this->handler->getSizeAndMetadataWithFallback( $file, $this->filePath . 'test.jpg' );
$expected = [
'ImageDescription' => 'Test file',
'XResolution' => '72/1',
'YResolution' => '72/1',
'ResolutionUnit' => 2,
'YCbCrPositioning' => 1,
'JPEGFileComment' => [
0 => 'Created with GIMP',
],
'MEDIAWIKI_EXIF_VERSION' => 2,
];
// Unserialize in case serialization format ever changes.
$this->assertEquals( unserialize( $expected ), unserialize( $res ) );
$this->assertEquals( $expected, $res['metadata'] );
}
/**

View file

@ -77,4 +77,23 @@ abstract class MediaWikiMediaTestCase extends MediaWikiIntegrationTestCase {
return new UnregisteredLocalFile( false, $this->repo,
"mwstore://localtesting/data/$name", $type );
}
/**
* Get a mock LocalFile with the specified metadata, specified as a
* serialized string. The metadata-related methods will return this
* metadata. The behaviour of the other methods is undefined.
*
* @since 1.37
* @param string $metadata
* @return LocalFile
*/
protected function getMockFileWithMetadata( $metadata ) {
return new class( $metadata ) extends LocalFile {
public function __construct( $metadata ) {
$this->loadMetadataFromString( $metadata );
$this->dataLoaded = true;
}
};
}
}

View file

@ -14,19 +14,23 @@ class PNGHandlerTest extends MediaWikiMediaTestCase {
}
/**
* @return string Value of PNGHandler::BROKEN_FILE
* @return array Expected metadata for a broken file. This tests backwards
* compatibility with existing DB rows, so can't be changed.
*/
private function brokenFile() : string {
$const = new ReflectionClassConstant( PNGHandler::class, 'BROKEN_FILE' );
return $const->getValue();
private function brokenFile() {
return [ '_error' => '0' ];
}
/**
* @covers PNGHandler::getMetadata
* @covers PNGHandler::getSizeAndMetadata
*/
public function testInvalidFile() {
$res = $this->handler->getMetadata( null, $this->filePath . '/README' );
$this->assertEquals( $this->brokenFile(), $res );
$res = $this->handler->getSizeAndMetadata( null, $this->filePath . '/README' );
$this->assertEquals(
[
'metadata' => $this->brokenFile()
],
$res );
}
/**
@ -56,7 +60,7 @@ class PNGHandlerTest extends MediaWikiMediaTestCase {
*/
public function testGetImageArea( $filename, $expected ) {
$file = $this->dataFile( $filename, 'image/png' );
$actual = $this->handler->getImageArea( $file, $file->getWidth(), $file->getHeight() );
$actual = $this->handler->getImageArea( $file );
$this->assertEquals( $expected, $actual );
}
@ -73,19 +77,19 @@ class PNGHandlerTest extends MediaWikiMediaTestCase {
* @param string $metadata Serialized metadata
* @param int $expected One of the class constants of PNGHandler
* @dataProvider provideIsMetadataValid
* @covers PNGHandler::isMetadataValid
* @covers PNGHandler::isFileMetadataValid
*/
public function testIsMetadataValid( $metadata, $expected ) {
$actual = $this->handler->isMetadataValid( null, $metadata );
public function testIsFileMetadataValid( $metadata, $expected ) {
$actual = $this->handler->isFileMetadataValid( $this->getMockFileWithMetadata( $metadata ) );
$this->assertEquals( $expected, $actual );
}
public function provideIsMetadataValid() {
// phpcs:disable Generic.Files.LineLength
return [
[ $this->brokenFile(), PNGHandler::METADATA_GOOD ],
[ '0', PNGHandler::METADATA_GOOD ],
[ '', PNGHandler::METADATA_BAD ],
[ null, PNGHandler::METADATA_BAD ],
[ 'a:0:{}', PNGHandler::METADATA_BAD ],
[ 'Something invalid!', PNGHandler::METADATA_BAD ],
[
'a:6:{s:10:"frameCount";i:0;s:9:"loopCount";i:1;s:8:"duration";d:0;s:8:"bitDepth";i:8;s:9:"colorType";s:10:"truecolour";s:8:"metadata";a:1:{s:15:"_MW_PNG_VERSION";i:1;}}',
@ -98,29 +102,55 @@ class PNGHandlerTest extends MediaWikiMediaTestCase {
/**
* @param string $filename
* @param string $expected Serialized array
* @dataProvider provideGetMetadata
* @covers PNGHandler::getMetadata
* @dataProvider provideGetSizeAndMetadata
* @covers PNGHandler::getSizeAndMetadata
*/
public function testGetMetadata( $filename, $expected ) {
public function testGetSizeAndMetadata( $filename, $expected ) {
$file = $this->dataFile( $filename, 'image/png' );
$actual = $this->handler->getMetadata( $file, "$this->filePath/$filename" );
// $this->assertEquals( unserialize( $expected ), unserialize( $actual ) );
$this->assertEquals( ( $expected ), ( $actual ) );
$actual = $this->handler->getSizeAndMetadata( $file, "$this->filePath/$filename" );
$this->assertEquals( $expected, $actual );
}
public static function provideGetMetadata() {
// phpcs:disable Generic.Files.LineLength
public static function provideGetSizeAndMetadata() {
return [
[
'rgb-na-png.png',
'a:6:{s:10:"frameCount";i:0;s:9:"loopCount";i:1;s:8:"duration";d:0;s:8:"bitDepth";i:8;s:9:"colorType";s:10:"truecolour";s:8:"metadata";a:1:{s:15:"_MW_PNG_VERSION";i:1;}}'
[
'width' => 50,
'height' => 50,
'bits' => 8,
'metadata' => [
'frameCount' => 0,
'loopCount' => 1,
'duration' => 0.0,
'bitDepth' => 8,
'colorType' => 'truecolour',
'metadata' => [
'_MW_PNG_VERSION' => 1,
],
],
],
],
[
'xmp.png',
'a:6:{s:10:"frameCount";i:0;s:9:"loopCount";i:1;s:8:"duration";d:0;s:8:"bitDepth";i:1;s:9:"colorType";s:14:"index-coloured";s:8:"metadata";a:2:{s:12:"SerialNumber";s:9:"123456789";s:15:"_MW_PNG_VERSION";i:1;}}'
[
'width' => 50,
'height' => 50,
'bits' => 1,
'metadata' => [
'frameCount' => 0,
'loopCount' => 1,
'duration' => 0.0,
'bitDepth' => 1,
'colorType' => 'index-coloured',
'metadata' => [
'SerialNumber' => '123456789',
'_MW_PNG_VERSION' => 1,
],
]
]
],
];
// phpcs:enable
}
/**

View file

@ -195,21 +195,21 @@ class SvgHandlerTest extends MediaWikiMediaTestCase {
$file = $this->getMockBuilder( File::class )
->disableOriginalConstructor()
->onlyMethods( [ 'getWidth', 'getHeight', 'getMetadata', 'getHandler' ] )
->onlyMethods( [ 'getWidth', 'getHeight', 'getMetadataArray', 'getHandler' ] )
->getMock();
$file->method( 'getWidth' )
->willReturn( $width );
$file->method( 'getHeight' )
->willReturn( $height );
$file->method( 'getMetadata' )
->willReturn( serialize( [
$file->method( 'getMetadataArray' )
->willReturn( [
'version' => SvgHandler::SVG_METADATA_VERSION,
'translations' => [
'en' => SVGReader::LANG_FULL_MATCH,
'ru' => SVGReader::LANG_FULL_MATCH,
],
] ) );
] );
$file->method( 'getHandler' )
->willReturn( $handler );
@ -295,10 +295,10 @@ class SvgHandlerTest extends MediaWikiMediaTestCase {
$metadata['version'] = SvgHandler::SVG_METADATA_VERSION;
$file = $this->getMockBuilder( File::class )
->disableOriginalConstructor()
->onlyMethods( [ 'getMetadata' ] )
->onlyMethods( [ 'getMetadataArray' ] )
->getMock();
$file->method( 'getMetadata' )
->willReturn( serialize( $metadata ) );
$file->method( 'getMetadataArray' )
->willReturn( $metadata );
$handler = new SvgHandler();
/** @var File $file */

View file

@ -21,24 +21,48 @@ class TiffTest extends MediaWikiIntegrationTestCase {
}
/**
* @covers TiffHandler::getMetadata
* @covers TiffHandler::getSizeAndMetadata
*/
public function testInvalidFile() {
$res = $this->handler->getMetadata( null, $this->filePath . 'README' );
$this->assertEquals( ExifBitmapHandler::BROKEN_FILE, $res );
$res = $this->handler->getSizeAndMetadata( null, $this->filePath . 'README' );
$this->assertEquals( [ 'metadata' => [ '_error' => ExifBitmapHandler::BROKEN_FILE ] ], $res );
}
/**
* @covers TiffHandler::getMetadata
* @covers TiffHandler::getSizeAndMetadata
*/
public function testTiffMetadataExtraction() {
$res = $this->handler->getMetadata( null, $this->filePath . 'test.tiff' );
$res = $this->handler->getSizeAndMetadata( null, $this->filePath . 'test.tiff' );
// phpcs:ignore Generic.Files.LineLength
$expected = 'a:16:{s:10:"ImageWidth";i:20;s:11:"ImageLength";i:20;s:13:"BitsPerSample";a:3:{i:0;i:8;i:1;i:8;i:2;i:8;}s:11:"Compression";i:5;s:25:"PhotometricInterpretation";i:2;s:16:"ImageDescription";s:17:"Created with GIMP";s:12:"StripOffsets";i:8;s:11:"Orientation";i:1;s:15:"SamplesPerPixel";i:3;s:12:"RowsPerStrip";i:64;s:15:"StripByteCounts";i:238;s:11:"XResolution";s:19:"1207959552/16777216";s:11:"YResolution";s:19:"1207959552/16777216";s:19:"PlanarConfiguration";i:1;s:14:"ResolutionUnit";i:2;s:22:"MEDIAWIKI_EXIF_VERSION";i:2;}';
$expected = [
'width' => 20,
'height' => 20,
'metadata' => [
'ImageWidth' => 20,
'ImageLength' => 20,
'BitsPerSample' => [
0 => 8,
1 => 8,
2 => 8,
],
'Compression' => 5,
'PhotometricInterpretation' => 2,
'ImageDescription' => 'Created with GIMP',
'StripOffsets' => 8,
'Orientation' => 1,
'SamplesPerPixel' => 3,
'RowsPerStrip' => 64,
'StripByteCounts' => 238,
'XResolution' => '1207959552/16777216',
'YResolution' => '1207959552/16777216',
'PlanarConfiguration' => 1,
'ResolutionUnit' => 2,
'MEDIAWIKI_EXIF_VERSION' => 2,
]
];
// Re-unserialize in case there are subtle differences between how versions
// of php serialize stuff.
$this->assertEquals( unserialize( $expected ), unserialize( $res ) );
$this->assertEquals( $expected, $res );
}
}

View file

@ -106,22 +106,71 @@ class WebPHandlerTest extends MediaWikiIntegrationTestCase {
}
/**
* @dataProvider provideTestGetImageSize
* @dataProvider provideTestGetSizeAndMetadata
*/
public function testGetImageSize( $path, $expectedResult ) {
public function testGetSizeAndMetadata( $path, $expectedResult ) {
$handler = new WebPHandler();
$this->assertEquals( $expectedResult, $handler->getImageSize( null, $path ) );
$this->assertEquals( $expectedResult, $handler->getSizeAndMetadata( null, $path ) );
}
public function provideTestGetImageSize() {
public function provideTestGetSizeAndMetadata() {
return [
// Public domain files from https://developers.google.com/speed/webp/gallery2
[ __DIR__ . '/../../data/media/2_webp_a.webp', [ 386, 395 ] ],
[ __DIR__ . '/../../data/media/2_webp_ll.webp', [ 386, 395 ] ],
[ __DIR__ . '/../../data/media/webp_animated.webp', [ 300, 225 ] ],
[
__DIR__ . '/../../data/media/2_webp_a.webp',
[
'width' => 386,
'height' => 395,
'metadata' => [
'compression' => 'lossy',
'animated' => false,
'transparency' => true,
'width' => 386,
'height' => 395,
'metadata' => [
'_MW_WEBP_VERSION' => 1,
],
],
]
],
[
__DIR__ . '/../../data/media/2_webp_ll.webp',
[
'width' => 386,
'height' => 395,
'metadata' => [
'compression' => 'lossless',
'width' => 386,
'height' => 395,
'metadata' => [
'_MW_WEBP_VERSION' => 1,
],
],
]
],
[
__DIR__ . '/../../data/media/webp_animated.webp',
[
'width' => 300,
'height' => 225,
'metadata' => [
'compression' => 'unknown',
'animated' => true,
'transparency' => true,
'width' => 300,
'height' => 225,
'metadata' => [
'_MW_WEBP_VERSION' => 1,
],
],
]
],
// Error cases
[ __FILE__, false ],
[
__FILE__,
[ 'metadata' => [ '_error' => '0' ] ],
],
];
}

View file

@ -15,38 +15,66 @@ class XCFHandlerTest extends MediaWikiMediaTestCase {
/**
* @param string $filename
* @param int $expectedWidth Width
* @param int $expectedHeight Height
* @dataProvider provideGetImageSize
* @covers XCFHandler::getImageSize
* @param array $expected
* @dataProvider provideGetSizeAndMetadata
* @covers XCFHandler::getSizeAndMetadata
*/
public function testGetImageSize( $filename, $expectedWidth, $expectedHeight ) {
public function testGetSizeAndMetadata( $filename, $expected ) {
$file = $this->dataFile( $filename, 'image/x-xcf' );
$actual = $this->handler->getImageSize( $file, $file->getLocalRefPath() );
$this->assertEquals( $expectedWidth, $actual[0] );
$this->assertEquals( $expectedHeight, $actual[1] );
$actual = $this->handler->getSizeAndMetadata( $file, $file->getLocalRefPath() );
$this->assertSame( $expected, $actual );
}
public static function provideGetImageSize() {
public static function provideGetSizeAndMetadata() {
return [
[ '80x60-2layers.xcf', 80, 60 ],
[ '80x60-RGB.xcf', 80, 60 ],
[ '80x60-Greyscale.xcf', 80, 60 ],
[
'80x60-2layers.xcf',
[
'width' => 80,
'height' => 60,
'bits' => 8,
'metadata' => [
'colorType' => 'truecolour-alpha',
]
],
],
[
'80x60-RGB.xcf',
[
'width' => 80,
'height' => 60,
'bits' => 8,
'metadata' => [
'colorType' => 'truecolour-alpha',
]
],
],
[
'80x60-Greyscale.xcf',
[
'width' => 80,
'height' => 60,
'bits' => 8,
'metadata' => [
'colorType' => 'greyscale-alpha',
]
]
],
];
}
/**
* @param string $metadata Serialized metadata
* @param int $expected One of the class constants of XCFHandler
* @dataProvider provideIsMetadataValid
* @covers XCFHandler::isMetadataValid
* @dataProvider provideIsFileMetadataValid
* @covers XCFHandler::isFileMetadataValid
*/
public function testIsMetadataValid( $metadata, $expected ) {
$actual = $this->handler->isMetadataValid( null, $metadata );
public function testIsFileMetadataValid( $metadata, $expected ) {
$actual = $this->handler->isFileMetadataValid( $this->getMockFileWithMetadata( $metadata ) );
$this->assertEquals( $expected, $actual );
}
public static function provideIsMetadataValid() {
public static function provideIsFileMetadataValid() {
return [
[ '', XCFHandler::METADATA_BAD ],
[ serialize( [ 'error' => true ] ), XCFHandler::METADATA_GOOD ],
@ -54,30 +82,4 @@ class XCFHandlerTest extends MediaWikiMediaTestCase {
[ serialize( [ 'colorType' => 'greyscale-alpha' ] ), XCFHandler::METADATA_GOOD ],
];
}
/**
* @param string $filename
* @param string $expected Serialized array
* @dataProvider provideGetMetadata
* @covers XCFHandler::getMetadata
*/
public function testGetMetadata( $filename, $expected ) {
$file = $this->dataFile( $filename, 'image/png' );
$actual = $this->handler->getMetadata( $file, "$this->filePath/$filename" );
$this->assertEquals( $expected, $actual );
}
public static function provideGetMetadata() {
return [
[ '80x60-2layers.xcf',
'a:1:{s:9:"colorType";s:16:"truecolour-alpha";}'
],
[ '80x60-RGB.xcf',
'a:1:{s:9:"colorType";s:16:"truecolour-alpha";}'
],
[ '80x60-Greyscale.xcf',
'a:1:{s:9:"colorType";s:15:"greyscale-alpha";}'
],
];
}
}

View file

@ -14,7 +14,7 @@ class MWFilePropsTest extends MediaWikiIntegrationTestCase {
'minor_mime' => null,
'mime' => null,
'sha1' => '',
'metadata' => '',
'metadata' => [],
'width' => 0,
'height' => 0,
'bits' => 0,
@ -28,7 +28,7 @@ class MWFilePropsTest extends MediaWikiIntegrationTestCase {
'minor_mime' => 'zip',
'mime' => 'application/zip',
'sha1' => 'rt7k3bexfau9i8jd5z41oxi3fqz7psb',
'metadata' => '',
'metadata' => [],
'width' => 0,
'height' => 0,
'bits' => 0,
@ -45,10 +45,12 @@ class MWFilePropsTest extends MediaWikiIntegrationTestCase {
'minor_mime' => 'jpeg',
'mime' => 'image/jpeg',
'sha1' => 'iqrl77mbbzax718nogdpirzfodf7meh',
'metadata' => 'a:2:{s:15:"JPEGFileComment";' .
'a:1:{i:0;s:58:"File source: ' .
'http://127.0.0.1:8080/wiki/File:Srgb_copy.jpg";}' .
's:22:"MEDIAWIKI_EXIF_VERSION";i:2;}',
'metadata' => [
'JPEGFileComment' => [
'File source: http://127.0.0.1:8080/wiki/File:Srgb_copy.jpg',
],
'MEDIAWIKI_EXIF_VERSION' => 2,
],
'media_type' => 'BITMAP',
] ],
];

View file

@ -77,6 +77,9 @@ EOF;
[
'nonanimated.gif',
[
'width' => 45,
'height' => 30,
'bits' => 1,
'comment' => [ 'GIF test file ⁕ Created with GIMP' ],
'duration' => 0.1,
'frameCount' => 1,
@ -87,6 +90,9 @@ EOF;
[
'animated.gif',
[
'width' => 45,
'height' => 30,
'bits' => 1,
'comment' => [ 'GIF test file . Created with GIMP' ],
'duration' => 2.4,
'frameCount' => 4,
@ -98,6 +104,9 @@ EOF;
[
'animated-xmp.gif',
[
'width' => 45,
'height' => 30,
'bits' => 1,
'xmp' => $xmpNugget,
'duration' => 2.4,
'frameCount' => 4,