Changes from patch set Icb93c79f4843b59dae80d3eda1a880457a1a68f2
Also some swaps from assertEquals to assertSame/True/False/Null
Change-Id: Ife497ae6cb1888b77eb25e85b76df72adc65641a
* Add automatic splitting of large metadata on upload or refresh. If
the reserializeMetadata option is enabled, metadata stored with PHP
serialization will be automatically reserialized to JSON.
* Inject configuration variable $wgUpdateCompatibleMetadata via
LocalRepo instead of accessing it directly.
* In refreshImageMetadata.php and rebuildImages.php, construct a new
LocalRepo with config overrides, instead of overwriting config
globals. Add a helper to RepoGroup to help with this.
* In refreshImageMetadata.php, add new options --convert-to-json and
--split which reserialize metadata and optionally split out large
items to blob storage.
Also, refreshImageMetadata.php was totally broken in the non-force mode
since metadata refresh on page view was disabled in b814245d9f. The
maintenance script was relying on newFileFromRow() magically upgrading
the row, which doesn't happen anymore. So, call maybeUpgradeRow()
directly.
Bug: T275268
Change-Id: I7bf7d9cef71641e287ca4346b568b381f4ada50e
* Optionally store metadata in the database in JSON format instead of
PHP serialization. The new JSON format has a top-level "envelope"
array which gives us a place to store things that are not part of the
handler metadata.
* Optionally split metadata items, putting items above a threshold into
the text table. The FileRepo and MediaHandler must both opt in.
* For staged deployment, the read side of these changes is always
active. Only the write side is configurable.
Bug: T275268
Change-Id: I876ea5c9d3a1881e278f689d2f8a3ae20240c703
Image metadata is usually a serialized string representing an array.
Passing the string around internally and having everything unserialize
it is an awkward convention.
Also, many image handlers were reading the file twice: once for
getMetadata() and again for getImageSize(). Often getMetadata()
would actually read the width and height and then throw it away.
So, in filerepo:
* Add File::getMetadataItem(), which promises to allow partial
loading of metadata per my proposal on T275268 in a future commit.
* Add File::getMetadataArray(), which returns the unserialized array.
Some file handlers were returning non-serializable strings from
getMetadata(), so I gave them a legacy array form ['_error' => ...]
* Changed MWFileProps to return the array form of metadata.
* Deprecate the weird File::getImageSize(). It was apparently not
called by anything, but was overridden by UnregisteredLocalFile.
* Wrap serialize/unserialize with File::getMetadataForDb() and
File::loadMetadataFromDb() in preparation for T275268.
In MediaHandler:
* Merged MediaHandler::getImageSize() and MediaHandler::getMetadata()
into getSizeAndMetadata(). Deprecated the old methods.
* Instead of isMetadataValid() we now have isFileMetadataValid(), which
only gets a File object, so it can decide what data it needs to load.
* Simplified getPageDimensions() by having it return false for non-paged
media. It was not called in that case, but was implemented anyway.
In specific handlers:
* Rename DjVuHandler::getUnserializedMetadata() and
extractTreesFromMetadata() for clarity. "Metadata" in these function
names meant an XML string.
* Updated DjVuImage::getImageSize() to provide image sizes in the new
style.
* In ExifBitmapHandler, getRotationForExif() now takes just the
Orientation tag, rather than a serialized string. Also renamed for
clarity.
* In GIFMetadataExtractor, return the width, height and bits per channel
instead of throwing them away. There was some conflation in
decodeBPP() which I picked apart. Refer to GIF89a section 18.
* In JpegMetadataExtractor, process the SOF0/SOF2 segment to extract
bits per channel, width, height and components (channel count). This
is essentially a port of PHP's getimagesize(), so should be bugwards
compatible.
* In PNGMetadataExtractor, return the width and height, which were
previously assigned to unused local variables. I verified the
implementation by referring to the specification.
* In SvgHandler, retain the version validation from unpackMetadata(),
but rename the function since it now takes an array as input.
In tests:
* In ExifBitmapTest, refactored some tests by using a provider.
* In GIFHandlerTest and PNGHandlerTest, I removed the tests in which
getMetadata() returns null, since it doesn't make sense when ported to
getMetadataArray(). I added tests for empty arrays instead.
* In tests, I retained serialization of input data since I figure it's
useful to confirm that existing database rows will continue to be read
correctly. I removed serialization of expected values, replacing them
with plain data.
* In tests, I replaced access to private class constants like
BROKEN_FILE with string literals, since stability is essential. If
the class constant changes, the test should fail.
Elsewhere:
* In maintenance/refreshImageMetadata.php, I removed the check for
shrinking image metadata, since it's not easy to implement and is
not future compatible. Image metadata is expected to shrink in
future.
Bug: T275268
Change-Id: I039785d5b6439d71dcc21dcb972177dba5c3a67d
Reapply the reverted commit which removed dynamic property assignment.
But continue to allow extra fields to be passed to newFromRow() for the
benefit of subclasses.
This reverts commit 9e8ff12a59.
Change-Id: Ia3c14c51541e06264891a51763a8dadde83fbab7
This reverts commit 29cd484c86.
Reason for revert: subclasses of LocalFile broke, cause they were relying on dynamic property assignment. For example OldLocalFile was loading oi_deleted this way. We could override loadFromDB, but since LocalFile is @stable to override, this becomes a breaking change. I'll leave it to Tim to re-revert in a way he thinks it's best.
Change-Id: I81e06c13128d3330972a39fb2628a1dd5f8c4658
Merge LocalFile::decodeRow() with its only caller loadFromRow(), and
replace dynamic assignment $this->$field with plain assignment. This
makes the data flow more obvious.
The point of dynamic assignment was defeated by the fact that almost
every field required some kind of normalization after loading it from
the database. There's no point looping over all fields when you need to
do something different for each field.
Dynamic assignment is retained in loadFromCache() and setProps(), where
there is less normalization.
Change-Id: Ie103884c4436ab63c29652edf3bab331a34081c9
RepoGroup::singleton(), ::destroySingletons() and ::setSingleton()
are all deprecated since 1.34.
Bug: T249020
Change-Id: Ie6c7db6331681ec96da646f945958ed8675e6b78
Some MediaHandler subclasses were setting custom properties on the File
object in order to cache file-associated state. So:
* Add File::getHandlerState() and File::setHandlerState().
* Put them in an interface, which will be used in a subsequent commit in
MediaHandler::getSizeAndMetadata().
* Use them in DjvuHandler.
* Provide a trivial implementation of the interface, for use in testing
and in the subsequent commit.
Change-Id: Ic365384ff13f7898c1203da38c4405abf03d7563
The global function wfLocalFile() is deprecated since 1.34 and unused.
This patch now makes it emit deprecation warnings.
Change-Id: Ib9f94b4f49e7720bd4455d019995037eaa4e3980
It is not entirely meaningless. It might be an indicator that
the number of calls to a method is intentionally unlimited.
This is similar to e.g. an @inheritDoc PHPDoc comment that
marks a method as being "intentionally undocumented".
However, what's the meaning of being "intentionally
unconstrained"? Let's just not have any constraint then.
I feel all these ->expects( $this->any() ) bloat the test
code so much that it's never worth it.
Change-Id: I9925e7706bd03e1666f6eb0b284cb42b0dd3be23
It's the same and makes the test code much more readable, I
would like to argue.
Because of the was I split all the changes I made into smaller
patches this patch contains some other changes in the same
lines where I could not split them off. E.g. removal of
->any(), which is the default anyway and doesn't do anything.
Change-Id: Ib297b989d4aec33b31a4e33fe9d5032865b39be0
Ended up using
grep -Prl '\->setMethods\(' . | xargs sed -r -i 's/setMethods\(/onlyMethods\(/g'
special-casing setMethods( null ) -> onlyMethods( [] )
and then manual fix of failing test (from PS2 onwards).
Bug: T278010
Change-Id: I012dca7ae774bb430c1c44d50991ba0b633353f1
Renamed several methods for clarity and updated various comments
Use explicit number_format() instead of casting floats to strings
when making purge values
Fix LocalRepoTest mocking of WANObjectCache::makeGlobalKey()
Bug: T264604
Change-Id: Ib03b97fcab8a34d38d3a8a12da6e28e12feef4ad
My personal best practice is to not document @params when there
is a @dataProvider. I mean, these test…() functions are not
meant to be called from anywhere. They do not really need
documentation. @param tags don't do much but duplicate what the
@dataProvider does. This is error-prone, as demonstrated by the
examples in this patch.
This patch also removes @throws tags from tests. A test…() can
never throw an exception. Otherwise the test would fail.
Most of these are found by the not yet released I10559d8.
Change-Id: I3782bca43f875687cd2be972144a7ab6b298454e
Needs to be a real path, not relative one
6) FileTest::testGetThumbnailSource with data set #2 (array(true,
array('/tmp'), 1024, 2048, '/tmp', 'Temporary path because temp f...
found'))
Temporary path because temp file was found
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'/tmp'
+'fsFilePath'
The file_exists in File::getThumbnailSource gets the path '/tmp' and
that does not exists on windows, because it is relative to one of the
core folder
Change-Id: I8618c8b34261451da42f88360a9d65353960e91f
The name is only used for local config and internal virtual URLs.
A key use case is having a LocalRepo named "local" on central repo
wiki and a ForeignDBViaLBRepo named "shared" on the client wikis,
the later being a reference to the former. The keys should be be
shared regardless of this naming difference.
Bug: T267668
Change-Id: Ic239bb980740ec5237edc878af4274885034a96f
The name change happened some time ago, and I think its
about time to start using the name name!
(Done with a find and replace)
My personal motivation for doing this is that I have started
trying out vscode as an IDE for mediawiki development, and
right now it doesn't appear to handle php aliases very well
or at all.
Change-Id: I412235d91ae26e4c1c6a62e0dbb7e7cf3c5ed4a6
Also rename image_redirect key to file_redirect while at it.
This assures that stale keys are not still in use.
Bug: T253405
Change-Id: I31a9bb6672b33fbfa1b974955d78fdfd8d58f5da
This should be the exact same. Its more a style change than anything.
So why do it then?
* I believe this is much less confusing than code mentioning a weird
"standard class". Barely anybody knows what this is, and what the
difference between "object" and "stdClass" is.
* The code is shorter.
* It's even faster. In my micro benchmark it's twice as fast.
Change-Id: I7ee0e8ae6d9264a89b6cd1dd861f0466ae620ccc
Add an endpoint to the core REST API to return the list of
media files embedded in a page, including certain metadata
for each media file.
Bug: T236169
Change-Id: I3188848ee7de365d209dfb7da5b885313e9f705b
The only use-case for this was if the local wiki's File namespace didn't
allow initial-lowercase names but a remote repo did. This case is
unlikely to be useful and was broken anyway, so it's now prohibited, and
getUserCaseDBKey is no longer needed. It's now an alias for getDBkey
until we remove it.
This includes a breaking change to
MediaWikiTitleCodec::splitTitleString, but that was always intended as
an internal method and I have now marked it officially as such. There is
one caller in code search outside core (JsonConfig), but it looks like
it will be unaffected.
Bug: T202094
Depends-On: I4b8ceb8a7f4624d6a3763aca6df41bf1a0d7293f
Depends-On: I724be15e93421f874fb202f0ec2ca6940e56a20a
Change-Id: I4fd64d4b0036b6dabdcfeee18766df18bf538542
Add public, protected or private to function missing a visibility
Enable the tests folder for the phpcs sniff
Change-Id: Ibefce76ea9984c47e08c94889ea2eafca7565e2c
assertSame() is guaranteed to never do any magic type conversion.
This can be critical when accidentially comparing empty strings (a
value PHP considers to be "falsy") to false, 0, 0.0, null, and such.
Change-Id: I2e2685c5992cae252f629a68ffe1a049f2e5ed1b
Some methods on LocalFile will fatal if called on a non-existing file.
ApiQueryImageInfo did not take that into account.
This patch changes LocalFile to avoid fatal errors, and ApiQueryImageInfo
to not try and report information on non-existing files.
NOTE: the modified code has NO test coverage! This should be fixed
before this patch is applied, or the patch needs to be thoroughly tested
manually.
Bug: T221812
Change-Id: I9b74545a393d1b7a25c8262d4fe37a6492bbc11e
