The '1.44' test data is the current serialization output.
The '1.44_native' test data is the output after
I9e6b924d62ccc3312f5c70989477da1e2f21c86b which uses native PageBundle
serialization. This is to establish forward-compatibility using the
procedure described at
https://www.mediawiki.org/wiki/Manual:Parser_cache/Serialization_compatibility
Change-Id: I8d53ff3e9c600cce16a0fc07f3665a91e5d8036b
(cherry picked from commit 9f6ee7ef03b2c3657aff60e81c4a0c17599cdc46)
By default this uses the existing ContentHandler::serializeContent() and
::unserializeContent() methods. But in cases where existing PHP
serialization preserved fields that ::serializeContent() did not,
provide an additional ContentHandler::serializeContentToJsonArray()
and ContentHandler::deserializeContentFromJsonArray() methods which
can be used. Use these in WikitextContentHandler to preserve the
PST flags.
Added test cases and a ContentSerializationTestTrait to make it
easy to ensure forward- and backward-compatibility in accord with
https://www.mediawiki.org/wiki/Manual:Parser_cache/Serialization_compatibility
The new JsonCodecable codec will be used to improve PageEditStashContent
serialization, which no longer has to PHP-serialize its Content object.
New test case added demonstrating compatibility.
Bug: T264389
Bug: T161647
Change-Id: I544625136088164561b9169a63aed7450cce82f5
(cherry picked from commit 21576d6c1893079777a1a51d0f81c4941c58e376)
This can cause PHP to try to load the param value as a class, with
potentailly disastrous results (though it’s not quite clear if this can
actually cause attacker-supplied code execution or not).
Bug: T377912
Change-Id: I0239b3e65cf516c6fdf287882f05e47a01f963c1
(cherry picked from commit cdf11b23474024dfe39057993372f2126302e244)
This was seen in WMF production during a train deployment, where namespacing of classes,
which had been serialized (for example CacheTime), along with our PSR-4 definitions may result
in cases where PHP tries to load the same PHP file more than once. Combined with non obvious
error messages, require_once gives us better behaviour and error messages.
More explicitly:
In T378006, the autoloader is entered from class_exists(), and the class has a filename
resolvable with PSR-4 which is already loaded by a non-PSR-4 class name. Using require_once
would allow class_exists() to return false in that case.
In T372500, the autoloader is entered from unserialize(). It looks like require_once would
just give you a more informative error message.
Bug: T378006
Bug: T372500
Change-Id: I928f29198af9baf81a3cae604b3adf41595c2176
(cherry picked from commit 03dd4ae7ae0e2ce7e45f0bf2cb913642eef842a8)
This adds support for serializing/deserializing objects which
implement the JsonCodecable interface from the wikimedia/json-codec
library used by Parsoid. JsonCodecable allows customizing the encoding
of objects of a given class using a class-specific codec object, and
JsonCodecable is an interface which is defined and can be used outside
mediawiki core.
In addition json-codec supports deserialization in the presence of
aliased class names, fixing T353883.
Backward and forward compatibility established via the mechanism
described in
https://www.mediawiki.org/wiki/Manual:Parser_cache/Serialization_compatibility
Test data generated by this patch was added in
I109640b510cef9b3b870a8c188f3b4f086d75d06 to ensure forward
compatibility with the output after this patch is merged.
Benchmarks:
PHP 7.4.33 PHP 8.2.19 PHP 8.3.6
BEFORE AFTER BEFORE AFTER BEFORE AFTER
Serialize: 926.7/s 1424.8/s 978.5/s 1542.4/s 1023.5/s 1488.6/s
Serialize (assoc): 930.2/s 1378.6/s 974.6/s 1541.9/s 1022.4/s 1463.4/s
Deserialize: 1942.7/s 1961.3/s 2118.8/s 2175.9/s 2129.8/s 2063.5/s
Deserialize (assoc): 1952.0/s 1905.7/s 2107.5/s 2192.1/s 2153.3/s 2011.1/s
These numbers definitely do not have as many significant digits as
written here. But they should be sufficient to demonstrate that
performance is not impaired by this patch and in fact serialization
speed improves slightly.
Bug: T273540
Bug: T327439
Bug: T346829
Bug: T353883
Depends-On: If1d70ba18712839615c1f4fea236843ffebc8645
Change-Id: Ia1017dcef462f3ac1ff5112106f7df81f5cc384f
This applies JSON Schema validation in phpunit tests where appropriate:
1) In ModuleSpecHandlerTest, the generated OpenApi specs are validated
against the OpenAPI 3 schema.
2) In RestStructureTest, module definition files are validated against
the mwapi schema.
This patch introduces a new trait to make it easy for phpunit test cases
to perform validation.
This patch also fixes some issues with the docs/rest/mwapi-1.0.json
schema and the includes/Rest/content.v1.json module definition.
Change-Id: I966cddb337c9373ed3a369496548a8d8c538ae84
Why:
* Maintenance scripts in core have low test coverage
* Improving this will reduce the chances of regressions and bugs
What:
* Create a MockLocalisationCacheTrait which is used to generate
a mock LocalisationCache instance which has a limited and fixed
set of message keys
** This is created by splitting code from LocalisationCacheTest
* Create AllTransTest and DumpMessagesTest which make use of this
new trait.
Bug: T371167
Change-Id: Ie814c8136504d2af94ec9377d64b19d3a0e25eed
* Re-use the same data provider explicitly instead of manually
keeping them in sync.
* Add test case names as array keys to improve debugging.
* Widen `@covers` annotations in unit tests.
Bug: T107289
Change-Id: I9a321400855ddd1f56334f6ecf85590fd8ed4aaf
MessageValue and friends are pure value objects and newable, so
it makes sense for them to be (de)serializable too. There are some
places where we want to serialize messages, such as in ParserOutput.
The structure of the resulting JSON is inspired by the way we
represent Message objects as plain values elsewhere in MediaWiki,
e.g. StatusValue::getStatusArray().
Co-Authored-By: C. Scott Ananian <cscott@cscott.net>
Depends-On: Ia32f95a6bdf342262b4ef044140527f0676402b9
Depends-On: I7bafe80cd36c2558517f474871148286350a4e76
Change-Id: Id47d58b5e26707fa0e0dbdd37418c0d54c8dd503
This is to make it clearer that they're related to converting serialized
content back into JSON, rather than stating that things are not
representable in JSON.
Change-Id: Ic440ac2d05b5ac238a1c0e4821d3f2d858bc3d76
PHPUnit 10 requires test data providers to be static. This patch doesn't
completely do the job, as some providers currently call the non-static
method `$this->markTestSkippedIfPhp()`, but it gets us closer to PHPUnit 10
compatibility.
This also lets us simplify the validateParserCacheSerializationTestData
maintenance script and make it slightly more generic.
Change-Id: Ie3696bfaa29aca9da45f54239126222e8c847ea9
The serialization test cases look for files based on the name of the
class they are testing. After the namespacing of ParserOutput, they
were looking for files named like:
1.42-MediaWiki\Parser\ParserOutput-binaryPageProperties.json
The embedded backslashes in these filenames would raise havoc on Windows
machines. What's more, none of the existing ParserOutput tests will
actually be checked anymore because the filenames don't match up
with what is expected after namespacing.
Fix this by stripping the namespace from the classname when forming
the test file names.
When this is done, the tests cases for GhostFieldAccess begin running
again, revealing that they were broken when GhostFieldTestClass was
re-namespaced. Add a class alias for the GhostFieldTestClass to fix
this.
Finally, PHP <= 8.1 does not deserialize private properties correctly
after a class is renamed and aliased, because the internal name of the
private property contains the "old" class name in the serialization.
Add a new ::restoreAliasedGhostField() method to the
GhostFieldAccessTrait to workaround this issue and restore proper
deserialization of ParserOutput.
Bug: T365060
Followup-To: I9c64a631b0b4e8e4fef8a72ee0f749d35f918052
Followup-To: I4c2cbb0a808b3881a4d6ca489eee5d8c8ebf26cf
Change-Id: I7bafe80cd36c2558517f474871148286350a4e76
When going through a ContentDOMTransformStage, we try to move the
PageBundle when transforming the document from and to DOM. In the
current version of this code, this adds DataParsoid, a non-serializable
class, to ExtensionData, which breaks on ParserCache storage in later
steps.
This patch is pretty hacky, but it transforms the PageBundle structure
back to a stdClass so that it can be re-serialized before cache
insertion. The added test fails without this patch.
Hopefully we'll get rid of these hacks when using a HTMLHolder later.
Bug: T365036
Change-Id: Icc74edd43ea5098faebc21a084b6d483d6ab99d1
Modules group together endpoints by a shared prefix. The idea is that each module has its own version and can generated self-contained self-documentation. This allows clients to have clear expectations about the endpoints of each module, no matter what wiki they are accessing. So far, each wiki may be exposing a different set of endpoints, with no way to provide a spec that describes that set of endpoints in a way that would be consistent across wikis and stable over time.
Bug: T362480
Change-Id: Iebcde4645d472d27eee5a30adb6eee12cc7d046b
Why: supporting JSON schema $refs is desired for the
CommunityConfiguration 2.0 project so configuration
schemas can reuse definitions coming from another
PHP JsonSchema class. The https://json-schema.org/
specification supports relative and absolute URIs
as the content of a $ref. The change aims
to support only the former. Meaning that any
$ref defined in a schema should point to a
class that can be resolved by PHP.
What: update schema traversing in JsonSchemaTrait
to resolve any reference specified as:
[ 'class' => SomeClass:class, 'field' => 'someClassField' ]
into an actual json schema compliant $ref, eg:
$ref: '#/$defs/SomeDefinitionName'. Also annotate
new definitions found in a "$defs" array with the
resolved value of the refed class field.
Bug: T357718
Change-Id: I130326e5762e706be4889e308eeec45a6c7683c5
This moves a code out of file scope into classes to make it
testable. The code is left in the same structure as it was before,
global functions have been converted into methods on the new
ThumbnailEntryPoint and Thumbnail404EntryPoint classes.
This test introduces comprehensive phpunit tests covering all functional
code paths in ThumbnailEntryPoint. This is intended to support
refactoring of this code.
Change-Id: I459abc7b11d0ab4ee682a863c9525a945048296f
FileBackendIntegrationTest was running tests against different backends
in an unconventional way, using a combination of wrapper test cases that
run tests against two different classes, and CLI options which don't
really exist anymore and have an associated fixme.
So:
* Move the bulk of FileBackendIntegrationTest to a new abstract base
class under tests/phpunit/integration.
* Add subclasses for the FS and multiwrite test cases. This allows us to
eliminate the wrappers.
* Add a subclass for MemoryFileBackend.
* Add a Swift subclass which replaces the main use case for
the CLI option --use-filebackend. It is automatically enabled when
a Swift backend is configured, similar to PostgreSQL tests.
* Some miscellaneous tests with a medium level of integration, not
requiring backend setup and teardown, were moved to new classes
FileBackendMultiWriteTest and FileBackendStoreTest.
Change-Id: I0da531349d7627970a7bcb34f3c1f5fd7e05cb21
Leave class aliases behind because they might be being used somewhere,
but we don't normally flag these kinds of things in the release notes,
do we?
Bug: T357823
Change-Id: I7fc7f34494d5c4df81f6746d63df1d0f990f8ae9
1) Introduce EntryPointEnvironment which wraps functions that interact
with the PHP runtime, so they can be mocked for testing.
2) Allow server info fields to be overwritten in FauxRequest.
3) Make MediaWikiEntryPoint use WebResponse to set headers
Bug: T354216
Change-Id: Ic21950c956de5d2b5a7dd66a1e2de58f807cfd9f
Abstract test classes are no longer allowed to end in "Test" as of
PHPUnit 9.6.
Follow-up: I53551ec6d6
Bug: T342110
Change-Id: I9638c2937f8b702851d080ab217fbc34620fabb6
Why:
* The temporary account configuration settings, which are
controlled via wgAutoCreateTempUser have many options
and so defining the config for a test is difficult without
keeping a private array for the default array value.
* Providing methods to easily disable and enable temporary account
creation in a one-line call is useful to fix existing tests
for temporary account creation.
What:
* Add the trait named TempUserTestTrait which provides two methods
that disable and enable temporary account autocreation.
* This trait is designed for use in integration tests as it modifies
configuration values.
* The ::enableAutoCreateTempUser method optionally takes an array
that can be used to override the default wgAutoCreateTempUser
configuration.
* The ::disableAutoCreateTempUser method optionally takes a value
for the reservedPattern key as this is still used if the 'enabled'
key is set to false.
Bug: T352693
Change-Id: I21c5c7d61370b9ce96059341e0744abc61545e09
Leave `composer phpunit` as the main PHPUnit entry point, a single
config file (phpunit.xml.dist) and a single bootstrap (bootstrap.php).
Deprecate the `composer phpunit:entrypoint`, making it print a
deprecation message and then fall back to `composer phpunit`. Deprecate
boostrap.integration.php as well. Update the release notes accordingly.
Add a temporary PHPUnit extension, only enabled in suite.xml, that
prints a deprecation message. Define a new constant in phpunit.php to
avoid printing 2 deprecation messages when phpunit.php is used (as it
auto-defaults to suite.xml).
Add some documentation to bootstrap.php.
Bug: T227900
Depends-On: Ie91c9f2882f12fd8ea822529528dd1dd9efe89db
Change-Id: If2ffe0522422fd6aadcc08988789d98c2403fc48
There are a couple of user options related classes already,
and the T321527 work on dynamic defaults is going to add
even more. Let's move them into a separate namespace
to make core a bit more organized.
Old name is kept as an alias for compatibility purposes.
Bug: T321527
Bug: T352284
Change-Id: I9822eb1553870b876d0b8a927e4e86c27d83bd52
Traditionally we tell people to avoid mocking the database, and we
haven't make a centralized way to do it. This has just led to it being
done poorly and verbosely in a lot of different ways.
So, add a MockDatabase class.
Change-Id: Ie420f2b42ee7d047378221c519845f7261c20c28
PHP 8.2 has deprecated dynamic property creation on classes that do not
explicitly allow it via the #[AllowDynamicProperties] annotation. The
recommended migration path for associating arbitrary data with objects
is the WeakMap class, but it is only available on PHP 8.0 and above.
Since MediaWiki still supports PHP 7.4, and the test framework needs to
be able to associate some state with objects it does not own, like DB
connection handles, introduce and use a new DynamicPropertyTestHelper
shim class that uses WeakMap if available and falls back to regular
dynamic property creation otherwise. Convert the DB setup-related
dynamic property usage in MediaWikiIntegrationTestCase to use this
class.
Bug: T326466
Change-Id: I1054f6f944d491b536949cada33e2ac670e026f8
This trait provides a robust way for tests to simulate user blocks.
Developed as an alternative approach to I7fc457eff88.
Change-Id: Ia5d813fcbdd77b94dbbc730ca4639d6efd2148a8
