The '1.44' test data is the current serialization output.
The '1.44_native' test data is the output after
I9e6b924d62ccc3312f5c70989477da1e2f21c86b which uses native PageBundle
serialization. This is to establish forward-compatibility using the
procedure described at
https://www.mediawiki.org/wiki/Manual:Parser_cache/Serialization_compatibility
Change-Id: I8d53ff3e9c600cce16a0fc07f3665a91e5d8036b
(cherry picked from commit 9f6ee7ef03b2c3657aff60e81c4a0c17599cdc46)
By default this uses the existing ContentHandler::serializeContent() and
::unserializeContent() methods. But in cases where existing PHP
serialization preserved fields that ::serializeContent() did not,
provide an additional ContentHandler::serializeContentToJsonArray()
and ContentHandler::deserializeContentFromJsonArray() methods which
can be used. Use these in WikitextContentHandler to preserve the
PST flags.
Added test cases and a ContentSerializationTestTrait to make it
easy to ensure forward- and backward-compatibility in accord with
https://www.mediawiki.org/wiki/Manual:Parser_cache/Serialization_compatibility
The new JsonCodecable codec will be used to improve PageEditStashContent
serialization, which no longer has to PHP-serialize its Content object.
New test case added demonstrating compatibility.
Bug: T264389
Bug: T161647
Change-Id: I544625136088164561b9169a63aed7450cce82f5
(cherry picked from commit 21576d6c1893079777a1a51d0f81c4941c58e376)
This can cause PHP to try to load the param value as a class, with
potentailly disastrous results (though it’s not quite clear if this can
actually cause attacker-supplied code execution or not).
Bug: T377912
Change-Id: I0239b3e65cf516c6fdf287882f05e47a01f963c1
(cherry picked from commit cdf11b23474024dfe39057993372f2126302e244)
This was seen in WMF production during a train deployment, where namespacing of classes,
which had been serialized (for example CacheTime), along with our PSR-4 definitions may result
in cases where PHP tries to load the same PHP file more than once. Combined with non obvious
error messages, require_once gives us better behaviour and error messages.
More explicitly:
In T378006, the autoloader is entered from class_exists(), and the class has a filename
resolvable with PSR-4 which is already loaded by a non-PSR-4 class name. Using require_once
would allow class_exists() to return false in that case.
In T372500, the autoloader is entered from unserialize(). It looks like require_once would
just give you a more informative error message.
Bug: T378006
Bug: T372500
Change-Id: I928f29198af9baf81a3cae604b3adf41595c2176
(cherry picked from commit 03dd4ae7ae0e2ce7e45f0bf2cb913642eef842a8)
This adds support for serializing/deserializing objects which
implement the JsonCodecable interface from the wikimedia/json-codec
library used by Parsoid. JsonCodecable allows customizing the encoding
of objects of a given class using a class-specific codec object, and
JsonCodecable is an interface which is defined and can be used outside
mediawiki core.
In addition json-codec supports deserialization in the presence of
aliased class names, fixing T353883.
Backward and forward compatibility established via the mechanism
described in
https://www.mediawiki.org/wiki/Manual:Parser_cache/Serialization_compatibility
Test data generated by this patch was added in
I109640b510cef9b3b870a8c188f3b4f086d75d06 to ensure forward
compatibility with the output after this patch is merged.
Benchmarks:
PHP 7.4.33 PHP 8.2.19 PHP 8.3.6
BEFORE AFTER BEFORE AFTER BEFORE AFTER
Serialize: 926.7/s 1424.8/s 978.5/s 1542.4/s 1023.5/s 1488.6/s
Serialize (assoc): 930.2/s 1378.6/s 974.6/s 1541.9/s 1022.4/s 1463.4/s
Deserialize: 1942.7/s 1961.3/s 2118.8/s 2175.9/s 2129.8/s 2063.5/s
Deserialize (assoc): 1952.0/s 1905.7/s 2107.5/s 2192.1/s 2153.3/s 2011.1/s
These numbers definitely do not have as many significant digits as
written here. But they should be sufficient to demonstrate that
performance is not impaired by this patch and in fact serialization
speed improves slightly.
Bug: T273540
Bug: T327439
Bug: T346829
Bug: T353883
Depends-On: If1d70ba18712839615c1f4fea236843ffebc8645
Change-Id: Ia1017dcef462f3ac1ff5112106f7df81f5cc384f
In T340552, the official PHP OpenTelemetry client was effectively
rejected for inclusion in MediaWiki due to its size. Implement a minimal
tracing library instead that eschews conformance with the OTEL client
specification in favor of simplicity, while remaining capable of
emitting trace data in OTLP format and thus retaining compatibility with
any ingestion endpoint capable of handling OTLP.
In its current state, the library supports a basic feature set that
should be sufficient for basic tracing integration:
* Span creation, inclusive span activation and automatic parent span
assignment,
* Span attributes and span kinds,
* Basic resource (process/request)-level metadata generation,
* Data export over OTLP.
Additional functionality, such as trace propagation, can then be
incrementally added to the library.
Bug: T340552
Change-Id: Ibc3910058cd7ed064cad293a3cdc091344e66b86
This applies JSON Schema validation in phpunit tests where appropriate:
1) In ModuleSpecHandlerTest, the generated OpenApi specs are validated
against the OpenAPI 3 schema.
2) In RestStructureTest, module definition files are validated against
the mwapi schema.
This patch introduces a new trait to make it easy for phpunit test cases
to perform validation.
This patch also fixes some issues with the docs/rest/mwapi-1.0.json
schema and the includes/Rest/content.v1.json module definition.
Change-Id: I966cddb337c9373ed3a369496548a8d8c538ae84
Why:
* Maintenance scripts in core have low test coverage
* Improving this will reduce the chances of regressions and bugs
What:
* Create a MockLocalisationCacheTrait which is used to generate
a mock LocalisationCache instance which has a limited and fixed
set of message keys
** This is created by splitting code from LocalisationCacheTest
* Create AllTransTest and DumpMessagesTest which make use of this
new trait.
Bug: T371167
Change-Id: Ie814c8136504d2af94ec9377d64b19d3a0e25eed
* Re-use the same data provider explicitly instead of manually
keeping them in sync.
* Add test case names as array keys to improve debugging.
* Widen `@covers` annotations in unit tests.
Bug: T107289
Change-Id: I9a321400855ddd1f56334f6ecf85590fd8ed4aaf
MessageValue and friends are pure value objects and newable, so
it makes sense for them to be (de)serializable too. There are some
places where we want to serialize messages, such as in ParserOutput.
The structure of the resulting JSON is inspired by the way we
represent Message objects as plain values elsewhere in MediaWiki,
e.g. StatusValue::getStatusArray().
Co-Authored-By: C. Scott Ananian <cscott@cscott.net>
Depends-On: Ia32f95a6bdf342262b4ef044140527f0676402b9
Depends-On: I7bafe80cd36c2558517f474871148286350a4e76
Change-Id: Id47d58b5e26707fa0e0dbdd37418c0d54c8dd503
This is to make it clearer that they're related to converting serialized
content back into JSON, rather than stating that things are not
representable in JSON.
Change-Id: Ic440ac2d05b5ac238a1c0e4821d3f2d858bc3d76
PHPUnit 10 requires test data providers to be static. This patch doesn't
completely do the job, as some providers currently call the non-static
method `$this->markTestSkippedIfPhp()`, but it gets us closer to PHPUnit 10
compatibility.
This also lets us simplify the validateParserCacheSerializationTestData
maintenance script and make it slightly more generic.
Change-Id: Ie3696bfaa29aca9da45f54239126222e8c847ea9
The serialization test cases look for files based on the name of the
class they are testing. After the namespacing of ParserOutput, they
were looking for files named like:
1.42-MediaWiki\Parser\ParserOutput-binaryPageProperties.json
The embedded backslashes in these filenames would raise havoc on Windows
machines. What's more, none of the existing ParserOutput tests will
actually be checked anymore because the filenames don't match up
with what is expected after namespacing.
Fix this by stripping the namespace from the classname when forming
the test file names.
When this is done, the tests cases for GhostFieldAccess begin running
again, revealing that they were broken when GhostFieldTestClass was
re-namespaced. Add a class alias for the GhostFieldTestClass to fix
this.
Finally, PHP <= 8.1 does not deserialize private properties correctly
after a class is renamed and aliased, because the internal name of the
private property contains the "old" class name in the serialization.
Add a new ::restoreAliasedGhostField() method to the
GhostFieldAccessTrait to workaround this issue and restore proper
deserialization of ParserOutput.
Bug: T365060
Followup-To: I9c64a631b0b4e8e4fef8a72ee0f749d35f918052
Followup-To: I4c2cbb0a808b3881a4d6ca489eee5d8c8ebf26cf
Change-Id: I7bafe80cd36c2558517f474871148286350a4e76
When going through a ContentDOMTransformStage, we try to move the
PageBundle when transforming the document from and to DOM. In the
current version of this code, this adds DataParsoid, a non-serializable
class, to ExtensionData, which breaks on ParserCache storage in later
steps.
This patch is pretty hacky, but it transforms the PageBundle structure
back to a stdClass so that it can be re-serialized before cache
insertion. The added test fails without this patch.
Hopefully we'll get rid of these hacks when using a HTMLHolder later.
Bug: T365036
Change-Id: Icc74edd43ea5098faebc21a084b6d483d6ab99d1
Modules group together endpoints by a shared prefix. The idea is that each module has its own version and can generated self-contained self-documentation. This allows clients to have clear expectations about the endpoints of each module, no matter what wiki they are accessing. So far, each wiki may be exposing a different set of endpoints, with no way to provide a spec that describes that set of endpoints in a way that would be consistent across wikis and stable over time.
Bug: T362480
Change-Id: Iebcde4645d472d27eee5a30adb6eee12cc7d046b
A number of tests have hardcoded expections that pass only in WMF CI
where Quibble has LocalSettings.php with $wgScript and $wgArticlePath
set a certain way.
We could fix these by adding setMwGlobals() in their tests, as we
often do, but these are so often forgotten that I'd rather we just
add them to TestSetup.php so that it is simply impossible to write a
test that that passes locally for you (if you have the same config)
but not for someone else.
There is a larger project in there somewhere about expanding this
slowly such that we basically only pluck DB-settings and extension
enablement from LocalSettings and otherwise run the tests with the
default settings in PHPUnit. Pretty much by definition, any (other)
setting you have in LocalSettings is irrelevant because it either:
1. has no effect on the test (majority, harmless either way),
2. has a custom default via TestSetup.php (which has precedence over
LocalSettings.php),
3. is relevant to the code being tested and the test case correctly
calls setMwGlobals() to ensure a consistent value during test.
4. is relevant to the tested code but has no override, thus only
passes if you happen to have the "right" value set for it
(undesirable).
Case 4 is already categorically impossible for the most common config
settings that influence random code because we give them a value
in TestSetup.php. This patch expands that to include $wgScript
and $wgArticlePath. Perhaps in the future we can think about a way
to do this automatically by either re-applying MainConfigSchema
(sans db settings) or by only selectively applying LocalSettings.php
in the first place.
This patch follows-up I072ddf89562fe, which added a test case in
WikitextContentHandlerIntegrationTest.php that assumed "/index.php"
as the value of $wgScript. This passes in WMF CI since Quibble uses
that value, but the tests failed in most local development installs
since those tend to use "/w" instead.
Rather than one-off fixing that one test with overrideConfigValues(),
switch to a more general fixture, since the precise values don't
matter for this test.
Bug: T349087
Bug: T277470
Change-Id: If4304b7ca4a838bd892d4516a0b5c6dfbc30986e
According to the dictionary, "per" (or more conventionally "as per")
means "according to". Refer OED "per" sense II.3.a. For example:
"No value was passed, so return null, as per default".
In this sentence, we are not specifying the default, we are referring
to the default. This correct usage of "per default" was used nowhere
in MediaWiki core as far as I can see.
Instead we have "per default" being used to mean "by default", that is,
giving the value to use when no explicit value was specified.
In OED, the phrase "by default" is blessed with its own section just
for computing usage:
"P.1.e. Computing. As an option or setting adopted automatically by a
computer program whenever an alternative is not specified by the user
or programmer. Cf. sense I.7a."
There are highly similar pre-computing usages of the same phrase,
whereas the phrase "per default" is not mentioned.
As a matter of style, I think "per default" should not be used even
when it is strictly correct, since the common incorrect usage makes it
ambiguous and misleading.
Change-Id: Ibcccc65ead864d082677b472b34ff32ff41c60ae
Why: supporting JSON schema $refs is desired for the
CommunityConfiguration 2.0 project so configuration
schemas can reuse definitions coming from another
PHP JsonSchema class. The https://json-schema.org/
specification supports relative and absolute URIs
as the content of a $ref. The change aims
to support only the former. Meaning that any
$ref defined in a schema should point to a
class that can be resolved by PHP.
What: update schema traversing in JsonSchemaTrait
to resolve any reference specified as:
[ 'class' => SomeClass:class, 'field' => 'someClassField' ]
into an actual json schema compliant $ref, eg:
$ref: '#/$defs/SomeDefinitionName'. Also annotate
new definitions found in a "$defs" array with the
resolved value of the refed class field.
Bug: T357718
Change-Id: I130326e5762e706be4889e308eeec45a6c7683c5
This moves a code out of file scope into classes to make it
testable. The code is left in the same structure as it was before,
global functions have been converted into methods on the new
ThumbnailEntryPoint and Thumbnail404EntryPoint classes.
This test introduces comprehensive phpunit tests covering all functional
code paths in ThumbnailEntryPoint. This is intended to support
refactoring of this code.
Change-Id: I459abc7b11d0ab4ee682a863c9525a945048296f
FileBackendIntegrationTest was running tests against different backends
in an unconventional way, using a combination of wrapper test cases that
run tests against two different classes, and CLI options which don't
really exist anymore and have an associated fixme.
So:
* Move the bulk of FileBackendIntegrationTest to a new abstract base
class under tests/phpunit/integration.
* Add subclasses for the FS and multiwrite test cases. This allows us to
eliminate the wrappers.
* Add a subclass for MemoryFileBackend.
* Add a Swift subclass which replaces the main use case for
the CLI option --use-filebackend. It is automatically enabled when
a Swift backend is configured, similar to PostgreSQL tests.
* Some miscellaneous tests with a medium level of integration, not
requiring backend setup and teardown, were moved to new classes
FileBackendMultiWriteTest and FileBackendStoreTest.
Change-Id: I0da531349d7627970a7bcb34f3c1f5fd7e05cb21
This was added a few weeks ago in change b8f8bcc63f (I49123a5021)
to fix various PHPUnit tests (e.g. CentralAuth, T341731) that failed
due to $wgWikimediaJenkinsCI no longer being set during Setup.php
(and thus the `onRegistration` callbacks in extensions).
This in turn was caused by change d2a30096f (I95a80c44f8d88) which
changed the way tests load LocalSettings.php (no longer in the global
scope), which is why Quibble's LocalSettings.php file, which sets
$wgWikimediaJenkinsCI wasn't taking effect as global variable.
Due to this and other shortcomings with $wgWikimediaJenkinsCI, Quibble
introduced the MW_QUIBBLE_CI env variable. That has the benefit of
always being available, even 1) without relying on a file being in
the global scope, 2) before Setup.php or LocalSettings apply, and
3) in contexts where these will never apply, such as "pure" phpunit
tests that run via `composer phpunit:unit` which don't load
LocalSettings.php at all (which helps address T331621).
This core patch depends on patches that fix current usage in
Wikimedia Gerrit hosted extensions.
Bug: T90875
Depends-On: I996f673add2e8d6e07e20154a90f23eb858ced6b
Depends-On: I0a7470277cdd6939fe3e6e3df92f4d812d320b9c
Depends-On: I25acee1e6e88ca745435cbfa0b398041f04c94d6
Depends-On: I576946230022e9d3518e2ace9ea6f8f1211fcdbe
Change-Id: Ia4df6350f849ca278efef98678850e8c5562f338
Leave class aliases behind because they might be being used somewhere,
but we don't normally flag these kinds of things in the release notes,
do we?
Bug: T357823
Change-Id: I7fc7f34494d5c4df81f6746d63df1d0f990f8ae9
* Switch out raw Exceptions, mostly for InvalidArgumentExceptions.
* Fake exceptions triggered to give Monolog a backtrace are for
some reason "traditionally" RuntimeExceptions, instead, so we
continue to use that pattern in remaining locations.
* Just entirely give up on PostgresResultWrapper's resource vs. object mess.
* Drop now-unneeded false positive hits.
Change-Id: Id183ab60994cd9c6dc80401d4ce4de0ddf2b3da0
1) Introduce EntryPointEnvironment which wraps functions that interact
with the PHP runtime, so they can be mocked for testing.
2) Allow server info fields to be overwritten in FauxRequest.
3) Make MediaWikiEntryPoint use WebResponse to set headers
Bug: T354216
Change-Id: Ic21950c956de5d2b5a7dd66a1e2de58f807cfd9f
Set $wgUsePigLatin in TestSetup.php. People tend to conflate
TestSetup with DevelopmentSettings. Really tests are meant to pass just
with TestSetup even though DevelopmentSettings is used in CI.
Change-Id: I1ccc84aad417350d28ca9e4f102a0c85d73c58d4
Abstract test classes are no longer allowed to end in "Test" as of
PHPUnit 9.6.
Follow-up: I53551ec6d6
Bug: T342110
Change-Id: I9638c2937f8b702851d080ab217fbc34620fabb6
