By default this uses the existing ContentHandler::serializeContent() and
::unserializeContent() methods. But in cases where existing PHP
serialization preserved fields that ::serializeContent() did not,
provide an additional ContentHandler::serializeContentToJsonArray()
and ContentHandler::deserializeContentFromJsonArray() methods which
can be used. Use these in WikitextContentHandler to preserve the
PST flags.
Added test cases and a ContentSerializationTestTrait to make it
easy to ensure forward- and backward-compatibility in accord with
https://www.mediawiki.org/wiki/Manual:Parser_cache/Serialization_compatibility
The new JsonCodecable codec will be used to improve PageEditStashContent
serialization, which no longer has to PHP-serialize its Content object.
New test case added demonstrating compatibility.
Bug: T264389
Bug: T161647
Change-Id: I544625136088164561b9169a63aed7450cce82f5
(cherry picked from commit 21576d6c1893079777a1a51d0f81c4941c58e376)
In WebP lossless chunks (identified by VP8L), width-minus-1 and height-minus-1 of the canvas are sequentially encoded as 14-bit integers. (spec: https://developers.google.com/speed/webp/docs/webp_lossless_bitstream_specification#3_riff_header)
WebPHandler, when decoding the canvas height, has been skipping two most-significant bits. This results in bogus values being read from larger losslessly-encoded files.
Change-Id: Ib5b26f36a15fa65e7990da2ebd94157faccc70c2
(cherry picked from commit 442b73cebbea6db7b7fc945189d5776602fabc8a)
In a future patch (Ia690f10ccbf4f60f9febca98915155c2df58f0d4) we will
use native JsonCodec serialization of the TOCData object. But first
we will add forward-compatibility code to deserialize TOCData, so that
if we need to rollback the future release we won't break the parser
cache.
New serialization test cases added, as per
https://www.mediawiki.org/wiki/Manual:Parser_cache/Serialization_compatibility
Bug: T327439
Change-Id: I4652b2709afd33ff5e469e36960391e993bc7bae
(cherry picked from commit bf61f6bc0eaf5013167e4b80860b0a610559c661)
Add a new test case for the "empty ToC" case, to complement the
existing serialization test cases for "missing ToC" and "non-empty
ToC".
Change-Id: I8e2ff94f871546232cbf099f98bfcce559de76f2
(cherry picked from commit cb2173c57276142adc466f7494016e2faf15b758)
Why:
* Maintenance scripts in core have low test coverage
* This can cause issues such as the findDeprecated.php maintenance
script not working, as described in T374832
* Testing the broken script after fixing it will avoid the script
breaking again.
What:
* Fix the findDeprecated.php maintenance script to actually
show when code is hard-deprecated.
* Modify the script to allow mocking of the MediaWiki install
path in PHPUnit tests
* Create FindDeprecatedTest which tests that the script produces
an expected output, without being specific about how the
code finds the methods (to allow the method to be changed
as discussed in T243403)
** To do this, create a folder in the data folder for the PHPUnit
tests that has a mock file structure allowing the test to
use a fixed list of soft and hard deprecated methods.
Bug: T374832
Bug: T371167
Change-Id: Ic4933cef95ef1af7fa3939625ac1747106c71230
Why:
- When temporary users are enabled, creating IP actors is disallowed
apart from specific cases, such as importing revisions authored by
anonymous users.
- If such a revision includes a category link and
wgRCWatchCategoryMembership is true, MediaWiki will fire a job to
create a corresponding RC entry, which will attempt to attribute the RC
to the anonymous IP that authored the imported revision and fail in
doing so.
What:
- Track whether a category membership change job was triggered by an
import, and allow RecentChange objects created by such jobs to create
anonymous actors.
Test Plan:
1. On a wiki with temporary accounts enabled and wgRCWatchCategoryMembership = true,
import a revision via Special:Import that was authored by an anonymous user
and contains a category link.
2. Verify that the import succeeds and that the corresponding RC entry
shows up.
Bug: T373318
Change-Id: I89abdca9c4ab8796a211df8b37c1bd7173a496e5
Why:
* Maintenance scripts in core are mostly untested and testing them
will help to avoid regressions.
* Doing this for frequently run scripts is important, as they
may fail in ways not caught by running them in CI.
What:
* Create CheckComposerLockUpToDateTest that fully tests the
associated maintenance script.
** Move test case composer JSON files used by the LockFileCheckerTest
class into the 'phpunit/data' folder instead of placing them
in the same directory to allow the new test class to use these
files too.
Bug: T371167
Change-Id: Id473ef440c6b7f8ecbb1c869f9cc0cf98705a0e9
Developers can use this to test their local version of Codex with
MediaWiki by pointing $wgCodexDevelopmentDir to their local clone of the
Codex repo, e.g. $wgCodexDevelopmentDir = '/home/yourname/git/codex';
Setting $wgCodexDevelopmentDir affects where the following things come
from:
- Codex JS/CSS files for the full library
- Codex JS/CSS files for code-split chunks, and the manifest.json file
that points to them
- Icons retrieved by CodexModule::getIcons()
- CSS-only icons imported in Less
- Design tokens imported in Less
Other changes in this patch:
- Add CodexModule::makeFilePath() to centralize the repeated path
concatenation. This makes it easier to switch out the regular path for
the dev mode path.
- Replace all uses of $IP (which is deprecated) and MW_INSTALL_PATH in
CodexModule with the BaseDirectory config setting.
- Make CodexModule::getIcons() reset its static cache if the path to the
icons file changes. Without this, it's impossible to make the unit
tests pass.
- Move the i18n messages code from the CodexModule constructor to
getMessages(). It can't be in the constructor because makeFilePath()
doesn't work there (it fails because the Config object hasn't been set
up yet).
- Add a 'mediawiki.skin.codex' import path so that we can stop
hard-coding the path to the Codex mixins file. Without this, we can't
make the Codex mixins come from the right place in development mode.
- Consider $wgCodexDevelopmentDir in setting the cache key for compiled
Less code, since changing this setting can change the output of Less
compilation (by changing design tokens, icons or mixins).
- Add unit tests for (the non-dev mode behavior of)
CodexModule::getIcons() and the i18n message key handling.
Bug: T314507
Change-Id: I11c6a81a1ba34fe10f4b1c98bf76f0db40c1ce98
Previously, ResourceLoader's CodexModule class was using a hard-coded
list of message keys to add to the payload any time Codex components
get used. But now we can get the list of messages directly from
Codex. This change replaces the hardcoded list of strings with
a call to file_get_contents().
The list of messages is defined in a file called "messageKeys.json"
in the Codex package. In the next release of Codex, this file will
be pulled in via foreign-resources.yml; for now it has been added
manually to the Codex files in resources/lib.
Bug: T371330
Change-Id: Ib7ca66d67153dfba72d8d49b0181d49b007eedce
Why:
* The MaintenanceRunnerTest class tests using real maintenance
scripts and the code being tested can include the maintenance
script files.
* However, if running all tests together (such as when generating
test coverage) the maintenance scripts may have already been
loaded by integration tests that exist for those scripts.
* Instead of fixing the specific instances, we should instead
ensure that (where necessary) the MaintenanceRunnerTest tests
using fake maintenance scripts only loaded by that test class.
* Doing this prevents the files from being loaded anywhere else
and ensures that the tests should not fail in this way again.
What:
* Update MaintenanceRunner::expandScriptFile to move the use of
MS_INSTALL_PATH to a protected method that can then be mocked
in MaintenanceRunnerTest.
* Update MaintenanceRunnerTest to use fake maintenance scripts
where necessary. This includes:
** No longer setting the current working directory as the
MW_INSTALL_PATH, but instead set it as __DIR__.
** Creating several fake maintenance scripts to replace the uses
of the real ones.
** Updating the data providers to reflect the changes.
Bug: T371188
Change-Id: Ifacdcb8de5af53d2d5f4ed354af0d209384d7325
Ensure forward compatibility with
Ia1017dcef462f3ac1ff5112106f7df81f5cc384f by adding
forward-compatibility test data generated by
Ia1017dcef462f3ac1ff5112106f7df81f5cc384f and verifying that tests
pass.
Change-Id: I109640b510cef9b3b870a8c188f3b4f086d75d06
I realized that even though we have a lot of ParserCache serialization
tests, and we have unit tests for JsonCodec, we don't have any tests
which bring these together and include JsonDeserializable data inside
ParserOutput objects.
Change-Id: I9dfe1050c18010d1447e77e0670e9e601449b110
MessageValue and friends are pure value objects and newable, so
it makes sense for them to be (de)serializable too. There are some
places where we want to serialize messages, such as in ParserOutput.
The structure of the resulting JSON is inspired by the way we
represent Message objects as plain values elsewhere in MediaWiki,
e.g. StatusValue::getStatusArray().
Co-Authored-By: C. Scott Ananian <cscott@cscott.net>
Depends-On: Ia32f95a6bdf342262b4ef044140527f0676402b9
Depends-On: I7bafe80cd36c2558517f474871148286350a4e76
Change-Id: Id47d58b5e26707fa0e0dbdd37418c0d54c8dd503
This matches the current status of the ParserOutput serialization for
REL1_42, which uses an empty string for the unused TOCHTML property.
Change-Id: Idcad27d0403dabf7478268e43aeb710da9af9fcd
CODEX_MODULE_DIR is always the /modules subdirectory of
CODEX_LIBRARY_DIR, so remove it and derive it that way instead.
- Update tests accordingly
- Rename test data files accordingly
- Use CODEX_LIBRARY_DIR in one place where we had neglected to use a
constant
Change-Id: Ieb347c2d128f3addfb11f6aca79d63969434f342
In addition to the webp spec, this also detects two non-standard
conventions sometimes found in real files: Allowing a fourcc
of "XMP\0" instead of the standard "XMP " and allowing the prefix
"Exif\0\0" at the beginning of the EXIF chunk.
Bug: T338341
Bug: T353981
Change-Id: I0ddc3322e1c2f4d4d7bb11732e05fd4ffb5eb692
This is an output transform to resolve the mw:I18n and mw:LocalizedAttrs
to their localized forms.
Bug: T358191
Change-Id: Id32bc05ff72eb2d9fba7f8c2f192c9f7812cbc70
was disabled in I60b66ff86e4c62 due to some weird issues with supporting
doctrine/dbal 2 and 3.
Regenerating the sqlite .sql patch with dbal 3
Follow-Up: I17d74705cdd5a2e1af0bda66bfa466067b8387a8
Change-Id: I2349e2cbb1760edef12e857c6269056322f57205
Importing them without using (reference) causes comments from the Codex
file to be output (but only if they're /* comments */, not // comments),
which will break these tests when we upgrade to Codex 1.5.0.
Bug: T363712
Change-Id: Idb5b70e342d3f08edcabda2e8477ec2d259846cb
The TranslationAliasesDirs configuration allows defining translatable
aliases in JSON files. The value should be a name or names of folders
that contains files that have localized aliases. Each language should
have a separate file.
Currently, it supports defining special page aliases but in the
future can be extended to support magic words and namespace aliases.
The patch adds a script: ConvertExtensionsMessagesToTranslationAlias
that can be used to convert existing ExtensionMessagesFiles to the new
format.
Bug: T89947
Change-Id: Ief16a48a8dc8742854f67301791aa2a0b0531116
Less.php no longer requires mapping color names (9bdb3fda04/lib/Less/Colors.php (L9))
to their respective hex values, see: I0e6fa0fa88b4ab8dc41c55524661
(T352866).
If the color name is valid CSS, the color keyword (e.g. "green" is
preseved by the less compiler).
Change-Id: Ic91c9bb1d7b931f389cfaf3457d2fee845b9c3f0
Covered:
- `testMustRender`: Confirmed handler's necessity to render the file.
- `testGetThumbType`: Verified thumb type returned for BMP files.
- `testGetSizeAndMetadata`: Tests with various BMP file sizes.
Change-Id: Ib01914e1ab66ca10ff4e089a0fd591d0b544edf5
If a generated chunk is listed in the codexComponents array,
an exception will be thrown informing the developer that the
item is not an export of Codex and cannot be required.
Text fixtures and expected values were updated to match
recent changes to the manifest that made composables
explict entries.
Bug: T353191
Change-Id: Ie3e35501993b1ad5d29ebecbcc8203e7f4ea8c81
This requires the path to the Codex modules directory
to be configurable, so we can use static manifest and
component files in tests. This is achieved by changing
the constant storing the Codex modules dir from private
to protected so it can be overridden when creating a
test module.
Bug: T350054
Change-Id: Icd3c6c5d2a82f5a5fb1c38b52e54893034e7375c
Ironically, our replacements to improve readability of Vue templates in
debug mode actually ended up eating newlines in the string that is sent
to the Vue template compiler. This caused bugs when inline comments were
used in multi-line JS expressions in templates.
Bug: T351771
Change-Id: I6b3ad5736a836ba75f57a49fd9b7ca2e237074c0
This has been constantly mentioned as buggy and broken and there is no
official version of latin or Arabic (see the ticket for more details).
This can be turned back as an extension if needed by third party users.
Bug: T350684
Bug: T268143
Depends-On: I6180dca2c49b3119751766268acc56087aaf8414
Change-Id: Ifbf3c8954d885daf891f8d9efc11743d898302f0
This does a string check on animated and @keyframe in style elements
for SVGs. If there is a match it will treat this SVG as animated.
While in theory this could be a false positive, without fulling
parsing the CSS, the chances of that seem minor and there are no
real negative consequences either.
Bug: T332790
Change-Id: I4b8b0781e8f9135d9ab856f3ec06f5a76c66c9a8
* Unrelated to the ResourcesTest structure test as it isn't testing
any of core's resources.
* Moved to the libs/Minify repo in change Ia9018a966a5325d.
Change-Id: If87035777633635fe4ad0f01790e7084f56f65a0
In preparation for Codex's upcoming 1.0 release, this patch makes
some additional changes in how the library is vendored into MW.
* foreign-resources.yml: codex-search is removed as a stand-alone
package (the upstream NPM package has also been deprecated).
The search-specific build of Codex now lives in the main "codex"
package. The foreign-resources.yml file has been updated to pull
in the codex-search scripts and styles from this new location.
* resources.php: The resource module definitions for codex-search
and codex-search-styles have been updated to use the relocated
CSS/JS files.
* UMD bundles: Upstream, Codex now designates the CJS bundle as the
primary entry point. A UMD bundle of the main Codex library is still
produced but this is intended mainly for usage through CDNs or basic
script-tag inclusion; in the future the codex.umd.js bundle may be
removed from MediaWiki and users should not rely on it. The
codex-search.umd.js bundle has been removed.
* MJS bundles: Codex also produces .mjs (ESM) bundles upstream;
for users outside of MediaWiki these are the recommended bundles
to use. However, within MediaWiki only the CommonJS build should be
used, as ResourceLoader does not support delivery of ESM scripts at
present. The codex.mjs and codex-search.mjs bundles have been removed
from MediaWiki with this patch.
Bug: T335324
Bug: T340020
Bug: T340119
Bug: T344848
Bug: T345281
Bug: T345688
Bug: T346099
Bug: T346435
Bug: T346988
Change-Id: I84b26ddf5f9a0a98faf0df8c7acaa5f678032786
Down from about 1000ms to 200ms. By using much smaller images. The
absolute size of the images is not relevant for this test. Only the
ratio.
There was nothing special about the "landscape-plain.jpg" image. But
"portrait-rotated.jpg" was: It's technically stored in landscape, but
marked with an EXIF tag for rotation. I used exiftool to copy the EXIF
data to the smaller version, so it behaves the same as before.
Change-Id: Ie977dfbceff02e4dfde38a5ef82e8ff81b4d6980
The new block in testRecacheExtensionMessagesFiles() covers the
underlying issue of T343343 and T343375; see change Icc3324aca7 for an
explanation of the issue.
In order for this test to not crash, we also need to adjust some of the
other tests and data. MessagesEn.php defines $preloadedMessages, and
LocalisationCache assumes (quite rightly so, really) that all of these
messages will be found in the configured MessagesDirs. But in the test,
we override MessagesDirs to point at the test’s JSON files, while still
using the original MessagesEn.php with its $preloadedMessages. Now that
we are triggering the preload mechanism (I think it wasn’t reached by
the tests previously?), it triggers a PHP deprecation (since PHP 8) on
the missing messages, which makes the test fail:
> explode(): Passing null to parameter #2 ($string) of type string is deprecated
I decided to fix this by adding fake entries for all $preloadedMessages
to the test en.json file. This makes the other tests that compare the
entire messages array fail, so instead only compare the three messages
those tests care about ("present-*"). I don’t like this very much, but I
dislike the other two approaches I can think of even more: check
defined( 'MW_PHPUNIT_TEST' ) in the original MessagesEn.php file, or add
a test-specific MessagesEn.php using the Language::getMessagesFileName
hook.
Bug: T342418
Bug: T343343
Bug: T343375
Change-Id: Iee11cb5c52cb9dd777b70a1daa06f41f2c3ca187
Previously, it was technically possible to set some keys in extension
messages files that didn’t make much sense (e.g. $rtl['en'] = true;).
This prevents optimizing language creation, so going forward, we will no
longer support that; ALL_KEYS is now split into CORE_ONLY_KEYS and
ALL_EXCEPT_CORE_ONLY_KEYS (with a test verifying that no key is missing
or overlapping or anything), and ALL_EXCEPT_CORE_ONLY_KEYS are silently
skipped when loading extension messages files.
To demonstrate that it’s okay to silently skip these keys, patch set 1
of this change on Gerrit instead raised a deprecation warning; CI
indicated that this warning was never hit. Codesearch [1] also suggests
that no known extension was actually using any of these keys. (The
DonationInterface [2] and LandingCheck [3] codesearch results can be
ignored: both of these define es-419 as a new language, using the
Language::getMessagesFileName hook to register their MessagesEs_419.php
as a “core” message file, not an extension message file.)
[1]: https://codesearch.wmcloud.org/search/?q=^\%24(fallback|rtl|(digit|separator)TransformTable|fallback8bitEncoding|link(PrefixExtension|Trail|PrefixCharset)|date(Formats|Preferences|PreferenceMigrationMap)|defaultDateFormat|digitGroupingPattern).*%3D&files=\.php%24
[2]: f8b5fe95f7/gateway_common/messages/MessagesEs_419.php (11)
[3]: 2537439aee/messages/MessagesEs_419.php (11)
Bug: T342418
Change-Id: Ia3dffea390d4efdfa3a3cea549d079507718ef48
Given that we now have abstract schema, any schema produced for SQLite
will be produced specificly for SQLite, no need to do regex gymnastics
that are hard to understand, maintain and obviously prone to break.
Bug: T326181
Change-Id: I58741ff82460cfd9a350440d50b293ba67e3c939
Importing Less files from @wikimedia/codex-design-tokens/ doesn't work,
on purpose. Instead of letting these imports fail with a nondescript
"file not found" error that doesn't help the developer understand why it
didn't work, provide a descriptive error message that points them in the
right direction.
Bug: T328602
Change-Id: I992ccde79a59ad51c7ebfe3ac7548a6e531f4a59
This makes all Codex mixins available anywhere that the skin variables
are imported, so MediaWiki code doesn't need to import these mixins
separately. This also ensures that the tokens are present when the
mixins are used.
Bug: T328602
Change-Id: If2b4073115d9082829e4917c87c167d18f5431d9
