Commit graph

195 commits

Author SHA1 Message Date
jenkins-bot
7061fd2ee2 Merge "Reapply "Move limit report rendering to ParserOutput"" 2021-11-11 00:19:44 +00:00
Ed Sanders
9fe2995989 build: Update eslint-config-wikimedia to 0.21.0
* Replace `substr`/`substring` with `slice`. The second argument
  to `substring` is length, not end index as in `substr`/`slice`,
  so convert where necessary.
* Replace `new Date().getTime()` with `Date.now()`
* Documentation fixes
* Replace `throw Error` with `throw new Error`

Change-Id: I532500ea4c99d8ebec01efb21273c8df21626e59
2021-11-09 21:08:55 +00:00
Ppchelko
643fc535c3 Reapply "Move limit report rendering to ParserOutput"
This reverts commit 2bcb3fe567.

Reason for revert: this is a good change,
just needed more work to not break CI

Change-Id: I23768bee242e3cf81b1493a740cf070e7ad1e224
2021-11-09 11:08:08 -08:00
Fomafix
4f71fcdfcc Remove the executable flag from test files
chmod a-x tests/phpunit/data/MSCompoundFileReader/*

Change-Id: Ic2fa5f118b40126bea429d2299563c03fe6b3d1a
2021-10-16 11:23:02 +00:00
Petr Pchelko
a1aa3e0827 Hard-deprecate all public property access on CacheTime and ParserOutput.
- Added a test where ParserOutput objects with CacheTime
properties set are unserialized from previous versions.
- Generate new serialization tests for 1.38

Now all serialization in production is JSON, so changing
property visibility shouldn't affect ParserCache.

Bug: T263851
Depends-On: I283340ff559420ceee8f286ba3ef202c01206a23
Change-Id: I70d6feb1c995a0a0f763b21261141ae8ee6dc570
2021-10-13 13:27:16 -04:00
Amir Sarabadani
c3fff36540 Add 1.36 and 1.37 snapshots to sqlite db tests
To make sure upgrades from those versions work properly

Change-Id: I570cf965286bc5358a43c9601217b8781bb09b0e
2021-09-29 20:13:38 +02:00
Amir Sarabadani
753a4ecdd3 Bump minimum required version for upgrade to 1.29
Since the branch cut has happened, we can bump and get rid of legacy
cruft. According to the policy we can go up to 1.31 but let's keep it
that way to avoid major distruptions.

Change-Id: I9d697445a3bb5047726c8b2a7f808edb8403cdda
2021-09-28 17:57:57 +02:00
Fomafix
ff7f41f928 resourceloader: Append newline to JavaScript only if missing
In debug mode of a packageFiles module there is now only one newline
between the JavaScript code and the closing }.

This change reuses ResourceLoader::ensureNewline and make this to a
interal public static function.

Change-Id: I89519896e3dc56d966c4a63102904686bff6fac9
2021-08-28 12:25:11 +00:00
jenkins-bot
da9f37a177 Merge "resourceloader: Add integration test for ResourceModuleSkinStyles" 2021-08-19 01:32:26 +00:00
Timo Tijhof
76245d70ff resourceloader: Add integration test for ResourceModuleSkinStyles
This was covered in small ways already, but we did not have coverage
of the registration logic where the skin attribute is injected into
the module info, and this then correctly getting read and composed
into a single stylesheet.

This is prep for an optimisation in a subsequent commit.

Change-Id: Ia6fff44a345bb4f5811956638cada8b3e6b6ea71
2021-08-19 01:14:00 +00:00
Fomafix
6157bdaef0 CSSJanus: Update from version 1.3.0 to version 2.0.0
This version fixes a flipping bug.

Bug: T288943
Depends-On: Ida34372a4e5acb3147d83c7a1d008850c8382e32
Change-Id: I0528443e4eae0338f5a37491352db03944c6bd4c
2021-08-16 18:39:41 +00:00
libraryupgrader
5357695270 build: Updating dependencies
composer:
* mediawiki/mediawiki-codesniffer: 36.0.0 → 37.0.0
  The following sniffs now pass and were enabled:
  * Generic.ControlStructures.InlineControlStructure
  * MediaWiki.PHPUnit.AssertCount.NotUsed

npm:
* svgo: 2.3.0 → 2.3.1
  * https://npmjs.com/advisories/1754 (CVE-2021-33587)

Change-Id: I2a9bbee2fecbf7259876d335f565ece4b3622426
2021-07-22 03:36:05 +00:00
Tim Starling
2e507003ca Ignore invalid chunks in PNG files, instead of aborting metadata extraction
* Factor out file read errors and unexpected EOF errors.
* For errors relating to chunk content, instead of throwing an
  exception which is silently discarded, just log an error and continue
  to the next chunk. This allows the dimensions to be extracted even if
  other metadata is mangled.
* As an additional sanity check, verify the CRC of each chunk.

Bug: T286273
Change-Id: I11d0186496324e0bb1bb0a143f438e0368a8e902
2021-07-13 11:11:03 +10:00
Timo Tijhof
3371e4031c tests: Remove unused composer.json sample data
Follows-up c351c4f06e.

Change-Id: I842690849590f2ee57eb951099b9131728ecb7d7
2021-06-08 23:56:01 +00:00
MacFan4000
c351c4f06e Freenode -> Libera per wikimedia moving from freenode to libera
Bug: T283247
Change-Id: Iebbb13ac53830a8d77b42bf2954cf80d76d78467
2021-05-20 22:10:24 +00:00
Timo Tijhof
39e2abd886 registration: Allow multi-line strings in "@" note keys
Change-Id: Ifb1a0f04cc22b122ed99308c096d0ce4296e9097
2021-04-29 02:47:47 +01:00
Ammarpad
e497482ee4 DoctrineSchemaBuilder: Do not add prefix placeholder for Postgres at all
Adding it (only to be removed later) is not useful and interferes with
the internals of the schema builder because it masks the actual table
name during the table sql building and at some stages, this actual name
is required to be known

Bug: T191231
Change-Id: I57cc9b83c81f5c2d0d50c7b9e0dc334e7251c74f
2021-04-06 00:23:14 +00:00
jenkins-bot
e623a91aab Merge "Fix replacement of control chars in DJVU text output" 2021-03-29 15:38:33 +00:00
Inductiveload
32ea3a3fbe Fix replacement of control chars in DJVU text output
The control characters are presented as text, not actual
control characters, so the regexes to replace them are
incorrect.

Added a column and para to the Djvu text on the first page
of the test LoremIpsum.djvu file

Bug: T230415
Change-Id: I4970bc30b3935ce4da062ee7ff687aa667027a00
2021-03-29 06:03:34 +00:00
Derk-Jan Hartman
114d8e1cc5 JPEG2000 MIME fixes
- jp2 is the preferred file extension
- add mime aliases
- add jpg/jpx (jpeg2000 part 2)
- add magic byte detection for both

Bug: T161934
Change-Id: I00f81d9c31bf53a3b950e8a1d2bf2684c3998745
2021-03-15 21:43:24 +01:00
DannyS712
c0d41c57cd test data: remove old cssmin test files
No longer needed, now in the minify repo

Follow-up: I8d71fc64aeecdb31db218b02e361ae9bb0d19b48
Change-Id: Idb8175eb032a3521d08982da5be24041ab8758d3
2021-03-11 20:02:39 +00:00
mainframe98
9b8d6612c6 Support Composer 2.0 in ComposerInstalled
Composer 2.0 provides installed packages under the 'packages' key
in installed.json.

Bug: T266419
Change-Id: I80b690fde1653c7a15f8e211782fa8e60971daa9
2021-03-04 09:29:34 +01:00
James D. Forrester
5a622b6a2e build: Upgrade eslint-config-wikimedia from 0.17.0 to 0.18.1
Change-Id: I5e3687be2b197134578126e1b890ee37dbc1bc1b
2021-02-18 08:39:09 -08:00
Amir Sarabadani
78e6c48156 Rework sqlite schema changes tests
We don't support upgrading from below 1.31 officially and from below 1.27, the
upgrade system doesn't work at all.

The files are copied from the release branches.

Bug: T259771
Change-Id: Ib5251f7ae38148e00ac2dccb7bde76529c9d5076
2021-02-05 16:21:24 +00:00
jenkins-bot
607c4d3910 Merge "CacheTime: remove mVersion field" 2021-01-25 19:12:05 +00:00
Timo Tijhof
f7e8bc6a69 exception: Add test for MWExceptionHandler trace formatting
Change-Id: I00b9607ba7e17c1f75ef065e83b83d2e1a82870c
2021-01-13 23:26:48 +00:00
Timo Tijhof
8ad97d7c32 resourceloader: Fail gracefully if a LESS message is not found
The message cache is originally meant for mw.messages in JS,
which expects non-existent messages to be cleanly omitted.
There is minimal server-side and client-side handling in place for this.

For LESS, however, we were assuming that the blob is complete,
thus not feeding anything to the LESS compiler, thus leading to
a run-time failure where the LESS file can't be parsed at all
due to an undeclared variable.

This could happen sometimes during development or after upgrading
a wiki with a stale LocalisationCache that is still being updated
at that time.

Bug: T267785
Change-Id: I60ff4eb7dce1fee56470acc177afd29ee14b764f
2021-01-07 15:41:02 -08:00
Petr Pchelko
2010a912c2 CacheTime: remove mVersion field
Has to be merged at least one train after the parent patch.

Bug: T269393
Change-Id: Iee98950ad0452a50088e755252da8468f0bec1ae
2021-01-04 10:29:52 -06:00
Daniel Kinzler
48172f794d Revert "Hard-deprecate all public property access on CacheTime and ParserOutput."
This reverts commit b1a30eb0c4.

Reason for revert: T269396

Bug: T269396
Change-Id: I374ca13ccc30418b8fe3bf98f5090f7643aac4d7
2020-12-04 11:47:55 +00:00
Petr Pchelko
b1a30eb0c4 Hard-deprecate all public property access on CacheTime and ParserOutput.
Bug: T263851
Change-Id: I3d3ff7b5b6899150df836e445b56896dfd5b887e
2020-11-19 10:12:41 -07:00
Petr Pchelko
b956c77d27 Merge CacheTime and ParserOutput accessedOptions properties
Change-Id: I5785596d68e8923f8bcbd182ace0b1991bd75c9a
2020-11-19 10:12:39 -07:00
Petr Pchelko
dbdc2a3cd3 Introduce JsonCodec to help with serialization/deserialization
Change-Id: I5433090ae8e2b3f2a4590cc404baf838025546ce
2020-11-19 08:32:21 -07:00
jenkins-bot
d009d75c73 Merge "Start generateSchemaChangeSql.php" 2020-11-16 08:40:36 +00:00
Petr Pchelko
7c68ae9296 Safe ParserOutput extension data and JsonUnserializable helper.
One major difference with what we've had before is that now we
actually write class names into the serialization - given that
this new mechanism is extencible, we can't establish any kind
of mapping of allowed classes. I do not think it's a problem
though.

Bug: T264394
Change-Id: Ia152f3b76b967aabde2d8a182e3aec7d3002e5ea
2020-11-10 11:21:09 -07:00
Petr Pchelko
017cfcf016 Forward-compat for merging CacheTime and ParserOutput mOptions
CacheTime::mUsedOptions and ParserOutput::mAccessedOptions
do exactly the same thing and has to be merged into a single property.
This patch adds forward-compatibility and needs to be deployed
at least one train before the patch which actually merges the properties.

Change-Id: Ic9d71a443994e2545ebf2a826b9155c82961cb88
2020-11-10 07:09:41 -07:00
daniel
cac89b547c ParserOutput: add support for binary properties in JSON.
This introduces a mechanism for encoding binary data in
strings set via setProperty(). This is needed to accommodate compressed
data as used by TemplateData, which uses gzip compression to make the
data fit into the page_props table.

Bug: T266200
Change-Id: I19fa0dea8c25d93fcdec9dc5ddd6f3c9c162b621
2020-11-04 18:52:09 +01:00
Amir Sarabadani
1125167b8d Start generateSchemaChangeSql.php
Creating schema changes from abstract schema and even abstracting a
schema change.

Bug: T230420
Change-Id: If626e866642af820dd70c5f9b0fe7c6a951e0a25
2020-11-01 21:14:53 +01:00
Ammar Abdulhamid
d095b56e25 Expand DoctrineSchemaBuilderTest
* Test the generated table for each platform
* Use the handy dirname() function instead of full raw string path

Bug: T191231
Depends-On: If344395615087c360597a5b3d66ea03e930b7d9b
Change-Id: I936c5d10924a46743c7f76df8526105eec7ff48d
2020-10-17 21:17:46 +00:00
daniel
600f64029f Use JSON for parser cache
This adds JSON serialization and deserialization capabilities
to CacheTime and ParserOutput.

NOTE: JSON serialization is disabled for now. Merging this patch
should not change behavior in production.

Bug: T263579
Change-Id: I18187e8bce573d21f6f1bd29106e07c63a6d2f4d
2020-10-13 16:28:52 -07:00
Petr Pchelko
8a879605d9 Add deserialization acceptance tests for ParserOutput
Bug: T264397
Change-Id: I6476fd9b8eff0e1b61ce5f43280d1cd9b7aaa77c
2020-10-12 08:55:32 +00:00
daniel
6eea7d7ed5 Add test infra for ParserCache serialization/deserialization
Based on Daniel's work at Ia6e70179b7ee5ce4e93888585ccc30d92da165c3
however was changed enough to move into a separate changeset.

More acceptance tests and data will be added in a followup commit.

Bug: T264397
Change-Id: I135187e83cbfa02b97c5656f0752f8bf1ceb58d0
2020-10-09 08:14:57 -06:00
Timo Tijhof
0c01d8cc52 resourceloader: Add skin-based 'mediawiki.skin.variables.less' import
Add the SkinLessImportPaths attribute for skin-specific LESS import
paths, which skins can use to override the mediawiki.skin.variables.less
file.

As a starting point, add the following 5 variables:

* device widths (3x)
  To help phase out 'mediawiki.ui/variables'. These are
  commonly used by MobileFrontend.

* @font-family-sans
  Recommended by Volker. Used by multiple skins.

* @border-radius-base
  Recommended by Volker as example of something that we currently
  hardcode in MediaWiki core for Vector and OOUI/WikimediaUI
  in 'mediawiki.widgets.datetime' but should instead be allowed
  to vary by skin and OOUI theme.

  Remove the hardcoded value for '@border-radius-base' in
  various places in favour of importing from mediawiki.skin.
  The default is a bare default of 0 (as border-radius is off
  by default in the browser).

  The value for Vector is restored there by I47da304667811.
  The value for MonoBook is improved by I000f319ab31b.

Bug: T112747
Change-Id: Icf86c930a3b5524254bb549624737d3b9dccb032
2020-09-15 10:05:53 +00:00
David Causse
01eada74ef Write category parent hierarchy when handling categorization
Since we do brute force update (DELETE ALL+INSERT ALL) of the parent
categories we need to always write the parent category hierarchy.

Bug: T246568
Change-Id: Id9c3282a97bf3fc3d5355cc08a0c9d833c72c9b2
2020-07-31 12:14:13 +02:00
David Causse
6cbe99618e categoryChangesAsRdf: Add test for categorization
The test is bit slow as it has to propagate the categorisation
changes through several jobs. Users have suffered multiple regressions
on this script that it might be worth the additional cost.

Change-Id: I666d0039cfe4ba4e2a17210e8070f836767598a1
2020-07-31 12:14:11 +02:00
Cindy Cicalese
c69489f44c Check for unknown slot.
Bug: T220525
Change-Id: I89ac1679ea70cb403433f8b97fe11e8eb6811280
2020-07-20 10:39:28 -04:00
daniel
7ffacacc6b MCR import
Bug: T220525
Change-Id: I7d41a8af12e33aed802526eb39a0880292c4f534
2020-07-18 20:34:16 -04:00
daniel
cac44ca333 Add import/export round trip test.
This creates the infrastructure we want to use later to test
improt and export of multiple slots.

Bug: T220525
Change-Id: I8e02927bd5532fe9cee0694d48a4c7d9ec060ba1
2020-07-14 15:55:55 +02:00
Roan Kattouw
8c6c1c9adc VueComponentParser: Use RemexHtml instead of PHP's HTML parser
This fixes an issue with HTML tags inside the <script> tag.
Remex also doesn't throw errors on attributes like @click, although it
does mangle them when producing DOM. To work around this, don't use DOM
serialization for the template HTML, but parse everything again using a
Remex parse+serialize pipeline that extracts the template and
(optionally) removes comments and strips whitespace.

One important effect of this change is that we'll have to forbid
self-closing tags in Vue templates, because Remex doesn't handle those
correctly (or rather, handles them *too* correctly). But on the up side,
we can now allow shorthands for v-bind/v-on/v-slot again.

Bug: T253334
Bug: T255587
Depends-On: I2253a2317187fe0d781ba5bfefab95e0f97d0a80
Change-Id: Id9a9728b7163601cc60bc587be07b70977d41970
2020-06-22 19:54:15 +00:00
Tim Starling
b9907ca9b6 Remove DatabaseIntegrationTest::testStoredFunctions()
As was pointed out in CR on the original change from December 2011, the
test requires elevated privileges when it is run against a database with
binlogging enabled, which is a nuisance when I'm trying to test replication-
related code. The author commented that the test was to support GeoData,
but that was only true until the "schema revamp" of February 2012.

I considered mocking the database and making it into a test of
streamStatementEnd, since that code was introduced at the same time, but
we already have DatabaseMysqlBaseTest::testStreamStatementEnd().

I also removed the restoreFlags() call since that was apparently left
over from testFlagSetting(), which has been moved to DatabaseTest.

Refs:

* https://www.mediawiki.org/wiki/Special:Code/MediaWiki/107376
* https://www.mediawiki.org/wiki/Special:Code/MediaWiki/108603
* https://www.mediawiki.org/wiki/Special:Code/MediaWiki/110649
* 67f08d6990

Change-Id: I1a6b9d0cd91c8539ac1080a423b519743088733e
2020-06-02 10:47:29 +10:00
Amir Sarabadani
dbbb3af6dc Small improvements to DoctrineSchemaBuilder and generateSchemaSql
Adding scope to const, adding docstring, adding comments to generated
files, fix default path.

Bug: T230428
Change-Id: I955a446421ffe5c8fa42aeedc0f962230e92e02a
2020-05-09 16:08:44 +02:00