Commit graph

180 commits

Author SHA1 Message Date
Timo Tijhof
39e2abd886 registration: Allow multi-line strings in "@" note keys
Change-Id: Ifb1a0f04cc22b122ed99308c096d0ce4296e9097
2021-04-29 02:47:47 +01:00
Ammarpad
e497482ee4 DoctrineSchemaBuilder: Do not add prefix placeholder for Postgres at all
Adding it (only to be removed later) is not useful and interferes with
the internals of the schema builder because it masks the actual table
name during the table sql building and at some stages, this actual name
is required to be known

Bug: T191231
Change-Id: I57cc9b83c81f5c2d0d50c7b9e0dc334e7251c74f
2021-04-06 00:23:14 +00:00
jenkins-bot
e623a91aab Merge "Fix replacement of control chars in DJVU text output" 2021-03-29 15:38:33 +00:00
Inductiveload
32ea3a3fbe Fix replacement of control chars in DJVU text output
The control characters are presented as text, not actual
control characters, so the regexes to replace them are
incorrect.

Added a column and para to the Djvu text on the first page
of the test LoremIpsum.djvu file

Bug: T230415
Change-Id: I4970bc30b3935ce4da062ee7ff687aa667027a00
2021-03-29 06:03:34 +00:00
Derk-Jan Hartman
114d8e1cc5 JPEG2000 MIME fixes
- jp2 is the preferred file extension
- add mime aliases
- add jpg/jpx (jpeg2000 part 2)
- add magic byte detection for both

Bug: T161934
Change-Id: I00f81d9c31bf53a3b950e8a1d2bf2684c3998745
2021-03-15 21:43:24 +01:00
DannyS712
c0d41c57cd test data: remove old cssmin test files
No longer needed, now in the minify repo

Follow-up: I8d71fc64aeecdb31db218b02e361ae9bb0d19b48
Change-Id: Idb8175eb032a3521d08982da5be24041ab8758d3
2021-03-11 20:02:39 +00:00
mainframe98
9b8d6612c6 Support Composer 2.0 in ComposerInstalled
Composer 2.0 provides installed packages under the 'packages' key
in installed.json.

Bug: T266419
Change-Id: I80b690fde1653c7a15f8e211782fa8e60971daa9
2021-03-04 09:29:34 +01:00
James D. Forrester
5a622b6a2e build: Upgrade eslint-config-wikimedia from 0.17.0 to 0.18.1
Change-Id: I5e3687be2b197134578126e1b890ee37dbc1bc1b
2021-02-18 08:39:09 -08:00
Amir Sarabadani
78e6c48156 Rework sqlite schema changes tests
We don't support upgrading from below 1.31 officially and from below 1.27, the
upgrade system doesn't work at all.

The files are copied from the release branches.

Bug: T259771
Change-Id: Ib5251f7ae38148e00ac2dccb7bde76529c9d5076
2021-02-05 16:21:24 +00:00
jenkins-bot
607c4d3910 Merge "CacheTime: remove mVersion field" 2021-01-25 19:12:05 +00:00
Timo Tijhof
f7e8bc6a69 exception: Add test for MWExceptionHandler trace formatting
Change-Id: I00b9607ba7e17c1f75ef065e83b83d2e1a82870c
2021-01-13 23:26:48 +00:00
Timo Tijhof
8ad97d7c32 resourceloader: Fail gracefully if a LESS message is not found
The message cache is originally meant for mw.messages in JS,
which expects non-existent messages to be cleanly omitted.
There is minimal server-side and client-side handling in place for this.

For LESS, however, we were assuming that the blob is complete,
thus not feeding anything to the LESS compiler, thus leading to
a run-time failure where the LESS file can't be parsed at all
due to an undeclared variable.

This could happen sometimes during development or after upgrading
a wiki with a stale LocalisationCache that is still being updated
at that time.

Bug: T267785
Change-Id: I60ff4eb7dce1fee56470acc177afd29ee14b764f
2021-01-07 15:41:02 -08:00
Petr Pchelko
2010a912c2 CacheTime: remove mVersion field
Has to be merged at least one train after the parent patch.

Bug: T269393
Change-Id: Iee98950ad0452a50088e755252da8468f0bec1ae
2021-01-04 10:29:52 -06:00
Daniel Kinzler
48172f794d Revert "Hard-deprecate all public property access on CacheTime and ParserOutput."
This reverts commit b1a30eb0c4.

Reason for revert: T269396

Bug: T269396
Change-Id: I374ca13ccc30418b8fe3bf98f5090f7643aac4d7
2020-12-04 11:47:55 +00:00
Petr Pchelko
b1a30eb0c4 Hard-deprecate all public property access on CacheTime and ParserOutput.
Bug: T263851
Change-Id: I3d3ff7b5b6899150df836e445b56896dfd5b887e
2020-11-19 10:12:41 -07:00
Petr Pchelko
b956c77d27 Merge CacheTime and ParserOutput accessedOptions properties
Change-Id: I5785596d68e8923f8bcbd182ace0b1991bd75c9a
2020-11-19 10:12:39 -07:00
Petr Pchelko
dbdc2a3cd3 Introduce JsonCodec to help with serialization/deserialization
Change-Id: I5433090ae8e2b3f2a4590cc404baf838025546ce
2020-11-19 08:32:21 -07:00
jenkins-bot
d009d75c73 Merge "Start generateSchemaChangeSql.php" 2020-11-16 08:40:36 +00:00
Petr Pchelko
7c68ae9296 Safe ParserOutput extension data and JsonUnserializable helper.
One major difference with what we've had before is that now we
actually write class names into the serialization - given that
this new mechanism is extencible, we can't establish any kind
of mapping of allowed classes. I do not think it's a problem
though.

Bug: T264394
Change-Id: Ia152f3b76b967aabde2d8a182e3aec7d3002e5ea
2020-11-10 11:21:09 -07:00
Petr Pchelko
017cfcf016 Forward-compat for merging CacheTime and ParserOutput mOptions
CacheTime::mUsedOptions and ParserOutput::mAccessedOptions
do exactly the same thing and has to be merged into a single property.
This patch adds forward-compatibility and needs to be deployed
at least one train before the patch which actually merges the properties.

Change-Id: Ic9d71a443994e2545ebf2a826b9155c82961cb88
2020-11-10 07:09:41 -07:00
daniel
cac89b547c ParserOutput: add support for binary properties in JSON.
This introduces a mechanism for encoding binary data in
strings set via setProperty(). This is needed to accommodate compressed
data as used by TemplateData, which uses gzip compression to make the
data fit into the page_props table.

Bug: T266200
Change-Id: I19fa0dea8c25d93fcdec9dc5ddd6f3c9c162b621
2020-11-04 18:52:09 +01:00
Amir Sarabadani
1125167b8d Start generateSchemaChangeSql.php
Creating schema changes from abstract schema and even abstracting a
schema change.

Bug: T230420
Change-Id: If626e866642af820dd70c5f9b0fe7c6a951e0a25
2020-11-01 21:14:53 +01:00
Ammar Abdulhamid
d095b56e25 Expand DoctrineSchemaBuilderTest
* Test the generated table for each platform
* Use the handy dirname() function instead of full raw string path

Bug: T191231
Depends-On: If344395615087c360597a5b3d66ea03e930b7d9b
Change-Id: I936c5d10924a46743c7f76df8526105eec7ff48d
2020-10-17 21:17:46 +00:00
daniel
600f64029f Use JSON for parser cache
This adds JSON serialization and deserialization capabilities
to CacheTime and ParserOutput.

NOTE: JSON serialization is disabled for now. Merging this patch
should not change behavior in production.

Bug: T263579
Change-Id: I18187e8bce573d21f6f1bd29106e07c63a6d2f4d
2020-10-13 16:28:52 -07:00
Petr Pchelko
8a879605d9 Add deserialization acceptance tests for ParserOutput
Bug: T264397
Change-Id: I6476fd9b8eff0e1b61ce5f43280d1cd9b7aaa77c
2020-10-12 08:55:32 +00:00
daniel
6eea7d7ed5 Add test infra for ParserCache serialization/deserialization
Based on Daniel's work at Ia6e70179b7ee5ce4e93888585ccc30d92da165c3
however was changed enough to move into a separate changeset.

More acceptance tests and data will be added in a followup commit.

Bug: T264397
Change-Id: I135187e83cbfa02b97c5656f0752f8bf1ceb58d0
2020-10-09 08:14:57 -06:00
Timo Tijhof
0c01d8cc52 resourceloader: Add skin-based 'mediawiki.skin.variables.less' import
Add the SkinLessImportPaths attribute for skin-specific LESS import
paths, which skins can use to override the mediawiki.skin.variables.less
file.

As a starting point, add the following 5 variables:

* device widths (3x)
  To help phase out 'mediawiki.ui/variables'. These are
  commonly used by MobileFrontend.

* @font-family-sans
  Recommended by Volker. Used by multiple skins.

* @border-radius-base
  Recommended by Volker as example of something that we currently
  hardcode in MediaWiki core for Vector and OOUI/WikimediaUI
  in 'mediawiki.widgets.datetime' but should instead be allowed
  to vary by skin and OOUI theme.

  Remove the hardcoded value for '@border-radius-base' in
  various places in favour of importing from mediawiki.skin.
  The default is a bare default of 0 (as border-radius is off
  by default in the browser).

  The value for Vector is restored there by I47da304667811.
  The value for MonoBook is improved by I000f319ab31b.

Bug: T112747
Change-Id: Icf86c930a3b5524254bb549624737d3b9dccb032
2020-09-15 10:05:53 +00:00
David Causse
01eada74ef Write category parent hierarchy when handling categorization
Since we do brute force update (DELETE ALL+INSERT ALL) of the parent
categories we need to always write the parent category hierarchy.

Bug: T246568
Change-Id: Id9c3282a97bf3fc3d5355cc08a0c9d833c72c9b2
2020-07-31 12:14:13 +02:00
David Causse
6cbe99618e categoryChangesAsRdf: Add test for categorization
The test is bit slow as it has to propagate the categorisation
changes through several jobs. Users have suffered multiple regressions
on this script that it might be worth the additional cost.

Change-Id: I666d0039cfe4ba4e2a17210e8070f836767598a1
2020-07-31 12:14:11 +02:00
Cindy Cicalese
c69489f44c Check for unknown slot.
Bug: T220525
Change-Id: I89ac1679ea70cb403433f8b97fe11e8eb6811280
2020-07-20 10:39:28 -04:00
daniel
7ffacacc6b MCR import
Bug: T220525
Change-Id: I7d41a8af12e33aed802526eb39a0880292c4f534
2020-07-18 20:34:16 -04:00
daniel
cac44ca333 Add import/export round trip test.
This creates the infrastructure we want to use later to test
improt and export of multiple slots.

Bug: T220525
Change-Id: I8e02927bd5532fe9cee0694d48a4c7d9ec060ba1
2020-07-14 15:55:55 +02:00
Roan Kattouw
8c6c1c9adc VueComponentParser: Use RemexHtml instead of PHP's HTML parser
This fixes an issue with HTML tags inside the <script> tag.
Remex also doesn't throw errors on attributes like @click, although it
does mangle them when producing DOM. To work around this, don't use DOM
serialization for the template HTML, but parse everything again using a
Remex parse+serialize pipeline that extracts the template and
(optionally) removes comments and strips whitespace.

One important effect of this change is that we'll have to forbid
self-closing tags in Vue templates, because Remex doesn't handle those
correctly (or rather, handles them *too* correctly). But on the up side,
we can now allow shorthands for v-bind/v-on/v-slot again.

Bug: T253334
Bug: T255587
Depends-On: I2253a2317187fe0d781ba5bfefab95e0f97d0a80
Change-Id: Id9a9728b7163601cc60bc587be07b70977d41970
2020-06-22 19:54:15 +00:00
Tim Starling
b9907ca9b6 Remove DatabaseIntegrationTest::testStoredFunctions()
As was pointed out in CR on the original change from December 2011, the
test requires elevated privileges when it is run against a database with
binlogging enabled, which is a nuisance when I'm trying to test replication-
related code. The author commented that the test was to support GeoData,
but that was only true until the "schema revamp" of February 2012.

I considered mocking the database and making it into a test of
streamStatementEnd, since that code was introduced at the same time, but
we already have DatabaseMysqlBaseTest::testStreamStatementEnd().

I also removed the restoreFlags() call since that was apparently left
over from testFlagSetting(), which has been moved to DatabaseTest.

Refs:

* https://www.mediawiki.org/wiki/Special:Code/MediaWiki/107376
* https://www.mediawiki.org/wiki/Special:Code/MediaWiki/108603
* https://www.mediawiki.org/wiki/Special:Code/MediaWiki/110649
* 67f08d6990

Change-Id: I1a6b9d0cd91c8539ac1080a423b519743088733e
2020-06-02 10:47:29 +10:00
Amir Sarabadani
dbbb3af6dc Small improvements to DoctrineSchemaBuilder and generateSchemaSql
Adding scope to const, adding docstring, adding comments to generated
files, fix default path.

Bug: T230428
Change-Id: I955a446421ffe5c8fa42aeedc0f962230e92e02a
2020-05-09 16:08:44 +02:00
Reedy
26cf0fc2e8 tests: Add test file to test SPDX license expressions
Bug: T251642
Change-Id: I6950ab0c9d0ef080dbe247f5b73458aac4d27f8a
2020-05-02 12:47:31 +01:00
jenkins-bot
fbf4346a10 Merge "build: Provide linting for .vue files from upstream" 2020-04-16 01:11:55 +00:00
Roan Kattouw
bed84e3281 build: Provide linting for .vue files from upstream
Was first attempted in f1eaa06d5e,
and reverted in 0b3fe4fe8c. Now this
is done in eslint-config-wikimedia for all repos.

Change-Id: I9ee735eef40085b6373b565838d67e186e438657
2020-04-15 17:40:31 -07:00
daniel
5a58e4421c Make dump tests based on fixture files.
This adds test coverage for dumping non-main slots.

NOTE: This invents a syntax for XML file "templates" that are
expected to "match" the actual output file.

Bug: T240213
Change-Id: I05e46175809c4562f3530377fe37db911bee244a
2020-04-06 15:21:00 -07:00
Timo Tijhof
75ccdc6147 languages: Move default $wgNamespaceAliases to MessagesEn.php
These are not configuration but business logic, similar to the
canonical names that are in NamespaceInfo.php, these must always
exist and cannot be altered or unset.

They were previously unconditionally assigned during all requests
in Setup.php and passed down as "site configuration".

Changes:

* Move them to MessagesEn.php where they can be cached and
  processed the same way as other core-provided aliases.

  Document and confirm with tests that this is a mergeable
  attribute that follows the language chain.

* Remove the duplicated code in a few places that was reading
  this variable + Language::getNamespaceAliases(), to instead
  just call the latter and move the logic there, centralised,
  and tested.

  In doing so I noticed that these were applied in an
  inconsistent order. Sometimes the config won, sometimes not.
  There's no obvious right or wrong way here, but I've chosen
  to standardise on the way that Language::getNamespaceIds() did
  it, which is that config wins. This because that method seems
  to be most widely used of the three (it decides how URLs and
  titles are parsed), and thus the one I least want to change
  the behaviour of.

* Document that $wgNamespaceAliases may only be used to
  define (extra) aliases, it is and never was a way to access
  the complete list of aliases.

Bug: T189966
Change-Id: Ibb14181aba8c1b509264ed40523e9ab4000fd71a
2020-03-14 19:27:40 +00:00
Roan Kattouw
ca46126e98 resourceloader: Support single-file component .vue files
Allows .vue files to be used in package modules as if they were .js
files: they can be added to the 'packageFiles' array in module
definitions, and require()d from JS files.

In the load.php output, each .vue file is transformed to a function that
contains the JS from the <script> tag, then a line that sets
module.exports.template to the contents of the <template> tag (encoded
as a string). The contents of the <style> tag are added to the module's
styles.

Internally, the type of a .vue file is inferred as 'script-vue', and the
file is parsed with VueComponentParser, which extracts the three parts.
After the transformation, the file's type is set to 'script+style', and
files of this type contribute to both getScript() and getStyles().

This change also adds caching to getPackageFiles(), because it now needs
to be called twice (in getScript() and getStyles()).

Change-Id: Ic0a5c771901450a518eb7d24456053956507e1ed
2020-03-12 14:32:41 -07:00
Roan Kattouw
f8861b7622 TemplateParserTest: Add test case with syntax error
Change-Id: Id4860269e55345176ad58fd75dcf3522a8617fa2
2020-01-29 23:15:53 +00:00
jenkins-bot
9318f46267 Merge "Improve tests in DoctrineSchemaBuilder" 2020-01-06 11:00:54 +00:00
Kunal Mehta
d6390ba1ee registration: Implement lazy-loaded attributes
These attributes are only accessed at a time when performance isn't
critical (e.g. tests), so we can skip caching them and read each
extension.json file again when we do need them.

At the same time, we want to reduce the size of the array that we're
caching due to PHP 7 performance regressions with APC (T187154).

'QUnitModules' is the first attribute to be converted into a lazy-loaded
attribute.

Bug: T220994
Change-Id: I224cdace05dbac00ff947c41daecb74b07c967bb
2019-12-15 19:04:50 +00:00
Timo Tijhof
3e6db51725 resourceloader: Add tests for module-level 'noflip' option
Also widen up @covers for the integration tests to let the
coverage percentage increase from those.

Change-Id: Ib9d42b124774b4c968c52f34e587059c63e8ffc2
2019-12-03 15:58:36 +00:00
Amir Sarabadani
6d4dc90831 Improve tests in DoctrineSchemaBuilder
Also changing its structure to be more readable and more like
https://w.wiki/CXo

My plan is to add more tables to this test.

Bug: T230428
Change-Id: Ia19d5c875cc95238f7fd7cd9adbed1ddb92af078
2019-11-22 19:53:10 +01:00
Max Semenik
32df52aa37 ComposerInstalledTest: remove mentions of phpunit-mock-objects
We're not using it anymore, so removing it from unrelated tests to
prevent it from appearing in searches.

Change-Id: I83550477faea36f7d739e91fb449e0ee3f07ea18
2019-11-21 12:59:45 -08:00
Fomafix
d607ef8512 Remove traling double newlines from text files
Performed using:

git ls-files \*.json \*.php \*.xml \*.xsd \*.html \*.txt \*.sql | xargs sed -i -e :a -e '/^\n*$/{$d;N;};/\n$/ba'

Excluded tests/parser/preprocess.

Change-Id: I513538ff13c942acfe589f15c5128a8df2b3ace9
2019-10-23 21:17:43 +02:00
Max Semenik
dc3fba4619 tests: Nuke everything HHVM-related
Change-Id: Ieccce5ebba25132b2433fc13a575b65a5bd538f2
2019-10-05 12:51:07 -07:00
Derk-Jan Hartman
9ce26a564d Fix XMP parser errors due to trailing nullchar
JPEG files can have trailing \0 chars at the end of the XMP value. Use
trim() to remove these from the string value.

Bug: T118799
Change-Id: Id4ab223ef432e5d2c0dd3b4e332320db02422700
2019-09-11 22:34:33 +00:00