Commit graph

224 commits

Author SHA1 Message Date
jenkins-bot
251e02be96 Merge "Drop 1.34 db updates" 2023-02-15 12:51:34 +00:00
jenkins-bot
1ec7ef8fe4 Merge "upload: Allow attributes starting with "on" in inkscape SVG namespace" 2023-02-13 00:58:17 +00:00
Derk-Jan Hartman
c64a8faab6 upload: Allow attributes starting with "on" in inkscape SVG namespace
Inkscape has an attribute only-selected in it's namespace which
is not interpreted by browsers and should be safe upload.

Bug: T288186
Change-Id: I044f225aef813efc708eb96f588fc55733d165f3
2023-02-13 00:41:22 +00:00
Alexander Vorwerk
599b5a0918 Drop 1.34 db updates
Per official policy, updates from versions older than two TLS releases
are not supported.

Change-Id: I3dc98fe61c8a508aa3ade694f8c0a6a00115031b
2023-02-09 23:28:10 +00:00
Umherirrender
9bcd6f7f80 tests: Move all language converter tests into /includes/
Move tests from /languages/ to /includes/language/ similiar to the
classes in f5644ba
Clean up suite config too not longer run that folder as test,
remove from coverage, there no php files to cover

Remove converter tests from Language*Test files,
all identical to the *ConverterTest cases (or remove the whole file)

Change-Id: I098ba241bd61d8ed2ee9b65393416077e3544e79
2023-02-01 20:10:56 +01:00
Kevin Israel
8ae7194fae AutoLoader: Drop support for $wgAutoloadAttemptLowercase
This setting has been deprecated since 1.35.

Bug: T231412
Change-Id: I88cb45a2dcbb19c2976824279bf95a66ae9afd91
2022-12-04 02:38:57 -05:00
Subramanya Sastry
bcb7009c41 Use real section metadata in tests
* Most of the files were generated from the validate* script.
* Post-processing of these generated files to fix problems:
  - Some of the files were binary-edited via "vi -b" to fix some
    issues with bad property names used in the prior step.
    1.36, 1.38, 1.39 files were all fixed up this way.
  - In addition, the 1.36 file had bad data (not sure if the wrong
    php version was used) but I fixed this by splicing in data
    from the 1.38 file to revert incorrect changes to "Categories"
    and "IndexPolicy" properties.
  - The 1.35 data file was binary edited by splicing data from the
    now 1.36 version.

Change-Id: I4e22b94ce30c2ad9b1f544c15e1c3cd0dd0bce6b
2022-11-23 12:45:27 -05:00
Subramanya Sastry
af9eb85fdf ParserCache data: Delete data for unsupported versions
1.31, 1.34 versions are no longer supported.

Change-Id: I24530ed4ff62c80a60c3bc7bc69da60d3b9e19b1
2022-11-23 10:35:46 -05:00
Subramanya Sastry
623625e8f2 Followup to fb747bc0: Fix bad property names
Change-Id: I362b0cf8feca13a91fd91961d400579f2e4ea97e
2022-11-18 16:12:06 -06:00
Subramanya Sastry
fb747bc038 Add section metadata parsercache serialization tests for MW 1.40
* Generate data files for 1.40 only since the new formats only
  showed up in 1.40 and won't be present in the parser cache
  for older MW versions.

Change-Id: I6f297e3091ec2faab7c2203c138800551b01e32a
2022-11-17 15:48:15 -06:00
Tim Starling
0b12f8b698 Improve LocalisationCache post-merge validation check
When I deleted MessagesEn.php, neither the exception on line 498 nor
the exception modified here was thrown. Extensions register namespaces,
hiding the lack of default namespaces. If namespaceNames was missing,
RHS line 1004 would raise a warning but would create the key.

Also, phan was complaining that the key might be absent.

So, use 'rtl' instead of 'namespaceNames' as our test for data validity,
since it's less likely to be overridden by extensions. Move the check up
above the normalization of individual keys. Use isset() to satisfy phan.

Bug: T322278
Change-Id: I44c7506205fbe9ba6ffbd1155a6cdcf3ff0960eb
2022-11-09 12:57:23 +11:00
C. Scott Ananian
d96207ab86 Auto-discover core parser test files
Make parser test discover in core work the same way as it does in
extensions: any file ending with *.txt under tests/parser is run
as a parser test file.

This search is recursive, which is motivation to also move some
unrelated files under tests/parser/preprocess over to
tests/phpunit/data/preprocess where they belong; they are used
by tests/phpunit/includes/parser/PreprocessorTest.php and are
unrelated to the parser test infrastructure.

Change-Id: I8c84b4b853e1309929dceb700aab1e79a598d8ab
2022-10-13 10:41:15 -04:00
Amir Sarabadani
4bde067722 Drop 1.32 and 1.33 db updates
Per official policy, updates from versions older than two TLS releases
are not supported.

Finding the implicit marker is a bit tricky. The user might try to
upgrade from a really old version that doesn't have tag_summary table
and mistakenly think it's new and allow upgrade. So turning the check
logic into a boolean AND where it must be new enough to have
change_tag_def table AND new enough not to have tag_summary table.

Change-Id: I5ff9630a6539a587a47930847e108ac53757106f
2022-09-26 18:34:31 +02:00
Zabe
0c4fa03647 Add 1.39 snapshot to sqlite db tests
Change-Id: I6bf00a239fd3181c2815ddffa4b7f37bd4738dac
2022-09-06 18:32:46 +00:00
Derk-Jan Hartman
ae943daee4 SVG Language selector should default to page language
Taking into account the pagelanguage should help take into account the
language variant on image pages in languages that have variants.

Bug: T310445
Change-Id: Ic821b2de66ed876e47c74ac21cedaad7a70a80fc
2022-06-13 21:17:25 +00:00
Bartosz Dziewoński
566b185e9c resourceloader: Tweak RL\FilePath handling in packageFiles
ResourceLoader's FilePath is designed to allow a FileModule
to include files that exist outside of the module's localBasePath,
to allow skins and extensions to extend core MediaWiki modules.
This is accomplished by having the FileModule use the FilePath's
localBasePath instead, in FileModule::getLocalPath.
(Similarly for remoteBasePath, but these are going out of fashion.)

However, the code processing 'packageFiles' did not handle this right:
it used FilePath's localBasePath when it appeared as a 'file',
but not when it was returned by a 'callback' or 'versionCallback',
using the FileModule's localBasePath instead in that case. Most
existing uses of FilePath in 'packageFiles' required the same base
path as the module, so it was convenient to avoid providing it twice.

To keep that convenience (already relied on by some extensions too)
while also allowing skins and extensions to add files from their own
directories to existing modules, the code processing 'packageFiles'
now uses FilePath's base paths in all cases, but they are optional,
and will fall back to the FileModule's paths when not provided.

Follow-up to 2890bca27d.

Change-Id: I38a0761ae4633a567b685b52c1d73b6ce280ffb7
2022-05-27 22:40:41 +01:00
daniel
37af531721 AutoLoader: provide methods for registering classes
This introduces static methods for registering classes and namespaces,
so we don't have to mess with global variables and static members
fields.

Bug: T240535
Change-Id: I1abb27bd70898d04c9253e0b7467447fa96d15a4
2022-05-10 13:55:53 +10:00
Tim Starling
910d146679 Call ZipDirectoryReader from MimeAnalyzer
Remove the special case GIFAR vulnerability check from UploadBase. I
put it there because the GIFAR vulnerability was so severe that we
needed to check for it even if $wgVerifyMimeType was false. Now that
support for Java applets has been dropped from browsers, the
vulnerability is not so severe and we can wind back the paranoia.

Add a ZipDirectoryReader invocation to MimeAnalyzer, to replace some
terrible regex hacks. This allows MimeAnalyzer to detect docx files
saved by LibreOffice (T291750).

This is also the new place for the GIFAR check. Add application/java
to $wgMimeTypeExclusions so that JAR uploads are rejected.

Since ZipDirectoryReader cannot look inside the contents of files in the
package, it can't detect the subtype of Open Document files. Use the
file extension instead, like what we do for Microsoft OPC files.

Remove public method detectZipType -- no callers in code search.

Bug: T291750
Change-Id: Iff1611c7adda9c0f0ed31593bad6dfffc9c9a086
2022-05-03 14:43:26 +10:00
C. Scott Ananian
46acc1a257 Use uniform representation for ParserOutput "index policy"
As part of the project of enforcing uniform semantics for
combining ParserOutput objects (T300979) use standard boolean flags
for the 'index' and 'noindex' index policy metadata.

The forward-compatibility "1.39_wmf.7-ParserCache-*" serialization
test cases have been renamed to "1.39-ParserCache-*" in this commit;
backward compatibility with the prior representation of index policy
will continue to be tested via the "1.38-ParserCache-*" cases.

Bug: T300979
Change-Id: I683e5ae054a0425b03c60a4af8c845b576414c1d
2022-03-31 09:51:21 -04:00
C. Scott Ananian
664012e23f Ensure forward compatibility with new ParserOutput index policy fields
Instead of ParserOutput::$mIndexPolicy, a future MW version
(I683e5ae054a0425b03c60a4af8c845b576414c1d) will use two boolean fields
ParserOutput::$mIndexSet and ::$mNoIndexSet.  For parser cache migration
purposes, ensure that core can deserialize the new version so that
rollback are safe.

Add serialization test cases with the new boolean fields as
"1.39_wmf.7-ParserOutput-*"; compatibility with the existing
"mIndexPolicy" serialization will continue to be tested with the
"1.38-ParserOutput-*" cases.

Change-Id: I5e4fc68cea18b31ecb028b3867537dcbd86b93cd
2022-03-31 09:51:21 -04:00
C. Scott Ananian
ea4bedee9d Remove misleading test in SerializationTestTrain::validateObjectEquality()
The initial object comparison using `==` compares all the property values
using `==` as well.  This causes (for example) the string "1" to compare
as equal to the number `1`, the empty string `""` to compare as equal
to the number `0` and other surprising things.

It appears that this comparison was done at an attempt at performance
optimization, but that has little place in a test suite.  Use the full
recursive strict comparison instead.

This requires updating one ParserCache test case which was apparently
created incorrectly (or not properly updated) and relied on the
previous behavior (numbers comparing as equal to strings) to pass.

Change-Id: Ife0e9ccc7be0f4933975bb326203693bc15a9658
2022-03-31 09:50:51 -04:00
Alexander Vorwerk
d66fe8b7c7 Bump minimum required version for upgrade to 1.31
The policy allows this and since 1.39 is going to be the next LTS
release, I think it is fine to do this now.

Change-Id: If426e0ee349252ccc0ba9c4222c7d6865ab57fa2
2022-03-15 22:12:59 +01:00
Alexander Vorwerk
71a092e513 Add 1.38 snapshot to sqlite db tests
Change-Id: I72e703473679dbe982c8d851a6c357b17de415eb
2022-03-15 17:05:42 +01:00
mainframe98
de0c4819d1 Add validation for abstract schema
This adds an option to the schema generating maintenance scripts to
validate abstract schemas and schema changes and a structure test to
validate exisiting schemas and schema changes. Schemas are also
validated when generating.

The validation for the schema doesn't impose limits on table, index or
column names as I couldn't find any reliable conventions for them.

The structure tests only cover MediaWiki itself as there is no
convention on where extensions store their abstract schema.
Ideally, auto detection would be possible for sql/, but for now
extensions have to define their own (thankfully trivial) tests.

A couple of invalid definitions were fixed thanks to these tests.

I aimed to be thorough, but not all parts of the abstract schema
are completely clear, and Doctrine's documentation is not complete.
As a result, not everything has a description field.

Bug: T298320
Change-Id: I681d265317d4d1584869142ebb23d4098c06885f
2022-02-22 17:41:08 +00:00
jenkins-bot
99dee6855a Merge "Change return value of ParserOutput::getPageProperty() when property is missing" 2022-02-19 00:49:48 +00:00
C. Scott Ananian
c39ef6c6c9 Change return value of ParserOutput::getPageProperty() when property is missing
The old ParserOutput::getProperty() method returned `false` when a property
was missing.  This requires callers to use the `?:` syntax to supply default
values, which then causes any falsey value to be treated as missing.
So, for example, setting the defaultsort to '0' will cause the default
sort to be ignored.

Modern php convention is to use `null` for missing values, and the `??`
syntax is a better/more restrictive alternative to `?:`.

We renamed `ParserOutput::getProperty()` to `::getPageProperty()` in
1.38 (Ie963eea5aa0f0e984ced7c4dfa0fd65d57313cfa/T287216) but kept the
return value convention.  Before this actually makes it into a 1.38
release, take the opportunity to fix the return value for the new
`ParserOutput::getPageProperty()` method to return `null` when the
property is missing.

We need to do some temporary workarounds to the places we'd
already swapped over to use the new `::getPageProperty()` method
to allow them to handle either `false` or `null` as a return value;
we'll clean that up once this is merged.

Code search:
https://codesearch.wmcloud.org/deployed/?q=-%3EgetPageProperty%5C%28|T301915&i=nope&files=&excludeFiles=&repos=

Bug: T301915
Depends-On: I3f11ce604970e47b41fc1c123792df8c3045626f
Depends-On: Ie7533f49fe4cad01ebfda29760d23c61e9867b10
Depends-On: Ic5c09f5caa4c897bc553c614fbae9cee159566a2
Depends-On: I0278b2eafd90e77e4fee41c45a1165fb79ddf47e
Depends-On: I383abb6b7dc5e96c0061af13957609f6e31a1065
Depends-On: I79f9f4078e415284af29b15047bafd1c823d7f5b
Depends-On: I02276c48c49f5d2d241a69eb0a6cdf439b572d8b
Depends-On: I71628661b4539a4e35ae32846e719f92bcf782e0
Depends-On: I7e215cb43de0ce150a6bcc00f92481dcdcfed383
Change-Id: Iaa25c390118d2db2b6578cdd558f2defd5351d15
2022-02-18 21:15:58 +00:00
C. Scott Ananian
3c211fdb3c Update ParserCache serialization test cases to use valid category keys
Category keys are supposed to be non-null strings.

The test cases use bogus integer values, which causes issues when
refactoring more strictly enforces validity checks on category sort
key values.

Change-Id: If2937a694ba6bd4c522336f33aa58d40949b5a54
2022-02-17 12:12:53 -05:00
C. Scott Ananian
b46dfcc351 Update ParserCache serialization test cases to use a valid index policy
Valid values for ParserCache::$mIndexPolicy are '', 'noindex', and 'index'.

The test cases use the bogus value 'policy1', which causes issues when
refactoring more strictly enforces validity checks on index policy
values.

Change-Id: I2d00ff4e3ded043d18942c8482a39fac14ec60bc
2022-02-09 12:47:27 -05:00
Ed Sanders
9e234e6a37 eslint: Only apply client rules to client files
Change-Id: I757b9616ce69b6020e9f0e556e006d39de59eb01
2022-02-09 02:40:11 +00:00
jenkins-bot
7061fd2ee2 Merge "Reapply "Move limit report rendering to ParserOutput"" 2021-11-11 00:19:44 +00:00
Ed Sanders
9fe2995989 build: Update eslint-config-wikimedia to 0.21.0
* Replace `substr`/`substring` with `slice`. The second argument
  to `substring` is length, not end index as in `substr`/`slice`,
  so convert where necessary.
* Replace `new Date().getTime()` with `Date.now()`
* Documentation fixes
* Replace `throw Error` with `throw new Error`

Change-Id: I532500ea4c99d8ebec01efb21273c8df21626e59
2021-11-09 21:08:55 +00:00
Ppchelko
643fc535c3 Reapply "Move limit report rendering to ParserOutput"
This reverts commit 2bcb3fe567.

Reason for revert: this is a good change,
just needed more work to not break CI

Change-Id: I23768bee242e3cf81b1493a740cf070e7ad1e224
2021-11-09 11:08:08 -08:00
Fomafix
4f71fcdfcc Remove the executable flag from test files
chmod a-x tests/phpunit/data/MSCompoundFileReader/*

Change-Id: Ic2fa5f118b40126bea429d2299563c03fe6b3d1a
2021-10-16 11:23:02 +00:00
Petr Pchelko
a1aa3e0827 Hard-deprecate all public property access on CacheTime and ParserOutput.
- Added a test where ParserOutput objects with CacheTime
properties set are unserialized from previous versions.
- Generate new serialization tests for 1.38

Now all serialization in production is JSON, so changing
property visibility shouldn't affect ParserCache.

Bug: T263851
Depends-On: I283340ff559420ceee8f286ba3ef202c01206a23
Change-Id: I70d6feb1c995a0a0f763b21261141ae8ee6dc570
2021-10-13 13:27:16 -04:00
Amir Sarabadani
c3fff36540 Add 1.36 and 1.37 snapshots to sqlite db tests
To make sure upgrades from those versions work properly

Change-Id: I570cf965286bc5358a43c9601217b8781bb09b0e
2021-09-29 20:13:38 +02:00
Amir Sarabadani
753a4ecdd3 Bump minimum required version for upgrade to 1.29
Since the branch cut has happened, we can bump and get rid of legacy
cruft. According to the policy we can go up to 1.31 but let's keep it
that way to avoid major distruptions.

Change-Id: I9d697445a3bb5047726c8b2a7f808edb8403cdda
2021-09-28 17:57:57 +02:00
Fomafix
ff7f41f928 resourceloader: Append newline to JavaScript only if missing
In debug mode of a packageFiles module there is now only one newline
between the JavaScript code and the closing }.

This change reuses ResourceLoader::ensureNewline and make this to a
interal public static function.

Change-Id: I89519896e3dc56d966c4a63102904686bff6fac9
2021-08-28 12:25:11 +00:00
jenkins-bot
da9f37a177 Merge "resourceloader: Add integration test for ResourceModuleSkinStyles" 2021-08-19 01:32:26 +00:00
Timo Tijhof
76245d70ff resourceloader: Add integration test for ResourceModuleSkinStyles
This was covered in small ways already, but we did not have coverage
of the registration logic where the skin attribute is injected into
the module info, and this then correctly getting read and composed
into a single stylesheet.

This is prep for an optimisation in a subsequent commit.

Change-Id: Ia6fff44a345bb4f5811956638cada8b3e6b6ea71
2021-08-19 01:14:00 +00:00
Fomafix
6157bdaef0 CSSJanus: Update from version 1.3.0 to version 2.0.0
This version fixes a flipping bug.

Bug: T288943
Depends-On: Ida34372a4e5acb3147d83c7a1d008850c8382e32
Change-Id: I0528443e4eae0338f5a37491352db03944c6bd4c
2021-08-16 18:39:41 +00:00
libraryupgrader
5357695270 build: Updating dependencies
composer:
* mediawiki/mediawiki-codesniffer: 36.0.0 → 37.0.0
  The following sniffs now pass and were enabled:
  * Generic.ControlStructures.InlineControlStructure
  * MediaWiki.PHPUnit.AssertCount.NotUsed

npm:
* svgo: 2.3.0 → 2.3.1
  * https://npmjs.com/advisories/1754 (CVE-2021-33587)

Change-Id: I2a9bbee2fecbf7259876d335f565ece4b3622426
2021-07-22 03:36:05 +00:00
Tim Starling
2e507003ca Ignore invalid chunks in PNG files, instead of aborting metadata extraction
* Factor out file read errors and unexpected EOF errors.
* For errors relating to chunk content, instead of throwing an
  exception which is silently discarded, just log an error and continue
  to the next chunk. This allows the dimensions to be extracted even if
  other metadata is mangled.
* As an additional sanity check, verify the CRC of each chunk.

Bug: T286273
Change-Id: I11d0186496324e0bb1bb0a143f438e0368a8e902
2021-07-13 11:11:03 +10:00
Timo Tijhof
3371e4031c tests: Remove unused composer.json sample data
Follows-up c351c4f06e.

Change-Id: I842690849590f2ee57eb951099b9131728ecb7d7
2021-06-08 23:56:01 +00:00
MacFan4000
c351c4f06e Freenode -> Libera per wikimedia moving from freenode to libera
Bug: T283247
Change-Id: Iebbb13ac53830a8d77b42bf2954cf80d76d78467
2021-05-20 22:10:24 +00:00
Timo Tijhof
39e2abd886 registration: Allow multi-line strings in "@" note keys
Change-Id: Ifb1a0f04cc22b122ed99308c096d0ce4296e9097
2021-04-29 02:47:47 +01:00
Ammarpad
e497482ee4 DoctrineSchemaBuilder: Do not add prefix placeholder for Postgres at all
Adding it (only to be removed later) is not useful and interferes with
the internals of the schema builder because it masks the actual table
name during the table sql building and at some stages, this actual name
is required to be known

Bug: T191231
Change-Id: I57cc9b83c81f5c2d0d50c7b9e0dc334e7251c74f
2021-04-06 00:23:14 +00:00
jenkins-bot
e623a91aab Merge "Fix replacement of control chars in DJVU text output" 2021-03-29 15:38:33 +00:00
Inductiveload
32ea3a3fbe Fix replacement of control chars in DJVU text output
The control characters are presented as text, not actual
control characters, so the regexes to replace them are
incorrect.

Added a column and para to the Djvu text on the first page
of the test LoremIpsum.djvu file

Bug: T230415
Change-Id: I4970bc30b3935ce4da062ee7ff687aa667027a00
2021-03-29 06:03:34 +00:00
Derk-Jan Hartman
114d8e1cc5 JPEG2000 MIME fixes
- jp2 is the preferred file extension
- add mime aliases
- add jpg/jpx (jpeg2000 part 2)
- add magic byte detection for both

Bug: T161934
Change-Id: I00f81d9c31bf53a3b950e8a1d2bf2684c3998745
2021-03-15 21:43:24 +01:00
DannyS712
c0d41c57cd test data: remove old cssmin test files
No longer needed, now in the minify repo

Follow-up: I8d71fc64aeecdb31db218b02e361ae9bb0d19b48
Change-Id: Idb8175eb032a3521d08982da5be24041ab8758d3
2021-03-11 20:02:39 +00:00