Commit graph

2264 commits

Author SHA1 Message Date
Arlo Breault
c44a3958a3 Don't apply French spacing in raw text elements
This also means we don't need to take special care for French spacing in
attributes, since it's no longer applied there.

Adds a test that captures this change.

Note that the test "Nowiki and french spacing" wonders whether this
escaping should be applied to nowiki content.

Bug: T255007
Change-Id: Ic8965e81882d7cf024bdced437f684064a30ac86
2021-02-16 19:26:29 -05:00
jenkins-bot
183d83b572 Merge "parser: new BeforeParserFetchTemplateRevisionRecord hook" 2021-02-16 04:31:16 +00:00
jenkins-bot
5ea464b234 Merge "Use static closures where safe to use" 2021-02-14 23:05:48 +00:00
Umherirrender
8de3b7d324 Use static closures where safe to use
This is micro-optimization of closure code to avoid binding the closure
to $this where it is not needed.

Created by I25a17fb22b6b669e817317a0f45051ae9c608208

Change-Id: I0ffc6200f6c6693d78a3151cb8cea7dce7c21653
2021-02-11 00:13:52 +00:00
Thiemo Kreuz
947a8cfc00 Use ?? instead of isset/array_key_exists where possible
This patch touches all uncontroversial (I hope) places where a chain
of isset(), array_key_exist() and the ternary ?: operator can be
replaced with the much shorter ?? feature from PHP 7.

?? does the same. It checks if the element before the ?? is set and
not null. When this check fails, the element after the ?? is used.

Change-Id: Id612e2782ae928164b26b6f0de676c6c7d8302f3
2021-02-10 20:32:25 +01:00
jenkins-bot
15ea8da052 Merge "Improve docs from parser objects PPNode/PPFrame" 2021-02-05 21:52:37 +00:00
C. Scott Ananian
0676256ccc parser: new BeforeParserFetchTemplateRevisionRecord hook
This new hook provides for the use case in T47096 (allowing the
Translate extension to transclude a page from another language) by
adding a new hook which would let us deprecate and replace two awkward
legacy hooks (one with an embarrassing capitalization issue).  The new
hook is a little more tightly scoped in terms of what it allows and
gives access to, and it uses the new RevisionRecord API.

In addition, the new hook uses LinkTarget instead of Title per
current best practices. (PageIdentity is not appropriate for
reasons documented at the hook invocation site.)

The original BeforeParserFetchTemplateAndtitle (sic) hook allowed
redirecting the revision id of a template inclusion, but not the
title. The only known current use is Extension:ApprovedRevs; the
FlaggedRevs extension replaces the entire function using
ParserOptions::setCurrentRevisionRecordCallback().

Extension:Translate would like to redirect the title as well, possibly
recursively (for a limited number of hops) to handle fallback
languages.  That is, when invoked on Foo/fr, including Template:Bar
would redirect to Template:Bar/fr -- and, if that doesn't exist, then
Template:Bar/fr would redirect to its fallback language, say
Template:Bar/en. It uses the top-level page title as context to set
the desired page language.  This would require 2 invocations of the
hook; we've set the recursion limit to 3 to provide a little bit
of future-proofing.

The hook added in this patch uses RevisionRecord instead of int
$rev_id, and thus can handle the case where the redirect is to a page
which doesn't exist (by setting the RevisionRecord to a
MutableRevisionRecord with the correct title and no main slot content)
in the fallback language case above.

The new hook deprecates BeforeParserFetchTemplateAndtitle and replaces
ParserFetchTemplate as well (deprecated in 1.35).  Code search:

https://codesearch.wmcloud.org/search/?q=BeforeParserFetchTemplateAndtitle&i=nope&files=&repos=

Bug: T47096
Change-Id: Ia5b5d339706ce4084c16948300e0e3418b11792e
2021-02-05 09:37:32 -05:00
jenkins-bot
0c5cd09cbf Merge "Fix CacheTime::getCacheExpiry for php8" 2021-02-04 22:39:10 +00:00
Ammarpad
af303cafe4 ParserOptions: Fix param/return type doc
onAccessCallback can be null, and setTidy() does not return boolean

Also use variable for MW services call.

Change-Id: If4e0c986ce47221d75888029bc32d788a9528681
2021-02-02 20:52:52 +00:00
Umherirrender
76126158cb Fix CacheTime::getCacheExpiry for php8
CacheTime::mCacheTime can contain the empty string,
which makes it fail on php8

PHP 8 changes the handling for string and int comparison.
These non-numeric strings are no longer silently converted to int 0,
which previously allowed for the comparison to become false without
actually checking against non-integer values.
https://wiki.php.net/rfc/saner-numeric-strings

Bug: T248925
Change-Id: Idecd3a5cd41fc2d54968db4b02787fe2a81af9d2
2021-02-01 20:05:54 +00:00
Umherirrender
39a4ddf689 Improve docs from parser objects PPNode/PPFrame
Change-Id: Ie1bd0fb52a16277482ce461a50b16550332b6768
2021-02-01 19:37:33 +00:00
jenkins-bot
f7b1bf1c72 Merge "Make ParserOptions::isSafeToCache more robust" 2021-01-28 19:03:57 +00:00
jenkins-bot
03e2d471c4 Merge "Rewrite <langconvert> to support BCP 47 tags" 2021-01-28 16:30:31 +00:00
Petr Pchelko
b964bd2e8d Make ParserOptions::isSafeToCache more robust
Bug: T273120
Change-Id: I516133d07d9ff4d7930de68b88ed4b40ca1c6150
2021-01-27 18:34:26 -06:00
jenkins-bot
d491f23b90 Merge "Respect used options for ParserOptions::isSafeToCache" 2021-01-25 19:13:53 +00:00
jenkins-bot
607c4d3910 Merge "CacheTime: remove mVersion field" 2021-01-25 19:12:05 +00:00
jenkins-bot
e845067ab6 Merge "Adopt pipe trick with Arabic comma" 2021-01-16 03:29:36 +00:00
David Kamholz
cdbd2e791d Rewrite <langconvert> to support BCP 47 tags
This validates langconvert's "from" and "to" arguments as valid BCP 47 tags. For example, it will accept "sr-Cyrl" and "sr-cyrl" and reject the non-standard internal MediaWiki code "sr-ec". I made the BCP 47 matching case insensitive as that seems to conform with how MediaWiki handles it elsewhere and case sensitive matching would probably be a headache for users.

Bug: T271758
Change-Id: I9f765fe650279820d61c3a7e499ca99468df3d14
2021-01-13 19:00:47 -08:00
Niklas Laxström
8d42b76017 Fix @deprecated for Parser::enableOOUI
Change-Id: I0d6937a82413f50a054ba0e3869efa3863a6632b
2021-01-13 15:12:50 +01:00
jenkins-bot
7200e36c9f Merge "parser: convert Preprocessor to WANCache and inject dependencies" 2021-01-12 05:04:47 +00:00
Aaron Schulz
de6eeead21 parser: convert Preprocessor to WANCache and inject dependencies
Make the caching logic use getWithSetCallback() and simplify
the code given that there is only one Preprocessor subclass.
Also, keep the cached values JSON serializable but rely on
the serialization in BagOStuff instead for simplicity.

Add related class constants for injecting preprocessor flags.

Bug: T254608
Change-Id: I72f9f0c0bc352ed5120469090c71294ff0c24999
2021-01-11 20:21:10 -08:00
Ebrahim Byagowi
9fe1d1f734 Adopt pipe trick with Arabic comma
Currently MediaWiki turns `[[test, abc]]` to `[[test, abc|test]]`
while saving the page but that comma isn't in use in Persian
so this patch makes MediaWiki to treat Arabic comma the same way
as regular comma.

Change-Id: Ib8051023abc25b7c4f97a3f50246f35650057ec9
2021-01-11 21:43:33 +00:00
jenkins-bot
a3a3e9f3af Merge "CoreTagHooks: First argument passed to parser tags can be null" 2021-01-07 17:29:28 +00:00
Petr Pchelko
7e8d1a11c8 Return back accidentally removed ParserCache 'hit' metric
Change-Id: Ibd69e532a2f373f9d0129ac2a2c6ac70039c9bec
2021-01-05 14:44:19 -06:00
C. Scott Ananian
a41f284324 CoreTagHooks: First argument passed to parser tags can be null
Document and enforce the correct type for the first argument to
a Parser tag hook, which will be `null` if the tag is self-closed.

Mark the methods in CoreTagHooks @internal.  They are apparently
unused outside MediaWiki core:
  https://codesearch.wmcloud.org/search/?q=CoreTagHooks&i=nope&files=&repos=

Add coverage test cases to ensure that all tag hooks properly handle
the `null` value of the first argument; prior to this patch the
`<html>` tag emitted a broken strip tag in this case.  The other hooks
passed the null to other callees in violation of their type
signatures, but eventually every other hook managed to safely cast the
null to the empty string without throwing an exception or emitting a
warning.  For those, this patch does not change existing behavior---it
just makes the cast to the empty string much more obvious to the
reader.

Change-Id: I69fde6c06eabb2db27bb1cc23d2cb19b99273391
2021-01-05 14:19:44 -05:00
Petr Pchelko
46b66f093a Respect used options for ParserOptions::isSafeToCache
Bug: T269293
Change-Id: Ic3cf908265ad470815f0ac81442d33bde04a5665
2021-01-04 10:32:34 -06:00
Petr Pchelko
2010a912c2 CacheTime: remove mVersion field
Has to be merged at least one train after the parent patch.

Bug: T269393
Change-Id: Iee98950ad0452a50088e755252da8468f0bec1ae
2021-01-04 10:29:52 -06:00
Umherirrender
a30fe542ae build: Enable SecurityCheck-DoubleEscaped and suppress issues
This issue type was globally suppressed in
I849ac4f120fd15b483e8939d4db45c98dc351259 to make reviewer easier.

This adds inline suppressions or @suppress directives on function
docs for false positives, mostly restoring those removed in
I849ac4f120fd15b483e8939d4db45c98dc351259

Bug: T231311
Change-Id: I1b1d814bd907e9d49fcc39f777982936574fc7c6
2020-12-30 23:34:20 +00:00
Umherirrender
e9e784a09e build: Enable phan-taint-check-plugin and suppress issues
Taint check checks for possible security issues by tracking html
escaping and more by using phan.
This slows done the phan-job a bit and requires more ram

Keep the DoubleEscaped issues out to make reviewer easier

Adds suppression for false positives
Adds taint-annotation to help taint-check
Removes suppression for code phan now understand better by the tracking
of keys in taint-check
Fix some small issues by adding int cast or htmlspecialchars calls

Bug: T216348
Bug: T268920
Change-Id: I849ac4f120fd15b483e8939d4db45c98dc351259
2020-12-30 19:02:22 +01:00
jenkins-bot
4257367a01 Merge "Record all used options in metadata." 2020-12-16 17:59:54 +00:00
jenkins-bot
add80561d2 Merge "ParserOptions: Deprecate fallback to $wgUser" 2020-12-16 15:17:05 +00:00
Petr Pchelko
71bb51ed55 ParserCache: general code cleanup, abstracted expiration checks.
Change-Id: I7374f30d582064236b8f782e6a2528eb692e3010
2020-12-16 12:09:55 +00:00
DannyS712
25db391d53 ParserOptions: Deprecate fallback to $wgUser
Emit deprecation notices in the constructor
if falling back to $wgUser, and fix core
calls

Bug: T246861
Depends-On: I51117931d527a3bdda468b48de577a7faafbcd69
Change-Id: Ibd0a8ffd0494c17a378cc43e6b6164166130adf4
2020-12-16 09:03:29 +00:00
jenkins-bot
f0d9e5e182 Merge "Implement caching for old revision HTML endpoint" 2020-12-16 00:41:27 +00:00
daniel
637f630fe9 Implement caching for old revision HTML endpoint
Bug: T269663
Change-Id: I2d17ec37d25f3a6e1c4836c05576bf0fabb7d429
2020-12-15 23:40:08 +01:00
jenkins-bot
b3a8ea7076 Merge "Remove 'stubthreshold' from ParserCache key." 2020-12-15 20:21:14 +00:00
jenkins-bot
ea12ffec23 Merge "Remove ParserCacheFactory::getInstance" 2020-12-15 19:25:04 +00:00
jenkins-bot
c8edeeeaed Merge "CacheTime: Remove the version check." 2020-12-15 19:00:26 +00:00
Petr Pchelko
5413678f06 Record all used options in metadata.
Bug: T269293
Change-Id: I69ac24fcf8bf81de1d25c5a331b2155d9fe21977
2020-12-15 12:51:14 -06:00
jenkins-bot
a10ba8d59e Merge "Implement <langconvert> tag" 2020-12-15 06:18:11 +00:00
David Kamholz
a7ad0547bc Implement <langconvert> tag
The <langconvert> tag takes two attributes: from (language variant from) and to (language variant to). It returns the content of the tag converted using LanguageConverter. It returns an error if the attributes are not present, if the variants do not exist, or if the variants belong to different languages. Currently it does not work for IuConverter, because the variants use the code ike rather than iu, and ike isn't in the list of languages with converters available.

This patchset reimplements from a parser function to a tag, and renames from transliterate to langconvert.

Bug: T263082
Change-Id: Idc3a32c66d5a0466c63e7ce8753d2619354c30b0
2020-12-14 19:40:31 -08:00
Petr Pchelko
b56a3c0d23 CacheTime: Remove the version check.
Fot the sake for forward compatibility, first we need
to remove the check for version in case of a train rollback.

Bug: T269393
Change-Id: Ic7011e20a23e8ad99eb872b050e97baf1ef5c6a2
2020-12-14 19:29:24 -06:00
Petr Pchelko
5a995333fb Remove ParserCacheFactory::getInstance
Added in 1.36, no deprecation process needed.

Bug: T269869
Depends-On: I5d81622d5857bc1d06a3600d8e9554882532714a
Change-Id: I182cb0cb7e4c569a64e2455ca80eda2d1f570608
2020-12-14 19:12:04 -06:00
jenkins-bot
f3e1a5aa22 Merge "api: Use GetLinkColours hook to export CSS classes in page info" 2020-12-15 00:27:48 +00:00
C. Scott Ananian
f9433de05e api: Use GetLinkColours hook to export CSS classes in page info
Several important extensions (Disambiguator, ProofreadPage, and
SoftRedirector) use the GetLinkColours hook to add additional CSS
classes to links on article pages. Parsoid previously relied on
backdoor knowledge of the way Disambiguator used the page property
table to support these, but they should be exported properly from the
API.

Bug: T237538
Change-Id: I945940aa872541d7e01f1e543ca854231c857fe2
2020-12-14 17:51:50 -05:00
daniel
00a3439dce Introduce RevisionOutputCache
Bug: T267981
Change-Id: Ib1dc641ed10d786918362b25bd655780d5844ba1
2020-12-14 16:50:28 +00:00
Umherirrender
f46ca9a63c build: Updating mediawiki/mediawiki-phan-config to 0.10.5
Change-Id: I343d2bae626a3903eb1e67c05bf5caef4314b7dd
2020-12-12 14:42:25 +01:00
Petr Pchelko
235c56d649 Remove 'stubthreshold' from ParserCache key.
Stubthreshold option used to be a cache-varying option,
but in all places where we interact with the ParserCache
we are checking that it's 0 before using the cache.

Instead, we can just remove all the special cases for
stubthreshold option, remove it from cache key and rely
on ParserOptions::isSafeToCache to avoid caching non-default
stubthreshold outputs.

Bug: T264351
Change-Id: Ifaf69a3e651eef21c88da3aa3044b490059958ca
2020-12-07 14:47:05 -06:00
Petr Pchelko
66cc685b45 Make ParserCache use CachedBagOStuff
Bug: T269593
Change-Id: I21e6e39eccad22b781252b142c1e5b079c1ee0b4
2020-12-07 10:28:30 -06:00
Daniel Kinzler
48172f794d Revert "Hard-deprecate all public property access on CacheTime and ParserOutput."
This reverts commit b1a30eb0c4.

Reason for revert: T269396

Bug: T269396
Change-Id: I374ca13ccc30418b8fe3bf98f5090f7643aac4d7
2020-12-04 11:47:55 +00:00