In T208070 / I120ca25a77b7b933de4afddd1d458e36a95e26da we added a
check whether we were processing the last line of input, in order
to avoid emitting extra trailing newlines. But if the number of
input lines is large, StringUtils::explode() will return an
iterator which doesn't implement Countable for efficiency.
I22eebb70af1b19d7c25241fc78bfcced4470e78a fixed this, but at the
cost of scanning the string twice: once just to count the number
of newlines before we begin to iterate over the lines.
This patch uses Iterator::valid() to determine if we're on the
last iteration without having to scan the string twice.
Bug: T208070
Bug: T218817
Change-Id: I41a45266d266195aa6002d3854e018cacf052ca6
The previous fix for T218817 (I22eebb70af1b19d7c25241fc78bfcced4470e78a)
was a bit premature: we didn't notice that ExplodeIterator *also*
used a different Iterator::key() than ArrayIterator -- it used
the string position as a key, not the line number. Combined with
an inequality test for "not the last line" meant that almost every
line was now the "last line" and we were missing a lot of needed
newlines.
Count the lines ourselves to fix the problem.
Bug: T208070
Bug: T218817
Change-Id: I55a2c4c0ec304292162c51aa88b206fea0142392
StringUtils::explode() returns an ExplodeIterator if the number of
separators is too high, which doesn't implement count.
So count the way that explode does.
Bug: T218817
Change-Id: I22eebb70af1b19d7c25241fc78bfcced4470e78a
This now at least matches the function names even though what's actually
meant is more like 'block-level tag'. Section is a poor choice of name
since there're wikitext sections unrelated to this.
Change-Id: Ic83aff4d862800b778441c28884194480b7e7d96
HTML, generated by some infoboxes and perhaps other places, gets
stripped in a way that merges words together that should not be
merged. Add tr, th, and td to the list of tags that should force
word separation.
Bug: T218001
Change-Id: Ib374339628b1f543ea4e07f24aa3e3b76f3117b5
It's a temporary feature flag not included in any release, just
removing it outright. The functonality will now be always enabled.
Bug: T205040
Change-Id: Ia9da82e6f6b2d270f1790a99fc8c35ad5e6aee5e
The addModuleScripts() methods were deprecated in 1.31 and 1.32,
these are now removed.
The getModuleScripts() are now deprecated as well, always returning
an empty array. To be removed in 1.34.
Depends on commits for bundled/wmf-deployed extensions that
remove the last few remaining callers to the deprecated functions
in: 3D, Collection, Flow, GlobalUserPage, and Wikibase.
Bug: T188689
Depends-On: If9f0bc6aef85117587fa1929f34f8861c8d80314
Depends-On: Ia8d41b97fbf6822f5f8f7ac889408acce1ac9a3a
Depends-On: I503b919739ea474ff33726815b0da55e2f7e2724
Depends-On: I236ef637fd03b810a46eb361e25067a037e9d183
Depends-On: I62e17779753b977a452cc0c9694947941e999cc3
Change-Id: I5a19b8f164ccf666485d2971202194b747f882df
I would argue that these comments do not add any information that
would not be there already. Having them adds mental overhead, because
one needs to read both the comment and the next line of code first to
understand they say the exact same. I don't find this helpful, but
more distracting.
Change-Id: I39c98f25225947ebffdcc2fd8f0243e7a6c070d7
This might hint at an edge-case in the PHP CodeSniffer sniff that should
detect if methods are separated by a single empty line. Feel free to
investigate. I, personally, can't invest more time in this than
suggesting this quick fix.
Change-Id: Ib3c60eac76f255b4fe929f7933de256222716576
This was originally a global search and replace. I manually checked all
replacements and reverted them if (due to the lack of type hints) either
null (that would be 0 when counted) or a Countable object can end in the
variable or property in question.
Now this patch only touches places where I'm sure nothing can break.
For the sanity of the honorable reviewers this patch is exclusively touching
negated counts. You should not find a single `!== []` in this patch, that
would be a mistake.
Change-Id: I5eafd4d8fccdb53a668be8e6f25a566f9c3a0a95
HTML doesn't allow certain semicolon-less HTML entities in attribute
values to avoid breaking legacy markup like:
<a href="http://example.com?foo¶m=bar">...</a>
(Note that the & in that URL is not properly entity-escaped as `&`.)
Unlike wikitext, HTML generally allows semicolon-less legacy entities
in text.
Our alt and link option processing shove text through
Sanitizer::stripAllTags, which does entity decoding including these
legacy semicolon-less entities. Wikitext doesn't allow semicolon-less
entities, so escape & characters where appropriate to protect alt/link
options and avoid breaking URLs.
This was a "regression" in how alt options were handled starting in
ddb4913f53 when we switched to using
Remex for Sanitizer::stripAllTags -- semicolon-less entities (previously
invalid in wikitext) were now being decoded when stripAllTags was
called on alt text. This change became a problem when
ad80f0bca2 sent link option text through
Sanitizer::stripAllTags (with the new semicolon-less entity decode)
instead of PHP's strip_tags (which, in addition to its other faults,
doesn't do entity decode at all). This suddenly started decoding
"non-wikitext" entities like `¶` inside URLs, breaking links.
Filed T210437 as a follow-up to consider changing the behavior
of Sanitizer::stripAllTags() globally to prevent it from decoding
semicolon-less entities for all callers.
Bug: T209236
Change-Id: I5925e110e335d83eafa9de935c4e06806322f4a9
Remex is pure PHP so there is no reason to use an external tidy any
more. Configuration variables and implementation classes were
deprecated in 1.32 or earlier. We've kept only $wgTidyConfig
which can be used for experimental features or debugging Remex.
Bug: T198214
Change-Id: I99d48f858d97b6e1d1e6cd76a42c960cc2c61f9f
This adds a method to LinkFilter to build the query conditions necessary
to properly use it, and adjusts code to use it.
This also takes the opportunity to clean up the calculation of el_index:
IPs are handled more sensibly and IDNs are canonicalized.
Also weird edge cases for invalid hosts like "http://.example.com" and
corresponding searches like "http://*..example.com" are now handled more
regularly instead of being treated as if the extra dot were omitted,
while explicit specification of the DNS root like "http://example.com./"
is canonicalized to the usual implicit specification.
Note that this patch will break link searches for links where the host
is an IP or IDN until refreshExternallinksIndex.php is run.
Bug: T59176
Bug: T130482
Change-Id: I84d224ef23de22dfe179009ec3a11fd0e4b5f56d
Future parsers will not support the output generated with tidy disabled.
Parser tests using untidied output will also be deprecated (and
rewritten) in a follow-up patch.
No new release notes necessary since user-visible tidy configuration
was deprecated previously (in 1.32), and individual methods which had
disabled tidy during execution were individually release-noted as they
were updated.
Bug: T198214
Depends-On: I0f417f75a49dfea873e9a2f44d81796a48b9f428
Depends-On: If5c619cdd3e7f786687cfc2ca166074d9197ca11
Change-Id: I592e0e0dfef7d929f05c60ffe4d60e09725b39cc
Previously, they were always displayed in defult language unless
forced explicitly in wikitext, e.g. [[File:Foo.svg|lang=ru]].
This change adds a feature flag that would enable always trying to
display in page language.
* If enabled, Parser will pass a new parameter - 'pagelang' - to
the media handler.
* SvgHandler uses page language when determining what language to
render the image in.
* 'pagelang' can always be overridden by 'lang'.
* If no translation in page language is available, the default
language (English) will be used for thumbnail URLs, to prevent
cluttering media storage and HTTP caches with useless copies.
Performance: this requires accessing image's metadata during parsing.
My testing indicates there were no code path where this wasn't the
case already, so no performance hit is expected, however we should
still keep an eye on page save performance.
Bug: T205040
Change-Id: I348840ef405e1370cc0c17d69051bce30153c9c0
bacd87e49 moved the displaytitle error message from the content to
outside of the content. Only the content is converted by the language
conversion. The error message outside of the content is not converted.
Therefor markNoConversion is not needed here anymore.
This change removes the -{R|...}- around the displaytitle in the error
message when the language converter is active.
Bug: T208249
Change-Id: Ieec43e9af045d19b0b7a82afb889e076b347eed1
We are deprecating the non-tidy modes of the parser.
ParserOptions::getCanonicalOverrides() has always set `tidy` to `true` at
any rate, so this isn't going to invalidate any parser cache entries.
Change-Id: Ib703a041edf8a8d57e94f136965f72d9bbfcf222
This was already deprecated in the release notes, and is not used in
production, but I'd overlooked adding an appropriate hard deprecation
notice in MWTidy::factory() to notify downstream users.
Change-Id: I8f4d8154a1d8a233017f54f0fb4bcfdf4a0373e1
