This reverts commit 82da9cf14b.
Passing through Remex seems to have unexpected consequences to be
investigated but, for the sake of unbreaking the UBN, let's revert this
first.
Bug: T353920
Change-Id: Iaac7942aa77aee5ab525852ac5b41dd516ff13c9
The previous implementation was using an ad-hoc regular expression which
was matching inside the data-mw attribute of Parsoid output, eg:
<sup about="#mwt42" [...] typeof="mw:Extension/ref mw:Error" data-mw="{"name":"ref","attrs":{"name":"infobox_stats_ref_rail"},"body":{"html":"<style data-mw-deduplicate=\"TemplateStyles:r1133582631\" typeof=\"...">
After substitution, the <link> element inserted contained " instead of
" and so broke out of the attribute.
Instead use a proper HTML tokenizer (via wikimedia/remex-html) so that
we don't allow bogus matches inside attribute values.
To fix up tests:
* Don't deduplicate styles when parsing UX messages (also helps performance)
* Don't deduplicate styles in ContentHandler integration tests
* Don't deduplicate styles by default in parser tests
(unless explicit option is set)
Depends-On: Id9801a9ff540bd818a32bc6fa35c48a9cff12d3a
Depends-On: I5111f1fdb7140948b82113adbc774af286174ab3
Followup-To: Ic0b17e361bf6eb0e71c498abc17f5f67f82318f8
Change-Id: I32d3d1772243c3819e1e1486351d16871b6e21c4
This replaces the previous `variant` name for this option to help avoid
confusion: Parsoid maintains both an 'html variant' and a 'wikitext variant'
and this helps us ensure we're all talking about the same thing.
Change-Id: I01b5eef72e8b4433510bc8cb9b684b1e37715821
It is difficult to distinguish this method from OutputPage::addJsConfigVars()
in code search:
https://codesearch.wmcloud.org/deployed/?q=%5BOo%5Dut%28put%29%3F%28%5C%28%5C%29%29%3F-%3EgetCategories%5C%28&files=&excludeFiles=&repos=
We generally try to replace $output with $parserOutput or $pOutput
as we touch code to improve the ability of codesearch to dig up
deprecated ParserOutput methods.
Bug: T305161
Depends-On: I02dd4f61c43c225b0ef6dc51c3e4f9d967a0a272
Depends-On: I61d2d77591579d825ad9d37f902e40366be55dd6
Depends-On: I91155106b7a9e10d3334f95ba4936d02851bfb11
Depends-On: Iaca745c79d9587571af03b23b21d76a6cba0ebf1
Depends-On: Id10a171c44411b1233ee4d6cf8fbd3dc57744eef
Depends-On: I47a25c011d9bd4b1a15dda4e673e32c25eb64f2b
Depends-On: I683fc768aba50b801f46467fcfa1668fa8731ea6
Change-Id: I5a2ac1c99b8b199102e12f0d32dd6ec5cdc24054
The parser test reader (imported from Parsoid) already converts
comma-separated lists to an array; we get an exception from explode()
if we then try to split that array by comma.
Change-Id: Id17ed22d81cc880f41f584406369323232c00bdf
The method should never be called directly, so make it throw an exception.
Nonetheless, mark it as deprecated and detect overrides in the
constructor, so that anyone who tries to override this method will see a
warning.
Fix the few tests that were relying on the existence of the test page.
Bug: T342428
Depends-On: Ic64ded5e2c0b59e7c888ece9566076058a125be4
Change-Id: I308617427309815062d54c14f3438cab31b08a73
Was introduced in I6c1e9bfad9790cf805809c28a3f8d45952cbb981 and
later rendered unused in Ie62250242965d3d90873909795ced2cbda506ddb.
Since we want to make sure we're using the right instance passed in
closures, let's keep getting it from the global service container
then keep locally in a variable.
Change-Id: I834b09933efc8835c73b9fafaad80cc7041757b6
The interaction between the title cache, the link cache and the parser
tests is very strange. With different parser tests and different
extension enabled it can fail and seems not very deterministic
Follow-Up: Ie4b67106512fb1a3a1b595dc4f6036276db96378
Change-Id: I55501eea7de739cac044b22caec150089183620b
The instance in the title cache does not see the reset of the id (lazy
loaded Title::mArticleId). When a title instance with Title::newFromText
is used from cache the instance can assume the title does not exists,
while the page was created. Remove the cached instance to get a fresh
instance which can do a fresh db lookup for the id when needed.
This should remove the issues from the parser test
with wrong title states
Follow-Up: Id056580c7b869ae4984de5e2c89fb4687eecf7bd
Follow-Up: Ibc8e0ddbe9e53c3334b9c26ec2d1eda976c2a62b
Change-Id: Ie4b67106512fb1a3a1b595dc4f6036276db96378
Avoid clone of Parser object, create a new one
Reset the UrlUtils services after changing the server setting,
it is a dependency of the ParserFactory and gets initalizied earlier now
Bug: T250448
Change-Id: Ie62250242965d3d90873909795ced2cbda506ddb
In the parser tests for LabelledSectionTransclusion a page transcludes
another page. On creation of the first page the second page does not
exists. The parser parsed the first page and after that the title cache
contains a title object for the second page with an article id of 0,
after the creation of the second page the article id in the title object
needs to be reset as it gets reused in the test.
Before 880fc5da the title object was in the title cache on page
creation, as the addArticle function was using Title::newFromText, which
results in the correct reset of the id in the cached object.
Bug: T342875
Change-Id: Id056580c7b869ae4984de5e2c89fb4687eecf7bd
Parsoid CI broke with b42062e7. It looks like the title cache clear
is still needed.
For anyone interested in digging deeper into this, you can revert
this patch and run the command below to see a failing test.
* composer phpunit:entrypoint -- --testsuite parsertests --filter 'multiple templates'
Change-Id: Ibc8e0ddbe9e53c3334b9c26ec2d1eda976c2a62b
Promote the deprecation to an error in the context of PHPUnit tests. The
point of hard deprecations is to make tests fail and this will help with
that, and also with eventually promoting the deprecation to an error
outside of tests.
Adjust code in parser tests that was accessing MediaWikiServices via
Title too early.
Avoid hack of resetting the error handler after loading Setup.php, and
conditionally install MW's hadler instead. This is particularly
important in scenarios where an exception is thrown before the handler
is reset, because MW's exception handler may also access
MediaWikiServices.
Bug: T227900
Bug: T273261
Change-Id: I7c5234046379cf4abd25d65e78c0a99ac9f32600
This reapplied commit b4e797510c with
some fixes for the parsoid tests, which can be checked by diff the
patchset 1 and the latest patchset.
- Factored out resetting of services that only related to language
and variant into a function, sorted all services roughly by their
dependency relationship.
Also, only reset when the language or variant is configured by the
test case.
- Replaced manual redefine of ContentLanguage service with reset
method like others.
- Reset necessary services after setting the default language code in
staticSetup(), so the override in addArticles() can be removed.
- Use the 'skin' param of getText() for setting skin, so we don't need
to touch the context.
- Removed the override of wgUser, wgLang, and wgOut. They didn't live
in parser-related codes anymore.
- Setting user option is not the correct way here, the problem is
UserOption(Lookup|Manager) didn't get reset, so the default user
language is wrong; and the context have a cached Language object.
When the default language option updated, purge the cache with
setUser() should be sufficient.
Change-Id: I4ebeaef98ed9b7682701c0385c68145ee1e78951
This reverts commit b4e797510c.
That commit breaks Parsoid CI, as demonstrated by tests on empty
commits Iad5e05eda4b94ce9f5708c84526c59e25cafa7a0 (passing,
depending on this patch) and I8a85aa11a29bcc568dd8079bce01320b087e04ac
(failing, not depending on this patch).
Change-Id: Ifaf295d45c00783a37e056e01bee98567d4a7cf5
- Factored out resetting of services that only related to language
and variant into a function, sorted all services roughly by their
dependency relationship.
Also, only reset when the language or variant is configured by the
test case.
- Replaced manual redefine of ContentLanguage service with reset
method like others.
- Reset necessary services after setting the default language code in
staticSetup(), so the override in addArticles() can be removed.
- Use the 'skin' param of getText() for setting skin, so we don't need
to touch the context.
- Removed the override of wgUser, wgLang, and wgOut. They didn't live
in parser-related codes anymore.
- Setting user option is not the correct way here, the problem is
UserOption(Lookup|Manager) didn't get reset, so the default user
language is wrong; and the context have a cached Language object.
When the default language option updated, purge the cache with
setUser() should be sufficient.
Change-Id: I94103b86a02d6b971f70a0bb7ece1f22cd16e715
The Hooks class contains deprecated functions and the whole class is
going to get removed, so remove the convenience function and inline the
code.
Bug: T335536
Change-Id: I8ef3468a64a0199996f26ef293543fcacdf2797f
Note that the metadata isn't even checked unless the wt2html is passing
it may take several runs to get all the tests updated.
Follows-Up: Ieaca9152b9f0d0a853c0dfaff1bdca808110539e
Change-Id: I10f5b54a8ebffaf10111d57aa66e1220c5418ca7
Follow up to I854f89bd823aab297efe29cd4fdee675afd77752
Returns the behaviour to what it was before that patch.
Change-Id: I743fa1118c4c78863f3857f4dc70d82f6bf4f0ac
Clean up to bypass skipped tests early in both legacy and
parsoid test runs without duplicating the skipped test check.
Also got rid of two FIXMEs with this refactoring.
Change-Id: I854f89bd823aab297efe29cd4fdee675afd77752
This is now enabled in production (Ic5a4a9950d51f63b17f4c5e70516bec87b981aa5)
and not something we want to remain configurable.
It is removed from Parsoid in I52ddfd21ff2e72a34cb5eb68742e3dfb85c6ccf6
Change-Id: I6a4d7d33fb42270fc5da3a922aa0a959180fb33f
The TOC used to be language-converted in ParserOutput::getText(), but
it wasn't possible to apply custom rules defined in the wikitext
article body at ::getText() time. Remove the various hacks that we'd
added in an attempt to do so, which were made unnecessary by
I321cd31dae64bbf845d53282e5d28a55bc4ec319.
Bug: T306862
Change-Id: Ib12cd02e9ade91d5794462e8833f2aa3b45a51f2
* ParserTestRunner: LocalisationCache needs to be reset since it has a
reference to LanguageNameUtils which has a copy of
$wgUsePigLatinVariant. Also factor out some
MediaWikiServices::getInstance() calls.
* In some other tests, set the variable.
Change-Id: I6c1e9bfad9790cf805809c28a3f8d45952cbb981
Two bugs here: first, we were silently skipping a needed file update in
::updateKnownFailures() if the file didn't previously exist. We're going
to still avoid writing the file if it didn't previously exist, out of an
abundance of caution, but at least we'll now fail noisily so the problem
can be fixed.
Second, the `--parsoid` flag was overriding the result of
::getFileSkipMessage() so we were processing files for
`--updateKnownFailures` that should have been skipped because they
are not marked parsoid-compatible. This override made sense when
we were still debugging the integrated-mode parsoid support in the
ParserTestRunner, but it is not needed anymore.
Change-Id: Iba961ea327e54bb6bdc87399dbcba87cd57b6b20
This adds support for various options which add metadata
information to the parser test output, including 'showtocdata'.
This builds on I845694d4f2109a8b9125410e8533ca69bbea50fa in treating
the metadata output as a separate section.
Bug: T270312
Depends-On: I8023931d31e494df325b16d1b922539e20b58c51
Change-Id: I0c42ec2dc93c358f1cddab77324b229bcc163e83
This provides a bit of isolation from the actual layout and names
of properties in the object, as well as being a touch more readable
when debugging test failures.
Change-Id: I5ddca850f577b2ac24e237a2518f03983e79a51d
If a ParserTest mixes HTML output and metadata properties, it can
complicate HTML normalization and other test processes, especially
for Parsoid-mode bidirectional tests.
Support splitting metadata output into a separate section, named
`!! metadata`, with the standard options for legacy and parsoid
variants, like `!! metadata/php` and `!! metadata/parsoid` and
`!! metadata/parsoid+integrated` etc.
For compatibility, if the metadata flags are present on the test
and the new section is not present, we'll continue to handle the
metadata output as we have before, aka append or prepend the metadata
to the HTML.
Code search for uses of these options (uses in parsoid and core can
be ignored; uses of 'pst' are harmless when they are not combined
with another option):
https://codesearch.wmcloud.org/search/?q=%28%5E%7C%20%29%28%28showtitle%7Cshowindicators%7Cill%7Ccat%7Cpst%7Cshowflags%29%28%20%7C%24%29%7C%28extension%3D%7Cproperty%3D%29%29&i=nope&files=%5Etests%2Fparser%2F.*%5C.txt&excludeFiles=&repos=
Change-Id: I845694d4f2109a8b9125410e8533ca69bbea50fa
