* Unclear if this is actually useful or helpful, but at least the
legacy and Parsoid test modes do all setup close together now.
* See I776b22a973fbcc945106fbe807376b858dd79e48 for a more detailed
explanation. This patch is a followup from that patch.
Change-Id: I720d31c3573b57b66956bdd98d11e4f89df9cca0
* There are two places in the parser test runner where test options
are processed: perTestSetup, and setupParserOptions. There isn't
enough guidance as to what should be set up where and why.
But, anything that is really a wiki config/global should ideally
be set up in perTestSetup because the ParserOptions object loads
all its defaults from the config values anyway.
* Parsoid gets wiki-specific options from SiteConfig and page-specific
options from PageConfig.
In 61c14054, "externallinktarget" was treated as a wiki-specific opt
by adding it to SiteConfig. So, if Parsoid is to see a test opt,
(a) the corresponding config/global should be initialized
(b) it should be processed before SiteConfig is constructed
* So, all that said, any test option that impacts Parsoid's SiteConfig
should therefore be set in perTestSetup as well whch also matches
my understanding of how it should be set up for legacy tests as well.
* This fix now ensures that external link target tests pass in Parsoid's
integrated mode as well.
* Based on above, couple other things that I'll fix up in followups:
- There are a couple other configs (maxtemplatedepth, maxincludesize)
that are being initialized in setupParserOptions which should
probably be moved to perTestSetup.
- runLegacyTest calls perTestSetup and setupParserOptions back-to-back
but runParsoidTest doesn't do that. Those are separated by code
that sets up SiteConfig and DataAccess. This shouldn't matter if the
above guidelines for initialization are followed appropriately,
but just for consistency's sake, I might try to move those two
closer together in another followup.
Followup-To: f7d6c0190d
Change-Id: I776b22a973fbcc945106fbe807376b858dd79e48
Eliminate a difference between the magic variable (no-arg) and
parser function (1-arg) forms; aka the difference between
{REVISIONTIMESTAMP}} and {{REVISIONTIMESTAMP:{{PAGENAME}}}}.
This is a follow up to I8d25755e4d92bd91988cfb706d85bdb170abb207.
The magic variable contains a MAX_TTS optimisation which reduces
the use of vary-revision-timestamp, since it has severe performance
implications; this patch applies the same optimisation to the
parser function.
The ParserTestRunner has a small issue with test setup:
ParserOptions::setTimestamp() was called with a unix-format timestamp,
where it expected a TS_MW format timestamp. This issue was
fixed, along with tweaking the test timestamps so that a timestamp
coming from ParserOptions would still be distinguishable from one
coming from the revision.
Bug: T204370
Change-Id: I883d42d67013b6fb0da57c61e715b51d3a807879
As a follow-up to Ife14d7319ecbb7b588cbac97fae109b14230c44a, we want to
check that the legacy parser and Parsoid align on what's expected for
nofollow/noreferrer/noopener. Since the wgExternalLinkTarget setting is
modified on the Parsoid side during specific tests, we align the test
runner on the legacy parser to be able to test for this.
Bug: T186241
Change-Id: I2824182eaa5c50cd3e8e0193726d896da285a3a8
* This ensures that titles don't conflict across test runs.
I am surprised this hasn't bitten any one in non parsoid
parser test runs in case titles are repeated across test files.
* I wonder if this was also the source of some of the transient
errors seen in T312709 and T312517.
* Scribunto calls fetchModuleFromParser in two places and passes
it a title. In one case, the title comes from Title::newFromText
(in the call in LuaEngine.php when asked to load a module from
within Lua code). In another case, the title comes from
Title::makeTitleSafe (in Hooks.php to run #invoke on a module).
Title::newFromText uses a title cache whereas Title::makeTitleSafe
constructs a new title.
And, it turned out that in Parsoid runs, the revision lookup
for the module had the wrong revid when it came from Lua code.
This then caused the package lookup to fail only when the module
was loaded in Lua code.
That this test only failed in Parsoid test runs is because we
ran Parsoid tests after legacy parser tests. If we had switched
the order, legacy parser tests would have failed.
I haven't connected all the dots from the cached title in Title.php
to it getting an incorrect revid (but not an incorrect pageid).
But, clearing caches in the parser test runner seems to fix
this problem.
I welcome someone else helping with fleshing out this trail as to
how the title caching leads to only the revid going out of sync.
* Since I could only reproduce this in Jenkins CI (and not locally),
debugging this turned out to be painful.
Bug: T316331
Change-Id: I7c419e1d508c337328db1d5132454fb805bfb1b5
* This should hopefully make the code a bit dryer and easier to read.
* While this should not matter in practice, when testing against Parsoid,
don't create two different mutable revision records for tests where
'lastsavedrevision' option is set.
Change-Id: I5da7ec9ec688df0a1f84f7c1817775fd94a1e092
This reverts commit 0442728ed8, as the issue was fixed upstream in Parsoid. Further, this "fix" inadvertently disabled all but "selser" test modes for Parsoid parser tests run by CI.
Depends-On: Iebd6f6b25aff779efaa55ce9c25eec117403884c
Change-Id: I34d719f43f11e1793b012395d359af2e260f8743
This matches the DB domain logic enforcement in the other class methods.
Cleanup several type hints in related code.
Change-Id: I6668852ccebe16349b38b5a793543bd3ec0a35b8
* processResults continued to report failures even when a test was
newly passing.
* Updated result processing to recognize newly passing tests.
* I removed the CI-focused performance optimization that recognized
an expected failure and bailed early without trying to determine
if there was an unexpected passing test. This could lead to devs
not recognizing changes to test results and not updating the known
failures file to remove newly passing tests.
Change-Id: Iee0b2b3388028b327c30d63b17394149dd8cdc42
In particular, there was a test of __NEWSECTIONLINK__ added by
4e717c0158 (way back when) which seemed
to be intended to check that the appropriate ParserOutputFlag was set,
but that output flag was never included in the `flags=` parser test
output. Address that to improve coverage.
Change-Id: Iccd80e63e9fd724bf7dfb7f243f823a2b9f9915d
* Prevent crashers when selser tests are skipped.
* Don't run processResults when running in 'cache' mode.
* Added a isCachingMode() helper to TestMode.php -- not sure
if this was overkill, but did it anyway.
Change-Id: I2b100bd852306b9ce1de69117c6ca7d7d9c9ce3f
* This matches behavior in standalone mode.
* If we want to run these tests (especially selser),
we should probably add html section. For wt2wt tests,
we may end up tweaking the wikitext to reduce spurious
wt2wt failures because of normalization.
* So for now, skip these tests.
Change-Id: I2b9444ecae1bbf012aebdb81a134ee07ed4582c7
* This let us pass mocked revisions in the parser test runner while
running in Parsoid mode.
* This leads to improvement in wt2html tests results where a revision
id is queried. I've verified this in the Cite extension repo as
also the main parserTests.text file but I cannot enable Parsoid
integrated testing on the main parser tests file without doing a
sweep over all parser tests and adding appropriate test sections
* Currently, PageConfigFactory doesn't have unit tests. Will look
into adding them separately in a followup.
* Moved the setupParsoidTransform function to a more suitable place
in the ParserTestRunner.php file.
Bug: T270310
Change-Id: I94d68c8528bb2f7b367c68d80d14ebc1ab904a7f
* 3 more tests now pass that would only fail when run with other tests
but pass when run with --filter.
Change-Id: Ib0f97cd29b55ff27d891eee1160b66da1e95129c
When a test sets options to change global config, we have to rebuild
the ParsoidSiteConfig instance to pick up the test options since
the service container caches the SiteConfig instance.
For now, we are unconditionally resetting SiteConfig before every
<test,mode> combination. At a later time, we can figure out a way
to do it more conditionally; filed T310283 for that.
Bug: T310283
Change-Id: Ia2dfd37e3da3e1e8b31ab917b1f831eaea5eb5c5
Previously versions of Parsoid represented the !!config section as
a string which then had to be parsed by the ParserTestRunner.
Bug: T307720
Depends-On: Ieaac08e8171fd4c2a233646a84caac70ec30db6f
Change-Id: I3c5666d061c469049bf150ff0fa3010cb907a493
We introduced a new type for "ParserTestMode" and then uniformly pass
this around alongside the ParserTest object itself. (We started by
using a string for the $mode but the "explicit changetree" mode
requires more structured data. We *could* encode this in JSON just to
keep the string type around, but it seems cleaner and more future
proof to wrap this in a proper class type.)
Removed the old TestFileReader wrapper class, which served only as
a thunk to convert the new Test class into the same sort of array
which the legacy parser test runner code expected. There's still
some remaining ::testToArray() usage here for the TestRecorder
framework, but that will be cleaned up in a future patch.
Also updated the TestRecorder with stronger types: use the same
<Test,TestMode> pair when invoking the TestRecorder, instead of the
loosely typed array we'd used previously.
Change-Id: Iec4c0c7972a655b4faccacc67bdc3ca66d4c163d
Run phan over classes in tests/parser
The dependency of classes between parser and phpunit is not clear.
Classes used by both possible needs part of /common/
Change-Id: I2ceca6b7cd447876c127ed3b14e09f479defbd93
This reverts commit 123f785cdf.
Reason for revert: This breaks parserTests.php script since the ParserTestRunner script has references to both TestFileReader classes and this patch ends up using the same reader everywhere.
Change-Id: I22bdd05f9a86674e88335cc9055f1ce6cfa128d1
The ParserTestRunner code was old enough that it used a number of different
mechanisms to supply defaults for options passed via an array; uniformly use
the standard PHP "array add" mechanism instead.
Change-Id: I1791f7f8b1ae2cc6f4dd23351e9810a98b598be1
Long term there isn't going to be a "TestFileReader" and
"ParsoidTestFileReader", the legacy test file reader is just a
temporary hack. And these aren't "Parsoid Test Files" they
are *parser test files* (ie, implementation independent,
more or less). Long story short, adding "Parsoid" to the
class name is a step in the wrong direction.
Change-Id: I9b520c39c6c5b52334c658ec2a97bd38612dbf61
* This adds all of Parsoid's abilities to the parserTests.php script
* Parsoid testing mode requires a "--parsoid" CLI option.
* This supports "--wt2html" and the other testing modes,
"--updateKnownFailures", "--knownFailures", and "--changetree"
options that are part of the Parsoid's parserTests.php script.
* Verified that the output of parserTests.php test run matches
the output of the PHPUnit test run on the Cite extensions'
citeParserTests.txt file.
Change-Id: I87a1f621d0a2579ac47fa80d34cf74d4456bcf62
This patch improves the parser test runner in the following ways:
1. Adds Parsoid selser auto-edit test support
Since it isn't possible to dynamically add tests in PHPUnit
(unlike the home-grown parser test runners), to support selser
auto-edit tests, we create a single composite test that includes
all the individual edits. If a single test fails, the whole test
is considering failing.
The test output from all the individual tests are combined into
composite expected / actual strings and test failure diffs reveals
diffs for individual tests.
Future patches could enhance output further.
2. Ensure that we skip HTML-based Parsoid test modes without appropriate
input HTML
3. Pass additional options to the parser test runner via the
ParserTestTopLevelSuite constructor. These support extended
functionality of the Parsoid parser test runner which are being
made available to core.
At the moment the constructor will attempt to JSON-decode the
PARSERTEST_FLAGS environment variable to supply defaults for
these flags, which provides a hacky sort of way to pass arguments
on the command line. This will be integrated with the existing
$flags passed to ParserTestTopLevelSuite in the future (T308367).
4. Add support for running tests in specific modes as well
as ability to update the known failures json file.
This includes some missing code to run per-test teardown functions
in runParsoidTest.
Unlike the Parsoid parser test runner, you can combine
updateKnownFailures with --filter as well as with subset of
test modes, including possibly specific selser changetrees.
If `{"updateKnownFailures": true}` is set in the PARSERTEST_FLAGS var,
you can also update the knownFailures result for only the tests
that are run.
However, note that this could change the sort order in the
*-knownFailures.json file. This leads to dirty (git) diffs when
a full updateKnownFailures is run on the file. I am not going to
tackle this issue for now since this is an obscure use case that
is unlikely to be used much, if at all.
Overall, if I copy over the -knownFailures.json file from the Parsoid
repo into the Cite repo and update it with a phpunit.php run in core,
it updates the known failures to reflect changes to test results from
an integrated test run. A second run doesn't lead to any additional
changes to the knownFailures.json file or to any test failures. I had
to fix some bugs (referenced in the previous paragraph) to ensure
this idempotence beyond the first run.
This patch also cleans up some function signatures to use Test object
It maps the Test object from the Parsoid repo to an associative array
that a bunch of (legacy) parser test functions expect.
This patch squashes the following previous patches:
I2f46b4b94b44aec76e33403455eb466899376a6b
Ib01991089ec410dd45f77a20b136c8e05d855fa4
Id0ef01e333580892aa6d415b8f66c4393b06870d
Change-Id: I9952ab5ee3edef8e2f00b0e65cbffc42f46c2ea7
Instead of parsing the configuration section here in ParserTestRunner,
it should be done by the parser test file parser.
Change-Id: I75473106cd7dcde6cc289a904f5669eaac717be8
All the other ways of doing it were ridiculous and much harder to read,
and usually required repeating the needle expression (to get its
length). I found these occurrences by grepping for various expressions,
but I undoubtedly missed some.
I didn't try replacing the many instances of strpos(...) === 0 with
str_starts_with(...), because I think they're readable enough as-is
(although less efficient). Likewise I didn't try porting strpos(...) !==
false to str_contains(...). For case-insensitive comparisons, Tim
Starling requested that we stick with substr_compare() because it's more
efficient than calling strtolower().
On PHP < 8 these functions will be included with a polyfill via
vendor/autoload.php. This is included at the beginning of
includes/AutoLoader.php, so if our autoloader has been included the
polyfill will be available. This means it should be safe to call these
functions from any code that would not be usable without our autoloader.
Three uses that Tim Starling identified as being performance-sensitive
have been split out to a separate commit for porting after the switch to
PHP 8.
Change-Id: I113a8d052b6845852c15969a2f0e6fbbe3e9f8d9
Move resetting the bad file lookup service into addArticles, since it
needs to be reset whenever articles are added, and that happens in the
setUp of ParserTestFileSuite as well.
Bug: T75581
Change-Id: Iedc4cfe9b76762d64f9c0f9d813c64de5fb98dce
