Locally this page doesn't load reliably for me, as
I have extensions which register broken resource loader
modules (eg. Math depends on VE, which I don't have
locally). This will lead to mw.loader.using rejecting
its promise directly, before any of the (working) tests
have been loaded. In order to work around this, wait for
each test module seperately and only start qunit
once all are resolved/ rejected.
Note: This shouldn't start more requests than the old
version as all modules will already be requested by
a single mw.loader.load call running before this logic
is invoked.
Change-Id: Ibee18d5a96068a6f00e5e0bae58355662324c2b4
This will be obsolete once we adopt version=2 but WMF CI isn't ready
for that yet per <https://phabricator.wikimedia.org/T250045#8671019>.
Meanwhile, we can reduce use of the global static method in one place.
SpecialJavaScriptTest has two main methods: renderPage() and exportJS().
renderPage() renders the HTML special page, and is where one might want
$wgResourceLoaderDebug to apply (which is the only thing that
inDebugMode does, in addition to just casting $_GET from string to
int, and then back to a string again, to be passed to RL\Context
to become an int again). This makes sense as OutputPage also uses
inDebugMode() like this.
exportJS() responds to the internal JS request made from the above
HTML page. This doesn't need to call this as it already gets the
debug= parameter set by the HTML. While impossible in practice, it
would make no sense for this to be in debug mode when the page isn't.
Like for index.php/OutputPage and load.php, we only call inDebugMode
on index.php. On load.php it gets debug= explicitly passed by the first
one and doesn't need to evaluate its own global state.
Bug: T250045
Bug: T36738
Change-Id: I2064a6cdc5970aec42d6bfaf70ea0c577642d465
This is the recommendation upstream (admittedly edited by me, but
it's been that way for 10+ years) as per
<https://qunitjs.com/intro/#in-the-browser>.
The reason we did it in JS is because of karma-runner, which loads
the test suite directly into an empty browser tab without the visual
HTML page. Thus to get the fixture there, we had to create it
ourselves. But, this is a bug in the karma-qunit adapater as it is
meant to take responsibility for creating this.
And, since at least 2015, it does precisely that per
<https://github.com/karma-runner/karma-qunit/pull/22> so this is no
longer needed!
Bug: T250045
Change-Id: I2a2bf7d231d34bf54dc6c0ff3173eada74ae3ff1
This reverts commit 1f5b4764d5.
Reason for revert: It appears the Apache server in CI is flaky in a way
we previously weren't able to observe in practice, but is now causing
regularly one of the ~100 or so quick concurrent debug=2 requests to
fail. Previously, there were only a handful of bundled non-debug reqs.
Bug: T330314
Bug: T330293
Change-Id: I65773e01bfe90ee509d1c1b422f093fb8da11c54
I don't recall why I added this. Possibly in a confused effort
to match /tests/phpunit, except /tests/phpunit/suites is not
where test cases live, they live under /tests/phpunit/* directly,
mostly /tests/phpunit/includes named after the source directory.
The correct equivalent to that is /tests/qunit/resources for JS.
While at it, also remove mention of this concept from various other
places where it doesn't add value. It's one more word/concept to
learn, process, understand, or translate mentally. They're just tests,
or for the one or two places where we care about how they are
internally transmitted, a "test module".
Bug: T250045
Change-Id: I5ea22e4965d190357aa69883f29f9049ee8ebf13
Move ResourceLoader classes to their own namespace. Strip the
"ResourceLoader" prefix from all except ResourceLoader itself.
Move the tests by analogy.
I used a namespace alias "RL" in some callers since RL\Module is less
ambiguous at the call site than just "Module".
I did not address DependencyStore which continues to have a non-standard
location and namespace.
Revert of a241d83e0a.
Bug: T308718
Change-Id: Id08a220e1d6085e2b33f3f6c9d0e3935a4204659
This reverts commit e08ea8ccb9.
Reason for revert: Breaks Phan in extensions, and as far as I’m aware,
this change isn’t urgently needed for anything, so the simplest fix is
to revert it again for now. After PHP 7.4 it should be safer to try this
again (we hopefully won’t need the two “hack” classes by then).
Bug: T308443
Change-Id: Iff3318cbf97a67f821f78e60da62a583f63e389e
Move ResourceLoader classes to their own namespace. Strip the
"ResourceLoader" prefix from all except ResourceLoader and
ResourceLoaderContext.
Move the tests by analogy.
I used a namespace alias "RL" in some callers since RL\Module is less
ambiguous at the call site than just "Module".
I did not address DependencyStore which continues to have a non-standard
location and namespace.
Change-Id: I92998ae6a82e0b935c13e02a183e7c324fa410a3
Now largely automated:
VARS=$(grep -o "'[A-Za-z0-9_]*'" includes/MainConfigNames.php | \
tr "\n" '|' | sed "s/|$/\n/;s/'//g")
sed -i -E "s/'($VARS)'/MainConfigNames::\1/g" \
$(grep -ERIl "'($VARS)'" includes/)
Then git add -p with lots of error-prone manual checking. Then
semi-manually add all the necessary "use" lines:
vim $(grep -L 'use MediaWiki\\MainConfigNames;' \
$(git diff --cached --name-only --diff-filter=M HEAD^))
I didn't bother fixing lines that were over 100 characters unless they
were over 120 and triggered phpcs.
Bug: T305805
Change-Id: I74e0ab511abecb276717ad4276a124760a268147
This covers all occurrences of /onfig->.*get( '/ in includes/.
Undoubtedly there are still plenty more to go.
Change-Id: I33196c4153437778496f40436bcde399638ac361
As part of making sure tests are not accidentally depending on global
state or some aspect of the skin, Special page, logged-out user, etc,
continue the removal of skin-specific code from SpecialJavaScriptTest.
This follows T131389 which previously removed the HTML layout,
stylesheets, and additional scripts from the test runner page already.
Bug: T89434
Depends-On: I9485b006804e243f9e022ca0668b290cf9d53f32
Depends-On: Ib20c1f5277c5d5a99f5117c82b7cbdeab75d3279
Change-Id: Iaf989bc50363a2525a922f745c0a1a6a992404f9
Usage:
* DiscussionTools tests: `grunt qunit --qunit-component=DiscussionTools`
* MediaWiki core tests only: `grunt qunit --qunit-component=MediaWiki`
It might be nice to have this more granular, to run tests for a single
file, or a single ResourceLoaderModule, but will leave that to another
time; it is already fast.
Note that some projects may fail, e.g. GrowthExperiments has a test
failure due to not mocking all of the dependencies needed for the
test code; that can be worked around by adjusting the application code
or the test registration to include all needed dependencies.
Example fixes:
* GrowthExperiments: Iea3a89a902cd17dd91696d2a37144baa9e452077
* MinervaNeue: I8f747e3df196055361d53d00fea833fb9de892f0
The only other extension I see that has one broken test when run
this way is VisualEditor.
Bug: T250045
Change-Id: I977150ece26f5878a0ccc42d39bed6c7e74e9602
* Use the int result of `ResourceLoader::inDebugMode()` directly in
SpecialJavaScriptTest so that debug=2 is passed on to the next
request, like it does on regular page views.
This is likely temporary, because
part of T85805 is to make debug=true default to debug=2, and
part of T250045 is to hardcode debug=2 for JavaScriptTest.
* Add a few line breaks to improve HTML output readability in debug
mode. This is a no-op for prod as they get minified away.
Today I learned: PHP string heredoc and nowdoc syntax strip not
only the shared common indentation (which I knew), it also strips
the first and last newline. Hence, switching ClientHtml to use
double quotes.
* Document the order in which lang/skin/debugScripts execute.
Bug: T85805
Bug: T250045
Change-Id: I2ad179401cee6cf6ba30c0b462f0d48a8529ba74
This name is consist with the rest of the setter and getter methods
in ParserOutput. Renamed the methods in OutputPage, ImageHistoryList,
ImageHistoryPseudoPager, and ContribsPager as well for consistency;
it also makes chasing down lingering references in codesearch easier.
Soft-deprecated the old name for 1.38. Hard-deprecation will follow,
but there are a number of users in production that should be chased
down first.
Code search:
https://codesearch.https://codesearch.wmcloud.org/deployed/?q=(allow%7Cprevent)Clickjacking&i=nope&files=&excludeFiles=&repos=
Bug: T287216
Change-Id: I9822c60c180d204bd30cb4447a1120155d456da4
The distinction between the two was lost several years years ago
when the "position top" queue ceased to be a thing. Since then,
the two have been loaded and executed together in the same batch,
and are doing similar things.
mediawiki.page.ready is publicly used in several places
as dependendency, but mediawiki.page.startup is entirely internal
to core, which makes it the easier one of the two to dissolve.
Keep an alias for two weeks for cache compat to avoid console
warnings about unknown modules. Although even some cache still
refers to it, this is harmless since the errors are recoverable
and the correct module was also loaded by the cached pages
already.
Bug: T260210
Change-Id: Ic418c23a7400abba22fd07b17f173d3c5f1d1d10
Follows-up ba50b32556, (introduce standalone "plain" mode),
and 0f9e4ca0fb (remove the skinned mode).
While both the HTML and JS payloads are already standalone, they did
internally use a fake ResourceLoaderContext object that had
the "real" skin and user language set (e.g. from the currently
logged-in user and/or site defaults).
This basically only affects 'skinStyles' and 'messages', neither
of which should be used by tests in any meaningful way.
Bug: T250045
Change-Id: Ib31934cd6dfdc5e37bccecf918f94a74e28a04b4
== What
This feature is disabled initially because we want to be able to
test it. When mw.loader's own tests are testing mw.loader.store,
they are mocking setTimeout and mw.requestIdleCallback, so that
they can then make mw.loader.store schedule its "flush" callback
and via the mock timer control when it executes and then assert
its result.
== The Bad
Previously an inline JS hack was concatenated right after startup.js,
and that seems like it should be early enough to prevent anything
from initialising mw.loader.store and scheduling a real 2s flush timer.
There is no obvious sign here that the startup module response would
request or implement a module (which could inititialise mw.loader.store).
However, there is.
== …, the Ugly
The startup response contains RLQ processing (which is empty in
the standalone test runner, so no worries here) and a call to
mw.loader.load for RLPAGEMODULES. The RLPAGEMODULES list is empty,
but it does still make a call to mw.loader.load(). And that
expands the empty array to include jquery+mediawiki.base and thus
makes a proper module request, which then initialises mw.loader.store
and schedules its 2s flush timeout.
This hasn't caused failing tests in CI so far, because there are
generally at least 2 seconds of unrelated tests that run first.
So by the time mw.loader's test suite comes around, it has
been disabled and the previous flush has already completed.
== … and the Good
Change I5f1067feb0a43d makes the 'mediawiki.jqueryMsg' test suite
super fast (previously took 2+ seconds, longer than any other test).
This exposes the fact that mw.loader.store was in fact not
actually properly disabled from the get go, and so tests would
be failing.
Bug: T250045
Change-Id: I38c3ad2a9a5813215dbb210bddafcc3cdd70295d
* Remove calls to setHeaders(), disallowUserJs().
These never do anything because the 10 lines of standalone HTML
page outputs don't consult OutputPage or Skin.
The exception was the 404 handler for if you visited
Special:JavaScriptTest/bogus, where there is nothing to protect,
not to mention that the page is disabled by default and in any
production environment anyhow.
* Inline the 404 handler logic.
* Update the outdated documentation for exportQUnit().
Bug: T250045
Change-Id: I2b9d712f439b270ae998e994113b7f3cd8601abf
The intro text visually renders the same without it. The parsed
message already has its own block (a paragraph).
This div is a left-over from when the test runner was a skinned
page, and we had some CSS to style it in a fancy way. All that
was removed many years a go in favour of the 10-line standalone
HTML testrunner we have today.
Bug: T250045
Change-Id: Iffcca90857cf8e5df09a9287ab22fad109df7b79
Much less indirection this way, making the test runner a bit easier
to reason about (maintenance-wise) and easier to debug for developers.
Minification doesn't help here anyway (quite the opposite).
This also means the legacy option to load a module synchronously
from ResourceLoader\ClientHtml can be removed. This option existed
solely to bootstrap the QUnit test runner, and can be removed in a
subsequent commit.
Bug: T250045
Change-Id: I73985048382e9cc754753ed84f04d25214c07599
For back-compat, keep 'user.tokens' as deprecated alias to 'user.options'
for one release cycle (to be removed in MW 1.36).
== user.options ==
As before, 'user.options' arrives immediately on every page view,
embedded in the HTML. It has an async dependency on 'user.defaults',
which is not downloaded until there is a known demand on
'user.options'. Once that arrives, the implementation closure
of 'user.options' will execute, and the module becomes 'ready'.
== user.options "empty" ==
Before this change, UserOptionsModule used isKnownEmpty to consider the
module "empty" for logged-out users (as well as for logged-in users that
haven't yet set any preferences).
This was a mistake. It is invalid in ResourceLoader to mark a module as
"empty" if that module has dependencies (see also T191596 and c3f200849).
This broke the state machine. The impact was minimal given that it is unlikely
for features to read keys from mw.user.options for logged-out users, which
if attempted would have simply returned null for all keys.
== New HTML ==
The user.options module is always embedded (never empty), and always
has a dependency on user.defaults.
== Cached HTML ==
The cached HTML for anons sets user.options's state to ready without
waiting for any dependency. Per the above, this was already causing
subtle bugs with mw.user.options.get() likely returning null for anons,
which was fairly innocent. For tokens a bottom value of null would be
problematic as the default for tokens must be "+\" instead. To make
sure that is available for cached page views, set this directly
in mediawiki.base.js. The cached HTML does contain an implement call for
'user.tokens' that contains the same defaults, but new code will not
be asking for or waiting for user.tokens, so that is unused.
Bug: T235457
Change-Id: I51e01d6fa604578cd2906337bde5a4760633c027
The test-only modules registered by QUnitTestResources.php are currently
were previously caught by the array_keys() catch-all in registerTestModules()
which meant that modules like 'test.sinonjs' would be requested on
SpecialJavaScriptTest despite not doing anything by itself, nor executing
at the "right" time per se through this means.
In order for it to execute at the right time, the testrunner has to depend
on it (which it does, already). But, that also means it doesn't need to
be requested separately. Doing so could be confusing.
This is neccecary in order to move 'jquery.qunit' from Resources.php
to QUnitTestResources.php as otherwise, listing in QUnitTestResources.php,
would implicitly mean SpecialJavaScriptTest.php thinks it's a test suite
and load it. That is a problem, because when we run the tests headless from
the command-line with Karma, the environment already has a QUnit interface
defined, and should not be loaded a second time by MW.
Change-Id: I08b31cd1dee516cf0d26bafdb8cc7c1223633bad
The ResourceLoaderModule::isRaw() feature and the ability to magically
switch a regular load.php request into raw mode is being removed soon.
Instead, specify raw=1 in the request url where that behaviour is needed.
Bug: T201483
Change-Id: Ie4564ec8e26ad53f2de1a43330d18a35b0498a63
This commit implements step 4 and step 5 of the plan outlined at T192623.
Before this task began, the typical JavaScript execution flow was:
* HTML triggers request for startup module (js req 1).
* Startup module contains registry, site config, and triggers
a request for the base modules (js req 2).
* After the base modules arrive (which define jQuery and mw.loader),
the startup module invokes a callback that processes RLQ,
which is what will request modules for this page (js req 3).
In past weeks, we have:
* Made mediawiki.js independent of jQuery.
* Spun off 'mediawiki.base' from mediawiki.js – for everything
that wasn't needed for defining `mw.loader`.
* Moved mediawiki.js from the base module request to being embedded
as part of startup.js.
The concept of dependencies is native to ResourceLoader, and thanks to the
use of closures in mw.loader.implement() responses, we can download any
number of interdependant modules in a single request (or parallel requests).
Then, when a response arrives, mw.loader takes care to pause or resume
execution as-needed. It is normal for ResourceLoader to batch several modules
together, including their dependencies.
As such, we can eliminate one of the two roundtrips required before a
page can request modules. Specifically, we can eliminate "js req 2" (above),
by making the two remaining base modules ("jquery" and "mediawiki.base") an
implied dependency for all other modules, which ResourceLoader will naturally
fetch and execute in the right order as part of the batch request.
Bug: T192623
Change-Id: I17cd13dffebd6ae476044d8d038dc3974a1fa176
If the jQuery promise returned by mw.loader.using() is rejected
synchronously (e.g. because one of the modules has a dependency on
non-existent module), the function passed to fail() also executes
synchronously. Note that this is true also with jQuery 3.
The exception it throws was caught by the catch() below, which
resulted in start() being called twice, which resulted in QUnit
throwing 'Uncaught Error: Called start() outside of a test
context too many times' rather than actually starting.
Change-Id: I5c6b50647c0af0fdec6547aaa59165f6b4a42642
If the mw.loader.using() call's promise is rejected, we end up
passing an Error object as the 'count' parameter to QUnit.start().
This seems to be harmless, but is very confusing when debugging.
Change-Id: I44caca5285dbced5a5876d9d7ff6236dbd3efc35
ResourceLoader errors, like invalid dependencies, are
hard to spot and only result in the special page
not finding any tests.
This is not a perfect solution but it would have
saved me a full day of troubleshooting.
Change-Id: I247174f89772b84b4cad31deffb03152921df020
HTML formatting of the queue was distributed over several OutputPage methods.
Each method demanding a snippet of HTML by calling makeResourceLoaderLink()
with a limited amount of information. As such, makeResourceLoaderLink() was
unable to provide the client with the proper state information.
Centralising it also allows it to better reduce duplication in HTML output
and maintain a more accurate state.
Problems fixed by centralising:
1. The 'user' module is special (due to per-user 'version' and 'user' params).
It is manually requested via script-src. To avoid a separate (and wrong)
request from something that requires it, we set state=loading directly.
However, because the module is in the bottom, the old HTML formatter could
only put state=loading in the bottom also. This sometimes caused a wrong
request to be fired for modules=user if something in the top queue
triggered a requirement for it.
2. Since a464d1d4 (T87871) we track states of page-style modules, with purpose
of allowing dependencies on style modules without risking duplicate loading
on pages where the styles are loaded already. This didn't work, because the
state information about page-style modules is output near the stylesheet,
which is after the script tag with mw.loader.load(). That runs first, and
mw.loader would still make a duplicate request before it learns the state.
Changes:
* Document reasons for style/script tag order in getHeadHtml (per 09537e83).
* Pass $type from getModuleStyles() to getAllowedModules(). This wasn't needed
before since a duplicate check in makeResourceLoaderLink() verified the
origin a second time.
* Declare explicit position 'top' on 'user.options' and 'user.tokens' module.
Previously, OutputPage hardcoded them in the top. The new formatter doesn't.
* Remove getHeadScripts().
* Remove getInlineHeadScripts().
* Remove getExternalHeadScripts().
* Remove buildCssLinks().
* Remove getScriptsForBottomQueue().
* Change where Skin::setupSkinUserCss() is called. This methods lets the skin
add modules to the queue. Previously it was called from buildCssLinks(),
via headElement(), via prepareQuickTemplate(), via OutputPage::output().
It's now in OutputPage::output() directly (slightly earlier). This is needed
because prepareQuickTemplate() calls bottomScripts() before headElement().
And bottomScript() would lazy-initialise the queue and lock it before
setupSkinUserCss() is called from headElement().
This makes execution order more predictable instead of being dependent on
the arbitrary order of data extraction in prepareQuickTemplate (which varies
from one skin to another).
* Compute isUserModulePreview() and isKnownEmpty() for the 'user' module early
on so. This avoids wrongful loading and fixes problem 1.
Effective changes in output:
* mw.loader.state() is now before mw.loader.load(). This fixes problem 2.
* mw.loader.state() now sets 'user.options' and 'user.tokens' to "loading".
* mw.loader.state() now sets 'user' (as "loading" or "ready"). Fixes problem 1.
* The <script async src> tag for 'startup' changed position (slightly).
Previously it was after all inline scripts and stylesheets. It's still after
all inline scripts and after most stylesheets, but before any user styles.
Since the queue is now formatted outside OutputPage, it can't inject the
meta-ResourceLoaderDynamicStyles tag and user-stylesheet hack in the middle
of existing output. This shouldn't have any noticable impact.
Bug: T87871
Change-Id: I605b8cd1e1fc009b4662a0edbc54d09dd65ee1df
Leaving behind only the so-called "plain" mode.
Also removed related unused messages (follows-up 6b758fc).
The execute() method continues to enforce 404 Not Found for arbitrary
subpage urls so that we keep the door open to add other sub resources or
test frameworks in the future.
Bug: T131389
Change-Id: I4c22666fb98e54c47ed1b4d12776af6fc43ee473
We're almost ready to drop the non-plain mode of Special:JavaScriptTest
in favour of Special:JavaScript/qunit/plain. There's a few mobile-related
extensions still using the non-karma mode for qunit testing.
However, none of them make use of the Skin selector, which was mainly a debug
thing I added in the initial version. It no longer makes sense since our tests
now enforce an anti-dependency on skin html and other context. Encouraging
testing in multiple skins in the old UI therefore no longer makes sense.
This also fixes one of the most frequent errors in resourceloader logs and gets
rid of an ugly hack in Resources.php that causes a small amount of overhead
in ResourceLoader::__construct().
> MessageBlobStore failed to find skinname-fallback
> MessageBlobStore failed to find skinname-apioutput
Change-Id: Idaacf718703883c6a7e83a17ccd3f41ebdca53d1
Page startup:
* Due to the startup module and top queue being asynchronous now,
move client-nojs/client-js class handling to OutputPage to ensure
there is no flashes of wrongly styled or unstyled content.
To preserve compatibility for unsupported browsers, undo the
class swap at runtime after the isCompatible() check.
ResourceLoader startup module:
* Load the startup module with <script async>.
* Use DOM methods instead of 'document.write' to create base module request (jquery|mediawiki).
mw.loader:
* Drop 'async' parameter from mw.loader.load().
* Remove the now-unused code paths for synchronous requests.
OutputPage:
* Drop '$loadCall' parameter from makeResourceLoaderLink().
Asynchronous is now the default and only way to load JavaScript.
This means the 'user' module "conditional document-write scripts"
are now a simple "mw.loader.load( url )" call.
* Fix incorrect @return of makeResourceLoaderLink(). This returns
an array not a string.
* Improve documentation of makeResourceLoaderLink().
* Drop '$inHead' parameter from getScriptsForBottomQueue(). No longer used.
Compatibility with the $wgResourceLoaderExperimentalAsyncLoading
feature is maintained. It just no longer needs to change the
way the queue works since it's always asynchronous. The feature
flag now only controls whether the bottom queue starts at the bottom
or starts at the top.
* Remove jQuery.ready() optimisation.
This was mostly there to avoid the setTimeout() loop jQuery does to detect
dom-ready in IE6/IE7 (which we no longer serve JavaScript at all).
And for a bug in Firefox with document.write (which is no longer used as of
this commit).
Bug: T107399
Change-Id: Icba6d7a87b239bf127a221bc6bc432cfa71a4a72
To make unit testing easier, allow any module to be loaded within
the unit test suite. Regardless of the intended 'target'.
Targets are meant for restricting front-end scope in production.
Enforcing that in the test suite causes various test suites to
get de-registered at run time client-side.
Otherwise, in order to truly run all unit tests, Jenkins would
have to re-run the entire test suite in all known targets. This
wouldn't make sense because modules have to be globally uniquely
named (no conflicts) and unit tests are atomic. They can all run
in the same suite.
To prevent modules being comitted with incompatible target
dependencies, we already have a Structure test in the PHPUnit
suite to catch those issues in the module registry.
This makes the main 'qunit' build for MobileFrontend more useful,
where currently many modules aren't being tested due to them not
being in the 'desktop' target.
Bug: T103027
Change-Id: I69f735eb56c1362189298d9859d3add576faaadb
These were never enabled or used in production and are not
compatible with the upcoming async changes (T107399). To avoid
having to maintain compatibility with this, remove it for now.
The current on-going request to operations for ESI support is unrelated
to this code.
Considered making makeResourceLoaderLink() protected as it's not
used anywhere in @wikimedia Git outside mediawiki-core. And the unit
test actually treated it as protected already. However it's called
in SpecialJavaScriptTest so leaving that as-is for now.
In Icba6d7a87b239 the signature will change again with the removal
of the $loadCall parameter, which is obsolete in an async world
due to document.write being forbidden.
Change-Id: I9f557cc794638ffd15329934865e21e1027f7cfa
Currently there's only one framework, so having an error landing page
when visiting Special:JavaScriptTest isn't helpful. DWIM and send the
user to Special:JavaScriptTest/qunit/plain if that is the only framework
that is configured.
Also add the testing help link to the "/plain" view.
Change-Id: Ifc473d080ecf6f0a9add0510480ba9dad76050e9
