The canonical way to enable debug mode is and will remain
through via 'debug=true'.
During the transition debug=2 will opt you in to the experimental
newer way. Anything that needs to be pinned to the old way for
compat can already start doing so by using debug=1 explicitly.
Once v2 is "ready", the default will flip and debug=1 will remain
for the foreseeable future to trigger the legacy behaviours.
Bug: T85805
Change-Id: Ieaf04e0c289646dd5d5b027b4f1f8278167b2d57
Instead of exporting this as a global variable, export it as an extension
attribute.
The $wgResourceModules configuration variable will continue to be supported
for its main purpose of adding modules. However it may no longer be used to
access the complete registry. In actuality, it was already incomplete because
modules defined via 'ResourceLoaderRegisterModules' hooks were already absent.
The full copy is now owned only by the ResourceLoader class and publicly
available via isModuleRegistered() and getModuleNames().
This opens the door to making it a lazy-loaded attribute, which would help
reduce the size of the "main" APCu cache key for ExtensionRegistry.
This is not possible with global variables as those must be exported
unconditionally from Setup.php.
Bug: T32956
Bug: T247265
Change-Id: I13cf05af458131ce8439eee770d7a62bd9404288
Instead of exporting this as a global variable, export it as an extension
attribute.
The $wgResourceModuleSkinStyles configuration variable will continue
to be supported for its main purpose of setting skin styles. However
it may no longer be used to read these settings. The canonical copy
of this is now restricted to the ResourceLoader class and not (yet)
made publicly available (I found no use cases for it).
This opens the door to making it lazy-loaded attribute, which would help
reduce the size of the "main" APCu cache key for ExtensionRegistry.
This is not possible with global variables as those must be exported
unconditionally from Setup.php.
Bug: T32956
Bug: T247265
Change-Id: I4ecf558d9c630c91959786d2573c34e619223cef
In a nut shell:
* We very often (52% of modules on enwiki) pad the hash with a zero,
which means the amount of bits we currently compute already fit in
6 characters already for most modules. For some modules (3%) we
even padded two zeroes.
* For the (now documented) use cases, the space of 78 Giga
(78 billion, or 78 milliard) seems more than we need. The space of
60 million should be enough.
This follows-up dfd046412f from 2016, which previously shortened the hash
down from 8 chars of base 64 (or 12 chars of hex) to 7 chars of base 32.
Before that change, the space was 281 Tera (64^8, or 16^12).
For more details see the added inline comment for ResourceLoader::makeHash,
and also the data at <https://phabricator.wikimedia.org/T229245>.
Bug: T229245
Change-Id: I9ad11772a33b3a44cb625275b1d7353e1393ee49
Reduces output by not needlessly performing a change client-side
for which we already know the result server-side.
Bug: T231168
Change-Id: I4b8749f976d04d24f85236ddd641c7a4c7f6c23a
This was done as a "clever" shortcut to make sure tests a little
but shorter, but also made them less consistent with normal code.
Remove this in favour of 'class' or 'factory' options as needed.
Also remove a bunch of unneeded register() calls.
The tests cover everything affected by this change.
Side fix - isFileModule should reject modules with 'factory'
the same way it rejected raw objects and non-FileModule 'class'
cases already. This is now covered by tests as well.
Bug: T222637
Change-Id: I3996317dbcd780cc6e0f82c84e769c08a3fc42bb
The ResourceLoaderModule::isRaw() feature and the ability to magically
switch a regular load.php request into raw mode is being removed soon.
Instead, specify raw=1 in the request url where that behaviour is needed.
Bug: T201483
Change-Id: Ie4564ec8e26ad53f2de1a43330d18a35b0498a63
This changeset implements T89432 and related tickets and is based on exploration
done at the Prague Hackathon. The goal is to identify tests in MediaWiki core
that can be run without having to install & configure MediaWiki and its dependencies,
and provide a way to execute these tests via the standard phpunit entry point,
allowing for faster development and integration with existing tooling like IDEs.
The initial set of tests that met these criteria were identified using the work Amir did in
I88822667693d9e00ac3d4639c87bc24e5083e5e8. These tests were then moved into a new subdirectory
under phpunit/ and organized into a separate test suite. The environment for this suite
is set up via a PHPUnit bootstrap file without a custom entry point.
You can execute these tests by running:
$ vendor/bin/phpunit -d memory_limit=512M -c tests/phpunit/unit-tests.xml
Bug: T89432
Bug: T87781
Bug: T84948
Change-Id: Iad01033a0548afd4d2a6f2c1ef6fcc9debf72c0d
lang=qqx and skin=fallback are the default values.
This change removes the default values from the load request of the
html5shiv module.
Before this change
<script src="/w/load.php?lang=qqx&modules=html5shiv&only=scripts&skin=fallback&sync=1"></script>
With this change
<script src="/w/load.php?modules=html5shiv&only=scripts&sync=1"></script>
Change-Id: Ie384ce0f7ab1bd0b6c2d3f0ca4a990c3cf3a7f15
Use global variables directly as globals, except for the specific
case of accessing it when it may not exist. In those cases we
use the fact that undefined properties of an object yield the
`undefined` value which we can cast to false. Accessing an undefined
variable would yield a ReferenceError exception.
Change-Id: I1d9e9aa5845ba3c756ad6e31358d8594e003b04b
Turn these embedded function calls into variable declarations,
later consumed by startup.js.
This has a few benefits:
* Variables can be declared immediately, without needing to
defer it to after mw.loader arrives from startup, via an RLQ
callback. This makes the inline script simpler and a tiny
bit smaller.
* By moving the actual function calls to startup.js, they could
easily be re-arranged in the future or migrated in other ways
without needing to worry about HTML caching and keeping concurrent
compatibility with both orders of execution.
Change-Id: I1c995a9572d9eb3201a565341c19bdf81ab00eff
One of the assertions already did this, but the others not yet.
The reason we do this is because the return value is a WrappedString
object. The assertEquals() method is able to satisfy the expected
value because of loose comparison casting it to a string for the
case where it matches.
But, when it fails during development, PHPUnit was still printing
the object as "Actual" and the string as "Expected" which makes
it difficult to understand what's going on, and also no diff.
Change-Id: Ice9fe5061fb76867e9bee2f34c88d3ddac3144f9
The test was dual-purposing the same array both to create
a ResourceLoaderContext object, and to pass further down as
$extraQuery parameter.
This meant that ResourceLoaderClientHtml::makeLoad, and indirectly
ResourceLoader::makeLoaderQuery, did not have any control over
the formatting of the query string, due to it forcing the key/value
pairs regardless (in addition to being redundant).
This is needed in order to omit debug=false by default, which is
the next commit.
Change-Id: I3c098a9a5055ec8f2b64319c6cadd13c030ba76a
Reduce our reliance on static state and configuration, and
propagate more state in explicit ways, through context, and
request parameters.
OutputPage creates ResourceLoaderContext and ResourceLoaderClientHtml
based on the configuration (via ResourceLoader::inDebugMode).
Everything within those classes should not need to check it
again.
* ResourceLoaderClientHtml:
Already doesn't check MW config, but it's test was still
mocking it. Removed now, and confirmed that it passes both
with true and false. The individual test cases set
debug=true/false as needed already.
It's sets were previously relying on the accidental behaviour
that within a unit test, we don't serialise over HTTP, which
meant that a pure PHP boolean would survive. With the new
raw `=== 'true'` check, this no longer works. Set it as a
string explicitly instead, which is the only thing we support
outside unit tests as well.
* ResourceLoaderContext:
Remove fallback to MW config when 'debug' is unset.
This is never unset in practice given that all load.php
urls have it set by OutputPage based on ResourceLoader::inDebugMode.
This change means that manually constructed ad-hoc load.php
urls that are missing 'debug=' parameter, will now always be
read as debug=false. This was the default already, but could
previously be changed through wgResourceLoaderDebug.
When changing wgResourceLoaderDebug, everything will still have
debug=true as before. The only change is when constructing load.php
urls manually and explicitly not set it.
Bug: T32956
Change-Id: Ie3424be46e2b8311968f3068ca08ba6a1139224a
The addModuleScripts() methods were deprecated in 1.31 and 1.32,
these are now removed.
The getModuleScripts() are now deprecated as well, always returning
an empty array. To be removed in 1.34.
Depends on commits for bundled/wmf-deployed extensions that
remove the last few remaining callers to the deprecated functions
in: 3D, Collection, Flow, GlobalUserPage, and Wikibase.
Bug: T188689
Depends-On: If9f0bc6aef85117587fa1929f34f8861c8d80314
Depends-On: Ia8d41b97fbf6822f5f8f7ac889408acce1ac9a3a
Depends-On: I503b919739ea474ff33726815b0da55e2f7e2724
Depends-On: I236ef637fd03b810a46eb361e25067a037e9d183
Depends-On: I62e17779753b977a452cc0c9694947941e999cc3
Change-Id: I5a19b8f164ccf666485d2971202194b747f882df
Expose this constant for internal use by Navigation Timing,
so that it can compute mwLoadEnd based on when these modules
finish loading.
The way mwLoadEnd is currently computed is by building a list
of all registered module names, and narrowing it down to the
ones in 'loading' state at the time that ext.navigationTiming.js
executes. The problem with doing that, is that it is includes
various lazy-loaded modules that aren't critical to the page and
aren't meant to be tracked by that metric. For example:
* Preloading of modules from various extensions (including VE,
and Popups).
* Background chains for EventLogging schemas from mw.loader.using,
including the one started by Navigation Timing itself.
On my local install, the list of filtered down modules always
includes 'schema.SaveTiming', '...rumSpeedIndex', etc.
Exporting the list passed to the initial load() call as constant
will enable Navigation Timing to instead only await only those
modules (and their dependneices).
Bug: T204426
Change-Id: Ida134b4dfee218db16c2d1f88d4f26e8d761e154
Follows-up If35a106c7. These log messages are not criticial and
should not be in the <head> competing with stylesheets and article
content. Move them to the end of <body> instead, nearby other
low-priority script tags.
The getBodyHtml() method from ClientHtml was empty, but has been
non-empty in the past. It's fine to repopulate.
Also, while ClientHtml::getBodyHtml was empty, there are additional
RLQ scripts created by OutputPage that do exist even without this.
Namely, there is a <script> for wgPageParseReport, and one for
wgBackendResponseTime etc.
Change-Id: Ibda7091bdcd5ed207395b20196cdc33df926a24c
This effectively applies safemode to the mw.loader client,
without the client itself needing specific knowledge of safemode.
Test Plan:
* Unchanged: When viewing a page in safemode, the 'user' and
'site' modules are still not queued by OutputPage.
* New: mw.loader.getState('site'), previously would yield
'registered', but will now yield null.
* New: mw.loader.load('site'), previously loaded the module,
it now logs a dependency warning for unknown module, like for
any other unknown module.
* New: mw.loader.using('site'), previously resolved, it is now
rejected.
Bug: T185303
Change-Id: I672e3891c8e1b3c2d13655fa134d0f1d031b8247
Follows-up 70941efd35 which broke various public
signatures of the ClientHtml class that I'd prefer to handle
differently.
This commit mainly restores support for all previously public
signatures, and either removes the need for a parameter, or moves
it to the end of the original signature (as optional param).
* ClientHtml::getHeadHtml: Remove the positional/required parameter
that was added. Restoring the method to being a stateless computer
that requires no parameters. Pass the option via construct instead.
* ClientHtml::makeLoad:
- Make $nonce optional.
- Restore $extraQuery as optional.
* ResourceLoader::makeInlineScript: Document $nonce as optional
(matching the implementation).
Change-Id: Iaf33f2a060048e6606fba8d875b6d2953b21ef45
The deprecation warning for the module 'mediawiki.ui' (used
e.g. on Special:UserLogin) is now actually shown.
Change-Id: If35a106c77622dbf7e8b5628fbea28f9e7ffd76d
The primary goal here is a defense in depth measure to
stop an attacker who found a bug in the parser allowing
them to insert malicious attributes.
This wouldn't stop someone who could insert a full
script tag (since at current it can't distinguish between
malicious and legit user js). It also would not prevent
DOM-based or reflected XSS for anons, as the nonce value
is guessable for anons when receiving a response cached
by varnish. However, the limited protection of just stopping
stored XSS where the attacker only has control of attributes,
is still a big win in my opinion. (But it wouldn't prevent
someone who has that type of xss from abusing things like
data-ooui attribute).
This will likely break many gadgets. Its expected that any
sort of rollout on Wikimedia will be done very slowly, with
lots of testing and the report-only option to begin with.
This is behind feature flags that are off by default, so
merging this patch should not cause any change in default
behaviour.
This may break some extensions (The most obvious one
is charinsert (See fe648d41005), but will probably need
some testing in report-only mode to see if anything else breaks)
This uses the unsafe-eval option of CSP, in order to
support RL's local storage thingy. For better security,
we may want to remove some of the sillier uses of eval
(e.g. jquery.ui.datepicker.js).
For more info, see spec: https://www.w3.org/TR/CSP2/
Additionally see:
https://www.mediawiki.org/wiki/Requests_for_comment/Content-Security-Policy
Bug: T135963
Change-Id: I80f6f469ba4c0b608385483457df96ccb7429ae5
When a module has group=user specified, it means that its module contents
can vary by user. These kinds of requests have two special needs:
1) They need an additional "user" parameter in their load.php request,
so that the response knows which user-context to use.
2) They need to have their 'version' hash pre-computed based on which assets
will be loaded for this user. The general 'version' hash associated with
this module name in the main registry (modules=startup) will be "wrong"
as that is computed based on logged-out status.
We do this by omitting the module name from the `mw.load.load(Array modules)`
call in the HTML, and instead output a request for the full url.
This currently works fine for most cases, such as the 'user' module loaded
by MediaWiki core. The branch in getData() dealing with legacy 'only=scripts'
behaviour also covers this case.
But the case of an extension registering a group=user module and loading it the
general way (e.g. not with legacy only=scripts behaviour), would currently end
up in the Array-queue and dynamically loaded by the client-side without knowing
the correct version hash. Fortunately, no code exists that I know of that meets
these three critera (extension registered, group=user, non-legacy). However,
for the GlobalCssJs extension to migrate from legacy to non-legacy, they will
need to start doing this. This commit makes sure that that will work.
The makeLoad() method in ClientHtml has code ensuring the full-url form (with
pre-computed 'version' hash) is used for any modules with group=user. Before
this patch, we didn't get to call makeLoad() because getData() was assuming
that we only need makeLoad() when either the module should be embedded (group=private),
or when it is a style/scripts-only module. It didn't consider group=user.
Bug: T188689
Change-Id: Iaab15e5f5c12e7e28b8c81beab90948cd07cd352
In preparation for passing down 'safemode' from OutputPage.
Only used in one place in Wikimedia Git: OutputPage::getRlClient().
Bug: T185303
Change-Id: If01eca96986ff8d7dcdaab6910bf183ba7c7311f
The getData() tests did have a test case for a 'user' group module with
isKnownEmpty in the script-only queue, but not yet a non-empty one.
The makeLoad() tests did have both already. Arguably the makeLoad()
tests are higher level and observe its behaviour already (outputting
a script tag means 'loading', outputting nothing means 'ready'),
but adding it to the getData() tests for consistency.
Bug: T188689
Change-Id: I842febf996ba8bb0ea25b5ca3b5ac4503db87376
This parameter is a no-op for these tests, as such, nothing was being
tested by specifying it in tests. The behaviour 'test' and 'test.top'
was identical.
Bug: T184257
Change-Id: Ia2bb731f00d4b4175f7b75174aeafaca9412329e
Clean up use of @codingStandardsIgnore
- @codingStandardsIgnoreFile -> phpcs:ignoreFile
- @codingStandardsIgnoreLine -> phpcs:ignore
- @codingStandardsIgnoreStart -> phpcs:disable
- @codingStandardsIgnoreEnd -> phpcs:enable
For phpcs:disable always the necessary sniffs are provided.
Some start/end pairs are changed to line ignore
Change-Id: I92ef235849bcc349c69e53504e664a155dd162c8
Used by Special:JavaScriptTest, and we may need to use this
in OutputPage as well (for html5shiv).
Change-Id: If7d7c12056dc3aab78486050f0798d42b158d9a9
It was already supported for only=scripts and only=styles, but
not yet for general modules loaded as 'combined'.
This is especially useful for general modules that are private/embedded,
as this allows us to skip the loader.implement() boilerplate.
Bug: T176159
Change-Id: I83e9f74dad75867c10ebd0a94b87c31e4adbb57f
Rather than only the 'private' group triggering embedding, allow modules
to explicitly specify if they should be embedded.
The default is still to only embed when the group is 'private', and the
'private' group is still special in that ResourceLoader::respond() will
still refuse to serve it from load.php.
Change-Id: Ib9a043c566822e278baecc15e87f9c5cebc2eb98
If a type=general module is enqueued, don't try to load it as a
stylesheet.
* Per a464d1d41d, state tracking is already disabled for
these loads (as otherwise we wrongly claim state=ready, when in
fact only the styles and not the scripts were loaded).
* The warning was added in a464d1d41d.
* Default install (tested in Vagrant), Wikimedia Beta cluster, and
Wikimedia production have seen zero violations of this warning
in the past 7 days.
Raise severity to ERROR and add the 'continue' statement so that
these are now not loaded at all.
Bug: T92459
Change-Id: I211d56ac2df479ebf5b98667c613ecf81489539b
The example for a mixed user module is outdated. Current user modules
all separate styles and scripts.
To make the effective change by the next commit easier to review,
update these tests first seperately.
Change-Id: I76a8a96eed8a8c39863c535362b1d0144ef2f866
Replaces \TestingAccessWrapper (defined in core) with
\Wikimedia\TestingAccessWrapper (defined in the composer package
wikimedia/testing-access-wrapper).
See https://gerrit.wikimedia.org/r/#/q/topic:librarize-testing-access-wrapper
for downstream patches.
The core version of the class is kept around for a while to avoid
circular dependency problems.
Bug: T163434
Change-Id: I52cc257e593da3d6c3b01a909e554a950225aec8
Previously, style modules were only in a predictable order for production mode.
In debug mode, the order was determined by order in which modules were added
to queue at run time. This made it sometimes hard to debug, especially when
dealing with gadgets that apply in a different order among each other.
Change-Id: I4bff0c91d127e4ad8015cd8c1775220fe460cbc3
* The styles queue has always been top-only
(except for a few months in 2015).
* The top queue loads asynchronous since mid-2015. (T107399)
And LocalStorage eval, previously the last remaining non-async part
of module loading, is also async as of October 2016. (T142129)
* This change merges the bottom 'mw.loader.load()' queue with the top queue.
It also moves any other snippets potentially in the bottom queue still:
- embed: I couldn't find any private modules with position=bottom
(doesn't make sense due to their blocking nature). If any do exist,
(third-party extensions?), they'll now be embedded in the <head>.
- scripts: Any legacy 'only=scripts' requests will now initiate
from the <head>.
Bug: T109837
Change-Id: I6c21e3e47c23df33a04c42ce94bd4c1964599c7f
Follows-up 6fa1e56. This is already fixed for http caches by
shortening the Cache-Control max-age in case of a version mismatch.
However the client still cached it blindly in mw.loader.store.
Resolve this by communicating to the client what version of the module
was exported. The client can then compare this version to the version
it originally requested and decide not to cache it.
Adopt the module key format (name@version) from mw.loader.store
in mw.loader.implement() as well.
Bug: T117587
Change-Id: I1a7c44d0222893afefac20bef507bdd1a1a87ecd
HTML formatting of the queue was distributed over several OutputPage methods.
Each method demanding a snippet of HTML by calling makeResourceLoaderLink()
with a limited amount of information. As such, makeResourceLoaderLink() was
unable to provide the client with the proper state information.
Centralising it also allows it to better reduce duplication in HTML output
and maintain a more accurate state.
Problems fixed by centralising:
1. The 'user' module is special (due to per-user 'version' and 'user' params).
It is manually requested via script-src. To avoid a separate (and wrong)
request from something that requires it, we set state=loading directly.
However, because the module is in the bottom, the old HTML formatter could
only put state=loading in the bottom also. This sometimes caused a wrong
request to be fired for modules=user if something in the top queue
triggered a requirement for it.
2. Since a464d1d4 (T87871) we track states of page-style modules, with purpose
of allowing dependencies on style modules without risking duplicate loading
on pages where the styles are loaded already. This didn't work, because the
state information about page-style modules is output near the stylesheet,
which is after the script tag with mw.loader.load(). That runs first, and
mw.loader would still make a duplicate request before it learns the state.
Changes:
* Document reasons for style/script tag order in getHeadHtml (per 09537e83).
* Pass $type from getModuleStyles() to getAllowedModules(). This wasn't needed
before since a duplicate check in makeResourceLoaderLink() verified the
origin a second time.
* Declare explicit position 'top' on 'user.options' and 'user.tokens' module.
Previously, OutputPage hardcoded them in the top. The new formatter doesn't.
* Remove getHeadScripts().
* Remove getInlineHeadScripts().
* Remove getExternalHeadScripts().
* Remove buildCssLinks().
* Remove getScriptsForBottomQueue().
* Change where Skin::setupSkinUserCss() is called. This methods lets the skin
add modules to the queue. Previously it was called from buildCssLinks(),
via headElement(), via prepareQuickTemplate(), via OutputPage::output().
It's now in OutputPage::output() directly (slightly earlier). This is needed
because prepareQuickTemplate() calls bottomScripts() before headElement().
And bottomScript() would lazy-initialise the queue and lock it before
setupSkinUserCss() is called from headElement().
This makes execution order more predictable instead of being dependent on
the arbitrary order of data extraction in prepareQuickTemplate (which varies
from one skin to another).
* Compute isUserModulePreview() and isKnownEmpty() for the 'user' module early
on so. This avoids wrongful loading and fixes problem 1.
Effective changes in output:
* mw.loader.state() is now before mw.loader.load(). This fixes problem 2.
* mw.loader.state() now sets 'user.options' and 'user.tokens' to "loading".
* mw.loader.state() now sets 'user' (as "loading" or "ready"). Fixes problem 1.
* The <script async src> tag for 'startup' changed position (slightly).
Previously it was after all inline scripts and stylesheets. It's still after
all inline scripts and after most stylesheets, but before any user styles.
Since the queue is now formatted outside OutputPage, it can't inject the
meta-ResourceLoaderDynamicStyles tag and user-stylesheet hack in the middle
of existing output. This shouldn't have any noticable impact.
Bug: T87871
Change-Id: I605b8cd1e1fc009b4662a0edbc54d09dd65ee1df
