This allows ResourceLoader::filter() to handle this case slightly
faster, since it searches for this annotation from the beginning.
In practice this is a negligible performance optimization, but let's
set a good example for the future.
Also tweak the comments and whitespace:
* Move comment about the FILTER_NOMIN from doc comment to code comment,
it's an implementation detail and not meant for public documentation
* Remove duplicated documentation from the parent class
* Change whitespace to be identical in both cases
Change-Id: I624914ff28d903027ba58710708ccc3c66af9e24
Follows-up 6fa4893928 (T84960), which disabled minification
for 'user.tokens'.
In 2014 'user.tokens' module was changed to change tokens on every page view,
even for the same user within a short period of time. This led to a huge
minify-cache growth, and we subsequently disabled caching for its minification
result.
Since then, we have also done:
* Disable minification for 'user.tokens' more generally, given
that it's just a simple JSON blob and we already pass down the 'debug' mode
flag down to the creation of that blob, so there's virtually nothing to left
to minify.
* Disable minification for mw.config.set(). Config values are exported by
ResourceLoaderClientHtml and not part of any module. Given that since 2015
we minify per-module and not per-response (b7eb243d92) that means config
data naturally doesn't go through minification, which is good, because
just like for 'user.tokens', the config data is JSON which is already created
without whitespace. Minification would be pointless.
However, 'user.options' is still being minified and cached, and makes up about
25% of the 'resourceloader:minify-js.*' keys in APC on Wikimedia servers.
Unlike 'user.tokens', the minify cache for 'user.options' does actually get
used by subsequent page views from the same user (unless preferences changed).
However, it isn't useful because it's plain JSON and already compressed enough.
Besides, the blob is so small that it's not worth the overhead of cache-checking.
If we would want to minify it, I'd recommend we re-minify each view without cache.
Change-Id: Ic0ffb0e23df9b40e2c1283c89bd876a5c0555951
This allows ResourceLoaderClientHtml to output state=ready instead
of state=loading with a no-op mw.loader.implement() call being embedded.
Test Plan:
* View source on page view when logged-out.
* Before:
- loader.state({"user.options":"loading"})
- loader.implement("user.options@..",function(){ .. user.options.state([]) })
* After:
- loader.state({"user.options":"ready"})
Bug: T176159
Change-Id: I18f76eaa960da9b0ca77f28e10f768587447a838
HTML formatting of the queue was distributed over several OutputPage methods.
Each method demanding a snippet of HTML by calling makeResourceLoaderLink()
with a limited amount of information. As such, makeResourceLoaderLink() was
unable to provide the client with the proper state information.
Centralising it also allows it to better reduce duplication in HTML output
and maintain a more accurate state.
Problems fixed by centralising:
1. The 'user' module is special (due to per-user 'version' and 'user' params).
It is manually requested via script-src. To avoid a separate (and wrong)
request from something that requires it, we set state=loading directly.
However, because the module is in the bottom, the old HTML formatter could
only put state=loading in the bottom also. This sometimes caused a wrong
request to be fired for modules=user if something in the top queue
triggered a requirement for it.
2. Since a464d1d4 (T87871) we track states of page-style modules, with purpose
of allowing dependencies on style modules without risking duplicate loading
on pages where the styles are loaded already. This didn't work, because the
state information about page-style modules is output near the stylesheet,
which is after the script tag with mw.loader.load(). That runs first, and
mw.loader would still make a duplicate request before it learns the state.
Changes:
* Document reasons for style/script tag order in getHeadHtml (per 09537e83).
* Pass $type from getModuleStyles() to getAllowedModules(). This wasn't needed
before since a duplicate check in makeResourceLoaderLink() verified the
origin a second time.
* Declare explicit position 'top' on 'user.options' and 'user.tokens' module.
Previously, OutputPage hardcoded them in the top. The new formatter doesn't.
* Remove getHeadScripts().
* Remove getInlineHeadScripts().
* Remove getExternalHeadScripts().
* Remove buildCssLinks().
* Remove getScriptsForBottomQueue().
* Change where Skin::setupSkinUserCss() is called. This methods lets the skin
add modules to the queue. Previously it was called from buildCssLinks(),
via headElement(), via prepareQuickTemplate(), via OutputPage::output().
It's now in OutputPage::output() directly (slightly earlier). This is needed
because prepareQuickTemplate() calls bottomScripts() before headElement().
And bottomScript() would lazy-initialise the queue and lock it before
setupSkinUserCss() is called from headElement().
This makes execution order more predictable instead of being dependent on
the arbitrary order of data extraction in prepareQuickTemplate (which varies
from one skin to another).
* Compute isUserModulePreview() and isKnownEmpty() for the 'user' module early
on so. This avoids wrongful loading and fixes problem 1.
Effective changes in output:
* mw.loader.state() is now before mw.loader.load(). This fixes problem 2.
* mw.loader.state() now sets 'user.options' and 'user.tokens' to "loading".
* mw.loader.state() now sets 'user' (as "loading" or "ready"). Fixes problem 1.
* The <script async src> tag for 'startup' changed position (slightly).
Previously it was after all inline scripts and stylesheets. It's still after
all inline scripts and after most stylesheets, but before any user styles.
Since the queue is now formatted outside OutputPage, it can't inject the
meta-ResourceLoaderDynamicStyles tag and user-stylesheet hack in the middle
of existing output. This shouldn't have any noticable impact.
Bug: T87871
Change-Id: I605b8cd1e1fc009b4662a0edbc54d09dd65ee1df
This greatly simplifies logic required to compute module versions.
It also makes it significantly less error-prone.
Since f37cee996e, we support hashes as versions (instead of timestamps).
This means we can build a hash of the content directly, instead of compiling a
large array with all values that may influence the module content somehow.
Benefits:
* Remove all methods and logic related to querying database and disk for
timestamps, revision numbers, definition summaries, cache epochs, and more.
* No longer needlessly invalidate cache as a result of no-op changes to
implementation datails. Due to inclusion of absolute file paths in the
definition summary, cache was always invalidated when moving wikis to newer
MediaWiki branches; even if the module observed no actual changes.
* When changes are reverted within a certain period of time, old caches can now
be re-used. The module would produce the same version hash as before.
Previously when a change was deployed and then reverted, all web clients (even
those that never saw the bad version) would have re-fetch modules because the
version increased.
Updated unit tests to account for the change in version. New default version of
empty test modules is: "mvgTPvXh". For the record, this comes from the base64
encoding of the SHA1 digest of the JSON serialised form of the module content:
> $str = '{"scripts":"","styles":{"css":[]},"messagesBlob":"{}"}';
> echo base64_encode(sha1($str, true));
> FEb3+VuiUm/fOMfod1bjw/te+AQ=
Enabled content versioning for the data modules in MediaWiki core:
* EditToolbarModule
* JqueryMsgModule
* LanguageDataModule
* LanguageNamesModule
* SpecialCharacterDataModule
* UserCSSPrefsModule
* UserDefaultsModule
* UserOptionsModule
The FileModule and base class explicitly disable it for now and keep their
current behaviour of using the definition summary. We may remove it later, but
that requires more performance testing first.
Explicitly disable it in the WikiModule class to avoid breakage when the
default changes.
Ref T98087.
Change-Id: I782df43c50dfcfb7d7592f744e13a3a0430b0dc6
By providing context as a parameter in getDependencies, we allow
modules to dyanamically determine dependencies based on context.
Note: To ease rollout, the parameter is optional in this patch. It is expected
that it will be made non-optional in the near future.
The use case is for CentralNotice campaigns to be able to add special
modules ahead of deciding which banner to show a user. The dynamically
chosen RL modules would replace ad-hoc JS currently sent with some banners.
A list of possible campaigns and banners is already sent as a PHP-
implemented RL module; that's the module that will dynamically choose other
modules as dependencies when appropriate. This approach will save a round
trip as compared to dynamically loading the modules client-side.
For compatibility, extensions that override
ResourceLoaderModule::getDependencies() should be updated with the new
method signature. Here are changes for extensions currently deployed on
Wikimedia wikis:
* CentralNotice: I816bffa3815e2eab7e88cb04d1b345070e6aa15f
* Gadgets: I0a10fb0cbf17d095ece493e744296caf13dcee02
* EventLogging: I67e957f74d6ca48cfb9a41fb5144bcc78f885e50
* PageTriage: Ica3ba32aa2fc76d11a44f391b6edfc871e7fbe0d
* UniversalLanguageSelector: Ic63e617f51702c27104e123d4bed91983a726b7f
* VisualEditor: I0ac775ca286e64825e31a9213b94648e41a5bc30
For more on the CentralNotice use case, please see I9f80edcbcacca2.
Bug: T98924
Change-Id: Iee61e5b527321d01287baa03ad9b4d4f526ff3ef
Follows-up f37cee996e which replaced the getHashMtime() and
getDefinitionMtime() methods with dummies that always return 1.
These getModifiedTime() implementations were only tracking the
definition summary or custom hash, which is already tracked
by getVersionHash().
Notes:
* SpecialCharacterDataModule: This one was odd as it was tracking
both the mtime *and* the file contents.
* UserCSSPrefsModule/UserOptionsModule: Remove redundant caching.
Already taken care of by getVersionHash() as of f37cee996e.
Bug: T94074
Change-Id: I6e37c3c2f85ef4599a8643b0efabc18de2f51ec4
* Use time() instead of:
- wfTimestamp()
- wfTimestamp( TS_UNIX )
- wfTimestamp( TS_UNIX, 0 )
- wfTimestamp( TS_UNIX, time() )
- intval( wfTimestamp( TS_UNIX, time() ) )
* Consistently use 1 as default instead of 0. Previously the
unwritten convention was that anything "final" max()'ed with 1,
and any internal method would use 0, but this wasn't applied
consistently made it fragile. There doesn't seem to be any
value in returning 0 only to have it maxed up to 1 (because if
the 0 would ever make it out alive, we'd be in trouble).
* wfTimestamp returns a string for TS_UNIX. In PHP this doesn't
matter much. In fact, max() takes number-like integers so
transparently, it even preserves it:
> max( 1, 3, '2' );
< 3
> max( 1, '3', 2 );
< "3"
Just cast it in one place at the very end (StartupModule)
instead of doing intval( wfTimestamp() ).
* Fix weird documentation claiming getModifiedTime can return
an array, or mixed.
* Remove 'version > 1 ? version : 1' logic in
ResourceLoader::makeLoaderRegisterScript. The client doesn't
have "0 means now" behaviour so this isn't needed. And the method
was only doing it for variadic argument calls.
Removal of quotes around timestamps reduced the size of the startup
module from 26.8KB to 25.9KB before gzip. After gzip the size was
and still is 5.7KB, though. (From 5456 bytes to 5415 bytes.)
Change-Id: If92ca3e7511e78fa779f2f2701e2ab24db78c8a8
We currently embed the full set of user options in a <script> tag in the HTML
output of every page. This is grossly inefficient, because the full set of
options is usually largely made up of site defaults which the user hasn't
customized.
So instead of doing that, let's emit the default options using one
ResourceLoader module and then apply the user's customizations on top.
This has the effect of slightly increasing the total bytes of JavaScript code
(because options that the user has customized will be emitted twice: once with
their default value in the user.defaults module, and then again with the
customized value in user.options). But this is more than offset by the
fact that the bulk of user options code (~4 kB uncompressed on enwiki) becomes
cacheable across requests.
Bonus round:
* Varnish gets to cache 4 kB fewer per page.
* Changes to the default options don't take 30 days to propagate.
Change-Id: I5a7e258d2d69159381bf5cc363227088b8fd6019
Introduces ResourceLoaderContext::getUserObj(), which gets
a (possibly cached) User object for the context's username.
Use this instead of the $wgUser global.
Change-Id: Ifd9f634db145381625ab68067ae67791a3f494b8
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Change-Id: Ifbb1da2a6278b0bde2a6f6ce2e7bd383ee3fb28a
Split the variable assignment and the return statement in two lines for
better readability.
When there was two return statements in one method the logic was swapped
to have only one return statement.
Change-Id: Id7a01b4a2df96036435f9e1a9be5678dd124b0af
This will help with improving human readability of JS and JSON
objects encoded by both ResourceLoader and the API. This patch
also adds new "utf8" parameter to the JSON formatter of the API.
Changes to FormatJson class:
* Added escaping of '<', '>', and '&' by default to protect against XSS.
* Removed unnecessary escaping of '/' and added an additional option to
unescape non-ASCII characters (those above U+007F) as well.
* Added PHP 5.3 pretty printing code (to replace Services_JSON) that
uses a four-space indent as PHP 5.4 does.
Changes to Xml class:
* Defined Xml::encodeJsVar() in terms of FormatJson::encode()
and added a pretty printing option. Also added a pretty printing
option to Xml::encodeJsCall() as well.
* Deprecated Xml::escapeJsString() and QuickTemplate::jstext();
callers have to add quotes themselves, hence the escaping of
both double quotes and apostrophes.
Bug: 26818
Change-Id: I1987190f1ba5bf41738e7bd611209706c1f6bb5c
* (bug 35317) CSRF in Special:Upload
Revert r56793, which removed the CSRF check for Special:Upload for normal file
uploads. Cross-site posting of file uploads without user interaction has been
possible since at least as early as Chrome 8 (late 2010) and Firefox 6 (mid
2011).
Commonist has used api.php since version 0.4.0 (April 2010), and the API
already requires an edit token, so Commonist 0.4.0+ is not affected by this
change.
* (bug 34907) Fix for CSRF vulnerability due to mw.user.tokens. Patch by Roan
Kattouw and Tim Starling.
* Filter out private modules early in ResourceLoader::makeResponse() and just
pretend they weren't specified. This means these modules cannot be loaded
through load.php . This filtering must not happen in makeModuleResponse(),
because that would break inlining.
* Force inlining of private modules in OutputPage::makeResourceLoaderLink(),
disregarding $wgResourceLoaderInlinePrivateModules
* Remove $wgResourceLoaderInlinePrivateModules
* Remove special treatment of private modules ($private) in
ResourceLoader::makeResponse() and sendResponseHeaders(), because we're not
allowing private modules to be loaded through here any more
* Remove identity checks in ResourceLoaderUserOptionsModule and
ResourceLoaderUserCSSPrefsModule, they didn't make a lot of sense before but
they're certainly useless now.
* Factored out error comment construction in ResourceLoader.php and stripped
comment terminations from exception messages. I didn't find an XSS
vulnerability but it looked scary.
Patchset2:
Removes whitespace error that prevented automatic merge by Gerrit:
includes/resourceloader/ResourceLoaderUserOptionsModule.php
Change-Id: I2dec8b8caf9db3c64919763865cc10cccdd6a1a3
* Fix css flipping: for Simple, it still depended on the content language (however, no css actually needs flipping there). For Standard/CologneBlue, I re-added flipping even though it's not needed there either. Just in case css is added that does need flipping, so it is future-proof :). Consequently, I added noflip to the existing css.
* For this flipping I added a parameter to addInlineStyle(), which I assume to be useful in other cases as well
* Make ResourceLoaderUserOptionsModule and ResourceLoaderUserTokensModule depend on mw.user
* Load mw.user.{tokens,options,groups} load as TYPE_COMBINED instead of TYPE_SCRIPT. The latter wouldn't wrap the code in mw.loader.implement()
** ...but make sure 'user' (user scripts) is excluded, that one needs to not be wrapped in a closure
* Make TYPE_COMBINED actually work in makeResourceLoaderLink()
* Add a comment in makeModuleResponse() to explain what the weird is_array( $scripts ) stuff is all about
* Add FIXME about how mw.user.options should split off the CSS part into a separate module
TODO: Since the alias is globally available from the start, the 'mw' argument in the wrapper is redundant, so that should be removed at some point as well.
TODO:
* Are there instances where we might want to restrict CSS as well as JS?
* Would a $wg config option and/or user preference and/or index.php GET parameter to limit inclusion be useful?
* Can we deprecate any of the existing $wg config options?
* What's going on with the duplicated code between OutputPage and SkinTemplate?
* Modified Xml::encodeJsVar() to allow it to pass through JS expressions without encoding, using a special object.
* In ResourceLoader::makeModuleResponse(), renamed $messages to $messagesBlob to make it clear that it's JSON-encoded, not an array.
* Fixed MessageBlobStore to store {} for an empty message array instead of [].
* In ResourceLoader::makeMessageSetScript(), fixed call to non-existent function mediaWiki.msg.set.
* For security, changed the calling convention of makeMessageSetScript() and makeLoaderImplementScript() to require explicit object construction of XmlJsCode() before interpreting their input as JS code.
* Documented several ResourceLoader static functions.
* In ResourceLoaderWikiModule, for readability, reduced the indenting level by flipping some if blocks and adding continue statements.
* In makeCustomLoaderScript(), allow non-numeric $version. The only caller I can find is already sending a non-numeric $version, presumably it was broken. Luckily there aren't any loader scripts in existence, I had to make one to test it.
* wfGetDb -> wfGetDB
* Added an extra line break in the startup module output, for readability.
* In ResourceLoaderStartUpModule::getModuleRegistrations(), fixed another assignment expression
* Break long lines.
* Convert long or unnecessary ternary operator usages to if/else.
* Fixed excessively clever assignment expressions.
* Rename $cache to $cacheEntry.
* Removed unnecessary web invocation guards. Their perlish form was making me uncomfortable. BTW, unlike in Perl, die() is not a function, it's a special case in the PHP grammar which very roughly simulates the Perl syntax:
die "x"; // works
0 || die("x"); // works
0 || (die); // works
0 || (die "x"); // fail!
