This greatly simplifies logic required to compute module versions.
It also makes it significantly less error-prone.
Since f37cee996e, we support hashes as versions (instead of timestamps).
This means we can build a hash of the content directly, instead of compiling a
large array with all values that may influence the module content somehow.
Benefits:
* Remove all methods and logic related to querying database and disk for
timestamps, revision numbers, definition summaries, cache epochs, and more.
* No longer needlessly invalidate cache as a result of no-op changes to
implementation datails. Due to inclusion of absolute file paths in the
definition summary, cache was always invalidated when moving wikis to newer
MediaWiki branches; even if the module observed no actual changes.
* When changes are reverted within a certain period of time, old caches can now
be re-used. The module would produce the same version hash as before.
Previously when a change was deployed and then reverted, all web clients (even
those that never saw the bad version) would have re-fetch modules because the
version increased.
Updated unit tests to account for the change in version. New default version of
empty test modules is: "mvgTPvXh". For the record, this comes from the base64
encoding of the SHA1 digest of the JSON serialised form of the module content:
> $str = '{"scripts":"","styles":{"css":[]},"messagesBlob":"{}"}';
> echo base64_encode(sha1($str, true));
> FEb3+VuiUm/fOMfod1bjw/te+AQ=
Enabled content versioning for the data modules in MediaWiki core:
* EditToolbarModule
* JqueryMsgModule
* LanguageDataModule
* LanguageNamesModule
* SpecialCharacterDataModule
* UserCSSPrefsModule
* UserDefaultsModule
* UserOptionsModule
The FileModule and base class explicitly disable it for now and keep their
current behaviour of using the definition summary. We may remove it later, but
that requires more performance testing first.
Explicitly disable it in the WikiModule class to avoid breakage when the
default changes.
Ref T98087.
Change-Id: I782df43c50dfcfb7d7592f744e13a3a0430b0dc6
By providing context as a parameter in getDependencies, we allow
modules to dyanamically determine dependencies based on context.
Note: To ease rollout, the parameter is optional in this patch. It is expected
that it will be made non-optional in the near future.
The use case is for CentralNotice campaigns to be able to add special
modules ahead of deciding which banner to show a user. The dynamically
chosen RL modules would replace ad-hoc JS currently sent with some banners.
A list of possible campaigns and banners is already sent as a PHP-
implemented RL module; that's the module that will dynamically choose other
modules as dependencies when appropriate. This approach will save a round
trip as compared to dynamically loading the modules client-side.
For compatibility, extensions that override
ResourceLoaderModule::getDependencies() should be updated with the new
method signature. Here are changes for extensions currently deployed on
Wikimedia wikis:
* CentralNotice: I816bffa3815e2eab7e88cb04d1b345070e6aa15f
* Gadgets: I0a10fb0cbf17d095ece493e744296caf13dcee02
* EventLogging: I67e957f74d6ca48cfb9a41fb5144bcc78f885e50
* PageTriage: Ica3ba32aa2fc76d11a44f391b6edfc871e7fbe0d
* UniversalLanguageSelector: Ic63e617f51702c27104e123d4bed91983a726b7f
* VisualEditor: I0ac775ca286e64825e31a9213b94648e41a5bc30
For more on the CentralNotice use case, please see I9f80edcbcacca2.
Bug: T98924
Change-Id: Iee61e5b527321d01287baa03ad9b4d4f526ff3ef
Follows-up f37cee996e which replaced the getHashMtime() and
getDefinitionMtime() methods with dummies that always return 1.
These getModifiedTime() implementations were only tracking the
definition summary or custom hash, which is already tracked
by getVersionHash().
Notes:
* SpecialCharacterDataModule: This one was odd as it was tracking
both the mtime *and* the file contents.
* UserCSSPrefsModule/UserOptionsModule: Remove redundant caching.
Already taken care of by getVersionHash() as of f37cee996e.
Bug: T94074
Change-Id: I6e37c3c2f85ef4599a8643b0efabc18de2f51ec4
* Use time() instead of:
- wfTimestamp()
- wfTimestamp( TS_UNIX )
- wfTimestamp( TS_UNIX, 0 )
- wfTimestamp( TS_UNIX, time() )
- intval( wfTimestamp( TS_UNIX, time() ) )
* Consistently use 1 as default instead of 0. Previously the
unwritten convention was that anything "final" max()'ed with 1,
and any internal method would use 0, but this wasn't applied
consistently made it fragile. There doesn't seem to be any
value in returning 0 only to have it maxed up to 1 (because if
the 0 would ever make it out alive, we'd be in trouble).
* wfTimestamp returns a string for TS_UNIX. In PHP this doesn't
matter much. In fact, max() takes number-like integers so
transparently, it even preserves it:
> max( 1, 3, '2' );
< 3
> max( 1, '3', 2 );
< "3"
Just cast it in one place at the very end (StartupModule)
instead of doing intval( wfTimestamp() ).
* Fix weird documentation claiming getModifiedTime can return
an array, or mixed.
* Remove 'version > 1 ? version : 1' logic in
ResourceLoader::makeLoaderRegisterScript. The client doesn't
have "0 means now" behaviour so this isn't needed. And the method
was only doing it for variadic argument calls.
Removal of quotes around timestamps reduced the size of the startup
module from 26.8KB to 25.9KB before gzip. After gzip the size was
and still is 5.7KB, though. (From 5456 bytes to 5415 bytes.)
Change-Id: If92ca3e7511e78fa779f2f2701e2ab24db78c8a8
We currently embed the full set of user options in a <script> tag in the HTML
output of every page. This is grossly inefficient, because the full set of
options is usually largely made up of site defaults which the user hasn't
customized.
So instead of doing that, let's emit the default options using one
ResourceLoader module and then apply the user's customizations on top.
This has the effect of slightly increasing the total bytes of JavaScript code
(because options that the user has customized will be emitted twice: once with
their default value in the user.defaults module, and then again with the
customized value in user.options). But this is more than offset by the
fact that the bulk of user options code (~4 kB uncompressed on enwiki) becomes
cacheable across requests.
Bonus round:
* Varnish gets to cache 4 kB fewer per page.
* Changes to the default options don't take 30 days to propagate.
Change-Id: I5a7e258d2d69159381bf5cc363227088b8fd6019
Introduces ResourceLoaderContext::getUserObj(), which gets
a (possibly cached) User object for the context's username.
Use this instead of the $wgUser global.
Change-Id: Ifd9f634db145381625ab68067ae67791a3f494b8
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Change-Id: Ifbb1da2a6278b0bde2a6f6ce2e7bd383ee3fb28a
Split the variable assignment and the return statement in two lines for
better readability.
When there was two return statements in one method the logic was swapped
to have only one return statement.
Change-Id: Id7a01b4a2df96036435f9e1a9be5678dd124b0af
This will help with improving human readability of JS and JSON
objects encoded by both ResourceLoader and the API. This patch
also adds new "utf8" parameter to the JSON formatter of the API.
Changes to FormatJson class:
* Added escaping of '<', '>', and '&' by default to protect against XSS.
* Removed unnecessary escaping of '/' and added an additional option to
unescape non-ASCII characters (those above U+007F) as well.
* Added PHP 5.3 pretty printing code (to replace Services_JSON) that
uses a four-space indent as PHP 5.4 does.
Changes to Xml class:
* Defined Xml::encodeJsVar() in terms of FormatJson::encode()
and added a pretty printing option. Also added a pretty printing
option to Xml::encodeJsCall() as well.
* Deprecated Xml::escapeJsString() and QuickTemplate::jstext();
callers have to add quotes themselves, hence the escaping of
both double quotes and apostrophes.
Bug: 26818
Change-Id: I1987190f1ba5bf41738e7bd611209706c1f6bb5c
* (bug 35317) CSRF in Special:Upload
Revert r56793, which removed the CSRF check for Special:Upload for normal file
uploads. Cross-site posting of file uploads without user interaction has been
possible since at least as early as Chrome 8 (late 2010) and Firefox 6 (mid
2011).
Commonist has used api.php since version 0.4.0 (April 2010), and the API
already requires an edit token, so Commonist 0.4.0+ is not affected by this
change.
* (bug 34907) Fix for CSRF vulnerability due to mw.user.tokens. Patch by Roan
Kattouw and Tim Starling.
* Filter out private modules early in ResourceLoader::makeResponse() and just
pretend they weren't specified. This means these modules cannot be loaded
through load.php . This filtering must not happen in makeModuleResponse(),
because that would break inlining.
* Force inlining of private modules in OutputPage::makeResourceLoaderLink(),
disregarding $wgResourceLoaderInlinePrivateModules
* Remove $wgResourceLoaderInlinePrivateModules
* Remove special treatment of private modules ($private) in
ResourceLoader::makeResponse() and sendResponseHeaders(), because we're not
allowing private modules to be loaded through here any more
* Remove identity checks in ResourceLoaderUserOptionsModule and
ResourceLoaderUserCSSPrefsModule, they didn't make a lot of sense before but
they're certainly useless now.
* Factored out error comment construction in ResourceLoader.php and stripped
comment terminations from exception messages. I didn't find an XSS
vulnerability but it looked scary.
Patchset2:
Removes whitespace error that prevented automatic merge by Gerrit:
includes/resourceloader/ResourceLoaderUserOptionsModule.php
Change-Id: I2dec8b8caf9db3c64919763865cc10cccdd6a1a3
* Fix css flipping: for Simple, it still depended on the content language (however, no css actually needs flipping there). For Standard/CologneBlue, I re-added flipping even though it's not needed there either. Just in case css is added that does need flipping, so it is future-proof :). Consequently, I added noflip to the existing css.
* For this flipping I added a parameter to addInlineStyle(), which I assume to be useful in other cases as well
* Make ResourceLoaderUserOptionsModule and ResourceLoaderUserTokensModule depend on mw.user
* Load mw.user.{tokens,options,groups} load as TYPE_COMBINED instead of TYPE_SCRIPT. The latter wouldn't wrap the code in mw.loader.implement()
** ...but make sure 'user' (user scripts) is excluded, that one needs to not be wrapped in a closure
* Make TYPE_COMBINED actually work in makeResourceLoaderLink()
* Add a comment in makeModuleResponse() to explain what the weird is_array( $scripts ) stuff is all about
* Add FIXME about how mw.user.options should split off the CSS part into a separate module
TODO: Since the alias is globally available from the start, the 'mw' argument in the wrapper is redundant, so that should be removed at some point as well.
TODO:
* Are there instances where we might want to restrict CSS as well as JS?
* Would a $wg config option and/or user preference and/or index.php GET parameter to limit inclusion be useful?
* Can we deprecate any of the existing $wg config options?
* What's going on with the duplicated code between OutputPage and SkinTemplate?
* Modified Xml::encodeJsVar() to allow it to pass through JS expressions without encoding, using a special object.
* In ResourceLoader::makeModuleResponse(), renamed $messages to $messagesBlob to make it clear that it's JSON-encoded, not an array.
* Fixed MessageBlobStore to store {} for an empty message array instead of [].
* In ResourceLoader::makeMessageSetScript(), fixed call to non-existent function mediaWiki.msg.set.
* For security, changed the calling convention of makeMessageSetScript() and makeLoaderImplementScript() to require explicit object construction of XmlJsCode() before interpreting their input as JS code.
* Documented several ResourceLoader static functions.
* In ResourceLoaderWikiModule, for readability, reduced the indenting level by flipping some if blocks and adding continue statements.
* In makeCustomLoaderScript(), allow non-numeric $version. The only caller I can find is already sending a non-numeric $version, presumably it was broken. Luckily there aren't any loader scripts in existence, I had to make one to test it.
* wfGetDb -> wfGetDB
* Added an extra line break in the startup module output, for readability.
* In ResourceLoaderStartUpModule::getModuleRegistrations(), fixed another assignment expression
* Break long lines.
* Convert long or unnecessary ternary operator usages to if/else.
* Fixed excessively clever assignment expressions.
* Rename $cache to $cacheEntry.
* Removed unnecessary web invocation guards. Their perlish form was making me uncomfortable. BTW, unlike in Perl, die() is not a function, it's a special case in the PHP grammar which very roughly simulates the Perl syntax:
die "x"; // works
0 || die("x"); // works
0 || (die); // works
0 || (die "x"); // fail!
