This ensures 'wikibits' and 'mediawiki.util' (if so configured)
finish loading before executing other modules.
This was previously implicitly provided for the bottom queue by
being in the top-queue, but since this is now asynchronous that
is no lower guaranteed. Fix this by making this explicit instead
of implicit.
Keep them in the top queue as-is to ensure consistency with cached
pages and it allows them to preload and batch in the same request
instead of being discovered later at run-time as a separate request.
Bug: T108124
Change-Id: I74e0cbe616404da927ea46d06308a7bae930eb69
Page startup:
* Due to the startup module and top queue being asynchronous now,
move client-nojs/client-js class handling to OutputPage to ensure
there is no flashes of wrongly styled or unstyled content.
To preserve compatibility for unsupported browsers, undo the
class swap at runtime after the isCompatible() check.
ResourceLoader startup module:
* Load the startup module with <script async>.
* Use DOM methods instead of 'document.write' to create base module request (jquery|mediawiki).
mw.loader:
* Drop 'async' parameter from mw.loader.load().
* Remove the now-unused code paths for synchronous requests.
OutputPage:
* Drop '$loadCall' parameter from makeResourceLoaderLink().
Asynchronous is now the default and only way to load JavaScript.
This means the 'user' module "conditional document-write scripts"
are now a simple "mw.loader.load( url )" call.
* Fix incorrect @return of makeResourceLoaderLink(). This returns
an array not a string.
* Improve documentation of makeResourceLoaderLink().
* Drop '$inHead' parameter from getScriptsForBottomQueue(). No longer used.
Compatibility with the $wgResourceLoaderExperimentalAsyncLoading
feature is maintained. It just no longer needs to change the
way the queue works since it's always asynchronous. The feature
flag now only controls whether the bottom queue starts at the bottom
or starts at the top.
* Remove jQuery.ready() optimisation.
This was mostly there to avoid the setTimeout() loop jQuery does to detect
dom-ready in IE6/IE7 (which we no longer serve JavaScript at all).
And for a bug in Firefox with document.write (which is no longer used as of
this commit).
Bug: T107399
Change-Id: Icba6d7a87b239bf127a221bc6bc432cfa71a4a72
To make unit testing easier, allow any module to be loaded within
the unit test suite. Regardless of the intended 'target'.
Targets are meant for restricting front-end scope in production.
Enforcing that in the test suite causes various test suites to
get de-registered at run time client-side.
Otherwise, in order to truly run all unit tests, Jenkins would
have to re-run the entire test suite in all known targets. This
wouldn't make sense because modules have to be globally uniquely
named (no conflicts) and unit tests are atomic. They can all run
in the same suite.
To prevent modules being comitted with incompatible target
dependencies, we already have a Structure test in the PHPUnit
suite to catch those issues in the module registry.
This makes the main 'qunit' build for MobileFrontend more useful,
where currently many modules aren't being tested due to them not
being in the 'desktop' target.
Bug: T103027
Change-Id: I69f735eb56c1362189298d9859d3add576faaadb
Most wikis only use user signatures on pages set aside from discussion,
(Talk namespaces and sometimes project/main namespaces depending on
the wiki), so having the button available everywhere is confusing.
The few wikis that need the button (especially non-content,
internal/corporate/planning wikis), can add relevant namespaces to the
$wgExtraSignatureNamespaces array in LocalSettings.
This would make it possible to solve bugs like T59727 or T53154.
Since this is a change to default behavior, a release note is added.
Bug: T7645
Change-Id: I7ccf1093b888c7b33721234349ca0ac054c3cd3f
The code is easier to maintain in an actual JavaScript file.
Especially with how variables were declared and concatenated in
a different order.
Change-Id: I758acb78de1cdf2128e81c86f992807ef0dbf444
- Removed space after casts
- Removed spaces in array index
- Added spaces around string concat
- Added space after words: switch, foreach
- else if -> elseif
- Removed parentheses around require_once, because it is not a function
- Added newline at end of file
- Removed double spaces
- Added spaces around operations
- Removed repeated newlines
Bug: T102609
Change-Id: Ib860222b24f8ad8e9062cd4dc42ec88dc63fb49e
By providing context as a parameter in getDependencies, we allow
modules to dyanamically determine dependencies based on context.
Note: To ease rollout, the parameter is optional in this patch. It is expected
that it will be made non-optional in the near future.
The use case is for CentralNotice campaigns to be able to add special
modules ahead of deciding which banner to show a user. The dynamically
chosen RL modules would replace ad-hoc JS currently sent with some banners.
A list of possible campaigns and banners is already sent as a PHP-
implemented RL module; that's the module that will dynamically choose other
modules as dependencies when appropriate. This approach will save a round
trip as compared to dynamically loading the modules client-side.
For compatibility, extensions that override
ResourceLoaderModule::getDependencies() should be updated with the new
method signature. Here are changes for extensions currently deployed on
Wikimedia wikis:
* CentralNotice: I816bffa3815e2eab7e88cb04d1b345070e6aa15f
* Gadgets: I0a10fb0cbf17d095ece493e744296caf13dcee02
* EventLogging: I67e957f74d6ca48cfb9a41fb5144bcc78f885e50
* PageTriage: Ica3ba32aa2fc76d11a44f391b6edfc871e7fbe0d
* UniversalLanguageSelector: Ic63e617f51702c27104e123d4bed91983a726b7f
* VisualEditor: I0ac775ca286e64825e31a9213b94648e41a5bc30
For more on the CentralNotice use case, please see I9f80edcbcacca2.
Bug: T98924
Change-Id: Iee61e5b527321d01287baa03ad9b4d4f526ff3ef
As of b1e4006b4, the tokens are different on every request.
Caching these is completely useless because the cache entry is
simply unreachable and is extra overhead on every request for
logged-in users to save content to Memcached.
Whether they should be minified at all and whether they perhaps
shouldn't change on every request is a separate matter.
Bug: T84960
Change-Id: I6016e4b01e44e0acbfd6d49f7d99555e2290c9cb
Modules now track their version via getVersionHash() instead of getModifiedTime().
== Background ==
While some resources have observeable timestamps (e.g. files stored on disk),
many other resources do not. E.g. config variables, and module definitions.
For static file modules, one can e.g. revert one of more files in a module to a
previous version and not affect the max timestamp.
Wiki modules include pages only if they exist. The user module supports common.js
and skin.js. By default neither exists. If a user has both, and then the
less-recently modified one is deleted, the max-timestamp remains unchanged.
For client-side caching, batch requests use "Math.max" on the relevant timestamps.
Again, if a module changes but another module is more recent (e.g. out-of-order
deployment, or out-of-order discovery), the change would not result in a cache miss.
More scenarios can be found in the associated Phabricator tasks.
== Version hash ==
Previously we virtually mapped these variables to a timestamp by storing the current
time alongside a hash of the value in ObjectCache. Considering the number of
possible request contexts (wikis * modules * users * skins * languages) this doesn't
work well. It results in needless cache invalidation when the first time observation
is purged due to LRU algorithms. It also has other minor bugs leading to fewer
cache hits.
All modules automatically get the benefits of version hashing with this change.
The old getDefinitionMtime() and getHashMtime() have been replaced with dummies
that return 1. These functions are often called from getModifiedTime() in subclasses.
For backward-compatibility, their respective values (definition summary and hash)
are now included in getVersionHash directly.
As examples, the following modules have been updated to use getVersionHash directly.
Other modules still work fine and can be updated later.
* ResourceLoaderFileModule
* ResourceLoaderEditToolbarModule
* ResourceLoaderStartUpModule
* ResourceLoaderWikiModule
The presence of hashes in place of timestamps increases the startup module size on
a default MediaWiki install from 4.4k to 5.8k (after gzip and minification).
== ETag ==
Since timestamps are no longer tracked, we need a different way to implement caching
for cache proxies (e.g. Varnish) and web browsers. Previously we used the
Last-Modified header (in combination with Cache-Control and Expires).
Instead of Last-Modified (and If-Modified-Since), we use ETag (and If-None-Match).
Entity tags (new in HTTP/1.1) are much stricter than Last-Modified by default.
They instruct browsers to allow usage of partial Range requests. Since our responses
are dynamically generated, we need to use the Weak version of ETag.
While this sounds bad, it's no different than Last-Modified. As reassured by
RFC 2616 <http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.3.3> the
specified behaviour behind Last-Modified follows the same "Weak" caching logic as
Entity tags. It's just that entity tags are capable of a stricter mode (whereas
Last-Modified is inherently weak).
== File cache ==
If $wgUseFileCache is enabled, ResourceLoader uses ResourceFileCache to cache
load.php responses. While the blind TTL handling (during the allowed expiry period)
is still maxage/timestamp based, tryRespondNotModified() now requires the caller to
know the expected ETag.
For this to work, the FileCache handling had to be moved from the top of
ResoureLoader::respond() to after the expected ETag is computed.
This also allows us to remove the duplicate tryRespondNotModified() handling since
that's is already handled by ResourceLoader::respond() meanwhile.
== Misc ==
* Remove redundant modifiedTime cache in ResourceLoaderFileModule.
* Change bugzilla references to Phabricator.
* Centralised inclusion of wgCacheEpoch using getDefinitionSummary. Previously this
logic was duplicated in each place the modified timestamp was used.
* It's easy to forget calling the parent class in getDefinitionSummary().
Previously this method only tracked 'class' by default. As such, various
extensions hardcoded that one value instead of calling the parent and extending
the array. To better prevent this in the future, getVersionHash() now asserts
that the '_cacheEpoch' property made it through.
* tests: Don't use getDefinitionSummary() as an API.
Fix ResourceLoaderWikiModuleTest to call getPages properly.
* In tests, the default timestamp used to be 1388534400000 (which is the unix time
of 20140101000000; the unit tests' CacheEpoch). The new version hash of these
modules is "XyCC+PSK", which is the base64 encoded prefix of the SHA1 digest of:
'{"_class":"ResourceLoaderTestModule","_cacheEpoch":"20140101000000"}'
* Add sha1.js library for client-side hash generation.
Compared various different implementations for code size (after minfication/gzip),
and speed (when used for short hexidecimal strings).
https://jsperf.com/sha1-implementations
- CryptoJS <https://code.google.com/p/crypto-js/#SHA-1> (min+gzip: 2.5k)
http://crypto-js.googlecode.com/svn/tags/3.1.2/build/rollups/sha1.js
Chrome: 45k, Firefox: 89k, Safari: 92k
- jsSHA <https://github.com/Caligatio/jsSHA>
https://github.com/Caligatio/jsSHA/blob/3c1d4f2e/src/sha1.js (min+gzip: 1.8k)
Chrome: 65k, Firefox: 53k, Safari: 69k
- phpjs-sha1 <https://github.com/kvz/phpjs> (RL min+gzip: 0.8k)
https://github.com/kvz/phpjs/blob/1eaab15d/functions/strings/sha1.js
Chrome: 200k, Firefox: 280k, Safari: 78k
Modern browsers implement the HTML5 Crypto API. However, this API is asynchronous,
only enabled when on HTTPS in Chromium, and is quite low-level. It requires boilerplate
code to actually use with TextEncoder, ArrayBuffer and Uint32Array. Due this being
needed in the module loader, we'd have to load the fallback regardless. Considering
this is not used in a critical path for performance, it's not worth shipping two
implementations for this optimisation.
May also resolve:
* T44094
* T90411
* T94810
Bug: T94074
Change-Id: Ibb292d2416839327d1807a66c78fd96dac0637d0
The patch did not improve performance. I'd like to think that the increased
control over when inline scripts are executed makes the patch worthwhile
regardless, but that is post hoc justification and possibly a bit of personal
ego. Krinkle agrees that we may use some of the ideas in this patch in the
future but he thinks we're better off not heading down this path before we
have a better sense of where we're going, and I trust his judgment.
This reverts commit e86e5f8460.
Change-Id: I151f74a41dd664b5a0aa5cfd99fcc95e2686a1e6
The current ordering of scripts and stylesheets in <head> causes all major
browsers to serialize and defer requests that could be performed in parallel.
The problem is that external stylesheets are loaded before inline scripts. As
Steven Souders explains, "all major browsers preserve the order of CSS and
JavaScript. The stylesheet has to be fully downloaded, parsed, and applied
before the inline script is executed. And the inline script must be executed
before the remaining resources can be downloaded. Therefore, resources that
follow a stylesheet and inline script are blocked from downloading."[1]
In other words: the browser could start loading body images, but it refuses to
do that until it has executed inline scripts in head. And it refuses to execute
those scripts until the external CSS is downloaded, parsed and applied. You can
see the effect of this in this image, showing the request waterfall for
[[en:Gothic Alphabet]]: [2]. Notice how no images were requested before the
browser had finished processing the three load.php requests at the top.
To fix this, we want to move the inline scripts above the external CSS. This is
a little bit tricky, because the inline scripts depend on mw.loader, which is
loaded via an external script. If we move the external script so that it too is
above the external stylesheet, we force the browser to serialize requests,
because the browser will not retrieve the external CSS until it has retrieved
and executed the external JS code. So what we want is to move the inline
scripts above the external stylesheet, but keep the external script (which the
inline scripts depend on) below the external stylesheet.
We can do this by wrapping the inline script code in a closure (which binds
'mw') and enqueuing the closure in a global array which will be processed by
the startup module at just the right time.
Net result: external CSS and JS is retrieved in parallel, retrieval of images
(and other external assets) is unblocked, but the order in which code is
evaluated remains the same.
[1]: <http://www.stevesouders.com/blog/2009/05/06/positioning-inline-scripts/>
[2]: <http://people.wikimedia.org/~ori/enwiki-waterfall.png> (excerpted from
<http://www.webpagetest.org/result/150316_0C_7MB/1/details/>.
Change-Id: I98d383a6299ffbd10210431544a505338ca8643f
Xhprof generates this data now. Custom profiling of various
sub-function units are kept.
Calls to profiler represented about 3% of page execution
time on Special:BlankPage (1.5% in/out); after this change
it's down to about 0.98% of page execution time.
Change-Id: Id9a1dc9d8f80bbd52e42226b724a1e1213d07af7
We already did this, but it was rather convoluted with lots of
if/elseif sequences checking all the possible values.
Remove this logic from ResourceLoaderStartUpModule. Simplying it
simply create the array and pass it to ResourceLoader::makeLoaderRegisterScript.
In makeLoaderRegisterScript, we apply a filter to the array(s) that
trim empty values.
While at it:
* As with other registration properties' default values (like for dependencies,
group, and skip) also use 'null' for the default value of 'source'.
The mediawiki.js client was already compatible with this, and the server omitted
it if it was the last value in the list. But in all other cases it explicitly
outputs "local". Use null instead of simplicity sake. This also gains us a
few characters in the output, and a relatively larger win after gzip since
there's lots more re-using of "null".
* Remove stray casting of $version to int. This only happened in case of
registering a single module (which don't do anywhere), and is redundant.
Change-Id: I1f321e7b8bd3b5cffc550b51169957a3da9b971d
By using the existing indexes of modules in the array being passed to
mw.loader.register we can reduce the size of the startup module by about
6% after gzip (nearly 20% before) on a wiki with very few modules (such
as my localhost). Comparing data from en.wikipedia.org shows about 9%
after gzip (nearly 30% before).
The technique adds a function to mediawiki.js which resolves the indexes
before registering the modules, which costs a little bit of data in that
payload, but it's negligible (118 bytes after gzip) in comparison to the
overall reduction.
Also, cleaned up lies in documentation and strange use of "m" as an
iterator variable.
Bonus: fix ISO8601 timestamp instead of UNIX timestamp being passed
to custom loader scripts.
Change-Id: If12991413fa6129cd20ceab0e59a3a30a4fdf5ce
* Use time() instead of:
- wfTimestamp()
- wfTimestamp( TS_UNIX )
- wfTimestamp( TS_UNIX, 0 )
- wfTimestamp( TS_UNIX, time() )
- intval( wfTimestamp( TS_UNIX, time() ) )
* Consistently use 1 as default instead of 0. Previously the
unwritten convention was that anything "final" max()'ed with 1,
and any internal method would use 0, but this wasn't applied
consistently made it fragile. There doesn't seem to be any
value in returning 0 only to have it maxed up to 1 (because if
the 0 would ever make it out alive, we'd be in trouble).
* wfTimestamp returns a string for TS_UNIX. In PHP this doesn't
matter much. In fact, max() takes number-like integers so
transparently, it even preserves it:
> max( 1, 3, '2' );
< 3
> max( 1, '3', 2 );
< "3"
Just cast it in one place at the very end (StartupModule)
instead of doing intval( wfTimestamp() ).
* Fix weird documentation claiming getModifiedTime can return
an array, or mixed.
* Remove 'version > 1 ? version : 1' logic in
ResourceLoader::makeLoaderRegisterScript. The client doesn't
have "0 means now" behaviour so this isn't needed. And the method
was only doing it for variadic argument calls.
Removal of quotes around timestamps reduced the size of the startup
module from 26.8KB to 25.9KB before gzip. After gzip the size was
and still is 5.7KB, though. (From 5456 bytes to 5415 bytes.)
Change-Id: If92ca3e7511e78fa779f2f2701e2ab24db78c8a8
site-wide
UploadWizard also uses wgFileCanRotate and set the variable by
I3c50b56e6a11742cfc1c02d3d8dbeb0c151b0e91 itself.
Change-Id: Iff6a952c8f4f21c0e27bd6b4191f3b0b453eec03
BitmapHandler has a lot of generic-ish functionality that could
be re-usable by extension classes (Such as how it organizes
$scalerParams array, or various image magick escaping methods).
However it's combined with a lot of very format specific things,
such as the shell-out call to image magick.
Try to separate out the more generic stuff into
TransformationalImageHandler. In order to do this, I also made
canRotate, autoRotateEnabled, and getScalerType non-static. No
extensions in our repo appeared to be using these methods, and they
don't really make sense to be static (imo).
In particular, I think code duplication can be reduced in
PagedTiffHandler by extending this new class. See comments
on I1b9a77a4a56eeb65.
Change-Id: Id3a8b25a598942572cb5791a95e86054d7784961
Prevents modules from depending on raw modules, like 'mediawiki' & 'jquery'.
To do that, those modules (as designated by raw=>true), aren't
registered in the module registry.
Change-Id: Iac37a236964dc40d6259988a0baaaa6331b54ec6
We currently have a few legacy requests to the load.php end point
that bypass the ResourceLoader client by coding a request to
load.php via a "<script src>" directly. Most prominently the
request for the 'site' wiki module (aka MediaWiki:Common.js).
Remove the manual wrapping of embedded private modules as this
is now taken are of by ResourceLoader::makeModuleResponse itself.
Misc:
* Mark "jquery" and "mediawiki" as Raw modules. While the startup
module had this already, these didn't. Without this, they'd
get the conditional wrap – which would be a problem since mediawiki.js
can't be conditional on 'window.mw' for that file defines that
namespace itself.
* Strip the cache-key comment in the unit tests because the hash
no longer matches and using the generic 'wiki' dbname was breaking
DB queries.
* Relates to bug 63587.
* See also 05d0f6fefd which expands the reach of the non-JS
environment to IE6 and helped expose this bug.
Change-Id: Icf6ede09b51ce212aa70ff6be4b341762ec75b4d
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling
Change-Id: I8ebfbcea0e2ae2670553822acedde49c1aa7e98d
- Removed spaces after not operator (!)
- Removed spaces inside array index
- use tab as indent instead of spaces
- Add newline at end of file
- Removed spaces after casts
Change-Id: I9ba17c4385fcb43d38998d45f89cf42952bc791b
Deprecated only since 1.23, but universally reviled, and used only by
MwEmbedSupport, which no longer uses it as of I43af3c87a).
Change-Id: I36c89b273fdb0ec3a76034c7a6d2f48a15d5457f
A module can be registered with a skip function. Such function,
if provided, will be invoked by the client when a module is
queued for loading. If the function returns true, the client will
bypass any further loading action and mark the module as 'ready'.
This can be used to implement a feature test for a module
providing a shim or polyfill.
* Change visibility of method ResourceLoader::filter to public.
So that it can be invoked by ResourceLoaderStartupModule.
* Add option to suppress the cache key report in ResourceLoader::filter.
We usually only call the minifier once on an entire request
reponse (because it's all concatenated javascript or embedded
javascript in various different closures, still valid as one
large script) and only add a little bottom line for the cache
key. When embedding the skip function we have to run the minifier
on them separately as they're output as strings (not actual
functions). These strings are typically quite small and blowing
up the response with loads of cache keys is not desirable in
production.
* Add method to clear the static cache of ResourceLoader::inDebugMode.
Global static state is evil but, as long as we have it, we at
least need to clear it after switching contexts in the test suite.
Also:
* Remove obsolete setting of 'debug=true' in the FauxRequest in
ResourceLoaderTestCase. It already sets global wgResourceLoaderDebug
in the setUp() method.
Bug: 66390
Change-Id: I87a0ea888d791ad39f114380c42e2daeca470961
Provides functonality similar to WebRequest#getCookie and WebResponse#setcookie.
Wraps $.cookie and automatically takes care of wgCookiePrefix etc.
Bug: 49156
Change-Id: I217ef258aecf1acd335e2cea56ae08b22541c7d4
Co-Author: Matthew Flaschen <mflaschen@wikimedia.org>
Co-Author: Timo Tijhof <krinklemail@gmail.com>
This partially reverts r73950 which removed $wgServerName on the ground that it
was only used for {{SERVERNAME}}. When it was pointed out that $wgServerName was
also used by several extensions, the response was not to restore the variable, but
to proceed to remove it from extensions as well.
It is a useful variable to have, as the discussion on Id819246a9 makes clear
(see Tim's comment on PS12 and Timo's reply). So let's reintroduce it, and expose
it in mw.config and ApiQuerySiteInfo as well.
Change-Id: I40a6fd427d38c64c628f70a2f407b145443ea204
The optimization basically works like this:
* Given module A with the dependencies B and C and module B with the
dependency C.
* Don't tell the client that A depends on C, as that's already included
in module B.
This way we can reduce the amount of data for module registration sent
to the client.
The code here isn't polished yet, but it works and should be good enough
to demonstrate my idea and implementation.
Change-Id: I7732a3b1d879c5eef059e136a5241d6d48046872
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Change-Id: Ifbb1da2a6278b0bde2a6f6ce2e7bd383ee3fb28a
In bug 63240, Roan calls it a 'misfeature', which I think is exactly right. It
has been used as a lazy workaround for having to think about load-order,
dependency management, and race conditions.
Bug: 63240
Change-Id: Ic48ad39c6d3d166d6bb8f60dfdd5f95aa7ed16a6
This will make it easier to debug this code and generally to interact
with the registry data before it gets sent out.
Change-Id: I01808ce5d6a0ac6f7a15719bdd2aa90908cb2fbd
