- Removed space after casts
- Removed spaces in array index
- Added spaces around string concat
- Added space after words: switch, foreach
- else if -> elseif
- Removed parentheses around require_once, because it is not a function
- Added newline at end of file
- Removed double spaces
- Added spaces around operations
- Removed repeated newlines
Bug: T102609
Change-Id: Ib860222b24f8ad8e9062cd4dc42ec88dc63fb49e
Centralise the building of module packages into this method so
that it can be easily re-used.
This is in preparation for providing the option for modules to use
content-based hashing (instead of based on meta-data) in the version
string of modules. Having a getModuleContent method allows that to
be implemented in a way that is well-cached (as we'll be calling it
from multiple multiple code paths) and without duplicating this logic.
Bug: T98087
Change-Id: I376233caaabe44b6101565b70a50904abdf8ab4f
* Change "private module denied" level to 'debug'.
This is quite unimportant and occupies over 90% of the logstash
entries for resourceloader.
* Changes all failure signals to 'warning' and using context data.
Left out method in most case since the messages are unambiguous.
Change-Id: Ibb2faaa54020ab0fd9685551689d9a89161e1ce7
Also:
* Update wfHttpError() to use uppercase DOCTYPE, to match other code
such as Html.php, wfThumbError(), HttpError.php, etc.
Change-Id: I4027e7fe1a138b03f78797b6d1bfe7bd1064d360
Graphite expects name components to be dot-separated, so our habit of using
dashes doesn't really make sense. Change metric names to be more compatible
with Graphite, except the job queue's, since that will require a gdash
dashboard definition migration.
Change-Id: I77d0ff7606a8fc88434e4352d23415a9a8f4725a
On HHVM, (a) the cost of a ResourceLoader::filter cache miss is smaller, and
(b) APC is faster and more resilient than PHP's. These two things tilt the
scales in favor of using APC rather than Memcached.
Change-Id: I30b490e5bc815849d4b0bb06f2dbbb68825ed225
As of b1e4006b4, the tokens are different on every request.
Caching these is completely useless because the cache entry is
simply unreachable and is extra overhead on every request for
logged-in users to save content to Memcached.
Whether they should be minified at all and whether they perhaps
shouldn't change on every request is a separate matter.
Bug: T84960
Change-Id: I6016e4b01e44e0acbfd6d49f7d99555e2290c9cb
* Split off OOjs UI's module definitions to a separate file from
Resources.php.
* Extend ResourceLoaderImageModule to support per-skin images and
variants.
* Allow skins to specify their preferred OOjs UI theme using new
skin registry attribute 'SkinOOUIThemes'. The default remains the
'mediawiki' theme, 'apex' can also be chosen now.
* Implement custom ResourceLoaderOOUIImageModule which implements some
special sauce required to shoehorn OOjs UI image set definitions into
ResourceLoaderImageModule with skin-specific styles support.
Bug: T100895
Change-Id: I3fbacbce95126a4cc29221352f3ef846f41e7b1b
Modules now track their version via getVersionHash() instead of getModifiedTime().
== Background ==
While some resources have observeable timestamps (e.g. files stored on disk),
many other resources do not. E.g. config variables, and module definitions.
For static file modules, one can e.g. revert one of more files in a module to a
previous version and not affect the max timestamp.
Wiki modules include pages only if they exist. The user module supports common.js
and skin.js. By default neither exists. If a user has both, and then the
less-recently modified one is deleted, the max-timestamp remains unchanged.
For client-side caching, batch requests use "Math.max" on the relevant timestamps.
Again, if a module changes but another module is more recent (e.g. out-of-order
deployment, or out-of-order discovery), the change would not result in a cache miss.
More scenarios can be found in the associated Phabricator tasks.
== Version hash ==
Previously we virtually mapped these variables to a timestamp by storing the current
time alongside a hash of the value in ObjectCache. Considering the number of
possible request contexts (wikis * modules * users * skins * languages) this doesn't
work well. It results in needless cache invalidation when the first time observation
is purged due to LRU algorithms. It also has other minor bugs leading to fewer
cache hits.
All modules automatically get the benefits of version hashing with this change.
The old getDefinitionMtime() and getHashMtime() have been replaced with dummies
that return 1. These functions are often called from getModifiedTime() in subclasses.
For backward-compatibility, their respective values (definition summary and hash)
are now included in getVersionHash directly.
As examples, the following modules have been updated to use getVersionHash directly.
Other modules still work fine and can be updated later.
* ResourceLoaderFileModule
* ResourceLoaderEditToolbarModule
* ResourceLoaderStartUpModule
* ResourceLoaderWikiModule
The presence of hashes in place of timestamps increases the startup module size on
a default MediaWiki install from 4.4k to 5.8k (after gzip and minification).
== ETag ==
Since timestamps are no longer tracked, we need a different way to implement caching
for cache proxies (e.g. Varnish) and web browsers. Previously we used the
Last-Modified header (in combination with Cache-Control and Expires).
Instead of Last-Modified (and If-Modified-Since), we use ETag (and If-None-Match).
Entity tags (new in HTTP/1.1) are much stricter than Last-Modified by default.
They instruct browsers to allow usage of partial Range requests. Since our responses
are dynamically generated, we need to use the Weak version of ETag.
While this sounds bad, it's no different than Last-Modified. As reassured by
RFC 2616 <http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.3.3> the
specified behaviour behind Last-Modified follows the same "Weak" caching logic as
Entity tags. It's just that entity tags are capable of a stricter mode (whereas
Last-Modified is inherently weak).
== File cache ==
If $wgUseFileCache is enabled, ResourceLoader uses ResourceFileCache to cache
load.php responses. While the blind TTL handling (during the allowed expiry period)
is still maxage/timestamp based, tryRespondNotModified() now requires the caller to
know the expected ETag.
For this to work, the FileCache handling had to be moved from the top of
ResoureLoader::respond() to after the expected ETag is computed.
This also allows us to remove the duplicate tryRespondNotModified() handling since
that's is already handled by ResourceLoader::respond() meanwhile.
== Misc ==
* Remove redundant modifiedTime cache in ResourceLoaderFileModule.
* Change bugzilla references to Phabricator.
* Centralised inclusion of wgCacheEpoch using getDefinitionSummary. Previously this
logic was duplicated in each place the modified timestamp was used.
* It's easy to forget calling the parent class in getDefinitionSummary().
Previously this method only tracked 'class' by default. As such, various
extensions hardcoded that one value instead of calling the parent and extending
the array. To better prevent this in the future, getVersionHash() now asserts
that the '_cacheEpoch' property made it through.
* tests: Don't use getDefinitionSummary() as an API.
Fix ResourceLoaderWikiModuleTest to call getPages properly.
* In tests, the default timestamp used to be 1388534400000 (which is the unix time
of 20140101000000; the unit tests' CacheEpoch). The new version hash of these
modules is "XyCC+PSK", which is the base64 encoded prefix of the SHA1 digest of:
'{"_class":"ResourceLoaderTestModule","_cacheEpoch":"20140101000000"}'
* Add sha1.js library for client-side hash generation.
Compared various different implementations for code size (after minfication/gzip),
and speed (when used for short hexidecimal strings).
https://jsperf.com/sha1-implementations
- CryptoJS <https://code.google.com/p/crypto-js/#SHA-1> (min+gzip: 2.5k)
http://crypto-js.googlecode.com/svn/tags/3.1.2/build/rollups/sha1.js
Chrome: 45k, Firefox: 89k, Safari: 92k
- jsSHA <https://github.com/Caligatio/jsSHA>
https://github.com/Caligatio/jsSHA/blob/3c1d4f2e/src/sha1.js (min+gzip: 1.8k)
Chrome: 65k, Firefox: 53k, Safari: 69k
- phpjs-sha1 <https://github.com/kvz/phpjs> (RL min+gzip: 0.8k)
https://github.com/kvz/phpjs/blob/1eaab15d/functions/strings/sha1.js
Chrome: 200k, Firefox: 280k, Safari: 78k
Modern browsers implement the HTML5 Crypto API. However, this API is asynchronous,
only enabled when on HTTPS in Chromium, and is quite low-level. It requires boilerplate
code to actually use with TextEncoder, ArrayBuffer and Uint32Array. Due this being
needed in the module loader, we'd have to load the fallback regardless. Considering
this is not used in a critical path for performance, it's not worth shipping two
implementations for this optimisation.
May also resolve:
* T44094
* T90411
* T94810
Bug: T94074
Change-Id: Ibb292d2416839327d1807a66c78fd96dac0637d0
Follows-up a4379d8f9e which broke the warning output.
Previously it was appending warnings to a non-existent "$response"
variable and then re-assigning it with the actual response, thus
never outputting this anywhere.
Change-Id: I6c53e7ccb4ae4783d496c06c7174c9d8a1a88b55
Other code paths in this class already make use of this method,
but this was still embedding arbitrary strings directly, at the
risk of producting invalid CSS or invalid JavaScript.
Change-Id: Ied6c420536a3dd1ef8f3816a078705136e8475b1
Fatal error: [] operator not supported for strings in
includes/resourceloader/ResourceLoader.php on line 354
This would occur when a skin customized ResourceModuleSkinStyles using
the '+' syntax, which preserves default skinStyles, and the default
skinStyles was defined as a string rather than array.
Change-Id: I6cd12a638d562c75bc4704e098de84f3882248b6
There was a code path that sent plain 304s without Expires or any
other headers. Fix this by moving the call to sendResponseHeaders()
into tryLastModified().
Bug: 51283
Change-Id: I15d13c5d32102f53bf3e3aaac01d76967e968f78
No skin in gerrit uses the feature of per-module local or remote paths,
and not supporting it will make implementing extension/skin.json support
easier.
Change-Id: I22863ef29426dbacd65439a12aeae56d4d757951
Follows-up ebeb29723, 1f393b6da, 0e719ce23.
Also:
* Add tests for ResourceLoader::makeLoaderImplementScript().
* Apply ResourceLoader::trimArray to makeLoaderImplementScript (new in c0c221bf).
This commit changes the load.php response to omit empty parameters.
These parameters were required until recently. The client has been
updated (1f393b6da and 0e719ce23) to make these optional, thus supporting
both the old server format and the change this commit makes
Clients with a tab open from before 0e719ce23 are naturally not
compatible with load.php responses from after this commit. Ensure
this is deployed several days after 0e719ce23 to reduce race
conditions of this nature.
(This is a re-submitted version of 4ce0c0da4)
Bug: T88879
Change-Id: I9e998261ee9b0b745e3339bc3493755c0cb04b6a
The test for OutputPage::makeResourceLoaderLink was triggering database
queries through MessageBlobStore even though it doesn't use any
messages.
In bb03d1a8e0 I had made MessageBlobStore a singleton instead of static
functions, however there's no need for it to be one since the class is
stateless. Callers can just create a new MessageBlobStore instance and
call functions upon it. Using getInstance() is now deprecated.
ResourceLoader now has a setMessageBlobStore setter to allow overriding
which MessageBlobStore instance will be used. OutputPageTest uses this
to set a NullMessageBlobStore, which makes no database queries.
Change-Id: Ica7436fb6f1ea59bd445b02527829ab0742c0842
The patch did not improve performance. I'd like to think that the increased
control over when inline scripts are executed makes the patch worthwhile
regardless, but that is post hoc justification and possibly a bit of personal
ego. Krinkle agrees that we may use some of the ideas in this patch in the
future but he thinks we're better off not heading down this path before we
have a better sense of where we're going, and I trust his judgment.
This reverts commit e86e5f8460.
Change-Id: I151f74a41dd664b5a0aa5cfd99fcc95e2686a1e6
The current ordering of scripts and stylesheets in <head> causes all major
browsers to serialize and defer requests that could be performed in parallel.
The problem is that external stylesheets are loaded before inline scripts. As
Steven Souders explains, "all major browsers preserve the order of CSS and
JavaScript. The stylesheet has to be fully downloaded, parsed, and applied
before the inline script is executed. And the inline script must be executed
before the remaining resources can be downloaded. Therefore, resources that
follow a stylesheet and inline script are blocked from downloading."[1]
In other words: the browser could start loading body images, but it refuses to
do that until it has executed inline scripts in head. And it refuses to execute
those scripts until the external CSS is downloaded, parsed and applied. You can
see the effect of this in this image, showing the request waterfall for
[[en:Gothic Alphabet]]: [2]. Notice how no images were requested before the
browser had finished processing the three load.php requests at the top.
To fix this, we want to move the inline scripts above the external CSS. This is
a little bit tricky, because the inline scripts depend on mw.loader, which is
loaded via an external script. If we move the external script so that it too is
above the external stylesheet, we force the browser to serialize requests,
because the browser will not retrieve the external CSS until it has retrieved
and executed the external JS code. So what we want is to move the inline
scripts above the external stylesheet, but keep the external script (which the
inline scripts depend on) below the external stylesheet.
We can do this by wrapping the inline script code in a closure (which binds
'mw') and enqueuing the closure in a global array which will be processed by
the startup module at just the right time.
Net result: external CSS and JS is retrieved in parallel, retrieval of images
(and other external assets) is unblocked, but the order in which code is
evaluated remains the same.
[1]: <http://www.stevesouders.com/blog/2009/05/06/positioning-inline-scripts/>
[2]: <http://people.wikimedia.org/~ori/enwiki-waterfall.png> (excerpted from
<http://www.webpagetest.org/result/150316_0C_7MB/1/details/>.
Change-Id: I98d383a6299ffbd10210431544a505338ca8643f
Currently if code wants to check whether a module is registered it
has to call getModule() and see if the response !== null.
Change-Id: I4b470083ddaa5d8cd6be50d5c5b690d4b99b6c4a
Add new hook (ResourceLoaderGetLessVars) called in ResourceLoader::getLessVars to
allow context-based less variables. Cache the resulting array to avoid multiple runs
of this hook.
Change-Id: I5a73bbd0ab58f8fe34519931c4f26c90998e3451
Works as intended, but didn't account for the first implement() parameter
being omitted client-side. Revert until that is accounted for, then re-try after
that fix is rolled out for > 1 week.
This reverts commit 4ce0c0da42.
Change-Id: I36c1619991663c0303636d1d3f037b0021ac79bf
Follows-up ebeb297236.
Also:
* Add tests for ResourceLoader::makeLoaderImplementScript().
* Apply ResourceLoader::trimArray to makeLoaderImplementScript (new in c0c221bf).
As always, the client (updated in Ie32e7d6a3c) is backward-compatible with old
(cached) load.php module responses. However, the old client is not compatible
new load.php responses after this commit.
That's generally not an issue, as we don't cache the client very long (~ 5 min).
However people with their browser open and mw.loader clients initialised can
still make new module requests (e.g. modules loaded on-demand, such as when
previewing edits, clicking buttons etc.). That can easily be several hours after
initial page load. As such, client/server bound changes should always be
back-compat and deployed a reasonable time apart to reduce chances of active
sessions making subsequent requests. Ideally we'd find some solution to this in
the long-term, but handling this at all is better than what we usually do...
For deployment: Ensure this is deployed several days after Ie32e7d6a3c09f.
Change-Id: Ic8d7efe49b5d45e3f95a2f04e3a26a014b10af16
Xhprof generates this data now. Custom profiling of various
sub-function units are kept.
Calls to profiler represented about 3% of page execution
time on Special:BlankPage (1.5% in/out); after this change
it's down to about 0.98% of page execution time.
Change-Id: Id9a1dc9d8f80bbd52e42226b724a1e1213d07af7
They are prone to being stripped by CSS and JS minification and can't
be used for non-CSS non-JS responses, like images.
We've already been keeping some state related to errors in the
$hasErrors member variable, let's go all the way and replace it with
$errors array, which tracks all errors accumulated during current
respond() call. When processing completes, all errors are added to the
response, if possible.
Change-Id: I6643f010174cb1f17780622e8a63db03cffe19e1
ResourceLoaderImageModule needs a set of SVG files and some data in
the module definition, and produces styles for a set of CSS classes,
one for each image, optionally with differently colored variants,
generated in SVG and PNG, data-URI-embedded if possible, compatible
with all browsers, and generally slick.
The intended usage is to ship icon libraries with MediaWiki that can
be used throughout the pages with no additional code.
* ResourceLoaderImageModule implements all of the logic for data
parsing and CSS generation.
* ResourceLoaderImage implements the logic for SVG image colorization
(for variants) and rasterization.
* ResourceLoader and ResourceLoaderContext were extended to serve a
new kind of load.php request that delivers a single image file. This
is used for fallback PNG images served to browsers that don't
understand SVG.
See change Ic6a76bfb for a demo.
Bug: T76473
Co-Authored-By: Trevor Parscal <trevorparscal@gmail.com>
Co-Authored-By: Bartosz Dziewoński <matma.rex@gmail.com>
Change-Id: Idf6ff4eb8e94f45946f15d283d34108b881fae6e
We already did this, but it was rather convoluted with lots of
if/elseif sequences checking all the possible values.
Remove this logic from ResourceLoaderStartUpModule. Simplying it
simply create the array and pass it to ResourceLoader::makeLoaderRegisterScript.
In makeLoaderRegisterScript, we apply a filter to the array(s) that
trim empty values.
While at it:
* As with other registration properties' default values (like for dependencies,
group, and skip) also use 'null' for the default value of 'source'.
The mediawiki.js client was already compatible with this, and the server omitted
it if it was the last value in the list. But in all other cases it explicitly
outputs "local". Use null instead of simplicity sake. This also gains us a
few characters in the output, and a relatively larger win after gzip since
there's lots more re-using of "null".
* Remove stray casting of $version to int. This only happened in case of
registering a single module (which don't do anywhere), and is redundant.
Change-Id: I1f321e7b8bd3b5cffc550b51169957a3da9b971d
By using the existing indexes of modules in the array being passed to
mw.loader.register we can reduce the size of the startup module by about
6% after gzip (nearly 20% before) on a wiki with very few modules (such
as my localhost). Comparing data from en.wikipedia.org shows about 9%
after gzip (nearly 30% before).
The technique adds a function to mediawiki.js which resolves the indexes
before registering the modules, which costs a little bit of data in that
payload, but it's negligible (118 bytes after gzip) in comparison to the
overall reduction.
Also, cleaned up lies in documentation and strange use of "m" as an
iterator variable.
Bonus: fix ISO8601 timestamp instead of UNIX timestamp being passed
to custom loader scripts.
Change-Id: If12991413fa6129cd20ceab0e59a3a30a4fdf5ce
* Use time() instead of:
- wfTimestamp()
- wfTimestamp( TS_UNIX )
- wfTimestamp( TS_UNIX, 0 )
- wfTimestamp( TS_UNIX, time() )
- intval( wfTimestamp( TS_UNIX, time() ) )
* Consistently use 1 as default instead of 0. Previously the
unwritten convention was that anything "final" max()'ed with 1,
and any internal method would use 0, but this wasn't applied
consistently made it fragile. There doesn't seem to be any
value in returning 0 only to have it maxed up to 1 (because if
the 0 would ever make it out alive, we'd be in trouble).
* wfTimestamp returns a string for TS_UNIX. In PHP this doesn't
matter much. In fact, max() takes number-like integers so
transparently, it even preserves it:
> max( 1, 3, '2' );
< 3
> max( 1, '3', 2 );
< "3"
Just cast it in one place at the very end (StartupModule)
instead of doing intval( wfTimestamp() ).
* Fix weird documentation claiming getModifiedTime can return
an array, or mixed.
* Remove 'version > 1 ? version : 1' logic in
ResourceLoader::makeLoaderRegisterScript. The client doesn't
have "0 means now" behaviour so this isn't needed. And the method
was only doing it for variadic argument calls.
Removal of quotes around timestamps reduced the size of the startup
module from 26.8KB to 25.9KB before gzip. After gzip the size was
and still is 5.7KB, though. (From 5456 bytes to 5415 bytes.)
Change-Id: If92ca3e7511e78fa779f2f2701e2ab24db78c8a8
A base ResourceLoaderModule::getTemplates() exists for subclasses
to override. An implementation is provided for ResourceLoaderFileModule.
For file modules, templates can be specified in the following manner:
'example' => array(
'templates' => array(
'bar' => 'templates/foo.html',
),
'scripts' => 'example.js',
),
The delivery system is template language agnostic, and currently
only supports "compiling" plain HTML templates.
This also adds template support to the following modules as a POC:
* mediawiki.feedback
* mediawiki.action.view.postEdit
* mediawiki.special.upload
Works with $wgResourceLoaderStorageEnabled
Change-Id: Ia0c5c8ec960aa6dff12c9626cee41ae9a3286b76
Not ready for merging, and Roan says that the +2 was
most likely accidental and meant to be a -1.
This reverts commit d146934f94.
Change-Id: I3926c9ae9e3c8026fceb3aeedd3b1f1d9b91667b
Preparation work for templating in core.
RL should allow us to ship HTML / template markup from server to client.
Use in Special:Upload and mediawiki.feedback as a proof of concept.
Separation of concerns etc...
See Also:
Ia63d6b6868f23a773e4a41daa0036d4bf2cd6724
Change-Id: I6ff38c12897e3164969a1090449e626001926c3b
Follows-up 9272bc6c47, 03c503da22, 1e063f6078.
One can't wrap arbitrary JavaScript in an if-statement and have
its inner-body mean exactly the same.
Certain statements are only allowed in the top of a scope (such
as hoisted function declarations). These are not allowed inside
a block. They're fine in both global scope and local function
scope, but not inside an if-block of any scope.
The ECMAScript spec only describes what is an allowed token.
Any unexpected token should result in a SyntaxError.
Chrome's implementation (V8) allows function declarations in
blocks and hoists them to *outside* the condition. Firefox's
SpiderMonkey silently ignores the statement. Neither throw a
SyntaxError.
Rgular ResourceLoader responses only contain mw.loader.implement()
and mw.loader.state() call which could be wrapped without issues.
However such responses don't need wrapping as they're only made
by mediawiki.js (in which case mw is obviously loaded). The
wrapping is for legacy scripts that execute in the global scope.
For those, let's wrap the script tag itself (instead of the
response). That seems like the most water-tight and semantically
correct solution.
Had to bring in $isRaw from ResourceLoader.php, else the startup
module would have been wrapped as well (added regression test).
Bug: 69924
Change-Id: Iedda0464f734ba5f7a884726487f6c7e07d444f1
Previously ResourceLoader would store any arbitrary data about a
source, provided it had a 'loadScript' key. It would register
the 'local' source with an additional 'apiScript' key, which was
also documented in DefaultSettings.php. However, it was
completely unused outside of the ForeignAPIGadgetRepo class in
Gadgets 2.0, which should be changed to take an API url as a
parameter. This was not useful as it was not ever formally
exposed, and it could not be depended upon that a source had
registered an 'apiScript' key.
For backwards compatability, both ResourceLoader::addSource()
and mw.loader.addSource() will both take an array/object, but
discard all parameters except for 'loadScript'.
Also added tests for ResourceLoader::addSource().
Bug: 69878
Change-Id: I4205cf788cddeec13b619be0c3576197dec1b8bf
There was only one warning left.
> 1184 | WARNING | Line exceeds 100 characters; contains 107 characters
> | | (Generic.Files.LineLength.TooLong)
Change-Id: I99c729e02dac6aaf13a05568fd04e7bb5ad7bcb5
For easier testability and other things. There are no uses
of this class in any extensions in gerrit.
Change-Id: I606de4259239e128ed7e0477fc98b84c647430c4
