Commit graph

119 commits

Author SHA1 Message Date
Timo Tijhof
28f6d7fbde resourceloader: Convert FileModule to use version hashing
Enabling the module content versionining is not feasible for FileModule as that
would involve Lessc and CSSJanus just to compute the version hash.

Instead, we can keep the existing logic that exists for the timestamp-based
versioning (which already has a comprehensive grip on tracking all involved
factors that cause a module to change) and convert it to use hashing instead.

This way the version hashes will be deterministic. Currently module versions
tend to be invalidated too often (and sometimes not often enough) due to Git and
other transport mechanisms not preserving file timestamps.

== Research ==

* <https://phabricator.wikimedia.org/T98087#1412712>
  It'd take upto 10 seconds to run startup if FileModule enables build versioning.

* <https://phabricator.wikimedia.org/T104950#1433142>
  Checking all files in resources/** took only ~30m longer in total with
  sha1_file compared to filemtime.

Bug: T104950
Change-Id: I732fa4db32258c634e32b507952f76eac7fc9395
2015-08-05 19:30:33 +00:00
Timo Tijhof
2581453fac resourceloader: Enable module content version for data modules
This greatly simplifies logic required to compute module versions.
It also makes it significantly less error-prone.

Since f37cee996e, we support hashes as versions (instead of timestamps).
This means we can build a hash of the content directly, instead of compiling a
large array with all values that may influence the module content somehow.

Benefits:
* Remove all methods and logic related to querying database and disk for
  timestamps, revision numbers, definition summaries, cache epochs, and more.

* No longer needlessly invalidate cache as a result of no-op changes to
  implementation datails. Due to inclusion of absolute file paths in the
  definition summary, cache was always invalidated when moving wikis to newer
  MediaWiki branches; even if the module observed no actual changes.

* When changes are reverted within a certain period of time, old caches can now
  be re-used. The module would produce the same version hash as before.
  Previously when a change was deployed and then reverted, all web clients (even
  those that never saw the bad version) would have re-fetch modules because the
  version increased.

Updated unit tests to account for the change in version. New default version of
empty test modules is: "mvgTPvXh". For the record, this comes from the base64
encoding of the SHA1 digest of the JSON serialised form of the module content:
> $str = '{"scripts":"","styles":{"css":[]},"messagesBlob":"{}"}';
> echo base64_encode(sha1($str, true));
> FEb3+VuiUm/fOMfod1bjw/te+AQ=

Enabled content versioning for the data modules in MediaWiki core:
* EditToolbarModule
* JqueryMsgModule
* LanguageDataModule
* LanguageNamesModule
* SpecialCharacterDataModule
* UserCSSPrefsModule
* UserDefaultsModule
* UserOptionsModule

The FileModule and base class explicitly disable it for now and keep their
current behaviour of using the definition summary. We may remove it later, but
that requires more performance testing first.

Explicitly disable it in the WikiModule class to avoid breakage when the
default changes.

Ref T98087.

Change-Id: I782df43c50dfcfb7d7592f744e13a3a0430b0dc6
2015-06-18 20:39:38 +00:00
Andrew Green
7e0d3369c2 resourceloader: Add context param to ResourceLoaderModule::getDependencies
By providing context as a parameter in getDependencies, we allow
modules to dyanamically determine dependencies based on context.
Note: To ease rollout, the parameter is optional in this patch. It is expected
that it will be made non-optional in the near future.

The use case is for CentralNotice campaigns to be able to add special
modules ahead of deciding which banner to show a user. The dynamically
chosen RL modules would replace ad-hoc JS currently sent with some banners.
A list of possible campaigns and banners is already sent as a PHP-
implemented RL module; that's the module that will dynamically choose other
modules as dependencies when appropriate. This approach will save a round
trip as compared to dynamically loading the modules client-side.

For compatibility, extensions that override
ResourceLoaderModule::getDependencies() should be updated with the new
method signature. Here are changes for extensions currently deployed on
Wikimedia wikis:

* CentralNotice: I816bffa3815e2eab7e88cb04d1b345070e6aa15f
* Gadgets: I0a10fb0cbf17d095ece493e744296caf13dcee02
* EventLogging: I67e957f74d6ca48cfb9a41fb5144bcc78f885e50
* PageTriage: Ica3ba32aa2fc76d11a44f391b6edfc871e7fbe0d
* UniversalLanguageSelector: Ic63e617f51702c27104e123d4bed91983a726b7f
* VisualEditor: I0ac775ca286e64825e31a9213b94648e41a5bc30

For more on the CentralNotice use case, please see I9f80edcbcacca2.

Bug: T98924
Change-Id: Iee61e5b527321d01287baa03ad9b4d4f526ff3ef
2015-06-09 03:10:20 +01:00
Gilles Dubuc
d6b4d3c537 Add ability to load plain CSS files at the bottom
Currently all styles modules added to the page using
addModuleStyles are put into the head, regardless
of their "position" value.

Bug: T97420
Change-Id: Ie4287e17d6f298cc63f42f257b1f67ee36961b77
2015-05-25 21:10:14 +00:00
Timo Tijhof
f37cee996e resourceloader: Replace timestamp system with version hashing
Modules now track their version via getVersionHash() instead of getModifiedTime().

== Background ==

While some resources have observeable timestamps (e.g. files stored on disk),
many other resources do not. E.g. config variables, and module definitions.

For static file modules, one can e.g. revert one of more files in a module to a
previous version and not affect the max timestamp.

Wiki modules include pages only if they exist. The user module supports common.js
and skin.js. By default neither exists. If a user has both, and then the
less-recently modified one is deleted, the max-timestamp remains unchanged.

For client-side caching, batch requests use "Math.max" on the relevant timestamps.
Again, if a module changes but another module is more recent (e.g. out-of-order
deployment, or out-of-order discovery), the change would not result in a cache miss.

More scenarios can be found in the associated Phabricator tasks.

== Version hash ==

Previously we virtually mapped these variables to a timestamp by storing the current
time alongside a hash of the value in ObjectCache. Considering the number of
possible request contexts (wikis * modules * users * skins * languages) this doesn't
work well. It results in needless cache invalidation when the first time observation
is purged due to LRU algorithms. It also has other minor bugs leading to fewer
cache hits.

All modules automatically get the benefits of version hashing with this change.
The old getDefinitionMtime() and getHashMtime() have been replaced with dummies
that return 1. These functions are often called from getModifiedTime() in subclasses.

For backward-compatibility, their respective values (definition summary and hash)
are now included in getVersionHash directly.

As examples, the following modules have been updated to use getVersionHash directly.
Other modules still work fine and can be updated later.

* ResourceLoaderFileModule
* ResourceLoaderEditToolbarModule
* ResourceLoaderStartUpModule
* ResourceLoaderWikiModule

The presence of hashes in place of timestamps increases the startup module size on
a default MediaWiki install from 4.4k to 5.8k (after gzip and minification).

== ETag ==

Since timestamps are no longer tracked, we need a different way to implement caching
for cache proxies (e.g. Varnish) and web browsers. Previously we used the
Last-Modified header (in combination with Cache-Control and Expires).

Instead of Last-Modified (and If-Modified-Since), we use ETag (and If-None-Match).

Entity tags (new in HTTP/1.1) are much stricter than Last-Modified by default.
They instruct browsers to allow usage of partial Range requests. Since our responses
are dynamically generated, we need to use the Weak version of ETag.

While this sounds bad, it's no different than Last-Modified. As reassured by
RFC 2616 <http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.3.3> the
specified behaviour behind Last-Modified follows the same "Weak" caching logic as
Entity tags. It's just that entity tags are capable of a stricter mode (whereas
Last-Modified is inherently weak).

== File cache ==

If $wgUseFileCache is enabled, ResourceLoader uses ResourceFileCache to cache
load.php responses. While the blind TTL handling (during the allowed expiry period)
is still maxage/timestamp based, tryRespondNotModified() now requires the caller to
know the expected ETag.

For this to work, the FileCache handling had to be moved from the top of
ResoureLoader::respond() to after the expected ETag is computed.

This also allows us to remove the duplicate tryRespondNotModified() handling since
that's is already handled by ResourceLoader::respond() meanwhile.

== Misc ==

* Remove redundant modifiedTime cache in ResourceLoaderFileModule.

* Change bugzilla references to Phabricator.

* Centralised inclusion of wgCacheEpoch using getDefinitionSummary. Previously this
  logic was duplicated in each place the modified timestamp was used.

* It's easy to forget calling the parent class in getDefinitionSummary().
  Previously this method only tracked 'class' by default. As such, various
  extensions hardcoded that one value instead of calling the parent and extending
  the array. To better prevent this in the future, getVersionHash() now asserts
  that the '_cacheEpoch' property made it through.

* tests: Don't use getDefinitionSummary() as an API.
  Fix ResourceLoaderWikiModuleTest to call getPages properly.

* In tests, the default timestamp used to be 1388534400000 (which is the unix time
  of 20140101000000; the unit tests' CacheEpoch). The new version hash of these
  modules is "XyCC+PSK", which is the base64 encoded prefix of the SHA1 digest of:
  '{"_class":"ResourceLoaderTestModule","_cacheEpoch":"20140101000000"}'

* Add sha1.js library for client-side hash generation.
  Compared various different implementations for code size (after minfication/gzip),
  and speed (when used for short hexidecimal strings).
  https://jsperf.com/sha1-implementations
  - CryptoJS <https://code.google.com/p/crypto-js/#SHA-1> (min+gzip: 2.5k)
    http://crypto-js.googlecode.com/svn/tags/3.1.2/build/rollups/sha1.js
    Chrome: 45k, Firefox: 89k, Safari: 92k
  - jsSHA <https://github.com/Caligatio/jsSHA>
    https://github.com/Caligatio/jsSHA/blob/3c1d4f2e/src/sha1.js (min+gzip: 1.8k)
    Chrome: 65k, Firefox: 53k, Safari: 69k
  - phpjs-sha1 <https://github.com/kvz/phpjs> (RL min+gzip: 0.8k)
    https://github.com/kvz/phpjs/blob/1eaab15d/functions/strings/sha1.js
    Chrome: 200k, Firefox: 280k, Safari: 78k

  Modern browsers implement the HTML5 Crypto API. However, this API is asynchronous,
  only enabled when on HTTPS in Chromium, and is quite low-level. It requires boilerplate
  code to actually use with TextEncoder, ArrayBuffer and Uint32Array. Due this being
  needed in the module loader, we'd have to load the fallback regardless. Considering
  this is not used in a critical path for performance, it's not worth shipping two
  implementations for this optimisation.

May also resolve:
* T44094
* T90411
* T94810

Bug: T94074
Change-Id: Ibb292d2416839327d1807a66c78fd96dac0637d0
2015-05-19 22:28:17 +00:00
Timo Tijhof
58ecbdb61b resourceloader: De-duplicate $files in ResourceLoaderFileModule::getModifiedTime()
lessc::allParsedFiles() always includes the main file we already
know about. As such needlessly causing it to be queried multiple
times.

Change-Id: Icd2c11d93d6491664fcf58a18bfe6e2932689774
2015-03-31 19:28:44 +01:00
Bartosz Dziewoński
7aeadb7f68 resourceloader: Throw InvalidArgumentException for invalid constructor arguments
(But not for exceptions about missing files.)

Per Legoktm's comments on 8edbfb5feb.

Change-Id: I725f846476d2d97d3d820bc22674f7b5aab812bb
2015-03-30 19:15:36 +02:00
jdlrobson
7c52c9265b resourceloader: Add template compiler for Mustache JS
Compatible library for PHP backend added by I281acc49c1.
This adds the frontend compiler.

Change-Id: I2190276a629ca882f3f5535e4e73635d2494e944
2015-03-23 11:10:05 -07:00
Timo Tijhof
1fc5803a78 resourceloader: Use parent getDefinitionSummary() in ResourceLoaderFileModule
Right now ResourceLoaderModule::getDefinitionSummary just returns an array
with the class name, but may be extended in the future. Either way, it shouldn't
be repeated in the subclass.

Change-Id: Ic1d58f8d330fb2176daffc7ad39bc0751c196181
2015-03-23 04:09:13 +00:00
Aaron Schulz
4ff8136807 Removed remaining profile calls
Change-Id: I31c81c78715048004fc8fca0f27d09c1fa71c118
2015-01-08 02:49:33 -08:00
Chad Horohoe
aa21e125a3 Remove obvious function-level profiling
Xhprof generates this data now. Custom profiling of various
sub-function units are kept.

Calls to profiler represented about 3% of page execution
time on Special:BlankPage (1.5% in/out); after this change
it's down to about 0.98% of page execution time.

Change-Id: Id9a1dc9d8f80bbd52e42226b724a1e1213d07af7
2015-01-07 11:14:24 -08:00
Reedy
4d9143c7f5 Add lots of @throws
Change-Id: I09d0c13070f966fcf23d2638d8fc1328279a5995
2014-12-24 13:49:20 +00:00
umherirrender
2a7c95acda Fixed spacing
- Changed spaces to tabs for indentation
- space after 'function'/'if'
- Added/Removed space after parenthesis/brackets/end of line
- Removed space after cast

Change-Id: I0e8e6a19b84b5e1308b632a0266cb78f688494ee
2014-10-30 17:50:19 +01:00
jdlrobson
ebeb297236 resourceloader: Add support for delivering templates
A base ResourceLoaderModule::getTemplates() exists for subclasses
to override. An implementation is provided for ResourceLoaderFileModule.

For file modules, templates can be specified in the following manner:

'example' => array(
	'templates' => array(
		'bar' => 'templates/foo.html',
	),
	'scripts' => 'example.js',
),

The delivery system is template language agnostic, and currently
only supports "compiling" plain HTML templates.

This also adds template support to the following modules as a POC:
* mediawiki.feedback
* mediawiki.action.view.postEdit
* mediawiki.special.upload

Works with $wgResourceLoaderStorageEnabled

Change-Id: Ia0c5c8ec960aa6dff12c9626cee41ae9a3286b76
2014-10-29 19:31:16 +00:00
Legoktm
de24e3099e Revert "Add RL template module with HTML markup language"
Not ready for merging, and Roan says that the +2 was
most likely accidental and meant to be a -1.

This reverts commit d146934f94.

Change-Id: I3926c9ae9e3c8026fceb3aeedd3b1f1d9b91667b
2014-10-17 23:19:21 +00:00
jdlrobson
d146934f94 Add RL template module with HTML markup language
Preparation work for templating in core.
RL should allow us to ship HTML / template markup from server to client.
Use in Special:Upload and mediawiki.feedback as a proof of concept.
Separation of concerns etc...

See Also:
Ia63d6b6868f23a773e4a41daa0036d4bf2cd6724

Change-Id: I6ff38c12897e3164969a1090449e626001926c3b
2014-10-17 09:23:04 -07:00
Kunal Mehta
c6289d013e ResourceLoaderFileModule: Support fallback in 'languageScripts'
Bug: 58139
Change-Id: Idd7657aa48a9eb2b075a85f77e066b425f330890
2014-09-25 10:51:16 -07:00
Timo Tijhof
eaac07ac27 resourceloader: Clean up $localBasePath variable assignment
No need to set a variable to value of itself. Also consistent
with the way $remoteBasePath is set, already.

Change-Id: I9b71322592a5ca2d85927fcf6732096f758847e9
2014-09-25 01:32:45 +00:00
Bartosz Dziewoński
4c01f8b2bc Make "/*@noflip*/ /*@embed*/" annotation work without CSSJanus hacks
This reverts most of commit 2d842f1425,
leaving only the test added in it, and reimplements the same
functionality better.

Instead of stripping /*@noflip*/ annotations in CSSJanus, which is
incompatible with other implementations that preserve it, extend
CSSMin to allow other CSS comments to be present before the
rule-global @embed annotation. (This required making the regex logic
in it even worse than it was, but it's actually slightly less terrible
than I expected it would be. Good thing we have tests!)

Bug: 69698
Change-Id: I58603ef64f7d7cdc6461b34721a4d6b15f15ad79
2014-09-23 22:47:54 +00:00
Roan Kattouw
ef26c7ad4f Followup 1cf5a6e: use $wgResourceBasePath to refer to resources/assets, not $wgScriptPath
Bonus: actually make $wgResourceBasePath default to $wgScriptPath, rather than
special-casing it in ResourceLoaderFileModule.

Change-Id: I608435cef00d3e77a5bbdb0a0122d3e7e1a4eb78
2014-09-17 16:21:28 -07:00
Bartosz Dziewoński
285c52039b Revamp classic edit toolbar not to hardcode paths in HTML
Also, try out a way to have per-module LESS variables defined in PHP.
This might come in handy in the future… Maybe for skin theme support?

(I recommend reviewing the file changes in the order below. :D)

includes/resourceloader/ResourceLoaderFileModule.php
  * Pass the context (ResourceLoaderContext) deeper down via
    readStyleFiles() and readStyleFile(). We need it to compile the
    .less files for the right language.
  * Extract LESS compiler creation to getLessCompiler().
  * Allow passing a LESS compiler instance to compileLessFile(), rather
    than getting one after the method is called.

  All of the changes are backwards-compatible.

includes/resourceloader/ResourceLoaderEditToolbarModule.php
  * New module to support getting the language data and passing it to
    LESS variables.

  It might be a good idea to factor out a reusable class for a LESS
  module with additional variables, but that would require more
  attention to design than I gave it.

resources/src/mediawiki.action/mediawiki.action.edit.toolbar/mediawiki.action.edit.toolbar.less
  * Glue code to use the language data defined by the module above and
    put it in final CSS.

includes/EditPage.php
  * Do not hardcode image URLs in output HTML, as they are provided in
    CSS now. This gets rid of some usage of globals.

  In fact, we should be able to finally move the inline JavaScript
  calls out of getEditToolbar(), but I'm already introducing too many
  changes for one patch. That can be done later.

languages/Language.php
  * Add getImageFiles() to complement existing getImageFile() method.
    Misleadingly named, it returns paths for images for the toolbar
    only (and no other ones at all).

skins/common/ → resources/src/mediawiki.action/mediawiki.action.edit.toolbar/
  * Moved all of the button images to new location.

  Also, boring cleanup that was harder before because we treated the
  paths as public API:
  * Placed default ones in en/ subdirectory.
  * Renamed cyrl/ to ru/.
  * Renamed ksh/button_S_italic.png → ksh/button_italic.png.

languages/messages/
  * Adjusting paths and filenames for the changes above.

resources/src/mediawiki.action/mediawiki.action.edit.css
resources/src/mediawiki.action/mediawiki.action.edit.js
  * Added styles and updated the script to make it possible to have
    non-<img> elements as toolbar buttons.
  * Consolidated styles that were already required, but defined
    somewhere else:
    * `cursor: pointer;` (from shared.css)
    * `vertical-align: middle;` (from commonElements.css)

Bug: 69277
Change-Id: I39d8ed4258c7da0fe4fe4c665cdb26c86420769c
2014-09-13 23:16:23 +02:00
jenkins-bot
08e50bcecd Merge "Make "/*@noflip*/ /*@embed*/" annotation work" 2014-08-29 19:45:06 +00:00
Bartosz Dziewoński
1538edce7b ResourceLoaderFileModule: Do not separately cache .less files
We used to have a second level of caching for them – when a module's
cache was invalidated, individual .less files that comprise it would
only be recompiled (with all their @imports) if their mktime changed
(or if $wgResourceLoaderLESSVars changed).

Given that practice has shown that RL modules including .less files
usually just include one file that @imports tons of stuff, and given
that RL already is very careful about invalidating caches, I think
this is unnecessary at best and harmful at worst.

The behavior proved problematic when I tried to implement support for
per-module LESS variables, values of which were language-dependent –
files would not be recompiled for different languages, and
getLessCacheKey() is called several levels deep in code that doesn't
have access to the context in which the module is being compiled.

Change-Id: I667f063507583e4cd3a669e32398d440f1431f7e
2014-08-20 20:26:46 +02:00
Bartosz Dziewoński
cd22ba020e Consistently use 'Less' rather than 'LESS' in function names
PHP function names are case-insensitive, so this is a fully
backwards-compatible change.

Change-Id: Ide04ad542ac5c3a26b6064093ae272cf9aeef2d1
2014-08-20 18:48:10 +02:00
Bartosz Dziewoński
2d842f1425 Make "/*@noflip*/ /*@embed*/" annotation work
To do it, just remove /*@noflip*/ annotations in CSSJanus after
we're done processing. They are not needed anymore and some obscure
interactions with CSSMin logic for preserving comments caused
`/*@noflip*/ /*@embed*/ background-image: url(…)` not to work
correctly (it would not be embedded).

This also requires us to always do CSSJanus processing, even when we
don't need flipping, to consistently handle the annotations.
I'm not entirely sure if this is worth it, but I still greatly prefer
doing it to documenting this stupid limitation. :)

Bug: 69698
Change-Id: I311b12b08b2dff9d45efb584db08cf4a11318f59
2014-08-18 17:40:51 +02:00
umherirrender
c2de24efcd Add missing @param to function docs
Change-Id: Ib7ac94d05a04490f25dfd40b46b27973cbab582c
2014-08-14 19:38:57 +00:00
Alexandre Emsenhuber
cc03986ca1 resourceloader: Fix undefined variable in ResourceLoaderFileModule::getSkipFunction
Follows-up 75c08916b0. Looking at the call chain of
validateScriptFile() up to JSTokenzier, the first parameter is the file
name, otherwise "[inline]" is used; so that variable should be $localPath.

Bug: 69214
Change-Id: If7f36449cb352f50ba795a6d306e5d949a3dbd29
2014-08-12 22:25:20 +00:00
jenkins-bot
2268ae08a0 Merge "includes/resourceloader/: Use Config instead of globals" 2014-08-08 17:25:42 +00:00
Kunal Mehta
4b1a87c96e includes/resourceloader/: Use Config instead of globals
Change-Id: Iaa1f4ae9c397575c9bb86dacbc342456e6f5f532
2014-08-07 18:47:34 +01:00
Kunal Mehta
0d051f6a4f ResourceLoaderFileModule::extractBasePaths: Simplify if condition
Change-Id: I69f090595629dabefaa0c6ddaa9a793a2f86ecfd
2014-08-07 11:27:36 +01:00
umherirrender
473b7d925e Fixed docs
- Use short form of boolean
- Use capital at begin of doc text

Change-Id: Ic5afacfa7298b1938d3b45ffd0cac5ce01f2f9db
2014-08-04 12:00:15 +02:00
Bartosz Dziewoński
3971d0646c resourceloader: Allow skins to provide additional styles for any module
The newly introduced $wgResourceModuleSkinStyles global enables skins to
provide additional stylesheets to existing ResourceLoader module.

This both makes it easier (or at all possible) to override default
styles and lowers the style footprint by making it possible not to
load styles unused on most pages.

----

Example:

Use the file 'foo-styles.css' for the 'mediawiki.foo' module when using
the MySkin skin:

  $wgResourceModuleSkinStyles['myskin'] = array(
    'mediawiki.foo' => 'foo-styles.css',
    'remoteSkinPath' => 'MySkin',
    'localBasePath' => __DIR__,
  );

For detailed documentation, see the doc comment in DefaultSettings.php.
For a practical usage example, see Vector.php.

----

Implementation notes:

* The values defined in $wgResourceModuleSkinStyles are embedded into
  the modules as late as possible (in ResourceLoader::register()).
* Only plain file modules are supported, setting module skin styles
  for other module types has no effect.
* ResourceLoader and ResourceLoaderFileModule now support loading
  files from arbitrary paths to make this possible, defined using
  ResourceLoaderFilePath objects.
  * This required some adjustments in seemingly unrelated places for
    code which didn't handle the paths fully correctly before.
* ResourceLoader and ResourceLoaderFileModule are now a bit more
  tightly coupled than before :(
* Included a tiny example change for the Vector skin, a lot more of
  similar cleanup is possible and planned for the future.
* Many of the non-essential mediawiki.* modules defined in
  Resources.php should be using `'skinStyles' => array( 'default' => … )`
  instead of `'styles' => …` to allow more customizations, this is
  also planned for the future after auditing which ones would actually
  benefit from this.

Change-Id: Ica4ff9696b490e35f60288d7ce1295766c427e87
2014-07-29 00:53:41 +02:00
umherirrender
dd8921c9d9 Cleanup some docs (includes/[m-r])
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling

Change-Id: I8ebfbcea0e2ae2670553822acedde49c1aa7e98d
2014-07-24 19:43:25 +02:00
Sam Smith
0bb3d2a499 Make ResourceLoaderFileModule#getAllStyleFiles include all skin styles
* Add the ResourceLoaderFileModule#getAllSkinStyleFiles method,
  which returns all of the skinStyles files for a given module
* The LessFileCompilationTest and checkLess.php script, which use
  the #getAllStyleFile method, now validate all LESS style files

Bug: 63343
Change-Id: Ib2eb5761919c648aea4ae58f8d0531799fe7982b
2014-06-17 19:38:58 +00:00
Daniel Friesen
07b80668be ResourceLoaderFileModule: Implement remoteSkinPath option
remoteSkinPath works the same as remoteExtPath but is relative to skins/ instead of extensions/

This will allow skins to register modules using:
 $wgResourceModules['skin.myskin'] = array(
   // ...
   'localBasePath' => __DIR__,
   'remoteSkinPath' => 'myskin',
 );

Instead of using:
 'remoteBasePath' => $GLOBALS['wgStylePath'] . '/myskin', // or
 'remoteBasePath' => &$GLOBALS['wgStylePath'],

Change-Id: I0e8c4a37a224e9528c9c5aa5417f0f56dbb89b97
2014-06-17 16:05:19 +02:00
Timo Tijhof
75c08916b0 resourceloader: Implement "skip function" feature
A module can be registered with a skip function. Such function,
if provided, will be invoked by the client when a module is
queued for loading. If the function returns true, the client will
bypass any further loading action and mark the module as 'ready'.

This can be used to implement a feature test for a module
providing a shim or polyfill.

* Change visibility of method ResourceLoader::filter to public.

So that it can be invoked by ResourceLoaderStartupModule.

* Add option to suppress the cache key report in ResourceLoader::filter.

We usually only call the minifier once on an entire request
reponse (because it's all concatenated javascript or embedded
javascript in various different closures, still valid as one
large script) and only add a little bottom line for the cache
key. When embedding the skip function we have to run the minifier
on them separately as they're output as strings (not actual
functions). These strings are typically quite small and blowing
up the response with loads of cache keys is not desirable in
production.

* Add method to clear the static cache of ResourceLoader::inDebugMode.

Global static state is evil but, as long as we have it, we at
least need to clear it after switching contexts in the test suite.

Also:
* Remove obsolete setting of 'debug=true' in the FauxRequest in
  ResourceLoaderTestCase. It already sets global wgResourceLoaderDebug
  in the setUp() method.

Bug: 66390
Change-Id: I87a0ea888d791ad39f114380c42e2daeca470961
2014-06-12 03:48:26 +02:00
Timo Tijhof
0888ccbcad Installer: Re-use ResourceLoader methods instead of duplicating
Also:
* Fix incorrect documentation comment of ResourceLoaderModule::getStyles(),
  it doesn't return a string, it never has.
* Remove spurious space in count() call.
* Fix spelling of in "proper".

Change-Id: I000393636f7b9015ad1bacfbb3eb8a6ad8695d72
2014-06-07 10:02:55 +02:00
Siebrand Mazeland
0b2fcf3785 Pass phpcs-strict on includes/resourceloader/
Change-Id: I5c63e0612c2aae240abcdbf21da9cdadd80dee31
2014-05-10 10:39:37 +02:00
umherirrender
dcf6955e5c Fixed some @params documentation (includes/*)
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.

Change-Id: Ifbb1da2a6278b0bde2a6f6ce2e7bd383ee3fb28a
2014-04-20 23:33:05 +02:00
Timo Tijhof
66b736e5fb resourceloader: Fix broken skinStyles loop in #getModifiedTime
The way we interpret the 'skinStyles' array in #getModifiedTime
is broken. For comparision, look at the loop in #getStyleFiles
which we use for the creation of the actual stylesheets (which
works fine).

It mixed up the logic causing it to always end up with an empty
array. So it's not so much that it fails to detect the files
having changed after an update, it never included mtimes of these
files in the first place. If there'd be a module with only
skinStyles resources, it would have timestamp 0 (1970).

Bug: 62068
Change-Id: I2e772a13183e66e4bfbf95057ebfd7f5e0d817ec
2014-02-28 19:24:51 +00:00
umherirrender
add8b9a976 Unroll array_map in ResourceLoaderFileModule::readStyleFiles
ResourceLoaderFileModule::readStyleFile can throw exceptions which
produces warnings when using array_map.

Produces useful stacktraces when there are issues with style file like LESS compilation errors

Bug: 47844
Bug: 59858
Change-Id: I7c48b2c94752eee81f0eeb8e00d1f2b526d80dfc
2014-02-13 06:27:17 +00:00
umherirrender
073abe3e12 No variable assignment on return statement
Split the variable assignment and the return statement in two lines for
better readability.

When there was two return statements in one method the logic was swapped
to have only one return statement.

Change-Id: Id7a01b4a2df96036435f9e1a9be5678dd124b0af
2014-01-02 09:43:35 +00:00
Timo Tijhof
d3bdda3221 resourceloader: Add definition hashing to improve cache invalidation
Currently we invalidate module caches based on timestamps from
various places (including message blob, file mtimes and more).

This meant that when a module changes such that the maximum
detectable timestamp is still the same, the cache would not
invalidate.

Module classes can now implement a method to build a summary.
In most cases this should be (a normalised version of) the
definition array originally given to ResourceLoader::register
(such as $wgResourceModules items).

The most common scenarios this addresses:

* File lists (scripts, styles) being re-ordered.
* Files being removed when more recently changed files remain
  part of the module (e.g. removing an older file).
* Files or messages being added to a module while other more
  recently changed things are already part of the module.
  E.g. you create a message and use it in a js file, but forget
  to add the message to the module. Then later you add the msg
  key. This last update would be ignored, because the mtime of
  the message is no newer than the total which already included
  the (same) mtime of the js file added in the previous update.
* Change the concatenation order of pages in a Gadget definition.

Bug: 37812
Change-Id: I00cf086c981a84235623bf58fb83c9c23aa2d619
2013-12-02 21:45:33 -08:00
umherirrender
0bc583af2c Move closing parenthesis from multi line if and function to own line
The Line continuation Coding conventions prefers the closing parenthesis
on the same line than the beginning curly braces. This is done for ifs
and functions.
Also move some boolean operator from the end of a line to the beginning
and changed some indentation to make the condition hopefully better
readable.

Change-Id: Id0437b06bde86eb5a75bc59eefa19e7edb624426
2013-12-01 21:39:00 +01:00
umherirrender
f153998317 Fixed spacing
- Removed double spaces
- Added space after if/switch/foreach
- Removed space on elseif
- Added space around parentheses
- Added newline at end of file
- Removed space before semicolon at end of line

Change-Id: Id40b87e04786c6111e6686d7f7eea1e588bdf37d
2013-11-19 19:03:54 +01:00
Timo Tijhof
f46f21915a resourceloader: Use state "error" instead of "missing" in case of exceptions
Changes:
* If we catch an exception while making the response for a module, set its
  client state to "error" instead of "missing".
  State "missing" should only be used if the module could not be
  found in the registry.
  This matches the behaviour of the client.

Clean up:
* Merge the two mw.loader.state calls into one.
* Add @throws documentation for compileLESSFile (follows-up cdc8b9e).
* Don't use 3 different ways to assert an array being empty,
  using 1 and sticking to it.
  - !$arr
  - $arr === array()
  - count( $arr )

Change-Id: I54c3de6d836702ffbe3044bc58a38e83e758bc33
2013-11-15 02:32:48 +01:00
Timo Tijhof
cdc8b9e011 resourceloader: Don't catch LESS error in ResourceLoaderFileModule
Instead let it throw up to ResourceLoader::makeModuleResponse.. By using
the central logic for exception handling the parse error becomes more
discoverable and obvious.

* The module is marked as in "error" state (currently it would still serve
  the result of file A and C, even if file B has a parse error).
  This matches the behaviour of the javascript checker we have.
* The exception comment is produced on top of the load.php response
  (where users expect it) instead of buried somewhere between the
  different modules and the different files within the modules.
* The exception comment will not be stripped out by CSSMin.
* DRY :)

Without this, debugging less parse errors is harder than it should be.

Bug: 55442
Change-Id: I6510b43f02cf82568b7fd2421209a8e636d5e115
2013-10-14 17:49:21 +00:00
Max Semenik
4d80400cdf Fix comment typos from 5c51cb96ea
Change-Id: I56f37010bce8a325af65eab62e7add232df108f4
2013-10-01 19:32:33 +00:00
Max Semenik
5c51cb96ea Maintenance script to check LESS files for validity
A reworked version of script from I068686854ad79e2f63a08d81b1af02f373110613

Move leccs instantiation code to ResourceLoader because it doesn't depend on any
particular module's state.

Change-Id: I733b53171dca77f50a30e5bd0bd5f1b456e4c85d
2013-09-30 21:37:13 +04:00
Timo Tijhof
93cb656139 less: Use new addParsedFile method, replacing embeddedFiles hack
Follows-up b67b9e1, lessphp now has a public method to add to the
list of files for compilation cache.

Change-Id: I62a6c7cdeb34ea742fa7547e3ca10e24ee484b97
2013-09-29 05:16:00 +00:00