Not used since 2011 (MediaWiki 1.18).
In an early version of ResourceLoader, we ran JSMinPlus syntax
validation on-demand on all served JavaScript content. This
was identified as cause of slowdown and high memory use, and
generally not considered as useful in production.
The reason it was there originally was not for the purpose of
validating static files, but for user-generated content.
So in MediaWiki 1.18, the behaviour of wgResourceLoaderValidateJS
was changed to only apply to user-generated content, and the
rest was disabled behind wgResourceLoaderValidateStaticJS, which
we then never use. Not even in development, given that we now
have superior experience through ESLint, even within IDEs where
supported.
Follows-up 49d3d18033 (r91914).
Change-Id: Ie25109a4fb23ee93fed0db4af5db4b11fe9ffe7f
The use of global variables was deprecated in favour of
ResourceLoaderModule::getLessVars() on a per-module basis.
Also moved testLessFileCompilation case to the appropiate file as it
covers ResourceLoaderFileModule.php, not ResourceLoader.php.
Bug: T140804
Depends-On: Ib1b2808df2384473bfac47f53a5d25d7c9bbca2b
Depends-On: I96047f69d01c4736306df2719267e6347daf556f
Change-Id: If708087c85c80355c7e78f1768529b5f2e16ed07
Add @covers for various helper methods used by public methods, where the helper
methods actually contain most of the logic being tested in FileModuleTest.
I've changed these methods from protected to private (confirmed no usage)
to further pin down that their contract doesn't matter beyond making the
public methods work.
Change-Id: I2aef0d322b38bc3595e7d2c2339112b16fc66b8d
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '
(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)
Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).
Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
This prevents cache churn when the wiki-global LESS variables vary
between wikis because the cache key is used as a "global" instead of
db-local. This is good for the common case, but should still explicitly
vary if the vars differ between wikis.
Bug: T191937
Change-Id: If12fd07a7062792205384150d6f5fd9a83f996cc
I can see that "parent::__construct" literally calls the parent
constructor. I can see that stuff preceeded by the keyword "protected"
is protected. I really (really) don't need comments explaining such.
Change-Id: I7458e714976a6acd3ba6a7c93fdc27d03903df83
This already worked as expected for any module that uses the new
enableModuleContentVersion model, but for the majority of file modules
this is not yet the case for performance reasons. As such, make
sure lessVars are included in our manual tracking.
Include it conditionally to avoid changing the array for other modules,
which would needlessly invalidate their cache.
Bug: T171809
Change-Id: Ib250068e0ecfc29a09ca33c23bef901ee0482bf2
Should be useful for cases where we pull in an external library
that already manages RTL flipping in its styles, and CSSJanus
does the wrong thing without extra markup.
Example:
'ext.tmh.video-js' => $baseExtensionResource + [
'scripts' => 'resources/videojs/video.js',
'styles' => 'resources/videojs/video-js.css',
'noflip' => true,
...
],
Bug: T148572
Bug: T148565
Change-Id: Icbad20d8a6e9a0d354ad159f5816f4fb67cc2775
ResourceLoader modules can now carry a 'deprecated' option which can
be a boolean or an object with message key. This message or a default
deprecation message will be show whenever that module is used in production.
Note: This will not work in debug mode for ResourceLoaderFile modules
and this is deemed acceptable for the time being. We can revisit later.
Bug: T137772
Change-Id: Ib9ebd2d39a59fd41d8537e06884699f77b03580c
This allows dynamically loaded modules to depend on page-style modules
without it causing the page-style module to be loaded a second time.
* New method Module::getType() indicates whether a module is
a page-style module or supposed to be dynamically loaded.
* Emit warning from addModuleStyles() when given a module that is
not a page-style module (to be enforced later)
Bug: T92459
Bug: T87871
Change-Id: I8b6c6a10d965e73965f877c42e995d04202524f3
We read files and concatenate their contents. Files may start with a BOM character.
BOM characters are only allowed at the beginning of a file, not half way.
Stripping it should be safe, since we already assume that everything is UTF-8.
Change-Id: I14ad698a684e78976e873e9ae2c367475550a063
Follows-up 8f5cd11d82.
The old getLocalFileReferences() method is no longer used anywhere.
Remove it and rename getAllLocalFileReferences back to it.
Change-Id: I864258aad128ba9b54464c7bc854543f2937f977
The output of this function is not deterministic. This function
is what tranforms urls to include content hashes and also embeds
content as data URI. The calls to this function are expected to be
up to date and matching the version hash as constructed by the
current web request.
Previously, the content hash as constructed based on the collective
file hash of all references images was accurate but the generated
css associated with it could be stale.
Unless we abuse MemoizedCallable to take a separate cache buster
(with $localFileRefs?), it's best to remove this as it isn't
a deterministic function and shouldn't be memoized.
Bug: T128668
Change-Id: Icb87ddc585d7320ac48619446bb8b9cbe5f4780e
This way, they automatically bypass any stale client or server cache.
Remove an old wfExpandUrl() call that would interfere with
transformResourcePath()'s ability to recognise the path.
Expanding the url there wasn't needed anyway. Whether it's on a separate host
(eg. bits.wikimedia.org), or path-absolute, either way works as-is.
Bug: T90983
Change-Id: I64eb3291adcfc3733ef96690399c09c06e19b3aa
Cache Less variables in the instance instead of statically.
This allows tests to populate their own less variabless via ResourceLoaderLESSVars.
Make getLessCompiler() and getLessVars() regular public methods and
update callers.
Change-Id: I95506b8bb20a4b2b3f82014a7b0fcee5f28973e6
MessageBlobStore class:
* Make logger aware.
* Log an error if json encoding fails.
* Stop using the DB table. WANObjectCache supports everything we need:
- Batch retrieval.
- Invalidate keys with wildcard selects or cascading check keys.
* Update tests slightly since the actual update now happens on-demand as
part of get() instead of within updateMessage().
ResourceLoader class:
* Remove all interaction with the msg_resource table. Remove db table later.
* Refactor code to use a hash of the blob instead of a timestamp.
Timestamps are unreliable and roll over too frequently for message blob store
because there is no authoritative source. The timestamps were inferred based on
when a change is observed. Message overrides from the local wiki have an
explicit update event when the page is edited. All other messages, such as
from MediaWiki core and extensions using LocalisationCache, have a single
timestamp for all messages which rolls over every time the cache is rebuilt.
A hash is deterministic, and won't cause needless invalidation (T102578).
* Remove redundant pre-fetching in makeModuleResponse().
This is already done by preloadModuleInfo() in respond().
* Don't bother storing and retreiving empty "{}" objects.
Instead, detect whether a module's message list is empty at runtime.
ResourceLoaderModule class:
* Make logger aware.
* Log if a module's message blob was not preloaded.
cleanupRemovedModules:
* Now that blobs have a TTL, there's no need to prune old entries.
Bug: T113092
Bug: T92357
Change-Id: Id8c26f41a82597e34013f95294cdc3971a4f52ae
This effectively reverts d6b4d3c537 and declines T97420.
This was previously attempted in b7c0e537eb.
Drop support for position "bottom" for addModuleStyles().
This feature was only recently introduced with the intent
to optimise page load performance, but had an adverse effect.
It increases chances of FOUC due to late discovery of these styles.
It also caused minor problems for some gadgets and extensions
that did not or were unable to set these flags. Some mobile
code was introduced around the same time as this feature and
was never given position=top.
Stylesheets that don't affect initial render or are only needed
on interaction should be loaded via addModules() instead; which
is handled by the asynchronous loader in JavaScript.
Change-Id: Ib9821b7b87cfc8a59062bb6ca358974fdb01ced1
Use BagOStuff::makeKey() and BagOStuff::makeGlobalKey() instead of
wfMemcKey() and wfGlobalCacheKey().
Change-Id: Id4e58c0e616b74de220faf13ba7c9ea606fef8c1
Using the following command line, I have found doc comments (and
a wfDeprecated() call) mentioning "1.26" when they should mention
"1.27" instead, which I have fixed manually:
git diff -M REL1_26 | grep --color=always -C 10 -iP \
'^\+.*\D1\.26(\D|$)' | aha > oldver.html
Follows-up these commits:
* 047b60b96d
* 169b7b98b5
* 25a44aa3e4
* 2fb2a3f14b
* 3f1e9fa268 [1]
* 8c84af70b6
* c0cb80beac
* d04a92a551
* d3b85592ea
[1] Release notes moved in I195dd1cf.
Because a release note stating that the UserRights hook is deprecated
was added in 37062a0c0d (before the branch cut), this commit does not
change the Hooks::run call that had been changed in 21206c8fbe.
Change-Id: I5a427f003e7e3b4559fe377bcdfdca466a570708
Add a simple class to `libs/` for memoizing functions by caching return values
in APC. I decided not to make this an external library just yet because I see
this as potentially a part of a larger functional programming library.
Doesn't use APCBagOStuff for two reasons: (1) avoid dependency on MediaWiki
code; (2) ability to pass third &$success parameter to apc_store, to
distinguish between cache misses and cached false values.
Use this in ResourceLoaderFileModule to cache CSSMin::remap.
Change-Id: I00a47983a2583655d4631ecc9c6ba17597e36b5f
As Timo noted in a review of I1ff6115, using '+=' to combine file dependencies
is incorrect here. I fixed it in one place but not the other, causing the file
dependencies to vary based on whether a .lesscache file was used.
Change-Id: Ie3f48048febc32c5c2d45acd87e10f946f6dfc94
Re-use the md_skin database field for now as to not need a
schema change, beause this table is going away soon (T113916).
Bug: T113868
Change-Id: I7c7546ec58fd9be0447604989b908dd2084b0fe3
Make paths stored in the module_deps table relative to $IP. This ensures that when
the MediaWiki install path changes and/or if the location of the extension or skins
directory changes, that ResourceLoader's internal model is still accurate.
Previously when these paths change, ResourceLoader would exhibit various bugs.
1. Unable to detect changes in the module (if the directory no longer exists).
2. Point #1 is usually preceeded by one last cache invalidation as the content hash
of the file path changes (from a valid hash to an empty string).
3. Unnecessary cache invalidation (if both old and new directories exist). This
happens when a file is both an entry point (in the 'scripts' or 'styles' array)
and also a file dependency. At first they are de-duplicated by array_unique.
But after the disk path changes, the next check will result in the old path
being fetched from module_deps, and the new path from the live configuration.
This causes two changes that result in needless cache invalidation:
- The hash list contains one more item (T111481).
- The hash list contains both the old and new version of a file.
(or even alternate versions, e.g. when a change is backported to the old
wmf branch; it also affects wikis on the new branch due to the stale
file path still in the database).
It seems unusual to move a MediaWiki install, and usually we recommend third
parties to run update.php if site administrators do move their wiki. However
Wikimedia's deployment system implicitly moves the MediaWiki install continously
from e.g. "/srv/mediawiki/php-1.26wmf5" to "/srv/mediawiki/php-1.26wmf6".
This caused virtually all ResourceLoader caching layers to get invalidated every
week when another wmf-branch is deployed, thus breaking these file paths, which
changes the version hash, which then invalidates the cache.
Bug: T111481
Change-Id: I173a9820b3067c4a6598a4c8d77e239797d2659c
* Remove ResourceLoaderFileModule::getLessCompiler(). There is no reason for a
module to need to get a compiler in a different manner than
ResourceLoader::getLessCompiler().
* Add ResourceLoaderModule::getLessVars(). This method provides a means for
subclasses to easily inject custom LESS variables. The default implementation
simply returns an empty array.
* Make the $context parameter for ResourceLoaderFileModule::readStyleFiles()
non-optional (via graceful deprecation). The only callers I found were either
already calling it with a ResourceLoader context, or had a perfectly usable
ResourceLoaderContext in local scope.
* Make ResourceLoaderFileModule::{readStyleFile,getLessCompiler} require a
context. These methods are protected, so I didn't bother with a deprecation.
* Call ksort() on the LESS variables array in the only place it matters -- when
hashing its serialized representation to construct a cache lookup key. This
relieves getLessVars() subclasses from having to remember to re-sort the
variables array if they modify it.
* These changes make it possible to substantially simplify
ResourceLoaderEditToolbarModule, because the only thing it needs to do now is
implement its own getLessVars() method.
* This also allows it to be versioned like any other ResourceLoaderFileModule,
rather than having to use enableModuleContentVersion().
Change-Id: Ic3eab71691e502bfe19bdf4eb6f82cc679a7782f
Most of it was already there (SELECT in preloadInfo, and in-class holder via
setFileDependencies) but the logic to write to the database was within
the file module class.
Moving it the other way around may make more sense in the future, but for
the moment ResourceLoader.php makes assumptions about all modules being
able to store setFileDependencies as read from the database. Whether or not
a class currently implements a way to write to that table.
Change-Id: I06b5da4144798197478c66e8b9ccd4cc62cf6fb6
Caching the output of a LESS compiler is tricky, because a LESS file may
include additional LESS files via @imports, in which case the cache needs
to vary as the contents of those files vary (and not just the contents of
the primary LESS file).
To solve this, we first introduce a utility class, FileContentsHasher. This
class is essentially a smart version of md5_file() -- given one or more file
names, it computes a hash digest of their contents. It tries to avoid
re-reading files by caching the hash digest in APC and re-using it as long as
the files' mtimes have not changed. This is the same approach I used in
I5ceb8537c.
Next, we use this class in ResourceLoaderFileModule in the following way:
whenever we compile a LESS file, we cache the result as an associative array
with the following keys:
* `files` : the list of files whose contents influenced the compiled CSS.
* `hash` : a hash digest of the combined contents of those files.
* `css` : the CSS output of the compiler itself.
Before using a cached value, we verify that it is still current by asking
FileContentHasher for a hash of the combined contents of all referenced files,
and we compare that against the value of the `hash` key of the cached entry.
Bug: T112035
Change-Id: I1ff61153ddb95ed17e543bd4af7dd13fa3352861
The way this is implemented is really dirty... but it found us
a few pre-existing bugs already (T111518, T111519, T111771).
I think it might be worth it.
* CSSMin: Add new method getAllLocalFileReferences() which skips the
file_exists() check.
* ResourceLoaderModule: Make use of it to track missing files too.
* ResourcesTest: Verify that the missing files are missing.
Change-Id: I5a3cdeb7d53485f161ccf8133e76850cdf5b4579
These properties don't affect the module response. They are either
only consumed by OutputPage, or provided by the startup module
directly, which detects its own changes.
Changes to these properties should not invalidate module cache for
these properties.
Follows-up 2f30ff7a86 which added a hook to getDependencies()
but still included the raw 'dependencies' in the definition summary.
But as it turns out, dependencies don't need to be included.
Change-Id: I468e6a6edc352d845dc7a5d06b0ae2a6d809700b
In Wikimedia production this tends to contain the branch name,
which causes the version to change for no reason after a MediaWiki
deployment. For example:
["remoteBasePath"]=>
string(43) "https://test.wikipedia.org/static/1.26wmf19"
Bug: T102578
Change-Id: I50b8afd72183cae875d4aaa0aaf0da4e7d664284
Enabling the module content versionining is not feasible for FileModule as that
would involve Lessc and CSSJanus just to compute the version hash.
Instead, we can keep the existing logic that exists for the timestamp-based
versioning (which already has a comprehensive grip on tracking all involved
factors that cause a module to change) and convert it to use hashing instead.
This way the version hashes will be deterministic. Currently module versions
tend to be invalidated too often (and sometimes not often enough) due to Git and
other transport mechanisms not preserving file timestamps.
== Research ==
* <https://phabricator.wikimedia.org/T98087#1412712>
It'd take upto 10 seconds to run startup if FileModule enables build versioning.
* <https://phabricator.wikimedia.org/T104950#1433142>
Checking all files in resources/** took only ~30m longer in total with
sha1_file compared to filemtime.
Bug: T104950
Change-Id: I732fa4db32258c634e32b507952f76eac7fc9395
This greatly simplifies logic required to compute module versions.
It also makes it significantly less error-prone.
Since f37cee996e, we support hashes as versions (instead of timestamps).
This means we can build a hash of the content directly, instead of compiling a
large array with all values that may influence the module content somehow.
Benefits:
* Remove all methods and logic related to querying database and disk for
timestamps, revision numbers, definition summaries, cache epochs, and more.
* No longer needlessly invalidate cache as a result of no-op changes to
implementation datails. Due to inclusion of absolute file paths in the
definition summary, cache was always invalidated when moving wikis to newer
MediaWiki branches; even if the module observed no actual changes.
* When changes are reverted within a certain period of time, old caches can now
be re-used. The module would produce the same version hash as before.
Previously when a change was deployed and then reverted, all web clients (even
those that never saw the bad version) would have re-fetch modules because the
version increased.
Updated unit tests to account for the change in version. New default version of
empty test modules is: "mvgTPvXh". For the record, this comes from the base64
encoding of the SHA1 digest of the JSON serialised form of the module content:
> $str = '{"scripts":"","styles":{"css":[]},"messagesBlob":"{}"}';
> echo base64_encode(sha1($str, true));
> FEb3+VuiUm/fOMfod1bjw/te+AQ=
Enabled content versioning for the data modules in MediaWiki core:
* EditToolbarModule
* JqueryMsgModule
* LanguageDataModule
* LanguageNamesModule
* SpecialCharacterDataModule
* UserCSSPrefsModule
* UserDefaultsModule
* UserOptionsModule
The FileModule and base class explicitly disable it for now and keep their
current behaviour of using the definition summary. We may remove it later, but
that requires more performance testing first.
Explicitly disable it in the WikiModule class to avoid breakage when the
default changes.
Ref T98087.
Change-Id: I782df43c50dfcfb7d7592f744e13a3a0430b0dc6
By providing context as a parameter in getDependencies, we allow
modules to dyanamically determine dependencies based on context.
Note: To ease rollout, the parameter is optional in this patch. It is expected
that it will be made non-optional in the near future.
The use case is for CentralNotice campaigns to be able to add special
modules ahead of deciding which banner to show a user. The dynamically
chosen RL modules would replace ad-hoc JS currently sent with some banners.
A list of possible campaigns and banners is already sent as a PHP-
implemented RL module; that's the module that will dynamically choose other
modules as dependencies when appropriate. This approach will save a round
trip as compared to dynamically loading the modules client-side.
For compatibility, extensions that override
ResourceLoaderModule::getDependencies() should be updated with the new
method signature. Here are changes for extensions currently deployed on
Wikimedia wikis:
* CentralNotice: I816bffa3815e2eab7e88cb04d1b345070e6aa15f
* Gadgets: I0a10fb0cbf17d095ece493e744296caf13dcee02
* EventLogging: I67e957f74d6ca48cfb9a41fb5144bcc78f885e50
* PageTriage: Ica3ba32aa2fc76d11a44f391b6edfc871e7fbe0d
* UniversalLanguageSelector: Ic63e617f51702c27104e123d4bed91983a726b7f
* VisualEditor: I0ac775ca286e64825e31a9213b94648e41a5bc30
For more on the CentralNotice use case, please see I9f80edcbcacca2.
Bug: T98924
Change-Id: Iee61e5b527321d01287baa03ad9b4d4f526ff3ef
Currently all styles modules added to the page using
addModuleStyles are put into the head, regardless
of their "position" value.
Bug: T97420
Change-Id: Ie4287e17d6f298cc63f42f257b1f67ee36961b77
Modules now track their version via getVersionHash() instead of getModifiedTime().
== Background ==
While some resources have observeable timestamps (e.g. files stored on disk),
many other resources do not. E.g. config variables, and module definitions.
For static file modules, one can e.g. revert one of more files in a module to a
previous version and not affect the max timestamp.
Wiki modules include pages only if they exist. The user module supports common.js
and skin.js. By default neither exists. If a user has both, and then the
less-recently modified one is deleted, the max-timestamp remains unchanged.
For client-side caching, batch requests use "Math.max" on the relevant timestamps.
Again, if a module changes but another module is more recent (e.g. out-of-order
deployment, or out-of-order discovery), the change would not result in a cache miss.
More scenarios can be found in the associated Phabricator tasks.
== Version hash ==
Previously we virtually mapped these variables to a timestamp by storing the current
time alongside a hash of the value in ObjectCache. Considering the number of
possible request contexts (wikis * modules * users * skins * languages) this doesn't
work well. It results in needless cache invalidation when the first time observation
is purged due to LRU algorithms. It also has other minor bugs leading to fewer
cache hits.
All modules automatically get the benefits of version hashing with this change.
The old getDefinitionMtime() and getHashMtime() have been replaced with dummies
that return 1. These functions are often called from getModifiedTime() in subclasses.
For backward-compatibility, their respective values (definition summary and hash)
are now included in getVersionHash directly.
As examples, the following modules have been updated to use getVersionHash directly.
Other modules still work fine and can be updated later.
* ResourceLoaderFileModule
* ResourceLoaderEditToolbarModule
* ResourceLoaderStartUpModule
* ResourceLoaderWikiModule
The presence of hashes in place of timestamps increases the startup module size on
a default MediaWiki install from 4.4k to 5.8k (after gzip and minification).
== ETag ==
Since timestamps are no longer tracked, we need a different way to implement caching
for cache proxies (e.g. Varnish) and web browsers. Previously we used the
Last-Modified header (in combination with Cache-Control and Expires).
Instead of Last-Modified (and If-Modified-Since), we use ETag (and If-None-Match).
Entity tags (new in HTTP/1.1) are much stricter than Last-Modified by default.
They instruct browsers to allow usage of partial Range requests. Since our responses
are dynamically generated, we need to use the Weak version of ETag.
While this sounds bad, it's no different than Last-Modified. As reassured by
RFC 2616 <http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.3.3> the
specified behaviour behind Last-Modified follows the same "Weak" caching logic as
Entity tags. It's just that entity tags are capable of a stricter mode (whereas
Last-Modified is inherently weak).
== File cache ==
If $wgUseFileCache is enabled, ResourceLoader uses ResourceFileCache to cache
load.php responses. While the blind TTL handling (during the allowed expiry period)
is still maxage/timestamp based, tryRespondNotModified() now requires the caller to
know the expected ETag.
For this to work, the FileCache handling had to be moved from the top of
ResoureLoader::respond() to after the expected ETag is computed.
This also allows us to remove the duplicate tryRespondNotModified() handling since
that's is already handled by ResourceLoader::respond() meanwhile.
== Misc ==
* Remove redundant modifiedTime cache in ResourceLoaderFileModule.
* Change bugzilla references to Phabricator.
* Centralised inclusion of wgCacheEpoch using getDefinitionSummary. Previously this
logic was duplicated in each place the modified timestamp was used.
* It's easy to forget calling the parent class in getDefinitionSummary().
Previously this method only tracked 'class' by default. As such, various
extensions hardcoded that one value instead of calling the parent and extending
the array. To better prevent this in the future, getVersionHash() now asserts
that the '_cacheEpoch' property made it through.
* tests: Don't use getDefinitionSummary() as an API.
Fix ResourceLoaderWikiModuleTest to call getPages properly.
* In tests, the default timestamp used to be 1388534400000 (which is the unix time
of 20140101000000; the unit tests' CacheEpoch). The new version hash of these
modules is "XyCC+PSK", which is the base64 encoded prefix of the SHA1 digest of:
'{"_class":"ResourceLoaderTestModule","_cacheEpoch":"20140101000000"}'
* Add sha1.js library for client-side hash generation.
Compared various different implementations for code size (after minfication/gzip),
and speed (when used for short hexidecimal strings).
https://jsperf.com/sha1-implementations
- CryptoJS <https://code.google.com/p/crypto-js/#SHA-1> (min+gzip: 2.5k)
http://crypto-js.googlecode.com/svn/tags/3.1.2/build/rollups/sha1.js
Chrome: 45k, Firefox: 89k, Safari: 92k
- jsSHA <https://github.com/Caligatio/jsSHA>
https://github.com/Caligatio/jsSHA/blob/3c1d4f2e/src/sha1.js (min+gzip: 1.8k)
Chrome: 65k, Firefox: 53k, Safari: 69k
- phpjs-sha1 <https://github.com/kvz/phpjs> (RL min+gzip: 0.8k)
https://github.com/kvz/phpjs/blob/1eaab15d/functions/strings/sha1.js
Chrome: 200k, Firefox: 280k, Safari: 78k
Modern browsers implement the HTML5 Crypto API. However, this API is asynchronous,
only enabled when on HTTPS in Chromium, and is quite low-level. It requires boilerplate
code to actually use with TextEncoder, ArrayBuffer and Uint32Array. Due this being
needed in the module loader, we'd have to load the fallback regardless. Considering
this is not used in a critical path for performance, it's not worth shipping two
implementations for this optimisation.
May also resolve:
* T44094
* T90411
* T94810
Bug: T94074
Change-Id: Ibb292d2416839327d1807a66c78fd96dac0637d0
lessc::allParsedFiles() always includes the main file we already
know about. As such needlessly causing it to be queried multiple
times.
Change-Id: Icd2c11d93d6491664fcf58a18bfe6e2932689774
Right now ResourceLoaderModule::getDefinitionSummary just returns an array
with the class name, but may be extended in the future. Either way, it shouldn't
be repeated in the subclass.
Change-Id: Ic1d58f8d330fb2176daffc7ad39bc0751c196181
Xhprof generates this data now. Custom profiling of various
sub-function units are kept.
Calls to profiler represented about 3% of page execution
time on Special:BlankPage (1.5% in/out); after this change
it's down to about 0.98% of page execution time.
Change-Id: Id9a1dc9d8f80bbd52e42226b724a1e1213d07af7
- Changed spaces to tabs for indentation
- space after 'function'/'if'
- Added/Removed space after parenthesis/brackets/end of line
- Removed space after cast
Change-Id: I0e8e6a19b84b5e1308b632a0266cb78f688494ee
A base ResourceLoaderModule::getTemplates() exists for subclasses
to override. An implementation is provided for ResourceLoaderFileModule.
For file modules, templates can be specified in the following manner:
'example' => array(
'templates' => array(
'bar' => 'templates/foo.html',
),
'scripts' => 'example.js',
),
The delivery system is template language agnostic, and currently
only supports "compiling" plain HTML templates.
This also adds template support to the following modules as a POC:
* mediawiki.feedback
* mediawiki.action.view.postEdit
* mediawiki.special.upload
Works with $wgResourceLoaderStorageEnabled
Change-Id: Ia0c5c8ec960aa6dff12c9626cee41ae9a3286b76
Not ready for merging, and Roan says that the +2 was
most likely accidental and meant to be a -1.
This reverts commit d146934f94.
Change-Id: I3926c9ae9e3c8026fceb3aeedd3b1f1d9b91667b
Preparation work for templating in core.
RL should allow us to ship HTML / template markup from server to client.
Use in Special:Upload and mediawiki.feedback as a proof of concept.
Separation of concerns etc...
See Also:
Ia63d6b6868f23a773e4a41daa0036d4bf2cd6724
Change-Id: I6ff38c12897e3164969a1090449e626001926c3b
No need to set a variable to value of itself. Also consistent
with the way $remoteBasePath is set, already.
Change-Id: I9b71322592a5ca2d85927fcf6732096f758847e9
This reverts most of commit 2d842f1425,
leaving only the test added in it, and reimplements the same
functionality better.
Instead of stripping /*@noflip*/ annotations in CSSJanus, which is
incompatible with other implementations that preserve it, extend
CSSMin to allow other CSS comments to be present before the
rule-global @embed annotation. (This required making the regex logic
in it even worse than it was, but it's actually slightly less terrible
than I expected it would be. Good thing we have tests!)
Bug: 69698
Change-Id: I58603ef64f7d7cdc6461b34721a4d6b15f15ad79
Bonus: actually make $wgResourceBasePath default to $wgScriptPath, rather than
special-casing it in ResourceLoaderFileModule.
Change-Id: I608435cef00d3e77a5bbdb0a0122d3e7e1a4eb78
Also, try out a way to have per-module LESS variables defined in PHP.
This might come in handy in the future… Maybe for skin theme support?
(I recommend reviewing the file changes in the order below. :D)
includes/resourceloader/ResourceLoaderFileModule.php
* Pass the context (ResourceLoaderContext) deeper down via
readStyleFiles() and readStyleFile(). We need it to compile the
.less files for the right language.
* Extract LESS compiler creation to getLessCompiler().
* Allow passing a LESS compiler instance to compileLessFile(), rather
than getting one after the method is called.
All of the changes are backwards-compatible.
includes/resourceloader/ResourceLoaderEditToolbarModule.php
* New module to support getting the language data and passing it to
LESS variables.
It might be a good idea to factor out a reusable class for a LESS
module with additional variables, but that would require more
attention to design than I gave it.
resources/src/mediawiki.action/mediawiki.action.edit.toolbar/mediawiki.action.edit.toolbar.less
* Glue code to use the language data defined by the module above and
put it in final CSS.
includes/EditPage.php
* Do not hardcode image URLs in output HTML, as they are provided in
CSS now. This gets rid of some usage of globals.
In fact, we should be able to finally move the inline JavaScript
calls out of getEditToolbar(), but I'm already introducing too many
changes for one patch. That can be done later.
languages/Language.php
* Add getImageFiles() to complement existing getImageFile() method.
Misleadingly named, it returns paths for images for the toolbar
only (and no other ones at all).
skins/common/ → resources/src/mediawiki.action/mediawiki.action.edit.toolbar/
* Moved all of the button images to new location.
Also, boring cleanup that was harder before because we treated the
paths as public API:
* Placed default ones in en/ subdirectory.
* Renamed cyrl/ to ru/.
* Renamed ksh/button_S_italic.png → ksh/button_italic.png.
languages/messages/
* Adjusting paths and filenames for the changes above.
resources/src/mediawiki.action/mediawiki.action.edit.css
resources/src/mediawiki.action/mediawiki.action.edit.js
* Added styles and updated the script to make it possible to have
non-<img> elements as toolbar buttons.
* Consolidated styles that were already required, but defined
somewhere else:
* `cursor: pointer;` (from shared.css)
* `vertical-align: middle;` (from commonElements.css)
Bug: 69277
Change-Id: I39d8ed4258c7da0fe4fe4c665cdb26c86420769c
We used to have a second level of caching for them – when a module's
cache was invalidated, individual .less files that comprise it would
only be recompiled (with all their @imports) if their mktime changed
(or if $wgResourceLoaderLESSVars changed).
Given that practice has shown that RL modules including .less files
usually just include one file that @imports tons of stuff, and given
that RL already is very careful about invalidating caches, I think
this is unnecessary at best and harmful at worst.
The behavior proved problematic when I tried to implement support for
per-module LESS variables, values of which were language-dependent –
files would not be recompiled for different languages, and
getLessCacheKey() is called several levels deep in code that doesn't
have access to the context in which the module is being compiled.
Change-Id: I667f063507583e4cd3a669e32398d440f1431f7e
To do it, just remove /*@noflip*/ annotations in CSSJanus after
we're done processing. They are not needed anymore and some obscure
interactions with CSSMin logic for preserving comments caused
`/*@noflip*/ /*@embed*/ background-image: url(…)` not to work
correctly (it would not be embedded).
This also requires us to always do CSSJanus processing, even when we
don't need flipping, to consistently handle the annotations.
I'm not entirely sure if this is worth it, but I still greatly prefer
doing it to documenting this stupid limitation. :)
Bug: 69698
Change-Id: I311b12b08b2dff9d45efb584db08cf4a11318f59
Follows-up 75c08916b0. Looking at the call chain of
validateScriptFile() up to JSTokenzier, the first parameter is the file
name, otherwise "[inline]" is used; so that variable should be $localPath.
Bug: 69214
Change-Id: If7f36449cb352f50ba795a6d306e5d949a3dbd29
The newly introduced $wgResourceModuleSkinStyles global enables skins to
provide additional stylesheets to existing ResourceLoader module.
This both makes it easier (or at all possible) to override default
styles and lowers the style footprint by making it possible not to
load styles unused on most pages.
----
Example:
Use the file 'foo-styles.css' for the 'mediawiki.foo' module when using
the MySkin skin:
$wgResourceModuleSkinStyles['myskin'] = array(
'mediawiki.foo' => 'foo-styles.css',
'remoteSkinPath' => 'MySkin',
'localBasePath' => __DIR__,
);
For detailed documentation, see the doc comment in DefaultSettings.php.
For a practical usage example, see Vector.php.
----
Implementation notes:
* The values defined in $wgResourceModuleSkinStyles are embedded into
the modules as late as possible (in ResourceLoader::register()).
* Only plain file modules are supported, setting module skin styles
for other module types has no effect.
* ResourceLoader and ResourceLoaderFileModule now support loading
files from arbitrary paths to make this possible, defined using
ResourceLoaderFilePath objects.
* This required some adjustments in seemingly unrelated places for
code which didn't handle the paths fully correctly before.
* ResourceLoader and ResourceLoaderFileModule are now a bit more
tightly coupled than before :(
* Included a tiny example change for the Vector skin, a lot more of
similar cleanup is possible and planned for the future.
* Many of the non-essential mediawiki.* modules defined in
Resources.php should be using `'skinStyles' => array( 'default' => … )`
instead of `'styles' => …` to allow more customizations, this is
also planned for the future after auditing which ones would actually
benefit from this.
Change-Id: Ica4ff9696b490e35f60288d7ce1295766c427e87
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling
Change-Id: I8ebfbcea0e2ae2670553822acedde49c1aa7e98d
* Add the ResourceLoaderFileModule#getAllSkinStyleFiles method,
which returns all of the skinStyles files for a given module
* The LessFileCompilationTest and checkLess.php script, which use
the #getAllStyleFile method, now validate all LESS style files
Bug: 63343
Change-Id: Ib2eb5761919c648aea4ae58f8d0531799fe7982b
remoteSkinPath works the same as remoteExtPath but is relative to skins/ instead of extensions/
This will allow skins to register modules using:
$wgResourceModules['skin.myskin'] = array(
// ...
'localBasePath' => __DIR__,
'remoteSkinPath' => 'myskin',
);
Instead of using:
'remoteBasePath' => $GLOBALS['wgStylePath'] . '/myskin', // or
'remoteBasePath' => &$GLOBALS['wgStylePath'],
Change-Id: I0e8c4a37a224e9528c9c5aa5417f0f56dbb89b97
A module can be registered with a skip function. Such function,
if provided, will be invoked by the client when a module is
queued for loading. If the function returns true, the client will
bypass any further loading action and mark the module as 'ready'.
This can be used to implement a feature test for a module
providing a shim or polyfill.
* Change visibility of method ResourceLoader::filter to public.
So that it can be invoked by ResourceLoaderStartupModule.
* Add option to suppress the cache key report in ResourceLoader::filter.
We usually only call the minifier once on an entire request
reponse (because it's all concatenated javascript or embedded
javascript in various different closures, still valid as one
large script) and only add a little bottom line for the cache
key. When embedding the skip function we have to run the minifier
on them separately as they're output as strings (not actual
functions). These strings are typically quite small and blowing
up the response with loads of cache keys is not desirable in
production.
* Add method to clear the static cache of ResourceLoader::inDebugMode.
Global static state is evil but, as long as we have it, we at
least need to clear it after switching contexts in the test suite.
Also:
* Remove obsolete setting of 'debug=true' in the FauxRequest in
ResourceLoaderTestCase. It already sets global wgResourceLoaderDebug
in the setUp() method.
Bug: 66390
Change-Id: I87a0ea888d791ad39f114380c42e2daeca470961
Also:
* Fix incorrect documentation comment of ResourceLoaderModule::getStyles(),
it doesn't return a string, it never has.
* Remove spurious space in count() call.
* Fix spelling of in "proper".
Change-Id: I000393636f7b9015ad1bacfbb3eb8a6ad8695d72
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Change-Id: Ifbb1da2a6278b0bde2a6f6ce2e7bd383ee3fb28a
The way we interpret the 'skinStyles' array in #getModifiedTime
is broken. For comparision, look at the loop in #getStyleFiles
which we use for the creation of the actual stylesheets (which
works fine).
It mixed up the logic causing it to always end up with an empty
array. So it's not so much that it fails to detect the files
having changed after an update, it never included mtimes of these
files in the first place. If there'd be a module with only
skinStyles resources, it would have timestamp 0 (1970).
Bug: 62068
Change-Id: I2e772a13183e66e4bfbf95057ebfd7f5e0d817ec
ResourceLoaderFileModule::readStyleFile can throw exceptions which
produces warnings when using array_map.
Produces useful stacktraces when there are issues with style file like LESS compilation errors
Bug: 47844
Bug: 59858
Change-Id: I7c48b2c94752eee81f0eeb8e00d1f2b526d80dfc
Split the variable assignment and the return statement in two lines for
better readability.
When there was two return statements in one method the logic was swapped
to have only one return statement.
Change-Id: Id7a01b4a2df96036435f9e1a9be5678dd124b0af
Currently we invalidate module caches based on timestamps from
various places (including message blob, file mtimes and more).
This meant that when a module changes such that the maximum
detectable timestamp is still the same, the cache would not
invalidate.
Module classes can now implement a method to build a summary.
In most cases this should be (a normalised version of) the
definition array originally given to ResourceLoader::register
(such as $wgResourceModules items).
The most common scenarios this addresses:
* File lists (scripts, styles) being re-ordered.
* Files being removed when more recently changed files remain
part of the module (e.g. removing an older file).
* Files or messages being added to a module while other more
recently changed things are already part of the module.
E.g. you create a message and use it in a js file, but forget
to add the message to the module. Then later you add the msg
key. This last update would be ignored, because the mtime of
the message is no newer than the total which already included
the (same) mtime of the js file added in the previous update.
* Change the concatenation order of pages in a Gadget definition.
Bug: 37812
Change-Id: I00cf086c981a84235623bf58fb83c9c23aa2d619
The Line continuation Coding conventions prefers the closing parenthesis
on the same line than the beginning curly braces. This is done for ifs
and functions.
Also move some boolean operator from the end of a line to the beginning
and changed some indentation to make the condition hopefully better
readable.
Change-Id: Id0437b06bde86eb5a75bc59eefa19e7edb624426
- Removed double spaces
- Added space after if/switch/foreach
- Removed space on elseif
- Added space around parentheses
- Added newline at end of file
- Removed space before semicolon at end of line
Change-Id: Id40b87e04786c6111e6686d7f7eea1e588bdf37d
Changes:
* If we catch an exception while making the response for a module, set its
client state to "error" instead of "missing".
State "missing" should only be used if the module could not be
found in the registry.
This matches the behaviour of the client.
Clean up:
* Merge the two mw.loader.state calls into one.
* Add @throws documentation for compileLESSFile (follows-up cdc8b9e).
* Don't use 3 different ways to assert an array being empty,
using 1 and sticking to it.
- !$arr
- $arr === array()
- count( $arr )
Change-Id: I54c3de6d836702ffbe3044bc58a38e83e758bc33
Instead let it throw up to ResourceLoader::makeModuleResponse.. By using
the central logic for exception handling the parse error becomes more
discoverable and obvious.
* The module is marked as in "error" state (currently it would still serve
the result of file A and C, even if file B has a parse error).
This matches the behaviour of the javascript checker we have.
* The exception comment is produced on top of the load.php response
(where users expect it) instead of buried somewhere between the
different modules and the different files within the modules.
* The exception comment will not be stripped out by CSSMin.
* DRY :)
Without this, debugging less parse errors is harder than it should be.
Bug: 55442
Change-Id: I6510b43f02cf82568b7fd2421209a8e636d5e115
A reworked version of script from I068686854ad79e2f63a08d81b1af02f373110613
Move leccs instantiation code to ResourceLoader because it doesn't depend on any
particular module's state.
Change-Id: I733b53171dca77f50a30e5bd0bd5f1b456e4c85d
Follows-up b67b9e1, lessphp now has a public method to add to the
list of files for compilation cache.
Change-Id: I62a6c7cdeb34ea742fa7547e3ca10e24ee484b97
'embeddedImages' is a custom property we set on lessc compiler instances to
track the set of files that were embedded in the generated source as data URIs.
This is so done so we know to regenerate the CSS if the source file for an
embedded asset changes. (If my pull request[1] is merged, we won't have to use
a custom property at all.)
I realized that the property name should be 'embeddedFiles' rather than
'embeddedImages', because it's not just images that can be usefully embedded in
CSS, but fonts as well.
[1]: https://github.com/leafo/lessphp/pull/472
Change-Id: Ief3afaa23e4532f4a536e0dfef943d4fa20db80d
This patch adds support for the LESS stylesheet language to ResourceLoader.
LESS is a stylesheet language that compiles into CSS. The patch includes
lessphp, a LESS compiler implemented in PHP. The rationale for choosing LESS is
explained in a MediaWiki RFC which accompanies this patch, available at
<https://www.mediawiki.org/wiki/Requests_for_comment/LESS>.
LESS support is provided for ResourceLoader file modules. It is triggered by
the presence of the '.less' extension in stylesheet filenames. LESS files are
compiled by lessc, and the resultant CSS is subjected to the standard set of
transformations (CSSJanus & CSSMin). The immediate result of LESS compilation
is encoded as an array, which includes the list of LESS files that were
compiled and their mtimes. This array is cached. Cache invalidation is
performed by comparing the cached mtimes with the mtimes of the files on disk.
If the compiler itself throws an exception, ResourceLoader constructs a
compilation result which consists of the error message encoded as a CSS
comment. Failed compilation results are cached too, but with an expiration time
of five minutes. The expiration time is required because the full list of
referenced files is not known.
Three configuration variables configure the global environment for LESS
modules: $wgResourceLoaderLESSVars, $wgResourceLoaderLESSFunctions, and
$wgResourceLoaderLESSImportPaths. $wgResourceLoaderLESSVars maps variable names
to CSS values, specified as strings. Variables declared in this array are
available in all LESS files. $wgResourceLoaderLESSFunctions is similar, except
it maps custom function names to PHP callables. These functions can be called
from within LESS to transform values. Read more about custom functions at
<http://leafo.net/lessphp/docs/#custom_functions>. Finally,
$wgResourceLoaderLESSImportPaths specifies file system paths in addition to the
current module's path where the LESS compiler should look up files referenced
in @import statements.
The issue of handling of /* @embed */ and /* @noflip */ annotations is left
unresolved. Earlier versions of this patch included an @embed analog
implemented as a LESS custom function, but there was enough ambiguity about
whether the strategy it took was optimal to merit discussing it in a separate,
follow-up patch.
Bug: 40964
Change-Id: Id052a04dd2f76a1f4aef39fbd454bd67f5fd282f
Some were using generic wfDebug().
Removed obsolete new-line which is already added by wfDebugLog.
Bug: 44018
Change-Id: I9907b374fa868c04ff2ce40964238936b9084a4a
- A cast statement must not be followed by a space.
- The method parameter $context is never used.
- Avoid function calls in a FOR loop test part.
- Opening brace should be on the same line as closing parenthesis.
Change-Id: I0eba7fcc9ceab372003d1134857346690c525e87
Doxygen expects parameter types to come before the
parameter name in @param tags. Used a quick regex
to switch everything around where possible. This
only fixes cases where a primitve variable (or a
primitive followed by other types) is the variable
type. Other cases will need to be fixed manually.
Change-Id: Ic59fd20856eb0489d70f3469a56ebce0efb3db13
When the database is in read-only mode or writes fail for some other
reason, ResourceLoader should make every attempt to still serve JS and
CSS correctly.
* Surround a bunch of DB write code in MessageBlobStore.php in try-catch
blocks.
* Surround the DB write in ResourceLoaderFileModule.php in a try-catch
block rather than checking wfReadOnly(), because the DB may fail for
other reasons.
* In ResourceLoader::respond() and helpers, set a short cache timeout on
responses that include commented-out error messages.
Change-Id: Idc83a0fe042806263f9337c40ade8c38c56aa3cd
