Install the backtrace collector very early, so that we can get the
backtrace even if headers were sent from LocalSettings.php.
Bug: T157392
Change-Id: I9bc732b34481c95afb5362e135a87bd4302498e2
Follows-up 047b60b96d (ref T111481).
The if-condition compared the expanded paths, not the relative paths.
This meant there were two conditions under which the code will perform
a useless write that inserts *literally* the exact same JSON value.
1. The base directory ($IP) changes after a branch upgrade.
2. Paths contain '../', '//' or other unnormalized paths.
The latter caused various Echo and ULS methods to keep writing the
same value because one of their images is referenced in CSS using
'../'. When inserted in the database as relative path and then
expanded again at run-time and compared to the input value, they
don't match ("$IP/foo/../bar.png" != "$IP/bar.png") and cause a write.
Bug: T158813
Change-Id: I223c232d3a8c4337d09ecf7ec6e5cd7cf7effbff
Currently it was still going through fetchTitleInfo() with an empty array on
the majority of requests without wiki modules, e.g. load.php?modules=jquery.
Bug: T158813
Change-Id: Ie33a2b4da572bb30b2e7a69db07790724ec2f03f
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.
Change-Id: I6f59febaf8fc96e80f8cfc11f4356283f461142a
Previously, style modules were only in a predictable order for production mode.
In debug mode, the order was determined by order in which modules were added
to queue at run time. This made it sometimes hard to debug, especially when
dealing with gadgets that apply in a different order among each other.
Change-Id: I4bff0c91d127e4ad8015cd8c1775220fe460cbc3
Follows-up 1d15085bb3.
The column has a unique index for module name and skin/language pair.
Previously the write lock was on module name, which meant that
shortly after deployment, the following happens:
* Files change on disk.
* (1-5min pass)
* First startup module request after 5min http-cache expires. Detects
one or more changes and updates the version hash of that module.
* Web client subsequently requests this module (if used on that page).
The first time that request comes in, it's a varnish cache miss
and will make RL load all files from disk related to that module
and update the cache index in the module_deps table. At this point
most popular skin/lang pairs fail, except one. As a result, the
other rows remain stale.
* (7-30 days varnish expiry pass OR another change to the module deploys)
* Web client requests this module and tries to update its skin/lang pair
for that module.
One simple change in January 2016 changes jquery.tablesorter to load
a PNG file instead of a GIF file. Now, over a year later, there are
still a dozen skin/lang pairs in enwiki.module_deps with stale data,
which is causing various suble bugs, as well as filesystem calls for
files that don't exist.
Ref T113916 (refactor module_deps).
Ref T158105 (stale cache bug).
Bug: T158105
Change-Id: Ib6c024bfa8d35ce2d622ba4242291daedb507d5e
This should perform better and reduce internal lock contention on the
database server.
Bug: T158105
Change-Id: I1acfb0630946283b317cb929e8d7c3b2af757ecf
Use of &$this doesn't work in PHP 7.1. For callbacks to methods like
array_map() it's completely unnecessary, while for hooks we still need
to pass a reference and so we need to copy $this into a local variable.
Bug: T153505
Change-Id: I8bbb26e248cd6f213fd0e7460d6d6935a3f9e468
Make ResourceLoader error formatting the same as everywhere else.
Which means if wgShowExceptionDetails is enabled locally, the
trace will be included as well.
This matches logic in MWExceptionRenderer.
Also move the typical error handling used by respond() to a
utility method to reduce duplication of code and avoid mistakes.
Change-Id: If04ae99618e4a758ed0f9dd2b555496b76da29de
When getScript (or some other method used in a module response)
throws an error, only that module fails (by outputting mw.loader.state
instead of mw.loader.implement). Other modules will work.
This has always been the case and is working fine. For example,
"load.php?modules=foo|bar", where 'foo' throws, will return:
```js
/* exception message: .. */
mw.loader.implement('bar', ..)
mw.loader.state('foo', 'error')
```
The problem, however, is that during the generation of the startup
module, we iterate over all other modules. In 2011, the
getVersionHash method (then: getModifiedTime) was fairly simple
and unlikely to throw errors.
Nowadays, some modules use enableModuleContentVersion which will
involve the same code path as for regular module responses.
The try/catch in ResourceLoader::makeModuleResponse() suffices
for the case of loading modules other than startup. But when
loading the startup module, and an exception happens in getVersionHash,
then the entire startup response is replaced with an exception comment.
Example case:
* A file not existing for a FileModule subclass that uses
enableModuleContentVersion.
* A database error from a data module, like CiteDataModule or
CNChoiceData.
Changes:
* Ensure E-Tag is still useful while an error happens in production
because we respond with 200 OK and one error isn't the same as
another.
Fixed by try/catch in getCombinedVersion.
* Ensure start manifest isn't disrupted by one broken module.
Fixed by try/catch in StartupModule::getModuleRegistrations().
Tests:
* testMakeModuleResponseError: The case that already worked fined.
* testMakeModuleResponseStartupError: The case fixed in this commit.
* testGetCombinedVersion: The case fixed in this commit for E-Tag.
Bug: T152266
Change-Id: Ice4ede5ea594bf3fa591134bc9382bd9c24e2f39
Changes:
- uses int instead of number as param and return value type,
- uses stdClass instead of stdObject
- fixes ResourceLoaderClientHtml constructor's $target param type:
it is string|null, not an array (previously misspelled as "aray")
- changes the type of references to XML parser in XMP lib to resource
instead of not existing XMLParser
Change-Id: I98c363ebc6658d1f4dcabad97a9a92f3fcd7ea8c
This is a pure documentation change. It mostly removes empty lines from
comments (and entirely empty comments), as well as adds a few missing
documentation blocks and fixes a minor mistake. I hope it's ok to have
this in one patch. I can split it, please tell me.
Change-Id: I9668338602ac77b903ab6b02ff56bd52743c37c4
* The styles queue has always been top-only
(except for a few months in 2015).
* The top queue loads asynchronous since mid-2015. (T107399)
And LocalStorage eval, previously the last remaining non-async part
of module loading, is also async as of October 2016. (T142129)
* This change merges the bottom 'mw.loader.load()' queue with the top queue.
It also moves any other snippets potentially in the bottom queue still:
- embed: I couldn't find any private modules with position=bottom
(doesn't make sense due to their blocking nature). If any do exist,
(third-party extensions?), they'll now be embedded in the <head>.
- scripts: Any legacy 'only=scripts' requests will now initiate
from the <head>.
Bug: T109837
Change-Id: I6c21e3e47c23df33a04c42ce94bd4c1964599c7f
This change resulted in unreasonable feature loss (human-readable
limit report was gone). Three months and multiple followups later,
the functionality is still not completely restored. Given lack
of response from the original author, I think it is time to revert
and reconsider, especially since the 1.28 release is soon.
A machine-readable limit report would be a very useful feature,
but not at the cost of losing human-readable limit report.
This reverts the following commits:
* Move NewPP limit report HTML comments to JS variables
b7c4c8717f
* Only pretty-print the parser report JS vars
28adc4d7ee
* Show wgPageParseReport on page previews too
1255654ed5
* Re-add human readable parser limit report
0051f108b9
* Restore hooks.txt for ParserLimitReportFormat
4663e7a737
Resolved minor merge conflicts in OutputPage (with 80e5b160)
and release notes.
Bug: T110763
Bug: T142210
Change-Id: Id88c8066fae3f369e8977b4b7488f67071bdeeb7
Use HTTPS instead of HTTP where the HTTP link is a redirect to the HTTPS link.
Also update some defect links.
Change-Id: Ic3a5eac910d098ed5c2a21e9f47c9b6ee06b2643
It looks like there is something missing after the last statement
Also remove some other empty lines at begin of functions, ifs or loops
while at these files
Change-Id: Ib00b5cfd31ca4dcd0c32ce33754d3c80bae70641
* Load the data of this variable from a JSON file to the same
data structure that ResourceLoader uses for digitTransformTable,
pluralRules, etc.
* Change the JSON structure to ensure the order of the rules.
Otherwise JavaScript processes the keys in a random order.
* Delete the grammar code from JS and replace it with
the same logic that is used in PHP for processing the data.
For now this is done only for Russian.
The next step will be to make the PHP and JS
data processing logic reusable.
Bug: T115217
Change-Id: I6b9b29b7017f958d62611671be017f97cee73415
* Remove filter between $moduleStyles and preloadTitleInfo().
Previously, this was filtering out wiki modules created by
gadgets and extensions. Still causing a couple of direct queries
from getTitleInfo().
* Store __METHOD__ in $fname outside getWithSetCallback.
This way queries are logged as either getTitleInfo or preloadTitleInfo.
This regressed in 0852a000a5, after which the latter was logged
as "::{closure}".
Change-Id: I454e43f43f8ad3270b86f4cfdbd68192c305cc07
This is one of the top three DB queries showing up in xenon
reverse flamegraph profiling.
It works via a per-wiki check key that is bumped whenever
someone changes a .js or .css page on that wiki.
Change-Id: I73f419558864ba3403b4601a098f6aaf84a3e7c1
Should be useful for cases where we pull in an external library
that already manages RTL flipping in its styles, and CSSJanus
does the wrong thing without extra markup.
Example:
'ext.tmh.video-js' => $baseExtensionResource + [
'scripts' => 'resources/videojs/video.js',
'styles' => 'resources/videojs/video-js.css',
'noflip' => true,
...
],
Bug: T148572
Bug: T148565
Change-Id: Icbad20d8a6e9a0d354ad159f5816f4fb67cc2775
Follows-up 6fa1e56. This is already fixed for http caches by
shortening the Cache-Control max-age in case of a version mismatch.
However the client still cached it blindly in mw.loader.store.
Resolve this by communicating to the client what version of the module
was exported. The client can then compare this version to the version
it originally requested and decide not to cache it.
Adopt the module key format (name@version) from mw.loader.store
in mw.loader.implement() as well.
Bug: T117587
Change-Id: I1a7c44d0222893afefac20bef507bdd1a1a87ecd
Utility methods like getFuzzyBool already use getRawVal instead
of getVal (see 83df6d8). But there was still direct use of getVal
which meant we still pay for WebRequest normalisation in load.php.
ResourceLoaderContext::__construct's use of WebRequest::getVal
accounts for 0.12% of execution time according to a Xenon daily log
(2016-09-27.all.reversed.svgz). The total use of WebRequest::getVal
is 0.26% of execution time. So almost half of that comes from RL.
Change-Id: Id37619ceadb5c990abf83f7910104cbc95523d28
Follows-up dbe592df. Used to go in one-way only and silenced by
addObj(). Now that we're iterating directly as well, need a
conditional.
Bug: T146728
Change-Id: Ief9d44cc5cb8f5d687560f59cc696c5671546163
This class can be re-used on non textarea inputs
(e.g. ContentEditable or <input>).
The 'default' preference can't be achieved in CSS alone, so
for non-textarea cases, use 'monospace' as is used in the majority of
browsers/OSes. In future a client script could compute the default
font of textarea and add it to the -default rule.
Bug: T145015
Change-Id: I5f0433e9cae6f692396537412c11b79b61db062f
* Keep this out of makeLoaderImplementScript() to keep it more generic
and to simplify future refactoring.
* Remove now-broken test case that asserted that the output varies
by global debug mode.
* Make the test responsible for wrapping in XmlJsCode. Previously
this magically happened because the module name was "user" in the
last test case.
* Make makeLoaderImplementScript protected. It's not used anywhere
outside ResourceLoader and we should keep it that way.
Test plan:
* Verify output unchanged:
- load.php?modules=user&only=scripts&user=Admin (raw code)
- load.php?modules=user&user=Admin (implement with unwrapped string)
- load.php?modules=jquery.client (implement with closure)
Change-Id: I527d01926fb6e4ce68c931695d830cdb9ceb608c
Ensure that messages created using ResourceLoaderContext::msg() have a
title set so they don't trigger the GlobalTitleFail log. We use a dummy
title since there is no real title we can use here, and the cache
doesn't vary on a title anyways. This particular title was picked
especially for Roan.
This patch should quiet the GlobalTitleFail logs for
MFResourceLoaderParsedMessageModule and VisualEditorDataModule.
Change-Id: I502faa22776e1cb34a6ef17be96567f121c80081
Follows-up 6f8dc27ca2 and dbd11e04aa. You'd expect a bug in preloading
to fallback to fetching it on-demand but due to a title format mismatch
the preload method did use the same title format for the cache key, but
not the same title format for discovering the results. As such, it set
the right cache key to an empty array.
* Make relevant methods testable and mockable.
* Add regression test.
* Change call to array_interect_key to use the same format as
fetchTitleInfo(): Normalised title keys from Title::getPrefixedText.
Previously, the intersect used the declared titles from getPages() which
are not localised - causing an empty array to be returned from the
intersect on wikis where the namespace name is localised.
Bug: T145673
Change-Id: Ibe788157724d73c727b9e2127b6828db32ca9420
Test plan:
* Request load.php?modules=startup&only=scripts
* Debug log receives 1 entry in the 'queries' channel from
ResourceLoaderWikiModule, and it comes from preloadTitleInfo.
Bug: T46362
Change-Id: I82e215745af6b8446cd69282b2c943afbbd45065
While rev_sha1 was preferred to rev_id (page_latest) to allow
client-cache to be re-used in case of a bad change and revert,
this is no longer possible since we recently added support for
explicit purging by considering page_touched.
As such, use of rev_sha1 is no longer useful.
Change-Id: Iddb65305ca4655098fdea9fcf736fd4046035dd7
makeVersionQuery and getCombinedVersion return the same string in
most cases, but there are exceptions, and it could diverge further
in the future. Use the semantically correct method.
Before 6fa1e56 it didn't matter how 'version' was computed as long
as it's deterministic and sufficiently unique. Now that we validate
the hashes it's important all methods use the same logic.
Rename method to makeVersionQuery since it's no longer used just
for comparison against the expected value.
Change-Id: I19f5818e27c8a0920d3d1374b40aeb0b6ba0b614
Avoid manually crafting a url. The packing logic and key sorting logic
is already in this method and keeps it central that way.
Test plan:
* Check output of /w/load.php?modules=startup&only=scripts.
This is used to construct the url to 'jquery%2Cmediawiki'.
No change in output.
Change-Id: I51ba51b6e74d78761108341c0ee318699ca7952f
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.
The old constant is an alias now.
Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
Versioned load.php requests (load.php?modules=..&version=..) are highly
cacheable due to being versioned. When a module changes, the startup module
delivers new metadata to the client which naturally results in a cache miss
by producing a different url.
However, during upgrades and other deployments it was possible for a
multi-server installation to pollute edge caches with outdated content
in the following scenario:
* Deployment starts for an upgrade from version A to version B.
* Server 1 (on version B) gets a request for the startup manifest.
Client receives new manifest with version B information
and makes a versioned request for version B.
* Server 2 (still on version A) responds to this request with the
version it has (which is version A).
* Edge cache will store version A under the new url for version B.
This commit changes the last point by setting a low max-age when a request
has a 'version' hash mismatch.
Test plan:
* Look for a request in your browser like:
'/load.php?..modules=jquery%2Cmediawiki..&version=..'
* Verify it has a high Cache-Control max-age.
* Manipulate the 'version' parameter and verify it gets a low max-age.
Bug: T117587
Change-Id: Iba89c09b7b71d9fd2a8ff3ebe2618e26ea9daddf
* Fix up one last use of global config vars in this class.
Other places in this class already used $rl->getConfig().
This way we don't inherit all of MediaWikiTestCase.
* Add unit tests covering all of ResourceLoaderContext
except expandModuleNames and getImageObj (tested in better
places already with the right @covers).
* Increase coverage for expandModuleNames(), add missing case
of when modules are not in alphabetical order.
Change-Id: Id19b084d37a6c3a77b36e03509adffb6b156fee1
Before 93ed259cf and 7bb7315d4, 'site' and 'site.styles' module were one module
loaded in two parts. One part via the general queue (not in a separate request),
and the other part in the stylesheet queue (in a separate request).
This was achieved by hacking the stylesheet queue handler to hardcode the group
to be 'site' for a module named 'site' (later 'site.styles' in c20a76ee)
eventhough the module as a whole was not actually in that group.
Now that these modules are separated, this hack is no longer needed
and the module can be registered as being in group 'site' properly.
Change-Id: If193f556be211749b115f3808ee46536b578fb22
* Fix signature of makeLoaderSourcesScript() to match
the change in behaviour since e103ba265.
* Consistently order providers before the test.
* Simplify testRegisterValid() and remove needless @depends.
* Remove unused private method stripNoflip().
Coverage:
* Expand test coverage for register().
* Add tests for getModuleNames().
* Add tests for getModule().
* Expand test coverage for addSource().
(case of invalid array)
* Expand test coverage for makeLoaderImplementScript().
(case of unwrapped user script, and case of invalid scripts)
* Add tests for makeLoaderSourcesScript().
Change-Id: Ibca3e486fcd3664f171f135327a0f340ee6da9ee
Remove use of the deprecated getModifiedTime() method and incorporate
file information in getDefinitionSummary() directly, instead of relying
on the parent class to include getModifiedTime().
Change getDefinitionSummary() to append to the summary instead of
setting arbitrary keys in the array directly to avoid conflicts.
This matches the pattern used elsewhere.
Change ResourceLoaderImage to use file hashes instead of timestamps
to avoid needless cache invalidation. At Wikimedia these modules
tend to roll over every week due to git not storing timestamps.
See also T104950, 9112c9347b, f37cee996.
Change-Id: I5d019bfb991c3b8042d1db14a853ba46cc690315
Follows-up 80e5b160e, which moved filter logic to getRlClient()
so that ResourceLoaderClientHtml doesn't have to deal with this.
While that worked fine for most modules (including 'site'),
it no longer applied the filter for exempt/hardcoded modules
(such as 'site.styles' and 'startup').
Bug: T143066
Change-Id: Iec924003873bc47484a0dc2f1a215f87aa4afdfb
HTML formatting of the queue was distributed over several OutputPage methods.
Each method demanding a snippet of HTML by calling makeResourceLoaderLink()
with a limited amount of information. As such, makeResourceLoaderLink() was
unable to provide the client with the proper state information.
Centralising it also allows it to better reduce duplication in HTML output
and maintain a more accurate state.
Problems fixed by centralising:
1. The 'user' module is special (due to per-user 'version' and 'user' params).
It is manually requested via script-src. To avoid a separate (and wrong)
request from something that requires it, we set state=loading directly.
However, because the module is in the bottom, the old HTML formatter could
only put state=loading in the bottom also. This sometimes caused a wrong
request to be fired for modules=user if something in the top queue
triggered a requirement for it.
2. Since a464d1d4 (T87871) we track states of page-style modules, with purpose
of allowing dependencies on style modules without risking duplicate loading
on pages where the styles are loaded already. This didn't work, because the
state information about page-style modules is output near the stylesheet,
which is after the script tag with mw.loader.load(). That runs first, and
mw.loader would still make a duplicate request before it learns the state.
Changes:
* Document reasons for style/script tag order in getHeadHtml (per 09537e83).
* Pass $type from getModuleStyles() to getAllowedModules(). This wasn't needed
before since a duplicate check in makeResourceLoaderLink() verified the
origin a second time.
* Declare explicit position 'top' on 'user.options' and 'user.tokens' module.
Previously, OutputPage hardcoded them in the top. The new formatter doesn't.
* Remove getHeadScripts().
* Remove getInlineHeadScripts().
* Remove getExternalHeadScripts().
* Remove buildCssLinks().
* Remove getScriptsForBottomQueue().
* Change where Skin::setupSkinUserCss() is called. This methods lets the skin
add modules to the queue. Previously it was called from buildCssLinks(),
via headElement(), via prepareQuickTemplate(), via OutputPage::output().
It's now in OutputPage::output() directly (slightly earlier). This is needed
because prepareQuickTemplate() calls bottomScripts() before headElement().
And bottomScript() would lazy-initialise the queue and lock it before
setupSkinUserCss() is called from headElement().
This makes execution order more predictable instead of being dependent on
the arbitrary order of data extraction in prepareQuickTemplate (which varies
from one skin to another).
* Compute isUserModulePreview() and isKnownEmpty() for the 'user' module early
on so. This avoids wrongful loading and fixes problem 1.
Effective changes in output:
* mw.loader.state() is now before mw.loader.load(). This fixes problem 2.
* mw.loader.state() now sets 'user.options' and 'user.tokens' to "loading".
* mw.loader.state() now sets 'user' (as "loading" or "ready"). Fixes problem 1.
* The <script async src> tag for 'startup' changed position (slightly).
Previously it was after all inline scripts and stylesheets. It's still after
all inline scripts and after most stylesheets, but before any user styles.
Since the queue is now formatted outside OutputPage, it can't inject the
meta-ResourceLoaderDynamicStyles tag and user-stylesheet hack in the middle
of existing output. This shouldn't have any noticable impact.
Bug: T87871
Change-Id: I605b8cd1e1fc009b4662a0edbc54d09dd65ee1df
Follows-up ea862efa7, which enables use of Less_Cache as part
of the switch from leafo/lessphp to oyejorge/less.php.
Since then 12afb3607 introduced a better caching method in
FileModule::compileLessFile(), which:
* Uses APC where possible, with fallback to main objectcache.
* Verifies both the input file and the indirectly used files have
not changed before using the cache. The built-in cache system
only seems to check the root file.
As such, we're incurring the overhead of generating them and storing
on-disk, but raraly ever making use of them. (And even if we do
use it when the APC object first expires, it's risky due to the
lack of indirect file validation).
Bug: T121519
Change-Id: I6e3c66f86a536463488697efc166fb704be0715c
ResourceLoader modules can now carry a 'deprecated' option which can
be a boolean or an object with message key. This message or a default
deprecation message will be show whenever that module is used in production.
Note: This will not work in debug mode for ResourceLoaderFile modules
and this is deemed acceptable for the time being. We can revisit later.
Bug: T137772
Change-Id: Ib9ebd2d39a59fd41d8537e06884699f77b03580c
Fixes addModuleStyles() violation from T92459 (a464d1d41).
Similar to 93ed259cf and Id2342454b for the 'site' module.
Doesn't need to be in separate commits since per-user HTML isn't
cached the same way.
Bug: T92459
Bug: T108590
Change-Id: I195f67d061de1306c97413aada7919e9f1b1d12c
mw.Title.newFromFileName() will now correctly use $wgIllegalFileChars,
rather than partially and incorrectly duplicating it.
$wgIllegalFileChars is currently defined to contain ':', '/' and '\'.
Previously we incorrectly did not filter out backslashes '\'. The hash
'#' is already disallowed by $wgLegalTitleChars.
Bug: T140222
Change-Id: I17e3ef61e04ae002ef576c9416c8f265ee000e89
For legacy reasons, the site module has always been both a script
module and a page-style module at the same time. Meaning, the styles
and scripts portion of it don't directly relate to each other, rather
they both relate to the page directly. As such, they should be
loaded separately.
Loading it as a dynamic module is not an option as its styles should
be top-loaded and not depend on JavaScript. The restriction that
will be put it place for T92459 also requires the module be split.
For environments that use static file or proxy caching, this MUST
be deployed after Ic137cb494 and I8b6c6a1 have been deployed and all
cache has rolled over.
Bug: T92459
Bug: T108590
Change-Id: Id2342454bdc87a7c4d194f7350caa699ec4054b9
* Instead of having messy code to create a hidden HTML
comment of English strings at the bottom of the page,
expose the structured data of the parse information
to JS so tools can use it.
* Make makeConfigSetScript() use pretty output so these
variables are also easy to read in "view source".
* Remove ParserLimitReportFormat hook, since the data
is not formatted to HTML anymore.
Bug: T110763
Change-Id: I2783c46c6d80f828f9ecf5e71fc8f35910454582
This allows dynamically loaded modules to depend on page-style modules
without it causing the page-style module to be loaded a second time.
* New method Module::getType() indicates whether a module is
a page-style module or supposed to be dynamically loaded.
* Emit warning from addModuleStyles() when given a module that is
not a page-style module (to be enforced later)
Bug: T92459
Bug: T87871
Change-Id: I8b6c6a10d965e73965f877c42e995d04202524f3
SHA-1 is not secure enough to be used as a cryptographic hash function, and its
implementation in JavaScript is too long and too slow for it to be a good
general-purpose hash function. And we currently throw away most of the work:
SHA-1 produces 160-bit hash values, of which we keep 48.
Although the JavaScript implementation is not exported, SHA-1 is a well-known
hash function, and I'm willing to bet that sooner or later someone will move to
make it accessible to other modules, at which point usage will start to spread.
For ResourceLoader, the qualities we're looking for in a hash function are:
* Already implemented in PHP
* Easy to implement in JavaScript
* Fast
* Collision-resistant
The requirement that hashes be cheap to compute in JavaScript narrows the field
to 32-bit hash functions, because in JavaScript bitwise operators treat their
operands as 32 bits, and arithmetic uses double-precision floats, which have a
total precision of 53 bits. It's possible to work around these limitations, but
it's a lot of extra work.
The best match I found is the 32-bit variant of FNV-1, which is available in
PHP as of version 5.4 (as 'fnv1a32'). The fnv132 JavaScript function is
around ten times faster and eight times shorter than sha1.
Change-Id: I1e4fb08d17948538d96f241b2464d594fdc14578
No longer needed per doing Ic137cb494ba23 in a different way.
This may be useful to revisit, but for now preferring to keep
simplicity and removing this unused option.
This reverts commit 9e217bf42d.
Change-Id: I9c0c316a3b58a3d0a3d3282dd74c7fa4eef8e378
To be used for the 'site' module. This will make it easier to split up
'site' into 'site' and 'site.styles' (per T92459).
Change-Id: Iaac3e458d5107e4c10c2826bd64608d5c47e8b87
We read files and concatenate their contents. Files may start with a BOM character.
BOM characters are only allowed at the beginning of a file, not half way.
Stripping it should be safe, since we already assume that everything is UTF-8.
Change-Id: I14ad698a684e78976e873e9ae2c367475550a063
This is with T92459 in mind to simplify the process of splitting
the 'user' module for the styles-only queue.
Consequences:
* Cached HTML isn't relevant in practice since there is no caching for logged-in
users and this module is only for logged-in users. Even then, cached HTML will
work and may happen as browsers re-use HTML responses when revisiting a
privately cached page (after 304 Not Modified).
Note that OutputPage (via isKnownEmpty) only actually tries to load 'user.groups'
if the wiki has 'MediaWiki:Group-*.{js,css}' pages for the current user's groups.
- Old style queue request will continue to ask for user.groups which is now a
FileModule with no styles (simply concats the empty string to the bundle)
- Old load() request will resolve with an empty function.
* The are no known dependants of 'user.groups'. If there are, they will work
by proxy of it now being an empty module that just ensures 'user' is loaded.
* The security origin of 'user.groups' was USER_SITEWIDE. The origin of 'user'
is lower (USER_INDIVIDUAL). Pages that are restricted to USER_SITEWIDE
previously received user.groups, but won't anymore. This should be fine as
OutputPage::reduceAllowedModules() is mainly used to either allow everything
or restrict all the way down to CORE. The only exception is disallowUserJs()
if $wgAllowSiteCSSOnRestrictedPages is enabled (T73621) but that edge case was
made for Common.css, not Group-*.css.
Change-Id: I74cd2368ebd2989c5e1c22bea491a80beb0319dc
This aims to solve all the problems and fulfill all the use cases.
It allows the dialog to be configured for Wikimedia Commons without
hardcoding anything, and it should be flexible enough for third-party
use. The default configuration should be sane for any wiki.
The file upload dialog can be configured using $wgUploadDialog.
See DefaultSettings.php for documentation. Example configuration for
Wikimedia Commons: Id56370e2334c8fe34e88180356232b48c244b7c4.
Configuration is loaded using ResourceLoaderUploadDialogModule for
local uploads or using ApiQuerySiteinfo (action=query&meta=siteinfo)
for uploads to a foreign wiki. Custom localisation messages may be
loaded using action=query&meta=allmessages.
Renamed messages:
upload-form-label-own-work-message-local -> upload-form-label-own-work-message-generic-local
upload-form-label-not-own-work-message-local -> upload-form-label-not-own-work-message-generic-local
upload-form-label-not-own-work-local-local -> upload-form-label-not-own-work-local-generic-local
upload-form-label-own-work-message-default -> upload-form-label-own-work-message-generic-foreign
upload-form-label-not-own-work-message-default -> upload-form-label-not-own-work-message-generic-foreign
upload-form-label-not-own-work-local-default -> upload-form-label-not-own-work-local-generic-foreign
Deleted messages, moved to WikimediaMessages in Id2977e19330aeaf854157d4355cd17e5dc72f16a:
upload-form-label-own-work-message-shared
upload-form-label-not-own-work-message-shared
upload-form-label-not-own-work-local-shared
Bug: T118097
Bug: T120998
Bug: T121632
Bug: T121633
Bug: T127895
Change-Id: I3017b8f09c27625deb7a92d6f667895b71cc0637
Most code paths already wrongly assume this behaviour.
This patch removes the now-redundant check from the two modules
that did account for it.
Change-Id: Ic85258f184db8aa68e19d8e667e396d08ee5de0c
Follows-up da36f65433 which added this method with intent to use in
ResourceLoader::register(). However, the feature was redesigned to
not need this and the method was left behind.
Change-Id: I5ebc93805d0df6605bec94094bcd4eb2b70ff18d
* Send 'module' and 'require' parameters to module closures.
This depends on Ia925844cc22f143 being deployed one cycle earlier.
* Patch Moment and OOjs to ensure these libraries continue to expose
their module as globals as well. AMD/UMD-compatible libraries
only expose a global *OR* an export, not both. We need both
for back-compat.
* Update pluralRuleParser to make use of module export to allow
usage via require().
To test, check out the patch and run:
> mw.loader.load('moment');
> mw.loader.require('moment')()
> mw.loader.require('moment')('2011-04-01').fromNow()
Bug: T108655
Change-Id: Idbd054880ee70d659ec760aef8fcb38d0704a394
Purging a wiki page included in a WikiModule should bump its version hash.
This is not needed by design, but it's a workaround until our caching layers
are more resilient against cache poisoning.
This gives users the power to resolve issues themselves.
Change-Id: I37d3379e76a6b652268c5a007e0ad741470fe6e1
$wgForceUIMsgAsContentMsg forces Message::inContentLanguage() to not
actually do anything, thus falling back to the session user's language
which we can't use in ResourceLoader. And 'mainpage' is added to that
array on various multi-lingual wikis.
Bug: T127233
Change-Id: I9dc1e57922641881b36c70658083c2c8c8a608a0
Follows-up b5e69c4ef which did the same for already for ResourceLoader.php.
There's no need to expand the hostname into these urls. If load.php
is on the same domain (e.g. set to '/w/load.php') then this can be
resolved by the browser normally.
If load.php is on a custom domain, the stylesheet would be served
from there instead of from the wiki domain. However in that case
the hostname would already be part of $script value ($wgLoadScript).
Bug: T125292
Change-Id: I7242445335d69d7ae290da5f321a59edd537d819
* Remove trailing space in self-closing tag.
Brings parserTest output of Parser and Parsoid closer together.
* Remove various line breaks at begin and end of script contents.
* Remove FILTER_NOMIN from makeConfigSetScript() output.
This isn't part of any user- or page-dependent module and not minified.
And Xml::encodeJsCall already ensures compact output for prod mode.
Bug: T127328
Change-Id: I85a5a59fd0955c1a112e8b24b933f0d9e983a156
This was always false since introduction in 3c72b527 released in MediaWiki
1.19, and deprecated in 0ac4f998, released as part of MediaWiki 1.26. Any
code that still needs the mediawiki.util module can continue to use it via
expressing this dependency through their ResourceLoader manifest as usual.
Bug: T111077
Change-Id: Ic838af8727476c047f01ef0dbbeb952c85e263e1
This makes it easier to fetch messages without having to manually
call inLanguage() on each wfMessage() call, which is currently
causing some code forget this and use plain wfMessage() which
defaults to MediaWiki user language from session.
I've considered setting $wgLang or RequestContext::getMain(),
but that's still bad since modules get passed a context in their
methods and shouldn't be using global context either.
The warnings provided by MW_NO_SESSION are exactly what we want.
Change-Id: I1288fa5622d9f82d21bb66c8eb6518b90e7cddb4
Follows-up 8f5cd11d82.
The old getLocalFileReferences() method is no longer used anywhere.
Remove it and rename getAllLocalFileReferences back to it.
Change-Id: I864258aad128ba9b54464c7bc854543f2937f977
The output of this function is not deterministic. This function
is what tranforms urls to include content hashes and also embeds
content as data URI. The calls to this function are expected to be
up to date and matching the version hash as constructed by the
current web request.
Previously, the content hash as constructed based on the collective
file hash of all references images was accurate but the generated
css associated with it could be stale.
Unless we abuse MemoizedCallable to take a separate cache buster
(with $localFileRefs?), it's best to remove this as it isn't
a deterministic function and shouldn't be memoized.
Bug: T128668
Change-Id: Icb87ddc585d7320ac48619446bb8b9cbe5f4780e
Remove optimization that avoids unstubbing $wgUser because $wgUser
is not a plain object for the current user name.
It's a stub with mFrom='session' until methods are called that need
the info, such as getName(), which then lazy loads the object based on
the session information.
We want to make load.php session-less.
Bug: T127233
Change-Id: Ica482e5d1892cb29456e6f2a91cd70017cf414c5
This way, they automatically bypass any stale client or server cache.
Remove an old wfExpandUrl() call that would interfere with
transformResourcePath()'s ability to recognise the path.
Expanding the url there wasn't needed anyway. Whether it's on a separate host
(eg. bits.wikimedia.org), or path-absolute, either way works as-is.
Bug: T90983
Change-Id: I64eb3291adcfc3733ef96690399c09c06e19b3aa
* Fix errors spotted by new release
* Introduce "composer fix", which uses phpcbf to automatically fix some
errors spotted by phpcs.
* Drop $PHPCS_ARGS variable that didn't work on Windows, and add -s flag
* Remove rules from phpcs.xml that are now in MW-CS ruleset.
Change-Id: I13e2155695918c918b67497ac65b85a03897095e
We don't want it to accidentally end up in MediaWiki 1.27 release.
It can be restored again when needed.
This reverts commit d0e47d475c.
Bug: T120867
Change-Id: Ie1a90ad2d2ccdecf189313e18c3c5b24576363f4
Keep in CSSMin as-is for back-compat and to ensure library remains
independent of MediaWiki.
Moved down a few lines as there is no need to compute the md5 hash when we're
returning a data URI. Previously md5_file was called twice during module builds
(once for the fallback url, and another time when producing the embedded data uri).
Applied to logo in SkinModule as example. To be applied elsewhere as needed.
Without it, fallback is current behaviour (no cache invalidation).
Bug: T99096
Change-Id: I7f38bfc1bea5c241bc4f8ec4f4b640fd65f2c04f
Some users are reporting that DomDocument::load can't read the files
on their setups, while they can be read with file_get_contents. So use
that and pass the string to DomDocument::loadXml. The advantage of
DomDocument::load is supposed to be in handling large files, and the
icons here are supposed to be small.
Bug: T107198
Change-Id: I8e4dc4642f9d0c5f01ec5e4061e83bf09d0a4900
startup.js:
* Don't assign to "window.RLQ" and read from "RLQ". Use the same
identifier in both places instead of relying on global scope.
* Create the global window.RLQ only once. The temporary [] fallback
doesn't need to be exposed globally, so use a local variable for
the loop that processes the pre-existing queue built up while
the startup module was loading.
OutputPage:
* Simplify the RLQ wrap to be more idiomatic and less repetitive.
Matches the pattern used in Google and Facebook open-source projects.
Change-Id: I9176377bc05432e51add841696a356428feec8ce
Cache Less variables in the instance instead of statically.
This allows tests to populate their own less variabless via ResourceLoaderLESSVars.
Make getLessCompiler() and getLessVars() regular public methods and
update callers.
Change-Id: I95506b8bb20a4b2b3f82014a7b0fcee5f28973e6
By default user option 'underline' is '2' which means no skin override.
Once a upon a time, none of the popular MediaWiki skins specified whether links should have
underlines, and most browsers default to having an underline on links (when not hovering).
As such, there was an exception added for languages where underlines cause text to be unreadable.
Nowadays however all modern skins explicitly specify the intended design for links.
Neither the default skin (Vector) nor other popular skins I checked have underlines by default.
As such, this rule is redundant.
If a skin comes along that does specify underlines by default, then these rules should be placed
in that skins' stylesheet. Not in the HTML of all pages.
Incidentally, this was the last code branch in ResourceLoaderUserCSSPrefsModule that is true by
default. Which means from now on this <style> tag is automatically omitted from the HTML for
logged-out users and users that haven't changed the 'underline' or 'editfont' preference.
Implement the missing isKnownEmpty() method so that OutputPage does the automatic omission.
To test the new text-decoration rule in mediawiki.skinning:
* Change user preference language to "ar" and observe anchor links in the interface
not having underlines on-hover.
* Change $wgLanguageCode to "ar" and observe anchor links in the content not having
underlines on-hover.
History:
* 2011 (043f98a3; r101445) Move overrides from shared.css to ResourceLoaderUserOptionsModule
Also remove "!important" from underline css
* 2011 (33293c9b; r96261) Apply "!important" to underline user option css
* 2011 (81446abc; r91432; T31712) Move overrides from MessageXX.php to skins/common/shared.css
* 2006 (43b2fb56; r15823) Move overrides from LanguageXX.php to MessageX.php
* 2004 (f9f3e915) Add override to LanguageAr.php
Bug: T105313
Change-Id: I94dda30fb80b30e9c12a74a8b09065da55681929
It turns out that people click the checkbox affirming that they are
they author of the file and that they release it under CC BY-SA even
when neither of those is true. So we're trying some interfaces that
require a modicum of thought rather than just a click on "I agree".
Option 1: The form we have right now, with a single checkbox.
Option 2: Four checkboxes, each with a label explaining one facet of
the requirements (own work; no pictures of copyrighted work;
educational/useful; irrevocably released as CC BY-SA).
Option 3: Some Yes/No questions structured so that 'Yes' is not
always the right answer to continue uploading.
Option 4: Longer introduction before a single checkbox (as in option
1), with examples of good and unacceptable content.
As only logged in users are able to upload files, we're able to bucket
them into four groups by user ID number. When the user completes a
file upload, the bucket number is saved server-side in a change tag by
the companion patch I90cb12c505b2581f36113ec6b4f7bf732f0971b7 (we could
match the user IDs cross-wiki by username, but that sounds painful).
For testing and debugging, add '?uploadbucket=N' to the URL to force
given interface option to appear. Any completed upload won't count
towards the bucket.
Note that for expediency, the tested options all assume uploads to
'shared' repository (that is, Wikimedia Commons). The winner's
messages will be tweaked to work with 'local' and other targets too.
This patch DOES NOT ENABLE THE TEST yet, it just implements the options.
Enabling it on specific wikis can be done via config:
* $wgForeignUploadTestEnabled = true/false (defaults to 'false')
Whether the test is running.
* $wgForeignUploadTestDefault = 1/2/3/4 (defaults to '1')
Interface to use when the test is not running (and for anons).
Bug: T120867
Bug: T121021
Change-Id: I557056b867c6a55ef2c9af321eb48893312632a3
23cfe1bba changed ResourceLoaderImageModule::getCssDeclarations(),
but contrary to the documentation did not synchronize it with
mediawiki.mixins.less. This commit does that.
This commit brings the -o-linear-gradient rule back (this time with
a comment that explains why it's there), and removes the
-webkit-linear-gradient rules from mediawiki.mixins.less
and CSS code that copypasted it.
Change-Id: Ia7e9738d90919248748e6968192bf02e420cddd0
MessageBlobStore class:
* Make logger aware.
* Log an error if json encoding fails.
* Stop using the DB table. WANObjectCache supports everything we need:
- Batch retrieval.
- Invalidate keys with wildcard selects or cascading check keys.
* Update tests slightly since the actual update now happens on-demand as
part of get() instead of within updateMessage().
ResourceLoader class:
* Remove all interaction with the msg_resource table. Remove db table later.
* Refactor code to use a hash of the blob instead of a timestamp.
Timestamps are unreliable and roll over too frequently for message blob store
because there is no authoritative source. The timestamps were inferred based on
when a change is observed. Message overrides from the local wiki have an
explicit update event when the page is edited. All other messages, such as
from MediaWiki core and extensions using LocalisationCache, have a single
timestamp for all messages which rolls over every time the cache is rebuilt.
A hash is deterministic, and won't cause needless invalidation (T102578).
* Remove redundant pre-fetching in makeModuleResponse().
This is already done by preloadModuleInfo() in respond().
* Don't bother storing and retreiving empty "{}" objects.
Instead, detect whether a module's message list is empty at runtime.
ResourceLoaderModule class:
* Make logger aware.
* Log if a module's message blob was not preloaded.
cleanupRemovedModules:
* Now that blobs have a TTL, there's no need to prune old entries.
Bug: T113092
Bug: T92357
Change-Id: Id8c26f41a82597e34013f95294cdc3971a4f52ae
* Add logger to ResourceLoaderContext for convenient use within modules.
* Group related members within ResourceLoaderContext.
Change-Id: Ifbc3de1b6e0838386735f1573df328d9b331ac37
Follows-up e103ba265b.
This method doesn't need to be backward-compatible as this is already
normalised server-side.
When upgrading from before MediaWiki 1.24 the startup response will
change to the new format and also inline a request to the newer
version of mediawiki.js.
Also update method documentation to be up to date and remove
spurious return statements that aren't used.
Change-Id: I03d1b5666fcddedaa46ecf878ee7636042a5e3be
Follows-up aacdb664a1, which was reverted.
API action=query&meta=tokens has different token types than the old
action=tokens values. Use a map to maintain support in the JavaScript API for
old token types that now fold into the generic 'csrf'.
Aside from core token types, those added by extensions are no longer
actively used from the old token API.
Bug: T72059
Change-Id: Iec3a9f0f51d64d90c81a147cc18097dcf679c7c9
This parameter was introduced last year as a way to extract
bare scripts from modules without any 'mw.loader.state()' suffix.
When ResourceFileCache is used ($wgUseFileCache) this causes
cache pollution as it didn't include getRaw() in the hash.
Change-Id: I7b9f9b6a5756777462395b911abafb62468cbefa
* No effective change. Only code reformatting.
* Logically group related members.
* Break up items one per line in getHash().
Change-Id: I8ccbe9d071a5c39f5c3d36d0d990574fb0ed8d72
The direction is derived from the language code, which is included
already. The method was added for convenience to consuming code,
but including it in the cache key seems pointless.
Main rationale here is runtime performance. getDirection() incurs
Language::factory() and Language::getDir() which require loading
of LCStore files.
Change-Id: I397a1c483203ec2c4903046c9494cae1c9480f8c
Follows-up e9b9e4df7a, which started using target=test as reserved for
Special:JavaScriptTest. However that conflicted with this.
Without this change, the unit tests fail if the MediaWiki install has
$wgEnableJavaScriptTest enabled.
Change-Id: I471017e7aedb74dddb4dc18ee4e89c1706ff09b5
Also remove the -o- prefix which uses 'fallback', as without
that rule it will use 'fallback' anyway.
Bug: T118815
Change-Id: I6a6e3b24b1cda55dfa9675743c39d21e792275c7
This table is not needed because module names and their messages
array are available to the runtime environment at very little cost.
The only purpose it was serving is reverse lookup from message
key to module name (e.g. when MessageCache receives update that
need to propagate to MessageBlobStore). However that is better
achieved by simply looping through modules in PHP. The overhead
of a database is not worth this minor convenience.
MessageBlobStore
* insertMessageBlob: Doesn't need to update msg_resource_links.
* updateModule: Doesn't need to update msg_resource_links.
* getUpdatesForMessage: Reimplement with list from memory
instead of msg_resource_links.
The database table will be removed in If009e2620e59002e1.
Bug: T113092
Change-Id: Ia9131f570001f00c9800b260ac4b3469d54d2784
I have not had the time and wherewithal to write the follow-up code that
actually uses this value. So just remove it for now.
This reverts commit 3d5aca4a40.
Change-Id: Ice9b65f39802b55b3d4ddf801c06d1be3227fd5b
Follows-up r96280 (368dbc5f5b), and r82927 (1e67922842).
Language::isValidCode() (used by index.php) allows a very wide range of values,
which inflates the msg_resource cache quite a bit (T102058). This is a first
step toward locking it down.
This change affects both handling of incoming load.php requests, and the
formatting of request urls by OutputPage. As such, OutputPage will no longer
forward invalid uselang values that are valid for index.php to load.php.
Change-Id: I27857ce5949bc616c7179f5f47b24aa2f6765f5f
My first pass at what became Ie3499d9f2 was to keep the exception for duplicate
registrations with conflicting parameters but downgrade to a warning if the
registration attempts are identically parametrized. I then removed the
exception entirely, but left the if-block, which does not make much sense, as
it means we pass over silently module registrations that set different
parameters, and throw a warning only when they are the same.
Change-Id: I13da33faf2f058ac75ef4b1615c4fafc18f14505
This effectively reverts d6b4d3c537 and declines T97420.
This was previously attempted in b7c0e537eb.
Drop support for position "bottom" for addModuleStyles().
This feature was only recently introduced with the intent
to optimise page load performance, but had an adverse effect.
It increases chances of FOUC due to late discovery of these styles.
It also caused minor problems for some gadgets and extensions
that did not or were unable to set these flags. Some mobile
code was introduced around the same time as this feature and
was never given position=top.
Stylesheets that don't affect initial render or are only needed
on interaction should be loaded via addModules() instead; which
is handled by the asynchronous loader in JavaScript.
Change-Id: Ib9821b7b87cfc8a59062bb6ca358974fdb01ced1
If an attempt is made to register a ResourceLoader module which has already
been registered, instead of bailing with an error, issue a warning instead, and
allow the lattermost set of module parameters to stick.
Because modules are registered from extensions, and because modules are often
migrated from one extension to another as extensions are refactored, and
because the developers of different extensions are not always aware of each
other's work, throwing an error runs a high risk of causing an outage.
Bug: T116628
Change-Id: Ie3499d9f2ff3bf071f1c2b0176e19440ff55211c
Use BagOStuff::makeKey() and BagOStuff::makeGlobalKey() instead of
wfMemcKey() and wfGlobalCacheKey().
Change-Id: Id4e58c0e616b74de220faf13ba7c9ea606fef8c1
* The '.php5' entrypoints were deprecated in I68b1ae842, $wgScriptExtension
in I3690f78bc.
* Drop the associated ResourceLoader configuration variable, too. `mwgrep`
shows no usage in the MediaWiki namespace.
* Keep the scriptExtension configuration parameter for FileRepo for people who
would like to interoperate with older MediaWiki installations that still use
'.php5'.
Change-Id: I17c8a15484b7e82cd5970d34e688109a2aae3840
This seems like a (slightly) cleaner system. We'll need to change
some documentation and add some configuration, but at least this will
be easier to handle.
Bug: T114765
Change-Id: If962fd3066e25e43e745efd29058eae82195bfb1
Using the following command line, I have found doc comments (and
a wfDeprecated() call) mentioning "1.26" when they should mention
"1.27" instead, which I have fixed manually:
git diff -M REL1_26 | grep --color=always -C 10 -iP \
'^\+.*\D1\.26(\D|$)' | aha > oldver.html
Follows-up these commits:
* 047b60b96d
* 169b7b98b5
* 25a44aa3e4
* 2fb2a3f14b
* 3f1e9fa268 [1]
* 8c84af70b6
* c0cb80beac
* d04a92a551
* d3b85592ea
[1] Release notes moved in I195dd1cf.
Because a release note stating that the UserRights hook is deprecated
was added in 37062a0c0d (before the branch cut), this commit does not
change the Hooks::run call that had been changed in 21206c8fbe.
Change-Id: I5a427f003e7e3b4559fe377bcdfdca466a570708
Add a simple class to `libs/` for memoizing functions by caching return values
in APC. I decided not to make this an external library just yet because I see
this as potentially a part of a larger functional programming library.
Doesn't use APCBagOStuff for two reasons: (1) avoid dependency on MediaWiki
code; (2) ability to pass third &$success parameter to apc_store, to
distinguish between cache misses and cached false values.
Use this in ResourceLoaderFileModule to cache CSSMin::remap.
Change-Id: I00a47983a2583655d4631ecc9c6ba17597e36b5f
This is a micro-optimization, but ResourceLoader is so well-optimized at this
point that the call to PCRE accounts for 1.25% of all load.php CPU time. So
might as well making it a tiny bit faster.
Change-Id: Iefab804a6ca6d54ce230958513a3bea44f4e7c62
* Add support for a '/* @nomin */' annotation in ResourceLoader. If present in
JavaScript or CSS, the code will not be minified or cached. This allows
modules like the ResourceLoaderUserTokensModule to declare themselves unfit
for minification / caching without requiring a complicated refactor.
* Make ResourceLoader::filter() static, at the cost of not having minifier
errors in the ResourceLoader log bucket. (They will continue to be logged as
exceptions, however).
Change-Id: Ic1d802ee20565e61046bfbd8fd209bc56a4cbd6c
* Decline to cache minified private modules, because they exist in as many
variants as there are users, and are unlikely to be cache hits now that we
use APC.
* Other modules are minified individually, to improve cache hit rate.
Bug: T107377
Change-Id: Id6f5142062d73b5701126724e0fe8264105f7813
When minifying JavaScript, never put each statement on a separate line, and
always set a target maximum line length of 1000. These behaviors were
previously configurable via $wgResourceLoaderMinifierStatementsOnOwnLine and
$wgResourceLoaderMinifierMaxLineLength, respectively.
Change-Id: I0b0eb632875b5e16f728fd0aa62f7f5ecd79ef62
As Timo noted in a review of I1ff6115, using '+=' to combine file dependencies
is incorrect here. I fixed it in one place but not the other, causing the file
dependencies to vary based on whether a .lesscache file was used.
Change-Id: Ie3f48048febc32c5c2d45acd87e10f946f6dfc94
Re-use the md_skin database field for now as to not need a
schema change, beause this table is going away soon (T113916).
Bug: T113868
Change-Id: I7c7546ec58fd9be0447604989b908dd2084b0fe3
Make paths stored in the module_deps table relative to $IP. This ensures that when
the MediaWiki install path changes and/or if the location of the extension or skins
directory changes, that ResourceLoader's internal model is still accurate.
Previously when these paths change, ResourceLoader would exhibit various bugs.
1. Unable to detect changes in the module (if the directory no longer exists).
2. Point #1 is usually preceeded by one last cache invalidation as the content hash
of the file path changes (from a valid hash to an empty string).
3. Unnecessary cache invalidation (if both old and new directories exist). This
happens when a file is both an entry point (in the 'scripts' or 'styles' array)
and also a file dependency. At first they are de-duplicated by array_unique.
But after the disk path changes, the next check will result in the old path
being fetched from module_deps, and the new path from the live configuration.
This causes two changes that result in needless cache invalidation:
- The hash list contains one more item (T111481).
- The hash list contains both the old and new version of a file.
(or even alternate versions, e.g. when a change is backported to the old
wmf branch; it also affects wikis on the new branch due to the stale
file path still in the database).
It seems unusual to move a MediaWiki install, and usually we recommend third
parties to run update.php if site administrators do move their wiki. However
Wikimedia's deployment system implicitly moves the MediaWiki install continously
from e.g. "/srv/mediawiki/php-1.26wmf5" to "/srv/mediawiki/php-1.26wmf6".
This caused virtually all ResourceLoader caching layers to get invalidated every
week when another wmf-branch is deployed, thus breaking these file paths, which
changes the version hash, which then invalidates the cache.
Bug: T111481
Change-Id: I173a9820b3067c4a6598a4c8d77e239797d2659c
* Remove ResourceLoaderFileModule::getLessCompiler(). There is no reason for a
module to need to get a compiler in a different manner than
ResourceLoader::getLessCompiler().
* Add ResourceLoaderModule::getLessVars(). This method provides a means for
subclasses to easily inject custom LESS variables. The default implementation
simply returns an empty array.
* Make the $context parameter for ResourceLoaderFileModule::readStyleFiles()
non-optional (via graceful deprecation). The only callers I found were either
already calling it with a ResourceLoader context, or had a perfectly usable
ResourceLoaderContext in local scope.
* Make ResourceLoaderFileModule::{readStyleFile,getLessCompiler} require a
context. These methods are protected, so I didn't bother with a deprecation.
* Call ksort() on the LESS variables array in the only place it matters -- when
hashing its serialized representation to construct a cache lookup key. This
relieves getLessVars() subclasses from having to remember to re-sort the
variables array if they modify it.
* These changes make it possible to substantially simplify
ResourceLoaderEditToolbarModule, because the only thing it needs to do now is
implement its own getLessVars() method.
* This also allows it to be versioned like any other ResourceLoaderFileModule,
rather than having to use enableModuleContentVersion().
Change-Id: Ic3eab71691e502bfe19bdf4eb6f82cc679a7782f
- Removed space after cast
- Removed spaces in array index
- Removed double spaces
- Added spaces around string concat
- Fixed mixed tabs and spaces at begin of line
Change-Id: I38e849723f055d2d4c05cba72f5c245a28e8d5da
This changes the current:
> Internal error.
To be:
> [89466ede] 2015-09-23 16:40: Fatal exception of type Less_Exception
This matches how exceptions are reported on HTML pages by MediaWiki
(via MWException::getHTML in OutputPage).
The IDs are okay to be public and exposed on HTML pages even when
$wgShowExceptionDetails is disabled.
Follows-up d478ffde10. Ref T110659.
Change-Id: I501eaa629161184ba89d44e53cdac20b39cb0903
ResourceLoaderEditToolbarModule is clearly the wrong place for something so
generic, so this method needs a new home. We can either introduce a new class
or find a suitable existing home. I think CSSMin is suitable. It has public
methods that do things that are closely related, like `encodeStringAsDataURI`
and `buildUrlValue`.
Change-Id: Icc6dfb8f47199e6188dd71948f4645baee085e51
Most of it was already there (SELECT in preloadInfo, and in-class holder via
setFileDependencies) but the logic to write to the database was within
the file module class.
Moving it the other way around may make more sense in the future, but for
the moment ResourceLoader.php makes assumptions about all modules being
able to store setFileDependencies as read from the database. Whether or not
a class currently implements a way to write to that table.
Change-Id: I06b5da4144798197478c66e8b9ccd4cc62cf6fb6
Replace the one-off code for APC-assisted file content hashing from I5ceb8537c
with a call to the FileContentsHasher class introduced in I1ff61153.
Change-Id: I8e4c3d5a5e7cdf6455a43769261d742b0255622c
Caching the output of a LESS compiler is tricky, because a LESS file may
include additional LESS files via @imports, in which case the cache needs
to vary as the contents of those files vary (and not just the contents of
the primary LESS file).
To solve this, we first introduce a utility class, FileContentsHasher. This
class is essentially a smart version of md5_file() -- given one or more file
names, it computes a hash digest of their contents. It tries to avoid
re-reading files by caching the hash digest in APC and re-using it as long as
the files' mtimes have not changed. This is the same approach I used in
I5ceb8537c.
Next, we use this class in ResourceLoaderFileModule in the following way:
whenever we compile a LESS file, we cache the result as an associative array
with the following keys:
* `files` : the list of files whose contents influenced the compiled CSS.
* `hash` : a hash digest of the combined contents of those files.
* `css` : the CSS output of the compiler itself.
Before using a cached value, we verify that it is still current by asking
FileContentHasher for a hash of the combined contents of all referenced files,
and we compare that against the value of the `hash` key of the cached entry.
Bug: T112035
Change-Id: I1ff61153ddb95ed17e543bd4af7dd13fa3352861
Cache the content hash of module files in APC, if available. Include the
modification time in the cached value, so that touching the file invalidates
the cache.
Does this take us back to a world in which updates to a file's mtime cause a
module's version to change? No. The content hash cache will be invalidated,
and the content hash will be computed, but the actual value should remain
the same.
Change-Id: I5ceb8537c3cdb120beae19740635f085ee128272
The hash value generated by ResourceLoaderModule::safeFileHash() is used for
versioning and cache invalidation, so a cryptographic hash function is not
necessary. We can get better performance by using MD4.
Wikimedia on-CPU time over 24 hours of safeFileHash() in load.php:
* 6.68% with safeFileHash using SHA1 (day 1)
* 7.07% with safeFileHash using SHA1 (day 2)
* 2.84% with safeFileHash using MD4 (day 3)
Change-Id: I6ff728f1240268517c0f03e0823129316bc901cb
We don't allow modifying existing modules or registering the same module twice.
We do have an established convention of registering plugins for optional
dependencies conditionally.
E.g.
- VisualEditor should only register 'uls.data' if ULS extension isn't installed.
- The Math extension should only register its VE plugin if VE is installed.
Typically the dependency being checked is registered unconditionally. The
conditional one is the supplemental one in case the canonical source within
the MediaWiki landscape is absent.
Currently this means code will check both isModuleRegistered() and the global
$wgResourceModules. This doesn't make sense. Reverse this so that the
unconditional registry is processed first, and the programmatic one after.
This also helps exceptions come from a more sensible place in case of a duplicate
registration.
Change-Id: Ib2d01c88511ade75a97b2509bccfabb1a4cc5c15
Mechanical stacktraces are mostly program identifiers (which are English) with some
glue text.
When stacktraces are disabled by the site admin, the replacement message doesn't
seem valuable to localise. And one could argue it should be in the user language
instead of the site language in that case, which complicates the output buffer
and caching layers for this.
It also reduces coupling with MediaWiki (Ref T32956).
Change-Id: If446f1b815add2be4baf1c333f18a4bab0cc9c37
The way this is implemented is really dirty... but it found us
a few pre-existing bugs already (T111518, T111519, T111771).
I think it might be worth it.
* CSSMin: Add new method getAllLocalFileReferences() which skips the
file_exists() check.
* ResourceLoaderModule: Make use of it to track missing files too.
* ResourcesTest: Verify that the missing files are missing.
Change-Id: I5a3cdeb7d53485f161ccf8133e76850cdf5b4579
wfExpandUrl() should only be used if the url needs to be absolute
and/or needs to point to the canonical, internal or default wgServer
for the wiki.
For meta-data or API responses this can make sense, but for resources
loaded by the browser from the current page, there is no added value
in expanding these to the default domain.
This is causing problems in custom setups such as at Wikimedia,
with "zero" and "m" sub domains for which $wgServer points to the
canonical domain.
Bug: T106966
Bug: T112646
Change-Id: Ie25d3a8f69564319a1b29b5c5904f3d352924a31
* Use implode() instead of foreach concat.
* Use two new lines instead of just one since the exception message
with stacktrace spans multiple lines (makes it easier to distinguish
when there are indeed exceptions from multiple sources).
* Output a single comment instead of one for each.
* If context is a JavaScript response, also include a console.error
call with the erro message.
To try out:
* Break a file module descriptor in Resources.php by e.g. making
a typo in one of the scripts arrays.
* View a page on-wiki that uses the module (e.g. jquery.accessKeyLabel
is loaded on most pages).
* Observe error in the console.
Bug: T110659
Change-Id: I4272795f1fab96a2effe2a6c068a56421adaa512
Deprecated in 1.24, for reasons explained in a0c41ae39d. I don't see any
usage in core or extensions.
Change-Id: I46f9e04ae633e7ff1ee112b652e1865731172f1f
This makes it consistent with how we handle the resourceloader_cache hit metric.
When I added the miss metric I figured it was nice to get both counts and timings
in a single metric, but I can't trust StatsD/Graphite not to screw this up so
let's untangle them to be sure.
Change-Id: I24f68e34a9fa491806fdb6023bdd757f6688c2ed
Follows-up r88883 (8dab43f703). The IEUrlExtension library only considers the
query string if the base path has no extension (or if the dot is urlencoded).
Neither of which is the case for load.php urls.
If the server doesn't expose undecoded request URIs (per IEUrlExtension)
then it will look in the query string always. However even then it is
only an issue if the last query parameter in the url contains a dot,
which is very unlikely for load.php (they typically end in 'skin' or
'version' which don't contain plain dots).
Ref T30840.
Change-Id: I4a4ef7ef5ad7d74ecb47a44342e6324fa9844173
These properties don't affect the module response. They are either
only consumed by OutputPage, or provided by the startup module
directly, which detects its own changes.
Changes to these properties should not invalidate module cache for
these properties.
Follows-up 2f30ff7a86 which added a hook to getDependencies()
but still included the raw 'dependencies' in the definition summary.
But as it turns out, dependencies don't need to be included.
Change-Id: I468e6a6edc352d845dc7a5d06b0ae2a6d809700b
mw.ForeignApi is an extension of mw.Api, automatically handling
everything required to communicate with another MediaWiki wiki via
cross-origin requests (CORS).
Authentication-related MediaWiki extensions may extend it further to
ensure that the user authenticated on the current wiki will be
automatically authenticated on the foreign one. A CentralAuth
implementation is provided in I0fd05ef8b9c9db0fdb59c6cb248f364259f80456.
Bug: T66636
Change-Id: Ic20b9682d28633baa87d22e6e9fb71ce507da58d
