Commit graph

269 commits

Author SHA1 Message Date
jdlrobson
bc4e07b6f6 resourceloader: Implement modern module loading (2/2)
* Send 'module' and 'require' parameters to module closures.
  This depends on Ia925844cc22f143 being deployed one cycle earlier.
* Patch Moment and OOjs to ensure these libraries continue to expose
  their module as globals as well. AMD/UMD-compatible libraries
  only expose a global *OR* an export, not both. We need both
  for back-compat.
* Update pluralRuleParser to make use of module export to allow
  usage via require().

To test, check out the patch and run:
> mw.loader.load('moment');
> mw.loader.require('moment')()
> mw.loader.require('moment')('2011-04-01').fromNow()

Bug: T108655
Change-Id: Idbd054880ee70d659ec760aef8fcb38d0704a394
2016-04-13 15:43:41 +00:00
Timo Tijhof
dd2d7d0ffc OutputPage: Minor clean up of <head> and HTML
* Remove trailing space in self-closing tag.
  Brings parserTest output of Parser and Parsoid closer together.

* Remove various line breaks at begin and end of script contents.

* Remove FILTER_NOMIN from makeConfigSetScript() output.
  This isn't part of any user- or page-dependent module and not minified.
  And Xml::encodeJsCall already ensures compact output for prod mode.

Bug: T127328
Change-Id: I85a5a59fd0955c1a112e8b24b933f0d9e983a156
2016-03-24 03:24:31 +00:00
Kunal Mehta
b9668d6d80 build: Update mediawiki-codesniffer to 0.6.0, add "composer fix"
* Fix errors spotted by new release
* Introduce "composer fix", which uses phpcbf to automatically fix some
errors spotted by phpcs.
* Drop $PHPCS_ARGS variable that didn't work on Windows, and add -s flag
* Remove rules from phpcs.xml that are now in MW-CS ruleset.

Change-Id: I13e2155695918c918b67497ac65b85a03897095e
2016-02-17 02:54:42 -08:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Ricordisamoa
f9b4a217e9 resourceloader: Remove obsolete '$that = $this' closure pattern
Closures support $this as of PHP 5.4.

Change-Id: Ib00be459490569036604afff53bd99fe247737df
2016-02-10 18:17:47 +00:00
Timo Tijhof
0c8101570c resourceloader: Simplify and clean up RLQ wrap
startup.js:
* Don't assign to "window.RLQ" and read from "RLQ". Use the same
  identifier in both places instead of relying on global scope.
* Create the global window.RLQ only once. The temporary [] fallback
  doesn't need to be exposed globally, so use a local variable for
  the loop that processes the pre-existing queue built up while
  the startup module was loading.

OutputPage:
* Simplify the RLQ wrap to be more idiomatic and less repetitive.
  Matches the pattern used in Google and Facebook open-source projects.

Change-Id: I9176377bc05432e51add841696a356428feec8ce
2016-01-27 02:03:00 +00:00
jdlrobson
4d0a36f333 resourceloader: Change getLessCompiler() to not be static
Cache Less variables in the instance instead of statically.

This allows tests to populate their own less variabless via ResourceLoaderLESSVars.

Make getLessCompiler() and getLessVars() regular public methods and
update callers.

Change-Id: I95506b8bb20a4b2b3f82014a7b0fcee5f28973e6
2016-01-14 18:44:32 -08:00
Timo Tijhof
5d5b269e0e resourceloader: Migrate from msg_resource table to object cache
MessageBlobStore class:
* Make logger aware.
* Log an error if json encoding fails.
* Stop using the DB table. WANObjectCache supports everything we need:
  - Batch retrieval.
  - Invalidate keys with wildcard selects or cascading check keys.
* Update tests slightly since the actual update now happens on-demand as
  part of get() instead of within updateMessage().

ResourceLoader class:
* Remove all interaction with the msg_resource table. Remove db table later.
* Refactor code to use a hash of the blob instead of a timestamp.
  Timestamps are unreliable and roll over too frequently for message blob store
  because there is no authoritative source. The timestamps were inferred based on
  when a change is observed. Message overrides from the local wiki have an
  explicit update event when the page is edited. All other messages, such as
  from MediaWiki core and extensions using LocalisationCache, have a single
  timestamp for all messages which rolls over every time the cache is rebuilt.
  A hash is deterministic, and won't cause needless invalidation (T102578).
* Remove redundant pre-fetching in makeModuleResponse().
  This is already done by preloadModuleInfo() in respond().
* Don't bother storing and retreiving empty "{}" objects.
  Instead, detect whether a module's message list is empty at runtime.

ResourceLoaderModule class:
* Make logger aware.
* Log if a module's message blob was not preloaded.

cleanupRemovedModules:
* Now that blobs have a TTL, there's no need to prune old entries.

Bug: T113092
Bug: T92357
Change-Id: Id8c26f41a82597e34013f95294cdc3971a4f52ae
2015-12-08 14:02:38 +00:00
Timo Tijhof
45bb13d5a0 resourceloader: Log warning if module produces an invalid version hash
* Add logger to ResourceLoaderContext for convenient use within modules.
* Group related members within ResourceLoaderContext.

Change-Id: Ifbc3de1b6e0838386735f1573df328d9b331ac37
2015-12-07 16:12:03 +00:00
Timo Tijhof
13d9c4582f resourceloader: Remove redundant back-compat in mw.loader.addSource()
Follows-up e103ba265b.

This method doesn't need to be backward-compatible as this is already
normalised server-side.

When upgrading from before MediaWiki 1.24 the startup response will
change to the new format and also inline a request to the newer
version of mediawiki.js.

Also update method documentation to be up to date and remove
spurious return statements that aren't used.

Change-Id: I03d1b5666fcddedaa46ecf878ee7636042a5e3be
2015-11-20 01:19:45 +00:00
Timo Tijhof
70f8a31afe resourceloader: Remove use of msg_resource_links table
This table is not needed because module names and their messages
array are available to the runtime environment at very little cost.

The only purpose it was serving is reverse lookup from message
key to module name (e.g. when MessageCache receives update that
need to propagate to MessageBlobStore). However that is better
achieved by simply looping through modules in PHP. The overhead
of a database is not worth this minor convenience.

MessageBlobStore
* insertMessageBlob: Doesn't need to update msg_resource_links.
* updateModule: Doesn't need to update msg_resource_links.
* getUpdatesForMessage: Reimplement with list from memory
  instead of msg_resource_links.

The database table will be removed in If009e2620e59002e1.

Bug: T113092
Change-Id: Ia9131f570001f00c9800b260ac4b3469d54d2784
2015-11-09 19:16:38 +00:00
jenkins-bot
5f15ac0058 Merge "Replace newAccelerator() with getLocalServerInstance()" 2015-11-02 22:04:21 +00:00
Aaron Schulz
17c91ad610 Replace newAccelerator() with getLocalServerInstance()
The name is clearer and more consistent, with simpler arguments.

Change-Id: I7205a99ce033e8b086a52cd02c8a721e99c84b1e
2015-11-02 21:39:08 +00:00
Ori Livneh
8a0cd2b043 Revert "Add MediaWiki-Timestamp header to ResourceLoader requests"
I have not had the time and wherewithal to write the follow-up code that
actually uses this value. So just remove it for now.
This reverts commit 3d5aca4a40.

Change-Id: Ice9b65f39802b55b3d4ddf801c06d1be3227fd5b
2015-11-02 13:36:18 -08:00
umherirrender
96473ea6e4 phpcs: Fix some "Assignment expression not allowed"
Found by new version of mediawiki/codesniffer
https://integration.wikimedia.org/ci/job/mediawiki-core-phpcs/1939/consoleFull

Change-Id: I673f71fd0dfc8d6ba1ce6c3d5da21787ff95cb32
2015-11-01 21:07:00 +01:00
Ori Livneh
d3e3bcfd6d Follow-up to Ie3499d
My first pass at what became Ie3499d9f2 was to keep the exception for duplicate
registrations with conflicting parameters but downgrade to a warning if the
registration attempts are identically parametrized. I then removed the
exception entirely, but left the if-block, which does not make much sense, as
it means we pass over silently module registrations that set different
parameters, and throw a warning only when they are the same.

Change-Id: I13da33faf2f058ac75ef4b1615c4fafc18f14505
2015-10-27 00:44:47 -07:00
Ori Livneh
b1ea0612f8 resourceloader: Deduplicate module registration conflicts
If an attempt is made to register a ResourceLoader module which has already
been registered, instead of bailing with an error, issue a warning instead, and
allow the lattermost set of module parameters to stick.

Because modules are registered from extensions, and because modules are often
migrated from one extension to another as extensions are refactored, and
because the developers of different extensions are not always aware of each
other's work, throwing an error runs a high risk of causing an outage.

Bug: T116628
Change-Id: Ie3499d9f2ff3bf071f1c2b0176e19440ff55211c
2015-10-26 18:48:05 +00:00
Ori Livneh
e2547a73c9 Update ResourceLoader for Ib7fc2f939b
Use BagOStuff::makeKey() and BagOStuff::makeGlobalKey() instead of
wfMemcKey() and wfGlobalCacheKey().

Change-Id: Id4e58c0e616b74de220faf13ba7c9ea606fef8c1
2015-10-19 17:28:09 -07:00
Kevin Israel
b52502eec5 Change 1.26 to 1.27, mostly in doc comments
Using the following command line, I have found doc comments (and
a wfDeprecated() call) mentioning "1.26" when they should mention
"1.27" instead, which I have fixed manually:

git diff -M REL1_26 | grep --color=always -C 10 -iP \
'^\+.*\D1\.26(\D|$)' | aha > oldver.html

Follows-up these commits:
* 047b60b96d
* 169b7b98b5
* 25a44aa3e4
* 2fb2a3f14b
* 3f1e9fa268 [1]
* 8c84af70b6
* c0cb80beac
* d04a92a551
* d3b85592ea

[1] Release notes moved in I195dd1cf.

Because a release note stating that the UserRights hook is deprecated
was added in 37062a0c0d (before the branch cut), this commit does not
change the Hooks::run call that had been changed in 21206c8fbe.

Change-Id: I5a427f003e7e3b4559fe377bcdfdca466a570708
2015-10-07 01:41:41 -04:00
Ori Livneh
e9bafa0ed2 resourceloader: Tiny optimization to ResourceLoader::isValidModuleName()
This is a micro-optimization, but ResourceLoader is so well-optimized at this
point that the call to PCRE accounts for 1.25% of all load.php CPU time. So
might as well making it a tiny bit faster.

Change-Id: Iefab804a6ca6d54ce230958513a3bea44f4e7c62
2015-10-05 19:34:51 +00:00
Ori Livneh
ca30efa30a resourceloader: cache minified user and site modules
* Add support for a '/* @nomin */' annotation in ResourceLoader. If present in
  JavaScript or CSS, the code will not be minified or cached. This allows
  modules like the ResourceLoaderUserTokensModule to declare themselves unfit
  for minification / caching without requiring a complicated refactor.
* Make ResourceLoader::filter() static, at the cost of not having minifier
  errors in the ResourceLoader log bucket. (They will continue to be logged as
  exceptions, however).

Change-Id: Ic1d802ee20565e61046bfbd8fd209bc56a4cbd6c
2015-10-03 20:29:48 +00:00
Ori Livneh
60dcbf48e7 Fix-up for I21bb3f08e7f: ResourceLoaderModule::expandRelativePaths
ResourceLoaderModule::expandRelativePaths(), not ResourceLoader::expandRelativePaths().

Change-Id: Ibb2a8398641d360240889988189d2ef51f114c0b
2015-10-02 00:37:46 +00:00
Timo Tijhof
66b2060d22 resourceloader: Fix-up missing expandRelativePaths() in preloadModuleInfo
Follows-up 280d292.

Change-Id: I21bb3f08e7fc9c6b364df2934640c2d5ce1e86b0
2015-10-01 17:14:19 -07:00
Ori Livneh
b7eb243d92 resourceloader: Minify per-module instead of per-response
* Decline to cache minified private modules, because they exist in as many
  variants as there are users, and are unlikely to be cache hits now that we
  use APC.
* Other modules are minified individually, to improve cache hit rate.

Bug: T107377
Change-Id: Id6f5142062d73b5701126724e0fe8264105f7813
2015-10-01 23:12:33 +00:00
Ori Livneh
b91643ffdc resourceloader: Deprecate two pointless minification config vars
When minifying JavaScript, never put each statement on a separate line, and
always set a target maximum line length of 1000. These behaviors were
previously configurable via $wgResourceLoaderMinifierStatementsOnOwnLine and
$wgResourceLoaderMinifierMaxLineLength, respectively.

Change-Id: I0b0eb632875b5e16f728fd0aa62f7f5ecd79ef62
2015-09-30 22:04:33 +00:00
Timo Tijhof
280d29215a resourceloader: Vary module_deps on language (in addition to skin)
Re-use the md_skin database field for now as to not need a
schema change, beause this table is going away soon (T113916).

Bug: T113868
Change-Id: I7c7546ec58fd9be0447604989b908dd2084b0fe3
2015-09-30 00:40:56 +00:00
jenkins-bot
84ce1abcc9 Merge "resourceloader: Cleaned up makeModuleResponse() param docs" 2015-09-28 19:32:26 +00:00
jenkins-bot
c108dd32d9 Merge "resourceloader: Tidy up RL to simplify ResourceLoaderEditToolbarModule" 2015-09-28 19:29:01 +00:00
Aaron Schulz
a7c4a1c5f5 resourceloader: Cleaned up makeModuleResponse() param docs
Change-Id: Iabf53452598e13a98241793c317ceffb746aa46e
2015-09-28 19:22:16 +00:00
Ori Livneh
3f1e9fa268 resourceloader: Tidy up RL to simplify ResourceLoaderEditToolbarModule
* Remove ResourceLoaderFileModule::getLessCompiler(). There is no reason for a
  module to need to get a compiler in a different manner than
  ResourceLoader::getLessCompiler().
* Add ResourceLoaderModule::getLessVars(). This method provides a means for
  subclasses to easily inject custom LESS variables. The default implementation
  simply returns an empty array.
* Make the $context parameter for ResourceLoaderFileModule::readStyleFiles()
  non-optional (via graceful deprecation). The only callers I found were either
  already calling it with a ResourceLoader context, or had a perfectly usable
  ResourceLoaderContext in local scope.
* Make ResourceLoaderFileModule::{readStyleFile,getLessCompiler} require a
  context. These methods are protected, so I didn't bother with a deprecation.
* Call ksort() on the LESS variables array in the only place it matters -- when
  hashing its serialized representation to construct a cache lookup key. This
  relieves getLessVars() subclasses from having to remember to re-sort the
  variables array if they modify it.
* These changes make it possible to substantially simplify
  ResourceLoaderEditToolbarModule, because the only thing it needs to do now is
  implement its own getLessVars() method.
* This also allows it to be versioned like any other ResourceLoaderFileModule,
  rather than having to use enableModuleContentVersion().

Change-Id: Ic3eab71691e502bfe19bdf4eb6f82cc679a7782f
2015-09-28 12:18:12 -07:00
Amir E. Aharoni
4e7cc78b4f Make lines short to pass phpcs in files under includes/resourceloader
Bug: T102614
Change-Id: I2202438591e74a35570bb621f5aad3ac241fbc29
2015-09-27 16:24:07 +00:00
umherirrender
c572d18661 Fixed spacing
- Removed space after cast
- Removed spaces in array index
- Removed double spaces
- Added spaces around string concat
- Fixed mixed tabs and spaces at begin of line

Change-Id: I38e849723f055d2d4c05cba72f5c245a28e8d5da
2015-09-26 20:44:54 +00:00
Timo Tijhof
9dcc8c872b resourceloader: Include exception log id in error message
This changes the current:

> Internal error.

To be:

> [89466ede] 2015-09-23 16:40: Fatal exception of type Less_Exception

This matches how exceptions are reported on HTML pages by MediaWiki
(via MWException::getHTML in OutputPage).

The IDs are okay to be public and exposed on HTML pages even when
$wgShowExceptionDetails is disabled.

Follows-up d478ffde10. Ref T110659.

Change-Id: I501eaa629161184ba89d44e53cdac20b39cb0903
2015-09-25 23:40:03 +00:00
jenkins-bot
440d21b0d9 Merge "resourceloader: Log load.php exceptions to JavaScript console" 2015-09-21 12:53:05 +00:00
Timo Tijhof
926a83a07d resourceloader: Run ResourceLoaderRegisterModules after wgResourceModules
We don't allow modifying existing modules or registering the same module twice.

We do have an established convention of registering plugins for optional
dependencies conditionally.

E.g.
- VisualEditor should only register 'uls.data' if ULS extension isn't installed.
- The Math extension should only register its VE plugin if VE is installed.

Typically the dependency being checked is registered unconditionally. The
conditional one is the supplemental one in case the canonical source within
the MediaWiki landscape is absent.

Currently this means code will check both isModuleRegistered() and the global
$wgResourceModules. This doesn't make sense. Reverse this so that the
unconditional registry is processed first, and the programmatic one after.

This also helps exceptions come from a more sensible place in case of a duplicate
registration.

Change-Id: Ib2d01c88511ade75a97b2509bccfabb1a4cc5c15
2015-09-18 19:06:03 +00:00
Ori Livneh
ea862efa75 Replace leafo/lessphp with oyejorge/less.php
Bug: T112035
Change-Id: I826adf981ac28c44f0915ceb059d699d6b23a254
2015-09-18 16:49:58 +00:00
jeroendedauw
90aa56ebce resourceloader: Add missing argument to makeLoaderImplementScript
Change-Id: I661ec86ab53639be6999e15ac20c726477fae8a5
2015-09-16 15:37:36 +00:00
Timo Tijhof
19349af140 resourceloader: Don't localise hidden exception
Mechanical stacktraces are mostly program identifiers (which are English) with some
glue text.

When stacktraces are disabled by the site admin, the replacement message doesn't
seem valuable to localise. And one could argue it should be in the user language
instead of the site language in that case, which complicates the output buffer
and caching layers for this.

It also reduces coupling with MediaWiki (Ref T32956).

Change-Id: If446f1b815add2be4baf1c333f18a4bab0cc9c37
2015-09-16 04:14:39 +00:00
Timo Tijhof
b5e69c4ef6 resourceloader: Don't call wfExpandUrl() on load.php urls
wfExpandUrl() should only be used if the url needs to be absolute
and/or needs to point to the canonical, internal or default wgServer
for the wiki.

For meta-data or API responses this can make sense, but for resources
loaded by the browser from the current page, there is no added value
in expanding these to the default domain.

This is causing problems in custom setups such as at Wikimedia,
with "zero" and "m" sub domains for which $wgServer points to the
canonical domain.

Bug: T106966
Bug: T112646
Change-Id: Ie25d3a8f69564319a1b29b5c5904f3d352924a31
2015-09-15 21:21:04 +01:00
Timo Tijhof
d478ffde10 resourceloader: Log load.php exceptions to JavaScript console
* Use implode() instead of foreach concat.
* Use two new lines instead of just one since the exception message
  with stacktrace spans multiple lines (makes it easier to distinguish
  when there are indeed exceptions from multiple sources).
* Output a single comment instead of one for each.
* If context is a JavaScript response, also include a console.error
  call with the erro message.

To try out:
* Break a file module descriptor in Resources.php by e.g. making
  a typo in one of the scripts arrays.
* View a page on-wiki that uses the module (e.g. jquery.accessKeyLabel
  is loaded on most pages).
* Observe error in the console.

Bug: T110659
Change-Id: I4272795f1fab96a2effe2a6c068a56421adaa512
2015-09-15 04:20:30 +01:00
Ori Livneh
e176c76da2 resourceloader: Fully remove ResourceLoaderLESSFunctions
Deprecated in 1.24, for reasons explained in a0c41ae39d. I don't see any
usage in core or extensions.

Change-Id: I46f9e04ae633e7ff1ee112b652e1865731172f1f
2015-09-04 01:41:40 +00:00
Timo Tijhof
f28e51adaf resourceloader: Use increment() for resourceloader_cache miss metric
This makes it consistent with how we handle the resourceloader_cache hit metric.

When I added the miss metric I figured it was nice to get both counts and timings
in a single metric, but I can't trust StatsD/Graphite not to screw this up so
let's untangle them to be sure.

Change-Id: I24f68e34a9fa491806fdb6023bdd757f6688c2ed
2015-09-02 04:22:40 +00:00
Timo Tijhof
9d81e06992 resourceloader: Don't add "&*" query parameter to load.php urls
Follows-up r88883 (8dab43f703). The IEUrlExtension library only considers the
query string if the base path has no extension (or if the dot is urlencoded).
Neither of which is the case for load.php urls.

If the server doesn't expose undecoded request URIs (per IEUrlExtension)
then it will look in the query string always. However even then it is
only an issue if the last query parameter in the url contains a dot,
which is very unlikely for load.php (they typically end in 'skin' or
'version' which don't contain plain dots).

Ref T30840.

Change-Id: I4a4ef7ef5ad7d74ecb47a44342e6324fa9844173
2015-09-01 02:16:46 +00:00
Aaron Schulz
4f0b16b914 Fixed BufferingStatsdDataFactory::timing() callers to use ms
* The interface actually demands this

Change-Id: I1e334c2696a8a8eca73a6ae7f71428190cad3107
2015-08-19 01:01:49 +00:00
jenkins-bot
b73b245689 Merge "Revert "resourceloader: Add must-revalidate to Cache-Control"" 2015-08-18 07:00:00 +00:00
Ori Livneh
78d727453d ResourceLoader: apply minify-js filter to config scripts
Minify the JavaScript generated in ResourceLoader::makeConfigSetScript() unless
debug mode is set.

Change-Id: I45b5839803546c8e35e74b72c679777815294315
2015-08-14 11:51:31 -07:00
Ori Livneh
471a3bfb3d ResourceLoader: make cacheReport option false by default
In the three years or so that I have been around, we at Wikimedia have not to
my knowledge used the cache keys to manage the cache of ResourceLoader modules.
Instead, if we want to force a module to rebuild, we simply touch one of its
files. (Though even that is not often required these days.)

Change-Id: Ib62a792e053572f268d4be19a10f99f84c07abc5
2015-08-11 09:38:43 -07:00
Krinkle
4c161b134a Revert "resourceloader: Add must-revalidate to Cache-Control"
This reverts commit 1139fbe58c.

Change-Id: If0019bd5d925b76e23ad9384427080d399af5d1f
2015-08-11 01:13:45 +00:00
Timo Tijhof
7facd8b214 resourceloader: Fix RLQ script to support IE8 quirk
In IE8 there is a race condition. If the window.RLQ runs first (that is, the
startup module loads first) then 'var RLQ' does not get associated
properly to our custom window.RLQ object.

The other way around is fine, but never happens because <script async>
isn't supported in IE8.

Consistently use 'window.RLQ' (never 'var RLQ') to bypass this bug.

Also updating the startup.js file for consistency (though not strictly
needed since the implicit global reference without 'var' also works fine
in IE8.

Bug: T107954
Change-Id: I3f46fee7c4528abf806bb9c51fc767eceb795009
2015-08-04 16:25:11 -07:00
Ori Livneh
0ca893e2eb Fix spelling mistake in ResourceLoader inline documentation
Change-Id: Ie099eed12ba45aeb1d3b8b08c911222af25e42fa
2015-07-31 16:11:38 -07:00