Commit graph

258 commits

Author SHA1 Message Date
jenkins-bot
5f15ac0058 Merge "Replace newAccelerator() with getLocalServerInstance()" 2015-11-02 22:04:21 +00:00
Aaron Schulz
17c91ad610 Replace newAccelerator() with getLocalServerInstance()
The name is clearer and more consistent, with simpler arguments.

Change-Id: I7205a99ce033e8b086a52cd02c8a721e99c84b1e
2015-11-02 21:39:08 +00:00
Ori Livneh
8a0cd2b043 Revert "Add MediaWiki-Timestamp header to ResourceLoader requests"
I have not had the time and wherewithal to write the follow-up code that
actually uses this value. So just remove it for now.
This reverts commit 3d5aca4a40.

Change-Id: Ice9b65f39802b55b3d4ddf801c06d1be3227fd5b
2015-11-02 13:36:18 -08:00
umherirrender
96473ea6e4 phpcs: Fix some "Assignment expression not allowed"
Found by new version of mediawiki/codesniffer
https://integration.wikimedia.org/ci/job/mediawiki-core-phpcs/1939/consoleFull

Change-Id: I673f71fd0dfc8d6ba1ce6c3d5da21787ff95cb32
2015-11-01 21:07:00 +01:00
Ori Livneh
d3e3bcfd6d Follow-up to Ie3499d
My first pass at what became Ie3499d9f2 was to keep the exception for duplicate
registrations with conflicting parameters but downgrade to a warning if the
registration attempts are identically parametrized. I then removed the
exception entirely, but left the if-block, which does not make much sense, as
it means we pass over silently module registrations that set different
parameters, and throw a warning only when they are the same.

Change-Id: I13da33faf2f058ac75ef4b1615c4fafc18f14505
2015-10-27 00:44:47 -07:00
Ori Livneh
b1ea0612f8 resourceloader: Deduplicate module registration conflicts
If an attempt is made to register a ResourceLoader module which has already
been registered, instead of bailing with an error, issue a warning instead, and
allow the lattermost set of module parameters to stick.

Because modules are registered from extensions, and because modules are often
migrated from one extension to another as extensions are refactored, and
because the developers of different extensions are not always aware of each
other's work, throwing an error runs a high risk of causing an outage.

Bug: T116628
Change-Id: Ie3499d9f2ff3bf071f1c2b0176e19440ff55211c
2015-10-26 18:48:05 +00:00
Ori Livneh
e2547a73c9 Update ResourceLoader for Ib7fc2f939b
Use BagOStuff::makeKey() and BagOStuff::makeGlobalKey() instead of
wfMemcKey() and wfGlobalCacheKey().

Change-Id: Id4e58c0e616b74de220faf13ba7c9ea606fef8c1
2015-10-19 17:28:09 -07:00
Kevin Israel
b52502eec5 Change 1.26 to 1.27, mostly in doc comments
Using the following command line, I have found doc comments (and
a wfDeprecated() call) mentioning "1.26" when they should mention
"1.27" instead, which I have fixed manually:

git diff -M REL1_26 | grep --color=always -C 10 -iP \
'^\+.*\D1\.26(\D|$)' | aha > oldver.html

Follows-up these commits:
* 047b60b96d
* 169b7b98b5
* 25a44aa3e4
* 2fb2a3f14b
* 3f1e9fa268 [1]
* 8c84af70b6
* c0cb80beac
* d04a92a551
* d3b85592ea

[1] Release notes moved in I195dd1cf.

Because a release note stating that the UserRights hook is deprecated
was added in 37062a0c0d (before the branch cut), this commit does not
change the Hooks::run call that had been changed in 21206c8fbe.

Change-Id: I5a427f003e7e3b4559fe377bcdfdca466a570708
2015-10-07 01:41:41 -04:00
Ori Livneh
e9bafa0ed2 resourceloader: Tiny optimization to ResourceLoader::isValidModuleName()
This is a micro-optimization, but ResourceLoader is so well-optimized at this
point that the call to PCRE accounts for 1.25% of all load.php CPU time. So
might as well making it a tiny bit faster.

Change-Id: Iefab804a6ca6d54ce230958513a3bea44f4e7c62
2015-10-05 19:34:51 +00:00
Ori Livneh
ca30efa30a resourceloader: cache minified user and site modules
* Add support for a '/* @nomin */' annotation in ResourceLoader. If present in
  JavaScript or CSS, the code will not be minified or cached. This allows
  modules like the ResourceLoaderUserTokensModule to declare themselves unfit
  for minification / caching without requiring a complicated refactor.
* Make ResourceLoader::filter() static, at the cost of not having minifier
  errors in the ResourceLoader log bucket. (They will continue to be logged as
  exceptions, however).

Change-Id: Ic1d802ee20565e61046bfbd8fd209bc56a4cbd6c
2015-10-03 20:29:48 +00:00
Ori Livneh
60dcbf48e7 Fix-up for I21bb3f08e7f: ResourceLoaderModule::expandRelativePaths
ResourceLoaderModule::expandRelativePaths(), not ResourceLoader::expandRelativePaths().

Change-Id: Ibb2a8398641d360240889988189d2ef51f114c0b
2015-10-02 00:37:46 +00:00
Timo Tijhof
66b2060d22 resourceloader: Fix-up missing expandRelativePaths() in preloadModuleInfo
Follows-up 280d292.

Change-Id: I21bb3f08e7fc9c6b364df2934640c2d5ce1e86b0
2015-10-01 17:14:19 -07:00
Ori Livneh
b7eb243d92 resourceloader: Minify per-module instead of per-response
* Decline to cache minified private modules, because they exist in as many
  variants as there are users, and are unlikely to be cache hits now that we
  use APC.
* Other modules are minified individually, to improve cache hit rate.

Bug: T107377
Change-Id: Id6f5142062d73b5701126724e0fe8264105f7813
2015-10-01 23:12:33 +00:00
Ori Livneh
b91643ffdc resourceloader: Deprecate two pointless minification config vars
When minifying JavaScript, never put each statement on a separate line, and
always set a target maximum line length of 1000. These behaviors were
previously configurable via $wgResourceLoaderMinifierStatementsOnOwnLine and
$wgResourceLoaderMinifierMaxLineLength, respectively.

Change-Id: I0b0eb632875b5e16f728fd0aa62f7f5ecd79ef62
2015-09-30 22:04:33 +00:00
Timo Tijhof
280d29215a resourceloader: Vary module_deps on language (in addition to skin)
Re-use the md_skin database field for now as to not need a
schema change, beause this table is going away soon (T113916).

Bug: T113868
Change-Id: I7c7546ec58fd9be0447604989b908dd2084b0fe3
2015-09-30 00:40:56 +00:00
jenkins-bot
84ce1abcc9 Merge "resourceloader: Cleaned up makeModuleResponse() param docs" 2015-09-28 19:32:26 +00:00
jenkins-bot
c108dd32d9 Merge "resourceloader: Tidy up RL to simplify ResourceLoaderEditToolbarModule" 2015-09-28 19:29:01 +00:00
Aaron Schulz
a7c4a1c5f5 resourceloader: Cleaned up makeModuleResponse() param docs
Change-Id: Iabf53452598e13a98241793c317ceffb746aa46e
2015-09-28 19:22:16 +00:00
Ori Livneh
3f1e9fa268 resourceloader: Tidy up RL to simplify ResourceLoaderEditToolbarModule
* Remove ResourceLoaderFileModule::getLessCompiler(). There is no reason for a
  module to need to get a compiler in a different manner than
  ResourceLoader::getLessCompiler().
* Add ResourceLoaderModule::getLessVars(). This method provides a means for
  subclasses to easily inject custom LESS variables. The default implementation
  simply returns an empty array.
* Make the $context parameter for ResourceLoaderFileModule::readStyleFiles()
  non-optional (via graceful deprecation). The only callers I found were either
  already calling it with a ResourceLoader context, or had a perfectly usable
  ResourceLoaderContext in local scope.
* Make ResourceLoaderFileModule::{readStyleFile,getLessCompiler} require a
  context. These methods are protected, so I didn't bother with a deprecation.
* Call ksort() on the LESS variables array in the only place it matters -- when
  hashing its serialized representation to construct a cache lookup key. This
  relieves getLessVars() subclasses from having to remember to re-sort the
  variables array if they modify it.
* These changes make it possible to substantially simplify
  ResourceLoaderEditToolbarModule, because the only thing it needs to do now is
  implement its own getLessVars() method.
* This also allows it to be versioned like any other ResourceLoaderFileModule,
  rather than having to use enableModuleContentVersion().

Change-Id: Ic3eab71691e502bfe19bdf4eb6f82cc679a7782f
2015-09-28 12:18:12 -07:00
Amir E. Aharoni
4e7cc78b4f Make lines short to pass phpcs in files under includes/resourceloader
Bug: T102614
Change-Id: I2202438591e74a35570bb621f5aad3ac241fbc29
2015-09-27 16:24:07 +00:00
umherirrender
c572d18661 Fixed spacing
- Removed space after cast
- Removed spaces in array index
- Removed double spaces
- Added spaces around string concat
- Fixed mixed tabs and spaces at begin of line

Change-Id: I38e849723f055d2d4c05cba72f5c245a28e8d5da
2015-09-26 20:44:54 +00:00
Timo Tijhof
9dcc8c872b resourceloader: Include exception log id in error message
This changes the current:

> Internal error.

To be:

> [89466ede] 2015-09-23 16:40: Fatal exception of type Less_Exception

This matches how exceptions are reported on HTML pages by MediaWiki
(via MWException::getHTML in OutputPage).

The IDs are okay to be public and exposed on HTML pages even when
$wgShowExceptionDetails is disabled.

Follows-up d478ffde10. Ref T110659.

Change-Id: I501eaa629161184ba89d44e53cdac20b39cb0903
2015-09-25 23:40:03 +00:00
jenkins-bot
440d21b0d9 Merge "resourceloader: Log load.php exceptions to JavaScript console" 2015-09-21 12:53:05 +00:00
Timo Tijhof
926a83a07d resourceloader: Run ResourceLoaderRegisterModules after wgResourceModules
We don't allow modifying existing modules or registering the same module twice.

We do have an established convention of registering plugins for optional
dependencies conditionally.

E.g.
- VisualEditor should only register 'uls.data' if ULS extension isn't installed.
- The Math extension should only register its VE plugin if VE is installed.

Typically the dependency being checked is registered unconditionally. The
conditional one is the supplemental one in case the canonical source within
the MediaWiki landscape is absent.

Currently this means code will check both isModuleRegistered() and the global
$wgResourceModules. This doesn't make sense. Reverse this so that the
unconditional registry is processed first, and the programmatic one after.

This also helps exceptions come from a more sensible place in case of a duplicate
registration.

Change-Id: Ib2d01c88511ade75a97b2509bccfabb1a4cc5c15
2015-09-18 19:06:03 +00:00
Ori Livneh
ea862efa75 Replace leafo/lessphp with oyejorge/less.php
Bug: T112035
Change-Id: I826adf981ac28c44f0915ceb059d699d6b23a254
2015-09-18 16:49:58 +00:00
jeroendedauw
90aa56ebce resourceloader: Add missing argument to makeLoaderImplementScript
Change-Id: I661ec86ab53639be6999e15ac20c726477fae8a5
2015-09-16 15:37:36 +00:00
Timo Tijhof
19349af140 resourceloader: Don't localise hidden exception
Mechanical stacktraces are mostly program identifiers (which are English) with some
glue text.

When stacktraces are disabled by the site admin, the replacement message doesn't
seem valuable to localise. And one could argue it should be in the user language
instead of the site language in that case, which complicates the output buffer
and caching layers for this.

It also reduces coupling with MediaWiki (Ref T32956).

Change-Id: If446f1b815add2be4baf1c333f18a4bab0cc9c37
2015-09-16 04:14:39 +00:00
Timo Tijhof
b5e69c4ef6 resourceloader: Don't call wfExpandUrl() on load.php urls
wfExpandUrl() should only be used if the url needs to be absolute
and/or needs to point to the canonical, internal or default wgServer
for the wiki.

For meta-data or API responses this can make sense, but for resources
loaded by the browser from the current page, there is no added value
in expanding these to the default domain.

This is causing problems in custom setups such as at Wikimedia,
with "zero" and "m" sub domains for which $wgServer points to the
canonical domain.

Bug: T106966
Bug: T112646
Change-Id: Ie25d3a8f69564319a1b29b5c5904f3d352924a31
2015-09-15 21:21:04 +01:00
Timo Tijhof
d478ffde10 resourceloader: Log load.php exceptions to JavaScript console
* Use implode() instead of foreach concat.
* Use two new lines instead of just one since the exception message
  with stacktrace spans multiple lines (makes it easier to distinguish
  when there are indeed exceptions from multiple sources).
* Output a single comment instead of one for each.
* If context is a JavaScript response, also include a console.error
  call with the erro message.

To try out:
* Break a file module descriptor in Resources.php by e.g. making
  a typo in one of the scripts arrays.
* View a page on-wiki that uses the module (e.g. jquery.accessKeyLabel
  is loaded on most pages).
* Observe error in the console.

Bug: T110659
Change-Id: I4272795f1fab96a2effe2a6c068a56421adaa512
2015-09-15 04:20:30 +01:00
Ori Livneh
e176c76da2 resourceloader: Fully remove ResourceLoaderLESSFunctions
Deprecated in 1.24, for reasons explained in a0c41ae39d. I don't see any
usage in core or extensions.

Change-Id: I46f9e04ae633e7ff1ee112b652e1865731172f1f
2015-09-04 01:41:40 +00:00
Timo Tijhof
f28e51adaf resourceloader: Use increment() for resourceloader_cache miss metric
This makes it consistent with how we handle the resourceloader_cache hit metric.

When I added the miss metric I figured it was nice to get both counts and timings
in a single metric, but I can't trust StatsD/Graphite not to screw this up so
let's untangle them to be sure.

Change-Id: I24f68e34a9fa491806fdb6023bdd757f6688c2ed
2015-09-02 04:22:40 +00:00
Timo Tijhof
9d81e06992 resourceloader: Don't add "&*" query parameter to load.php urls
Follows-up r88883 (8dab43f703). The IEUrlExtension library only considers the
query string if the base path has no extension (or if the dot is urlencoded).
Neither of which is the case for load.php urls.

If the server doesn't expose undecoded request URIs (per IEUrlExtension)
then it will look in the query string always. However even then it is
only an issue if the last query parameter in the url contains a dot,
which is very unlikely for load.php (they typically end in 'skin' or
'version' which don't contain plain dots).

Ref T30840.

Change-Id: I4a4ef7ef5ad7d74ecb47a44342e6324fa9844173
2015-09-01 02:16:46 +00:00
Aaron Schulz
4f0b16b914 Fixed BufferingStatsdDataFactory::timing() callers to use ms
* The interface actually demands this

Change-Id: I1e334c2696a8a8eca73a6ae7f71428190cad3107
2015-08-19 01:01:49 +00:00
jenkins-bot
b73b245689 Merge "Revert "resourceloader: Add must-revalidate to Cache-Control"" 2015-08-18 07:00:00 +00:00
Ori Livneh
78d727453d ResourceLoader: apply minify-js filter to config scripts
Minify the JavaScript generated in ResourceLoader::makeConfigSetScript() unless
debug mode is set.

Change-Id: I45b5839803546c8e35e74b72c679777815294315
2015-08-14 11:51:31 -07:00
Ori Livneh
471a3bfb3d ResourceLoader: make cacheReport option false by default
In the three years or so that I have been around, we at Wikimedia have not to
my knowledge used the cache keys to manage the cache of ResourceLoader modules.
Instead, if we want to force a module to rebuild, we simply touch one of its
files. (Though even that is not often required these days.)

Change-Id: Ib62a792e053572f268d4be19a10f99f84c07abc5
2015-08-11 09:38:43 -07:00
Krinkle
4c161b134a Revert "resourceloader: Add must-revalidate to Cache-Control"
This reverts commit 1139fbe58c.

Change-Id: If0019bd5d925b76e23ad9384427080d399af5d1f
2015-08-11 01:13:45 +00:00
Timo Tijhof
7facd8b214 resourceloader: Fix RLQ script to support IE8 quirk
In IE8 there is a race condition. If the window.RLQ runs first (that is, the
startup module loads first) then 'var RLQ' does not get associated
properly to our custom window.RLQ object.

The other way around is fine, but never happens because <script async>
isn't supported in IE8.

Consistently use 'window.RLQ' (never 'var RLQ') to bypass this bug.

Also updating the startup.js file for consistency (though not strictly
needed since the implicit global reference without 'var' also works fine
in IE8.

Bug: T107954
Change-Id: I3f46fee7c4528abf806bb9c51fc767eceb795009
2015-08-04 16:25:11 -07:00
Ori Livneh
0ca893e2eb Fix spelling mistake in ResourceLoader inline documentation
Change-Id: Ie099eed12ba45aeb1d3b8b08c911222af25e42fa
2015-07-31 16:11:38 -07:00
Timo Tijhof
ad77bb10f2 resourceloader: Use WrappedString library to merge RLQ inline scripts
Update unit test to account for the internal 'html' prop now
being an array instead of string. And update expected values to
no longer have a trailing line break.

Bug: T27202
Change-Id: I105b6ef2e64ab8b891562e16940edb88592bd415
2015-07-31 17:45:06 +00:00
Timo Tijhof
7c9c60ba88 resourceloader: Restore minification for 'user' and 'site' module
Follows-up 19a40cd3ad, Id599b6be4.

TODO: Change minification to happen per-module instead of per-request so that
these hacks aren't needed (T107377).

Change-Id: Iaa281ee117f2ae7a51884d256dfbb1807224fc52
2015-07-30 21:38:10 +00:00
Timo Tijhof
59f1a1efa8 resourceloader: Convert inline statements to queued functions
Instead of having inline statements be plain statements wrapped
in an if-conditional block, convert them to inline functions
pushed into a queue.

The queue is kept in-memory until the startup module is loaded
at which point it transforms into a function that is immediately
invoked.

This is a prerequisite to making the top queue asynchronous.
Until then these functions will just run immediately as they
occur in the HTML after the startup module.

This is based on the previously reverted commit e86e5f8460.

Bug: T107399
Change-Id: Ifb38efca219c10ab973ad4c4ebb21c6a4239b005
2015-07-30 18:22:46 +00:00
Timo Tijhof
afce927c8f resourceloader: Ensure 'user' loads after 'site' (asynchronously)
Regression from 19a40cd3ad which made the 'site' module load asynchronously,
but the 'user' module was still loaded synchronously which meant it ran before
the site module finished.

Full test script at <https://gist.github.com/Krinkle/5db1d237da241b243485>.

Also:
* This changes the 'user' module to load asynchronously.
* Similar to 19a40cd3ad for site module, this makes the styles for the user
  module load twice. Harmless but doesn't look pretty internally.
* Remove the obsolete XXX-comment from 0b5389d98d (r56770).
* Add comment documenting the fact that the 'excludepages' feature can cause
  User/common.js and User/vector.js to be mis-ordered when the user previews
  common.js edits. This has always been the case (since 2009) and is merely
  being documented here.

Bug: T32358
Bug: T106736
Bug: T102077
Change-Id: Id599b6be42613529fb7f4dd3273f36ccadb3a09e
2015-07-30 01:13:22 +00:00
Timo Tijhof
19a40cd3ad resourceloader: Implement support for 'site' into mw.loader
* No longer a dedicated <script> with only=scripts.
  This means it creates no extra script request and becomes a versioned
  request using data from the startup module.

* No longer in group=site.
  This means it collapses into the existing bottom queue.
  Not even one dedicated script request, but zero.

* No longer exclude from module storage. This can be cached like any other module.
  It was previously excluded because it was already loaded separately.

* Change mw.loader#execute to special-case the 'site' module with $.globalEval.

* Add hack to ensure the styles of the 'site' module still load without
  JavaScript, in the top, and after the ResourceLoaderDynamicStyles marker.
  This unfortunately stays its own request. Not sure how to avoid that.

Bug: T32358
Bug: T106736
Bug: T102077
Change-Id: I291a8c3aae1a71760bec58161891c1bd77c9b724
2015-07-29 20:12:01 +00:00
Roan Kattouw
1139fbe58c resourceloader: Add must-revalidate to Cache-Control
Bug: T105255
Change-Id: Ifdd32560335dee3bdd3a2844c8169e5b963b18c5
2015-07-28 02:37:31 +00:00
jenkins-bot
612fbbf536 Merge "Add MediaWiki-Timestamp header to ResourceLoader requests" 2015-07-15 07:30:24 +00:00
Ori Livneh
3d5aca4a40 Add MediaWiki-Timestamp header to ResourceLoader requests
To help distinguish cache hits from cache misses, send a MediaWiki-Timestamp
header on ResourceLoader responses, indicating the current time as fractional
seconds since epoch, with microsecond precision.

Change-Id: I954a96ced1435a47d458de12411b257fe6b4ea34
2015-07-15 08:35:45 +02:00
Timo Tijhof
bdb4a243ea resourceloader: Set a TTL for minification cache entries
Follows-up 458e7cabbb. HHVM's APC cache doesn't have any
limitation or eviction logic. Thus entries without a TTL will
linger indefinitely.

At time of writing over 99% of APC entries were from "resourceloader:filter"
because there are far more unpopular resources than popular ones.

A fixed TTL is suboptimal as it will also cause popular resources
(e.g. startup module, or jquery) to be re-minified every day.

Beware this isn't one single minification. It's one minification,
for every JS/CSS resource, on every app server, for many different
lang/skin/config variations.

Bug: T104769
Change-Id: Ib62c320da0eeede68aad3f2294006ec7dcf1df55
2015-07-07 17:15:37 +00:00
Timo Tijhof
9bd84c11d2 resourceloader: Make minify cache keys globally shared instead of local
The keys already contain a hash of the contents and the version of
the filter system. No need for these to be fragmented by wiki.

Previously wiki farms would minify the same module build for
every wiki. For e.g. Wikimedia this should bring down minification
runs for a new module version from ~800x to something lower.
It won't be 1 since modules may still vary by language, or config.

This should speed up load.php responses when a new module version
is deployed.

It will also reduce response time for page views of logged-in
users due to improved minification cache of embedded modules.

Change-Id: Iee884208c5c4ba40b46abd332271df698c6afb6f
2015-07-01 03:44:55 +00:00
Timo Tijhof
71b7c21416 resourceloader: Move applyFilter() stats back to filter()
Follow-up b88e88b, which converted the count stats to include
timing information but incidentally moved the gathering point
inside the method, which meant it now also applied to uncached
minifications from logged-in user html views (called from one
branch higher in the filter method).

This should reset the 'resourceloader_cache.minify-js.miss.count'
back to normal levels.

Change-Id: I7761242f016e9026f5012352ec2f7297c1753f25
2015-06-30 02:12:40 +01:00