=== Why
* More speed
In debug mode, the server should regenerate the startup manifest
on each page view to ensure immediate effect of changes. But,
this also means more version recomputation work on the server.
For most modules, this was already quite fast on repeat views
because of OS-level file caches, and our file-hash caches and
LESS compile caches in php-apcu from ResourceLoader.
But, this makes it even faster.
* Better integration with browser devtools.
Breakpoints stay more consistently across browsers when the
URL stays the same even after you have changed the file and
reloaded the page. For static files, I believe most browsers ignore
query parameters. But for package files that come from load.php,
this was harder for browsers to guess correctly which old script URL
is logically replaced by a different one on the next page view.
=== How
Change Module::getVersionHash to return empty strings in debug mode.
I considered approaching this from StartupModule::getModuleRegistrations
instead to make the change apply only to the client-side manifest.
I decided against this because we have other calls to getVersionHash
on the server-side (such as for E-Tag calculation, and formatting
cross-wiki URLs) which would then not match the version queries that
mw.loader formats in debug mode.
Also, those calls would still be incurring some the avoidable costs.
=== Notes
* The two test cases for verifying the graceful fallback in production
if version hash computations throw an exception, were moved to a
non-debug test case as no longer happen now during the debug
(unminified) test cases.
* Avoid "PHP Notice: Undefined offset 0" in testMakeModuleResponseStartupError
by adding a fallback to empty string so that if the test fails,
it fails in a more useful way instead of aborting with this error
before the assertion happens. (Since PHPUnit generally stops on the
first error.)
* In practice, there are still "version" query parameters and E-Tag
headers in debug mode. These are not module versions, but URL
"combined versions" crafted by getCombinedVersion() in JS and PHP.
These return the constant "ztntf" in debug mode, which is the hash
of an empty string. We could alter these methods to special-case
when all inputs are and join to a still-empty string, or maybe we
just leave them be. I've done the latter for now.
Bug: T235672
Bug: T85805
Change-Id: I0e63eef4f85b13089a0aa3806a5b6f821d527a92
This is in preparation for making all version hashes the empty string
in debug mode, which will make things faster to work with, but also
helps solve T235672 in an easy way client-side without having to
add additional complexity to the mw.loader client that is specific
to debug mode.
Bug: T235672
Change-Id: I43204f22dfbcf5d236b35adc5b35df3da8021bad
In debug mode of a packageFiles module there is now only one newline
between the JavaScript code and the closing }.
This change reuses ResourceLoader::ensureNewline and make this to a
interal public static function.
Change-Id: I89519896e3dc56d966c4a63102904686bff6fac9
About 1.5% of load.php wall-time is spent in isFileModule() calls
during the ServiceWiring/getResourceLoader/register call early on.
Reduce the overhead of this cost by moving that logic to the Module
class.
There are two costs that we save this way:
1. The inherent cost of applying the skinStyles.
This is now limited to only the modules that are constructed in a
given web request. Thus, apart from the startup response (which
constructs all modules), for regular load.php requests and all
index.php page views, the vast majority of modules will never need
to be constructed, and thus won't pay this cost.
2. The overhead of predicting (and class-loading) for whether a module
is (or will become) a FileModule object.
This is what isFileModule() does and is the main reason I wrote
this patch. It involves class loading, and checks and conditions that
run 1000+ times at WMF. This is eliminated now because we no longer
have to calculate this decision. Instead, the logic applies when
it needs to (due to FileModule implementing it), and doesn't
when it doesn't!
Change-Id: Ia2db14f930800c96e767b94ef62fb00e9d52725b
We’re appending at least a closing brace to the source code, so if that
code ends with a line comment, we need to have a line break between the
end of the code and our appended brace.
Bug: T286996
Change-Id: I4e93b10f31445f6dab76c1614e546fb41fd6c0f6
Ended up using
grep -Prl '\->setMethods\(' . | xargs sed -r -i 's/setMethods\(/onlyMethods\(/g'
special-casing setMethods( null ) -> onlyMethods( [] )
and then manual fix of failing test (from PS2 onwards).
Bug: T278010
Change-Id: I012dca7ae774bb430c1c44d50991ba0b633353f1
Add the SkinLessImportPaths attribute for skin-specific LESS import
paths, which skins can use to override the mediawiki.skin.variables.less
file.
As a starting point, add the following 5 variables:
* device widths (3x)
To help phase out 'mediawiki.ui/variables'. These are
commonly used by MobileFrontend.
* @font-family-sans
Recommended by Volker. Used by multiple skins.
* @border-radius-base
Recommended by Volker as example of something that we currently
hardcode in MediaWiki core for Vector and OOUI/WikimediaUI
in 'mediawiki.widgets.datetime' but should instead be allowed
to vary by skin and OOUI theme.
Remove the hardcoded value for '@border-radius-base' in
various places in favour of importing from mediawiki.skin.
The default is a bare default of 0 (as border-radius is off
by default in the browser).
The value for Vector is restored there by I47da304667811.
The value for MonoBook is improved by I000f319ab31b.
Bug: T112747
Change-Id: Icf86c930a3b5524254bb549624737d3b9dccb032
This should be the exact same. Its more a style change than anything.
So why do it then?
* I believe this is much less confusing than code mentioning a weird
"standard class". Barely anybody knows what this is, and what the
difference between "object" and "stdClass" is.
* The code is shorter.
* It's even faster. In my micro benchmark it's twice as fast.
Change-Id: I7ee0e8ae6d9264a89b6cd1dd861f0466ae620ccc
* Where possible and easy to figure out, change `array` to something like
`array<K,V>` or `V[]` for improved static analysis to catch/prevent
regressions in CI.
* Minor doc improvements:
- consistently use the imperative mood for method briefs,
- consistently use @internal instead of @private,
- explain in @throws why they happen to inform when they should be caught
(and remove if they are not meant to be caught/handled by any caller).
* Simplify addSources() implementation as a simple loop instead recursing
(not worth the complexity, only called once or twice at runtime).
* Use more granular exceptions to distinguish between errors that indicate
a mistake on the caller (logic/invalid arguments error), and runtime
errors (which are more circumstantial).
* Update register() unit test for bad 'moduleInfo' type to use
a nested value, given that the second-parameter level type is
now verified by the signature already.
Change-Id: Id98ba1f28cb7f1c72f0a3e82f4151bcbd0f3db77
This change allows to use the context in the functions.
The following internal static functions from ResourceLoader get now a
reference to the ResourceLoaderContext object:
* makeLoaderImplementScript
* makeLoaderStateScript
* makeLoaderRegisterScript
* makeLoaderSourcesScript
ResouceLoader::encodeJsonForScript is duplicated to
ResourceLoaderContext::encodeJson loading the debug mode from context.
ResourceLoader::encodeJsonForScript is kept for other usages without
context.
The debug mode is loaded from $context->getDebug() instead of from
ResourceLoader::inDebugMode(). This does not support to enable the debug
mode by setting the cookie 'resourceLoaderDebug' or the configuration
variable wgResourceLoaderDebug. Only the URL parameter debug=true
enables the debug mode. This should be sufficient for the subsequent
ResourceLoader requests. The tests don't need the global variable
wgResourceLoaderDebug anymore. The initial ResourceLoader context in
OutputPage still uses ResourceLoader::inDebugMode() with cookie and
global configuration variable.
This change adds the parameter $context with a ResourceLoaderContext
object to ResourceLoaderModule::getDeprecationInformation and deprecates
omitting the parameter. Ifa1a3bb56b731b83864022a358916c6aca5d7c10
updates this in extension ExtJSBase.
Bug: T229311
Change-Id: I5341f18625209446a6d006f60244990f65530319
assertSame() is guaranteed to never do any magic type conversion.
This can be critical when accidentially comparing empty strings (a
value PHP considers to be "falsy") to false, 0, 0.0, null, and such.
Change-Id: I2e2685c5992cae252f629a68ffe1a049f2e5ed1b
In a nut shell:
* We very often (52% of modules on enwiki) pad the hash with a zero,
which means the amount of bits we currently compute already fit in
6 characters already for most modules. For some modules (3%) we
even padded two zeroes.
* For the (now documented) use cases, the space of 78 Giga
(78 billion, or 78 milliard) seems more than we need. The space of
60 million should be enough.
This follows-up dfd046412f from 2016, which previously shortened the hash
down from 8 chars of base 64 (or 12 chars of hex) to 7 chars of base 32.
Before that change, the space was 281 Tera (64^8, or 16^12).
For more details see the added inline comment for ResourceLoader::makeHash,
and also the data at <https://phabricator.wikimedia.org/T229245>.
Bug: T229245
Change-Id: I9ad11772a33b3a44cb625275b1d7353e1393ee49
* Add a test to confirm that the ResourceLoader::respond() logic
works as intended.
* Remove the client code for preventing it from being loaded.
This can never happen in production unless there is a bug.
Instead of optimising to avoid a pointless request that only
happens when the software is broken, instead optimise for when
the software is not broken by just letting it happen. The server
already handles it just fine.
This was originally added in 2015 with 1dd7390372 to reduce
logspam, but that was instead fixed in 6d6b037e12 by making the
log message debug-only (because it's not a software problem,
it's a client-error, e.g. a broken user script or a third
party trying out different things on the load.php entry point).
Removing this makes the client a bit smaller, too :)
Change-Id: Ic5420d9329a73514f4fc27baa46ae58d94addafb
This was done as a "clever" shortcut to make sure tests a little
but shorter, but also made them less consistent with normal code.
Remove this in favour of 'class' or 'factory' options as needed.
Also remove a bunch of unneeded register() calls.
The tests cover everything affected by this change.
Side fix - isFileModule should reject modules with 'factory'
the same way it rejected raw objects and non-FileModule 'class'
cases already. This is now covered by tests as well.
Bug: T222637
Change-Id: I3996317dbcd780cc6e0f82c84e769c08a3fc42bb
Being a raw module means that when it is requested from load.php with
"only=scripts" set, then the output is *not* wrapped in an
'mw.loader.implement' closure *and* there no 'mw.loader.state()' appendix.
Instead, it is served "raw".
Before 2018, the modules 'mediawiki' and 'jquery' were raw modules.
They were needed before the client could define 'mw.loader.implement', and
could never be valid dependencies. Module 'mediawiki' merged to 'startup',
and 'jquery' became a regular module (T192623). Based on the architecture
of modules being deliverable bundles, it doesn't make sense for there to
ever be raw modules again. Anything that 'startup' needs should be bundled
with it. Anything else is a regular module.
On top of that, we never actually needed this feature because specifying
the 'only=scripts' and 'raw=1' parameters does the same thing.
The only special bit about marking modules (not requests) as "raw" was that
it allowed the client to forget to specify "raw=1" and the server would
automatically omit the 'mw.loader.state()' appendix based on whether the
module is marked as raw. As of Ie4564ec8e26ad53f2, the two remaining use
cases for raw responses now specify the 'raw=1' request parameter, and we
can get rid of the "raw module" feature and all the complexity around it.
== Startup module
In the startup module there was an interesting use of isRaw() that has
little to do with the above. The "ATTENTION" warning there applies to the
startup module only, not raw modules in general. This is now fixed by
explicitly checking for StartupModule.
Above that warning, it talked about saving bytes, which was an optimisation
given that "raw" modules don't communicate with mw.loader, they also don't
need to be registered there because even if mw.loader would try to load
them, the server would never inform mw.loader about the module having
arrived. There are now no longer any such modules.
Bug: T201483
Change-Id: I8839036e7b2b76919b6cd3aa42ccfde4d1247899
The only remaining use of 'new ResourceLoader' is in tests, which have
been migrated in this commit to either passing the real config explicitly
(for integration tests), or by passing a HashConfig from a new
'getMinimalConfig' method which has only the keys required for the tests
to pass (e.g. avoid any ConfigExeption for unknown keys).
Also clean up some related code quality issues:
* Migrate wfScript() to $conf->get() so that the local Config is used,
instead of implicitly using global variables. This isn't deprecated for
MediaWiki generally, but done here to prepare ResourceLoader for becoming
a standalone library.
* Remove mocking of 'CacheEpoch' config, this is no longer used anywhere
in ResourceLoader.
* Change EmptyResourceLoader to use the minimal config by default and
remove code duplication by calling the parent.
Update the small number of uses that are integration tests, to explicitly
pass in the live config as needed. And for the one case that tests the
'startup' module, it no longer needs to register it manually given this
is part of ResourceLoader::__construct() by default.
Bug: T32956
Change-Id: I127346fd530fa66f205156e545758b1c29d0fac0
Mark the two basic integration tests as `@medium`.
These confirm that:
* The registration method throws on invalid names.
* The higher levels are able to support an empty string as name.
Change-Id: Ib8215408a28040986ae07f4b2421681bc885abad
Also restore the order of registrations as it was before
last week with 47422fabe2. (That is, core modules are registered
before extension modules, in case of conflicts with a warning, the
core one wins).
Bug: T32956
Change-Id: I3a50508178159dfc8e5db1e218a5e6d10e2d4b2a
This has always been an odd case, as indicicated by the cross-class
comment references, and the fact that its test cases are already
in ResourceLoaderTest.php, for convenience, as that's also where
the creation of 'module name strings' is done and tested.
Actually move it there instead of pretending it is there.
Change-Id: Ied9569436cc78704a5c1b75eeebb73f8631350f6
Reduce our reliance on static state and configuration, and
propagate more state in explicit ways, through context, and
request parameters.
OutputPage creates ResourceLoaderContext and ResourceLoaderClientHtml
based on the configuration (via ResourceLoader::inDebugMode).
Everything within those classes should not need to check it
again.
* ResourceLoaderClientHtml:
Already doesn't check MW config, but it's test was still
mocking it. Removed now, and confirmed that it passes both
with true and false. The individual test cases set
debug=true/false as needed already.
It's sets were previously relying on the accidental behaviour
that within a unit test, we don't serialise over HTTP, which
meant that a pure PHP boolean would survive. With the new
raw `=== 'true'` check, this no longer works. Set it as a
string explicitly instead, which is the only thing we support
outside unit tests as well.
* ResourceLoaderContext:
Remove fallback to MW config when 'debug' is unset.
This is never unset in practice given that all load.php
urls have it set by OutputPage based on ResourceLoader::inDebugMode.
This change means that manually constructed ad-hoc load.php
urls that are missing 'debug=' parameter, will now always be
read as debug=false. This was the default already, but could
previously be changed through wgResourceLoaderDebug.
When changing wgResourceLoaderDebug, everything will still have
debug=true as before. The only change is when constructing load.php
urls manually and explicitly not set it.
Bug: T32956
Change-Id: Ie3424be46e2b8311968f3068ca08ba6a1139224a
For config vars, add a virtual package file called 'config.js', and use
require( './config.js' ) in the module. For most data modules, add a
virtual package file called 'data.js', use require( './data.js' ) in the
module. Where needed, add wrapper files that put the data in the
relevant global object and in module.exports.
LanguageDataModule is the only special-purpose module not being removed
in this commit, because it uses languageScripts, and those are not
compatible with packageFiles (yet).
Also merge mediawiki.ForeignStructuredUpload.config into
mediawiki.ForeignStructuredUpload, since that was the only thing that
used it.
Change-Id: I203d4e3ecdeeeb16729eba2dcf40d11a41d2e582
Package files are files that are part of a module, but are not
immediately executed when the module executes. Instead, they are
lazy-excecuted when require() is called on them. Package files can be
scripts (JS) or data (JSON), and can be real files on the file system,
or virtual files generated by a callback.
Using virtual data files, server-side data and config variables can be
bundled with a module. Support for file-based require() allows us to
import npm modules into ResourceLoader more easily.
The require function passed to each script execution context, which was
previously a reference to the global mw.loader.require() function, is
changed to one that is scoped to the module and the file being executed.
This is needed to support relative paths: require( '../foo.js' ) can
mean a different file depending on the path of the calling file.
The results of require()ing each file (i.e. the value of module.exports
after executing it) are stored, and calling require() on the same file a
second time won't execute it again, but will return the stored value.
Miscellaneous changes:
- Add XmlJsCode::encodeObject(), which combines an associative array of
XmlJsCode objects into one larger XmlJsCode object. This is needed for
encoding the packageFiles parameter in mw.loader.implement() calls.
Bug: T133462
Change-Id: I78cc86e626de0720397718cd2bed8ed279579112
This might hint at an edge-case in the PHP CodeSniffer sniff that should
detect if methods are separated by a single empty line. Feel free to
investigate. I, personally, can't invest more time in this than
suggesting this quick fix.
Change-Id: Ib3c60eac76f255b4fe929f7933de256222716576
* There is only a single non-test caller to this method in the entire
codebase, and no callers elsewhere (Wikimedia Git, Codesearch).
It's only used with an array, so remove the other unused code paths,
and mark it internal (to my knowledge, nothing ever used it outside
RL in the past, either).
* Add test coverage for the module indexing logic.
Change-Id: I9e0f95416d5b2fdd87323288231ee6d8c85d88e7
Follows-up b7b84d55d4, which embedded the whole of the three 'mediawiki'
JS files inside of startup.js, but did so inside of the pre-existing
closure that was there.
This adds some overhead we don't get value of, so best to remove
it and embed it as a sibling of startup.js in the same top-level
scope.
In local testing (Chrome stable, CPU 1/6th) this reduced startup
run-time from 73-78ms to 63-65ms.
Also:
* Declare isCompatible() as a normal function.
Disable the implicit-globals warning given this isn't a regular module,
this file is intentionally in the global scope.
* Unfold the private startUp() function to its call site.
Change-Id: Ida51ab20898c9e4ae6cbf7ad2968d88d824a1483
The PHP interface still supports it without deprecation, but there
is no need for the client to support it given it is only ever called
from the startup module, which always passes it an object.
Follows-up 6815e35575, which did the same for `state(key, val)`.
Also simplify the implementation to avoid extra calls and optimise
for the only signature.
Mark the method as @private, and change the existing use of
@protected to @private (doesn't really make sense in JavaScript).
Change-Id: I24786f90bcfb256b2e7c37f7760bba1a5e399443
The use of global variables was deprecated in favour of
ResourceLoaderModule::getLessVars() on a per-module basis.
Also moved testLessFileCompilation case to the appropiate file as it
covers ResourceLoaderFileModule.php, not ResourceLoader.php.
Bug: T140804
Depends-On: Ib1b2808df2384473bfac47f53a5d25d7c9bbca2b
Depends-On: I96047f69d01c4736306df2719267e6347daf556f
Change-Id: If708087c85c80355c7e78f1768529b5f2e16ed07
- Cover case of simple module load.
The bulk of this use case is already covered by a lower-level
test for makeModuleResponse(). The added case here exists
to cover the wrapper method, ResourceLoader::respond().
- Cover logic for catching and logging internal errors.
Change-Id: I4315bb00137ff80ee2b790c6b4d4b5fbd93f6bc1
The last remaining users of this feature were MobileFrontend and Minerva,
which have been migrated to direct imports.
Bug: T140807
Change-Id: I1a66a2ad314bde332297798520e5ec3e0e3d4c9b
Adds coverage for line 1049-1056.
Also, follow-up 6292d54dff by simpliying the regex by using /s modifier to
enable PCRE_DOTALL which includes matching of new lines.
Change-Id: Icec34dfe107d418951b3d155234295c79410ffaa
This prevents cache churn when the wiki-global LESS variables vary
between wikis because the cache key is used as a "global" instead of
db-local. This is good for the common case, but should still explicitly
vary if the vars differ between wikis.
Bug: T191937
Change-Id: If12fd07a7062792205384150d6f5fd9a83f996cc
This was previously covered implicitly by an unrelated test.
Change that test (dependency.less) to use ../ to access the file
directly so that that test case is only about tracking dependencies
and testing the parser.
Then, add a second case that tests the use of import dirs.
Bug: T140807
Change-Id: Ie85abffe313922c03b3e146422f36b1d6a79743d
