Follows-up Ieaf04e0c289646dd5 which changed internal references to
bool(true) for 'debug' to the integer DEBUG_ constants, and introduced
a new debug=2 parameter.
In the refactor, I missed the setDebug() calls for
DerivativeResourceLoaderContext, which were still passing a boolean,
but more importantly were effectively passing debug=1 even if the
pageview carried debug=2. This isn't a problem yet in production since
debug=2 is currently identical in behaviour to debug=1.
The impact of this issue is mainly noticed through secondary CSS
requests. The URLs for primary stylesheets, and JS modules was already
forwarding the current "debug" version.
Test Plan:
* Open Main_Page?action=edit&debug=2
* Before this patch, e.g. on mediawiki.org today, secondary
stylesheet requests (part of a JS module) have debug=1.
For example "modules=jquery.makeCollapsible.styles&only=styles".
* After this, everything has debug=2 when the page view has debug=2.
Bug: T85805
Change-Id: Ia8fba4e30397bc5890033f13417b6739b0f83c38
=== Why
* More speed
In debug mode, the server should regenerate the startup manifest
on each page view to ensure immediate effect of changes. But,
this also means more version recomputation work on the server.
For most modules, this was already quite fast on repeat views
because of OS-level file caches, and our file-hash caches and
LESS compile caches in php-apcu from ResourceLoader.
But, this makes it even faster.
* Better integration with browser devtools.
Breakpoints stay more consistently across browsers when the
URL stays the same even after you have changed the file and
reloaded the page. For static files, I believe most browsers ignore
query parameters. But for package files that come from load.php,
this was harder for browsers to guess correctly which old script URL
is logically replaced by a different one on the next page view.
=== How
Change Module::getVersionHash to return empty strings in debug mode.
I considered approaching this from StartupModule::getModuleRegistrations
instead to make the change apply only to the client-side manifest.
I decided against this because we have other calls to getVersionHash
on the server-side (such as for E-Tag calculation, and formatting
cross-wiki URLs) which would then not match the version queries that
mw.loader formats in debug mode.
Also, those calls would still be incurring some the avoidable costs.
=== Notes
* The two test cases for verifying the graceful fallback in production
if version hash computations throw an exception, were moved to a
non-debug test case as no longer happen now during the debug
(unminified) test cases.
* Avoid "PHP Notice: Undefined offset 0" in testMakeModuleResponseStartupError
by adding a fallback to empty string so that if the test fails,
it fails in a more useful way instead of aborting with this error
before the assertion happens. (Since PHPUnit generally stops on the
first error.)
* In practice, there are still "version" query parameters and E-Tag
headers in debug mode. These are not module versions, but URL
"combined versions" crafted by getCombinedVersion() in JS and PHP.
These return the constant "ztntf" in debug mode, which is the hash
of an empty string. We could alter these methods to special-case
when all inputs are and join to a still-empty string, or maybe we
just leave them be. I've done the latter for now.
Bug: T235672
Bug: T85805
Change-Id: I0e63eef4f85b13089a0aa3806a5b6f821d527a92
This is in preparation for making all version hashes the empty string
in debug mode, which will make things faster to work with, but also
helps solve T235672 in an easy way client-side without having to
add additional complexity to the mw.loader client that is specific
to debug mode.
Bug: T235672
Change-Id: I43204f22dfbcf5d236b35adc5b35df3da8021bad
In debug mode of a packageFiles module there is now only one newline
between the JavaScript code and the closing }.
This change reuses ResourceLoader::ensureNewline and make this to a
interal public static function.
Change-Id: I89519896e3dc56d966c4a63102904686bff6fac9
The native "foreign module source" feature, as used by the GlobalCssJs
extension, did not work correctly in debug mode as the urls returned
by the remote wiki were formatted as "/w/load.php...", which would
be interpreted by the browser relative to the host document, instead
of relative to the parent script.
For example:
1. Page view on en.wikipedia.org.
2. Script call to
meta.wikimedia.org/w/load.php?debug=true&modules=ext.globalCssJs.user&user
This URL is formatted by getScriptURLsForDebug on en.wikipedia.org,
when building the article HTML. It knows the modules is on Meta, and
formats it as such.
So far so good.
3. meta.wikimedia.org responds with an array of urls for sub resources.
That array contained URLs like "/w/load.php...only=scripts".
These were formatted by getScriptURLsForDebug running on Meta,
no longer with a reason to make it a Meta-Wiki URL as it isn't
perceived as cross-wiki. It is indistinguishable from debugging
a Meta-Wiki page view from its perspective.
This patch affects scenario 3 by always expanding it relative to the
current-request's wgServer. We still only do this in debug mode. There
is not yet a need to do this in non-debug mode, and if there was we'd
likely want to find a way to avoid it in the common case to keep
embedded URLs short.
The ResourceLoader::expandUrl() method is similar to the one in
Wikimedia\Minify\CSSMin.
Test Plan:
* view-source:http://mw.localhost:8080/w/load.php?debug=1&modules=site
For Module base class.
Before, the array entries were relative. After, they are full.
* view-source:http://mw.localhost:8080/w/load.php?debug=1&modules=jquery
For FileModule.
Before, the array entries were relative. After, they are full.
* view-source:http://mw.localhost:8080/wiki/Main_Page?debug=true
Unchanged.
* view-source:http://mw.localhost:8080/wiki/Main_Page
Unchanged.
Bug: T255367
Change-Id: I83919744b2677c7fb52b84089ecc60b89957d32a
About 1.5% of load.php wall-time is spent in isFileModule() calls
during the ServiceWiring/getResourceLoader/register call early on.
Reduce the overhead of this cost by moving that logic to the Module
class.
There are two costs that we save this way:
1. The inherent cost of applying the skinStyles.
This is now limited to only the modules that are constructed in a
given web request. Thus, apart from the startup response (which
constructs all modules), for regular load.php requests and all
index.php page views, the vast majority of modules will never need
to be constructed, and thus won't pay this cost.
2. The overhead of predicting (and class-loading) for whether a module
is (or will become) a FileModule object.
This is what isFileModule() does and is the main reason I wrote
this patch. It involves class loading, and checks and conditions that
run 1000+ times at WMF. This is eliminated now because we no longer
have to calculate this decision. Instead, the logic applies when
it needs to (due to FileModule implementing it), and doesn't
when it doesn't!
Change-Id: Ia2db14f930800c96e767b94ef62fb00e9d52725b
… including PHPDoc tags like `@return <type> $variableName`.
A return value doesn't have a variable name. I can see that
some people do this intentionally, repeating the variable
name that was used in the final `return $var;` at the end
of a method. This can indeed be helpful. I leave a lot of
these untouched and removed them only when it's obviously
wrong, or does not provide any additional information in
addition to what the code already says.
Change-Id: Ia18cd9f25ef658b08ad25b97a744897e2a8deffc
Modules that set "es6": true in their module definition will error when
a non-ES6 client tries to load them.
To detect ES6 support, this looks for native Promise support,
RegExp.prototype.flags, and non-BMP characters in variable names. All
browsers that lack full ES6 support fail at least one of those checks.
To flag modules as requiring ES6, this adds a ! to the end of their
version string. This takes up much less space than adding another
register() parameter (which would have to be at the end). It's hacky,
but we expect this feature to be relatively temporary, until we require
ES6 for running any JS at all (probably in about a year).
For distinguishing different types of errors thrown from
sortDependencies(), use e.name. We can't subclass Error properly because
that requires ES6.
Bug: T272104
Change-Id: I45670c910ff12eb422ae54c9fcf372e45c7b2bf1
This is micro-optimization of closure code to avoid binding the closure
to $this where it is not needed.
Created by I25a17fb22b6b669e817317a0f45051ae9c608208
Change-Id: I0ffc6200f6c6693d78a3151cb8cea7dce7c21653
- Restructure as a more readable and easier to scan list,
rather than with large sections.
- Move examples out to their own section, and remove or combine
trivial examples.
- Link to mediawiki.org docs for Package files,
and document when the feature was added (since 1.33).
- Document the "factory" option, and use the same format
for specifying when it was added.
- Add more @see references to this for discovery.
(The @see syntax is required for Doxygen to link variables,
unlike class names which can be linked bare from class/method
descriptions.)
Bug: T232566
Change-Id: Idb295d034c0afbd75cc1f57bbed163654bda2ab5
Originally introduced in r91608 (0f201b19f4), which cached
the input verbatim in Memcached, which is somewhat wasteful.
Instead, use null for the 99% case of the script being valid,
and cache that.
Also follows-up 3a748592f8 which made this a shared cache key,
which I don't think makes sense here since since there aren't any
shared resources going through this code path.
As counter example, the minification cache applies also to JS
files we deploy from Git, which are the same on all wikis and
identified by a file path that will match regardless of wiki
content, and so benefit from shared caching and can safely do so.
User scripts, however, are always stored on a particular wiki.
Even if they are loaded across domains in a browser, they will
still come through the wiki where they are stored, so this has
no added value.
Change-Id: I1615359e9ae8762f177004a02a9d3f69178e05c1
Also remove use of AtEase while at it (per updated coding conventions),
which also helps to simplify the code in this case since it means
we no longer need a 'finally' clause for the case where there is both
an ignorable compiler warning and an exception.
Change-Id: I4ec7044553db7e070f9fdb0954a8dfc834b6a9b4
This includes fixing some mistakes, as well as removing
redundant text that doesn't add new information, either because
it literally repeats what the code already says, or is actually
duplicated.
Change-Id: I3a8dd8ce57192deda8916cc444c87d7ab1a36515
Also move ResourceLoaderRegisterModulesHook under the same
namespace. I forgot this one in commit b7ac554304.
This is mainly motivated by making the collaboration graph
on doc.wikimedia.org not huge by pulling in the full range
of core hook interfaces.
<https://doc.wikimedia.org/mediawiki-core/master/php/classResourceLoader.html>
Depends-On: Ifa53d96dd3e4592208dbc991e2af9e6b7598e44d
Depends-On: Ib4571d80f55a4716e28b76ba1d007f95b0fb2540
Change-Id: I576305a5edb36a2ffb513e6d557bbf1d337cb17e
For compliance with the new version of the table interface policy
(T255803).
This patch was created by an automated search & replace operation
on the includes/ directory.
Bug: T257789
Change-Id: Ie32c1b11b3d16ddfc0c83a757327d449ff80b2e4
For compliance with the new version of the table interface policy
(T255803).
This patch was created by an automated search & replace operation
on the includes/ directory.
Bug: T257789
Change-Id: I5ffbb91882ecce2019ab644839eab5e8fb8a1c5f
* Add a void return hint to methods that are not meant to return
anything. This helps catch accidental return statements in the
future, lets Phan better understand how methods are meant to be
used, and might also allow PHP to better optimise the compiled
code form (speculation).
I did not, however, add it to publicly extended methods as that
might mess with strict mode.
* Remove the internal getResourceLoader() method from MessageBlobStore.
This has been redundant since 3edaa0b37c.
The method was protected, and not considered stable to subclass
for extensions. Hence not a breaking change.
* Add Throwable typehint to formatException() and friends.
This is the narrowest one I could add given that the methods
called from here already enforce the same typehint.
Update the doc to match for now. There isn't a reason right now
to limit this only to Exception, and given this is the method
and not the catch statement itself, does not change behaviour.
* Remove unused ResourceLoader->getHookContainer().
Added within 1.35 cycle, safe to remove.
* Remove unexpected `@since` for `@internal` getHookRunner().
* Remove redundant `@internal` from ResourceLoaderMwUrlModule
methods, which itself is already `@internal`.
Change-Id: I68d33ff6feca7ef95282a7ff03eb9332adfde31c
ResourceLoader requires user scripts to be valid ES5.
Browsers have implemented most of the ES6 spec by now, with some
of the ES6 features actually having been around for a long time now.
As such, parse errors for otherwise valid ES6 scripts, from the POV of
the script author (which can include people editing their user scripts),
this error message can be inscrutable. Try to make it more explicit.
Change-Id: I20a6c705ac44c2c41b11e4a8f16675592b6d8a87
Migrate all callers of Hooks::run() to use the new
HookContainer/HookRunner system.
General principles:
* Use DI if it is already used. We're not changing the way state is
managed in this patch.
* HookContainer is always injected, not HookRunner. HookContainer
is a service, it's a more generic interface, it is the only
thing that provides isRegistered() which is needed in some cases,
and a HookRunner can be efficiently constructed from it
(confirmed by benchmark). Because HookContainer is needed
for object construction, it is also needed by all factories.
* "Ask your friendly local base class". Big hierarchies like
SpecialPage and ApiBase have getHookContainer() and getHookRunner()
methods in the base class, and classes that extend that base class
are not expected to know or care where the base class gets its
HookContainer from.
* ProtectedHookAccessorTrait provides protected getHookContainer() and
getHookRunner() methods, getting them from the global service
container. The point of this is to ease migration to DI by ensuring
that call sites ask their local friendly base class rather than
getting a HookRunner from the service container directly.
* Private $this->hookRunner. In some smaller classes where accessor
methods did not seem warranted, there is a private HookRunner property
which is accessed directly. Very rarely (two cases), there is a
protected property, for consistency with code that conventionally
assumes protected=private, but in cases where the class might actually
be overridden, a protected accessor is preferred over a protected
property.
* The last resort: Hooks::runner(). Mostly for static, file-scope and
global code. In a few cases it was used for objects with broken
construction schemes, out of horror or laziness.
Constructors with new required arguments:
* AuthManager
* BadFileLookup
* BlockManager
* ClassicInterwikiLookup
* ContentHandlerFactory
* ContentSecurityPolicy
* DefaultOptionsManager
* DerivedPageDataUpdater
* FullSearchResultWidget
* HtmlCacheUpdater
* LanguageFactory
* LanguageNameUtils
* LinkRenderer
* LinkRendererFactory
* LocalisationCache
* MagicWordFactory
* MessageCache
* NamespaceInfo
* PageEditStash
* PageHandlerFactory
* PageUpdater
* ParserFactory
* PermissionManager
* RevisionStore
* RevisionStoreFactory
* SearchEngineConfig
* SearchEngineFactory
* SearchFormWidget
* SearchNearMatcher
* SessionBackend
* SpecialPageFactory
* UserNameUtils
* UserOptionsManager
* WatchedItemQueryService
* WatchedItemStore
Constructors with new optional arguments:
* DefaultPreferencesFactory
* Language
* LinkHolderArray
* MovePage
* Parser
* ParserCache
* PasswordReset
* Router
setHookContainer() now required after construction:
* AuthenticationProvider
* ResourceLoaderModule
* SearchEngine
Change-Id: Id442b0dbe43aba84bd5cf801d86dedc768b082c7
This can be enabled via a configuration flag. Otherwise, SqlModuleDependencyStore
will be used in order to keep using the module_deps table.
Create a dependency store class, wrapping BagOStuff, that stores known module
dependencies. Inject it into ResourceLoader and inject the path lists into
ResourceLoaderModule directly and via callback.
Bug: T113916
Change-Id: I6da55e78d5554e30e5df6b4bc45d84817f5bea15
This is only rarely called and when it is, it is generally only
once or maybe a handful of times in the same request (e.g. number
of pages within a gadget).
The constructor performs no expensive logic, either.
Originally added via 0f201b19 (r91608), though unsure why.
Change-Id: I568028addaa2f60cb956b14ea7cc44ca46aaf7f5
This protected method was forgotten to be removed in MediaWiki 1.31,
when the public ResourceLoaderModule::getHashMtime() and ::getDefinitionMtime()
methods were removed (originally deprecated in MW 1.26).
Bug: T233059
Change-Id: Ia89d9b2fad99628e60063383a3f379db3fb49c47
Avoid use of wfTimestamp and wfDebugLog global functions.
Also simplify some of the error messages around processing of
'packageFiles' definitions and throw generic LogicException
instead of MWException.
Change-Id: I55ce1f107f53dfdfe673cbe4411b0a7c4e24b2ea
getDeprecationInformation without $context is deprecated since
commit 2fc229116d (I5341f18625209446a6d00).
Depends-On: Ifa1a3bb56b731b83864022a358916c6aca5d7c10
Change-Id: Idd4aba87c65101f9ec257a7ff745fe29ed19c7c9
It was mentioning what ResourceLoader needs to do with the method
internally (such as "call it from getVersionHash"), but that is
irrelevant to the developer implementing the method in sub classes.
Also update the rest and make it hopefully a bit easier to follow.
Change-Id: Idc73e344bb4ffc3f83e2efcd358ca34f98bc5443
This change allows to use the context in the functions.
The following internal static functions from ResourceLoader get now a
reference to the ResourceLoaderContext object:
* makeLoaderImplementScript
* makeLoaderStateScript
* makeLoaderRegisterScript
* makeLoaderSourcesScript
ResouceLoader::encodeJsonForScript is duplicated to
ResourceLoaderContext::encodeJson loading the debug mode from context.
ResourceLoader::encodeJsonForScript is kept for other usages without
context.
The debug mode is loaded from $context->getDebug() instead of from
ResourceLoader::inDebugMode(). This does not support to enable the debug
mode by setting the cookie 'resourceLoaderDebug' or the configuration
variable wgResourceLoaderDebug. Only the URL parameter debug=true
enables the debug mode. This should be sufficient for the subsequent
ResourceLoader requests. The tests don't need the global variable
wgResourceLoaderDebug anymore. The initial ResourceLoader context in
OutputPage still uses ResourceLoader::inDebugMode() with cookie and
global configuration variable.
This change adds the parameter $context with a ResourceLoaderContext
object to ResourceLoaderModule::getDeprecationInformation and deprecates
omitting the parameter. Ifa1a3bb56b731b83864022a358916c6aca5d7c10
updates this in extension ExtJSBase.
Bug: T229311
Change-Id: I5341f18625209446a6d006f60244990f65530319
* Add license header where missing.
* Add missing `@since` (1.17 for most classes), except
ResourceLoaderLessVarFileModule since 1.32 (1bc62c548c).
* Remove duplicate file-level description for class-only files,
merge with the class description instead.
* Remove my own `@author` annotation from one file.
* Mark core's own FileModule subclasses as `@internal`, except
for the following which we support use of in extensions:
ResourceLoaderLessVarFileModule,
ResourceLoaderOOUIIconPackModule, and
ResourceLoaderWikiModule.
Change-Id: I336af2e4ccdbe2512594e8861b72628d24194e41
Being a raw module means that when it is requested from load.php with
"only=scripts" set, then the output is *not* wrapped in an
'mw.loader.implement' closure *and* there no 'mw.loader.state()' appendix.
Instead, it is served "raw".
Before 2018, the modules 'mediawiki' and 'jquery' were raw modules.
They were needed before the client could define 'mw.loader.implement', and
could never be valid dependencies. Module 'mediawiki' merged to 'startup',
and 'jquery' became a regular module (T192623). Based on the architecture
of modules being deliverable bundles, it doesn't make sense for there to
ever be raw modules again. Anything that 'startup' needs should be bundled
with it. Anything else is a regular module.
On top of that, we never actually needed this feature because specifying
the 'only=scripts' and 'raw=1' parameters does the same thing.
The only special bit about marking modules (not requests) as "raw" was that
it allowed the client to forget to specify "raw=1" and the server would
automatically omit the 'mw.loader.state()' appendix based on whether the
module is marked as raw. As of Ie4564ec8e26ad53f2, the two remaining use
cases for raw responses now specify the 'raw=1' request parameter, and we
can get rid of the "raw module" feature and all the complexity around it.
== Startup module
In the startup module there was an interesting use of isRaw() that has
little to do with the above. The "ATTENTION" warning there applies to the
startup module only, not raw modules in general. This is now fixed by
explicitly checking for StartupModule.
Above that warning, it talked about saving bytes, which was an optimisation
given that "raw" modules don't communicate with mw.loader, they also don't
need to be registered there because even if mw.loader would try to load
them, the server would never inform mw.loader about the module having
arrived. There are now no longer any such modules.
Bug: T201483
Change-Id: I8839036e7b2b76919b6cd3aa42ccfde4d1247899
Remove use of a complete "namespace" for grouping and instead use dashes
within the first key segment for that.
This way, we can leverage the various features WANObjectCache provides
for statistics relating to a particular kind of thing being cached.
These statistics are currently combined for all of resourceloader,
which isn't useful. Mildly related to T223647.
Change-Id: I5df69e46ac436d04e765726a36fc3eb70697a7ed
Follows-up e7b57d881a, which changed it from replace() to upsert()
but lost one of the wrapping arrays in doing so.
Previously updated many more rows than expected on Postgresql, when it
should only be updating individual rows, not all rows that match either
criteria.
SQL query before:
WHERE ((md_module = 'jquery.makeCollapsible.styles') OR (md_skin = 'vector|en-gb'))
SQL query after:
WHERE ((md_module = 'jquery.makeCollapsible.styles' AND md_skin = 'vector|en-gb'))
Not a problem on MySQL as upsert() is implemented differently there.
Bug: T222385
Change-Id: If8a458bf4543b297b3a06f31e09c0e77666bf7e6
* Use early return instead of all-encapsulating conditional.
* Document why the try/catch is so big.
Change-Id: Ie19e18556e7ac0a12ad6b979367f8c6b786bbe31
