Commit graph

115 commits

Author SHA1 Message Date
Timo Tijhof
a5ce1d7792 resourceloader: Add Doxygen group and improve overall docs
* Add license header where missing.
* Add missing `@since` (1.17 for most classes), except
  ResourceLoaderLessVarFileModule since 1.32 (1bc62c548c).

* Remove duplicate file-level description for class-only files,
  merge with the class description instead.

* Remove my own `@author` annotation from one file.

* Mark core's own FileModule subclasses as `@internal`, except
  for the following which we support use of in extensions:
  ResourceLoaderLessVarFileModule,
  ResourceLoaderOOUIIconPackModule, and
  ResourceLoaderWikiModule.

Change-Id: I336af2e4ccdbe2512594e8861b72628d24194e41
2019-09-14 18:37:36 +00:00
Timo Tijhof
e9cc45629e resourceloader: Improve ResourceLoaderWikiModule test coverage
* Remove redundant getContent() cases that were all testing the same.
  The redirect logic should indeed be tested, but exists in getContentObj(),
  not getContent(). This test was also mocking getContentObj() thus not
  actually testing what the case claims to test.
  Fortunately, the right test already exists (testGetContentForRedirects),
  so this is redundant.

* Add actual coverage of successful outcomes for getContent (previously
  they were all error/null cases), with JS content, and with CSS content.

* Fix broken test case for "Bad content model". This was not working because
  it mocked out getContentObj, thus it wasn't actually testing "bad content model",
  but rather pointlessly duplicated the previous test case.
  Fix it by actually making it use a WikitextContent object, which makes it
  test the branch that handles incompatible content models.

Change-Id: I59af5318e536c730755352e9be8f995df1f56a86
2019-07-25 23:35:21 +01:00
Derick Alangi
339211a1ea Avoid usage of deprecated Revision::* constants, use RevisionRecord
Change-Id: I872fc89e5c02dd6a3ae9cd7e76640b95dc33f514
2019-07-21 15:03:03 +01:00
Timo Tijhof
253bee13bf resourceloader: Change cache keys to use first segment for grouping by purpose
Remove use of a complete "namespace" for grouping and instead use dashes
within the first key segment for that.

This way, we can leverage the various features WANObjectCache provides
for statistics relating to a particular kind of thing being cached.

These statistics are currently combined for all of resourceloader,
which isn't useful. Mildly related to T223647.

Change-Id: I5df69e46ac436d04e765726a36fc3eb70697a7ed
2019-06-13 18:04:17 +00:00
Timo Tijhof
472d3f4cb4 resourceloader: Remove unused logic for WikiModule::getDB() as null
This was intended at some point for GlobalCssJs but ended up
being implemented differently once we realised that not having
any information about the remote wiki was unacceptable.

The local placeholder that points to metawiki (in case of GlobalCssJs)
must still at the very least be able to query version information
so that it gets a new version hash after changes etc. to ensure
cache rolls over.

This is being removed in preparation for refactoring where
compatibility with this broken model is no longer kept.

Change-Id: I4f1a06ace881daf1a5f202bdba0075460804de57
2019-04-17 19:27:27 +00:00
Timo Tijhof
fa05976f5f resourceloader: Remove needless MW-specific fallback
This is never used by the mw.loader client code. For any
manually crafted requests or fringe usage where the load.php
might be used to fetch some raw JS code that doesn't vary by
language or skin, proceed with qqx/fallback.

In the future load.php might deny these requests and then we
could make this a constructor parameter to ResourceLoaderContext.

Bug: T32956
Change-Id: I4e4ee758cd22278cea9592d4745b4f7fc00e0add
2019-04-15 17:46:41 +00:00
Jack Phoenix
57bfd3d678 resourceloader: Fix variable spacings where need be
This fix is to make sure resource loader code conforms with convension
https://www.mediawiki.org/wiki/Manual:Coding_conventions/PHP#Spaces.

Change-Id: I34c0a0addbf7c373fc3a64b644a3098bb485a3f4
2019-04-06 13:09:05 +00:00
Aaron Schulz
a26eee40e0 Avoid use of deprecated ObjectCache::getMainWANInstance method
Change-Id: I429e68c8642a00afa812f5ae900c1fb4f8308528
2019-03-01 02:38:55 +00:00
Aaron Schulz
633be72f6f Make invalidateModuleCache() take a database domain ID
Change-Id: I5d9880a5aee4b055365549671258eb9f242a37d6
2019-02-06 12:40:11 -08:00
daniel
c5deaec28c Don't pass false as a wikiId to invalidateModuleCache()
Using false to represent the local wiki is supported in many places in core,
but not in invalidateModuleCache

Bug: T199416
Change-Id: I5cb93c173295261bf5fcf6c77c197a80c4aac1a2
2018-08-03 14:47:29 +02:00
Umherirrender
130ec2523d Fix PhanTypeMismatchDeclaredParam
Auto fix MediaWiki.Commenting.FunctionComment.DefaultNullTypeParam sniff

Change-Id: I865323fd0295aabd06f3e3c75e0e5043fb31069e
2018-07-07 00:34:30 +00:00
daniel
e8632ab0f6 [MCR] Factor PageUpdater out of WikiPage
This introduces PageUpdater to replace WikiPage::doEditContent,
and DerivedPageDataUpdater, to replace WikiPage::doEditUpdates
and WikiPage::prepareContentForEdit.

See docs/pageupdater.txt for a description of their
functionality.

MCR migration notes:

* The interface of PageUpdater is expected to
remain mostly stable after this patch. Code that has been using
WikiPage::doEditContent can be confidently migrated to using the
new mechanism for revision creation.

* This patch keeps the code inside PageUpdater largely aligned
with the old code in WikiPage, to make review easier to to avoid
mistakes. It is intended to be refactored further, moving
application logic into stateless services.

* DerivedPageDataUpdate is intended as a stepping stone for further
refactoring. Its behavior is designed to be compatible with
callback code that currently relies on
WikiPage::prepareContentForEdit. Much of the code that currently
lives in DerivedPageDataUpdate should be factored out into
services, all behavior relevant to calling code should be exposed
via narrow interfaces.

Bug: T174038
Bug: T196653
Change-Id: If610c68f4912e89af616cdcac1d35a1be3946afa
2018-06-14 13:22:13 +00:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
Brad Jorsch
3f1142045f Generalize ResourceLoader 'excludepage' functionality
There has long been a hack for previewing edits to user JS/CSS, where
OutputPage would pass an 'excludepage' parameter to
ResourceLoaderUserModule to tell it not to load one particular page and
would instead embed that page statically. That's nice, but there are
other places where we could use the same thing.

This patch generalizes it:
* DerivativeResourceLoaderContext may now contain a callback for mapping
  titles to replacement Content objects.
* ResourceLoaderWikiModule::getContent() uses the overrides, and
  requests embedding when they're used. All subclasses in Gerrit should
  pick it up automatically.
* OutputPage gains methods for callers to add to the override mapping,
  which it passes on to RL. It loses a bunch of the special casing it
  had for the 'user' and 'user.styles' modules.
* EditPage sets the overrides on OutputPage when doing the preview, as
  does ApiParse for prop=headhtml. TemplateSandbox does too in I83fa0856.
* OutputPage::userCanPreview() gets less specific to editing user CSS
  and JS, since RL now handles the embedding based on the actual
  modules' dependencies and EditPage only requests it on preview.

ApiParse also gets a new hook to support TemplateSandbox's API
integration (used in I83fa0856).

Bug: T112474
Change-Id: Ib9d2ce42931c1de8372e231314a1f672d7e2ac0e
2018-04-25 00:37:08 +00:00
jdlrobson
c3f200849b resourceloader: Consider having dependencies as non-empty in WikiModule::isKnownEmpty
If a module itself is empty, it must consider any dependencies
it has before bailing out as empty.

Bug: T191596
Change-Id: I2b45b948a6f78060e53513d3b4b77f48d7bf4a6b
2018-04-24 20:56:32 +00:00
Timo Tijhof
dfb754cbc6 resourceloader: Improve titleInfo docs and simplify title key
* Document the structure of the in-process $titleInfo cache.
  Specifically, specify that it is not the value from getTitleInfo(),
  but rather a container for zero or more versions of such values.
  The reason this is fragmented is because ResourceLoaderContext
  is a parameter to most methods and as such, makes everything
  variable. Tracked as T99107.

* Make various bits easier to understand by consistently refering
  to the container keys as "batchKey", and referring to the internal
  keys as "titleKey".

* Centralise title key logic by moving to private method.

* Replace the internal creation of titleKey to be based on LinkTarget
  with plain namespace IDs and db keys, instead of invoking the
  expensive getPrefixedTitle function which involves quite a lot
  of overhead (TitleCodec, GenderCache, Database, Language,
  LocalisationCache, ..).

Change-Id: I701e5156ef7815a0e36caefae5871524eff3f688
2018-04-10 19:44:40 +01:00
James D. Forrester
6d4e15476c Title: Refactor JS/CSS page handling to be more sane
Change-Id: Ia7837dc614dcc8896a7d4b6d663dc45b6bd4f7ee
2018-02-16 17:35:12 +00:00
daniel
6af796f3e0 MCR: Deprecate and gut Revision class
This is a re-submission of I4f24e7fbb68.

As a first major step towards Multi-Content-Revisions (MCR),
this patch turns the Revision class into a legacy proxy for
the new RevisionRecord and RevisionStore classes.

Backwards compatibility is maintained for all but some
rare edge cases, like constructing a completely empty
Revision object.

For more information on MCR, see
<https://www.mediawiki.org/wiki/Requests_for_comment/Multi-Content_Revisions>.

NOTE: once this is merged, verify create/delete/restore cycle on beta,
      ideally with emulated replication lag.

Bug: T174025
Change-Id: Ia4c20a91e98df0b9b14b138eb4825c55e5200384
2017-12-21 18:08:54 +00:00
Daniel Kinzler
09bf4f5bb2 Revert "[MCR] Turn Revision into a proxy to new code."
This reverts commit 9dcc56b3c9.

With this patch applied, newly created revisions are sometimes not found
just after submitting an edit, until replicas have caught up.

Our best theory is that it somehow interfere with ChronologyProtector,
but we don't have a good idea how.

Also, as legoktm mentioned, the commit message is terrible and needs fixing.

Change-Id: Idf3404f3fa8f8d08a7fb2ab8268726e2c1edecfe
2017-12-19 12:38:48 +00:00
daniel
9dcc56b3c9 [MCR] Turn Revision into a proxy to new code.
Change-Id: I4f24e7fbb683cb51f3fd8b250732bae9c7541ba2
2017-12-18 14:37:29 +00:00
Aaron Schulz
a005f1de98 Clean up wiki ID and DB domain ID handling
Bug: T174017
Change-Id: I42299a393c6691d39817db1d83a176a17df2474b
2017-09-30 10:25:03 +00:00
Aaron Schulz
d735dc562d Move Database and subclasses to Rdbms namespace
Change-Id: I52bef87512f9ddd155d1f4cc0052f6b7a0db5b42
2017-04-12 10:43:57 -07:00
Fomafix
0d247d202a Document return string as JavaScript code for getScript
Change-Id: I01055c2b6a11dbe6284d1aff2352ba428ed9bee2
2017-04-03 10:24:41 +02:00
Timo Tijhof
c505293cfd resourceloader: Improve code coverage for WikiModuleTest
preloadTitleInfo:
* Add missing case for empty $moduleNames.
* Add missing case for invalid page names.

getContent:
* Add missing case for bad title
* Add missing case for dead redirect.
* Add missing case for no content found.

Change-Id: I44dde13cb0db19d91c4ff15a5abefd17353cad90
2017-03-31 18:53:00 -07:00
Aaron Schulz
488a647831 Move IDatabase/IMaintainableDatabase to Rdbms namespace
Change-Id: If7e8a8ff574661fd827de8bcec11d2c39a687300
2017-03-28 15:32:38 -07:00
Kaldari
771e85be6c resourceloader: Follow redirects for JavaScript/CSS in WikiModule
Bug: T108653
Change-Id: I9a321fc5728f3bf9df950c060c92e4148cf3ef93
2017-03-14 19:03:26 -07:00
jenkins-bot
17eda64357 Merge "includes: Replace implicit Bugzilla bug numbers with Phab ones" 2017-02-28 00:51:57 +00:00
Timo Tijhof
49948457ec resourceloader: Optimize WikiModule preload for reqs without wiki modules
Currently it was still going through fetchTitleInfo() with an empty array on
the majority of requests without wiki modules, e.g. load.php?modules=jquery.

Bug: T158813
Change-Id: Ie33a2b4da572bb30b2e7a69db07790724ec2f03f
2017-02-22 21:42:16 +00:00
James D. Forrester
9635dda73a includes: Replace implicit Bugzilla bug numbers with Phab ones
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.

Change-Id: I6f59febaf8fc96e80f8cfc11f4356283f461142a
2017-02-21 18:13:24 +00:00
Timo Tijhof
189c906062 resourceloader: Remove unused getPosition() code
Unused as of bc374082fa (T109837).

Change-Id: I1d8f7109bbe49700f1824fdce0439e958e84f6fa
2016-11-18 19:02:12 +00:00
Timo Tijhof
baf878fb04 resourceloader: Use cached Revision::newKnownCurrent for WikiModule
Change-Id: I0c68c649783042a959dc20d9675daae790d82cb1
2016-11-07 23:10:24 +00:00
Timo Tijhof
458d37f205 OutputPage: Reduce getTitleInfo() calls (improve preloading)
* Remove filter between $moduleStyles and preloadTitleInfo().
  Previously, this was filtering out wiki modules created by
  gadgets and extensions. Still causing a couple of direct queries
  from getTitleInfo().

* Store __METHOD__ in $fname outside getWithSetCallback.
  This way queries are logged as either getTitleInfo or preloadTitleInfo.
  This regressed in 0852a000a5, after which the latter was logged
  as "::{closure}".

Change-Id: I454e43f43f8ad3270b86f4cfdbd68192c305cc07
2016-10-20 21:59:23 +00:00
Aaron Schulz
0852a000a5 Add caching to ResourceLoaderWikiModule::preloadTitleInfo()
This is one of the top three DB queries showing up in xenon
reverse flamegraph profiling.

It works via a per-wiki check key that is bumped whenever
someone changes a .js or .css page on that wiki.

Change-Id: I73f419558864ba3403b4601a098f6aaf84a3e7c1
2016-10-20 20:54:11 +00:00
Timo Tijhof
1e74a83a21 resourceloader: Fix undefined getPrefixedText()
Follows-up dbe592df. Used to go in one-way only and silenced by
addObj(). Now that we're iterating directly as well, need a
conditional.

Bug: T146728
Change-Id: Ief9d44cc5cb8f5d687560f59cc696c5671546163
2016-09-28 04:38:27 +00:00
Timo Tijhof
dbe592df6d resourceloader: Fix WikiModule preload to support localised titles
Follows-up 6f8dc27ca2 and dbd11e04aa. You'd expect a bug in preloading
to fallback to fetching it on-demand but due to a title format mismatch
the preload method did use the same title format for the cache key, but
not the same title format for discovering the results. As such, it set
the right cache key to an empty array.

* Make relevant methods testable and mockable.
* Add regression test.
* Change call to array_interect_key to use the same format as
  fetchTitleInfo(): Normalised title keys from Title::getPrefixedText.
  Previously, the intersect used the declared titles from getPages() which
  are not localised - causing an empty array to be returned from the
  intersect on wikis where the namespace name is localised.

Bug: T145673
Change-Id: Ibe788157724d73c727b9e2127b6828db32ca9420
2016-09-14 14:54:01 -07:00
Timo Tijhof
dbd11e04aa resourceloader: Preload WikiModule::getTitleInfo() in batches
Test plan:
* Request load.php?modules=startup&only=scripts
* Debug log receives 1 entry in the 'queries' channel from
  ResourceLoaderWikiModule, and it comes from preloadTitleInfo.

Bug: T46362
Change-Id: I82e215745af6b8446cd69282b2c943afbbd45065
2016-09-07 20:31:13 -07:00
Timo Tijhof
717fc889c4 resourceloader: Simplify WikiModule::getTitleInfo() DB query
While rev_sha1 was preferred to rev_id (page_latest) to allow
client-cache to be re-used in case of a bad change and revert,
this is no longer possible since we recently added support for
explicit purging by considering page_touched.

As such, use of rev_sha1 is no longer useful.

Change-Id: Iddb65305ca4655098fdea9fcf736fd4046035dd7
2016-09-08 03:29:58 +00:00
Aaron Schulz
950cf6016c Rename DB_SLAVE constant to DB_REPLICA
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.

The old constant is an alias now.

Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
2016-09-05 22:55:53 -07:00
Aaron Schulz
16266edff3 Change "slave" => "replica DB" in /includes
Change-Id: Icb716219c9335ff8fa447b1733d04b71d9712bf9
2016-09-05 21:01:01 +00:00
Timo Tijhof
a464d1d41d resourceloader: Track state of page-style modules
This allows dynamically loaded modules to depend on page-style modules
without it causing the page-style module to be loaded a second time.

* New method Module::getType() indicates whether a module is
  a page-style module or supposed to be dynamically loaded.

* Emit warning from addModuleStyles() when given a module that is
  not a page-style module (to be enforced later)

Bug: T92459
Bug: T87871
Change-Id: I8b6c6a10d965e73965f877c42e995d04202524f3
2016-07-14 20:28:40 +01:00
Krinkle
daf49115f9 Revert "resourceloader: Add support for variables in WikiModule"
No longer needed per doing Ic137cb494ba23 in a different way.
This may be useful to revisit, but for now preferring to keep
simplicity and removing this unused option.

This reverts commit 9e217bf42d.

Change-Id: I9c0c316a3b58a3d0a3d3282dd74c7fa4eef8e378
2016-06-09 19:27:34 +00:00
Timo Tijhof
9e217bf42d resourceloader: Add support for variables in WikiModule
To be used for the 'site' module. This will make it easier to split up
'site' into 'site' and 'site.styles' (per T92459).

Change-Id: Iaac3e458d5107e4c10c2826bd64608d5c47e8b87
2016-06-06 19:09:32 +01:00
Max Semenik
2fe7f8be7c Allow specifying targets for ResourceLoaderWikiModule
Change-Id: Ic8f29aecfbd1e6bfaef44405190043055ce6215a
2016-05-02 11:32:20 -07:00
Timo Tijhof
435db17da8 resourceloader: Make action=purge affect ResourceLoaderWikiModule
Purging a wiki page included in a WikiModule should bump its version hash.
This is not needed by design, but it's a workaround until our caching layers
are more resilient against cache poisoning.

This gives users the power to resolve issues themselves.

Change-Id: I37d3379e76a6b652268c5a007e0ad741470fe6e1
2016-04-03 02:41:11 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Timo Tijhof
afcfc3290c resourceloader: Consistently refer to the framework as ResourceLoader
Change-Id: Ia59e4eac9662723e80d62f7cfcb9e4292e3ee4de
2015-10-28 03:24:40 +00:00
Timo Tijhof
9435cd81b0 resoureloader: Consolidate styles-only queue at the top
This effectively reverts d6b4d3c537 and declines T97420.
This was previously attempted in b7c0e537eb.

Drop support for position "bottom" for addModuleStyles().
This feature was only recently introduced with the intent
to optimise page load performance, but had an adverse effect.

It increases chances of FOUC due to late discovery of these styles.
It also caused minor problems for some gadgets and extensions
that did not or were unable to set these flags. Some mobile
code was introduced around the same time as this feature and
was never given position=top.

Stylesheets that don't affect initial render or are only needed
on interaction should be loaded via addModules() instead; which
is handled by the asynchronous loader in JavaScript.

Change-Id: Ib9821b7b87cfc8a59062bb6ca358974fdb01ced1
2015-10-27 03:39:34 +00:00
Ori Livneh
32705d44cf resourceloader: Wrap another CSSMin::remap call in MemoizedCallable::call
Follows-up da3443bfa2.

Change-Id: Iabfe65b918ef5f8eb589bedd7a1e72b3ad9d2952
2015-10-20 21:05:37 +00:00
Kunal Mehta
ad9f14d662 Support redirects in JavaScriptContent
When a JavaScript page is moved, a "redirect" in the form of
mw.loader.load(...) will be left behind, so any other
JavaScript loading the page that way will still work, albeit
with an extra HTTP request.

This also implements Content::getRedirectTarget(), so redirects
are marked properly in the database, and users viewing them
are redirected properly. A magic "/* #REDIRECT */" comment
must be in front of the mw.loader.load call. This is done so
that pages which currently are just one mw.loader.load call
aren't turned into redirects.

Bug: 71200
Bug: 33973
Change-Id: I10fdff087a901da56fad64531f0e382f90ebcf37
2015-07-20 15:36:49 +00:00
umherirrender
70f3afd548 Remove unneeded empty lines at begin of if/else/foreach body
An if body must not begin with an empty line

Change-Id: I62b058be337fcc85a120fcd3dadce564db59a271
2015-06-19 20:05:45 +02:00