Commit graph

411 commits

Author SHA1 Message Date
Stanislav Malyshev
7e18cfc3b5 Infrastructure for augmenting search results
Bug: T117493
Change-Id: Ia5413a7846cc961026a2dc3542b619493bc76a23
2016-09-15 15:44:03 -07:00
jenkins-bot
0387ce5cc6 Merge "Rename LBFactory => LBFactoryMW and make LBFactory in /libs" 2016-09-15 17:39:35 +00:00
Aaron Schulz
0e5cd18b74 Rename LBFactory => LBFactoryMW and make LBFactory in /libs
The former extends the later with MW-specific logic.

Also removed a wf* method call from ChronologyProtector.

Change-Id: I325f59b7467ab9c2137731d1ce69816f5a020f03
2016-09-15 17:14:32 +00:00
jenkins-bot
efe855145d Merge "Move Linker::formatTemplates() to separate class, remove global usage" 2016-09-15 16:52:10 +00:00
Aaron Schulz
aeedfb8526 Remove unused LBFactoryFake class
Change-Id: I88acf72fc32c7e3c9bead81d11c5adfd8dff6151
2016-09-14 22:02:43 -07:00
Aaron Schulz
0bbba6e0d4 Move various DB helper classes to /libs/rdbms
Change-Id: I0724f1acce4f6c43b1f0983fa119e628e7c53ba5
2016-09-14 20:40:12 -07:00
Aaron Schulz
c1c68c92f2 Move LoadBalancer to /libs/rdbms
Change-Id: Ic87f1cc84ba44be20f345944406e1d8a34be1c96
2016-09-14 20:27:08 -07:00
jenkins-bot
21e3f9ca43 Merge "Exception cleanups for LoadBalancer" 2016-09-15 02:19:03 +00:00
Aaron Schulz
5820c6dcac Exception cleanups for LoadBalancer
* Make the error message itself be dumb/raw.
* Make the exception exposer show the same GUI message.
* Remove overzelous wgLang check in MWExceptionRenderer.

Change-Id: Ifffff3b3cc785ea3080e4975efe33b3c2cf304d6
2016-09-15 01:32:43 +00:00
jenkins-bot
7813e93a1c Merge "Move DBConnRef and ChronologyProtector to /libs/rdbms" 2016-09-15 00:46:03 +00:00
jenkins-bot
c78b405080 Merge "Move TransactionProfiler to /libs/rdbms" 2016-09-15 00:43:34 +00:00
Aaron Schulz
4f509e92ab Move DBConnRef and ChronologyProtector to /libs/rdbms
Change-Id: If2c4b314a5c39311328843f534d91bf90823e179
2016-09-14 17:29:54 -07:00
Aaron Schulz
9d419f69b1 Move IDatabase/ILoadBalancer to /libs/rdbms
Change-Id: I71f980b1024dc2fbca9cafbad2c9daebb6f28c9b
2016-09-14 16:34:50 -07:00
Aaron Schulz
048da5273b Move DBError to /libs/rdbms/exception
Change-Id: I2ad40db66a68f03fb34a63278a652e0505195076
2016-09-14 11:55:33 -07:00
Aaron Schulz
00bee02971 Add MWExceptionRenderer class and decouple DBError
* This handles the work of showing exceptions so that
  MWException does not have too.
* Simplify the DBError classes to regular Exception
  classes. Lots of pointless prettification has been
  removed, but DBConnectionError still gets the usual
  special treatment of a fallback page and Google form.
* Remove hacky file cache fallback code that probably
  did not work.
* Make MWExceptionHandler::report() wrap
  MWExceptionExposer::output().
* Make MWException::runHooks() wrap
  MWExceptionExposer::runHooks().

Change-Id: I5dfdc84e94ddac65417226cf7c84513ebb9f9faa
2016-09-14 11:53:55 -07:00
Aaron Schulz
531aa2906e Move TransactionProfiler to /libs/rdbms
Change-Id: I13173facc9f101bca69e3bf3e85f292f8468e855
2016-09-14 05:18:57 -07:00
Aaron Schulz
9c8d885060 Move LoadMonitor classes to libs/rdbms/loadmonitor
Change-Id: Ib18108f24ff8b9d05dc324bad132f597b3e2ddef
2016-09-14 10:56:52 +00:00
Aaron Schulz
6e014bb753 Add ILoadBalancer interface
Remove redundant LoadBalancer docs (those without @since).

Change-Id: I981177b6854cfdbea4a51b6db7e365dac0da258a
2016-09-13 19:53:36 -07:00
Kunal Mehta
b16f897a05 Move Linker::formatTemplates() to separate class, remove global usage
Linker::formatTemplates() was a static function that depended upon
global state like $wgLang (explicitly), $wgUser & $wgTitle (implicitly).
Moving it to a separate class allows us to clean it up a little bit and
use modern things like RequestContext and LinkRenderer.

Bug: T145177
Change-Id: Icdea8a2b299b4876feb3df3d66df3e4c104dd928
2016-09-08 22:56:57 -07:00
Aaron Schulz
05589cad6b Move PECL memcached class to /libs
Remove IP class dependency, which is simple enough here

Change-Id: I3aed6ae6747d8d2b9e75b492e5d5293ba4ce9edd
2016-09-07 22:55:11 -07:00
Timo Tijhof
b2585a95b2 filecache: Remove unused ObjectFileCache class
This was originally intended for use by ResourceLoader but it was
incomplete and not actually used by ResourceLoader at first.

Then 93d358cd added support for FileCache to ResourceLoader with
a separate ResourceFileCache class.

Also fixed typo in HTMLFileCache from 6559b8bf.

ObjectFileCache is not mentioned anywhere else in Wikimedia Git.

Change-Id: I69cca27ee7cd922da12f1793660432709c273be6
2016-09-06 17:09:50 -07:00
jenkins-bot
a206443c0e Merge "Change "slave" => "replica DB" in /maintenance" 2016-09-06 01:05:50 +00:00
Aaron Schulz
c0a9ab0f6d Change "slave" => "replica DB" in /maintenance
Change-Id: Ibd3d617901130378a935402326cd4eefbb382c9e
2016-09-06 00:13:08 +00:00
Aaron Schulz
e99bb1b7dc objectcache: allow for callbacks to mask SYNC_WRITE latency
Change-Id: I908222ad3788ebe330aa58831cda139da32becd8
2016-09-01 13:44:18 -07:00
Aaron Schulz
9b81de7960 Move some DB error classes with the rest as type DBUnexpectedError
Change-Id: I97d61d32495e6d88ce274772369390c76d7ff3c4
2016-08-29 23:22:21 +00:00
Bartosz Dziewoński
3b84eb02c2 Implement NumericUppercaseCollation
This collation orders text with numbers "naturally", so that
'Foo 1' < 'Foo 2' < 'Foo 12'.

Note that this only works in terms of sequences of digits, and the
behavior for decimal fractions or pretty-formatted numbers may be
unexpected.

This is only expected to work mostly correctly for English-language
text. Consider it a proof of concept. You probably want to use
an UCA collation with '-u-kn' suffix rather than this.

Bug: T8948
Change-Id: Ie268f2d92c5c75d0aaecf54ede2bdda1af3b309d
2016-08-23 18:41:01 +00:00
Aaron Schulz
420efbfb61 Move EnqueueableDataUpdate to a separate file
Change-Id: Iabc291cd1f3c5390ca8bcc5da64a0ff01a082575
2016-08-22 21:20:49 -07:00
Aaron Schulz
f16eb73ecf Move invalidatePages() to new PurgeJobUtils class
This does not really belong in SqlDataUpdate.

Change-Id: I7166e50696483371f95db3a8b6bce44b0f866ccd
2016-08-23 01:09:08 +00:00
jenkins-bot
6a72a63e31 Merge "Split DBLockManager classes into their own files" 2016-08-22 17:37:34 +00:00
Bartosz Dziewoński
89107070d1 Support 'hide-if' parameters in OOUI HTMLForm
For plain HTML forms, we just put the required data in the 'data-hide-if'
attribute. For OOUI, it's not so easy - while we could just call
->setAttribute(...) on the FieldLayout, this would disappear when
infusing (since it's not part of the config), and we have no control over
when some piece of JavaScript decides to infuse the element. Even if we
managed to handle it first, infusing replaces the DOM nodes for elements
with new ones, which would "disable" our event handlers.

To solve this, I'm creating two new layouts HTMLFormFieldLayout and
HTMLFormActionFieldLayout (subclassing FieldLayout and ActionFieldLayout)
with a common trait (mixin) HTMLFormElement. This is all implemented both
in PHP and JS. Right now it only serves to carry the 'hide-if' data from
PHP to JS code, but I imagine it'll be extended in the future for other
HTMLForm features not yet present in the OOUI version (e.g. 'cloner'
fields).

The code in hide-if.js has been modified to work with jQuery objects or
with OOjs UI Widgets with minimal changes. I had to duplicate the map of
HTMLFormField classes to modules they require there (from autoinfuse.js),
which is ugly - I'm fixing this in a follow-up commit
I3da75706209cbc16b19cc3f02b355e58ca75fec9.

Bug: T141558
Change-Id: I3b06a6f75eed01d3e0bdc5dd33e1b40b7a2fc0a2
2016-08-22 15:42:22 +00:00
Aaron Schulz
7f3faf091f Split DBLockManager classes into their own files
Change-Id: If903a90a5be2d6ff11504d34eb125e86c1ab1191
2016-08-20 10:33:37 -07:00
jenkins-bot
4b70bc2b28 Merge "Extract ParserOutput search index data fields from WikiTextContentHandler" 2016-08-19 18:40:17 +00:00
aude
64ee3d3269 Extract ParserOutput search index data fields from WikiTextContentHandler
Bug: T142491
Change-Id: I69b010b893135e53fac7f16f4b927b8fbcba06d2
2016-08-19 09:26:17 -04:00
Tim Starling
535501ed62 Add a script to build an HHVM RepoAuthoritative bytecode file
This script compiles a list of known MW sources, including core and
extensions, and generates a RepoAuthoritative bytecode file. Some ideas
were taken from Bryan Davis's scap-hhvm-compile shell script.

Also, renamed the old maintenance/hiphop directory to maintenance/hhvm,
respecting the rename of the upstream project.

Change-Id: I55798729d0553d2840e8099e81d604a814e8664c
2016-08-16 13:32:19 +10:00
jenkins-bot
3cfcd55011 Merge "resourceloader: Move queue formatting out of OutputPage" 2016-08-11 14:09:58 +00:00
Gergő Tisza
854a462dc0 Remove $wgDisableAuthManager
Change-Id: I2b2c9693a275fcc026916bd97f303e7a5c8df341
2016-08-09 23:00:27 +00:00
Timo Tijhof
80e5b160e0 resourceloader: Move queue formatting out of OutputPage
HTML formatting of the queue was distributed over several OutputPage methods.
Each method demanding a snippet of HTML by calling makeResourceLoaderLink()
with a limited amount of information. As such, makeResourceLoaderLink() was
unable to provide the client with the proper state information.

Centralising it also allows it to better reduce duplication in HTML output
and maintain a more accurate state.

Problems fixed by centralising:

1. The 'user' module is special (due to per-user 'version' and 'user' params).
   It is manually requested via script-src. To avoid a separate (and wrong)
   request from something that requires it, we set state=loading directly.
   However, because the module is in the bottom, the old HTML formatter could
   only put state=loading in the bottom also. This sometimes caused a wrong
   request to be fired for modules=user if something in the top queue
   triggered a requirement for it.

2. Since a464d1d4 (T87871) we track states of page-style modules, with purpose
   of allowing dependencies on style modules without risking duplicate loading
   on pages where the styles are loaded already. This didn't work, because the
   state information about page-style modules is output near the stylesheet,
   which is after the script tag with mw.loader.load(). That runs first, and
   mw.loader would still make a duplicate request before it learns the state.

Changes:

* Document reasons for style/script tag order in getHeadHtml (per 09537e83).

* Pass $type from getModuleStyles() to getAllowedModules(). This wasn't needed
  before since a duplicate check in makeResourceLoaderLink() verified the
  origin a second time.

* Declare explicit position 'top' on 'user.options' and 'user.tokens' module.
  Previously, OutputPage hardcoded them in the top. The new formatter doesn't.

* Remove getHeadScripts().
* Remove getInlineHeadScripts().
* Remove getExternalHeadScripts().
* Remove buildCssLinks().
* Remove getScriptsForBottomQueue().

* Change where Skin::setupSkinUserCss() is called. This methods lets the skin
  add modules to the queue. Previously it was called from buildCssLinks(),
  via headElement(), via prepareQuickTemplate(), via OutputPage::output().
  It's now in OutputPage::output() directly (slightly earlier). This is needed
  because prepareQuickTemplate() calls bottomScripts() before headElement().
  And bottomScript() would lazy-initialise the queue and lock it before
  setupSkinUserCss() is called from headElement().
  This makes execution order more predictable instead of being dependent on
  the arbitrary order of data extraction in prepareQuickTemplate (which varies
  from one skin to another).

* Compute isUserModulePreview() and isKnownEmpty() for the 'user' module early
  on so. This avoids wrongful loading and fixes problem 1.

Effective changes in output:

* mw.loader.state() is now before mw.loader.load(). This fixes problem 2.
* mw.loader.state() now sets 'user.options' and 'user.tokens' to "loading".
* mw.loader.state() now sets 'user' (as "loading" or "ready"). Fixes problem 1.

* The <script async src> tag for 'startup' changed position (slightly).
  Previously it was after all inline scripts and stylesheets. It's still after
  all inline scripts and after most stylesheets, but before any user styles.
  Since the queue is now formatted outside OutputPage, it can't inject the
  meta-ResourceLoaderDynamicStyles tag and user-stylesheet hack in the middle
  of existing output. This shouldn't have any noticable impact.

Bug: T87871
Change-Id: I605b8cd1e1fc009b4662a0edbc54d09dd65ee1df
2016-08-08 12:23:09 -07:00
jenkins-bot
356b3420d6 Merge "Make content handlers assemble content for search" 2016-08-04 15:54:36 +00:00
jenkins-bot
c5f68ec77a Merge "Introduce MediaHandlerFactory to create MediaHandler objects" 2016-08-04 09:27:23 +00:00
Kunal Mehta
b4596edd00 Introduce MediaHandlerFactory to create MediaHandler objects
This will allow further refactoring of override logic in parser tests.

Ideally the factory class would not use $wgMediaHandlers directly, but
that ends up breaking too many tests for now.

Change-Id: I34a63ee7089ff26f86f3dd6f3cd1a37928bc4005
2016-08-03 16:47:41 -07:00
Timo Tijhof
ae762b1157 resourceloader: Separate 'user.styles' module from 'user'
Fixes addModuleStyles() violation from T92459 (a464d1d41).

Similar to 93ed259cf and Id2342454b for the 'site' module.
Doesn't need to be in separate commits since per-user HTML isn't
cached the same way.

Bug: T92459
Bug: T108590
Change-Id: I195f67d061de1306c97413aada7919e9f1b1d12c
2016-08-02 22:07:07 +00:00
Bartosz Dziewoński
15692fa6d4 Move HTMLFormField subclasses to a separate directory
It's getting more difficult to navigate the files in includes/htmlform/
with every new field and every new helper class that is being added.

Change-Id: I92ce2356baf6151f17b2440970d5abdf86503820
2016-08-01 07:58:56 +00:00
Stanislav Malyshev
add1ebe2ab Make content handlers assemble content for search
Bug: T89733
Change-Id: Ie45de496ecc826211d98eea3a410c7639b4be0a4
2016-07-26 13:08:45 -07:00
Aaron Schulz
94ab94bd2d Renamed MWCallableUpdate class file to match the class
Change-Id: Ia595cee94a696b83b00fe9f2a91b49bce51c08dd
2016-07-26 03:42:57 -07:00
jenkins-bot
dda1b39062 Merge "Make statsd counts for MWCallableUpdate actually useful" 2016-07-22 23:30:27 +00:00
Aaron Schulz
16ad2a779e Make statsd counts for MWCallableUpdate actually useful
Include the method that made the callback in the key.

Change-Id: Ifc7c486ee5c8d57b2516456569bb724cf7dc2b99
2016-07-21 23:52:09 -07:00
Tim Starling
b2f7bb4d76 Preprocessor_Hash: use child arrays instead of linked lists
The singly-linked list data structure of Preprocessor_Hash was causing
stack exhaustion due to the need for a recursion depth proportional to
the number of children of a given PPNode, in serialize() and on
object destruction. So, switch to array-based storage. PPNode_* becomes
a temporary proxy around the underlying storage, which avoids circular
references and keeps the storage very compact. Preprocessor_DOM uses
similar temporary PPNode objects, so the fact that

  $node->getFirstChild() !== $node->getFirstChild()

should not cause any new problems.

* Increment cache version
* Use JSON serialization of the store array instead of serialize(),
  since JSON is more compact, even after gzipping.
* For efficiency, make $accum a plain array, and use it as an array
  where possible, instead of using helper functions.

Performance and memory usage for typical input are slightly improved:
something like 4% faster for the whole parse, and 20% less memory for
the tree.

Bug: T73486
Change-Id: I0d6c162b790d6dc1ddb0352aba6e4753854f4c56
2016-07-22 05:25:11 +00:00
Ori Livneh
f181cdec56 Add option for sharing info about this MediaWiki install via pingback
When $wgPingback is true, MediaWiki will periodically ping
https://www.mediawiki.org/beacon with basic information about the local
MediaWiki installation.  This data includes, for example, the type of system,
PHP version, and chosen database backend.

The pingback is sent via a deferred (post-send) update whenever $wgVersion
changes, using the updatelog table to ensure we don't send duplicate pingbacks.
A database lock ensures only one thread attempts to send the pingback, and a
cache key throttles attempts to no more than once per hour.

$wgPingback is false by default. The web installer has a checkbox for
controlling this option, and it is checked by default. This nudges new installs
to turn on pingbacks, but does not sneak this decision past sysops of existing
installs.

Change-Id: Ie43a6b46a07ebd9ccc1b9c3001f2ea02435d826b
2016-07-21 21:00:28 +00:00
Aaron Schulz
3c21f0e8e5 Add AutoCommitUpdate class and replace some onTransactionIdle callers
* This puts the complex logic here after the commit step for
  all DBs, making the main multi-DB transaction more likely
  to be atomic.
* Made some cleanups to AtomicSectionUpdate and made it cancel
  if the transaction is rolled back as it should.
* Also cleaned up some closures for PHP 5.4.

Change-Id: If2f7bb6b1ba6daf1cfdc934f27c32b0b10431a3d
2016-07-21 05:24:28 +00:00
jenkins-bot
392ea68622 Merge "Only store currently-existing categories in the categories table" 2016-07-19 19:26:56 +00:00