Commit graph

214 commits

Author SHA1 Message Date
Umherirrender
a9007e8baf Add missing & to @param documentation to match functon call
Change-Id: I81e68310abcbc59964b22e0e74842d509f6b1fb9
2017-08-11 18:47:46 +02:00
Matthew Flaschen
7362910408 WANCacheReapUpdate: Handle special pages without exception
Bug: T172560
Change-Id: I95a6da2c5303e5fc4e931734e39974825d4c3c53
2017-08-04 18:39:38 -04:00
Thiemo Mättig
91a920fd85 Remove auto-generated "Constructor" documentation on constructors
Having such comments is worse than not having them. They add zero
information. But you must read the text to understand there is
nothing you don't already know from the class and the method name.

This is similar to I994d11e. Even more trivial, because this here is
about comments that don't say anything but "constructor".

Change-Id: I474dcdb5997bea3aafd11c0760ee072dfaff124c
2017-07-21 12:19:30 +02:00
Aaron Schulz
3daad1264b Push all DeferredUpdates to POSTSEND queue when running that queue
This avoids putting updates in the PRESEND queue at a point where they
may never get run later in the request. The peculiarity lead to a
regression in 24842cfac.

Move "enqueue" logic to runUpdate() to simplify execute(). If job
insertion batching is strongly desired for a class, then it can use
MergeableUpdate.

Removed unused "update" field in $executeContext.

Bug: T168723
Change-Id: I40d16f6cd0adc8583797b99d859b76a836d362a8
2017-07-06 14:10:17 -07:00
David Causse
187ada1cf4 Fix phrase search
Partially revert I61dc536 that broke phrase search support.

Fix phrase search by making explicit that there are two
kind of legalSearchChars() usecases :

- the chars allowed to be part of the search query (including special
  syntax chars such as " and *). Used by SearchDatabase::filter() to
  cleanup the whole query string (the default).

- the chars allowed to be part of a search term (excluding special
  syntax chars) Used by search engine implementaions when parsing with
  a regex.

For future reference:
Originally this distinction was made "explicit" by calling directly
SearchEngine::legalSearchChars() during the parsing stage. This was
broken by Iaabc10c by enabling inheritance.
This patch adds a new optional param to legalSearchChars to make this
more explicit.

Also remove the function I introduced in I61dc536 (I wrongly assumed
that the disctinction made between legalSearchChars usecases was due
to a difference in behavior between indexing and searching).

Added more tests to prevent this from happening in the future.

Bug: T167798
Change-Id: Ibdc796bb2881a2ed8194099d8c9f491980010f0f
2017-07-03 11:44:48 +02:00
David Causse
f230f5dcc7 Fix highlighting for phrase queries
I think the bug was introduced during a cleanup in Iaabc10c.
I don't think that " should be part of the legalSearchChars at query
time, it seems to break the regex.
The strategy here is to distinguish legalSearchChars used query time vs
the ones used at index time by introducing:
SearchEngine::legalSearchCharsForUpdate()

Bug: T167798
Change-Id: I61dc53665e26d3c6c48caed78dd3bbde9a33def7
2017-06-26 09:53:13 +02:00
Aaron Schulz
95fdff36c2 Make DeferredUpdates detect LBFactory transaction rounds
Previously, tryOpportunisticExecute() tried to nest transaction rounds,
which would fail. Added LBFactory::hasTransactionRound() as needed.

Also cleaned up some unqualified class names in callbacks and set the
PRESEND flag for the JobQueueDB AutoCommitUpdate callback. Use the
proper getMasterDB() method while at it. These follow up 24842cfac.

Bug: T154425
Change-Id: Ib1d38f68bd217903d1a7d46fb15b7d7d9620daa6
2017-06-10 15:22:32 +00:00
jenkins-bot
9c8ff2fac0 Merge "Avoid scoped lock errors in Category::refreshCounts() due to nesting" 2017-06-02 23:28:03 +00:00
Aaron Schulz
c769e3f3e4 Add missing return value to getPendingUpdates() docs
Change-Id: I917a2eb97e6e99ccc895245b84717ba40b0c88f6
2017-06-01 21:10:44 -07:00
Seb35
d80fca05e1 Better handling of jobs execution in post-connection shutdown
In the postprocessing, some jobs can be executed but given the deferred
updates were already "closed", any new DeferredUpdate were directly called
(as explained by Krinkle on T165714), and the transactions opened by
classical jobs are badly mixed with transactions (directly) executed by
DeferredUpdates jobs, issuing a DBError, avoiding the job, which stays
in a 'claimed' status even if failed.

Quite similarly, some DeferredUpdates callables use JobQueueGroup::lazyPush
so it is needed to really push the generated jobs.

This change removes the run-immediately-deferred-updates behaviour even
in the post-connection shutdown, and given there is a call to
DeferredUpdates::doUpdates in JobRunner::execute it is not necessary to
add another one and hence execution of Web jobs is more similar to execution
of CLI jobs. In the same spirit to reconcile Web jobs and CLI jobs, the
call to JobQueueGroup::pushLazyJobs is done in JobRunner::execute.

Bug: T165714
Bug: T100085
Change-Id: I721e7167eca5b0b6227234fe516005243ab22388
2017-06-01 13:16:08 +02:00
Aaron Schulz
dbaec78dbb Avoid scoped lock errors in Category::refreshCounts() due to nesting
Bug: T166757
Change-Id: Ie59523a8b3315f063c914cd25d7b53c11e03fbcd
2017-05-31 22:18:12 -07:00
Kunal Mehta
72aaae7b0b SiteStatsUpdate: Avoid deprecated wfMemcKey()
And the deprecated ObjectCache::getMainStashInstance() while we're at
it.

Change-Id: Iebd4e029c3ab6443403087d991bb263ef83b1ed8
2017-05-25 11:20:23 -07:00
Aaron Schulz
869bea9c59 Release prior row locks beforehand in LinksUpdate::updateCategoryCounts
Bug: T163801
Change-Id: I6a026ee9a249da5a8fd9ceb216416340a9188d3f
2017-04-26 22:33:56 -07:00
Fomafix
464f0c72a5 Remove multiple spaces in PHP files
Change-Id: Id9c26ec5ca730a9536f8fdccb8853eb03cbe87a4
2017-04-12 17:27:19 +02:00
Aaron Schulz
488a647831 Move IDatabase/IMaintainableDatabase to Rdbms namespace
Change-Id: If7e8a8ff574661fd827de8bcec11d2c39a687300
2017-03-28 15:32:38 -07:00
WMDE-Fisch
7b5f08e703 Replace deprecated Context::getStats() with MWServices::getStatsdDataFactory()
Change-Id: I1756f69ca2ebd301a5049bf758d1a87c37771fe6
2017-03-17 12:07:03 +01:00
Antoine Musso
8357fe9e98 Add accessor to DeferredUpdates
Updates are stored in private variables which is not convenient when
testing deferred updates.  Add a getPendingUpdates() accessor. The stage
can be specified to only retrieve PRE or POST ones.

Change-Id: I1af730ec5e48bc0be555a8db4611a76eb9332444
2017-03-03 13:32:05 +00:00
jenkins-bot
17eda64357 Merge "includes: Replace implicit Bugzilla bug numbers with Phab ones" 2017-02-28 00:51:57 +00:00
Aaron Schulz
64df456b39 Move remaining LoadBalancer classes to Rdbms
The old names are left as aliases.

Change-Id: I52a327f2463a2ba7437324047b5b00d28cd1d758
2017-02-23 20:38:31 -08:00
James D. Forrester
9635dda73a includes: Replace implicit Bugzilla bug numbers with Phab ones
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.

Change-Id: I6f59febaf8fc96e80f8cfc11f4356283f461142a
2017-02-21 18:13:24 +00:00
Aaron Schulz
4a177b34ef Move LBFactory to Rdbms namespace
Change-Id: I5ae10783228d0252284807c9562bc8e328d4becb
2017-02-03 17:24:03 -08:00
Brad Jorsch
fb3ae6fbe3 Replace use of &$this
Use of &$this doesn't work in PHP 7.1. For callbacks to methods like
array_map() it's completely unnecessary, while for hooks we still need
to pass a reference and so we need to copy $this into a local variable.

Bug: T153505
Change-Id: I8bbb26e248cd6f213fd0e7460d6d6935a3f9e468
2017-01-31 23:01:54 -05:00
Aaron Schulz
2e5eb693de objectcache: add WANObjectCacheReaper for assuring purges
* This fixes keys based on some sort of change log.
  Updates are wrapped in a mutex and keep track of the
  last known good position.
* Make WANObjectReapUpdate class that cleans up title
  related keys using the recentchanges table. This triggers
  as a deferred updates on RC view.

Change-Id: I7f14b9ca2533032147e62b1a3cc004a23da86579
2017-01-27 02:21:06 +00:00
Thiemo Mättig
00c3f09566 Remove empty lines from PHP and JavaScript comment blocks
This is a pure documentation change. It mostly removes empty lines from
comments (and entirely empty comments), as well as adds a few missing
documentation blocks and fixes a minor mistake. I hope it's ok to have
this in one patch. I can split it, please tell me.

Change-Id: I9668338602ac77b903ab6b02ff56bd52743c37c4
2016-12-09 09:01:06 +00:00
umherirrender
34fe90ac52 Remove empty lines at end of functions
It looks like there is something missing after the last statement
Also remove some other empty lines at begin of functions, ifs or loops
while at these files

Change-Id: Ib00b5cfd31ca4dcd0c32ce33754d3c80bae70641
2016-11-05 11:55:10 +01:00
Aaron Schulz
e3b7bf4d0a Make updateCategoryCounts() have better lag checks
* Add the lag checks to LinksUpdate. Previously, only
  LinksDeletionUpdate had any such checks.
* Remove the transaction hook usage, since the only two callers are
  already lag/contention aware. Deferring them just makes the wait
  checks pointless and they might end up happening all at once.
* Also set the visibility on some neigboring methods.
* Clean up LinksUpdate $existing variables in passing. Instead of
  overriding the same variable, use a differently named variable
  to avoid mistakes.

Bug: T95501
Change-Id: I43e3af17399417cbf0ab4e5e7d1f2bd518fa7e90
2016-10-20 15:33:18 -07:00
Kunal Mehta
61adc1e146 Use namespaced ScopedCallback
The un-namespaced \ScopedCallback is deprecated.

Change-Id: Ie014d5a775ead66335a24acac9d339915884d1a4
2016-10-17 15:46:05 -07:00
Aaron Schulz
4a2085523f Make SessionBackend::save() update the user post-send
Bug: T92357
Change-Id: Id4f4991aca1ceeb74b59e980f09863041246a4fc
2016-10-12 11:14:45 -07:00
jenkins-bot
d3f9524df3 Merge "Add transaction ticket to LinksUpdateComplete hook" 2016-09-23 23:14:32 +00:00
Aaron Schulz
d31ecefc1c Add transaction ticket to LinksUpdateComplete hook
This lets callers use commitAndWaitForReplication() more easily.

Change-Id: I743bd1f989b8fb3b7ba3e5cc8ce1bb44c00f99af
2016-09-23 02:37:03 +00:00
Aaron Schulz
c5cba0ea15 Call setTransactionTicket() on DeferredUpdates sub-queue items too
This should lower the rate of "does not have outer scope" log
warnings in DBPerformance.

Also add traces to such logs.

Change-Id: I7d21ea745cae07e0fbbe4cd8de82e93f1d10e0a5
2016-09-22 17:03:56 -07:00
Aaron Schulz
4c08bf8d96 Avoid using cascadingDeletes()/cleanupTriggers()
Simplify DB callers by just having one code path.
All but some very old code paths bothered with these.

Change-Id: Iaf7a2f83146a0ed15995f9cfc74edcf16ae5a448
2016-09-18 02:05:07 +00:00
Aaron Schulz
a2790b1b80 Remove wfGetCaller() dependency from DatabaseBase
Change-Id: I3e240b2eb5c1f6a21f1bc974c3d28f5755c7451a
2016-09-15 14:40:00 -07:00
Aaron Schulz
b73ffae752 Make SiteStatsUpdate implement MergeableUpdate
Change-Id: I53c9bf133f0707f2fa203ddd1a7fcb65be8b1329
2016-09-14 23:18:20 +00:00
Aaron Schulz
d1e15787bd Make DeferredUpdates call setTransactionTicket() on all DataUpdate tasks
This assues that things like LinksUpdate/LinksDeletionUpdate will
have a proper ticket for commitAndWaitForReplication()

Change-Id: I8234510efb706394c39c4027ddf54ace76983aa9
2016-09-12 21:48:51 -07:00
Aaron Schulz
148f7bf0d1 Make DeferredUpdates::execute() protected
Update the only caller, which is a deprecated wrapper method.
Locking down this internal method makes it secure against
misuse with regards to recursion checks.

Change-Id: I3ed52dbe4c0ad52c7b5de92e81bfdc98a1737bcf
2016-09-13 04:11:24 +00:00
jenkins-bot
f9676b1950 Merge "Clarify and update DeferredUpdates docs" 2016-09-07 06:56:40 +00:00
Aaron Schulz
3642142dd0 Clarify and update DeferredUpdates docs
Follows-up dc0cdc8a4d

Change-Id: Ib70585d9d10f6f6493a625697243e0a67796c272
2016-09-07 06:48:49 +00:00
jenkins-bot
4e27dcee4c Merge "Make deferred updates fully own their transaction rounds" 2016-09-06 06:19:32 +00:00
Aaron Schulz
950cf6016c Rename DB_SLAVE constant to DB_REPLICA
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.

The old constant is an alias now.

Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
2016-09-05 22:55:53 -07:00
Aaron Schulz
b8b4f12663 Make deferred updates fully own their transaction rounds
This will matter when commitMasterChanges() enforcement is tighter.
For now, any thing can still call that method anytime.

Change-Id: I0b08e9e9118a6c6118c117e3856d0a8e0ca3d457
2016-09-03 06:44:31 -07:00
Aaron Schulz
dc0cdc8a4d Make DeferredUpdates able to run DataUpdates
* Also make ErrorPageError exceptions display themselves
  in PRESEND mode. Before they were always suppressed.
* Make DataUpdate::runUpdates() simply wrap
  DeferredUpdates::execute().
* Remove unused installDBListener() method, which was
  basically moved to Maintenance.
* Enable DBO_TRX for DeferredUpdates::execute() in CLI mode
* Also perform sub-DeferrableUpdate jobs right after their
  parent for better transaction locality.
* Made rollbackMasterChangesAndLog() clear all master
  transactions/rounds, even if there are no changes yet.
  This keeps the state cleaner for continuing.
* For sanity, avoid calling acquirePageLock() in link updates
  unless the transaction ticket is set. These locks are
  already redundant and weaker in range than the locks the
  Job classes that run them get. This helps guard against
  DBTransactionError.
* Renamed $type to $stage to be more clear about the order.

Change-Id: I1e90b56cc80041d70fb9158ac4f027285ad0f2c9
2016-09-02 04:12:50 +00:00
Aaron Schulz
ce504f2d19 Move run-on-add logic in DeferredUpdates to addUpdate()
Change-Id: I1b3ec21e07549f7250fa497f352fe1c131ddc88a
2016-09-02 00:02:53 +00:00
Aaron Schulz
c1ebcb0ee2 Remove $recursionGuard var from tryOpportunisticExecute()
Aside from there being no idle callback setting here, the
old addUpdate() code that runs updates allows nesting updates.
Make this support that for a few transitional commits. It will
be changed later to simply order the sub-updates after their
parent updates, keeping both outer scope for all updates and
locality of related updates.

Change-Id: I0ad4e9713a7893b981b7bb013e9db803eed663b2
2016-09-01 22:56:45 +00:00
Aaron Schulz
ef1c17b77d Remove unused custom transaction logic from DataUpdate
Change-Id: Ife65e4e90a35395e87f4f487f1cb871b67d92aa1
2016-09-01 15:03:16 -07:00
jenkins-bot
e117bb7cd9 Merge "Use transaction listener to run DeferredUpdates in CLI mode" 2016-09-01 06:45:00 +00:00
Aaron Schulz
44f486ffac Use transaction listener to run DeferredUpdates in CLI mode
This sets triggers on master position waits typically called
after commitMasterChanges() or in commitAndWaitForReplication().

Change-Id: I127a8fe3cfc319abfa84fcd221ee2dae191c6d3b
2016-09-01 03:30:00 +00:00
Aaron Schulz
27992c48f8 Deprecate SqlDataUpdate
Core subclasses have been updated.

Change-Id: Ic1beebeb7aec2b40d89c75637968e57e71a6c1a2
2016-09-01 00:27:24 +00:00
Aaron Schulz
a438b12b8b Links update class cleanups
* Avoid using deprecated functions.
* Switch to DataUpdate as the direct parent class as
  no benefit was provided from SqlDataUpdate (which
  should be deprecated soon).

Change-Id: I0f1c77128f3df658e6a0eaf4471ca48ac536c643
2016-08-31 17:11:51 -07:00
cenarium
d88d902735 LinksUpdate: Add functions returning changed page properties
This adds getAddedProperties and getRemovedProperties functions
to LinksUpdate. They are available only after the update, so for
extensions in the LinksUpdateComplete hook. This is useful for
example if an extension caches a page property; if the property
gets changed it may want to purge the cache.

This is similar to the getAddedLinks and getRemovedLinks
functions.

Change-Id: I0c73b3d181f32502da75687857ae9aeff731f559
2016-08-29 16:45:05 -07:00