Commit graph

44 commits

Author SHA1 Message Date
Aaron Schulz
488a647831 Move IDatabase/IMaintainableDatabase to Rdbms namespace
Change-Id: If7e8a8ff574661fd827de8bcec11d2c39a687300
2017-03-28 15:32:38 -07:00
Antoine Musso
8357fe9e98 Add accessor to DeferredUpdates
Updates are stored in private variables which is not convenient when
testing deferred updates.  Add a getPendingUpdates() accessor. The stage
can be specified to only retrieve PRE or POST ones.

Change-Id: I1af730ec5e48bc0be555a8db4611a76eb9332444
2017-03-03 13:32:05 +00:00
Aaron Schulz
64df456b39 Move remaining LoadBalancer classes to Rdbms
The old names are left as aliases.

Change-Id: I52a327f2463a2ba7437324047b5b00d28cd1d758
2017-02-23 20:38:31 -08:00
Aaron Schulz
4a177b34ef Move LBFactory to Rdbms namespace
Change-Id: I5ae10783228d0252284807c9562bc8e328d4becb
2017-02-03 17:24:03 -08:00
Aaron Schulz
4a2085523f Make SessionBackend::save() update the user post-send
Bug: T92357
Change-Id: Id4f4991aca1ceeb74b59e980f09863041246a4fc
2016-10-12 11:14:45 -07:00
Aaron Schulz
c5cba0ea15 Call setTransactionTicket() on DeferredUpdates sub-queue items too
This should lower the rate of "does not have outer scope" log
warnings in DBPerformance.

Also add traces to such logs.

Change-Id: I7d21ea745cae07e0fbbe4cd8de82e93f1d10e0a5
2016-09-22 17:03:56 -07:00
Aaron Schulz
d1e15787bd Make DeferredUpdates call setTransactionTicket() on all DataUpdate tasks
This assues that things like LinksUpdate/LinksDeletionUpdate will
have a proper ticket for commitAndWaitForReplication()

Change-Id: I8234510efb706394c39c4027ddf54ace76983aa9
2016-09-12 21:48:51 -07:00
Aaron Schulz
148f7bf0d1 Make DeferredUpdates::execute() protected
Update the only caller, which is a deprecated wrapper method.
Locking down this internal method makes it secure against
misuse with regards to recursion checks.

Change-Id: I3ed52dbe4c0ad52c7b5de92e81bfdc98a1737bcf
2016-09-13 04:11:24 +00:00
jenkins-bot
f9676b1950 Merge "Clarify and update DeferredUpdates docs" 2016-09-07 06:56:40 +00:00
Aaron Schulz
3642142dd0 Clarify and update DeferredUpdates docs
Follows-up dc0cdc8a4d

Change-Id: Ib70585d9d10f6f6493a625697243e0a67796c272
2016-09-07 06:48:49 +00:00
Aaron Schulz
b8b4f12663 Make deferred updates fully own their transaction rounds
This will matter when commitMasterChanges() enforcement is tighter.
For now, any thing can still call that method anytime.

Change-Id: I0b08e9e9118a6c6118c117e3856d0a8e0ca3d457
2016-09-03 06:44:31 -07:00
Aaron Schulz
dc0cdc8a4d Make DeferredUpdates able to run DataUpdates
* Also make ErrorPageError exceptions display themselves
  in PRESEND mode. Before they were always suppressed.
* Make DataUpdate::runUpdates() simply wrap
  DeferredUpdates::execute().
* Remove unused installDBListener() method, which was
  basically moved to Maintenance.
* Enable DBO_TRX for DeferredUpdates::execute() in CLI mode
* Also perform sub-DeferrableUpdate jobs right after their
  parent for better transaction locality.
* Made rollbackMasterChangesAndLog() clear all master
  transactions/rounds, even if there are no changes yet.
  This keeps the state cleaner for continuing.
* For sanity, avoid calling acquirePageLock() in link updates
  unless the transaction ticket is set. These locks are
  already redundant and weaker in range than the locks the
  Job classes that run them get. This helps guard against
  DBTransactionError.
* Renamed $type to $stage to be more clear about the order.

Change-Id: I1e90b56cc80041d70fb9158ac4f027285ad0f2c9
2016-09-02 04:12:50 +00:00
Aaron Schulz
ce504f2d19 Move run-on-add logic in DeferredUpdates to addUpdate()
Change-Id: I1b3ec21e07549f7250fa497f352fe1c131ddc88a
2016-09-02 00:02:53 +00:00
Aaron Schulz
c1ebcb0ee2 Remove $recursionGuard var from tryOpportunisticExecute()
Aside from there being no idle callback setting here, the
old addUpdate() code that runs updates allows nesting updates.
Make this support that for a few transitional commits. It will
be changed later to simply order the sub-updates after their
parent updates, keeping both outer scope for all updates and
locality of related updates.

Change-Id: I0ad4e9713a7893b981b7bb013e9db803eed663b2
2016-09-01 22:56:45 +00:00
Aaron Schulz
44f486ffac Use transaction listener to run DeferredUpdates in CLI mode
This sets triggers on master position waits typically called
after commitMasterChanges() or in commitAndWaitForReplication().

Change-Id: I127a8fe3cfc319abfa84fcd221ee2dae191c6d3b
2016-09-01 03:30:00 +00:00
Aaron Schulz
32a8eb1822 Make various MWCallableUpdate usages rollback-safe
Support for this added to MWCallableUpdate and its helper method
in DeferredUpdates.

Bug: T141988
Change-Id: Ief150ea655036e2708557dd8ae847379edf21933
2016-08-15 00:47:42 -07:00
Aaron Schulz
16ad2a779e Make statsd counts for MWCallableUpdate actually useful
Include the method that made the callback in the key.

Change-Id: Ifc7c486ee5c8d57b2516456569bb724cf7dc2b99
2016-07-21 23:52:09 -07:00
Aaron Schulz
67f74dcdb2 Add statsd logging of DeferredUpdates
Bug: T137326
Change-Id: Icce439210c6412c1824d8d5c411880825bb05643
2016-06-18 11:24:01 -07:00
Aaron Schulz
8d84be66f6 Do not let DataUpdate exceptions block other deferred updates
Change-Id: Ida563b29b5450c44358155fc937cd7eeb87090c2
2016-04-26 14:06:51 -07:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Aaron Schulz
58cffd04b8 Pass __METHOD__ to load balancer commit/rollback methods
Change-Id: I3fd87908c2a493fae49de6a29efe79f8d433c112
2015-12-22 18:30:20 -08:00
Aaron Schulz
d2c33200f7 Remove DeferredUpdates::forceDeferral() testing hack
Change-Id: I7f2ad8190e1a746b6c39190b86fa9c83c5b921fe
2015-12-04 21:18:29 +00:00
Aaron Schulz
99610822d8 Add mergeable update support DeferredUpdates
* DeferrableUpdate classes can implement MergeableUpdate.
  Duplicate updates will be merged via the merge() method.
* Make SquidUpdate support merge() so that duplicate URL
  purges are now caught accross the entire pre-send request
  execution.

Change-Id: Idffdd3e71d89e4a0f28281e65a881113caae497c
2015-12-04 19:08:51 +00:00
Aaron Schulz
34dd248e28 Add pre-send update support to DeferredUpdates
* PRESEND/POSTSEND constants can now be used in addUpdate()
  and addCallableUpdate() to control when the update runs.
  This is useful for updates that may report errors the client
  should see or to just get a head start on queued or pubsub
  based updates like CDN purges. The OutputPage::output() method
  can easily take a few 100ms.
* Removed some argument b/c code from doUpdates().
* Also moved DeferrableUpdate to a separate file.

Change-Id: I9831fe890f9f68f9ad8c4f4bba6921a8f29ba666
2015-12-04 19:08:27 +00:00
Aaron Schulz
9bb575be70 Make DeferredUpdates exception handling more robust
Change-Id: Ica2b8dce0978818a2f51684b97c93a3ba845d81a
2015-11-03 13:05:41 -08:00
Aaron Schulz
a8590172c8 Make DeferredUpdates::doUpdates always commit per task
* All callers are either using commit already or would be fine
  using it (e.g. Maintenance scripts and JobRunner that have
  no real transaction open).

Change-Id: I9f54b27619da6dac2cb63d255995aabc4ee78002
2015-10-22 01:27:08 +00:00
jenkins-bot
0e1c391d97 Merge "Change 1.26 to 1.27, mostly in doc comments" 2015-10-10 17:01:23 +00:00
Aaron Schulz
1846e2dc15 Made LinksUpdate on edit use the job queue
* LinksUpdate is now an EnqueueableDataUpdate
  and can yeild a prioritzed refreshLinks job.
* DeferredUpdates::runUpdates() now takes an enqueue
  flag to try to use jobs. This is set in restInPeace().
  Updates that change many links will be less likely to
  increase lag, as the runners are more strict about that.
* Also made the LinksDeletionUpdate job enqueue happen
  post-send on page deletion for consistency

Bug: T95501
Change-Id: I8863caef9c8f03234699d33e4d47d2310a0c8446
2015-10-08 01:02:03 -07:00
Aaron Schulz
ccab8f10f6 Make DeferredUpdates::doUpdates use DataUpdate::runUpdates
All DataUpdate objects will be managed by runUpdates() while
the other DeferrableUpdate objects will still be run here.
This respects the transaction sematics of DataUpdate a bit more.

Change-Id: Ia0d2dd26a38b0e8911589407b533b58d04d2b084
2015-10-08 07:21:47 +00:00
Kevin Israel
b52502eec5 Change 1.26 to 1.27, mostly in doc comments
Using the following command line, I have found doc comments (and
a wfDeprecated() call) mentioning "1.26" when they should mention
"1.27" instead, which I have fixed manually:

git diff -M REL1_26 | grep --color=always -C 10 -iP \
'^\+.*\D1\.26(\D|$)' | aha > oldver.html

Follows-up these commits:
* 047b60b96d
* 169b7b98b5
* 25a44aa3e4
* 2fb2a3f14b
* 3f1e9fa268 [1]
* 8c84af70b6
* c0cb80beac
* d04a92a551
* d3b85592ea

[1] Release notes moved in I195dd1cf.

Because a release note stating that the UserRights hook is deprecated
was added in 37062a0c0d (before the branch cut), this commit does not
change the Hooks::run call that had been changed in 21206c8fbe.

Change-Id: I5a427f003e7e3b4559fe377bcdfdca466a570708
2015-10-07 01:41:41 -04:00
Aaron Schulz
169b7b98b5 Added replication=async option to FileBackendMultiWrite
* This will defer writes to non-master backends till the
  end up the web request. This is useful for multi-DC setups.

Bug: T112708
Change-Id: I118c07764dd4a4f4f2590d4548238df12860e750
2015-10-01 23:30:41 +00:00
Aaron Schulz
d3f5c943ea Removed DeferredUpdates::addHTMLCacheUpdate()
* That method does not belong in DeferredUpdates
  as this class should agnostic to the work done

Change-Id: I8098ffb82bf9c33ccb48a31aa9dbad514e8724a8
2015-09-21 18:08:14 -07:00
Aaron Schulz
cc4ca3e86d Removed deprecated $wgDeferredUpdateList
Change-Id: I7072ad3b34dd9a0726df6d1c044616c3b4bcd79f
2015-09-21 16:30:16 -07:00
Aaron Schulz
8dbfe9d339 DeferredUpdates doc comment tweak
Change-Id: Ib58bcb065ef7fac58c3c579eb0ca45e51ccdae11
2015-05-30 17:18:12 +00:00
Aaron Schulz
99b05dcc6d Make sure $dbw is an object in DeferredUpdates::addUpdate()
Change-Id: Ib52574bc7f2e13bf06f595e349a7b3ccea425056
2015-05-20 23:06:05 -07:00
Aaron Schulz
04d11e6590 Make JobRunner flush DeferredUpdates after each job
Change-Id: Iff6625ddc04a15751d2bb07dc6558145e7ceb14a
2015-05-18 18:52:34 -07:00
Aaron Schulz
6921770414 Updated some try-catch statements: MWException -> Exception
Change-Id: I76601a86e30f4984e3b1a8c8ec5ef5a0f652433a
2015-01-09 17:20:22 -08:00
Ricordisamoa
2ae155da52 Fix phpcs errors in includes/
Mostly Squiz.WhiteSpace.SuperfluousWhitespace.EmptyLines

Change-Id: I678b2f0902f11cd1dfa1611b9da24e7237df9122
2015-01-08 20:15:07 +01:00
Chad Horohoe
aa21e125a3 Remove obvious function-level profiling
Xhprof generates this data now. Custom profiling of various
sub-function units are kept.

Calls to profiler represented about 3% of page execution
time on Special:BlankPage (1.5% in/out); after this change
it's down to about 0.98% of page execution time.

Change-Id: Id9a1dc9d8f80bbd52e42226b724a1e1213d07af7
2015-01-07 11:14:24 -08:00
bsitu
70b012c595 Add support for adding deferred update within a deferred update
Under some configurations all Echo notifications are inserted
via a DeferredUpdate.  When an extension delays its own notification
trigger via DeferredUpdate as well the Echo notification will not
be run.

Change-Id: Ib40fcd4939ede5b0ff986ab109432630efd1be74
2014-08-20 22:13:11 +00:00
umherirrender
02dc9da399 Cleanup some docs (includes/[a-d])
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling

Change-Id: I7b65fe04db431342cc58b469dc48f41a50c4e891
2014-07-24 19:42:45 +02:00
Siebrand Mazeland
7daa687d68 Update documentation and formatting for includes/deferred/
Change-Id: I56f35be6e549aec13c4fbad5ee6301aa29358031
2013-11-20 21:50:02 +00:00
Siebrand Mazeland
368a95dc65 Update formatting on includes/deferred/
Change-Id: Id3f143eae3781b26dcbc3ba63a566b7a6006c0cc
2013-11-20 21:48:48 +00:00
Tim Starling
64d0dc06a0 Move the DeferredUpdates hierarchy to its own directory
The grouping makes at least as much sense as job/, and certainly makes
more sense than cache/. With directories named after base classes, it is
fairly easy to tell what should go where. The grouping of
DeferredUpdates, DataUpdate and CallableUpdate would surely be
uncontroversial.

The move of SearchUpdate out of search/ demonstrates the conflict between
arrangement by module versus arrangement by type, which is the most
difficult design question here. I think arrangement by type is more
consistent with e.g. the arrangement of the core root, i.e. tests/,
resources/, maintenance/, etc. where a given feature will have its files
split up into a mostly type-based hierarchy.

I also tidied up AutoLoader.php by moving includes/content to the correct
location, sorted alphabetically by subdirectory.

Verified with AutoLoaderTest.

Change-Id: Ib369411d0caca38e72978084aa57348f1b892ed0
2013-11-04 10:32:40 +11:00
Renamed from includes/DeferredUpdates.php (Browse further)