This is a stop-gap solution for the problem described in T190178.
A better solution would probably include changing the behavior
of LegacyLogger.
Bug: T190178
Change-Id: I433be04b8ee725becd174e567270aa674d2661df
This allows scheduling of updates that need to start their own
transaction round. Specifically for cases where the ability to
commit early is not enough (which is already possible via LBFactory
getEmptyTransactionTicket and commitAndWaitForReplication).
Change-Id: I0910587b61c8ddf825f91e92c2f93582cc7ebd80
This avoids putting updates in the PRESEND queue at a point where they
may never get run later in the request. The peculiarity lead to a
regression in 24842cfac.
Move "enqueue" logic to runUpdate() to simplify execute(). If job
insertion batching is strongly desired for a class, then it can use
MergeableUpdate.
Removed unused "update" field in $executeContext.
Bug: T168723
Change-Id: I40d16f6cd0adc8583797b99d859b76a836d362a8
Previously, tryOpportunisticExecute() tried to nest transaction rounds,
which would fail. Added LBFactory::hasTransactionRound() as needed.
Also cleaned up some unqualified class names in callbacks and set the
PRESEND flag for the JobQueueDB AutoCommitUpdate callback. Use the
proper getMasterDB() method while at it. These follow up 24842cfac.
Bug: T154425
Change-Id: Ib1d38f68bd217903d1a7d46fb15b7d7d9620daa6
In the postprocessing, some jobs can be executed but given the deferred
updates were already "closed", any new DeferredUpdate were directly called
(as explained by Krinkle on T165714), and the transactions opened by
classical jobs are badly mixed with transactions (directly) executed by
DeferredUpdates jobs, issuing a DBError, avoiding the job, which stays
in a 'claimed' status even if failed.
Quite similarly, some DeferredUpdates callables use JobQueueGroup::lazyPush
so it is needed to really push the generated jobs.
This change removes the run-immediately-deferred-updates behaviour even
in the post-connection shutdown, and given there is a call to
DeferredUpdates::doUpdates in JobRunner::execute it is not necessary to
add another one and hence execution of Web jobs is more similar to execution
of CLI jobs. In the same spirit to reconcile Web jobs and CLI jobs, the
call to JobQueueGroup::pushLazyJobs is done in JobRunner::execute.
Bug: T165714
Bug: T100085
Change-Id: I721e7167eca5b0b6227234fe516005243ab22388
Updates are stored in private variables which is not convenient when
testing deferred updates. Add a getPendingUpdates() accessor. The stage
can be specified to only retrieve PRE or POST ones.
Change-Id: I1af730ec5e48bc0be555a8db4611a76eb9332444
This should lower the rate of "does not have outer scope" log
warnings in DBPerformance.
Also add traces to such logs.
Change-Id: I7d21ea745cae07e0fbbe4cd8de82e93f1d10e0a5
This assues that things like LinksUpdate/LinksDeletionUpdate will
have a proper ticket for commitAndWaitForReplication()
Change-Id: I8234510efb706394c39c4027ddf54ace76983aa9
Update the only caller, which is a deprecated wrapper method.
Locking down this internal method makes it secure against
misuse with regards to recursion checks.
Change-Id: I3ed52dbe4c0ad52c7b5de92e81bfdc98a1737bcf
This will matter when commitMasterChanges() enforcement is tighter.
For now, any thing can still call that method anytime.
Change-Id: I0b08e9e9118a6c6118c117e3856d0a8e0ca3d457
* Also make ErrorPageError exceptions display themselves
in PRESEND mode. Before they were always suppressed.
* Make DataUpdate::runUpdates() simply wrap
DeferredUpdates::execute().
* Remove unused installDBListener() method, which was
basically moved to Maintenance.
* Enable DBO_TRX for DeferredUpdates::execute() in CLI mode
* Also perform sub-DeferrableUpdate jobs right after their
parent for better transaction locality.
* Made rollbackMasterChangesAndLog() clear all master
transactions/rounds, even if there are no changes yet.
This keeps the state cleaner for continuing.
* For sanity, avoid calling acquirePageLock() in link updates
unless the transaction ticket is set. These locks are
already redundant and weaker in range than the locks the
Job classes that run them get. This helps guard against
DBTransactionError.
* Renamed $type to $stage to be more clear about the order.
Change-Id: I1e90b56cc80041d70fb9158ac4f027285ad0f2c9
Aside from there being no idle callback setting here, the
old addUpdate() code that runs updates allows nesting updates.
Make this support that for a few transitional commits. It will
be changed later to simply order the sub-updates after their
parent updates, keeping both outer scope for all updates and
locality of related updates.
Change-Id: I0ad4e9713a7893b981b7bb013e9db803eed663b2
This sets triggers on master position waits typically called
after commitMasterChanges() or in commitAndWaitForReplication().
Change-Id: I127a8fe3cfc319abfa84fcd221ee2dae191c6d3b
* DeferrableUpdate classes can implement MergeableUpdate.
Duplicate updates will be merged via the merge() method.
* Make SquidUpdate support merge() so that duplicate URL
purges are now caught accross the entire pre-send request
execution.
Change-Id: Idffdd3e71d89e4a0f28281e65a881113caae497c
* PRESEND/POSTSEND constants can now be used in addUpdate()
and addCallableUpdate() to control when the update runs.
This is useful for updates that may report errors the client
should see or to just get a head start on queued or pubsub
based updates like CDN purges. The OutputPage::output() method
can easily take a few 100ms.
* Removed some argument b/c code from doUpdates().
* Also moved DeferrableUpdate to a separate file.
Change-Id: I9831fe890f9f68f9ad8c4f4bba6921a8f29ba666
* All callers are either using commit already or would be fine
using it (e.g. Maintenance scripts and JobRunner that have
no real transaction open).
Change-Id: I9f54b27619da6dac2cb63d255995aabc4ee78002
* LinksUpdate is now an EnqueueableDataUpdate
and can yeild a prioritzed refreshLinks job.
* DeferredUpdates::runUpdates() now takes an enqueue
flag to try to use jobs. This is set in restInPeace().
Updates that change many links will be less likely to
increase lag, as the runners are more strict about that.
* Also made the LinksDeletionUpdate job enqueue happen
post-send on page deletion for consistency
Bug: T95501
Change-Id: I8863caef9c8f03234699d33e4d47d2310a0c8446
All DataUpdate objects will be managed by runUpdates() while
the other DeferrableUpdate objects will still be run here.
This respects the transaction sematics of DataUpdate a bit more.
Change-Id: Ia0d2dd26a38b0e8911589407b533b58d04d2b084
Using the following command line, I have found doc comments (and
a wfDeprecated() call) mentioning "1.26" when they should mention
"1.27" instead, which I have fixed manually:
git diff -M REL1_26 | grep --color=always -C 10 -iP \
'^\+.*\D1\.26(\D|$)' | aha > oldver.html
Follows-up these commits:
* 047b60b96d
* 169b7b98b5
* 25a44aa3e4
* 2fb2a3f14b
* 3f1e9fa268 [1]
* 8c84af70b6
* c0cb80beac
* d04a92a551
* d3b85592ea
[1] Release notes moved in I195dd1cf.
Because a release note stating that the UserRights hook is deprecated
was added in 37062a0c0d (before the branch cut), this commit does not
change the Hooks::run call that had been changed in 21206c8fbe.
Change-Id: I5a427f003e7e3b4559fe377bcdfdca466a570708
* This will defer writes to non-master backends till the
end up the web request. This is useful for multi-DC setups.
Bug: T112708
Change-Id: I118c07764dd4a4f4f2590d4548238df12860e750
