This lets the runJobs.php $wgCommandLineMode hack be removed.
Some fixes based on unit tests:
* Only call applyTransactionRoundFlags() for master connections
for transaction rounds from beginMasterChanges().
* Also cleaned up the commitAndWaitForReplication() reset logic.
* Removed deprecated DataUpdate::doUpdate() calls from jobs
since they cannot nest in a transaction round.
Change-Id: Ia9b91f539dc11a5c05bdac4bcd99d6615c4dc48d
It is now very easy to implement a simple low delay job runner:
while true; do php runJobs.php --wait; sleep 1; done;
Change-Id: I50cf00aed4b15d90384a2cc12bddd64d96d5a1ad
Move the MWLogger PSR-3 logging related classes into the
MediaWiki\Logger namespace. Create shim classes to ease migration of
existing MWLoggerFactory usage to the namespaced classes.
Bug: T93406
Change-Id: I359cc81fbd2dcf8937742311dcc7d3dee08747b0
* Also added an async flag to SpecialRunJobs so that it can be
set to false to get a JSON blob back with a regular 200 status.
Change-Id: I2f5763e017684c3c61f3d3f27ddf7f7834bdfce2
* The new job loop already manages timeouts and this version caused
some problems with the time-sharing approach that used --maxtime.
This killed runners too fast for some jobs and only handled CPU
time anyway, not wall clock time.
Change-Id: Id3462554b1ecba9ecdf7d49673645d54c0555cd6
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.
Change-Id: I727deec35a712de0f0c676cc87dfa661f1ee965b
If there is an unhandled exception during a job, rollback the
transaction to be safe. Otherwise its likely that the
transaction will be commited when next job is run, as $dbw->begin()
implicitly commits any open pending transactions. This can result
in database referential integrity issues.
Change-Id: I5807e64440ff6c6651fbbb4924645d05d843b98e
* Previously, it did not if --type was used
* A --nothrottle option was added to ignore throttling
Change-Id: Ib3ca899f1ce30250a63084096f1b660c96e359fe
Shouldn't be needed and aren't for any core operations. If any
extension relies on these $wgTitles being set in maintenance
environments those extensions are broken and should be fixed.
Change-Id: Ie02a5042ab96e155d783d56d5340dd0da8e3d55c
* This also adds a Job::workItemCount() method
* Removed unused USE_PRIORITY constant
* A few small cleanups in runJobs.php
Change-Id: Ife9370e488fa63dcd1f702ed98f3b7f26057f10c
We had a comment at the top of runJobs.php listing potential options to
use. That unsurprisingly ended up being outdated. Instead folks should
either:
* page down to the constructor to have the exhaustive list of options
* RTFM by invoking the script with --help
Change-Id: Ic87574b26145376bb9926894a7c35f9f0622a3e9
Follows-up I1343872de7, Ia533aedf63 and I2df2f80b81.
Also updated usage in text in documentation and the
installer LocalSettingsGenerator.
Most of them were handled by this regex:
- find: (require|include|require_once|include_once)\s*\(\s*(.+?)\s*\)\s*;$
- replace: $1 $2;
Change-Id: I6b38aad9a5149c9c43ce18bd8edbab14b8ce43fa
Squiz.WhiteSpace.LanguageConstructSpacing:
Language constructs must be followed by a single space;
expected "require_once expression" but found
"require_once(expression)"
It is a keyword (e.g. like `new`, `return` and `print`). As
such the parentheses don't make sense.
Per our code conventions, we use a space after keywords like
these. We appeared to have an unwritten exception for `require`
that doesn't make sense. About 60% of require/include usage
was missing the space and/or had superfluous parentheses.
It is as silly as print("foo") or return("foo"), it works
because keywords have no significance for whitespace between
it and the expression that follows, and since experessions can
be wrapped in parentheses for clarity (e.g. when doing string
concatenation or mathematical operations) the parenthesis
before and after basiclaly just ignored.
Change-Id: I2df2f80b8123714bea7e0771bf94b51ad5bb4b87
* Handle exceptions in run() better.
* Made the profiling more accurate.
* Simplified the code to handle non-boolean results of run().
wfWarn() was removed, and bugs reports were already filed.
Change-Id: Ic702319420efbdf12c17d51494255a64a6212d99
* The default class is JobQueueAggregatorMemc.
This essentially has the logic that nextJobDB.php used.
* Also created a JobQueueAggregatorRedis class.
This is much more efficient and more responsive.
* This can speed up calls to getQueuesWithJobs().
* Removed unused getDefaultQueuesWithJobs() function.
Change-Id: Ifb3c6c881decd643da1b662956ded69db4b39431
* Make the recycling a bit more periodic rather than based on
how often pop() gets called essentially. This works better if
a queue does not have jobs inserted very often.
Change-Id: I64fbc8afbb1cf096717ba4bfc6fe7b7715abdb72
* Jobs will be re-attempted if run() returns false.
This means that claimTTL is useful beyond just the case
where runners get killed in the middle of a job or an
uncaught exception is thrown.
* Moved MAX_ATTEMPTS constant up to base class.
* Clarified docs a bit.
Change-Id: Id7f970e82a63aa563e9a7a023ce32e5d6680433a
