JobQueueGroup::get( $type ) returns a new object for each job type even
if it's the same class (default class). Also it builds different queues
and push them separelty a couple lines lower limiting the ability to
batch push them in the default job queue object.
Given that JobQueue cares about job types being the same as the class
type (which is important for JobQueueDB implementation), adding a config
option called typeAgnostic to ignore type mismatch in JobQueue
implementations that don't care about the type (like JobQueueEventBus)
Bug: T292048
Change-Id: I151f067ca94a985c816446b545921c387b083911
This is micro-optimization of closure code to avoid binding the closure
to $this where it is not needed.
Created by I25a17fb22b6b669e817317a0f45051ae9c608208
Change-Id: I0ffc6200f6c6693d78a3151cb8cea7dce7c21653
If the root job timestamp keys are lost or otherwise unknown, they
will now be deductively recached with the best known values as jobs
are popped and executed. This means the running any of many child
jobs of a root job can restore the root timestamp if it was lost.
This does not need to use the main stash given this fact.
Bug: T227376
Change-Id: Iae0f3af15803af048ff49f3bf281b2bde18c87f2
Also rename throwDBException() to getDBException() and make the
callers throw the result to avoid phpstorm warnings. Remove $row
assignment that is always overridden as well.
Change-Id: I84bc4b11f10152eada6dd6f4788c4f79dcb4a2fb
Simplify the code of jobs that do not care about titles and removes
the direct Title dependency from JobQueue. Remove getTitle() from
IJobSpecification itself. Move all the Job::factory calls into a
single JobQueue::factoryJob() method.
Depends-on: Iee78f4baeca0c0b4d6db073f2fbcc56855114ab0
Change-Id: I9c9d0726d4066bb0aa937665847ad6042ade13ec
Also moved some WikiMap/$wgJobClasses checks to JobQueueGroup::pop
which is the method callers are supposed to use.
Change-Id: I2ab82d8adc4ae1f54697d2935afa2053539cf2db
This was originally a global search and replace. I manually checked all
replacements and reverted them if (due to the lack of type hints) either
null (that would be 0 when counted) or a Countable object can end in the
variable or property in question.
Now this patch only touches places where I'm sure nothing can break.
For the sanity of the honorable reviewers this patch is exclusively touching
negated counts. You should not find a single `!== []` in this patch, that
would be a mistake.
Change-Id: I5eafd4d8fccdb53a668be8e6f25a566f9c3a0a95
Using domains means thats JobQueueDB has the right value to use for calls
like LoadBalancer::getConnection(). The full domain includes the schema in
the case of Postgres. This makes calls to getConnection() less awkward by
not relying on the fallback logic in reallyOpenConnection() for null schemas.
Make getWikiIdFromDomain/isCurrentWikiDomain account for the schema if it
is both defined and is not simply the generic "mediawiki" schema MediaWiki
uses by default. If all wikis use the default schema, the wiki IDs can get
by with DB/prefix alone, which various config and methods may be built around.
Otherwise, the config callbacks must account for schema and the config must
include it in various wiki domain ID lists to properly disambiguate wikis.
Also, clean up SiteConfiguration::siteFromDB() since it is not meant
to handle schemas unless the callback method was taylored to do so.
Finally, add more comments to DefaultSettings.php about already existing
limitations of wiki domain IDs and their components.
Change-Id: I8d94a650e5c99a19ee50551c5be9544318eb05b1
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '
(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)
Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).
Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
I was bored. What? Don't look at me that way.
I mostly targetted mixed tabs and spaces, but others were not spared.
Note that some of the whitespace changes are inside HTML output,
extended regexps or SQL snippets.
Change-Id: Ie206cc946459f6befcfc2d520e35ad3ea3c0f1e0
The race would cause jobs to be be unclaimed but have
no actual job data, causing pop() to return false for
them.
Bug: T133089
Change-Id: Icc60388d4431ab7fa2ca2e46bbb0d212b2b94201
The missing "bool" should be obvious.
I'm also changing type hints from the implementation to the interface.
All public methods from the JobSpecification class are also in the
interface, except for two: toSerializableArray and newFromArray.
These two are not used here.
Change-Id: I36867cdfdf012a4f3233ac4730ab46dac1edc0ab
This class should manage the escaping it uses, rather than use some
random BagOStuff that has nothing to do with the job queue.
Change-Id: Ie716dc4a3429754a99c5f0670555e5e049b61aa1
* Track queues with non-abandoned jobs per partition server.
The s-queuesWithJobs key can easily be queried to see which
queues need to have periodic tasks run (or for debugging).
* This is requirement for the redis jobchron service to be able to
avoid hitting N=(no. types X no. wikis) queues for periodic tasks
when only a tiny fraction of those actually have any jobs. For WMF,
there are over 30K queues, most of them empty, so doing that can help
lower redis-server CPU (or at least make jobchron more responsive).
* This also allows for jobchron to manage the aggregator by taking the
per-server aggregator sets and merging them. This scales much better
as there are only a modest number of these daemons (18 for WMF) but
vastly more web thread pushing jobs. This cuts down on the connections
to the active aggregator server (the one with the hash table).
* Use Lua unpack() more for stylistic consistency.
Change-Id: I1549f0edc78cc4004dd887b475dec4c0ebd306c6
Follows-up 4844595. Which wrongly passed the delta as $type parameter.
This resulted in the creation of "jobqueue.inserts_actual.50" and other
random buckets.
The "all" key also got incremented with 1, but doesn't reflect the actual
value in any meaningful way (besides the number of data points pushed
to statsd).
Change-Id: Ic93e06946ce3fcf0c21d2b7504fb10b3711c6e2b
The `inserts` metric includes dupes. `inserts_actual` will track the rate at
which the job queue is actually growing.
Change-Id: Ie7c544fc8e59180a204d1d149b6a23ac48b8abab
* Since JobQueue metrics are qualified with 'jobqueue.', don't add a 'job-'
prefix to each metric.
* Separate the key from the job type with a dot rather than a dash.
* To avoid having a Graphite node that is both a "directory" and a metric, use
'.all' as a suffix for aggregates.
Change-Id: I2ac604d3c042dbfb0b3a27759800f435ec22041e
