Using domains means thats JobQueueDB has the right value to use for calls
like LoadBalancer::getConnection(). The full domain includes the schema in
the case of Postgres. This makes calls to getConnection() less awkward by
not relying on the fallback logic in reallyOpenConnection() for null schemas.
Make getWikiIdFromDomain/isCurrentWikiDomain account for the schema if it
is both defined and is not simply the generic "mediawiki" schema MediaWiki
uses by default. If all wikis use the default schema, the wiki IDs can get
by with DB/prefix alone, which various config and methods may be built around.
Otherwise, the config callbacks must account for schema and the config must
include it in various wiki domain ID lists to properly disambiguate wikis.
Also, clean up SiteConfiguration::siteFromDB() since it is not meant
to handle schemas unless the callback method was taylored to do so.
Finally, add more comments to DefaultSettings.php about already existing
limitations of wiki domain IDs and their components.
Change-Id: I8d94a650e5c99a19ee50551c5be9544318eb05b1
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '
(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)
Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).
Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
I was bored. What? Don't look at me that way.
I mostly targetted mixed tabs and spaces, but others were not spared.
Note that some of the whitespace changes are inside HTML output,
extended regexps or SQL snippets.
Change-Id: Ie206cc946459f6befcfc2d520e35ad3ea3c0f1e0
The race would cause jobs to be be unclaimed but have
no actual job data, causing pop() to return false for
them.
Bug: T133089
Change-Id: Icc60388d4431ab7fa2ca2e46bbb0d212b2b94201
The missing "bool" should be obvious.
I'm also changing type hints from the implementation to the interface.
All public methods from the JobSpecification class are also in the
interface, except for two: toSerializableArray and newFromArray.
These two are not used here.
Change-Id: I36867cdfdf012a4f3233ac4730ab46dac1edc0ab
This class should manage the escaping it uses, rather than use some
random BagOStuff that has nothing to do with the job queue.
Change-Id: Ie716dc4a3429754a99c5f0670555e5e049b61aa1
* Track queues with non-abandoned jobs per partition server.
The s-queuesWithJobs key can easily be queried to see which
queues need to have periodic tasks run (or for debugging).
* This is requirement for the redis jobchron service to be able to
avoid hitting N=(no. types X no. wikis) queues for periodic tasks
when only a tiny fraction of those actually have any jobs. For WMF,
there are over 30K queues, most of them empty, so doing that can help
lower redis-server CPU (or at least make jobchron more responsive).
* This also allows for jobchron to manage the aggregator by taking the
per-server aggregator sets and merging them. This scales much better
as there are only a modest number of these daemons (18 for WMF) but
vastly more web thread pushing jobs. This cuts down on the connections
to the active aggregator server (the one with the hash table).
* Use Lua unpack() more for stylistic consistency.
Change-Id: I1549f0edc78cc4004dd887b475dec4c0ebd306c6
Follows-up 4844595. Which wrongly passed the delta as $type parameter.
This resulted in the creation of "jobqueue.inserts_actual.50" and other
random buckets.
The "all" key also got incremented with 1, but doesn't reflect the actual
value in any meaningful way (besides the number of data points pushed
to statsd).
Change-Id: Ic93e06946ce3fcf0c21d2b7504fb10b3711c6e2b
The `inserts` metric includes dupes. `inserts_actual` will track the rate at
which the job queue is actually growing.
Change-Id: Ie7c544fc8e59180a204d1d149b6a23ac48b8abab
* Since JobQueue metrics are qualified with 'jobqueue.', don't add a 'job-'
prefix to each metric.
* Separate the key from the job type with a dot rather than a dash.
* To avoid having a Graphite node that is both a "directory" and a metric, use
'.all' as a suffix for aggregates.
Change-Id: I2ac604d3c042dbfb0b3a27759800f435ec22041e
* This unifies the retry and no-retry code paths so that its easier to use
the separate job runner service.
* All jobs will use the ack procedure. If no retries are enabled they will
just get deleted. As a side-effect, abandoned jobs can be seen for a few
days in the no-retry case, which used to not happen. This could actually
be useful to know about anyway.
Change-Id: I185092e3696fb336b9edcf19280dcd9a561161d9
I have not found documentation about the @params command, so this seems
not valid.
I have moved the text to the corresponding @param
Change-Id: I443866d602b60570278289e6e24a77d37314cede
The hhvm redis client returns false instead of null. This caused
JobQueueRedis to get stuck in an infinite loop. This works around the
difference by catching null as a signal for no more jobs.
It can be reverted when https://github.com/facebook/hhvm/pull/3127/ is
in all versions of hhvm we expect to run MediaWiki.
Bug: 67622
Change-Id: I9bbad42f36a80635097b8e0140b48b6492b2f0f5
