Commit graph

55 commits

Author SHA1 Message Date
Jakub Vrana
547312ae39 Do not pass unused parameter
Found by PHPStan.

Change-Id: I48a613d94624ef4f8c533b98b8c128b4cc9c47f5
2018-12-01 18:08:04 +01:00
Aaron Schulz
51945dbca3 Use DB domain in JobQueueGroup and make WikiMap domain ID methods stricter
Using domains means thats JobQueueDB has the right value to use for calls
like LoadBalancer::getConnection(). The full domain includes the schema in
the case of Postgres. This makes calls to getConnection() less awkward by
not relying on the fallback logic in reallyOpenConnection() for null schemas.

Make getWikiIdFromDomain/isCurrentWikiDomain account for the schema if it
is both defined and is not simply the generic "mediawiki" schema MediaWiki
uses by default. If all wikis use the default schema, the wiki IDs can get
by with DB/prefix alone, which various config and methods may be built around.
Otherwise, the config callbacks must account for schema and the config must
include it in various wiki domain ID lists to properly disambiguate wikis.

Also, clean up SiteConfiguration::siteFromDB() since it is not meant
to handle schemas unless the callback method was taylored to do so.

Finally, add more comments to DefaultSettings.php about already existing
limitations of wiki domain IDs and their components.

Change-Id: I8d94a650e5c99a19ee50551c5be9544318eb05b1
2018-11-07 04:46:56 +00:00
Aaron Schulz
c947f3288d jobqueue: clean up JobQueueDB::getCacheKey() to use makeGlobalKey()
Change-Id: I341720fda0d242c7822f78048cb49f63538d08bc
2018-10-13 00:14:38 -07:00
Umherirrender
a4caa4d0c6 build: Updating mediawiki/mediawiki-codesniffer to 22.0.0
Added spaces around .
Removed empty return statement which are not required
Removed return after phpunit markTestIncomplete,
which is throwing to exit the test, no need for a return

Change-Id: I2c80b965ee52ba09949e70ea9e7adfc58a1d89ce
2018-09-16 15:51:11 +00:00
Aaron Schulz
5938a6efdc rdbms: add more comments and sanity checks for CONN_TRX_AUTOCOMMIT
Change-Id: I69992cf2e2ae3ef62125b0bc733a0cb7274f814e
2018-08-09 01:39:07 +00:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
James D. Forrester
f8ac266287 Job: Remove insert() and batchInsert(), deprecated since 1.21
Bug: T179624
Change-Id: I0d78a926c8c513fd652a424e8a51ded20e524a4a
2018-05-21 20:10:09 +01:00
Aaron Schulz
b6cd5421b9 rdbms: rename onTransactionIdle() to onTransactionCommitOrIdle()
This is clearer and is consistent with onTransactionPreCommitOrIdle()

Change-Id: I3a34a0e9adea69ec55ed6ddfef47703e31e7c3b5
2018-05-09 21:07:06 +00:00
Aaron Schulz
c8085ad43f rdbms: make IDatabase::onTransaction* methods pass the DB handle for convenience
Change-Id: Ia45a26830d62326b103593268fbf34c907783c90
2018-04-24 16:45:11 -07:00
Aaron Schulz
f8c2486d15 rdbms: rename CONN_TRX_AUTO constant to CONN_TRX_AUTOCOMMIT
The "AUTO" means AUTOCOMMIT, not "automatic transactions"/DBO_TRX,
which is basically the opposite concept. The new name does not
suffer from that ambiguity.

Keep the old constant as an alias for backwards compatibility.

Also remove LoadBalancer comment about non-existing field

Change-Id: I63beeb061fc9be73f320308e4d6393b58628b8c8
2018-04-12 13:01:52 -07:00
Brad Jorsch
01a10dba5a Remove reference to deprecated IDatabase->nextSequenceValue()
The method was deprecated and made unnecessary in Ib308190c.

Change-Id: I1729ac0b3a88270a4c2f064187a2472112aaeb1e
2017-09-01 12:28:39 -04:00
Aaron Schulz
99c80a8fc7 rdbms: Support secondary autocommit connections in LoadBalancer
This is useful for things like SqlBagOStuff and JobQueueDB
being able to write in auto-commit mode while the main
transaction round still goes on.

SqlBagOStuff already had this sort of logic.

JobQueueDB now also takes this approach so it does not have to
defer insertion except for sqlite. This makes callers more likely
to know when insertion failed or not.

Make sure LoadBalancer sets "master"/"replica" LB info itself;
this fixes a bug found in the new tests.

Bug: T140338
Bug: T42451
Change-Id: I4199a9598d7afbf976a6efa8ed84b85b56da02bd
2017-08-18 01:28:34 +00:00
Aaron Schulz
c9544ab6b2 Remove excess getExternalLB() argument
Change-Id: I097b1ba46806649f014ca624d15d9064ee0f9ba7
2017-07-17 16:32:53 -07:00
Aaron Schulz
82e2c924e4 Remove "@author Aaron Schulz" annotations
Bug: T139301
Change-Id: Ib5248e8e27d60611c7373bce4b29dd5e85aa3489
2017-06-27 15:24:14 -07:00
Aaron Schulz
95fdff36c2 Make DeferredUpdates detect LBFactory transaction rounds
Previously, tryOpportunisticExecute() tried to nest transaction rounds,
which would fail. Added LBFactory::hasTransactionRound() as needed.

Also cleaned up some unqualified class names in callbacks and set the
PRESEND flag for the JobQueueDB AutoCommitUpdate callback. Use the
proper getMasterDB() method while at it. These follow up 24842cfac.

Bug: T154425
Change-Id: Ib1d38f68bd217903d1a7d46fb15b7d7d9620daa6
2017-06-10 15:22:32 +00:00
Seb35
24842cfac0 Use AutoCommitUpdate instead of Database->onTransactionIdle
This is needed for deferred updates LinksDeletionUpdate and LinksUpdate, else
callbacks registered with onTransactionIdle prevent other transactions from
being executed, at least in this case.

Bug: T154425
Bug: T154438
Bug: T157679
Change-Id: Iecd396d584a62ac936cd963915339159467b44cd
2017-06-06 14:23:37 +02:00
Aaron Schulz
dd359741cc Move DB errors to Rdbms namespace
Change-Id: I463bd86123501abc68fdb78b4cda6110f7af2549
2017-04-15 10:47:41 -07:00
Aaron Schulz
3c2a172c3f Move DBConnRef/MaintainableDBConnRef to Rdbms namespace
Change-Id: I7ca6b40f973b2aae83747de92e280ded222c87e3
2017-03-29 11:04:47 -07:00
Aaron Schulz
488a647831 Move IDatabase/IMaintainableDatabase to Rdbms namespace
Change-Id: If7e8a8ff574661fd827de8bcec11d2c39a687300
2017-03-28 15:32:38 -07:00
James D. Forrester
9635dda73a includes: Replace implicit Bugzilla bug numbers with Phab ones
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.

Change-Id: I6f59febaf8fc96e80f8cfc11f4356283f461142a
2017-02-21 18:13:24 +00:00
jenkins-bot
69ae945e8d Merge "Update weblinks in comments from HTTP to HTTPS" 2016-11-08 21:32:00 +00:00
Fomafix
202f695f67 Update weblinks in comments from HTTP to HTTPS
Use HTTPS instead of HTTP where the HTTP link is a redirect to the HTTPS link.

Also update some defect links.

Change-Id: Ic3a5eac910d098ed5c2a21e9f47c9b6ee06b2643
2016-11-07 15:24:46 +01:00
Aaron Schulz
1eb86e0e4d Rename JobQueueDB::getSlaveDB() to getReplicaDB()
Change-Id: I7a88c4876e76836c92cc0b45b4e72124a087121c
2016-10-29 00:04:39 -07:00
Kunal Mehta
61adc1e146 Use namespaced ScopedCallback
The un-namespaced \ScopedCallback is deprecated.

Change-Id: Ie014d5a775ead66335a24acac9d339915884d1a4
2016-10-17 15:46:05 -07:00
Aaron Schulz
a2790b1b80 Remove wfGetCaller() dependency from DatabaseBase
Change-Id: I3e240b2eb5c1f6a21f1bc974c3d28f5755c7451a
2016-09-15 14:40:00 -07:00
Aaron Schulz
fca4beb302 Avoid using deprecated methods in JobQueueDB
Change-Id: Ib35b8792e3e4902b52c3e708d8c6e756f35986bd
2016-09-12 19:56:37 -07:00
Aaron Schulz
950cf6016c Rename DB_SLAVE constant to DB_REPLICA
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.

The old constant is an alias now.

Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
2016-09-05 22:55:53 -07:00
Aaron Schulz
16266edff3 Change "slave" => "replica DB" in /includes
Change-Id: Icb716219c9335ff8fa447b1733d04b71d9712bf9
2016-09-05 21:01:01 +00:00
Aaron Schulz
f3cfdf0baa Remove commit() calls from JobQueueDB
These are not safe for the common case where the local DB
handle is used for the queue (and other table writes).

Change-Id: Ic24a05c18bf31e49bf7e9a3c058deb5d35271511
2016-08-23 17:24:58 +00:00
Aaron Schulz
1f313389e2 Remove direct rollback() calls from some places
Rely on the mass-rollback logic in MWExceptionHandler instead.
This results in a better chance of atomicity.

Change-Id: I2eb5661d4acc105a1323d69c5463268c234bd745
2016-08-23 01:19:21 +00:00
Aaron Schulz
b5d8f3aed1 Fix some JobQueue IDEA errors
Change-Id: I624f65ee1ca97c1acae9b54ca36d910eb4e42a70
2016-03-08 03:23:13 -08:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Ricordisamoa
7a97368c35 Stop doing $that = $this in includes/jobqueue
Closures support $this as of PHP 5.4

Change-Id: Icf4eb9ffeab410c01c68a031ec6704cd83c2681e
2016-02-10 17:51:47 +00:00
Reedy
00c426e3c2 Replace wfBaseConvert with Wikimedia\base_convert
Change-Id: Iadab3d018c3559daf79be90edb23d131729bdb68
2015-11-24 22:51:42 +00:00
Aaron Schulz
8ad6c52d4f Convert JobQueueDB to using the WAN cache
Change-Id: Ie5820d1439014572ca171c9303d51a8d3938ad00
2015-11-01 01:05:34 -08:00
Aaron Schulz
92bbfd5ea6 Migrate more callers away from $wgMemc
Callers should use more expliciy lazy-loaded
cache factory methods.

Change-Id: Ifa0bf389720a09a931ee6466b993f787d83a09a7
2015-10-26 19:07:12 +00:00
Aaron Schulz
fc662dcb71 Convert doBatchPushInternal to use startAtomic/endAtomic
This is less likely to cause nesting problems if the main
DB is used for the job table (e.g. stock installs)

Change-Id: I4c72688bd5d07a6b0626958869e6b386babbc5b3
2015-10-07 11:03:35 -07:00
Ori Livneh
427bdb6dbd jobqueue: use more sensible metric key names
* Since JobQueue metrics are qualified with 'jobqueue.', don't add a 'job-'
  prefix to each metric.
* Separate the key from the job type with a dot rather than a dash.
* To avoid having a Graphite node that is both a "directory" and a metric, use
  '.all' as a suffix for aggregates.

Change-Id: I2ac604d3c042dbfb0b3a27759800f435ec22041e
2015-06-14 22:38:02 -07:00
Aaron Schulz
dc4950a33b Implemented getAllAcquiredJobs in JobQueueDB
Change-Id: Ie9f0b9357b365f8bddd1f2fdcac11dec29aec876
2015-06-03 13:19:37 -07:00
Kunal Mehta
f138447de1 jobqueue: Record stats on how long it takes before a job is run
Bug: T101054
Change-Id: I5dc13d79a5ec2e8cb6679e3ff2535b5cb031ca30
2015-06-03 12:54:27 -07:00
Aaron Schulz
22734b3c0f Let deduplicateRootJob() accept JobSpecification for consistency
Change-Id: I872c2af40028e918fe6d9a8cd32ac97c70bffdae
2015-05-23 12:49:44 -07:00
jenkins-bot
2e89500994 Merge "Removed executeReadyPeriodicTasks() method" 2015-05-12 20:03:02 +00:00
Aaron Schulz
e5ff2d681f Send job ACKs to statsd
Change-Id: Idf6dcd6baf39484ac6fcf14465e13ae12405dd3a
2015-05-11 16:55:42 -07:00
Aaron Schulz
ec3da97659 Removed executeReadyPeriodicTasks() method
* Moved all these hacks to JobQueueDB, which is the only queue that
  should need this (for stock installs). Newer queues should always
  have the queue store manage stuff like this, not MediaWiki.
* This also avoids expensive object construction that does nothing
  when non-DB queues are used.

Change-Id: Id718cda25750be73044a049b39958cca55aa3172
2015-05-06 20:20:40 -07:00
Aaron Schulz
9c6721e38e Removed "empty" cache key from JobQueueDB for simplicity
Bug: T91815
Change-Id: I74d3f8257c728f3ec5d223cf8ca5e2f93aceab13
2015-04-27 20:49:35 +00:00
Ori.livneh
787f2dfc36 Revert "Added per-wiki queue stats information"
Creates too many metrics.
This reverts commit 340220080e.

Task: T95913
Change-Id: Ifa478996febcdd6b3687c4cb64b215495514a899
2015-04-20 19:31:03 +00:00
Aaron Schulz
f7f49dfb00 Made job factory() callers use Title::makeTitle
* The titles are not from user input and this simplifies the code

Change-Id: I067049cca5661b387076f0c28bc0b71d22162a0f
2015-04-13 21:31:29 +00:00
Aaron Schulz
37042262e3 Moved some JobQueueAggregator logic out of JobQueueGroup
Change-Id: I28ba1a25db225d4cf5f503a6c0f4405f13118151
2015-03-03 21:56:15 -08:00
Ricordisamoa
12dec5d85d Fix some stuttering in comments and documentation
Change-Id: I9c0088b9aab37335203cad45a1d6fa8ac3f43321
2014-12-17 19:44:10 +00:00
Aaron Schulz
87e33d67c9 Made JobQueueDB::doWaitForBackups wait for the right slaves
Change-Id: I6022fea7add5c317da89d5361d166a047bcc95e4
2014-09-24 13:07:34 -07:00