Commit graph

164 commits

Author SHA1 Message Date
Aaron Schulz
d2e1734390 rdbms: rename setDomainPrefix to setLocalDomainPrefix in ILoadBalancer
Update CloneDatabase and unit test callers, which are the only things
that are likely to call it.

Change-Id: Idbc6d2965cd3828c2c6d23012319bbf0fc3ffa4a
2018-11-26 10:38:15 -08:00
jenkins-bot
98fd48b57b Merge "rdbms: add ILBFactory::redefineLocalDomain method" 2018-11-26 16:51:39 +00:00
Aaron Schulz
1c205f1822 rdbms: add ILBFactory::redefineLocalDomain method
This is intended for use with scripts like addWiki.php to avoid mismatched domain errors.

Bug: T209483
Change-Id: Ie24f83f4e0fad7b01690d2055e1529a4cb0275b5
2018-11-22 23:51:44 -08:00
Aaron Schulz
35ebcebdac rdbms: add DBConnRef sanity check to LoadBalancer::closeConnection
Change-Id: I0473ffb1f89301530518f1921fe9d5ca68eb194c
2018-11-20 23:06:22 -08:00
stibba
3ee7eee360 Use new ScopedCallback::newScopedIgnoreUserAbort helper function
Introduced in Ib6e307d76f93.

This patch updates FileBackend, LoadBalancer and LBFactory to use a public
function newScopedIgnoreUserAbort in ScopedCallback instead of all using
the exact same function but duplicated.

Bug: T184044
Change-Id: I27d7dc16abfe4b9447d7f3d8bd89f0de3ddeb662
2018-11-01 17:18:01 -07:00
jenkins-bot
a14a638d41 Merge "rdbms: fix connection reuse logic in LoadBalancer for postgres" 2018-10-26 20:20:32 +00:00
jenkins-bot
d31ec23f2a Merge "rdbms: Use "mixed" type for $profiler field in LoadBalancer" 2018-10-25 21:16:59 +00:00
Aaron Schulz
5ad3a32361 rdbms: Use "mixed" type for $profiler field in LoadBalancer
Change-Id: I8d110d0420a155bb62f4212933d9de2562dc7661
2018-10-25 20:47:28 +00:00
Aaron Schulz
7f1a2fcd6b rdbms: fix connection reuse logic in LoadBalancer for postgres
Change-Id: Ic4b7af156fbed54aae0a34a0bbba76490cd505e0
2018-10-24 04:35:14 -07:00
Fomafix
43244db9a2 Use PHP 7 '??' operator instead of if-then-else
Change-Id: If9d4be5d88c8927f63cbb84dfc8181baf62ea3eb
2018-10-21 21:46:46 +02:00
Aaron Schulz
b06f020217 rdbms: re-add DB domain sanity checks to LoadBalancer
Also clean up empty schema handling in DatabaseDomain

This reverts commit f23ac02f4f.

Bug: T193565
Change-Id: I95fde5c069f180ca888a023fade25ec81b846d44
2018-10-16 23:35:05 +00:00
Krinkle
f23ac02f4f Revert "rdbms: add domain sanity checks to LoadBalancer connection methods"
This reverts commit b416e166a3.

Fails on Travis CI / PHP 7.1 / Postgres:

> Creating administrator user account
>
> UnexpectedValueException:
> Got connection to 'traviswiki-mediawiki', but expected 'traviswiki'.
>
> Backtrace:
> #0 includes/libs/rdbms/loadbalancer/LoadBalancer.php(887): Wikimedia\Rdbms\LoadBalancer->openForeignConnection(integer, string, integer)
> #1 includes/libs/rdbms/loadbalancer/LoadBalancer.php(766): Wikimedia\Rdbms\LoadBalancer->openConnection(integer, string, integer)
> #2 includes/libs/rdbms/loadbalancer/LoadBalancer.php(851): Wikimedia\Rdbms\LoadBalancer->getConnection(integer, array, string, integer)
> #3 includes/jobqueue/JobQueueDB.php(780): Wikimedia\Rdbms\LoadBalancer->getConnectionRef(integer, array, string, integer)
> #4 includes/jobqueue/JobQueueDB.php(761): JobQueueDB->getDB(integer)
> #5 includes/jobqueue/JobQueueDB.php(187): JobQueueDB->getMasterDB()
> #6 includes/jobqueue/JobQueue.php(332): JobQueueDB->doBatchPush(array, integer)
> #7 includes/jobqueue/JobQueue.php(302): JobQueue->batchPush(array, integer)
> #8 includes/jobqueue/JobQueueGroup.php(154): JobQueue->push(array)
> #9 includes/user/UserGroupMembership.php(162): JobQueueGroup->push(array)
> #10 includes/user/User.php(3733): UserGroupMembership->insert(boolean)
> #11 includes/installer/Installer.php(1680): User->addGroup(string)
> #12 includes/installer/Installer.php(1601): Installer->createSysop(PostgresInstaller)
> #13 includes/installer/CliInstaller.php(176): Installer->performInstallation(array, array)

Change-Id: I269df89d47cc5a183af630447cd1642d3d4a3deb
2018-10-13 00:35:55 +00:00
Aaron Schulz
b416e166a3 rdbms: add domain sanity checks to LoadBalancer connection methods
Bug: T193565
Change-Id: I94d905277e01b8e30ac3f6532ece07388bb20cce
2018-10-12 02:16:49 +00:00
Aaron Schulz
fe0af6cad5 rdbms: Database::selectDB() update the domain and handle failure better
LoadBalancer uses Database::getDomainId() for deciding which keys to use
in the foreign connection handle arrays. This method should reflect any
changes made to the DB selection.

If the query fails, then do not change domain field. This is the sort of
approach that LoadBalancer is expects in openForeignConnection(). Also,
throw an exception when selectDB() fails.

The db/schema/prefix fields of Database no longer exist in favor of just
using the newer currentDomain field.

Also:
* Add IDatabase::selectDomain() method and made selectDB() wrap it.
* Extract the DB name from sqlite files if not explicitly provided.
* Fix inconsistent open() return values from Database subclasses.
* Make a relationSchemaQualifier() method to handle the concern of
  omitting schema names in queries. The means that getDomainId() can
  still return the right value, rather than confusingly omitt the schema.
* Make RevisionStore::checkDatabaseWikiId() account for the domain schema.
  Unlike d2a4d614fc, this does not incorrectly assume the storage is
  always for the current wiki domain. Also, LBFactorySingle sets the local
  domain so it is defined even in install.php.
* Make RevisionStoreDbTestBase actually set the LoadBalancer local domain.
* Make RevisionTest::testLoadFromTitle() account for the domain schema.

Bug: T193565
Change-Id: I6e51cd54c6da78830b38906b8c46789c79498ab5
2018-10-10 12:03:30 -07:00
Gergő Tisza
cebdd50a49 Make unclosed transaction errors more useful
Move unclosed transaction error reporting to Database so it can
include information about the caller that started the transaction.

Change-Id: I834d957f172c03005de522f3029bb634b3c7220e
2018-10-09 17:50:17 +00:00
Umherirrender
b4016f14d1 Fix logged method in LoadBalancer
Seeing {closure} in the logs as caller is not helpful

Change-Id: I45d9be25acdcc70c7d9755c15f68f7bc6887e166
2018-09-30 17:38:35 +00:00
Aaron Schulz
a3160ceeba rdbms: make schema/prefix handling more consistent in LoadBalancer
Move "tablePrefix" logic to LoadBalancer::reallyOpenConnection and
make it more consistent for local vs foreign connections. The prefix
for local connections is now set even if $domain does not provide a
database.

Also set the schema even if $domain does not provide a database.

Typically a database domain is either fully specified or comes from
DatabaseDomain::newUnspecified(), but there's no reason not to use
at least the schema/prefix of otherwise unspecified domains. The only
other sensible option would be some sort of exception.

Change-Id: Ib52f4b69e429810e3c5eac92a48aecf7b22e86e6
2018-08-23 00:25:53 -07:00
Aaron Schulz
106f632996 rdbms: remove some dead code in LoadBalancer::setDomainPrefix
Change-Id: I594a390b446c00eb604b0be63f67b2c8cbd45758
2018-08-16 06:31:11 -07:00
Aaron Schulz
bfdde8b2ff rdbms: split out private LoadBalancer::openLocalConnection method
Change-Id: I2138b3d28f8f1a8ea706165fbe3a7e3f0d858447
2018-08-15 04:34:29 +00:00
Aaron Schulz
5938a6efdc rdbms: add more comments and sanity checks for CONN_TRX_AUTOCOMMIT
Change-Id: I69992cf2e2ae3ef62125b0bc733a0cb7274f814e
2018-08-09 01:39:07 +00:00
Umherirrender
130ec2523d Fix PhanTypeMismatchDeclaredParam
Auto fix MediaWiki.Commenting.FunctionComment.DefaultNullTypeParam sniff

Change-Id: I865323fd0295aabd06f3e3c75e0e5043fb31069e
2018-07-07 00:34:30 +00:00
Aaron Schulz
2fd62f5dc9 rdbms: add resolveDomainID() method to LBFactory/LoadBalancer
Also add LBFactory::getLocalDomainID to match the one in LoadBalancer

Change-Id: Ia31f0800bd3b692194c08b1eab9cfb2f43679c7a
2018-07-05 14:07:04 +01:00
Aaron Schulz
d7707fa814 rdbms: rearrange LoadBalancer fields and make some private
Put the object/callback fields first, followed by config fields
and then state fields, similar to LBFactory.

Change-Id: I40ceaa48842b505f15efe5e41ca7a6de437b6ff1
2018-06-24 01:29:59 +00:00
Marius Hoch
d74413bbe7 Allow passing the default DB group to use in Maintenance scripts
Also allows setting it via a global, although that is probably
less useful.

Bug: T147169
Change-Id: Ic51204a6f6ce9db4cc96108e823e388512724eff
2018-06-12 23:52:40 +02:00
Max Semenik
1e680456b4 Get rid of call_user_func(_array)(), part 3
Also cleaned up nearby code in a couple places.

Change-Id: Ibf44ee7c0ceb739d7e79406e4ff39303c316e285
2018-06-10 02:21:24 +00:00
jenkins-bot
5a6c78c441 Merge "Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient" 2018-05-31 19:01:07 +00:00
Tim Starling
25503cf011 Maintenance script to wrap the mysql binary
This is more or less equivalent to the WMF shell script "sql", but it is
reasonably portable and written in a less hackish way.

Change-Id: I240d8d4e65ce46680cac257ee376a9b026c13f92
2018-05-31 15:42:51 +10:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
Reedy
765370a6db Add @deprecated tags to various class_alias calls
Bug: T195576
Change-Id: I10cd8415891bfe4a278eee06c9cfe905b3e036dc
2018-05-29 13:10:20 -07:00
Aaron Schulz
86af2ef383 rdbms: fix callback stage errors in LBFactory::commitMasterChanges
Just like 082ed053b6 fixed pre-commit callback errors when new instances
of LoadBalancer are made during that step, do the same for post-commit
callbacks.

Bug: T194308
Change-Id: Ie79e0f22b3aced425cf067d0df6b67e368223e6c
2018-05-10 04:26:41 +00:00
Aaron Schulz
205cfc1854 rdbms: fix LBFactory::commitAll() round handling
This avoids "Transaction round stage must be approved (not cursory)".

Bug: T194308
Change-Id: I9dbfe9cede02b1b1904c1d5e5a9802306c2492a2
2018-05-09 14:51:18 -07:00
Aaron Schulz
082ed053b6 rdbms: fix finalization stage errors in LBFactory::commitMasterChanges
If a pre-commit callback caused a new LoadBalancer object to be created,
that object will be in the "cursory" state rather than the "finalized"
state. If any callbacks run on an LB instance, make LBFactory iterate
over them all again to finalize these new instances.

Make LoadBalancer::finializeMasterChanges allow calls to
already-finalized instances for simplicity.

Bug: T193668
Change-Id: I4493e9571625a350c0a102219081ce090967a4ac
2018-05-07 18:04:43 +00:00
Aaron Schulz
b32325bd23 rdbms: improve log warnings in runMasterPostTrxCallbacks()
Bug: T191282
Change-Id: Iba663c58224af920f90d7b401aab2eb21f921941
2018-05-01 13:24:39 -07:00
Aaron Schulz
5bfa77f8d9 rdbms: enforce and improve LBFactory/LoadBalancer callback handling
* Handle the case where an onTransaction* callback for one handle
  adds more onTransaction* callbacks to a different handle. Instead
  of supporting only a short chain of such callbacks, try to resolve
  the whole chain by using a loop in LoadBalancer and LBFactory.
* Add sanity checks to enforce the proper call order of LoadBalancer
  transaction methods, such as those that execute callbacks. This is
  the order that LBFactory already uses. Use ROUND_ERROR for problems
  that can ruin the instance state. Such problems require rollback.
* Correct setTrxEndCallbackSuppression() calls in beginMasterChanges()
  that were making tests fail.
* Make Database handle callback suppression for FLUSHING_ALL_PEERS
  instead of making LoadBalancer/LBFactory have to manage it.
* Simplify finalizeMasterChanges() given that suppression does not
  actually effect runOnTransactionPreCommitCallbacks().
* Make dangling callback warning in Database::close work properly.
* Actually use $fname in flushReplicaSnapshots().
* Use DBTransactionError instead of DBExpectedError in some places
  where stages fail.
* Fix failing testGetScopedLock() unit tests so everything passes.

Add more comments to setTransactionListener and onTransactionIdle.

Change-Id: I6a25a6e4e5ba666e0da065a24846cbab7e786c7b
2018-04-30 21:55:25 +00:00
Timo Tijhof
c7fc4ef9bf rdbms: Replace reportConnectionError() with direct throws
When reading through DatabaseMysqlBase::open(), it was not
obvious that execution would not continue after the conditional
`!$this->conn` block, given it ends in a method call, without
return or throw. I considered adding a return statement after it
for clarity, but it seems in this case it might make more sense
to throw directly given $error here has already gone through a
fallback to getLastError() a few lines up.

Replace the other three calls to reportConnectionError() as well,
which previously passed a useful string that was overwritten
with lastError(). Instead, log both. And make their call to
queryLogger->error() match the previous ones to have an 'error' as well.

This leaves reportConnectionError() as being unused, except for
a call from LoadBalancer. That call was problematic because
it was inside a conditional for IDatabase, but the method isn't
part of that interface. Replace it with a direct throw as well.

Deprecate the method as its now unused in core, and also remove its
'# New method' comment which hasn't made sense since r75341 (16cded8b32).

Change-Id: I0f2ef00ba44bf7090a3ce54edeb8c7e8e543e46a
2018-04-26 04:25:17 +00:00
jenkins-bot
6f1cb4c3aa Merge "rdbms: remove deprecated getLaggedSlaveMode() method alias" 2018-04-23 22:56:24 +00:00
Aaron Schulz
19b71ac216 rdbms: remove deprecated getLaggedSlaveMode() method alias
Change-Id: Iecdd39270d27ee1a2f9e1d714ec02c710fa241ef
2018-04-23 12:57:10 -07:00
Aaron Schulz
628a3a9b26 rdbms: make sure cpPosIndex cookie is applied to LBFactory in time
Once getMain() was called in setSchemaAliases(), the ChronologyProtector
was initialized and the setRequestInfo() call in Setup.php had no effect.
Only the request values read in LBFactory::__construct() were used, which
reflect $_GET but not cookie values.

Use the $wgDBtype variable to avoid this and add an exception when that
sort of thing happens.

Further defer instantiation of ChronologyProtector so that methods like
ILBFactory::getMainLB() do not trigger construction.

Bug: T192611
Change-Id: I735d3ade5cd12a5d609f4dae19ac88fec4b18b51
2018-04-23 15:44:02 +00:00
Aaron Schulz
fd1e608cae rdbms: remove "m" prefix from LoadBalancer comments
Change-Id: Ie239157eb6ee8b7991c45ef5c5c06a1caaba52c9
2018-04-16 21:20:58 -07:00
Aaron Schulz
e9d42c0e7e rdbms: add $flags argument to ILoadBalancer::getAnyOpenConnection
Make LoadMonitor use this flag for getting a connection.

Change-Id: I32ea9aadc0223c86db5979d6579d781a6af0ff53
2018-04-16 14:17:52 -07:00
jenkins-bot
e929c71631 Merge "rdbms: make LoadBalancer::doWait() cast $timeout to an integer" 2018-04-16 01:01:21 +00:00
Aaron Schulz
5739d1454b rdbms: make LoadBalancer::doWait() cast $timeout to an integer
Make sure that fractional values are not sent to masterPosWait()

Change-Id: I0a66471d4bf7005774b5741702c7f9c65bbfd5d0
2018-04-14 10:13:14 +00:00
jenkins-bot
fe4f16a0e1 Merge "rdbms: make $i in LoadBalancer::getConnection override $groups" 2018-04-13 16:30:37 +00:00
Aaron Schulz
daf0514345 rdbms: make $i in LoadBalancer::getConnection override $groups
Previously, passing a specific server index (even the master index)
would result in a connection to some server in $groups or the generic
read group being selected.

Change-Id: Ifb497f5f4bb61f16b310d4d30fb8315e3bcdce8e
2018-04-12 20:48:04 +00:00
jenkins-bot
6a3aadf097 Merge "rdbms: rename CONN_TRX_AUTO constant to CONN_TRX_AUTOCOMMIT" 2018-04-12 20:38:58 +00:00
Aaron Schulz
f8c2486d15 rdbms: rename CONN_TRX_AUTO constant to CONN_TRX_AUTOCOMMIT
The "AUTO" means AUTOCOMMIT, not "automatic transactions"/DBO_TRX,
which is basically the opposite concept. The new name does not
suffer from that ambiguity.

Keep the old constant as an alias for backwards compatibility.

Also remove LoadBalancer comment about non-existing field

Change-Id: I63beeb061fc9be73f320308e4d6393b58628b8c8
2018-04-12 13:01:52 -07:00
jenkins-bot
48f931b9cb Merge "rdbms: make LBFactory "cliMode" check for phpdbg" 2018-04-11 22:12:34 +00:00
Aaron Schulz
5bad47dbb4 rdbms: make LBFactory "cliMode" check for phpdbg
Change-Id: I618a975198f1432b21e03efa40520d4819a7ce5f
2018-04-10 20:39:26 +00:00
Brad Jorsch
8dfeac7446 rdbms: Add a deprecationLogger callback
Much like the existing errorLogger, but for logging deprecation
warnings.

The default in the RDBMS layer is to call trigger_error() with
E_USER_DEPRECATED. The default in MediaWiki (via MWLBFactory) is to log
to the 'deprecated' log group, much like wfDeprecated() does, although
unfortunately we can't effectively use that directly since we have no
idea of a proper $callerOffset to pass.

Change-Id: Id13625e249516e84d72b6310953bb338a90976da
2018-04-05 22:25:08 +00:00
Aaron Schulz
d4c31cf841 rdbms: clean up DBO_TRX behavior for onTransaction* callbacks
* Make onTransactionIdle() wait until any transaction round
  is gone, even if there is no SQL transaction active. This
  is what onTransactionPreCommitOrIdle() already does.
* Decouple "transaction round mode" (DBO_TRX) from whether a
  round is active via a 'trxRoundId' LB info field. If rounds
  are enabled, but not is started, then the transaction state
  should be interpreted as "idle".
* Improve related documentation.
* Add more related unit tests.

Change-Id: I3ab18f577ec0375897fcb63f18f4ee2deeb436e9
2018-04-03 23:16:45 -07:00