Make DBConnRef enforce the DB domain selected during its lifetime
and allow more nested and successive use of the same connection handle
via DBConnRef. This can avoid extra connections in some cases where
getConnection()/getConnectionRef() is used.
Also:
* Reduce the number of connection pools arrays from six to two
* Merge getLocalConnection()/getForeignConnection() into one method
* Expand various related code comments
Since LoadBalancer::getReadOnlyReason() no longer user the local domain
but rather DOMAIN_ANY, it should not result in "USE" errors if the local
domain does not have a database on the server.
This version of the patch removes the unused reuseConnectionInternal()
method (the method was previously added back to the patch by mistake).
Bug: T226595
Change-Id: I62502f4de4f86a54f25be1699c4d1a1c1baee60b
This effects:
* LoadBalancer::getReadOnlyReason()
* LoadBalancer::getLaggedReplicaMode()
* LoadBalancer::isPrimaryRunningReadOnly()
* LoadBalancer::getRandomNonLagged()
The lag/read-only mode is designed to be tracked per-server. The DB domain
is not useful here, and passing one complicates callers, causing needless
cache fragmentation, and cause can errors when a foreign LoadBalancer falls
back (or is provided) a local DB domain (which might not reside on the server).
Clarified some related "server" vs "DB" terminology in comments.
Bug: T318904
Change-Id: I3e09c1915d012db36be1bc505e00dcb0603fd388
The lag/read-only mode is designed to be tracked per-server.
The DB domain is not useful here, and passing one complicates callers,
causing needless cache fragmentation.
Change-Id: Ifd08659afadc4f7ae81b6d6f71c9b8032f3860b5
Make DBConnRef enforce the DB domain selected during its lifetime
and allow more nested and successive use of the same connection handle
via DBConnRef. This can avoid extra connections in some cases where
getConnection()/getConnectionRef() is used.
Also:
* Reduce the number of connection pools arrays from six to two
* Merge getLocalConnection()/getForeignConnection() into one method
* Expand various related code comments
Bug: T226595
Depends-On: If808cbab429d41e1f2289683533e4a781a4bdf5e
Change-Id: I540b08920997c57cad6445ddb09d8e663eaf4714
Various variables are left from ealier refactor are now unused
and can be removed to make the code easier to read
Change-Id: Id51770af1f08e85c7e7a02234a2cd2ab5b47ee7a
We also bring along Database::attributesFromType(), which relied on the
private ::getClass(). This requires us to inject DatabaseFactory through
the LBFactory/LoadBalancer hierarchy.
Database::factory() is now soft deprecated. All callers outside of
includes/installer/ still need migration.
Bug: T299691
Bug: T315270
Change-Id: I7d057a9438f1b097554679975e4e9b2fc99e7c2b
This method is unused and the naming is not very clear in terms of what
it returns (server name vs config map). Removing it reduces externally
exposed complexity.
Optimize DBConnRef::getServerName() when the server index is known.
Use this to eliminate the "topologicalMaster" parameter from Database.
Rename internal fields and paramters in Database to use "primary"
instead of "master", for consistency.
Add some additional clarifying comments.
Bug: T299691
Change-Id: I98515fa02a58a4c72a06f1ff283b249b1617c886
It's causing running Echo maint scripts to error with:
Cannot close DBConnRef instance; it must be shareable
Change-Id: Ic3c2c499a67e71ba9a3e3cd9af2d7eee5e582ac9
* Document what the LBFactory/LoadBalancer (sub)classes do.
* Move useful descriptions from file doc to class doc so that we don't
pointlessly maintain it in two places, and to allow for the file
block to be consistently visually ignored instead of sometimes
containing useful information.
This patch removes various non-applicable or unrelated descriptions
from the file block that were blindly copy-pasted, thus proving my
point. It also removes two duplicate/clashing definition of the
'Database' defgroup. Keeping only the primary one in IDatabase.php.
* Move ingroup tag to class block, as indexing the source file
in Doxygen creates noise in the navigation sidebar and does not
add add any benefit.
* Fix dead-end reference to a sqlite/README. I'll rework this
in a later patch.
Change-Id: Iad0e67d766f4a7d5b97e7a471b49f2d8e60c506b
This introduces $wgLBFactoryConf['configCallback'] which can be set to a
function that returns updates to be applied to $wgLBFactoryConf. The new
method LBFactory::autoreConfigure() can be called to check the callabck
and, if the config changed, reconfigure all existing LoadBalancers.
Reconfiguring the LoadBalancers causes all open connections to be
invalidated; however, any DBConnRef instances will remain valid and will
acquire a fresh connection from the LoadBalancer automatically when
appropriate.
As a proof of concept, this patch adds support for config reloding
into WikiExporter.
Bug: T298485
Change-Id: I6c3ffde62f6e038730736abe980befd90ec43e1a
According to docs on IDatabase::getDBName, a database does not
neccessarily have a name, so the function may return null in
such a case.
This fixes a unit test failure on php8.1 involving making cache
keys out of nulls
Bug: T313663
Change-Id: Iba880e2060d9c04ff07b883500cd653148eb54bf
Follows-up I3adb57cbb1fd which added this in 2016 "just in case".
This has been causing fatal errors for several year since due to
LoadBalancer::closeAll calling into objects such as $this->logger,
which have in turn already began their own shutdown.
Destructors should generally be minimal and self-contained, which
these were not.
I could not find what case this is for, but I do note that the
Database objects have their own `__destruct` which calls
IDatabase::closeConnection, as implemented by DatabaseMysqli,
which means even if php-mysqli fails to disconnect, we should still
perform our own disconnect at the end of the process via destructors.
I've also removed the debug message while at it, as it appears to be
unreachable in practice and there are no explicit callers to closeAll.
Bug: T214734
Change-Id: I1ca2ad394fe39d444602d95e64e33408da3bce03
* Export the LoadMonitor weight metric to statsd on each global cache
miss.
* Export the replication lag at the same time. This can replace the
getLagTimes.php cron job and should eliminate the sawtooth effect seen
in the cron job exports, which is presumably due to the offset
between cron job start time and heartbeat time. It should also work
around the bug which causes s3 to be missing.
Also:
* Fix a log message in TransactionProfiler. actualSeconds -> actual
since this log message is used for counts as well as time.
Bug: T313004
Change-Id: I61e45870d750019ed2c92f45b2f8b9c33a7e7d65
Change type bool in union types to false where true is not allowed
Add false to DatabaseSqlite::getFulltextSearchModule
Change-Id: I1199b261c4e5c3f6ff184c756f46f2650b16b0c9
If there are row or named locks held, ping() should be called to
make sure that the locks where not lost. If they were lost, then
no transactions should be committed.
Slightly optimize isWriteQuery() by checking QUERY_CHANGE_TRX.
Bug: T307133
Change-Id: I8f97e1e3a4aa72c31ac2d642511af8dadf070b07
This would simplify the code for its users a lot.
Bug: T255493
Depends-On: I6ab4440a13f4682dd3a6e9cfc6c629222fbeab8a
Change-Id: I6e7544763bde56fc1e19de0358b71ba984a978ca
Callback style iteration made sense before generators existed, but
generators make for simpler code. The "call method" variants made
sense before closures existed but defeat static analysis.
So, in LBFactory:
* Add ILBFactory::getAllLBs()
* Deprecate ILBFactory::forEachLB()
* Remove LBFactory::forEachLBCallMethod(), was protected.
* Add LBFactory::getLBsForOwner(), which is protected and has the
internal interface in @return. Adding a new abstract method breaks
Wikibase tests despite LBFactory not being stable to extend.
* Migrate callers. Generators allow you to return/break from the middle
of the loop, which implies a little rearrangement for some callers.
In LoadBalancer, connections supposedly of type IDatabase were
routinely type-hinted as Database in closure parameters so that methods
could be called that were not in the interface. So it's convenient to
get rid of public iteration methods entirely in favour of private
methods returning Database[].
* Hard-deprecate ILoadBalancerForOwner::forEachOpenConnection()
and replace it with a private generator method since nothing called
it externally except for core tests.
* Hard-deprecate ILoadBalancerForOwner::forEachOpenPrimaryConnection()
and replace it with a private generator. DeferredUpdates needed it for
iterating over IDatabase::explicitTrxActive(), so add
ILoadBalancer::explicitTrxActive() as a replacement.
* Replace private method LoadBalancer::forEachOpenReplicaConnection()
with a generator.
Depends-On: If0b382231e27d6d1197fb7b6aef6ab50335df4e5
Change-Id: I64514e77b9bfe737be5b12e1d3c9c49976bb522f
Instance ownership is supposed to protect LoadBalancer and Database
against unauthorized calls to internal methods other than by the owning
LoadBalancer/LBFactory. This seems like unnecessary complexity. It was
introduced for T231443 and T217819, but the link was speculative and in
the end it didn't help to fix or isolate those bugs. Since then it has
caused a production error (T303885) and an intermittent CI failure
(T292239).
Instead, split the ILoadBalancer interface, introducing
ILoadBalancerForOwner, which contains the methods which are only safe
to call by LBFactory or by the caller of LBFactory::newMainLB(). This
allows phan to statically detect inappropriate calls to internal
methods.
Ownership was used for convenience for two things unrelated to its
original purpose:
* Suppressing calls to ScopedCallback::newScopedIgnoreUserAbort() when
the caller has already called it. But nested calls are apparently
harmless, so I just called it unconditionally.
* Suppressing exceptions from Database::close(). I extended the
behaviour for owned instances to apply to all instances, so even
unowned instances will no longer throw on close.
CodeSearch suggests nothing in extensions is calling these methods with
an owner parameter. One extension (Wikibase) overrides a method with an
owner parameter in a test mock class and so needs to be simultaneously
updated.
Depends-On: Ib03aba9d8f5f05b875a321d00b14483633a636a8
Change-Id: I27ba4973d24d759c88b3868c95e7db875801ca0c
Rename canRecoverFromDisconnect() in order to better describe
its function. Make it use the transaction ID and query walltime
as arguments and return an ERR_* class constant instead of a bool.
Avoid retries of slow queries that yield lost connection errors.
Track session state errors caused by the loss of named locks or
temp tables (e.g. during connection loss). Such errors will prevent
further queries except for rollback() and flushSession(), which must
be issued to resolve the error.
Add flushPrimarySessions() methods to LBFactory/LoadBalancer
and use it in places where session state loss is meant to be
safely aknowledged.
Change-Id: I60532f86e629c83b357d4832d1963eca17752944
We really don't need this complexity and it prevents us from improving
connection management.
MaintainableDatabase should stay but the connection ref shouldn't.
Bug: T255493
Change-Id: I867301dc7fa07cac298f8faba9cf82ca4617f50e
This reverts commit ee3c65d541.
Reason for revert: Blocking the train, going to revert the whole chain.
Change-Id: I553115f310d93f98f7c6a2a77de28c4dda7762da
This reverts commit eed58f2f61.
Reason for revert: Blocking the train, going to revert the whole chain
Change-Id: I64f9e5a9dde106671783f958a686ca697182077b
Use the LoadBalancer id in flushPrimarySessions(), not the LBFactory one,
and use assertOwnership() to check $owner, similar to other methods.
In DatabaseMysqlBase::doFlushSession(), change RELEASE_ALL_LOCKS() query
to use RELEASE_LOCK(), since only newer MariaDB versions (>=10.5.2) support
it. No errors were thrown in the method since they are suppressed, but the
syntax error would cause the transaction to be placed in an error state.
Add assertion to testTransactionCallbackChains() that would otherwise fail.
Randomize lock names in lock() tests to avoid contention.
Bug: T292239
Bug: T303887
Follow-Up: ee3c65d541
Follow-Up: 4cac31de4e
Change-Id: I414d737028338cfd5369eee24576df4aa26a2f6f
Rename canRecoverFromDisconnect() in order to better describe
its function. Make it use the transaction ID and query walltime
as arguments and return an ERR_* class constant instead of a bool.
Avoid retries of slow queries that yield lost connection errors.
Add methods and class constants to track session state errors
caused by the loss of named locks or temp tables. Such errors can
be resolved by a "session flush" method.
Make assertQueryIsCurrentlyAllowed() better distinguish ROLLBACK
queries from ROLLBACK TO SAVEPOINT queries. For some scenarios,
only full tranasction ROLLBACK queries should be allowed.
Add flushSession() method to Database and flushPrimarySessions()
methods to LBFactory/LoadBalancer.
Also:
* Rename wasKnownStatementRollbackError() and make it take the
error number as an argument, similar to wasConnectionError().
Add mysql error codes for query timeouts since they only cause
statement rollbacks.
* Rename wasConnectionError() and mark it as protected. This is an
internal method with no outside callers.
* Rename wasQueryTimeout(), remove some HHVM-specific code, and
simplify the arguments.
* Make executeQuery() use a for loop for the query retry logic
to reduce code duplication.
* Move the error state setting logic in executeQueryAttempt() up
in order to reduce code duplication.
* Move the beginIfImplied() call in executeQueryAttempt() up to the
retry loop in executeQuery(). This narrows the executeQueryAttempt()
concerns to sending a single query and updating tracking fields.
* Make closeConnection() and doHandleSessionLossPreconnect() in
DatabaseSqlite more consistent with the base class by releasing named locks.
* Mark trxStatus() as @internal.
Bug: T281451
Bug: T293859
Change-Id: I200f90e413b8a725828745f81925b54985c72180
Fix the fixme in LoadBalancer::getServerConnection() and throw an
exception on connection loss if errors are not silenced. Otherwise a
server shutdown during a maintenance script always leads to an
InvalidArgumentException from DBConnRef::__construct().
Also, improve related log messages.
Change-Id: I8a5af3271264837d7a694382be4c70a735169f8d
This was soft-deprecated in 1.34 and codesearch doesn't bring anything
except this repo in github which is marked as defunct:
https://github.com/Brickimedia/GlobalContribs
Bug: T299392
Change-Id: I03943ca62688e2c9c41df3970ec49e644d46a8de
