Make getServer() always return a string, as documented, even with new
Database::NEW_UNCONNECTED handles that have yet to call open(). If the
'host' parameter to __construct() is ''/null, getServer() now returns
'localhost' instead of null. This avoids problems like fatal errors in
calls to TransactionProfiler::recordConnection().
Use Database constants for "connectionParams" field keys for better
static analysis.
Also:
* Add Database::getServerName() method that returns "readable" server
names in the style of LoadBalancer::getServerName(). Note that the
"hostName" field is already passed in from LoadBalancer.
* Migrate most getServer() callers to getServerName() for easier
debugging and more readable logging.
* Also, normalize Database/LoadBalancer SPI logging context to use
"db_server" and reduce logging code duplication in LoadBalancer.
Bug: T277056
Change-Id: I00ed4049ebb45edab1ea07561c47e226a423ea3b
This checks that the Database state has not diverged from the driver DB
handle nor server-side connection state due to an exception being thrown
in an unexpected place within internal Database methods. DB handles with
possible state corruption will not accept queries.
For example, a PHP extension like Excimer might be used to throw request
timeout exceptions. Such exceptions can trigger after any PHP or Zend
function returns, e.g. within DatabaseMysqlBase::doSelectDomain() after
the "USE" query completes but before $this->currentDomain gets updated.
Also:
* Make getApproximateLagStatus() catch getLag() errors since begin()
expects it to simply use "false" for the lag value on failure. This
helps assure that $this->trxAutomatic gets properly set.
* Unsuppress exceptions in runOnTransactionPreCommitCallbacks()
as the transaction needs to get aborted anyway (as already happens).
* Unsuppress exceptions in runOnAtomicSectionCancelCallbacks() since
the safest thing to do is just roll back the transaction.
* Only suppress DBError exceptions in runOnTransactionIdleCallbacks().
and runTransactionListenerCallbacks(). Return the array of errors
rather than throw the first one. Most of the callers had to catch
the errors, so it's easier to avoid throwing them to begin with.
* Avoid blanket try/catch in sourceStream(), doReplace(), upsert(),
doInsertSelectGenericand().
* Clarify various code comments and add missing @internal tags.
Bug: T193565
Change-Id: I6b7b02c02b24c2ff01094af3df54c989fe504af7
Also:
* Add more documentation and phan annotations
* Avoid code duplication in resetExpectations()
* Make setExpectation() a bit more readable
* Move non-static field initialization to __construct()
* Convert two fields into simple class constants
* Mark TransactionProfiler with @internal
* Use reportExpectationViolated() in more cases in order to
avoid similar logging code
* Make all transactionWritingOut() and recordQueryCompletion()
arguments mandatory since the only callers already gave them
Bug: T269789
Bug: T277056
Change-Id: I6315e9c6a96f65343d45184a60d12665cbc32fc9
Also:
* Add more documentation and phan annotations
* Avoid code duplication in resetExpectations()
* Make setExpectation() a bit more readable
* Add more type hints to methods
* Move non-static field initialization to __construct()
* Convert two fields into simple class constants
* Mark TransactionProfiler with @internal
* Use reportExpectationViolated() in more cases in order to
avoid similar logging code
* Make all transactionWritingOut() and recordQueryCompletion()
arguments mandatory since the only callers already gave them
Bug: T269789
Change-Id: I5d106ac5c95d0ea94c4f6bdee90ca51f8f7ddbad
Add several new internal methods to help with wrangling
the various formats that rows, conditions, options, and
unique key lists can come in. Remove now unused method
isMultiRowArray().
Add various sanity checks and logging for parameters to
upsert(), replace(), insert(), and insertSelect().
Move DatabasePostgresTest to the integration/ directory.
Change-Id: If5988a6f0816e8da2cbf2fd612e1a3e3a2e9c52f
Add a query builder class which encapsulates the parameters to
IDatabase::select() and related functions.
Override useIndexClause() and ignoreIndexClause() in DatabaseTestHelper
so that index hints can be tested.
Bug: T243051
Change-Id: I58eec4eeb23bd7fb05b8b77d0a748f1026afee52
Move the DBO_TRX init logic out of Database::__construct() and into
LoadBalancer since the later already handles setting and clearing this
flag based on transaction rounds starting and ending.
Add 'lazyMasterHandle', 'topologyRole', and 'topologicalMaster' parameters
to Database::factory() and inject them via LoadBalancer all at once in order
to avoid worrying about call order. Move some type casting code to
Database::__construct().
Add IDatabase::getTopologyRole()/getTopologicalMaster().
Use constants for getLBInfo()/setLBInfo() for better usage tracking and
typo resistance.
Change-Id: I437ce434326601e6ba36d9aedc55db396dfe4452
Add public, protected or private to function missing a visibility
Enable the tests folder for the phpcs sniff
Change-Id: Ibefce76ea9984c47e08c94889ea2eafca7565e2c
Rename Database::hasFlags() to sound less similar to getFlag()/setFlag().
The order of class fields is roughly:
* Objects and resources
* Configuration
* Mutable options that do not have to be kept in sync with the connection
* Session level state tracking (should be in sync with the connection)
* Transaction level state tracking (should be in sync with the connection)
* Results from or timestamp of the last time a certain event occurred
Change-Id: Ibdca2fefe7ed2c792344c5602b5191a950eed933
This avoids having two similar fields that have to stay
in sync. Clean up the related error handling for connections.
If a connection handle is unusable, like when essential SET
queries fail, then destroy it.
Also:
* Avoid use of transactions in DatabasePostgres::determineCoreSchema.
* Make sure all subclasses log on connection failure.
* Add schema sanity checks to mysql/sqlite classes.
* Add IDatabase::QUERY_NO_RETRY flag to simplify reasoning about
queries that already run on open() to begin with.
* Remove unused return value of Database::open.
* Remove deprecated Database::reportConnectionError method.
Change-Id: I97beba7ead1523085bda8784234d00c69ef1accc
Make IDatabase::lastDoneWrites() reflect creation and changes to
the cloned temporary unit test tables but not other temporary tables.
This effects the LB method hasOrMadeRecentMasterChanges(). Other tables
are assumpted to really just be there for temporary calculations rather
acting as test-only ephemeral versions of permanent tables. Treating
writes to the "fake permanent" temp tables more like real permanent
tables means that the tests better align with production.
At the moment, temporary tables still have to use DB_MASTER, given
the assertIsWritableMaster() check in query(). This restriction
can be lifted at some point, when RDBMs compatibility is robust.
Bug: T218388
Change-Id: I4c0d629da254ac2aaf31aae35bd2efc7bc064ac6
Mainly:
* Stash trxLevel as the variable $priorTransaction since
Database::replaceLostConnection might make it 0 when called.
* Factor out Database::beginIfImplied method and call it on
each query attempt of query(), not just the first one.
* Do not bother setting STATUS_TRX_ERROR if a query fails due to
connection issues and was recoverable since requiring ROLLBACK
in order to continue has no real advantage.
* Do not bother setting trxDoneWrites/lastWriteTime for temporary
table operations.
* Make Database::handleTransactionLoss() keep TransactionProfiler
cleaner by calling Database::transactionWritingOut().
Also:
* Make sure Database::wasKnownStatementRollbackError() calls are
right after the corresponding queries so it is easy to follow.
Having connection attempts in between seems fragile.
* Rename Database::doProfiledQuery => Database::attemptQuery and
move more logic to that method.
* Factor out Database::assertNeitherReplicaNorReadOnly method.
* Rename Database::assertOpen => Database::assertHasConnectionHandle.
* Fix wording of Database::wasKnownStatementRollbackError comments.
* Use $isEffectiveWrite variable name instead of $isNonTempWrite
and $isWrite in some places.
Bug: T218226
Change-Id: I2063e4080b41d5fc504f9207a56312ce92130ed7
The Profiler::profileIn and Profiler::profileOut methods are just stubs.
Use a callback to the Profiler::scopedProfileIn method instead.
Change-Id: I16068bce583bb880250fe91235f2283453be5e4c
Also improved the atomicity and affected row count logic for
insert/replace with sqlite.
Also remove unused "fileHandle" code from insert().
Change-Id: If7b9148fd44f3a958899885753c7c86ba66bf193
LoadBalancer uses Database::getDomainId() for deciding which keys to use
in the foreign connection handle arrays. This method should reflect any
changes made to the DB selection.
If the query fails, then do not change domain field. This is the sort of
approach that LoadBalancer is expects in openForeignConnection(). Also,
throw an exception when selectDB() fails.
The db/schema/prefix fields of Database no longer exist in favor of just
using the newer currentDomain field.
Also:
* Add IDatabase::selectDomain() method and made selectDB() wrap it.
* Extract the DB name from sqlite files if not explicitly provided.
* Fix inconsistent open() return values from Database subclasses.
* Make a relationSchemaQualifier() method to handle the concern of
omitting schema names in queries. The means that getDomainId() can
still return the right value, rather than confusingly omitt the schema.
* Make RevisionStore::checkDatabaseWikiId() account for the domain schema.
Unlike d2a4d614fc, this does not incorrectly assume the storage is
always for the current wiki domain. Also, LBFactorySingle sets the local
domain so it is defined even in install.php.
* Make RevisionStoreDbTestBase actually set the LoadBalancer local domain.
* Make RevisionTest::testLoadFromTitle() account for the domain schema.
Bug: T193565
Change-Id: I6e51cd54c6da78830b38906b8c46789c79498ab5
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '
(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)
Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).
Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
I532bc5201 added code to put the Database into an error state on error,
to prevent callers from catching and ignoring exceptions without rolling
back. But to avoid breaking everything relying on the ability to do so,
it didn't set the error state for certain types of errors.
To allow those broken callers to be cleaned up, log a deprecation
warning when we detect that someone has indeed ignored one of these
errors.
Bug: T189999
Change-Id: Ib7aca59639f30959e106fd4f1a1209e28bad2857
If we're not going to set trxStatus to an error state in this case, we
need to issue a rollback to be sure the database (i.e. PostgreSQL) isn't
still in an error state too.
Bug: T189999
Change-Id: Id6e203b216fff937b6a97d779b36c278e3366409
Handle all errors in query() that might have caused rollback by
putting the Database handle into an error state that can only be
resolved by cancelAtomic() or rollback(). Other queries will be
rejected until then.
This results in more immediate exceptions in some cases where
atomic section mismatch errors would have been thrown, such as a
an error bubbling up from a child atomic section. Most cases were
a try/catch block assumes that only the statement was rolled back
now result in an error and rollback.
Callers using try/catch to handle key conflicts should instead use
SELECT FOR UPDATE to find conflicts beforehand, or use IGNORE, or
the upsert()/replace() methods. The try/catch pattern is unsafe and
no longer allowed, except for some common errors known to just
rollback the statement. Even then, such statements can come from
child atomic sections, so committing would be unsafe. Luckily, in
such cases, there will be a mismatch detected on endAtomic() or a
dangling section detected in close(), resulting in rollback.
Remove caching from DatabaseMyslBase::getServerVariableSettings
in case some SET query changes the values.
Bug: T189999
Change-Id: I532bc5201681a915d0c8aa7a3b1c143b040b142e
Done using the PhpStorm refactor->rename tool.
Also move "defaultBigSelects" declaration to DatabaseMysqlBase
as no other classes uses that.
Change-Id: I424a2d9815de3a5d4cca2522f3db23a5efe6b592
* Update replace()/upsert() to combine the affected row
count for the non-native case
* Also make replace() atomic in the non-native case,
similar to how upsert() already works
Change-Id: I6c9bcba54eca6bcf4a93a9b230aaedf7f36aa877
Constructs a query for the union of permutations of a set of fields, for
use in situations where the database otherwise makes poor plans due to
inability to use indexes effectively (e.g. T149077 and T168010).
Change-Id: I20980dcada664486c09198b8c45896620bd83e81
Also make the temp table tracking catch plain "DROP TABLE"
in addition to the stricter "DROP TEMPORARY TABLE" clause.
Bug: T146300
Change-Id: Ia8306ec25e63adcdcf0dcc8f6a700dd01afdc948
This class is in /libs and cannot depend on all of MediaWiki.
Replace the call with a simple debug() call instead.
Also, make the legacy logger route/format errors from the two
new DB log types (DBConnection, DBQuery) to the old wfLogDBError
locations, including MWDebug::debugMsg().
Change-Id: I64895d3f5b9a000d8186ab6a6ffb4b76a7e9ff40
This avoids slave lag and makes query time account easier.
It also avoids table-level autoinc locking and slave drift
with statement-based replication in some setups.
Also refactored the use of $wgCommandLine mode in
DatabaseBase slightly, so that it can be injected.
Change-Id: I2dba6024ecf32c9ee24a3080cce3b02568c1458b
* All callers are calling it this way and it breaks
hhvm when declared statically and called dynamically.
Change-Id: I894e615fd828615384aa8457a16a759c8aa416ef
Created a DatabaseTestHelper class, which extends DatabaseBase and
implements STUBs for the interface methods and abstract methods
Change-Id: I9965b3604e78b2722077a35a7b4ce62a5bcb370e
