Commit graph

56 commits

Author SHA1 Message Date
jenkins-bot
f426a261ff Merge "rdbms: Remove deprecated IDatabase functions" 2022-03-15 20:28:56 +00:00
Amir Sarabadani
46374a8e3e rdbms: Remove deprecated IDatabase functions
Soft-deprecated in 1.37, hard-deprecated in 1.38

Bug: T286694
Change-Id: Icd23271149bba3e4632d595c057a733d13707503
2022-03-15 19:33:32 +01:00
Aaron Schulz
ee3c65d541 rdbms: provide $owner argument in LoadBalancer::flushPrimarySessions()
This avoids DBUnexpectedError in LBFactory::flushPrimarySessions()

Follow-up: 4cac31de4e
Change-Id: I6b6e274135d842f379d71fb9efb42758984afc78
2022-03-15 10:42:21 -07:00
Amir Sarabadani
0543fc9906 rdbms: Migrate TransactionProfiler calls to TransactionManager
Bug: T299698
Change-Id: I4bf6ab7768909233a7bf75fd6eb842449852c333
2022-02-26 00:46:50 +01:00
jenkins-bot
24aa34d06c Merge "phpcs: Disable Generic.Files.LineLength for test files" 2022-02-21 15:51:29 +00:00
jenkins-bot
26190f6e41 Merge "Remove deprecated LoadBalancer::openConnection() method" 2022-02-21 06:03:00 +00:00
Timo Tijhof
8d406bbcd6 phpcs: Disable Generic.Files.LineLength for test files
There is a common and reasonable need for longer lines in tests.
The nudge for shorter lines doesn't seem valuable here. The natural
breaks will likely still fall in 80-100 given the enforced practice
for non-test code, e.g. whether through habit, or 80-100 column markers
in text editors, or the finite width of diff and code review
interfaces.

Change-Id: I879479e13551789a67624ce66f0946d2f185e6ee
2022-02-18 18:32:05 +00:00
Amir Sarabadani
5c6c7a8568 rdbms: Move more code from Database to TransactionManager
Bug: T299698
Change-Id: I063d76243271b1291a38cc5d722af91cfcd63ed7
2022-02-16 04:45:40 +01:00
Aaron Schulz
70d0e5fde5 Remove deprecated LoadBalancer::openConnection() method
Change-Id: Id88dc030c31b5509078412a1b845814252589dda
2022-02-14 23:03:09 -08:00
Amir Sarabadani
e7f3f202dd rdbms: Move more internal code from Database to TransactionManager
Most of transaction status handling is now in TransactionManager.

Bug: T299698
Change-Id: I399380768dfa58504b2205403392c53eee73096e
2022-02-05 04:58:40 +00:00
Tim Starling
ff9675bbe7 Database::factorConds(): fix insufficient parenthesization
Follows-up 4ba64a1967 (Ic2d6b054b) which added the method with the bug,
and the LinksUpdate refactor (d3b2b8006, I472f4a023) which first
used the method.

Bug: T299095
Change-Id: I9746e69b7b4f79d0afb9b965da43b9ae36c6afe4
2022-01-12 23:46:26 +00:00
Tim Starling
bb9a442be9 Add per-table straight join option
Add support for MySQL's per-table syntax for straight joins, which is
a join type analogous to LEFT JOIN.

SelectQueryBuilder::straightJoin() previously enabled the whole-query
straight join option, but it makes more sense for it to work analogously
with leftJoin(). So I renamed the existing method to
straightJoinOption(). No callers found in CodeSearch.

The motivation is to allow us to prevent reordering of the change_tag
join without affecting the rest of the recentchanges query.

Bug: T298225
Change-Id: I2bfcffc2bf9a64f23afdea0dfaccf89938896e62
2022-01-07 13:26:25 +11:00
Amir Sarabadani
be11c21d21 rdbms: Remove IDatabase::fieldName() method
Deprecated and unused,
https://codesearch.wmcloud.org/search/?q=%5CbfieldName%5C(&i=fosho&files=%5C.php&excludeFiles=&repos=

Bug: T296960
Bug: T286694
Change-Id: I3c1a1a90a904c9ead03b7158a87f6c4ee005b987
2021-12-21 23:11:23 +00:00
Amir Sarabadani
050fbbec8f rdbms: Remove IDatabase::numFields() method
Soft-deprecated since 1.37.

Unused https://codesearch.wmcloud.org/search/?q=numFields&i=nope&files=&excludeFiles=&repos=

Bug: T286694
Bug: T296960
Change-Id: I722602c8b11dcfc5cab70593876a4fb336e9684a
2021-12-21 23:10:49 +00:00
jenkins-bot
d81f7ee110 Merge "rdbms: add query timeout support to Database::select()" 2021-12-13 20:33:22 +00:00
Aaron Schulz
46425659fa rdbms: add query timeout support to Database::select()
This will modify the SQL to add a timeout specifier, depending
on whether the server is running MySQL or MariaDB, and whether
such features are supported for the version number.

Bug: T129093
Bug: T195792
Change-Id: I12e095fe82622064e555a42f1812eb99d201f10c
2021-12-13 20:08:16 +00:00
Amir Sarabadani
8dca355526 rdbms: Remove five deprecated methods from IDatabase
They are not used in anything deployed to production
These methods are:
  - ::getTopologyRootMaster() since 1.37
  - ::masterPosWait() since 1.37
  - ::dataSeek() soft-deprecated since 1.37
  - ::onTransactionIdle() soft-deprecated since 1.32
  - ::getMasterPos() since 1.37

Bug: T296960
Change-Id: I1d196172b3f50e1200980c6325beccecac7db903
2021-12-13 09:07:54 +01:00
Tim Starling
4ba64a1967 Add IDatabase::factorConds()
Generalize makeWhere2d() to provide factorConds(), which simplifies any
boolean expression consisting of equality conditions in a
sum-of-products form.

Change-Id: Ic2d6b054b69fd0be0577807bcc4afb919af7d693
2021-12-08 11:02:46 +11:00
James D. Forrester
b81dfed73e DatabaseMysqlBase: Rename getMasterServerInfo to getPrimaryServerInfo
Hard-deprecating immediately as no known users in git outside of this repo.

Bug: T282894
Change-Id: I79310cafeae368aa2a06b4eec69359663fa11e9d
2021-09-03 20:45:05 +00:00
James D. Forrester
07f9d1db49 Database: Rename assertIsWritableMaster to assertIsWritablePrimary
Hard-deprecating immediately as no known users in git outside of this repo.

Bug: T282894
Change-Id: I5c8152b3d4b58e49a641b235532e40b06362b7c2
2021-09-03 20:11:17 +00:00
James D. Forrester
2cfcca23f5 ILoadBalancer/ILBFactory: Rename hasMasterChanges to hasPrimaryChanges
Bug: T282894
Change-Id: Ief9cda8fece28c7f515ca30d0154f1e24559f687
2021-09-02 16:14:01 -07:00
James D. Forrester
10324c232a ILoadBalancer/ILBFactory: Rename rollbackMasterChanges to rollbackPrimaryChanges
Bug: T282894
Change-Id: I31794e052d71160195dd3b6c29fea24bc98b356b
2021-09-02 12:50:52 -07:00
James D. Forrester
5ad7ca7eba ILoadBalancer/ILBFactory: Rename commitMasterChanges() to commitPrimaryChanges()
Bug: T282894
Change-Id: I0d80be56e683924254c4e38d05e1109ea74eeab5
2021-09-02 11:27:10 -07:00
James D. Forrester
577e1c3710 ILoadBalancer/ILBFactory: Rename beginMasterChanges() to beginPrimaryChanges()
Bug: T282894
Change-Id: I10e607215e6772c48670659719948f7135472a7f
2021-09-01 22:15:45 +00:00
Aaron Schulz
acd3a53799 rdbms: add unique key checks to upsert() and replace() in Database
Disallow changing unique key values of rows in the upsert() assignment.
The unique key values are used to decide which new and old rows collide.
Changing these values at the same time is too complex and ill-defined.

Disallow setting NULL values for columns of the identity key in upsert()
and replace(), unless they are all NULL for all of the rows. This edge
case can be treated as a plain INSERT, assuming that conflicts are not
triggered over weird DEFAULT column values.

Update the corresponding IDatabase documentation.

Change-Id: If6e0cecce32a5eedaf9a82b67ba41296bfa4a317
2021-08-18 19:18:40 -07:00
James D. Forrester
1d71fd91d6 ILoadBalancer and IDatabase: Rename getMasterPos to getPrimaryPos
Bug: T282894
Change-Id: I30a600335af735a13a6ad2d3df56a5f17e05bfab
2021-08-02 18:14:41 +00:00
James D. Forrester
d11c59538a Rename DB primary position interfaces to DBPrimaryPos and MySQLPrimaryPos
And replace all uses.

Bug: T282894
Change-Id: I5222a8568255ac9fa5e2350e2264b8d2ee5eb968
2021-08-02 17:59:39 +00:00
libraryupgrader
5357695270 build: Updating dependencies
composer:
* mediawiki/mediawiki-codesniffer: 36.0.0 → 37.0.0
  The following sniffs now pass and were enabled:
  * Generic.ControlStructures.InlineControlStructure
  * MediaWiki.PHPUnit.AssertCount.NotUsed

npm:
* svgo: 2.3.0 → 2.3.1
  * https://npmjs.com/advisories/1754 (CVE-2021-33587)

Change-Id: I2a9bbee2fecbf7259876d335f565ece4b3622426
2021-07-22 03:36:05 +00:00
DannyS712
ae4efe66c6 Don't pass unneeded variables into anonymous functions
Change-Id: Iad4d1dc4727828f9e0120c8bf99dc245a29ee14d
2021-07-13 19:48:26 +00:00
jenkins-bot
064eea6483 Merge "rdbms: disallow upsert()/replace() calls with multiple unique keys" 2021-05-07 01:55:56 +00:00
Aaron Schulz
99d5d2e8cc rdbms: cleanup getServer() and connection parameter fields in Database
Make getServer() always return a string, as documented, even with new
Database::NEW_UNCONNECTED handles that have yet to call open(). If the
'host' parameter to __construct() is ''/null, getServer() now returns
'localhost' instead of null. This avoids problems like fatal errors in
calls to TransactionProfiler::recordConnection().

Use Database constants for "connectionParams" field keys for better
static analysis.

Also:
* Add Database::getServerName() method that returns "readable" server
  names in the style of LoadBalancer::getServerName(). Note that the
  "hostName" field is already passed in from LoadBalancer.
* Migrate most getServer() callers to getServerName() for easier
  debugging and more readable logging.
* Also, normalize Database/LoadBalancer SPI logging context to use
  "db_server" and reduce logging code duplication in LoadBalancer.

Bug: T277056
Change-Id: I00ed4049ebb45edab1ea07561c47e226a423ea3b
2021-05-05 19:44:02 +00:00
Aaron Schulz
7cc4868758 rdbms: disallow upsert()/replace() calls with multiple unique keys
Change-Id: I4b6f07e8c8b3915c780885083f7465790328d2a3
2021-05-05 08:24:44 -07:00
Aaron Schulz
889a37f8c3 rdbms: Use server time in DatabaseMysqlBase::getLagFromPtHeartbeat()
The avoids problems stemming from clock skew among app and db servers
along with problems caused by per-request clock drift.

Bug: T248481
Change-Id: Ifb3db110ddf8f209e0e19126abeb44437dee3768
2021-04-30 15:29:36 -07:00
jenkins-bot
23d5d441c9 Merge "Replace uses of DB_MASTER with DB_PRIMARY" 2021-04-29 19:04:17 +00:00
James D. Forrester
df5eb22f83 Replace uses of DB_MASTER with DB_PRIMARY
Just an auto-replace from codesniffer for now.

Change-Id: I5240dc9ac5929d291b0ef1c743ea2bfd3f428266
2021-04-29 09:24:31 -07:00
Aaron Schulz
f39f0384f3 rdbms: remove whitespace padding from IDatabase::conditional()
Change-Id: Ib3d808966f8ea7a29fd503b0727373dae153d0ec
2021-04-27 11:03:09 -07:00
Thiemo Kreuz
b95a07380a Remove meaningless ->expects( $this->any() ) from all tests
It is not entirely meaningless. It might be an indicator that
the number of calls to a method is intentionally unlimited.
This is similar to e.g. an @inheritDoc PHPDoc comment that
marks a method as being "intentionally undocumented".

However, what's the meaning of being "intentionally
unconstrained"? Let's just not have any constraint then.

I feel all these ->expects( $this->any() ) bloat the test
code so much that it's never worth it.

Change-Id: I9925e7706bd03e1666f6eb0b284cb42b0dd3be23
2021-04-23 11:58:58 +02:00
Daimona Eaytoy
535d7abf59 phpunit: Mass-replace setMethods with onlyMethods and adjust
Ended up using
  grep -Prl '\->setMethods\(' . | xargs sed -r -i 's/setMethods\(/onlyMethods\(/g'

special-casing setMethods( null ) -> onlyMethods( [] )

and then manual fix of failing test (from PS2 onwards).

Bug: T278010
Change-Id: I012dca7ae774bb430c1c44d50991ba0b633353f1
2021-04-16 20:15:00 +02:00
jenkins-bot
1128084434 Merge "Move libs/rdbms/database tests to unit tests" 2021-04-14 02:34:20 +00:00
jenkins-bot
e8f0d25e29 Merge "rdbms: detect corrupt Database instances due to critical section failure" 2021-04-08 02:31:43 +00:00
DannyS712
27eb31153f Move libs/rdbms/database tests to unit tests
No integration is needed
Also switch to extending MediaWikiUnitTestCase

Change-Id: I5d91e2de223a8dff9d61e8888778994d73e604df
2021-04-07 13:46:54 +00:00
Ammarpad
17b471645a rdbms: Remove obsolete db connection params in DatabaseTest
They were used by MSSQL which is completely unsupported now.

Change-Id: Ibc4f8c13727964032a7cf5c39a6745d5c6c55334
2021-03-25 22:27:31 +00:00
Aaron Schulz
278bb5c23e rdbms: detect corrupt Database instances due to critical section failure
This checks that the Database state has not diverged from the driver DB
handle nor server-side connection state due to an exception being thrown
in an unexpected place within internal Database methods. DB handles with
possible state corruption will not accept queries.

For example, a PHP extension like Excimer might be used to throw request
timeout exceptions. Such exceptions can trigger after any PHP or Zend
function returns, e.g. within DatabaseMysqlBase::doSelectDomain() after
the "USE" query completes but before $this->currentDomain gets updated.

Also:
* Make getApproximateLagStatus() catch getLag() errors since begin()
  expects it to simply use "false" for the lag value on failure. This
  helps assure that $this->trxAutomatic gets properly set.
* Unsuppress exceptions in runOnTransactionPreCommitCallbacks()
  as the transaction needs to get aborted anyway (as already happens).
* Unsuppress exceptions in runOnAtomicSectionCancelCallbacks() since
  the safest thing to do is just roll back the transaction.
* Only suppress DBError exceptions in runOnTransactionIdleCallbacks().
  and runTransactionListenerCallbacks(). Return the array of errors
  rather than throw the first one. Most of the callers had to catch
  the errors, so it's easier to avoid throwing them to begin with.
* Avoid blanket try/catch in sourceStream(), doReplace(), upsert(),
  doInsertSelectGenericand().
* Clarify various code comments and add missing @internal tags.

Bug: T193565
Change-Id: I6b7b02c02b24c2ff01094af3df54c989fe504af7
2021-03-24 14:25:50 +00:00
Aaron Schulz
383a5bcad9 rdbms: avoid undefined "expectBy" notices in TransactionProfiler (II)
Also:
* Add more documentation and phan annotations
* Avoid code duplication in resetExpectations()
* Make setExpectation() a bit more readable
* Move non-static field initialization to __construct()
* Convert two fields into simple class constants
* Mark TransactionProfiler with @internal
* Use reportExpectationViolated() in more cases in order to
  avoid similar logging code
* Make all transactionWritingOut() and recordQueryCompletion()
  arguments mandatory since the only callers already gave them

Bug: T269789
Bug: T277056
Change-Id: I6315e9c6a96f65343d45184a60d12665cbc32fc9
2021-03-16 03:58:11 +00:00
Bartosz Dziewoński
a8e44cceb2 Revert "rdbms: avoid undefined "expectBy" notices in TransactionProfiler"
This reverts commit 9624e4f74a.

Bug: T277056
Change-Id: I93cdac28edf3e4220269667fd244b907ef189726
2021-03-15 23:13:59 +01:00
Aaron Schulz
9624e4f74a rdbms: avoid undefined "expectBy" notices in TransactionProfiler
Also:
* Add more documentation and phan annotations
* Avoid code duplication in resetExpectations()
* Make setExpectation() a bit more readable
* Add more type hints to methods
* Move non-static field initialization to __construct()
* Convert two fields into simple class constants
* Mark TransactionProfiler with @internal
* Use reportExpectationViolated() in more cases in order to
  avoid similar logging code
* Make all transactionWritingOut() and recordQueryCompletion()
  arguments mandatory since the only callers already gave them

Bug: T269789
Change-Id: I5d106ac5c95d0ea94c4f6bdee90ca51f8f7ddbad
2021-03-09 16:59:52 +00:00
Umherirrender
a1de8b8700 Tests: Mark more more closures as static
Result of a new sniff I25a17fb22b6b669e817317a0f45051ae9c608208

Bug: T274036
Change-Id: I695873737167a75f0d94901fa40383a33984ca55
2021-02-09 02:55:57 +00:00
Reedy
729f20afc8 Tests: Mark more closures as static
Bug: T274036
Change-Id: I911d3041cebe417d162934223b46ea295c6d20e3
2021-02-07 13:26:56 +01:00
jenkins-bot
8a03ce6564 Merge "Set empty closures in DatabaseTest to fix php8 tests" 2021-01-31 03:24:34 +00:00
jenkins-bot
eb4b304ed8 Merge "Add missing @param and @return to documentation in tests" 2021-01-30 15:22:33 +00:00