Commit graph

1596 commits

Author SHA1 Message Date
Amir Sarabadani
0f32376bf6 rdbms: Migrate buildGroupConcatField and buildSelectSubquery to platform
Bug: T307616
Change-Id: I4ebc7c219b51456c4eac435210fce46277e94ed5
2022-06-27 12:18:09 +02:00
Amir Sarabadani
d8f39205fc rdbms: Move methods delegated to platform to one central place
Making them sitting next to each other would make the class more
readable and also discouraging direct use of these methods outside of
rdbms (also implicitly by putting them at the bottom of the class).

These methods should not be stable to override anymore, their
counter-part in platform should be (for extensions planning to add
support for more rdbms types)

Bug: T307616
Change-Id: I5d6c5775f2a7378a73c69618ed09acb1b31374a8
2022-06-27 11:56:39 +02:00
Matěj Suchánek
1865180ae7 Do minor code cleanup
Remove dead code and fix typos. Should cause no change in behavior.

Change-Id: I5d293b842bc93a28b8bcd799a31b5e6e30fe692e
2022-06-24 13:52:42 +02:00
Aaron Schulz
3ce1067594 rdbms: add Database::flushSession() $mode safety parameter
This matches the begin(), commit(), and rollback() methods.

Change-Id: I3e66582052b2f42b2260bcf02b102caea8986f0a
2022-06-16 19:54:35 +00:00
Aryeh Gregor
c435212260 Get rid of warnings on PHP 8.1
This is mostly about adding return types to methods that implement PHP
interfaces, and not passing null to core functions that want a string.
After this patch, and an update to return types in RemexHtml,
tests/phpunit/integration/ has no more errors than in PHP 8.0.

Bug: T289879
Bug: T289926
Change-Id: Ia424f5cc897070f4188ae126b5bf6a1f552db0e1
2022-06-13 04:42:20 -04:00
jenkins-bot
69c012b938 Merge "rdbms: Move cpStash message to ChronologyProtector.php" 2022-06-09 21:13:21 +00:00
jenkins-bot
e5ea337b25 Merge "Database::queryMulti, buildExcludedValue cleanup" 2022-06-09 19:02:43 +00:00
jenkins-bot
264664abeb Merge "rdbms: allow merging IDatabase::upsert() rows with current ones" 2022-06-09 03:13:04 +00:00
Tim Starling
077dc706c8 Database::queryMulti, buildExcludedValue cleanup
* Move queryMulti() to IDatabase
* Provide default parameters
* Unpack complex expression in do/while condition
* Fix some comments
* In buildExcludedValue, remove rambling about bad alternatives.

Change-Id: I675fcbd397492e7efdb023debc215c30e21c7ffe
2022-06-09 12:54:34 +10:00
jenkins-bot
dadf6b8edd Merge "rdbms: Add multi-statement query support to Database" 2022-06-09 02:38:51 +00:00
Aaron Schulz
fdfe022ccf rdbms: allow merging IDatabase::upsert() rows with current ones
Rewrite the queries so that the column values of conflicted
proposed rows can be referenced in the SET expressions used to
update existing rows. Add IDatabase::buildExcludedValue() to
derive such references.

Make upsert() only use the first key in the unique key list as
the one to use for finding existing rows and merging them. Any
other key will just cause duplicate key errors/rollback upon
row collisions. For example, suppose a `user` table has unique
keys on both UUID and email address. It would be undesirable
for upsert() to clobber an existing `user` row just because
someone tried to make an account with the same email address.

Using upsert() makes sense when the following hold:
- The table has a single unique key that conveys "identity".
  AUTOINCREMENT and UUID columns are particularly good here.
  If row A existed at time t1, row B existed at time t2, and
  both rows have matching values for this "identity key", then
  they are conceptually the "same" row. It does not matter if
  other columns differ or if the row was deleted and recreated.
- Any other unique keys are just business constraints unrelated
  to the concept of identity. For example, an `accounts` table
  might have ID, name, and email address each as unique keys.
- For each proposed row, upsert() looks for an existing row
  that conflicts along the identity key. If there is one, then
  it gets updated, otherwise, the proposed row is inserted.
  Other unique key collisions result in operation rollback.

Bug: T113916
Change-Id: Iddd9f120ee966c76b3acb35e62ea14ec4c6f925d
2022-06-09 11:45:46 +10:00
Aaron Schulz
da9e4b52a5 rdbms: Add multi-statement query support to Database
Add Database::queryMulti(), which will execute an array of
queries as a batch with minimal roundtrips.

SQLite fallbacks to looping through each statement and
invoking doQuery().

Add QueryStatus class to reduce complexity in Database.

Rewrite doQuery() as doSingleStatementQuery().

Change-Id: I3d51083e36ab06fcc1d94558e51b38e106f71bb9
2022-06-09 11:45:38 +10:00
Aaron Schulz
f5e165317a rdbms: move mysql isQuotedIdentifier() override to SQLPlatform
Follow-up to dd9b44b1

Bug: T310214
Change-Id: I99b817b3d50ffcdf563561544560ac1e43f9d805
2022-06-08 14:59:34 -07:00
Aaron Schulz
d67c91a36a rdbms: make DatabaseSqlite::doSelectDomain() handle table prefix changes
Change-Id: I17dd8e6398de983b6c87163eb1d20e56097c4f07
2022-06-06 15:08:28 -07:00
jenkins-bot
69a9ffd30a Merge "rdbms: support mmap_size pragma for sqlite" 2022-06-05 19:39:06 +00:00
jenkins-bot
2198eec936 Merge "rdbms: Move selectSQLText to SQLPlatform" 2022-06-02 19:54:08 +00:00
Amir Sarabadani
dd9b44b167 rdbms: Move selectSQLText to SQLPlatform
Bug: T307616
Change-Id: I5218bca6633ad4d2704267e0c40926ca171fb1f7
2022-06-02 21:35:04 +02:00
Timo Tijhof
ac09cb7510 rdbms: Move cpStash message to ChronologyProtector.php
* It's now only logged when we actually read or write to CP.
  This makes it less confusing to see the message in cases where
  CP isn't or can't be used, as it would claim to use it when really
  it isn't/wouldn't.

* This was adding some overhead as it was the only non-wfDebug()
  thing I could find on all requests. Mainly noticable locally with
  mw-docker and writing to mw-www-debug.log. This helps a bit
  with T85805 and making dozens of load.php requests in debug mode
  less slow.

* It duplicated the channel name outside its class instead of
  using DI.

Change-Id: I0947b98d285f26ab5af8dd58a6dce2e3e0525d44
2022-06-02 01:41:13 +00:00
jenkins-bot
e451af4d0a Merge "rdbms: make approvePrimaryChanges() trigger ping() in more cases" 2022-06-01 22:46:42 +00:00
Aaron Schulz
b6b53a0412 rdbms: make approvePrimaryChanges() trigger ping() in more cases
If there are row or named locks held, ping() should be called to
make sure that the locks where not lost. If they were lost, then
no transactions should be committed.

Slightly optimize isWriteQuery() by checking QUERY_CHANGE_TRX.

Bug: T307133
Change-Id: I8f97e1e3a4aa72c31ac2d642511af8dadf070b07
2022-06-01 20:45:36 +00:00
Aaron Schulz
0a4569a7e7 rdbms: support mmap_size pragma for sqlite
Change-Id: I517f4be9a38a1782484d4040b1dc0c3081db3a0d
2022-05-31 23:10:14 +00:00
Aaron Schulz
496332e4ce rdbms: avoid DBReadOnlyRoleError in Database::doFlushSession()
Fix and simplify the flags to executeQuery(). The QUERY_CHANGE_ROWS flag
caused the DB_REPLICA role check to be violated.

Remove the UNLOCK TABLES query since lockTables()/unlockTables() have
since been removed.

Bug: T306632
Change-Id: I8fcf940028abd70c0d66bec1780308d3c2367b61
2022-05-31 23:09:41 +00:00
jenkins-bot
c60f455dec Merge "installer,rdbms: Remove hints to Oracle and Microsoft SQL Server" 2022-05-28 00:06:42 +00:00
Umherirrender
e3acf723de installer,rdbms: Remove hints to Oracle and Microsoft SQL Server
Fix i18n messages still refers to both rdbms

Bug: T230418
Change-Id: I830c192331786d0dcf58eaf09e34f49fa4c88a06
2022-05-27 23:47:42 +00:00
Tim Starling
472d60d249 rdbms: Deprecate DBO_SSL
Using a bitfield breaks server array template merging in LBFactoryMulti.

The only nice thing about flags compared to boolean constructor
parameters is the existence of setFlag(), clearFlag() and
restoreFlags(). But that is not useful for parameters that are used
during connect.

Bug: T134809
Change-Id: I088ef4a0e4c11a2cfe17415b325b455164004c1e
2022-05-27 14:57:31 +00:00
jenkins-bot
06f1de611b Merge "rdbms: Move handling index and table aliases to SQLPlatform" 2022-05-25 22:29:34 +00:00
Aaron Schulz
b815055256 rdbms: merge sql_mode logic with other connection variables
Change-Id: I62f570466452992f63cb4835c70c915f874bd8ef
2022-05-25 18:43:31 +00:00
Amir Sarabadani
80ddbf1032 rdbms: Move handling index and table aliases to SQLPlatform
Next step in moving selectSQLText() to SQLPlatform

Bug: T307616
Change-Id: Idd136c17a04e12b1fe91c81ce90677ec42ca0882
2022-05-23 12:08:08 +02:00
Amir Sarabadani
d83cbce70e rdbms: Move four more functions from Database to SQLPlatform
Bug: T307616
Change-Id: I0ff6318bcbca79830f013a16597d04d1e97ccf6d
2022-05-20 17:06:49 +02:00
jenkins-bot
f8117f9165 Merge "rdbms: fix Postgres return type for lastErrno()" 2022-05-18 08:38:10 +00:00
Aaron Schulz
92fb86f0dc rdbms: update various IDatabase comments
Change-Id: Icabb4e49d812bb7c629bbeeac8ffb13c0a2c422a
2022-05-18 00:22:52 +00:00
jenkins-bot
434c333d9b Merge "rdbms: Prepare for moving Database::selectSQLText to SQLPlatform" 2022-05-17 02:20:37 +00:00
jenkins-bot
a04d835b14 Merge "rdbms: Normalize DBTransactionSizeError message" 2022-05-13 20:00:20 +00:00
Amir Sarabadani
8c833750d0 rdbms: Prepare for moving Database::selectSQLText to SQLPlatform
This moves a lot of methods used in selectSQLText method so the move
would be easier. I tried moving it but the patch basically exploded.

Bug: T307616
Change-Id: Ib8c68b4506055b63c769926d6d7cfd1069d951f4
2022-05-12 07:59:22 +02:00
Amir Sarabadani
a294e715a4 rdbms: Replace getConnection with getLazyConnectionRef
This would simplify the code for its users a lot.

Bug: T255493
Depends-On: I6ab4440a13f4682dd3a6e9cfc6c629222fbeab8a
Change-Id: I6e7544763bde56fc1e19de0358b71ba984a978ca
2022-05-12 07:43:03 +02:00
jenkins-bot
51cad7cfa9 Merge "rdbms: Move out more functions from Database to SQLPlatform" 2022-05-11 17:21:57 +00:00
Amir Sarabadani
332b530e8a rdbms: Move out more functions from Database to SQLPlatform
Bug: T307616
Depends-On: Id1f477f5ca4c837cf50ee5ef450873735b732a42
Change-Id: I63308df985b9d596110718450997565de2b06aca
2022-05-11 16:51:53 +00:00
Aaron Schulz
fbdd56da35 rdbms: fix Postgres return type for lastErrno()
The error codes are strings (e.g. "55P03"). Casting to an integer
breaks comparisons by callers (all internal) and makes error strings
confusing.

Use "00000" instead of 0 to reduce this confusion (does not affect
current callers).

Follow-up to 478c4892.

Change-Id: I3dd8929f4c8e604eef669710e684724977381c74
2022-05-10 17:42:56 -07:00
Matěj Suchánek
e47c441078 Fix many typos in comments
Found using IntelliJ's "Typo" code inspection.

Change-Id: I746220ebe6e1e39f6cb503390ec9053e6518cf16
2022-05-10 12:46:11 +00:00
jenkins-bot
4ece9d46fe Merge "rdbms: Remove db name from error message" 2022-05-06 02:37:21 +00:00
Amir Sarabadani
cf3a41a7c0 rdbms: Normalize DBTransactionSizeError message
The actual seconds vary, this makes the errors not getting aggregated in
logstash

Bug: T300194
Change-Id: Ibc6d6a3d8831b23a507f9175a4f9030975a1d5d9
2022-05-05 07:24:28 +02:00
Amir Sarabadani
2387b0c774 rdbms: Remove db name from error message
This is to allow us aggregate errors coming from different dbs.

Simply add the dbname as an attribute

Bug: T306634
Change-Id: I7c4609ececbf727336c174616d525d2df9637559
2022-05-05 07:23:23 +02:00
Amir Sarabadani
ca5b162c23 rdbms: Start using SQLPlatform and move more methods there
The first of many changes decoupling SQL building blocks from Database
class.

Bug: T299691
Depends-On: I5d1d5b9b875bced7bda234f45d6d22ed59db4871
Change-Id: I784e78361f5ee629d31c68629d669ee0ddddf929
2022-05-04 18:34:13 +00:00
Aaron Schulz
9a9f441dc5 rdbms: remove lockTables()/unlockTables()
Bug: T294969
Change-Id: I94a2b1916c3d798f412ad2b89d8c31e09439d6c7
2022-05-03 16:03:57 -07:00
jenkins-bot
4872dbbfdd Merge "Replace LoadBalancer/LBFactory callback iteration with generators" 2022-05-03 18:10:12 +00:00
Aryeh Gregor
4851f512b3 IResultWrapper::next() now returns void
As required by the Iterator interface. In PHP 8.1 we can suppress the
warning with #[\ReturnTypeWillChange], but it's wrong and should change
anyway.

Depends-On: I576109808755b054e7876556f244ee8eafd12f9d
Change-Id: Ibaa7b4c81d8f114783db45c9100200ba14547c8f
2022-05-02 10:43:51 +03:00
Tim Starling
8632b5920a Replace LoadBalancer/LBFactory callback iteration with generators
Callback style iteration made sense before generators existed, but
generators make for simpler code. The "call method" variants made
sense before closures existed but defeat static analysis.

So, in LBFactory:

* Add ILBFactory::getAllLBs()
* Deprecate ILBFactory::forEachLB()
* Remove LBFactory::forEachLBCallMethod(), was protected.
* Add LBFactory::getLBsForOwner(), which is protected and has the
  internal interface in @return. Adding a new abstract method breaks
  Wikibase tests despite LBFactory not being stable to extend.
* Migrate callers. Generators allow you to return/break from the middle
  of the loop, which implies a little rearrangement for some callers.

In LoadBalancer, connections supposedly of type IDatabase were
routinely type-hinted as Database in closure parameters so that methods
could be called that were not in the interface. So it's convenient to
get rid of public iteration methods entirely in favour of private
methods returning Database[].

* Hard-deprecate ILoadBalancerForOwner::forEachOpenConnection()
  and replace it with a private generator method since nothing called
  it externally except for core tests.
* Hard-deprecate ILoadBalancerForOwner::forEachOpenPrimaryConnection()
  and replace it with a private generator. DeferredUpdates needed it for
  iterating over IDatabase::explicitTrxActive(), so add
  ILoadBalancer::explicitTrxActive() as a replacement.
* Replace private method LoadBalancer::forEachOpenReplicaConnection()
  with a generator.

Depends-On: If0b382231e27d6d1197fb7b6aef6ab50335df4e5
Change-Id: I64514e77b9bfe737be5b12e1d3c9c49976bb522f
2022-04-29 08:58:17 +10:00
Tim Starling
05701ffc40 rdbms: Remove instance ownership concept
Instance ownership is supposed to protect LoadBalancer and Database
against unauthorized calls to internal methods other than by the owning
LoadBalancer/LBFactory. This seems like unnecessary complexity. It was
introduced for T231443 and T217819, but the link was speculative and in
the end it didn't help to fix or isolate those bugs. Since then it has
caused a production error (T303885) and an intermittent CI failure
(T292239).

Instead, split the ILoadBalancer interface, introducing
ILoadBalancerForOwner, which contains the methods which are only safe
to call by LBFactory or by the caller of LBFactory::newMainLB(). This
allows phan to statically detect inappropriate calls to internal
methods.

Ownership was used for convenience for two things unrelated to its
original purpose:

* Suppressing calls to ScopedCallback::newScopedIgnoreUserAbort() when
  the caller has already called it. But nested calls are apparently
  harmless, so I just called it unconditionally.
* Suppressing exceptions from Database::close(). I extended the
  behaviour for owned instances to apply to all instances, so even
  unowned instances will no longer throw on close.

CodeSearch suggests nothing in extensions is calling these methods with
an owner parameter. One extension (Wikibase) overrides a method with an
owner parameter in a test mock class and so needs to be simultaneously
updated.

Depends-On: Ib03aba9d8f5f05b875a321d00b14483633a636a8
Change-Id: I27ba4973d24d759c88b3868c95e7db875801ca0c
2022-04-26 11:48:46 +10:00
jenkins-bot
48b613dc34 Merge "rdbms: fix stray parenthesis in DatabaseMysqlBase::doFlushSession()" 2022-04-25 13:24:46 +00:00
Aaron Schulz
bbfcbf9137 rdbms: fix stray parenthesis in DatabaseMysqlBase::doFlushSession()
Follows-up db36853718 from last month.

Change-Id: I7f3cbcbd118cecac2103090ca83769507c25d859
2022-04-25 13:05:55 +00:00