An INSERT SELECT in MySQL/MariaDB is unsafe for replication if a column
is getting values from auto-increment, statement-based replication is in
use, and the default innodb_autoinc_lock_mode is set.
I9173f655 added checks to force non-native insertSelect for the
statement-based replication and innodb_autoinc_lock_mode != 2 case, but
determining whether a column is getting values from auto-increment is
too hard to do automatically there.
Instead, let's add a flag to let the caller hint that the query isn't
getting any auto-increment values. And use it in MysqlUpdater when
appropriate.
Bug: T160993
Change-Id: If70450a64aa3bcbf763c62838bb21306d124ae3d
It would be easy for a call to nonNativeInsertSelect() to generate an
INSERT that's too big for the database to actually process. Add batching
to try to avoid that.
Bug: T160993
Change-Id: I1de994208d95926f0d75c0d7cab7b5fe1dd565c3
Named locks are session-level constructs and this transaction agnostic.
Also make lockIsFree() a bit more consistent when the thread has the
lock itself.
Change-Id: Ief51196161bbc50c798740f3c738fd0e39880508
Done using the PhpStorm refactor->rename tool.
Also move "defaultBigSelects" declaration to DatabaseMysqlBase
as no other classes uses that.
Change-Id: I424a2d9815de3a5d4cca2522f3db23a5efe6b592
This was added in r19879 (3de6fd8c8) with no provided use case.
There also appears to be nothing exploiting this.
Change-Id: I1641d4ed16d612ab6f08ee1eeca1580af73888bf
* Update replace()/upsert() to combine the affected row
count for the non-native case
* Also make replace() atomic in the non-native case,
similar to how upsert() already works
Change-Id: I6c9bcba54eca6bcf4a93a9b230aaedf7f36aa877
* Make sure all unique keys specified have all their values
provided to avoid large bogus DELETEs. Do not ignore them
in such cases either, as that would cause inconsistencies
between the native and non-native case. Use an exception.
* Make ChangeTags caller clearer that the list of indexes
is not a list of fields for a single index. Also, avoid
mentioning indexes for values not defined in the new
records, as this causes errors or inconsistencies with
the native vs non-native case.
* This also fixes the "Undefined index: ts_log_id" error
when running unit tests on postgres.
Change-Id: I30263df22066bd6d4836202b1bcad5d1aa1e7383
Follows-up 0ff2b7a776 (T120333) which removed support for the
deprecated php 'mysql' extension.
It surprised me that the 'DatabaseMysql' entry could safely be removed
from $classAliases. This was because the key is not direct input,
but based on $canonicalDBTypes, which maps mysql to mysqli.
Take this further by removing the indirection between type, driver
and class name.
* Map built-in db types directly to their class and driver.
* Change fallback to assign $class directly from $dbType, without
indirection of a $driver variable.
* Remove unused $classAliases.
Change-Id: I1200b07f66f23624410d848b6f382cb2dafa5c59
This PHP extension was deprecated in PHP 5.5 and removed in PHP 7.0.
The newer MySQL driver 'mysqli' is the default driver since MediaWiki
1.22 and 'mysql' was deprecated in MediaWiki 1.30.
Bug: T120333
Change-Id: Icff7a63ab47ae1483de87ba6486945974699362b
This introduces MediaWikiTestCase::getSchemaOverrides, which can be overwritten
to return information about which tables are going to be altered, and which SQL
files should be used to set up the target schema. This allows tests for a class
that interacts with the database can have a subclass for each supported database
schema.
NOTE: this has only been tested with MySQL.
Bug: T180705
Change-Id: I7a4071072d802a82ecf7d16fbf8882ff8c79287f
Discovered while debugging I7a4071072d802a82ecf7d16fbf8882ff8c79287f
DatabaseIntegration test alters the delimiter for the database instance
and it never gets set back.
Simply set the delimiter back to a default value at the start of
sourceSteam (for each new file pointer) which could have a new
delimiter.
Change-Id: Ib829b1c8d87271526fa6826910d40c304c778b32
phpdbg is a gdb-style debugger for PHP that is run from the command
line. However, it has a different PHP_SAPI value, so it was impossible
to run maintenance scripts with it (until now).
To avoid having to check both PHP_SAPI values in a bunch of places,
introduce wfIsCLI() to easily check whether running from the
command-line or not.
We're (CI team) interested in generating code coverage with phpdbg
instead of xdebug, hence this patch.
Bug: T184043
Change-Id: Id1f994ca146d7858cd8bb6ab6cdbb7718ff524fb
After 36f4daf32c was merged, Aaron pointed out on Gerrit that:
Transaction can be used on replicas for REPEATABLE-READ mode, so
queries in that transaction see the same version of the DB. Those
do not involve writes and things would probably break sometimes
without it (like reads following foreign keys and reference in
separate queries).
So allow no-write connections to call begin() and commit(), and only
throw an exception when a write query is attempted.
Bug: T183265
Change-Id: I138677a714df13ead08061c3e9ef794c3befbe9d
In the off chance something called this, it would break all
sorts of code that expects that either query result functions
either succeed or throw an error.
Callers are not expected to have to check if the result of
a query is meaningful or false due to an error.
Change-Id: I0b4fe1403f55a399ffd40817ed12f857087d6f83
The SQL standard supports parenthesized joins like
a JOIN (b JOIN c ON (...)) ON (...)
But it doesn't support parenthesizing a single table name, i.e. a
one-table "join", like
a JOIN (b) ON (...)
Detect the degenerate single-table case and omit the parentheses.
Bug: T181674
Change-Id: I82cacd80465092aa67ff19bdcfd6682001bf12ab
Don't include the trace in the actual message, that keeps us from
grouping these messages by type. Instead inject the trace as an
extra parameter we'll pick up in logstash
Change-Id: I48184c1af2560827ef50baff4fc0443f00697504
SQL supports parentheses for grouping in the FROM clause.[1] This is
useful when you want to left-join against a join of other tables.
For example, say you have tables 'a', 'b', and 'c'. You want all rows
from 'a', along with rows from 'b' + 'c' only where both of those
exist.
SELECT * FROM a LEFT JOIN b ON (a_b = b_id) JOIN c ON (b_c = c_id)
doesn't work, it'll only give you the rows where 'c' exists.
SELECT * FROM a LEFT JOIN b ON (a_b = b_id) LEFT JOIN c ON (b_c = c_id)
doesn't work either, it'll give you rows from 'b' without a
corresponding row in 'c'. What you need to do is
SELECT * FROM a LEFT JOIN (b JOIN c ON (b_c = c_id)) ON (a_b = b_id)
This patch implements this by extending the syntax for the $table
parameter to IDatabase::select(). When passing an array of tables, if a
value in the array is itself an array that is interpreted as a request
for a parenthesized join. To produce the example above, you'd do
something like
$db->select(
[ 'a', 'nest' => [ 'b', 'c' ] ],
'*',
[],
__METHOD__,
[],
[
'c' => [ 'JOIN', 'b_c = c_id ],
'nest' => [ 'LEFT JOIN', 'a_b = b_id' ],
]
);
[1]: In standards as far back as SQL-1992 (I couldn't find an earlier
version), and it seems to be supported by at least MySQL 5.6, MariaDB
10.1.28, PostgreSQL 9.3, PostgreSQL 10.0, Oracle 11g R2, SQLite 3.20.1,
and MSSQL 2014 (from local testing and sqlfiddle.com).
Change-Id: I1e0a77381e06d885650a94f53847fb82f01c2694
This is consistent with query error suppression, let the
listener callbacks have a chance to run. This also makes
LoadBalancer::rollbackMasterChanges move on to the other
connections.
Change-Id: Ic8b9dd4b868bfe69b04fb20f7be5fce11d864fc4
I0e6a9e6d overlooked the special handling PG needs (prior to 9.5 anyway)
to properly emulate MySQL's IGNORE option when delegating to the parent
implementation.
For now, then, don't use the native implementation in PG when IGNORE is
specified. Instead, fall back to the non-native implementation that does
a select() then an insert() where PG can handle the IGNORE properly.
In the future we might use the ON CONFLICT DO NOTHING clause added in PG
9.5 to be able to do native insertSelect() with IGNORE (and to better
handle multi-row insert() with IGNORE, and we could use the related ON
CONFLICT DO UPDATE to implement upsert()). All that is left for a future
patch.
Change-Id: I7987d59580543de03d5c6a5ed7fa3ce551ac12f3
Throw DBQueryTimeoutError if a database query error is detected
to be a timeout.
Only DatabaseMysqlBase has been updated here.
This is a subclass of DBQueryError, so existing catch'es will work.
Bug: T175775
Change-Id: I4749dc33ad530d9b22504f02106b1ca49e8eb167
For some varargs a variable name is added with suffix ,... as seen for
many other varargs
Some @param are swapped, because there are in the wrong order
Enable Sniff MediaWiki.Commenting.FunctionComment.ParamNameNoMatch
Change-Id: I60fec6025bce824d5c67563ab7b65ad6cd628ad8
This class is the driver for MySQL databases corresponding to the PHP
extension 'mysql'. This PHP extension was deprecated in PHP 5.5 and
removed in PHP 7.0. It was replaced by the PHP extension 'mysqli',
which is supported and the default MySQL driver since MediaWiki 1.22.
Bug: T120333
Change-Id: Id1478f73e348cdd1ad1f560b1d8ddefe56c52767
If no writes started a transaction yet, the callback would run
but not commit (by design, joining the request round). Later
writes will then pile on top of it.
The point of this method is to avoid such cases, so this edge
case has been fixed.
Change-Id: I9b44b19261d679de4aff6e44a9cfeb4f684ce02e
Constructs a query for the union of permutations of a set of fields, for
use in situations where the database otherwise makes poor plans due to
inability to use indexes effectively (e.g. T149077 and T168010).
Change-Id: I20980dcada664486c09198b8c45896620bd83e81
selectField() and selectFieldValues() are trivial, they just need to
pass it through to select(). In fact, selectFieldValues() was already
doing it, just no one ever updated IDatabase.
insertSelect() is a little more work. nativeInsertSelect() was
originally written as largely a copy-paste of select() and has since
gotten well out of sync. Now that we have selectSQLText(), we should be
able to just use that. DatabasePostgres's implementation can wrap the
parent implementation instead of being another copy-paste, but
DatabaseOracle seems to still need to be special.
Change-Id: I0e6a9e6daa510639d3212641606047a5db96c500
Will add @inheritdoc in a follow-up commit for functions
documented in the parent classes.
Part of 2017 MediaWiki Documentation Day
Change-Id: I002a1f6451940ecbcacea7b3ca2fc6ad0f4eba47
