Commit graph

54 commits

Author SHA1 Message Date
Aaron Schulz
d395dfb039 rdbms: make selectRowCount() use $var argument to exclude NULLs
If the $var argument is provided, then it will make the resulting
count exclude rows where the value for that column is NULL.

Also add buildSelectSubquery() method and Subquery
wrapper class for use with select() for calculated tables.

Change-Id: I549d629af99afdf370602de095f7fba6d1546c37
2018-03-18 01:34:33 +00:00
Brad Jorsch
6261b4187a rdbms: Add $join_conds to IDatabase::estimateRowCount()
So queries with joins can be estimated.

Change-Id: I9163cf9005d2c2001a88bb102eb4142f0322b0df
2018-03-12 12:15:14 -04:00
Aaron Schulz
55ab84e17d rdbms: avoid strange uses of empty()
Change-Id: Id1a8d1aae72cdee48e43ddb3227cd697516411e0
2018-03-01 20:30:07 -08:00
Aaron Schulz
60e8d3e16e rdbms: remove "m" member prefix from various classes
Change-Id: Iade8e8f70bb1307b96683d979d7e3650f4107515
2018-02-17 03:58:46 +00:00
Aaron Schulz
c9ad7037ce rdbms: do not bother making DBO_TRX transactions in IDatabase::lock()
Named locks are session-level constructs and this transaction agnostic.
Also make lockIsFree() a bit more consistent when the thread has the
lock itself.

Change-Id: Ief51196161bbc50c798740f3c738fd0e39880508
2018-02-15 16:32:35 -08:00
Aaron Schulz
ec550d4823 rdbms: remove "m" prefix from Database fields
Done using the PhpStorm refactor->rename tool.

Also move "defaultBigSelects" declaration to DatabaseMysqlBase
as no other classes uses that.

Change-Id: I424a2d9815de3a5d4cca2522f3db23a5efe6b592
2018-02-15 23:29:34 +00:00
Reedy
39f0f919c5 Update suppressWarning()/restoreWarning() calls
Bug: T182273
Change-Id: I9e1b628fe5949ca54258424c2e45b2fb6d491d0f
2018-02-10 08:50:12 +00:00
Aaron Schulz
6237fd11b6 rdbms: make affectedRows() work more consistently
* Update replace()/upsert() to combine the affected row
  count for the non-native case
* Also make replace() atomic in the non-native case,
  similar to how upsert() already works

Change-Id: I6c9bcba54eca6bcf4a93a9b230aaedf7f36aa877
2018-01-30 20:02:07 -08:00
Brad Jorsch
4bd3e80e7a Database: Support parenthesized JOINs
SQL supports parentheses for grouping in the FROM clause.[1] This is
useful when you want to left-join against a join of other tables.

For example, say you have tables 'a', 'b', and 'c'. You want all rows
from 'a', along with rows from 'b' + 'c' only where both of those
exist.

 SELECT * FROM a LEFT JOIN b ON (a_b = b_id) JOIN c ON (b_c = c_id)

doesn't work, it'll only give you the rows where 'c' exists.

 SELECT * FROM a LEFT JOIN b ON (a_b = b_id) LEFT JOIN c ON (b_c = c_id)

doesn't work either, it'll give you rows from 'b' without a
corresponding row in 'c'. What you need to do is

 SELECT * FROM a LEFT JOIN (b JOIN c ON (b_c = c_id)) ON (a_b = b_id)

This patch implements this by extending the syntax for the $table
parameter to IDatabase::select(). When passing an array of tables, if a
value in the array is itself an array that is interpreted as a request
for a parenthesized join. To produce the example above, you'd do
something like

 $db->select(
     [ 'a', 'nest' => [ 'b', 'c' ] ],
     '*',
     [],
     __METHOD__,
     [],
     [
         'c' => [ 'JOIN', 'b_c = c_id ],
         'nest' => [ 'LEFT JOIN', 'a_b = b_id' ],
     ]
 );

[1]: In standards as far back as SQL-1992 (I couldn't find an earlier
 version), and it seems to be supported by at least MySQL 5.6, MariaDB
 10.1.28, PostgreSQL 9.3, PostgreSQL 10.0, Oracle 11g R2, SQLite 3.20.1,
 and MSSQL 2014 (from local testing and sqlfiddle.com).

Change-Id: I1e0a77381e06d885650a94f53847fb82f01c2694
2017-10-13 19:12:16 +00:00
Alexander Mashin
c428c2cf36 Ensure only strings are passed to mysql_real_escape_string()
Under some conditions (Semantic MediaWiki, Gadgets), an integer is
passed to DatabaseMysqli::mysqlRealEscapeString (). This integer is, in
turn, passed to mysqli::real_escape_string (), which needs a string.

Under HHVM 3.19.1 (at least) this type mismatch causes an exception.

A typecast should prevent it.

I repeated the patch in other DB drivers where I could find a function
that escaped strings for SQL.

Bug: T163646
Change-Id: I1b7820bc064dc79498cf9f17747f745990c526b7
2017-10-08 03:45:12 +00:00
Brad Jorsch
d98a666d38 Fix insertSelect() with IGNORE in PostgreSQL
I0e6a9e6d overlooked the special handling PG needs (prior to 9.5 anyway)
to properly emulate MySQL's IGNORE option when delegating to the parent
implementation.

For now, then, don't use the native implementation in PG when IGNORE is
specified. Instead, fall back to the non-native implementation that does
a select() then an insert() where PG can handle the IGNORE properly.

In the future we might use the ON CONFLICT DO NOTHING clause added in PG
9.5 to be able to do native insertSelect() with IGNORE (and to better
handle multi-row insert() with IGNORE, and we could use the related ON
CONFLICT DO UPDATE to implement upsert()). All that is left for a future
patch.

Change-Id: I7987d59580543de03d5c6a5ed7fa3ce551ac12f3
2017-09-26 01:32:27 +00:00
Umherirrender
f739a8f368 Improve some parameter docs
Add missing @return and @param to function docs and fixed some @param

Change-Id: I810727961057cfdcc274428b239af5975c57468d
2017-09-10 20:32:31 +02:00
Brad Jorsch
0a9c55bfd3 Deprecate IDatabase::nextSequenceValue()
It's often forgotten because MySQL and Sqlite don't use it, the only
users are PostgreSQL and Oracle. And when used, if inserts to multiple
tables are being done it's easy to get the ordering wrong.

This patch reimplements DatabasePostgres::insertId() in terms of PG's
lastval() function, and adds triggers to the Oracle schema to make it
work the same as the other databases.

Bug: T164900
Change-Id: Ib308190c52673a9266c8495a589ae644f9fbefce
2017-08-31 11:16:26 -04:00
Brad Jorsch
91b86399b1 Fix various PostgreSQL failures
* Fix schema for image_comment_temp.
* Provide values in CommentStoreTest::provideInsertRoundTrip() for
  columns where the PG schema doesn't have a default value but the MySQL
  schema does.
* Call nextSequenceValue() from CommentStoreTest::testInsertRoundTrip().
* Correctly handle $options being the string 'FOR UPDATE' in
  DatabasePostgres::selectSQLText()
* Correctly handle the initial table in DatabasePostgres::selectSQLText() FOR
  UPDATE mangling.
* Correctly handle aliases in DatabasePostgres::selectSQLText() FOR
  UPDATE mangling.

Tests in PG are still going to be broken thanks to the fact that
nextSequenceValue() and insertId() can't be separated by another
nextSequenceValue()/insertId() pair. That should be taken care of by
T164898/T164900.

Change-Id: Ia770a003ca9170ab8bcc1436d8afe30700e00ada
2017-08-30 17:18:29 -04:00
jenkins-bot
ddf3d27b10 Merge "Make DatabasePostgres::serverIsReadOnly() check default_transaction_read_only" 2017-06-15 21:04:22 +00:00
Aaron Schulz
1ac90c76f4 Make DatabasePostgres::serverIsReadOnly() check default_transaction_read_only
Change-Id: If6088c15271a827fedc6a585e070bab6c1acc371
2017-06-15 01:36:39 -07:00
Aaron Schulz
b7aaa8cdbe Set flush mode in DatabasePostgres::reportQueryError()
Bug: T167927
Change-Id: Ib02608cf33d369939a9e587cebfc736336a1e979
2017-06-15 01:25:17 -07:00
jenkins-bot
7f3d6713e7 Merge "Fix Postgres support" 2017-06-14 08:19:53 +00:00
Brad Jorsch
31cb62dd2d DB: Add join conditions to selectField, selectFieldValues, and insertSelect
selectField() and selectFieldValues() are trivial, they just need to
pass it through to select(). In fact, selectFieldValues() was already
doing it, just no one ever updated IDatabase.

insertSelect() is a little more work. nativeInsertSelect() was
originally written as largely a copy-paste of select() and has since
gotten well out of sync. Now that we have selectSQLText(), we should be
able to just use that. DatabasePostgres's implementation can wrap the
parent implementation instead of being another copy-paste, but
DatabaseOracle seems to still need to be special.

Change-Id: I0e6a9e6daa510639d3212641606047a5db96c500
2017-06-14 05:07:42 +00:00
Aaron Schulz
6fb5844fec Fix TextPassDumperDatabaseTest::testPrefetchPlain postgres failures
The insertId() method was returning a string, which caused the
returnValueMap not to trigger due to int/string mismatches.

Also add sanity integer cast to WikiPage::insertOn().

Added a few more type docs.

Bug: T75174
Change-Id: Id1090f3e3d0481272a3d13c3af8f2588f06dc912
2017-05-26 21:43:17 +00:00
Aaron Schulz
44577c3962 Make sure unique keys carry over to postgres table duplicates
This fixes BotPasswordTest failures.

Change-Id: I81003187af0c57020f9fc4cbcd2606e71907a903
2017-05-16 19:20:42 -07:00
Aaron Schulz
6793b60eca Fix some postgres test failures
Bug: T75174
Change-Id: If9812ae301f0af84fa012e2e980b92036c1b29b2
2017-05-10 01:59:47 +00:00
mwjames
b5611ce6fa Fix Postgres support
See https://phabricator.wikimedia.org/T149454#3217259

This restores a line mistakenly deleted in
eef8fc45f3.

Bug: T149454
Change-Id: If6463904bb5269a7ccab5ddd69f69dba8104bc49
2017-05-08 21:06:55 +00:00
Aaron Schulz
dd359741cc Move DB errors to Rdbms namespace
Change-Id: I463bd86123501abc68fdb78b4cda6110f7af2549
2017-04-15 10:47:41 -07:00
Aaron Schulz
d735dc562d Move Database and subclasses to Rdbms namespace
Change-Id: I52bef87512f9ddd155d1f4cc0052f6b7a0db5b42
2017-04-12 10:43:57 -07:00
Aaron Schulz
1a1c43bdff Database: clean up lockTables() and add postgres support
A new method is now available to check whether session scope
locks are supported, which callers typically want when using lock().
Its usage can avoid deadlock prone and expensive row-level locks for
some maintenance tasks.

For Postgres, table locks are tied to the transaction. Trigger
startAtomic() in lockTables() and endAtomic() in unlockTables() to
assure that a transaction is present.

Also remove LOW_PRIORITY feature, which is ignored by mysql.

Change-Id: I499061bcc2763afb1ff4a43319064eed4ba3a8fe
2017-04-07 16:28:36 -07:00
Aaron Schulz
e01fd44388 Move ResultWrapper subclasses to Rdbms
Change-Id: I6f3f0e85e268b24c57c537aa6ad8016e0b4cdddb
2017-03-03 00:44:41 +00:00
Kunal Mehta
a57b64436c Use wikimedia/timestamp
Bug: T100924
Depends-On: I0a067367cda6885fa45631ed7c18799d653dc9bf
Change-Id: I69ba64e364df8af089c1c918cdf32f99454e693a
2017-02-28 21:15:38 -08:00
jenkins-bot
b444c2a3df Merge "includes/libs: Replace implicit Bugzilla bug numbers with Phab ones" 2017-02-28 00:50:31 +00:00
Bartosz Dziewoński
0da3d2ff2b Database: Remove weird commented-out block
Commented out in 033b6b9646 (r20329).
After ten years, I think it's safe to bet this won't ever be getting
un-commented.

Change-Id: Ibb1f3e2969b2d81f6f2a17fff57e9b05cc17d58b
2017-02-25 22:55:45 +01:00
James D. Forrester
b715f432f4 includes/libs: Replace implicit Bugzilla bug numbers with Phab ones
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.

Skipping jsminplus.php as those bug numbers aren't Wikimedia's, nor obviously
someone else's.

Change-Id: I9a2210e17852ee56f11282b980ac66d8c7a95671
2017-02-21 18:13:08 +00:00
Aaron Schulz
8a5d8c0c71 Move Field classes to Rdbms namespace
Update core callers and leave a class alias to \Field.

Change-Id: I4908282301d5de2a20baafe510557bd2c3867de5
2017-02-16 01:14:37 +00:00
Aaron Schulz
019fa7cb9c Move Blob class to Rdbms namespaces
Leave \Blob as an alias. Callers can now use the Rdbms\Blob class
for "extends"/"new" and the Rdbms\IBlob interface for type hints.

Change-Id: I983b76f181ac60c1eb92c350cd27ad77ec90a192
2017-02-13 21:07:00 +00:00
Brad Jorsch
fb3ae6fbe3 Replace use of &$this
Use of &$this doesn't work in PHP 7.1. For callbacks to methods like
array_map() it's completely unnecessary, while for hooks we still need
to pass a reference and so we need to copy $this into a local variable.

Bug: T153505
Change-Id: I8bbb26e248cd6f213fd0e7460d6d6935a3f9e468
2017-01-31 23:01:54 -05:00
Kevin Israel
cc495efff4 Database: Avoid internal use of ignoreErrors()
* Use $tempIgnore in DatabaseBase::tableExists() instead of ignoreErrors().
* DatabasePostgres::reportQueryError(), since 419221e4a7, calls
  ignoreErrors() before and after rollback(). This is probably not effective
  as a recursion guard (reportQueryError() would be called regardless), and
  for suppressing any log entry or exception, it is redundant (doRollback()
  already sets $tempIgnore when calling query()).
* ignoreErrors(), now unused, can be removed in a separate change set.

Change-Id: Iee768abf996ece213146ab8f64a15a2b407a2e5e
2016-12-27 05:20:30 -05:00
Aaron Schulz
f5dbb60c6e Depedency inject remappedTableName() map via MWLBFactory
Change-Id: I71d3481fc624f665bac81725026647bbb84ae608
2016-11-09 21:35:57 -08:00
jenkins-bot
69ae945e8d Merge "Update weblinks in comments from HTTP to HTTPS" 2016-11-08 21:32:00 +00:00
Fomafix
202f695f67 Update weblinks in comments from HTTP to HTTPS
Use HTTPS instead of HTTP where the HTTP link is a redirect to the HTTPS link.

Also update some defect links.

Change-Id: Ic3a5eac910d098ed5c2a21e9f47c9b6ee06b2643
2016-11-07 15:24:46 +01:00
Paladox
995d44ccde Make determineCoreSchema a public function again in DatabasePostgres
It was changed to a private function today, but it is called in
PostgresInstaller so it is causing this error

PHP Fatal error:  Call to private method
DatabasePostgres::determineCoreSchema() from context 'PostgresInstaller'
in /home/travis/build/paladox/mw/includes/installer/PostgresInstaller.php
on line 515

Caused by I2a067ca89a03f9ebf3f70a4f36ddae92e5b1e468

Change-Id: Ifac5a5c7dfeceeb8c7827d6d70e757633df77ca5
2016-10-29 22:13:14 +00:00
Aaron Schulz
3cc48cb058 Various small cleanups to DatabasePostgres
* Add missing method visibilites
* Removed redundant doc blocks
* Use empty string for mSchema for consistency with
  the base class

Change-Id: I2a067ca89a03f9ebf3f70a4f36ddae92e5b1e468
2016-10-28 22:14:26 -07:00
jenkins-bot
31a3a94d20 Merge "Postgres updater fixes to make update.php able to run" 2016-10-29 01:31:01 +00:00
Aaron Schulz
eef8fc45f3 Postgres updater fixes to make update.php able to run
* Remove redundant schema prefix from relname=x query. The
  schema filtering is already done via the JOIN. The relname
  portion is just the table name not <schema>.<table name>.
* Avoid explicit table schema qualification and rely on the
  search path, as MW 1.27 did. Previously it only used the
  global $wgDBschema var to pass to determineCoreSchema()
  instead of keeping it in mSchema.
* Clean up some code duplication in Database::tableName() and
  make the code comments clearer.
* Make DatabasePostgres::tableName() use parent::tableName()
  instead of a method that just wraps this method. The intent
  seems clearer this way.
* Remove unused return value in
  PostgresUpdater::rebuildTextSearch().

Bug: T148628
Change-Id: Id11d9576b7c2fdad22ff7f90727c12997217a632
2016-10-28 17:13:56 -07:00
Aaron Schulz
3e4b7663ec Add short-circuit to DatabasePostgres::schemaExists()
Change-Id: I5221f7d937be1e87689df5a21fd64e244dbd4c2a
2016-10-26 16:07:12 -07:00
Aaron Schulz
d1215eb7a5 Small DB classes IDEA warning fixes and cleanups
Change-Id: I65b5f452a364b329bb671540f4115dbc4f40eebd
2016-10-23 22:03:37 -07:00
Aaron Schulz
fda4d46fc4 Clean up postgres connection handling
* Remove non-connection magic case when no DB $user
  is given. This was removed from the base class.
* Use PGSQL_CONNECT_FORCE_NEW to let LoadBalancer
  handle connection reuse. This makes it work like
  the mysql classes.
* Make postgres connection error messages actually
  be useful by using the PHP error when possible.
  This makes it clear if the problem is authentication
  or something else and so on.

Change-Id: I3fd76c1e2db8d6008074f5347b201554579b549a
2016-10-19 01:08:49 +00:00
Aaron Schulz
66b0ad56df Postgres installation fixes
* Make isTransactableQuery() exclude CREATE/ALTER.
  Starting transactions for schema changes like this can cause
  errors as it is not supported for MySQL and some Postgres
  operations. Note that temporary tables are session-level,
  so they are not effected by this change.
* Clean up the transaction logic in determineCoreSchema()
  so a transaction is not left dangling.
* Fix broken getSchemaPath() call in PostgresInstaller.
* Avoid warnings in DatabasePostgres::closeConnection() if
  mConn is already unset.
* Commit master changes in doMaintenance.php before running
  deferred updates, just as MediaWiki.php does.
* Change E_WARNING to E_USER_WARNING to avoid notices in the
  default /rdbms error handlers.
* Also avoid trying to rollback in MWExceptionHandler if the
  LBFactory service is disabled, which just results in an error.

Bug: T147599
Change-Id: I64ccab7f9b74f60309ba0c9a8ce68337c42ffb0f
2016-10-17 15:06:38 -07:00
Kunal Mehta
090d0267da Use wikimedia/wait-condition-loop
Since the WaitConditionLoop class was first introduced in 1.28 (current
master), no back-compat alias is added.

Bug: T146256
Depends-On: Ia84774d83da79fea1e167fe065c69549981f753b
Change-Id: Ibd4f15c87105b8caccbd1f661b74b6efa012b77f
2016-10-01 00:25:43 -07:00
Aaron Schulz
0bdab07814 Move DatabaseBase to a class alias for type-hints
Change-Id: Ib6bbbd751358cbc2f2d2767c80721d3d8f142c0d
2016-09-29 06:09:03 +00:00
Aaron Schulz
63ce63820e Add DBO_* class constants and defines.php alias the class constants
Change-Id: If13b23ef849d4cf4c711b0ec2bf2e8a795f90738
2016-09-23 19:58:44 +00:00
Aaron Schulz
00d677347e Generalize core schema docs in DatabasePostgres
Change-Id: I52e64867c1533c50406623d5bc121c1e4da2459a
2016-09-22 21:41:19 +00:00