Commit graph

390 commits

Author SHA1 Message Date
Timo Tijhof
26c9849330 docs: Fix 'dependant' typos
The intended word in all these cases was the adjective "dependent".

Whilst the "dependant" does exist, it is a noun and generally
refers to a person. The word is rare used in general, but
especially so in a technology context.

Change-Id: Ic7e2d2ea6a566f4139ff1fdb77f38b0e962ccd9c
2021-02-18 16:59:20 +00:00
Umherirrender
8de3b7d324 Use static closures where safe to use
This is micro-optimization of closure code to avoid binding the closure
to $this where it is not needed.

Created by I25a17fb22b6b669e817317a0f45051ae9c608208

Change-Id: I0ffc6200f6c6693d78a3151cb8cea7dce7c21653
2021-02-11 00:13:52 +00:00
Thiemo Kreuz
50f8929efa Fix documentation of IDatabase::selectField()
I found code in an extension that actually does this and
provides an array of field names (obviously with a single
field name only). This might be an "unintentional feature".
Still I believe it's a good idea to support it, because all
other methods support it as well. This makes selectField less
of an outlier.

Change-Id: Ie63b06896b59dcaa629f720f98f28943674dc8d7
2021-02-05 08:01:44 +00:00
Umherirrender
62002cdcf1 build: Update mediawiki/mediawiki-codesniffer to 35.0.0
Change-Id: Idb413be4b8cba8611afdc022af59810ce1a4531e
2021-01-31 13:34:38 +00:00
Timo Tijhof
6404da3d39 rdbms: Remove outdated MySQL 4 references and fix doc URLs
- Remove or clarify ancient comments referencing MySQL 4.

- Oracle can't afford to host small HTML files for longer than two
  years, as such they make a point to break their URLs after a while.
  Update our references to the last 5.x release for now, where the same
  information resides (instead 404 or redirect to MySQL 8 docs).

Bug: T34217
Change-Id: I322cc8936f96c2545c63e7c9c1f5fa241e2b8c18
2021-01-30 22:47:33 +00:00
Ammarpad
10e58e2637 rbdms: Improve signature of Database::getCacheSetOptions
One of the passed args is allowed to be null, but this null will
just be ignored and does not seems to be useful here so this is
now deprecated as suggested by Krinle at Id837e81

Now the function takes variadic args, which may still contain null
for sometime (owing to the method being public api).  In 1.37+, the
variadic parameter can be typehinted, and then the check for
instanceof would no longer be necessary.

Change-Id: I3863c3981470c1dbc11e417bc2252339bdee391f
2021-01-18 07:07:52 +01:00
Aaron Schulz
f5dea84dfa rdbms: split out LoadBalancer::fieldHasBit to reduce code duplication
Also clean up Database::fieldHasBit parameter names

Change-Id: I9e94d7fff87e98e78e9a62672d4e81b812fd9a79
2021-01-11 19:09:50 -08:00
DannyS712
7cc3d0c240 Minor cleanup in Database.php, should be a no-op
- Some whitespace cleanup

- Use early returns in ::getClass()

- Declare $rtt in ::pingAndCalculateLastTrxApplyTime()

- Make error messages for selecting the '*' $var field
  when not allowed consistent

- Flip order of $first handling in ::makeList()

Change-Id: Ibda4b72157986b0c7e6e03fb9f49836bc9b018c9
2021-01-06 06:08:20 +00:00
Ammarpad
0a4cd7c671 Restore use of func_get_args() in getCacheSetOptions()
The documentation says this can be used "for several DBs" but
because there are only two real paramaters and the gotcha  explained
at I7d4804a, I unintentionally limited it to work for "only two"
databases while fixing a different issue.

I am not sure wether there are callers calling with 5, 10 or 20
database objects, but PHP will never raise any warning even if
there are, the remaining args will just be silently discarded.

This restores the behavior so it can work "for several DBs", as
it used to.

Follow-up: I7d4804a715063d95827fcec6d14a3ae059b234c3
Change-Id: Id837e813cda19b86dcd14ec36b520f8465e5855d
2021-01-04 07:47:14 +00:00
jenkins-bot
99a6d07f90 Merge "Prevent calling database method on null" 2021-01-03 04:20:25 +00:00
Ammarpad
9b887ac1cc Prevent calling database method on null
It's allowed to call Database::getCacheSetOptions() with null
as a second argument since the paramater is nullable.

However, this will lead to fatal error as we will end up calling
getSessionLagStatus() on the passed null.

The only minor safeguard to this now is if the second argument is
omitted, then func_get_args() will not contain the default as it
returns only a copy of the passed arguments, but this is not
something that should be relied upon. There's nothing that can
prevent calling code from passing the second null.

Also pass the arguments directly to foreach() without
calling func_get_args() since we already know the possible
passable arguments and the function is not variadic.

Change-Id: I7d4804a715063d95827fcec6d14a3ae059b234c3
2021-01-03 04:16:28 +01:00
Ammarpad
87ecf20aa7 Fix documentation in Database class
Empty condition is also disallowed for DELETE query.

Change-Id: I7c7b24821fa32b9edaf5026134ce745913350766
2021-01-03 03:33:38 +01:00
Ammarpad
9efb07c9d2 Use short array syntax to add array element
The short syntax is faster and recommended over the method call

Change-Id: I18a3adcfa66abe5eadb834c0b23b54c6853b8c91
2021-01-02 08:20:24 +01:00
Aaron Schulz
e3237d4b1d rdbms: rename transaction idle/precommit/postcommit callback fields in Database
This better conveys when the callbacks are supposed to be fired

Change-Id: If4a5bb3cce510657b03d1891cfb77608a0cc61b2
2020-12-13 21:15:28 +00:00
Daimona Eaytoy
b18344ba1e Clarify error message in Database::assertBuildSubstringParams
Change-Id: Iba115a6c514b49b02c2073ebfc81b6fc2ec117e9
2020-12-09 02:38:31 +01:00
Umherirrender
c85a43561e Improve class property documentation
Reformat existing documentation to match the format

Change-Id: I190b54b5e962f17bab6502dd1b3c02f11dc926d2
2020-10-30 10:38:58 +01:00
Umherirrender
d790580fda Fix typos related to repeated words
Change-Id: Ibc187d95b003017255bc87adf56afae7a59bd3db
2020-09-27 10:25:36 +00:00
Máté Szabó
ce50e1bff9 Permit temporary table writes on replica DB connections
In I8e17644d1b447416adee18e42cf0122b52a80b22, MediaWiki's DBAL was adjusted to
reject any write query on read-only DB replica connections. This poses a problem
for extensions that use temporary tables in their queries, as such queries now
have to be executed on the source DB rather than a replica to work around this
fact. An example of such an extension is Semantic MediaWiki, whose QueryEngine
uses temporary tables extensively in serving reads. The current situation, where
all writes, including non-persistent ones, must be executed on a source DB
connection, causes scalability issues since it's no longer possible to
distribute these queries between multiple replicas.

An old code comment in the DBAL cited MySQL bug 33669 as a potential blocker to
permitting temporary table operations on read-only connections. However, that
bug was closed a decade ago, and Fandom's Semantic MediaWiki cluster has been
permitting such operations on its MySQL 5.7 replica nodes (running with
--read-only) for several years now, without observing any adverse side-effect.

This patch accordingly relaxes the restrictions placed by the MediaWiki DBAL on
temporary table operations to enable executing them even on read-only replica DB
connections. Several unit tests were added to verify the conditions under which
a given write query may be allowed to execute on a connection.

Bug: T259362
Change-Id: I90a1427a15d0aee07e7b24ba4248b7ef4475c227
2020-07-31 17:47:06 +02:00
Umherirrender
c417320dd2 Fix Database::getTempTableWrites for multi table DDLs
Capture group with * does not return a array, only the last match.
Match all and split on the delimiter instead

Bug: T252183
Change-Id: I39701dd367bf2914e7c6fb24b3f14f09a059e483
2020-07-28 17:50:17 +02:00
Kunal Mehta
0b43c49465 Revert "Add a new type of database to the installer from extension"
It caused a 20% latency regression by unconditionally parsing extension.json
files on every single load instead of using the existing caching
infrastructure. There are further problems with the use of parsing/loading
extension.json files in a method that is incompatible with the existing
architecture.

This primarily reverts commit 46eabe275c.

Also needed to revert 16381261ae and 7c72347ec1.

Bug: T258664
Change-Id: I34a783c3f0df0447876a26441bb2d12e02368871
2020-07-22 16:05:31 -07:00
Aaron Schulz
5aad68403c rdbms: improve Database::commit() "out of sync" logging
Bug: T256394
Change-Id: If716a34bcd1d6fc10103ddcb3c19deb97cdacf17
2020-07-16 18:00:16 -07:00
daniel
3395978eb8 rdbms: Mark the Database class as stable for subclassing
Strategy used to determine which methods should be stable:
- Methods that encode specific SQL syntax
- Methods already overwritten by a subclass

Strategy used to determine which methods should NOT be stable:
- private or final (obviously)
- encodes logic for managing database connections
- wraps a call to a method that is intended for overriding

Note that such methods would be overwritten by an extension that
provides support for a specific database backend. Such extensions
are only now becoming possible in 1.35. Marking methods in Database
as stable for overriding is done to support the creation of
such extension (e.g. for MSSQL and Oracle).

Bug: T247862
Change-Id: Ief5386c0ff4239a8d95db2d6896338612aa56bb0
2020-07-13 19:26:46 +02:00
jenkins-bot
8b2f44b6e7 Merge "phan: Enable redundant_condition_detection" 2020-07-02 00:28:10 +00:00
Umherirrender
bc5cb7ae64 phan: Enable redundant_condition_detection
Remove duplicate casts
Suppress false positives

Bug: T248438
Change-Id: I2f89664a4bcd3b39b15e7cf850adda2f0c90ae6f
2020-07-01 20:13:07 +00:00
ArtBaltai
46eabe275c Add a new type of database to the installer from extension
Decouple Installer services
Implement injection class Autoloader and i18n messages from extension.json
Implement extension selector by type
Add i18n message key `version-database`
Extensions for testing:
- https://github.com/MWStake/PerconaDB - real Percona extension
- https://github.com/killev/mediawiki-dbext2 - fake extension for test

Bug: T226857, T255151
Change-Id: I9ec8a18ad19283f6be67ac000110ac370afc0815
2020-06-26 13:37:32 +03:00
jenkins-bot
f11d1c7d31 Merge "rdbms: Denormalize fname and errno in DBQuery log messages" 2020-06-22 22:32:53 +00:00
Timo Tijhof
5b7e86df0c rdbms: Denormalize fname and errno in DBQuery log messages
Instead of:

> (Count: 2003)
> {fname}	{db_server}	{errno}	{error}	{sql1line}

It might now log:

> (Count: 1751)
> Error 1062 from MovePage::moveTo, {error} {sql1line} {db_server}

> (Count: 252)
> Error 1064 from MediaWiki:__construct, {error} {sql1line} {db_server}

Bug: T255202
Change-Id: I54c9f10dfdf2a2d316460b1f946863034769a819
2020-06-12 18:14:15 +00:00
Umherirrender
88fe512c22 Pass function name to database functions (rdbms/installer)
Change-Id: Ia37d37f2471960c1d9d04fd9bd86224058e88413
2020-06-07 21:06:47 +02:00
Reedy
b3734a0ef4 Fix even more Squiz.Scope.MethodScope.Missing
Change-Id: I0c3f8ce7fddfa6886413cdaf1edc236c4dfff44e
2020-05-17 03:07:58 +01:00
Reedy
b3fd219187 Fix more libs PSR12.Properties.ConstantVisibility.NotFound
Change-Id: Ie514103708ed631b2c4e320441c7958662418197
2020-05-16 20:13:34 +00:00
Peter Ovchyn
64ea02b060 database: Disallow db->update() without condition
In order to prevent possible performance or replication issues, empty condition
for 'update' queries shouldn't be allowed

Bug: T243619
Depends-On: Ica5f4719c7c927a4e33ba818c40c9f6fc1a5ee7b
Change-Id: Ib728b639ec0c1b079046ac0f8492449def36f2a0
2020-04-29 19:25:27 +03:00
Aaron Schulz
27cf5ace45 rdbms: add IDatabase::QUERY_* flags to obviate isWriteQuery()
This reduces regex overhead and reliance on brittle assumptions.
This will also be useful for complex write queries involving WITH.
Some RDBMS types allow writes with in the WITH aliases themselves,
in addition to the main query itself. Checking raw SQL strings for
such things would get fairly complex.

Change-Id: I8ac4bc4d671abf02f97e82c5daf7b21271b85e5e
2020-04-28 00:49:11 +00:00
Aaron Schulz
cbc700e186 rbms: optimize and rename truncateTable() to truncate()
Allow truncation of multiple tables. This also provides for
a way to avoid risky keywords like CASCADE for Postgres.

For Postgres, use RESTART IDENTITY, which has been supported
since Postgres 8.4.

Avoid TRUNCATE/DELETE queries for empty temp tables, which is
useful for integrations tests that frequently call this method.

Reorganize and tweak the regexes in Database::getTempWrites().
It now recognizes multi-table DROP/TRUNCATE (Postgres-style).

Change-Id: Idd49f118b20ea5a0f7a3e8c00369aabcd45dd44e
2020-04-21 01:26:18 -07:00
Daimona Eaytoy
dbb23e3952 Upgrade phan to 0.10.2
Amongst other things, this version of phan bundles taint-check, which is
however disabled in the config file because there are lots of issues to
be fixed.

Upgrading phan alone now means that we can have a clean baseline for the taint-check upgrade.

Bug: T248630
Change-Id: I8ab7ef9a9e73952098664176aad6c2b3b88095ee
2020-04-16 00:28:13 +00:00
jenkins-bot
9a9be7305d Merge "rdbms: convert "Exception" try/catch blocks to "Throwable" or "RuntimeException"" 2020-04-02 01:49:53 +00:00
Aaron Schulz
303ec64395 rdbms: convert "Exception" try/catch blocks to "Throwable" or "RuntimeException"
Change-Id: I95c204c5c436c3504dcab9e7fe2ae27936ffb590
2020-04-02 01:14:17 +00:00
Tim Starling
507501d6ee Stop using SCRIPT_NAME where possible, rely on statically configured routing
It has become apparent that $_SERVER['SCRIPT_NAME'] may contain the same
thing as REQUEST_URI, for example in WMF production. PATH_INFO is not
set, so there is no way to split the URL into SCRIPT_NAME and PATH_INFO
components apart from configuration.

* Revert the fix for T34486, which added a route for SCRIPT_NAME to the
  PathRouter for the benefit of img_auth.php. In T235357, the route thus
  added contained $1, breaking everything.
* Remove calls to WebRequest::getPathInfo() from everywhere other than
  index.php. Dynamic modification of $wgArticlePath in order to make
  PathRouter work was weird and broken anyway. All that is really needed
  is a suffix of REQUEST_URI, so I added a function which provides that.
* Add $wgImgAuthPath, for use as a last resort workaround for T34486.
* Avoid the use of $_SERVER['SCRIPT_NAME'] to detect the currently
  running script.
* Deprecated wfGetScriptUrl(), a fairly simple wrapper for SCRIPT_NAME.
  Apparently no callers in core or extensions.

Bug: T235357
Change-Id: If2b82759f3f4aecec79d6e2d88cd4330927fdeca
2020-04-01 12:33:38 -04:00
Aaron Schulz
e1a8c4e2a0 rdbms: add IMaintainableDatabase::truncateTable()
Use it in MediaWikiIntegrationTestCase for resetting tables.

Also create Database::resetSequencesForTable() helper method from
the resetSequenceForTable() methods in the SQLite/Postgres classes.

Change-Id: I20945e20590e69340b1ce75f6bb2f6972375b00c
2020-03-25 01:36:06 +00:00
Aaron Schulz
13b11a946e rdbms: reduce duplication in Database via helper methods
Add several new internal methods to help with wrangling
the various formats that rows, conditions, options, and
unique key lists can come in. Remove now unused method
isMultiRowArray().

Add various sanity checks and logging for parameters to
upsert(), replace(), insert(), and insertSelect().

Move DatabasePostgresTest to the integration/ directory.

Change-Id: If5988a6f0816e8da2cbf2fd612e1a3e3a2e9c52f
2020-03-10 22:26:04 +00:00
Aaron Schulz
5dbd6797b5 rdbms: remove Database::bufferResults() and Database::tableNamesWithAlias()
Neither are used. One was deprecated and the other was internal.

Change-Id: Id7360da8f2efe3a2437695792cd93ac89625b5cd
2020-03-10 16:59:33 +00:00
jenkins-bot
44354f45eb Merge "rdbms: inject replLogger into Database and consolidate duplicated logging" 2020-03-04 21:29:23 +00:00
Aaron Schulz
ccebc2317b rdbms: improve IDatabase method documentation and parameter names
Change-Id: I87c0c6e418fe19bd837b38ebdfb967466f967fca
2020-03-02 22:59:12 +00:00
Aaron Schulz
6c5d937adb rdbms: inject replLogger into Database and consolidate duplicated logging
Bug: T235244
Change-Id: I9397f6f74f703a395ef1be4713702247060d8bd4
2020-02-23 00:33:33 +00:00
Aaron Schulz
5d6470d37d rdbms: make Database::build(Greatest|Least) support expressions
This makes it possible to use with counter UPDATE queries.

Also add some extra sanity checks for input types.

Change-Id: Ibc2b7173e28022b5ba7bb04d11c594313a47a101
2020-02-15 21:56:57 +00:00
jenkins-bot
c3f20ad3f7 Merge "Add SelectQueryBuilder" 2020-02-13 00:04:11 +00:00
Ricordisamoa
1b3bc281ac Clean up redundant Exception|Throwable union type
PHP 7.0 makes many error conditions throw instances of the new Error class
which does not extend the known Exception.
The Throwable interface provides a concise and type-safe way of handling
either, e.g. for logging purposes, but HHVM did not support it, requiring
tedious fallback checks.

This commit replaces occurrences of Exception in code paths equally
covered by Throwable, like Exception|Throwable parameter and return types
(also nullable), instanceof guards, duplicated `catch` blocks, as well as
related comments and documentation blocks, with the exception of $previous
parameter descriptions consistent with the manual at
https://www.php.net/manual/en/exception.construct.php

Proper type declarations have been added or reinstated where possible.

Change-Id: I5d3920d3cc66936a350314e2f19c4f6faeffd7c0
2020-02-12 20:28:40 +00:00
Tim Starling
d06a3e049b Add SelectQueryBuilder
Add a query builder class which encapsulates the parameters to
IDatabase::select() and related functions.

Override useIndexClause() and ignoreIndexClause() in DatabaseTestHelper
so that index hints can be tested.

Bug: T243051
Change-Id: I58eec4eeb23bd7fb05b8b77d0a748f1026afee52
2020-02-07 10:10:17 +11:00
Tim Starling
00c8a5cbde In Database::select() allow an empty array for $table
Previously it would give FROM followed by nothing which is always a
syntax error. Easier to fix it here than to convert empty arrays to
empty strings in SelectQueryBuilder.

Bug: T243051
Change-Id: I95a9b6a34cfb5c1ca4cf243c4226b5ed4f968035
2020-02-07 09:57:27 +11:00
Aaron Schulz
314efebb56 rdbms: add GREATEST/LEAST wrappers to IDatabase
Change-Id: I9de931123b03ce10713a3a9bbb34e1332dd5965b
2020-01-17 22:19:08 +00:00
James D. Forrester
0958a0bce4 Coding style: Auto-fix MediaWiki.Usage.IsNull.IsNull
Change-Id: I90cfe8366c0245c9c67e598d17800684897a4e27
2020-01-10 14:17:13 -08:00