Commit graph

46 commits

Author SHA1 Message Date
Timo Tijhof
4ef0891994 rdbms: Consolidate logger channels into one
Notable changes:

* In SqlBagOStuff::getConnectionFromServerInfo, only two loggers were
  injected. The rest implicitly got a NullLogger due to being absent.
  These are now effectively unsilenced.

* Database::__construct() required almost all parameters, even the
  loggers. I've wanted to move some of DatabaseFactory into the ctor
  here for a while. In order to make this change not a breaking
  change, the new 'logger' parameter is optional with NullLogger as
  default. This allowed some of the test cases, which were simply
  passing NullLogger, to be fixed by passing nothing instead of
  passing the new option name.

  The Database class is behind a dozen layers of indirection for
  real use, so this will still be injected just fine (DBF, LB, LBF,
  MWLBF, etc.).

* In LegacyLogger, the handling for $wgDBerrorLog was previously
  limited to DBConnection and DBQuery. This now includes errors
  from other (generally, newer) parts of Rdbms as well, which were
  previously missing.

  This only affects sites (typically CI and dev setup) where
  $wgDBerrorLog is used, as opposed to the more common
  $wgDebugLogGroups by-channel configuration.

* TransactionProfiler gets its logger injected in a rather odd way,
  via entrypoints (MediaWiki.php, ApiMain.php, and MaintenanceRunner)
  as opposed to service wiring. This is kept as-is for now.

* In LBFactoryTest, in particular testInvalidSelectDBIndependent2,
  there are cases that intentionally produce failures of which
  the result is then observed. In CI we assert that dberror.log is
  empty so instead of adding the missing logger fields to that
  LBFactory instance, the only one set (replLogger) is removed.
  The alternative is to set 'logger' now, which would naturally
  cause CI failures due to unexpected entries coming through to
  non-mocked error log.

Bug: T320873
Change-Id: I7ca996618e41b93f488cb5c4de82000bb36e0dd3
2023-01-03 22:46:38 +00:00
Aaron Schulz
23eab3ec39 rdbms: make SqlitePlatform::tableName() apply double quotes
The RDBMs classes treat table names as application-supplied rather
than user-supplied, since it does not make sense for the user to be
able to blindly pick the schema (likely causing query errors). Table
names are generally non-problematic [a-zA-Z0-9_]+ strings. However,
they still should be escaped as identifiers, since they might have
problematic characters. Fix the SQLite backend to apply escaping
here, similar to MySQL and Postgres.

Fix incorrect use of tableName() in DatabaseSqlite::fieldInfo().
It should work the same as in tableExists().

Bug: T72510
Change-Id: I3713f81e6e1eca84cc51c12c9307d4c5dc737043
2022-11-23 00:17:40 +00:00
Aaron Schulz
fe7301c9ba rdbms: remove getTopologyRootPrimary() and clean up related fields/parameters
This method is unused and the naming is not very clear in terms of what
it returns (server name vs config map). Removing it reduces externally
exposed complexity.

Optimize DBConnRef::getServerName() when the server index is known.
Use this to eliminate the "topologicalMaster" parameter from Database.

Rename internal fields and paramters in Database to use "primary"
instead of "master", for consistency.

Add some additional clarifying comments.

Bug: T299691
Change-Id: I98515fa02a58a4c72a06f1ff283b249b1617c886
2022-08-26 12:26:10 -07:00
Amir Sarabadani
2eb242f40a rdbms: Drastically simplify Sqlite db tests
Now that we moved considerable chunk of DatabaseSqlite to
SqlitePlatform, it's easier to mock and test without actually needing a
live db connection.

This allows moving a lot of tests from integration to unit tests making
them much faster.

Change-Id: I8ce7c470abed5144ba456a000c733e5d739c5c4e
2022-08-08 14:38:20 +02:00
jenkins-bot
26a8cd5bd3 Merge "Cast results of Sqlite test to string [php 8.1]" 2022-08-03 12:44:52 +00:00
Amir Sarabadani
9bb6189a30 rdbms: Migrate SchemaVar and replaceVars from Database to SQLPlatform
Bug: T307616
Change-Id: I0975dd0328236c34b4e6c6df846e2465da31b750
2022-08-02 10:14:54 +02:00
Brian Wolff
1e50e2e538 Cast results of Sqlite test to string [php 8.1]
It seems that on php 8.1, the data fetching from sqlite will
fetch integers for integer fields instead of strings. Cast
everything to strings so the test works on both php 8.1 and 7.x.

Bug: T313663
Change-Id: I0a2c2ab95b84565186ac7417f6cfdef2215f6bf1
2022-07-29 22:17:41 -07:00
Umherirrender
b0398654c5 tests: Do not use class alias in @covers
Also fix case for the XMLTypeCheck test

Change-Id: I809510c8085a36e20fd0eefb5e77d2671b3148f2
2022-05-28 00:37:43 +02:00
Aryeh Gregor
4851f512b3 IResultWrapper::next() now returns void
As required by the Iterator interface. In PHP 8.1 we can suppress the
warning with #[\ReturnTypeWillChange], but it's wrong and should change
anyway.

Depends-On: I576109808755b054e7876556f244ee8eafd12f9d
Change-Id: Ibaa7b4c81d8f114783db45c9100200ba14547c8f
2022-05-02 10:43:51 +03:00
Aaron Schulz
277d34818b tests: Split out DatabaseSqliteUpgradeTest class
The testUpgrades() method is slow and is easier to
parallelize via paratest if it lies in a separate class/file

Also make the test use a data provider

Bug: T50217
Change-Id: I5c8df6ae0ff34941f4cf7275680b309e79565925
2022-03-10 23:29:29 -08:00
Ammarpad
e9e3c4a75f Fix broken SQLite test and simplify.
This is not testing anything since tables.sql was emptied
in I9f33132

It's probably also no longer necessary, since the schema
is now generated by Doctrine per platform, but I am not sure
about that, so I am just fixing the test and simplifying it.

Change-Id: Icb4d04e6057958771eb00a31b967a42cc948d6a1
2022-03-02 22:39:53 +01:00
Amir Sarabadani
050fbbec8f rdbms: Remove IDatabase::numFields() method
Soft-deprecated since 1.37.

Unused https://codesearch.wmcloud.org/search/?q=numFields&i=nope&files=&excludeFiles=&repos=

Bug: T286694
Bug: T296960
Change-Id: I722602c8b11dcfc5cab70593876a4fb336e9684a
2021-12-21 23:10:49 +00:00
Amir Sarabadani
c3fff36540 Add 1.36 and 1.37 snapshots to sqlite db tests
To make sure upgrades from those versions work properly

Change-Id: I570cf965286bc5358a43c9601217b8781bb09b0e
2021-09-29 20:13:38 +02:00
Amir Sarabadani
753a4ecdd3 Bump minimum required version for upgrade to 1.29
Since the branch cut has happened, we can bump and get rid of legacy
cruft. According to the policy we can go up to 1.31 but let's keep it
that way to avoid major distruptions.

Change-Id: I9d697445a3bb5047726c8b2a7f808edb8403cdda
2021-09-28 17:57:57 +02:00
libraryupgrader
5357695270 build: Updating dependencies
composer:
* mediawiki/mediawiki-codesniffer: 36.0.0 → 37.0.0
  The following sniffs now pass and were enabled:
  * Generic.ControlStructures.InlineControlStructure
  * MediaWiki.PHPUnit.AssertCount.NotUsed

npm:
* svgo: 2.3.0 → 2.3.1
  * https://npmjs.com/advisories/1754 (CVE-2021-33587)

Change-Id: I2a9bbee2fecbf7259876d335f565ece4b3622426
2021-07-22 03:36:05 +00:00
Aaron Schulz
99d5d2e8cc rdbms: cleanup getServer() and connection parameter fields in Database
Make getServer() always return a string, as documented, even with new
Database::NEW_UNCONNECTED handles that have yet to call open(). If the
'host' parameter to __construct() is ''/null, getServer() now returns
'localhost' instead of null. This avoids problems like fatal errors in
calls to TransactionProfiler::recordConnection().

Use Database constants for "connectionParams" field keys for better
static analysis.

Also:
* Add Database::getServerName() method that returns "readable" server
  names in the style of LoadBalancer::getServerName(). Note that the
  "hostName" field is already passed in from LoadBalancer.
* Migrate most getServer() callers to getServerName() for easier
  debugging and more readable logging.
* Also, normalize Database/LoadBalancer SPI logging context to use
  "db_server" and reduce logging code duplication in LoadBalancer.

Bug: T277056
Change-Id: I00ed4049ebb45edab1ea07561c47e226a423ea3b
2021-05-05 19:44:02 +00:00
Daimona Eaytoy
535d7abf59 phpunit: Mass-replace setMethods with onlyMethods and adjust
Ended up using
  grep -Prl '\->setMethods\(' . | xargs sed -r -i 's/setMethods\(/onlyMethods\(/g'

special-casing setMethods( null ) -> onlyMethods( [] )

and then manual fix of failing test (from PS2 onwards).

Bug: T278010
Change-Id: I012dca7ae774bb430c1c44d50991ba0b633353f1
2021-04-16 20:15:00 +02:00
Umherirrender
a1de8b8700 Tests: Mark more more closures as static
Result of a new sniff I25a17fb22b6b669e817317a0f45051ae9c608208

Bug: T274036
Change-Id: I695873737167a75f0d94901fa40383a33984ca55
2021-02-09 02:55:57 +00:00
Amir Sarabadani
ba4a80facc Remove old exceptions from DatabaseSqliteTest
Follow up to Ib5251f7ae3814

Change-Id: I5e1503506cb55673c9283dcbe1d38a46cb7e1edd
2021-02-05 23:29:33 +00:00
Amir Sarabadani
78e6c48156 Rework sqlite schema changes tests
We don't support upgrading from below 1.31 officially and from below 1.27, the
upgrade system doesn't work at all.

The files are copied from the release branches.

Bug: T259771
Change-Id: Ib5251f7ae38148e00ac2dccb7bde76529c9d5076
2021-02-05 16:21:24 +00:00
Thiemo Kreuz
ac205fa84c Fix incomplete/bogus PHPDoc tags in various tests
Most of these are found by the not yet released I10559d8.

I remove the type MockObject in some cases when the calling
code really does not need to know if he get's a mock or the
real thing. However, I do this only in places that are very
closely related to the fixes.

Change-Id: I26a4c3c5a8ae141bf56161b52b54bce7e68f2e30
2021-01-14 19:02:00 +00:00
Daimona Eaytoy
2b37cfaf18 build: Bump mediawiki-codesniffer to 31.0.0
Done with `composer fix` and suppressing the rest (i.e. sniffs for
global variables, which for core should be suppressed anyway).

Additionally, add `-p` to `phpcbf`, as otherwise it just seems stuck.

Change-Id: Ide8d6cdd083655891b6d654e78440fbda81ab2bc
2020-05-30 14:56:28 +00:00
Amir Sarabadani
11f241589b Wire empty abstract schema into installer
For more information on schema changes see:
https://www.mediawiki.org/wiki/Manual:Schema_changes

Bug: T191231
Bug: T230428
Change-Id: I779b9c61e388e998738a2ed41b0b8e2afdecd232
2020-05-09 20:42:11 +02:00
Amir Sarabadani
6319ffa1b7 Do not assume case in checking NULL in DatabaseSqliteTest
Currently, when trying to automatically generate the schema, it
majestically explodes because "NULL" is not equal to "null".

Change-Id: I0facf313525eee1af31fdb307368ecf3bdda895b
2020-05-09 14:10:08 +00:00
Aaron Schulz
86533ae590 Drop DEFAULT from *_actor columns for SQLite tables that still have it
These were left out by mistake (made easier by a bug where some phpunit
tests never run). Fixing the test bug is blocked on fixing the SQLite
schema so that DatabaseSqliteTest.php passes when it actually runs.

Bug: T246077
Change-Id: I75d7a7fb56a07c2984e987b6136e1fd00615c421
2020-03-19 21:17:56 -07:00
Aaron Schulz
f13b3c8521 Fix obsolete version-based tests in DatabaseSqliteTest
These failed and servered no use after 6cf729838f

Change-Id: I29f5e761b977c1c831afd75da901b5e58a4bbde1
2020-03-18 14:32:26 -07:00
Aaron Schulz
13b11a946e rdbms: reduce duplication in Database via helper methods
Add several new internal methods to help with wrangling
the various formats that rows, conditions, options, and
unique key lists can come in. Remove now unused method
isMultiRowArray().

Add various sanity checks and logging for parameters to
upsert(), replace(), insert(), and insertSelect().

Move DatabasePostgresTest to the integration/ directory.

Change-Id: If5988a6f0816e8da2cbf2fd612e1a3e3a2e9c52f
2020-03-10 22:26:04 +00:00
jenkins-bot
44354f45eb Merge "rdbms: inject replLogger into Database and consolidate duplicated logging" 2020-03-04 21:29:23 +00:00
Aaron Schulz
a14a1f39a1 rdbms: support native batched replace() in DatabaseSqlite
Change-Id: Icc6d6ff7412d6a686b200e7d8432840b46377d10
2020-03-02 15:10:15 +00:00
Peter Ovchyn
9b34f6786a sqlite: Fix broken integration tests for SQLite db driver
Bug: T245995
Change-Id: I280487bf27e4864eba8c0f02f408306061b948b2
2020-02-26 21:57:09 +02:00
Timo Tijhof
f575721a06 Update all use of $wgVersion to MW_VERSION
Follows-up I04628de4152dd5.

Bug: T212738
Change-Id: I718474ec0d9fd29ac2c05477f0f2493615d8aff5
2020-02-25 02:16:12 +00:00
Aaron Schulz
6c5d937adb rdbms: inject replLogger into Database and consolidate duplicated logging
Bug: T235244
Change-Id: I9397f6f74f703a395ef1be4713702247060d8bd4
2020-02-23 00:33:33 +00:00
jenkins-bot
9b1067846d Merge "Coding style: Auto-fix MediaWiki.Usage.PHPUnit*" 2020-01-10 16:59:11 +00:00
James D. Forrester
5e9fca47b9 Coding style: Auto-fix MediaWiki.Usage.PHPUnit*
Change-Id: I86fc55a4fc8ceafe368692173211bbcd6d8581d7
2020-01-10 10:17:12 +00:00
Aaron Schulz
3c1c9b5098 rdbms: support native batched insert() in DatabaseSqlite
This was added in 4bd1b4b455 but reverted due to a bug in other
changes within that patch, namely, the replace() method changes.

Change-Id: Id73c3fed9e1862daeeecbf47bcfa53d1719c59bb
2020-01-06 18:17:46 +00:00
Daimona Eaytoy
f5529400cc Stop using assertContains with string haystack
This was done automatically by replacing every assertContains with
string *needle*. Then verifying the results.

Bug: T192167
Change-Id: Id8cbbf3b01e948f80046714183cc299f86be21fd
2019-12-15 00:22:44 +00:00
Daimona Eaytoy
6365eaab8d Autofix 94 PHPUnit 8 compat issues
Done automatically using the master version of MW codesniffer and
running composer fix.

Bug: T192167
Change-Id: If6b40f515fde32ab5eff074a90e821c30c791827
2019-12-13 15:29:10 +01:00
Timo Tijhof
0da1ecfd89 profiler: Remove ProfilerOutputDb and profileinfo.php entry point
Bug: T231366
Change-Id: I211c8192200d9af00914f9847608a6daf2898f91
2019-11-06 15:28:00 -05:00
Max Semenik
48a323f702 tests: Add explicit return type void to setUp() and tearDown()
Bug: T192167
Depends-On: I581e54278ac5da3f4e399e33f2c7ad468bae6b43
Change-Id: I3a21fb55db76bac51afdd399cf40ed0760e4f343
2019-10-30 14:31:22 -07:00
jenkins-bot
662c41a0fb Merge "Revert "rdbms: optimize insert(), replace(), and upsert() for sqlite when possible"" 2019-10-08 17:06:53 +00:00
Anomie
be8ef0f2c3 Revert "rdbms: optimize insert(), replace(), and upsert() for sqlite when possible"
This reverts commit 4bd1b4b455.

Reason for revert: Incorrect syntax for SQLite "ON CONFLICT (...) DO UPDATE".

Bug: T234795
Change-Id: I987e6e7052368d5d9d0b6de9dd189edc4feaebe5
2019-10-08 16:51:37 +00:00
Umherirrender
fb3914739e Type hint against IResultWrapper
IDatabase::select returns the interface IResultWrapper,
not the implementation ResultWrapper

Change-Id: I8904eff2db48ff00a4a9486be842266c638bc249
2019-10-04 19:58:19 +02:00
Aaron Schulz
4bd1b4b455 rdbms: optimize insert(), replace(), and upsert() for sqlite when possible
Change-Id: Ic884a4ce42a99333f5176f7b681f8a8bba15d2a1
2019-10-03 01:45:24 +00:00
Thiemo Kreuz
32a429e8c4 tests: Prefer assertSame() when comparing the integer 0
assertSame() is guaranteed to not do any type conversion. This can be
critical when acciden tially comparing, for example, 0 to 0.0.

Change-Id: Iffcc9bda69573623ba14af655dcd697d0fcce525
2019-09-19 15:35:23 +00:00
Kosta Harlan
c860482d29 Tests: Set dbname for DatabaseSqliteTest
Bug: T230780
Change-Id: I93d3771233c74443f76b0a402a4820e1e1e30ee7
2019-08-20 11:28:29 +02:00
Kosta Harlan
214750d8d2 Define unit and integration test suites
Following discussion in Ibb8175981092d7f41864e641cc3c118af70a5c76, this patch
proposes to further reduce the scope of what unit tests may access, by removing
the loading of DefaultSettings and GlobalFunctions.php. This also has the
implied effect of disabling the storage backend, as well as the global service
locator.

MediaWikiTestCase is renamed to MediaWikiIntegrationTestCase so it's scope and
purpose is more clear. Whether we still need to keep `@group Database`
annotation around is debatable, as it's unclear to me what the performance costs
are of implying database access for all tests which extend IntegrationTestCase.
As far as I can tell, `@group Database` is primarily used in CI to run faster
tests before slower ones, and with the new UnitTestCase the annotation seems
redundant.

To run all testsuites, use `composer phpunit`. Other composer scripts:

- `composer phpunit:unit` to run unit tests
- `composer phpunit:integration` to run integration tests
- `composer phpunit:coverage` to generate code coverage reports from unit
   tests (requires XDebug).

Note that you can pass arguments to composer scripts with `--`, e.g. `composer
phpunit:integration --exclude-group Dump`.

Other changes:

- Rename bootstrap.php to bootstrap.maintenance.php so it's clear it's part of
  the legacy PHPUnit-as-maintenance-class setup
- Create new bootstrap.php which loads the minimal configuration necessary for
  the tests, and do additional setup in the run() method of the unit/integration
	test case classes
- Move the unit-tests.xml file to phpunit.xml.dist in preparation for this being
  the default test configuration

For a follow-up patch:

- Find unit/integration tests for extensions/skins
- Migrate other test suites from suite.xml
- Support running all tests via vendor/bin/phpunit

Bug: T84948
Bug: T89432
Bug: T87781
Change-Id: Ie717b0ecf4fcfd089d46248f14853c80b7ef4a76
2019-06-28 12:18:18 -04:00
Renamed from tests/phpunit/includes/db/DatabaseSqliteTest.php (Browse further)