Commit graph

90 commits

Author SHA1 Message Date
James D. Forrester
1d71fd91d6 ILoadBalancer and IDatabase: Rename getMasterPos to getPrimaryPos
Bug: T282894
Change-Id: I30a600335af735a13a6ad2d3df56a5f17e05bfab
2021-08-02 18:14:41 +00:00
James D. Forrester
d11c59538a Rename DB primary position interfaces to DBPrimaryPos and MySQLPrimaryPos
And replace all uses.

Bug: T282894
Change-Id: I5222a8568255ac9fa5e2350e2264b8d2ee5eb968
2021-08-02 17:59:39 +00:00
James D. Forrester
719cf161f2 More master -> primary documentation and internal var renaming
Bug: T254646
Change-Id: I63cc8895033714bdfbf09aee933a8f0a43b387f3
2021-07-15 11:20:20 +01:00
Fomafix
ea1b06c150 Fix various typos in documentation
Bug: T201491
Change-Id: Ifa1b84b28ea97dfb4212658b02f24ee6881c1cb0
2021-06-21 00:26:07 +00:00
James D. Forrester
df5eb22f83 Replace uses of DB_MASTER with DB_PRIMARY
Just an auto-replace from codesniffer for now.

Change-Id: I5240dc9ac5929d291b0ef1c743ea2bfd3f428266
2021-04-29 09:24:31 -07:00
jenkins-bot
89b00fcfbe Merge "Remove unnecessary ->equalTo() from tests" 2021-04-23 14:43:04 +00:00
Thiemo Kreuz
c1ee8250e9 Remove unnecessary ->equalTo() from tests
This is the default anyway when using ->with(). The test code
becomes so much more readable without this, I would like to
argue. Let it just say "with these values".

Because of the way I split my changes into multiple patches
there are a few other changes in this patch I could not split,
e.g. removing unnecessary ->any(). This is the default anyway
and doesn't make the test more specific.

Change-Id: I34990799fa9258ba8dc64c7e78ec43f7903b7681
2021-04-23 12:02:42 +02:00
Thiemo Kreuz
e2ec44d73f Replace ->exactly( 1 ) with ->once() in tests
It's shorter, the more canonical form, and easier to read
because of this.

Change-Id: I9c68a49c1f31bddfac91a74a6592fe3545ca00a7
2021-04-22 09:42:28 +02:00
Aaron Schulz
65947212ba rdbms: improve ChronologyProtector comments and variable/method names
Also added some basic type hints

Change-Id: I5ddf18cba9517deb077625bc04c1351a8b49ea66
2021-04-13 01:11:12 +00:00
Aaron Schulz
bd7cf4dce9 Add $wgChronologyProtectorStash and improve $wgMainStash comments
Remove WRITE_SYNC flag from ChronologyProtector since the current
plan is to simply use a datacenter-local storage cluster.

Move the touched timestamps into the same stash key that holds the
replication positions. Update the ChronologyProtector::getTouched()
comments.

Also:
* Use $wgMainCacheType as a $wgChronologyProtectorStash fallback
  since the main stash will be 'db-replicated' for most sites.
* Remove HashBagOStuff default for position store since that can
  result in timeouts waiting on a write position index to appear
  since the data does not actually persist accress requests.
* Rename ChronologyProtector::saveSessionReplicationPosition()
  since it does not actually save replication positions to storage.
* Make ChronologyProtector::getTouched() check the "enabled" field.
* Allow mocking the current time in ChronologyProtector.
* Mark some internal methods with @internal.
* Migrate various comments from $wgMainStash to BagOStuff.
* Update some other ObjectCache related comments.

Bug: T254634
Change-Id: I0456f5d40a558122a1b50baf4ab400c5cf0b623d
2021-03-01 20:34:34 +00:00
Umherirrender
a1de8b8700 Tests: Mark more more closures as static
Result of a new sniff I25a17fb22b6b669e817317a0f45051ae9c608208

Bug: T274036
Change-Id: I695873737167a75f0d94901fa40383a33984ca55
2021-02-09 02:55:57 +00:00
Reedy
eb41565a9a Tests: Start marking some closures as static
Bug: T274036
Change-Id: Ib738ecd3bc23d34900bc268c8246702ac3655746
2021-02-06 19:57:42 +00:00
addshore
959bc315f2 MediaWikiTestCase to MediaWikiIntegrationTestCase
The name change happened some time ago, and I think its
about time to start using the name name!
(Done with a find and replace)

My personal motivation for doing this is that I have started
trying out vscode as an IDE for mediawiki development, and
right now it doesn't appear to handle php aliases very well
or at all.

Change-Id: I412235d91ae26e4c1c6a62e0dbb7e7cf3c5ed4a6
2020-06-30 17:02:22 +01:00
Thiemo Kreuz
6aa6d10e86 Replace all call_user_func(_array) in all tests
There is native support for all of this now in PHP, thanks to changes
and additions that have been made in later versions. There should be no
need any more to ever use call_user_func() or call_user_func_array().

Reviewing this should be fairly easy: Because this patch touches
exclusivly tests, but no production code, there is no such thing as
"insufficent test coverage". As long as CI goes green, this should be
fine.

Change-Id: Ib9690103687734bb5a85d3dab0e5642a07087bbc
2020-06-06 18:41:20 +02:00
Tim Starling
fbf4fe4357 In ChronologyProtector fix confusion between DB name and master name
The one caller of LBFactory::getChronologyProtectorTouched() was calling
it with a domain ID instead of a master server name. Using the master
server name to identify replication position makes sense for
ChronologyProtector, since the replication position may be reset when
the master changes, but it is an odd convention for LBFactory. So:

* Rename all $dbName variables in ChronologyProtector to $masterName,
  for clarity.
* Interpret the first parameter to
  ILBFactory::getChronologyProtectorTouched() as a database domain, to
  make its only existing caller work.
* Change the first parameter to ChronologyProtector::getTouched() from a
  string to a strongly typed ILoadBalancer, by analogy with
  applySessionReplicationPosition(). This removes the master name concept
  from the public interface.
* Mark ChronologyProtector @internal. The accessor in LBFactory is
  protected, so extensions can't use it anyway. This is just to clarify
  why I think changing the parameter to getTouched() without b/c is OK.
* Add a simple test which mostly just checks that ChronologyProtector gets
  called with the correct parameters. It's an
  LBFactory/ChronologyProtector integration test.

Change-Id: I3b4832b5a4d7410e94b9c51577b30b31d49bc63d
2020-06-01 12:54:35 +10:00
Daimona Eaytoy
2b37cfaf18 build: Bump mediawiki-codesniffer to 31.0.0
Done with `composer fix` and suppressing the rest (i.e. sniffs for
global variables, which for core should be suppressed anyway).

Additionally, add `-p` to `phpcbf`, as otherwise it just seems stuck.

Change-Id: Ide8d6cdd083655891b6d654e78440fbda81ab2bc
2020-05-30 14:56:28 +00:00
Reedy
229b2c15e8 Fix a plethora of class and function call case mismatches
Bug: T231412
Change-Id: I597a25de3294a6673424f30475760280ef209a8a
2020-05-26 14:14:46 +01:00
Aaron Schulz
fcfea80fa3 rdbms: add "loadMonitor" parameter to LBFactoryMulti/LBFactorySimple
This allows full configuration of the LoadMonitor, not just setting
the class name like loadMonitorClass allows. Deprecate the later.

Also fix obsolete LoadBalancer constructor documentation and
standardize the style of LBFactoryMulti::__construct() comments.

Change-Id: Icfdc0d6bbf227dce56e65ad679b93ce3f604e9f7
2020-03-27 20:43:00 -07:00
Aaron Schulz
160370afcd rdbms: dependency inject domain aliases into LBFactory
Add new ILBFactory::setDomainAliases() method for injection database
domain aliases and call it in MWLBFactory::setDomainAliases().

Also:
* Remove overkill "last db/section" caching in LBFactoryMulti
* Clean up some LBFactoryMulti code comments
* Split out separate MWLBFactoryTest test file

Change-Id: If180a58c61178969ca7587c4a06b8786574c7254
2020-02-15 22:34:38 +00:00
James D. Forrester
4f2d1efdda Coding style: Auto-fix MediaWiki.Classes.UnsortedUseStatements.UnsortedUse
Change-Id: I94a0ae83c65e8ee419bbd1ae1e86ab21ed4d8210
2020-01-10 09:32:25 -08:00
James D. Forrester
83d76f4cb5 phpcs: Enable MediaWiki.Commenting.PhpunitAnnotations.ForbiddenExpectedException* and make pass
Change-Id: I63f97497714a32236268be6965c5e181dade6c58
2019-10-14 12:48:48 -07:00
Aaron Schulz
fb621c26a3 rdbms: various cleanups to LoadBalancer::reallyOpenConnection()
Move the DBO_TRX init logic out of Database::__construct() and into
LoadBalancer since the later already handles setting and clearing this
flag based on transaction rounds starting and ending.

Add 'lazyMasterHandle', 'topologyRole', and 'topologicalMaster' parameters
to Database::factory() and inject them via LoadBalancer all at once in order
to avoid worrying about call order. Move some type casting code to
Database::__construct().

Add IDatabase::getTopologyRole()/getTopologicalMaster().

Use constants for getLBInfo()/setLBInfo() for better usage tracking and
typo resistance.

Change-Id: I437ce434326601e6ba36d9aedc55db396dfe4452
2019-10-11 11:35:02 -07:00
Thiemo Kreuz
c2211946f7 tests: Replace PHPUnit's loose assertEquals(null) with assertNull()
assertEquals( null, … ) still succeeds when the actual value is 0, false,
an empty string, even an empty array. All these should be reported as a
failure, I would argue.

Note this patch previously also touched assertSame( null ). I reverted
these. The only benefit would have been consistency within this codebase,
but there is no strict reason to prefer one over the other. assertNull()
and assertSame( null ) are functionally identical.

Change-Id: I92102e833a8bc6af90b9516826abf111e2b79aac
2019-09-27 19:15:38 +00:00
jenkins-bot
0fc1dee35e Merge "tests: Prefer PHPUnit's assertSame() when comparing empty strings" 2019-09-20 15:55:08 +00:00
Thiemo Kreuz
e06ce9f467 tests: Prefer PHPUnit's assertSame() when comparing empty strings
assertSame() is guaranteed to never do any magic type conversion.
This can be critical when accidentially comparing empty strings (a
value PHP considers to be "falsy") to false, 0, 0.0, null, and such.

Change-Id: I2e2685c5992cae252f629a68ffe1a049f2e5ed1b
2019-09-20 15:27:58 +00:00
Thiemo Kreuz
32a429e8c4 tests: Prefer assertSame() when comparing the integer 0
assertSame() is guaranteed to not do any type conversion. This can be
critical when acciden tially comparing, for example, 0 to 0.0.

Change-Id: Iffcc9bda69573623ba14af655dcd697d0fcce525
2019-09-19 15:35:23 +00:00
Aaron Schulz
1fbb16c712 rdbms: expand on LoadBalancer ownership concept
Enforce this pattern for the remaining LoadBalancer methods.
Carry this over into Database::close() to decide how loud the
error handling should be.

In LBFactory, clean up ownership of newMainLB()/newExternalLB().
The should have a null owner if called from outside the class
since the LBFactory does not track nor care about them anymore
in that case. Disable newMainLB() for LBFactorySingle as it
makes no sense and was broken.

Also remove some redundant abstract LBFactory methods that
just duplciate ILBFactory.

Bug: T231443
Bug: T217819
Depends-On: I7ed5c799320430c196a9a8e81af901999e2de7d0
Change-Id: I90b30a79754cfcc290277d302052e60df4fbc649
2019-09-05 12:08:13 -07:00
Urbanecm
6ded91313c Revert "rdbms: make LoadBalancer::reallyOpenConnection() handle setting DBO_TRX"
This reverts commit 45831e619c.

Reason for revert: Caused beta not work at all.

Bug: T231162
Change-Id: Icc5c1fa0dc01082a622641ad96c22c939cd56d48
2019-08-25 16:39:33 +00:00
Aaron Schulz
45831e619c rdbms: make LoadBalancer::reallyOpenConnection() handle setting DBO_TRX
Make LoadBalancer::reallyOpenConnection() handle initializing DBO_TRX
instead of Database::__construct().

Also:
* Avoid having the "catch" block appear like it returns a
  half-constructed Database.
* Use the variable name $conn instead of $db to be consistent
  throughout the class. Only send Database::__construct() parameters
  that it recognizes instead of mixing in setLBInfo() data.

Change-Id: Iffc3d1d0713051a164adb51a4c4ee12e4ac887c3
2019-08-24 20:06:17 +00:00
Amir Sarabadani
4d10bb14e8 Drop Oracle and Mssql
After approval of RFC T191231, we are going to drop oracle and mssql
and it will be possible to bring back the support using the abstract schema

Adding to release notes will be done in a follow-up

Bug: T230418
Change-Id: I90bd5cfcc3e18011b193c965fdb1fa54675040b5
2019-08-14 11:31:41 +00:00
Aaron Schulz
a3a6dfed2e rdbms: allow automatic PDO creation of SQLite database files
Define missing DatabaseSqlite::doSelectDomain() method to handle attempts
to change the database, prefix, and/or schema.

Also add sanity check to serverIsReadOnly() to make sure open() was called

Change-Id: I72c25bf4dab5e01def3fb9472217e7637aede1d4
2019-08-07 01:05:35 +00:00
Aaron Schulz
318306ed17 rdbms: add ILoadBalancer::getReplicaResumePos method
This does what ChronologyProtector wants more rigorously and is better
named. Not all replica servers will have the same position, so they
should be compared to get the highest one.

Simplify the getMasterPos() method to only return master positions
as the other current callers do not need anything else. It will now
connect if needed as well. This should make the method naming better.

Reducing the use of replica derived replication postitions (instead
of those from the master) makes certain GTID issues less likely,
such as the matter of obsolete domain IDs.

Increase general test coverage of LoadBalancer.

Bug: T224422
Change-Id: I5420721ee339a24d09c26c38709500c7bbe797c2
2019-07-12 18:16:15 +01:00
Aaron Schulz
69c503148f rdbms: add replica server counting methods to ILoadBalancer
This is slightly more robust and makes the intent much clearer
than random calling code checking getServerCount() all over the
place. In addition, this yields better separation of concern.

Also, cleanup the LoadBalancer constructer a bit and make the
validation a bit stricter.

Make some server index comparisons strict while at it.

Change-Id: Icc1a35bd65c6862ff81faa3ab9b2aa7cafe29443
2019-06-20 12:47:23 +01:00
Aaron Schulz
17de186917 rdbms: fix ChronologyProtector client IDs to not be empty
Follow up to 8107fddd8f

Also improve debug logging and add some unit tests

Bug: T223310
Bug: T223978
Change-Id: I35484385e4da2912bc10f5e8d2fb07cb1097347e
2019-05-24 23:42:32 -07:00
Umherirrender
a78251721b Mock IDatabase::class instead of DatabaseMysqli::class
Change-Id: I4c15c32b17e1e5a70ad59b25f976a2fc0e7907fe
2019-05-01 17:38:11 +02:00
Aaron Schulz
8107fddd8f rdbms: add "secret" parameter to ChronologyProtector to use HMAC client IDs
Also make $posIndex mandatory and clean up some IDE warnings in LBFactory.

Change-Id: I9e686b670bc86eb377f14ca57a94e1aa3fd901d5
2019-04-19 16:22:04 -07:00
Aaron Schulz
deaf6b0e5d Change Database type hints to use IDatabase
Change-Id: Ie1e86cd961a619335138402ba22fe2a73a3e62fd
2019-04-07 01:15:50 +00:00
Aaron Schulz
c4e284f113 rdbms: codify DatabaseDomain table "_" prefix convention
Alos simplify isCompatible() slightly and make the string
case in convertToString() explicit.

Change-Id: Ifb61bb5fb012491520525bbebfbde2269fa55b52
2019-03-26 21:04:51 +00:00
Aaron Schulz
321640b117 rdbms: use a direct "USE" query for doSelectDomain() for mysql
This should give better error messages on failure.

Bug: T212284
Change-Id: I55260c6e3db1770f01e3d6a6a363b917a57265be
2019-03-26 18:50:28 +00:00
Thiemo Kreuz
eee735a176 rdbms: Use more narrow type hints in LBFactoryTest
Originally I wanted to fix the broken type hint for the `Database`
class. But I ended limiting the scope as much as possible and only
type hint at the interfaces that contain the methods the code actually
needs.

Change-Id: I450a55e80612f6e676f7c803dc904b4b653426d8
2019-02-13 18:27:26 +00:00
Aaron Schulz
d2e1734390 rdbms: rename setDomainPrefix to setLocalDomainPrefix in ILoadBalancer
Update CloneDatabase and unit test callers, which are the only things
that are likely to call it.

Change-Id: Idbc6d2965cd3828c2c6d23012319bbf0fc3ffa4a
2018-11-26 10:38:15 -08:00
Aaron Schulz
1c205f1822 rdbms: add ILBFactory::redefineLocalDomain method
This is intended for use with scripts like addWiki.php to avoid mismatched domain errors.

Bug: T209483
Change-Id: Ie24f83f4e0fad7b01690d2055e1529a4cb0275b5
2018-11-22 23:51:44 -08:00
James D. Forrester
903e8b63de doc: Modernise parameter names and documentation for 'replica' DBs
Non-breaking change. Remaining uses are public interfaces (a constant, two
globals, a config sub-parameter, SQL queries, storage function names), one i18n
message key, and a whole lot of maintenance scripts with calls to the deprecated
function wfWaitForSlaves().

Change-Id: I6ee5ca92ccf6a80c08f53d9efe38ebb4b05064d7
2018-10-31 10:36:48 -07:00
Aaron Schulz
fe0af6cad5 rdbms: Database::selectDB() update the domain and handle failure better
LoadBalancer uses Database::getDomainId() for deciding which keys to use
in the foreign connection handle arrays. This method should reflect any
changes made to the DB selection.

If the query fails, then do not change domain field. This is the sort of
approach that LoadBalancer is expects in openForeignConnection(). Also,
throw an exception when selectDB() fails.

The db/schema/prefix fields of Database no longer exist in favor of just
using the newer currentDomain field.

Also:
* Add IDatabase::selectDomain() method and made selectDB() wrap it.
* Extract the DB name from sqlite files if not explicitly provided.
* Fix inconsistent open() return values from Database subclasses.
* Make a relationSchemaQualifier() method to handle the concern of
  omitting schema names in queries. The means that getDomainId() can
  still return the right value, rather than confusingly omitt the schema.
* Make RevisionStore::checkDatabaseWikiId() account for the domain schema.
  Unlike d2a4d614fc, this does not incorrectly assume the storage is
  always for the current wiki domain. Also, LBFactorySingle sets the local
  domain so it is defined even in install.php.
* Make RevisionStoreDbTestBase actually set the LoadBalancer local domain.
* Make RevisionTest::testLoadFromTitle() account for the domain schema.

Bug: T193565
Change-Id: I6e51cd54c6da78830b38906b8c46789c79498ab5
2018-10-10 12:03:30 -07:00
Reedy
71badb429e Remove global namespaced LBFactory
Change-Id: Ia87aa5c9426af83627a1bbc9bd914546c89c1e47
2018-09-21 15:42:42 +00:00
Reedy
1c5399b471 tests: Stop using deprecated LBFactory, use namespaced version
Change-Id: Ifbbbfe1bc58e57739b88588e3b6feb703f20534d
2018-09-21 02:37:12 +00:00
Kunal Mehta
b92565ca18 Disable PHPUnit tests that fail under sqlite
So we can make the job voting, preventing other tests from regressing.
These tests can be re-enabled whenever they're made to pass.

Change-Id: I959710138e1e1b37b6ba69561c2920a78208bf12
2018-08-01 00:19:43 -07:00
Aaron Schulz
2fd62f5dc9 rdbms: add resolveDomainID() method to LBFactory/LoadBalancer
Also add LBFactory::getLocalDomainID to match the one in LoadBalancer

Change-Id: Ia31f0800bd3b692194c08b1eab9cfb2f43679c7a
2018-07-05 14:07:04 +01:00
Aaron Schulz
44b47b43ee rdbms: make getCPInfoFromCookieValue() stricter about allowed values
All components, not just the write index, must now be present.

Bug: T194403
Change-Id: I279ba3e16d470aca09fdb74cec91d28efb5e2f95
2018-06-12 18:18:54 +00:00
Aaron Schulz
fb51330084 rdbms: include client ID hash in ChronologyProtector cookies
Previously, if an internal service forwarded the cookies for a
user (e.g. for permissions) but not the User-Agent header or not
the IP address (e.g. XFF), ChronologyProtector could timeout
waiting for a matching writeIndex to appear for the wrong key.

The cookie now tethers the client to the key that holds the
DB positions from their last state-changing request.

Bug: T194403
Bug: T190082
Change-Id: I84f2cbea82532d911cdfed14644008894498813a
2018-06-02 03:57:30 +00:00