Commit graph

1476 commits

Author SHA1 Message Date
Kunal Mehta
4a8ae0a0b4 Use callable type-hint in a few places
Mostly places which immediately had a:
 if ( !is_callable( $callback ) ) {
  throw new Exception(...);
 }
check at the beginning of the function.

Change-Id: Ia78663b2231629010816bd1cda8814b996968d1d
2016-06-07 16:39:22 -07:00
Brion Vibber
8749184856 Add LBFactory::closeAll() method
Needed to allow long-running background tasks to close out database
connections in a way that allows them to reopen later when needed
again.

Bug: T97641
Change-Id: I4974b691392d4225b15e7b771a28a2cefeb9d8e4
2016-05-25 07:43:23 +00:00
jenkins-bot
9e23ffa395 Merge "Support non-replicating DB clusters for static datasets" 2016-05-24 23:59:24 +00:00
Aaron Schulz
e2c75bb3bd Do not count 0-load servers in getMaxLag()
JobRunner::run() and API endpoints use this to bail out if
lag is too high. A dedicated "vslow" host with no normal
load should not trigger this logic, for example.

Bug: T135809
Change-Id: If6879aa1a4379857779a4eacf2a8b7400c0b434f
2016-05-23 18:42:56 +00:00
Aaron Schulz
dad87efa5a Support non-replicating DB clusters for static datasets
* Make the slave wait methods work properly in this case if
  the config flag is set.
* Also fixed a small IDEA error.

Bug: T135690
Change-Id: I9a12f1fc54c0b426f64c19ac2bf720e683fcf2fe
2016-05-18 16:03:27 -07:00
jenkins-bot
2196031b72 Merge "Upgrade mismatched begin() warnings to exceptions" 2016-05-06 00:03:34 +00:00
jenkins-bot
f7dad57c64 Merge "Allow reset of global services (redux)." 2016-05-04 22:07:44 +00:00
daniel
d7410db0fd Allow reset of global services (redux).
(This is part of I6ec374ac9 wich was a re-submit of Ie98bf5af5
which got reverted by Ide7ab563)

This change provides a mechanism to reset global service instances
in an orderly manner. There are three use cases for this:

* the installation process
* integration tests (which most of the existing phpunit tests are)

In contrast to I6ec374ac9, this change does not cause singeltons
of legacy services to be reset. It is assumed that legacy services
use global state to access services and configuration, so any
change in confuguration would affect them immediately.

NOTE: the original I6ec374ac9 would cause session information to
get lost if the user session was creatsed before initialization
was complete. This was apparently triggered by the MobileFrontend
extension under some circumstances. Check with Addshore and Catrope.

Change-Id: Ie06782ffb96e675c0aa55dc26fb8f22037e8517d
2016-05-04 19:18:29 +02:00
Aaron Schulz
f8d5075656 Upgrade mismatched begin() warnings to exceptions
No WMF logs for these events in the last 7 days.

Change-Id: I79f2f2c8117ffa887a550a793a9e490c2a26f721
2016-05-03 19:17:51 +00:00
jenkins-bot
6a4c5f37dc Merge "Quote column name aliases" 2016-05-02 21:38:38 +00:00
jenkins-bot
2530a4c0a1 Merge "Update Microsoft SQL Server schema" 2016-05-02 21:34:50 +00:00
Ryan Schmidt
eaa91d7abf Quote column name aliases
Sometimes, a column name alias is chosen that is a reserved word or is
otherwise invalid (such as an alias named "user" on postgres or mssql).
Let's unconditionally quote the alias so that no matter what the user
specifies, it will not cause the query to error out.

Bug: T118973
Change-Id: If3ea7324980355e78afcdb3bf2596d23bb96412d
2016-05-02 21:22:59 +00:00
Ryan Schmidt
8efca2635d Update Microsoft SQL Server schema
The MSSQL schema is now brought in-line with the MySQL schema.
Additionally, various issues that prevented successful installation or
updates via MSSQL were fixed, notably with respect to creating bits of
the database should other bits already exist as well as issues with
previous updater patches not working correctly.

Additional MSSQL bugfixes will come in separate patches, as they are less
related to the schema/install/upgrade process.

Change-Id: If3eea625499d3cb14abba40f528208173067a53a
2016-05-02 13:02:36 -05:00
jenkins-bot
2f885ee6b7 Merge "Fix escaping for MSSQL LIKE queries." 2016-04-25 05:01:51 +00:00
U-REDMOND\emadelw
59136fdaf0 Fix escaping for MSSQL LIKE queries.
MSSQL allows for more operators than standard LIKE queries. In addition,
an ESCAPE clause must be specified in order to backslash-escapes to work.

Bug: T73207
Change-Id: Idadf9d56cadc48cf47d000598d8a3214c684f9d5
2016-04-25 04:51:32 +00:00
Brian Wolff
8974d19995 Modify documentation of select() method for Database.
* Add a note in Database.php that docs are in IDatabase.php,
maybe its just me, but I didn't realize that was where the docs
are (even though in retrospect that should have been obvious) and
wasted a lot of time because I didn't realize that.
* Change references to IDatabase::tableName() in doc to
DatabaseBase::tableName(), since tableName is not in the IDatabase
interface.
* Be explicit in docs for select() about which parts are safe for
user input. I think its important to be very explicit about this.

No code changes, just doc changes.

Change-Id: I3a66477afc6a5f54855062482ef04c2966e468f6
2016-04-17 16:20:50 -04:00
Ori.livneh
4cff45e67c Revert "Measure commitMasterChanges() run time"
This reverts commit efb4418839. This is the biggest metric
by statsd traffic volume on the Wikimedia cluster. Please re-introduce this
only with sampling.

Bug: T101141
Change-Id: I7bc3b3aaa98dcefe1a80a4f458a83232c29a27ea
2016-04-12 19:20:01 +00:00
Catrope
1162ed5f88 Revert "Allow reset of global services."
Completely breaks login.

This reverts commit 8e7a0a0912.

Change-Id: Ide7ab5632e987e81374c21173df6ab3998649df7
2016-04-11 13:40:28 -07:00
jenkins-bot
e523771f05 Merge "Add templateOverridesBySection to multi LBFactory" 2016-04-11 18:39:36 +00:00
daniel
8e7a0a0912 Allow reset of global services.
This change provides a mechanism to reset global service instances
in an orderly manner. There are three use cases for this:

* the installation process
* forking processes
* integration tests (which must of the existing phpunit tests are)

Depends-On: I5d638ad415fc3840186a0beaa09ac02ea688539b
Change-Id: Ie98bf5af59208f186dba59a9e971c72ea0b63e69
2016-04-11 16:22:40 +02:00
jenkins-bot
8eac2feedb Merge "Use pg_result_error, not pg_last_error, as the latter gives false negatives." 2016-04-08 23:56:05 +00:00
Max Semenik
943563062f Kill mbstring fallbacks
In the age when we require PHP 5.5, pretending that mbstring emulation
is not slow and silly is silly.

Bug: T129435
Change-Id: Ic8235c9da9a926df63ec7388900c44eab454eebe
2016-04-06 22:14:48 +00:00
Reedy
f6f40a0965 LoadBalancer::pickRandom() was removed
Bug: T122754
Change-Id: I85d0c4387195cb60c70e24a0a7ab338723e6e7f4
2016-04-02 01:31:50 +01:00
Aaron Schulz
6e38db159c Support custom query in pt-heartbeat lag detection
Bug: T111266
Change-Id: Ib0f9ca8072d9becebacfa8c9afe9d61567f73912
2016-04-01 23:41:14 +00:00
Aaron Schulz
2121abd764 Add templateOverridesBySection to multi LBFactory
This makes it easier to support pt-heartbeat with shard "channels"
since the channel is based on the section name.

Change-Id: Ie0128ee2914e8dc0063c759704373b2440df5ed8
2016-03-22 14:19:07 -07:00
Reedy
1834ee3d8e Fix numerous class/function casing
Change-Id: I23982bfa0548c9ea3bdb432be7982f1563930715
2016-03-18 23:14:49 +00:00
Jan Zerebecki
061e45aef7 DB: Correct argument documentation to match code.
Change-Id: I1cc3910932d8c4f04b1dbbc05ada88299c3ce10b
2016-03-08 19:39:49 +01:00
Siebrand Mazeland
5b119a0e44 Replace uses of join() by implode()
All of core uses implode() consistently now.

Change-Id: Iba50898c64c43f356d1caf8869f484e90d9ff651
2016-03-08 18:24:16 +00:00
jenkins-bot
9fe2ba22f7 Merge "Fixes to masterPosWait() for master switchovers" 2016-03-08 12:09:57 +00:00
Aaron Schulz
b64a178a21 Fixes to masterPosWait() for master switchovers
* Clean up return value types and docs.
* Handle master switch-over better w.r.t the job queue due
  to binlog name changes (the host portion). Previously the method
  would fail and trigger read-only mode when waiting on former
  master positions. Assume the the switch-over was done properly
  and thus return immediately.

Bug: T126436
Change-Id: Ib8c05a5c72d03a5c98e41b23c5653fc194b6d130
2016-03-08 03:37:44 -08:00
Bartosz Dziewoński
c161c46d26 Improve code suffering from PHP 5.3's lack of support for foo()[]
I searched for /\$(\S+) = (.+?\(.*?\);)\n.*?\$\1\[/, ignored
everything involving isset(), unset() or array assigments, then
skimmed through the remaining results and changed things where they
made sense. These changes were not automated, so please review them.

Change-Id: Ib37b4c66fc57648470f151ad412210b3629c2538
2016-02-28 22:49:20 +01:00
jenkins-bot
d2193f9d1d Merge "Ignore noop DB transactions errors on connection loss" 2016-02-26 16:09:11 +00:00
Matthew Flaschen
d4d086b0ce IDatabase::commit: Docs re 'flush' exception on explicit transaction
Change-Id: I44087e92d2186f4e931b3f0766caba1f90f4ebc4
2016-02-24 18:22:30 -05:00
Aaron Schulz
38f0c07279 Ignore noop DB transactions errors on connection loss
This avoids some needless exceptions for longer running scripts.

Bug: T127428
Change-Id: I3098141aa77a4c73cb2a0c784b38b15c712f82a9
2016-02-24 00:35:03 +00:00
jenkins-bot
d42754e477 Merge "database: Add extra sanity check to selectFieldValues()" 2016-02-18 14:46:29 +00:00
Aaron Schulz
ff23e83f41 database: Add extra sanity check to selectFieldValues()
Change-Id: Ia2788ea84bd099550655930188365984e5274678
2016-02-18 14:38:44 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Aaron Schulz
ef44f4308e Upgrade mismatched commit() warnings to exceptions
I saw no WMF logs for these in the last 7 days.

Change-Id: I725535925ba3fc68cee485767f18937fb8f239bf
2016-02-12 21:32:00 +00:00
Greg Sabino Mullane
290829ae0d Use pg_result_error, not pg_last_error, as the latter gives false negatives.
Specifically in the savepoint part of the code, where a failed query inside the
savepoint gives a non-false reply to pg_last_error by the time we call it,
which then leads to us doing 'RELEASE' instead of 'ROLLBACK TO', and causes
all sorts of mayhem.

The whole system needs a little cleanup, but this is an important bandaid.

Change-Id: I0223a3599816db5650f45863400e6194468d4361
2016-02-10 23:10:19 -05:00
Ricordisamoa
1e37987beb Follow-up to 02818c441c
* Remove obsolete @protected tag from docstring
  of DatabaseBase::indexName(), now really protected

* Make DatabaseSqlite::indexName() protected too

Change-Id: I90f894878379d0850dba61cd483af16864c209ea
2016-02-11 00:09:05 +01:00
Ricordisamoa
02818c441c Stop doing $that = $this in includes/db
Closures support $this as of PHP 5.4

Also made DatabaseBase::indexName() protected

Change-Id: Iff31e7d9186832a855a953b923ac182f1c66ffa0
2016-02-10 18:08:17 +01:00
Aaron Schulz
39ef7941b0 Style and commenting tweaks to CategoryMembershipChangeJob
Also renamed safeWaitForPos() => safeWaitForMasterPos()

Change-Id: Ic89e2a0b17cd6e6ef09cf703bbbcea1988a5bde9
2016-01-30 10:27:28 -08:00
jenkins-bot
c32764db69 Merge "Do not auto-reconnect to DBs if named locks where lost" 2016-01-30 18:07:35 +00:00
Aaron Schulz
79d12a284b Do not auto-reconnect to DBs if named locks where lost
Otherwise, transactions might be committed without cooperative
locks that were supposed to be present. This is similar to the
logic of not auto-reconnecting if a transaction was started to
avoid committing incomplete changes.

Change-Id: Ia7bc6b188bb5ee53a5bf7c5a30718bc7c4dd0ba9
2016-01-30 09:17:17 -08:00
Aaron Schulz
4e62537a8b Make sure getScopedLockAndFlush() commits before unlocking
This is necessary for callers to see changes from prior callers

Change-Id: I5e05215541e641b5b5a4bc55c91d6ec8ef774ca1
2016-01-30 08:52:06 -08:00
jenkins-bot
4ebf6581f5 Merge "Improve safeGetLag() return docs" 2016-01-30 13:35:39 +00:00
Aaron Schulz
b088acbb63 Improve safeGetLag() return docs
Change-Id: I7b2d0d96901412abe8bdf9abee39e6d545c94644
2016-01-29 15:09:59 -08:00
Aaron Schulz
982869d6b0 Fix IDEA error in safeWaitForPos()
Change-Id: Iecb023925177be05282f67e48056aeb8f2a9703c
2016-01-29 14:47:38 -08:00
Aaron Schulz
69c71cc45e Add LoadBalancer::safeWaitForPos()
This is useful for waiting on a specific slave handle to
reach a given master position.

Change-Id: Ia4569c4fc82c25eb4089cf7198f01e349636e2c7
2016-01-29 13:05:41 -08:00
Aaron Schulz
60013aa328 Add IDatabase::getScopedLockAndFlush() method
This method is less manual and avoids the usual pitfalls of
not unlocking for a return statement or not flushing out any
prior transaction.

Change-Id: Ib1681244767de860105a68210e181e2f024ee525
2016-01-29 20:32:05 +00:00