Commit graph

514 commits

Author SHA1 Message Date
Aaron Schulz
a129edcc5e Made DatabaseSqlite::__construct always caller super
* Also fixed misuse of private mTrxAtomicLevels var

Change-Id: I711508cb3906a5192be1a244a7e511b1720141ca
2015-04-28 08:37:21 -07:00
jenkins-bot
661f171575 Merge "Added $wgJobSerialCommitThreshold setting" 2015-04-27 08:29:46 +00:00
Aaron Schulz
7c821caef5 Added $wgJobSerialCommitThreshold setting
* This is used to avoid lag by certain jobs

Bug: T95501
Change-Id: Id707c9a840fa23d56407e03aaae4e25149a1f906
2015-04-24 11:38:16 -07:00
Aaron Schulz
fe72859716 Allow setting MySQL session variables via config
Change-Id: I39c7e0f5ffe53a3fb15da489f4dddc36af99a4b3
2015-04-24 11:07:02 -07:00
Aaron Schulz
4dd8b5cf45 Added a simple pendingWriteQueryDuration() DB method
Change-Id: I924cf78d8bb96d526a7ba7444f0532d7eb223bf2
2015-04-23 03:13:40 +00:00
jenkins-bot
dd7df34a22 Merge "Clean up handling of 'infinity'" 2015-04-14 18:57:16 +00:00
Aaron Schulz
b6ef721900 Reduce begin/commit activity in startAtomic with DBO_TRX
Change-Id: I6c1334967aeff54fb18091c23749fcb0b49624b9
2015-04-09 15:49:15 -07:00
Aaron Schulz
419221e4a7 Fixed bug where catching DB errors left ignoreErrors() on
* Also fixed an actual rollback loop possible in postgres

Change-Id: I41508127f74e1bbee4c020546fed85ab53318ab7
2015-04-07 18:12:06 +00:00
Aaron Schulz
4d58457784 Added slave/master fallback logic in Revision
* This is a more specific form of the logic removed in 3c2bc32ae1.
  It does not suffer the problem of causing constant master DB
  queries due to a bad template reference or such.
* It will use the master if writes from the current thread
  are pending or were recently committed. This deals with the
  common problem of code that needs to read things it just wrote,
  such as diffs on rollback or edit hooks.
* This commit reverts 8624e261f by making the hack obsolete.

Bug: T93866
Bug: T94407
Change-Id: Ib9ecb75e1236e767bdc86d124d5e22a03ae0fb5f
2015-03-31 23:39:28 +00:00
Brad Jorsch
ac6f81d9ad Clean up handling of 'infinity'
There's a bunch of stuff that probably only works because the database
representation of infinity is actually 'infinity' on all databases
besides Oracle, and Oracle in general isn't maintained.

Generally, we should probably use 'infinity' everywhere except where
directly dealing with the database.

* Many extension callers of Language::formatExpiry() with $format !==
  true are assuming it'll return 'infinity', none are checking for
  $db->getInfinity().
* And Language::formatExpiry() would choke if passed 'infinity', despite
  callers doing this.
* And Language::formatExpiry() could be more useful for the API if we
  can override the string returned for infinity.
* As for core, Title is using Language::formatExpiry() with TS_MW which
  is going to be changing anyway. Extension callers mostly don't exist.
* Block already normalizes its mExpiry field (and ->getExpiry()),
  but some stuff is comparing it with $db->getInfinity() anyway. A few
  external users set mExpiry to $db->getInfinity(), but this is mostly
  because SpecialBlock::parseExpiryInput() returns $db->getInfinity()
  while most callers (including all extensions) are assuming 'infinity'.
* And for that matter, Block should use $db->decodeExpiry() instead of
  manually doing it, once we make that safe to call with 'infinity' for
  all the extensions passing $db->getInfinity() to Block's contructor.
* WikiPage::doUpdateRestrictions() and some of its callers are using
  $db->getInfinity(), when all the inserts using that value are using
  $db->encodeExpiry() which will convert 'infinity'.

This also cleans up a slave-lag issue I noticed in ApiBlock while
testing.

Bug: T92550
Change-Id: I5eb68c1fb6029da8289276ecf7c81330575029ef
2015-03-13 11:19:53 -04:00
Aaron Schulz
c39a4530b0 Made master connection expectations actually work
Bug: –T86862
Change-Id: If918904c6c02ea83473dfaf34dfc2787ab610c6f
2015-03-03 13:18:18 -08:00
Aaron Schulz
61df5e785f Made SqlBagOStuff avoid tripping TransactionProfiler
* Set a custom profiler with no expectations to avoid tripping
  the "0 write" expection. This avoids useless log entries.

Change-Id: Iac849a729eb36b1a8affb0dbc8b8c195fab4b03a
2015-02-27 17:45:48 +00:00
jenkins-bot
bd5fd843d6 Merge "Always decode Blob objects from Database::addQuotes" 2015-02-25 21:17:06 +00:00
jenkins-bot
19d6957824 Merge "DatabaseBase constructor cleanups" 2015-02-23 21:18:23 +00:00
Aaron Schulz
2cfcb390c6 DatabaseBase constructor cleanups
* The array based format is now required
* Pass any additional parameters through factory() to __construct() even
  if the base clase does not recognize them. This makes DB specific
  options simpler.
* Also added missing bits to Sqlite constructors

Bug: T90288
Change-Id: I29b79a3c1bb2b5e51c1c8f5e04cd08c71e0662a3
2015-02-23 13:02:28 -08:00
Kevin Israel
3c16633c4f DatabaseBase: Document behavior of nulls in array conditions
Follows-up 26235c7337.

Change-Id: Id008557e53a8c27f49e87cb2120298ce6478b6df
2015-02-21 23:47:43 -05:00
Aaron Schulz
7df76bae3b Made TransactionProfiler log regardless of Profiler
* Also avoid the use of slow closures for profiling

Change-Id: Id437e39038fd9ba80b899329a61e9f14229481f2
2015-02-20 02:11:11 +00:00
Aaron Schulz
fc22415ba9 Pass the right variable to recordQueryCompletion()
Change-Id: I0d75ed7dd55e40b63bd823ec43b8d3a8f5bed53b
2015-02-15 09:31:00 +00:00
Tim Starling
88982d4ccf Fix spammy "Writes done" debug message
Was broken by I1a8830eaf, which accidentally caused the log message to
be output for every write query, instead of only the first write query.

Change-Id: I4a53914bcb28804ff4c31b4e0418591793d411b6
2015-02-13 15:59:37 +11:00
jenkins-bot
c3d802cc30 Merge "Added query/connection expectation support to TransactionProfiler" 2015-02-12 21:38:58 +00:00
Aaron Schulz
505394a5a9 Added query/connection expectation support to TransactionProfiler
* Master connections and writes on read requests will now be logged to
  get visibility into code that reduces performance and site availability.

bug: T88445
bug: T86862
Change-Id: Ic2ae95c33facbb54e062aef2ce67d6182caa044a
2015-02-11 18:57:03 +00:00
jenkins-bot
23650785ef Merge "Moved RecentChange::purgeExpiredChanges to a job" 2015-02-10 21:32:55 +00:00
Kevin Israel
5f3957498b Database: Cast to int in estimateRowCount(), selectRowCount()
Doc comments state that these methods return ints. In order to ensure
that, values must be cast to int before they are returned.

With respect to selectRowCount(), follows-up 65f81d2843.

Change-Id: I108221ce4ad1b5b103b015fe875de54e04781741
2015-02-06 05:03:45 -05:00
Aaron Schulz
52a5954068 Moved "large write query" code to TransactionProfiler
Change-Id: Ic05e832eb21545a4e639b52aca7b3a5811a890ce
2015-02-05 01:42:25 +00:00
Erik Bernhardson
7fd481c770 Always decode Blob objects from Database::addQuotes
The current API for Database::encodeBlob/Database::addQuotes requires
the code that is outputting binary data to have a database handle, so
that it may call Database::encodeBlob to get either a plain string or
a Blob object back. All database implementations other than MySQL
return a Blob object from Database::encodeBlob.

This is a rather inconvenient API, it tightly couples the creation of
binary data with the Database object unnecessarily.  If all database
objects accept a Blob via Database::addQuotes then code can simply
wrap its arguments in Blob and know that any database it ends up at
will be properly handled.

This patch changes the default implementation of Database::addQuotes
to recognize a Blob object was passed in, and use Blob::fetch to turn
it back into a string.  Database implementations other than MySQL all
handle this Blob object already.  The postgresql implementation had
to be adjusted slightly. Now when it sees a Blob object that it did
not create it will encode that appropriately.

Bug: 72367
Change-Id: I12fb4bd339be19137fffba2e47a70741382f6a8c
2015-02-02 17:43:12 +00:00
Aaron Schulz
99e6b43ab8 Moved RecentChange::purgeExpiredChanges to a job
* Also added a selectFieldValues helper method to the DB classes
  since this use case keeps coming up.

Change-Id: I62cdbb497dc2c8fe4758e756d13688b85165e869
2015-01-16 17:39:31 -08:00
Aaron Schulz
6921770414 Updated some try-catch statements: MWException -> Exception
Change-Id: I76601a86e30f4984e3b1a8c8ec5ef5a0f652433a
2015-01-09 17:20:22 -08:00
Sumit Asthana
26235c7337 Database::makeList() : Handle NULL when building 'IN' clause
Amends to Database::makeList() to check if 'NULL' is present in array while
building IN clause. Presently a NULL in array intended to build IN clause, is
treated as a usual value. This creates
problem:https://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#function_in
Therefore null is appended at the end of the IN clause as " $field IS NULL"
rather than within the IN clause.
DatabaseSQLTest.php: Some test cases added for the modified behaviour.

Bug: T50853
Change-Id: Ic255c37e9702fd44d8173027ab8036ddc48f9d2e
2015-01-03 22:20:35 +05:30
Aaron Schulz
61e23c0578 Cleanup DatabaseBase::query implicit transaction code
* Add DatabaseBase::isTransactableQuery() for checking whether a query could
  benefit from being executed in a transaction.

Change-Id: Ie5b116bc726b47c68459e6525a1bb43b96bd9f30
2014-12-25 22:43:57 +00:00
Reedy
4d9143c7f5 Add lots of @throws
Change-Id: I09d0c13070f966fcf23d2638d8fc1328279a5995
2014-12-24 13:49:20 +00:00
Aaron Schulz
8029513503 Use generalizeSQL for DBPerformance log entries
Change-Id: I6612e01ba6382bcdc1642848009f6b8e13bd79f2
2014-12-18 15:00:57 -08:00
Aaron Schulz
9c246d1019 Log to DBPerformance when queries affect many rows
Change-Id: If82b596187b638888760f800bba2fea20fc7ae8a
2014-12-12 19:35:57 +00:00
Tim Starling
90fd90d8ce Fix ProfilerStandard sorting and a profile error
* Sort $this->collated even if collateOnly is set. Also I don't think
  arsort works that way.
* Fix a profiling error in every DB query, which was due to the two
  scoped profiling sections being destroyed in the wrong order.

Change-Id: I6af05f37a5c0391acfa80d54ecbca7a08ad81250
2014-12-11 18:14:13 +00:00
Aaron Schulz
a13b950a40 Added some abstract methods to DatabaseBase
Change-Id: I75781ee55d48d9fda40d600228100708d6524df8
2014-12-09 16:22:59 -08:00
umherirrender
489d793882 Fixed spacing
- Added/removed spaces around parenthesis
- Added newline in empty blocks
- Added space after switch/foreach/function
- Use tabs at begin of line
- Add newline at end of file

Change-Id: I244cdb2c333489e1020931bf4ac5266a87439f0d
2014-12-05 22:28:07 +01:00
Marius Hoch
3d3d36fcdd Fix DatabaseBase::replaceVars for PHP 5.3
Change-Id: I78a535abfe3f444b2ccb99638a63b6f16c9f18af
2014-12-04 23:14:00 +01:00
Chad Horohoe
52f3b13312 Remove Profiler::isStub()
Just check for ProfilerStub directly.

Change-Id: I503916599f182df4206da5282193ae6ec9324ee6
2014-12-03 20:19:56 -08:00
Aaron Schulz
d5fa948648 Removed low signal wfDebugLog call in DatabaseBase
Change-Id: I231766b6a1a1651508bb03fa137e6b2b55c55e6a
2014-12-03 14:09:33 -08:00
Aaron Schulz
efde278c82 Log backtraces for DB reconnections
Change-Id: Ibe7badde80bf6a44e292b89fe6c085cbda5748e2
2014-12-03 21:55:26 +00:00
Kevin Israel
c8c1023596 Database: Remove use of strencode() in replaceVars()
Almost all known uses of the /*$var*/ syntax are for either the
table prefix or table options, neither of which need string
encoding. Use of that syntax inside strings has been discouraged
for a while anyway.

This is to allow deprecating strencode().

Also changed the method to replace in one pass (to not find
variables inside values of other variables).

Change-Id: Ia697ce5f14082e737b3e1c4e174bd3f6223acc5c
2014-12-03 05:08:17 -05:00
Aaron Schulz
240152117c Added custom frame support to Profiler
* Made use of it in the DatabaseBase classes
* For the xhprof class, this only works in HHVM for now

Change-Id: I95d7cc128d4a770328fbdd2b546972d3fc2e2e8a
2014-11-25 14:52:21 -07:00
Aaron Schulz
5b19c6e54d Removed unused $trx variable
Change-Id: Ie4bc7c32c09f80dccf45946569cbc1ec5f136bdb
2014-11-22 08:18:12 +00:00
Aaron Schulz
be8b4d4d28 TransactionProfiler now shows the delay periods between queries
* Also tweaked the slave query display threshold to show them more

Change-Id: I0d9002b2f12e846a7355992443287d4ec43a7e88
2014-11-21 20:46:41 +00:00
Aaron Schulz
315ffffcaa Avoid sending empty function names to TransactionProfiler
* This could happen if the Profiler was a stub

Change-Id: Id4f635f899ee3c0ca85acf1b1e4cde47dcbbdc75
2014-11-19 11:39:00 -08:00
Bryan Davis
2792ea9783 Add logging context to database logs
* Add optional $context parameter to wfDebug, wfDebugLog, wfLogDBError
  and wfErrorLog that will be passed to MWLogger.
* Add support for PSR-3 style log message parameter interpolation in
  MWLoggerLegacyLogger.
* Add context information to wfLogDBError calls made from DatabaseBase,
  DatabaseMysqlBase and LoadBalancer instances.
* Deprecate wfDebugTimer() which now appears to be unused.

Change-Id: Ic90d593d00a2b0b5b80ed205908cbe624042603c
2014-11-19 17:59:55 +00:00
Aaron Schulz
db5d7a807f Removed incomplete/unused DatabaseType interface
Change-Id: I95b64f0a74f0203092464c4900fef2ac97e3ba35
2014-11-09 22:56:23 -08:00
Aaron Schulz
671bcd8d27 Changed TransactionProfiler to only work via the DB classes
* This makes it work with xhprof/xdebug, though it will only examine DB queries.

Change-Id: I999b66e6c3af88dda408b2252c248e085af9eea4
2014-11-06 10:30:51 -08:00
Sergio Santoro
10dcbd6794 DatabaseBase::tableName() fix limit while invoking explode()
Limit for explode() needs to be set to 3 to allow entering
branch at line 2350.

Change-Id: Idc2ba81a4bafa3e5511982db73a5c1549f1f9ef0
2014-10-15 23:14:11 +02:00
Aaron Schulz
958a13b025 Created DatabaseBase::trxTimestamp method and added usage in ActiveUsers query
Change-Id: Iba3c83ce8010583908f9dbaa8348f7226f524793
2014-09-24 22:38:36 +00:00
Aaron Schulz
fe029f8777 Added DatabaseBase::selectRowCount() method
* Useful for doing COUNT(*) with a LIMIT for performance
* Made MergeHistory use it

Change-Id: If1cfc52ea22fd09d34915c29b45d0b487d28d15a
2014-09-05 13:11:58 -07:00