Commit graph

504 commits

Author SHA1 Message Date
Aaron Schulz
c39a4530b0 Made master connection expectations actually work
Bug: –T86862
Change-Id: If918904c6c02ea83473dfaf34dfc2787ab610c6f
2015-03-03 13:18:18 -08:00
Aaron Schulz
61df5e785f Made SqlBagOStuff avoid tripping TransactionProfiler
* Set a custom profiler with no expectations to avoid tripping
  the "0 write" expection. This avoids useless log entries.

Change-Id: Iac849a729eb36b1a8affb0dbc8b8c195fab4b03a
2015-02-27 17:45:48 +00:00
jenkins-bot
bd5fd843d6 Merge "Always decode Blob objects from Database::addQuotes" 2015-02-25 21:17:06 +00:00
jenkins-bot
19d6957824 Merge "DatabaseBase constructor cleanups" 2015-02-23 21:18:23 +00:00
Aaron Schulz
2cfcb390c6 DatabaseBase constructor cleanups
* The array based format is now required
* Pass any additional parameters through factory() to __construct() even
  if the base clase does not recognize them. This makes DB specific
  options simpler.
* Also added missing bits to Sqlite constructors

Bug: T90288
Change-Id: I29b79a3c1bb2b5e51c1c8f5e04cd08c71e0662a3
2015-02-23 13:02:28 -08:00
Kevin Israel
3c16633c4f DatabaseBase: Document behavior of nulls in array conditions
Follows-up 26235c7337.

Change-Id: Id008557e53a8c27f49e87cb2120298ce6478b6df
2015-02-21 23:47:43 -05:00
Aaron Schulz
7df76bae3b Made TransactionProfiler log regardless of Profiler
* Also avoid the use of slow closures for profiling

Change-Id: Id437e39038fd9ba80b899329a61e9f14229481f2
2015-02-20 02:11:11 +00:00
Aaron Schulz
fc22415ba9 Pass the right variable to recordQueryCompletion()
Change-Id: I0d75ed7dd55e40b63bd823ec43b8d3a8f5bed53b
2015-02-15 09:31:00 +00:00
Tim Starling
88982d4ccf Fix spammy "Writes done" debug message
Was broken by I1a8830eaf, which accidentally caused the log message to
be output for every write query, instead of only the first write query.

Change-Id: I4a53914bcb28804ff4c31b4e0418591793d411b6
2015-02-13 15:59:37 +11:00
jenkins-bot
c3d802cc30 Merge "Added query/connection expectation support to TransactionProfiler" 2015-02-12 21:38:58 +00:00
Aaron Schulz
505394a5a9 Added query/connection expectation support to TransactionProfiler
* Master connections and writes on read requests will now be logged to
  get visibility into code that reduces performance and site availability.

bug: T88445
bug: T86862
Change-Id: Ic2ae95c33facbb54e062aef2ce67d6182caa044a
2015-02-11 18:57:03 +00:00
jenkins-bot
23650785ef Merge "Moved RecentChange::purgeExpiredChanges to a job" 2015-02-10 21:32:55 +00:00
Kevin Israel
5f3957498b Database: Cast to int in estimateRowCount(), selectRowCount()
Doc comments state that these methods return ints. In order to ensure
that, values must be cast to int before they are returned.

With respect to selectRowCount(), follows-up 65f81d2843.

Change-Id: I108221ce4ad1b5b103b015fe875de54e04781741
2015-02-06 05:03:45 -05:00
Aaron Schulz
52a5954068 Moved "large write query" code to TransactionProfiler
Change-Id: Ic05e832eb21545a4e639b52aca7b3a5811a890ce
2015-02-05 01:42:25 +00:00
Erik Bernhardson
7fd481c770 Always decode Blob objects from Database::addQuotes
The current API for Database::encodeBlob/Database::addQuotes requires
the code that is outputting binary data to have a database handle, so
that it may call Database::encodeBlob to get either a plain string or
a Blob object back. All database implementations other than MySQL
return a Blob object from Database::encodeBlob.

This is a rather inconvenient API, it tightly couples the creation of
binary data with the Database object unnecessarily.  If all database
objects accept a Blob via Database::addQuotes then code can simply
wrap its arguments in Blob and know that any database it ends up at
will be properly handled.

This patch changes the default implementation of Database::addQuotes
to recognize a Blob object was passed in, and use Blob::fetch to turn
it back into a string.  Database implementations other than MySQL all
handle this Blob object already.  The postgresql implementation had
to be adjusted slightly. Now when it sees a Blob object that it did
not create it will encode that appropriately.

Bug: 72367
Change-Id: I12fb4bd339be19137fffba2e47a70741382f6a8c
2015-02-02 17:43:12 +00:00
Aaron Schulz
99e6b43ab8 Moved RecentChange::purgeExpiredChanges to a job
* Also added a selectFieldValues helper method to the DB classes
  since this use case keeps coming up.

Change-Id: I62cdbb497dc2c8fe4758e756d13688b85165e869
2015-01-16 17:39:31 -08:00
Aaron Schulz
6921770414 Updated some try-catch statements: MWException -> Exception
Change-Id: I76601a86e30f4984e3b1a8c8ec5ef5a0f652433a
2015-01-09 17:20:22 -08:00
Sumit Asthana
26235c7337 Database::makeList() : Handle NULL when building 'IN' clause
Amends to Database::makeList() to check if 'NULL' is present in array while
building IN clause. Presently a NULL in array intended to build IN clause, is
treated as a usual value. This creates
problem:https://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#function_in
Therefore null is appended at the end of the IN clause as " $field IS NULL"
rather than within the IN clause.
DatabaseSQLTest.php: Some test cases added for the modified behaviour.

Bug: T50853
Change-Id: Ic255c37e9702fd44d8173027ab8036ddc48f9d2e
2015-01-03 22:20:35 +05:30
Aaron Schulz
61e23c0578 Cleanup DatabaseBase::query implicit transaction code
* Add DatabaseBase::isTransactableQuery() for checking whether a query could
  benefit from being executed in a transaction.

Change-Id: Ie5b116bc726b47c68459e6525a1bb43b96bd9f30
2014-12-25 22:43:57 +00:00
Reedy
4d9143c7f5 Add lots of @throws
Change-Id: I09d0c13070f966fcf23d2638d8fc1328279a5995
2014-12-24 13:49:20 +00:00
Aaron Schulz
8029513503 Use generalizeSQL for DBPerformance log entries
Change-Id: I6612e01ba6382bcdc1642848009f6b8e13bd79f2
2014-12-18 15:00:57 -08:00
Aaron Schulz
9c246d1019 Log to DBPerformance when queries affect many rows
Change-Id: If82b596187b638888760f800bba2fea20fc7ae8a
2014-12-12 19:35:57 +00:00
Tim Starling
90fd90d8ce Fix ProfilerStandard sorting and a profile error
* Sort $this->collated even if collateOnly is set. Also I don't think
  arsort works that way.
* Fix a profiling error in every DB query, which was due to the two
  scoped profiling sections being destroyed in the wrong order.

Change-Id: I6af05f37a5c0391acfa80d54ecbca7a08ad81250
2014-12-11 18:14:13 +00:00
Aaron Schulz
a13b950a40 Added some abstract methods to DatabaseBase
Change-Id: I75781ee55d48d9fda40d600228100708d6524df8
2014-12-09 16:22:59 -08:00
umherirrender
489d793882 Fixed spacing
- Added/removed spaces around parenthesis
- Added newline in empty blocks
- Added space after switch/foreach/function
- Use tabs at begin of line
- Add newline at end of file

Change-Id: I244cdb2c333489e1020931bf4ac5266a87439f0d
2014-12-05 22:28:07 +01:00
Marius Hoch
3d3d36fcdd Fix DatabaseBase::replaceVars for PHP 5.3
Change-Id: I78a535abfe3f444b2ccb99638a63b6f16c9f18af
2014-12-04 23:14:00 +01:00
Chad Horohoe
52f3b13312 Remove Profiler::isStub()
Just check for ProfilerStub directly.

Change-Id: I503916599f182df4206da5282193ae6ec9324ee6
2014-12-03 20:19:56 -08:00
Aaron Schulz
d5fa948648 Removed low signal wfDebugLog call in DatabaseBase
Change-Id: I231766b6a1a1651508bb03fa137e6b2b55c55e6a
2014-12-03 14:09:33 -08:00
Aaron Schulz
efde278c82 Log backtraces for DB reconnections
Change-Id: Ibe7badde80bf6a44e292b89fe6c085cbda5748e2
2014-12-03 21:55:26 +00:00
Kevin Israel
c8c1023596 Database: Remove use of strencode() in replaceVars()
Almost all known uses of the /*$var*/ syntax are for either the
table prefix or table options, neither of which need string
encoding. Use of that syntax inside strings has been discouraged
for a while anyway.

This is to allow deprecating strencode().

Also changed the method to replace in one pass (to not find
variables inside values of other variables).

Change-Id: Ia697ce5f14082e737b3e1c4e174bd3f6223acc5c
2014-12-03 05:08:17 -05:00
Aaron Schulz
240152117c Added custom frame support to Profiler
* Made use of it in the DatabaseBase classes
* For the xhprof class, this only works in HHVM for now

Change-Id: I95d7cc128d4a770328fbdd2b546972d3fc2e2e8a
2014-11-25 14:52:21 -07:00
Aaron Schulz
5b19c6e54d Removed unused $trx variable
Change-Id: Ie4bc7c32c09f80dccf45946569cbc1ec5f136bdb
2014-11-22 08:18:12 +00:00
Aaron Schulz
be8b4d4d28 TransactionProfiler now shows the delay periods between queries
* Also tweaked the slave query display threshold to show them more

Change-Id: I0d9002b2f12e846a7355992443287d4ec43a7e88
2014-11-21 20:46:41 +00:00
Aaron Schulz
315ffffcaa Avoid sending empty function names to TransactionProfiler
* This could happen if the Profiler was a stub

Change-Id: Id4f635f899ee3c0ca85acf1b1e4cde47dcbbdc75
2014-11-19 11:39:00 -08:00
Bryan Davis
2792ea9783 Add logging context to database logs
* Add optional $context parameter to wfDebug, wfDebugLog, wfLogDBError
  and wfErrorLog that will be passed to MWLogger.
* Add support for PSR-3 style log message parameter interpolation in
  MWLoggerLegacyLogger.
* Add context information to wfLogDBError calls made from DatabaseBase,
  DatabaseMysqlBase and LoadBalancer instances.
* Deprecate wfDebugTimer() which now appears to be unused.

Change-Id: Ic90d593d00a2b0b5b80ed205908cbe624042603c
2014-11-19 17:59:55 +00:00
Aaron Schulz
db5d7a807f Removed incomplete/unused DatabaseType interface
Change-Id: I95b64f0a74f0203092464c4900fef2ac97e3ba35
2014-11-09 22:56:23 -08:00
Aaron Schulz
671bcd8d27 Changed TransactionProfiler to only work via the DB classes
* This makes it work with xhprof/xdebug, though it will only examine DB queries.

Change-Id: I999b66e6c3af88dda408b2252c248e085af9eea4
2014-11-06 10:30:51 -08:00
Sergio Santoro
10dcbd6794 DatabaseBase::tableName() fix limit while invoking explode()
Limit for explode() needs to be set to 3 to allow entering
branch at line 2350.

Change-Id: Idc2ba81a4bafa3e5511982db73a5c1549f1f9ef0
2014-10-15 23:14:11 +02:00
Aaron Schulz
958a13b025 Created DatabaseBase::trxTimestamp method and added usage in ActiveUsers query
Change-Id: Iba3c83ce8010583908f9dbaa8348f7226f524793
2014-09-24 22:38:36 +00:00
Aaron Schulz
fe029f8777 Added DatabaseBase::selectRowCount() method
* Useful for doing COUNT(*) with a LIMIT for performance
* Made MergeHistory use it

Change-Id: If1cfc52ea22fd09d34915c29b45d0b487d28d15a
2014-09-05 13:11:58 -07:00
Aaron Schulz
60b6dd6ffe Fixed wrong comment in DatabaseBase::getFlag
Change-Id: Ic299f538b322a07c0ee66786bbe70cb48de3b22e
2014-08-28 23:13:42 -07:00
rillke
9665a657b0 Add "chemical" major MIME type to the image tables
The American Chemical Society suggested a new major MIME type for files
containing chemical data in 1998: http://dx.doi.org/10.1021/ci9803233
This suggestion got widely adopted and is now a de-facto-standard despite
not registered with IANA.

Applying this patch will allow us to continue with extension MolHandler
and PDBHandler.
http://fab.wmflabs.org/T352

- Fixes bug 66412 by creating a logic that will prevent running unneeded
  updates.

Bug: 66412
Change-Id: Ic45dc1bce796a0406ed8a84e6274df1c4bda4967
2014-08-25 19:13:35 -03:00
umherirrender
7c6a25856c Add missing @return to function docs
Change-Id: I45b9d02f94ecc58372268ec5e6a0b572a0b7e2a9
2014-08-23 23:14:57 +02:00
Aaron Schulz
9d03f95894 Fixed bogus mysql error code 0 on mid-transaction disconnect
* Now the proper error (e.g. "server has gone away") and code is shown.

Change-Id: If0255101151441c7d05141ffc7cb416ad79e6bd8
2014-08-21 09:58:03 -07:00
jenkins-bot
25b73462a1 Merge "Separate select() parameter summary from detail" 2014-08-01 22:49:36 +00:00
Aaron Schulz
cafcb6b225 Vastly increase how long wfWaitForSlaves() can block in CLI mode
* All maintenance scripts pretty much assumed this behavior, but it
  was not actually the case. This can avoid some massive lag problems.

Change-Id: I5894409d6abade16afd8144f6689c2aff3fa7327
2014-07-31 23:28:11 -07:00
Aaron Schulz
5e0ac5814d Log errors when several callbacks give exceptions in onTransaction* methods
Change-Id: I055f46d98c255749dd64b30bfa2db190ddc73d6b
2014-07-28 17:53:34 -07:00
umherirrender
02dc9da399 Cleanup some docs (includes/[a-d])
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling

Change-Id: I7b65fe04db431342cc58b469dc48f41a50c4e891
2014-07-24 19:42:45 +02:00
Brad Jorsch
053c3c5fd4 Add $wgDebugDumpSqlLength
When $wgDebugDumpSql is set, the logged queries are truncated to 500
bytes. This is often not enough if you're wanting to actually debug
the queries instead of just seeing that queries were executed.

Change-Id: Iad3abd20c11d647834aa5546a1a9c82916c6519f
2014-07-23 11:15:49 +00:00
jenkins-bot
4c0f5ab1da Merge "Leave table names with numbers untouched in DatabaseBase::generalizeSQL" 2014-07-21 14:47:35 +00:00