Thijs/wiki.techinc.nl
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Configurable log sampling via $wgDebugLogGroups

Browse source 
Make it possible to configure a log-group-specific sampling factor by allowing
$wgDebugLogGroups values to be associative arrays with 'destination' & 'sample'
keys.

The immediate use-case I have in mind is the memcached log, which is too
verbose to enable in production. We could sample on the receiving end, but
doing so would not help MediaWiki, which would still be on the hook for
processing and dispatching every single record.

Idea was vetted by Dr. Manhattan:

<TimStarling> I think you could have each element in $wgDebugLogGroups be an
  associative array
<TimStarling> e.g. array( 'sample' => 1000, 'target' => "$host:$port")
<TimStarling> that would allow for more features to be added to it in
  future

Change-Id: Ib0ece5d5d9c2aadef13b77ac38995b0e47ac086f
This commit is contained in:
Ori Livneh 2013-11-05 11:42:49 -08:00
parent 0d1beb5d13
commit 2c25e14642
2 changed files with 47 additions and 9 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

21
includes/DefaultSettings.php
View file

@ -4895,10 +4895,29 @@ $wgDebugDBTransactions = false;
$wgDebugDumpSql = false;
/**
* Set to an array of log group keys to filenames.
* Map of string log group names to log destinations.
*
* If set, wfDebugLog() output for that group will go to that file instead
* of the regular $wgDebugLogFile. Useful for enabling selective logging
* in production.
*
* Log destinations may be string values specifying a filename or URI, or they
* may be filename or an associative array mapping 'target' to the desired
* filename. The associative array may also contain a 'sample' key with an
* integer value, specifying a sampling factor.
*
* @par Example:
* @code
* $wgDebugLogGroups['redis'] = '/var/log/mediawiki/redis.log';
* @endcode
*
* @par Advanced example:
* @code
* $wgDebugLogGroups['memcached'] = (
* 'target' => '/var/log/mediawiki/memcached.log',
* 'sample' => 1000, // log 1 message out of every 1,000.
* );
* @endcode
*/
$wgDebugLogGroups = array();

35
includes/GlobalFunctions.php
View file

@ -1008,7 +1008,12 @@ function wfDebugMem( $exact = false ) {
/**
* Send a line to a supplementary debug log file, if configured, or main debug log if not.
* $wgDebugLogGroups[$logGroup] should be set to a filename to send to a separate log.
* To configure a supplementary log file, set $wgDebugLogGroups[$logGroup] to a string
* filename or an associative array mapping 'target' to the desired filename. The
* associative array may also contain a 'sample' key with an integer value, specifying
* a sampling factor.
*
* @since 1.23 support for sampling log messages via $wgDebugLogGroups.
*
* @param $logGroup String
* @param $text String
@ -1018,14 +1023,28 @@ function wfDebugMem( $exact = false ) {
function wfDebugLog( $logGroup, $text, $public = true ) {
global $wgDebugLogGroups;
$text = trim( $text ) . "\n";
if ( isset( $wgDebugLogGroups[$logGroup] ) ) {
$time = wfTimestamp( TS_DB );
$wiki = wfWikiID();
$host = wfHostname();
wfErrorLog( "$time $host $wiki: $text", $wgDebugLogGroups[$logGroup] );
} elseif ( $public === true ) {
wfDebug( "[$logGroup] $text", false );
if ( !isset( $wgDebugLogGroups[$logGroup] ) ) {
if ( $public === true ) {
wfDebug( "[$logGroup] $text", false );
}
return;
}
$logConfig = $wgDebugLogGroups[$logGroup];
if ( is_array( $logConfig ) ) {
if ( isset( $logConfig['sample'] ) && mt_rand( 1, $logConfig['sample'] ) !== 1 ) {
return;
}
$destination = $logConfig['destination'];
} else {
$destination = strval( $logConfig );
}
$time = wfTimestamp( TS_DB );
$wiki = wfWikiID();
$host = wfHostname();
wfErrorLog( "$time $host $wiki: $text", $destination );
}
/**