2004-02-18 02:15:00 +00:00
|
|
|
<?php
|
2004-09-02 23:28:24 +00:00
|
|
|
/**
|
2012-05-08 12:51:21 +00:00
|
|
|
* Localisation messages cache.
|
|
|
|
|
*
|
|
|
|
|
* This program is free software; you can redistribute it and/or modify
|
|
|
|
|
* it under the terms of the GNU General Public License as published by
|
|
|
|
|
* the Free Software Foundation; either version 2 of the License, or
|
|
|
|
|
* (at your option) any later version.
|
|
|
|
|
*
|
|
|
|
|
* This program is distributed in the hope that it will be useful,
|
|
|
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
|
* GNU General Public License for more details.
|
|
|
|
|
*
|
|
|
|
|
* You should have received a copy of the GNU General Public License along
|
|
|
|
|
* with this program; if not, write to the Free Software Foundation, Inc.,
|
|
|
|
|
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
|
|
|
|
|
* http://www.gnu.org/copyleft/gpl.html
|
|
|
|
|
*
|
WARNING: HUGE COMMIT
Doxygen documentation update:
* Changed alls @addtogroup to @ingroup. @addtogroup adds the comment to the group description, but doesn't add the file, class, function, ... to the group like @ingroup does. See for example http://svn.wikimedia.org/doc/group__SpecialPage.html where it's impossible to see related files, classes, ... that should belong to that group.
* Added @file to file description, it seems that it should be explicitely decalred for file descriptions, otherwise doxygen will think that the comment document the first class, variabled, function, ... that is in that file.
* Removed some empty comments
* Removed some ?>
Added following groups:
* ExternalStorage
* JobQueue
* MaintenanceLanguage
One more thing: there are still a lot of warnings when generating the doc.
2008-05-20 17:13:28 +00:00
|
|
|
* @file
|
|
|
|
|
* @ingroup Cache
|
2004-09-02 23:28:24 +00:00
|
|
|
*/
|
2003-12-14 14:32:19 +00:00
|
|
|
|
2004-09-02 23:28:24 +00:00
|
|
|
/**
|
|
|
|
|
*
|
|
|
|
|
*/
|
2011-01-26 17:41:18 +00:00
|
|
|
define( 'MSG_LOAD_TIMEOUT', 60 );
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
define( 'MSG_LOCK_TIMEOUT', 30 );
|
|
|
|
|
define( 'MSG_WAIT_TIMEOUT', 30 );
|
2007-01-05 18:08:29 +00:00
|
|
|
define( 'MSG_CACHE_VERSION', 1 );
|
2003-12-14 14:32:19 +00:00
|
|
|
|
2004-09-02 23:28:24 +00:00
|
|
|
/**
|
|
|
|
|
* Message cache
|
2007-01-05 18:08:29 +00:00
|
|
|
* Performs various MediaWiki namespace-related functions
|
WARNING: HUGE COMMIT
Doxygen documentation update:
* Changed alls @addtogroup to @ingroup. @addtogroup adds the comment to the group description, but doesn't add the file, class, function, ... to the group like @ingroup does. See for example http://svn.wikimedia.org/doc/group__SpecialPage.html where it's impossible to see related files, classes, ... that should belong to that group.
* Added @file to file description, it seems that it should be explicitely decalred for file descriptions, otherwise doxygen will think that the comment document the first class, variabled, function, ... that is in that file.
* Removed some empty comments
* Removed some ?>
Added following groups:
* ExternalStorage
* JobQueue
* MaintenanceLanguage
One more thing: there are still a lot of warnings when generating the doc.
2008-05-20 17:13:28 +00:00
|
|
|
* @ingroup Cache
|
2004-09-02 23:28:24 +00:00
|
|
|
*/
|
2005-12-04 18:27:59 +00:00
|
|
|
class MessageCache {
|
2010-08-05 18:38:42 +00:00
|
|
|
/**
|
|
|
|
|
* Process local cache of loaded messages that are defined in
|
|
|
|
|
* MediaWiki namespace. First array level is a language code,
|
|
|
|
|
* second level is message key and the values are either message
|
|
|
|
|
* content prefixed with space, or !NONEXISTENT for negative
|
|
|
|
|
* caching.
|
|
|
|
|
*/
|
|
|
|
|
protected $mCache;
|
|
|
|
|
|
|
|
|
|
// Should mean that database cannot be used, but check
|
|
|
|
|
protected $mDisable;
|
2008-07-05 15:39:10 +00:00
|
|
|
|
2010-08-05 18:38:42 +00:00
|
|
|
/// Lifetime for cache, used by object caching
|
|
|
|
|
protected $mExpiry;
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Message cache has it's own parser which it uses to transform
|
|
|
|
|
* messages.
|
|
|
|
|
*/
|
|
|
|
|
protected $mParserOptions, $mParser;
|
2004-09-25 02:23:04 +00:00
|
|
|
|
2010-08-05 18:38:42 +00:00
|
|
|
/// Variable for tracking which variables are already loaded
|
|
|
|
|
protected $mLoadedLanguages = array();
|
2004-09-25 02:23:04 +00:00
|
|
|
|
2011-01-26 15:42:04 +00:00
|
|
|
/**
|
|
|
|
|
* Singleton instance
|
2011-04-25 22:41:54 +00:00
|
|
|
*
|
|
|
|
|
* @var MessageCache
|
2011-01-26 15:42:04 +00:00
|
|
|
*/
|
|
|
|
|
private static $instance;
|
|
|
|
|
|
2011-04-18 12:43:53 +00:00
|
|
|
/**
|
|
|
|
|
* @var bool
|
|
|
|
|
*/
|
|
|
|
|
protected $mInParser = false;
|
|
|
|
|
|
2011-01-26 15:42:04 +00:00
|
|
|
/**
|
|
|
|
|
* Get the signleton instance of this class
|
|
|
|
|
*
|
|
|
|
|
* @since 1.18
|
|
|
|
|
* @return MessageCache object
|
|
|
|
|
*/
|
|
|
|
|
public static function singleton() {
|
|
|
|
|
if ( is_null( self::$instance ) ) {
|
|
|
|
|
global $wgUseDatabaseMessages, $wgMsgCacheExpiry;
|
|
|
|
|
self::$instance = new self( wfGetMessageCacheStorage(), $wgUseDatabaseMessages, $wgMsgCacheExpiry );
|
|
|
|
|
}
|
|
|
|
|
return self::$instance;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
2011-01-26 17:41:18 +00:00
|
|
|
* Destroy the singleton instance
|
2011-01-26 15:42:04 +00:00
|
|
|
*
|
|
|
|
|
* @since 1.18
|
|
|
|
|
*/
|
|
|
|
|
public static function destroyInstance() {
|
|
|
|
|
self::$instance = null;
|
|
|
|
|
}
|
|
|
|
|
|
2010-08-05 18:26:13 +00:00
|
|
|
function __construct( $memCached, $useDB, $expiry ) {
|
2010-08-05 18:19:34 +00:00
|
|
|
if ( !$memCached ) {
|
|
|
|
|
$memCached = wfGetCache( CACHE_NONE );
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
$this->mMemc = $memCached;
|
2003-12-14 14:32:19 +00:00
|
|
|
$this->mDisable = !$useDB;
|
|
|
|
|
$this->mExpiry = $expiry;
|
2008-06-30 03:02:06 +00:00
|
|
|
}
|
2008-06-03 20:41:57 +00:00
|
|
|
|
2008-07-05 15:39:10 +00:00
|
|
|
/**
|
|
|
|
|
* ParserOptions is lazy initialised.
|
2011-04-18 12:43:53 +00:00
|
|
|
*
|
|
|
|
|
* @return ParserOptions
|
2008-07-05 15:39:10 +00:00
|
|
|
*/
|
2006-07-26 07:15:39 +00:00
|
|
|
function getParserOptions() {
|
|
|
|
|
if ( !$this->mParserOptions ) {
|
|
|
|
|
$this->mParserOptions = new ParserOptions;
|
2012-05-19 16:23:24 +00:00
|
|
|
$this->mParserOptions->setEditSection( false );
|
2006-07-26 07:15:39 +00:00
|
|
|
}
|
|
|
|
|
return $this->mParserOptions;
|
|
|
|
|
}
|
|
|
|
|
|
2006-01-07 13:09:30 +00:00
|
|
|
/**
|
2008-07-05 15:39:10 +00:00
|
|
|
* Try to load the cache from a local file.
|
|
|
|
|
*
|
2013-03-11 17:15:01 +00:00
|
|
|
* @param string $hash the hash of contents, to check validity.
|
2008-07-05 15:39:10 +00:00
|
|
|
* @param $code Mixed: Optional language code, see documenation of load().
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
* @return The cache array
|
2005-11-08 11:54:04 +00:00
|
|
|
*/
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
function getLocalCache( $hash, $code ) {
|
|
|
|
|
global $wgCacheDirectory;
|
2005-11-08 11:54:04 +00:00
|
|
|
|
* Introduced a new system for localisation caching. The system is based around fast fetches of individual messages, minimising memory overhead and startup time in the typical case. It handles both core messages (formerly in Language.php) and extension messages (formerly in MessageCache.php). Profiling indicates a significant win for average throughput.
* The serialized message cache, which would have been redundant, has been removed. Similar performance characteristics can be achieved with $wgLocalisationCacheConf['manualRecache'] = true;
* Added a maintenance script rebuildLocalisationCache.php for offline rebuilding of the localisation cache.
* Extension i18n files can now contain any of the variables which can be set in Messages*.php. It is possible, and recommended, to use this feature instead of the hooks for special page aliases and magic words.
* $wgExtensionAliasesFiles, LanguageGetMagic and LanguageGetSpecialPageAliases are retained for backwards compatibility. $wgMessageCache->addMessages() and related functions have been removed. wfLoadExtensionMessages() is a no-op and can continue to be called for b/c.
* Introduced $wgCacheDirectory as a default location for the various local caches that have accumulated. Suggested $IP/cache as a good place for it in the default LocalSettings.php and created this directory with a deny-all .htaccess.
* Patched Exception.php to avoid using the message cache when an exception is thrown from within LocalisationCache, since this tends to fail horribly.
* Removed Language::getLocalisationArray(), Language::loadLocalisation(), Language::load()
* Fixed FileDependency::__sleep()
* In Cdb.php, fixed newlines in debug messages
In MessageCache::get():
* Replaced calls to $wgContLang capitalisation functions with plain PHP functions, reducing the typical case from 99us to 93us. Message cache keys are already documented as being restricted to ASCII.
* Implemented a more efficient way to filter out bogus language codes, reducing the "foo/en" case from 430us to 101us
* Optimised wfRunHooks() in the typical do-nothing case, from ~30us to ~3us. This reduced MessageCache::get() typical case time from 93us to 38us.
* Removed hook MessageNotInMwNs to save an extra 3us per cache hit. Reimplemented the only user (LocalisationUpdate) using the new hook LocalisationCacheRecache.
2009-06-28 07:11:43 +00:00
|
|
|
$filename = "$wgCacheDirectory/messages-" . wfWikiID() . "-$code";
|
2005-11-08 11:54:04 +00:00
|
|
|
|
2008-07-05 15:39:10 +00:00
|
|
|
# Check file existence
|
2005-12-30 08:50:59 +00:00
|
|
|
wfSuppressWarnings();
|
2005-11-08 11:54:04 +00:00
|
|
|
$file = fopen( $filename, 'r' );
|
2005-12-30 08:50:59 +00:00
|
|
|
wfRestoreWarnings();
|
2005-11-08 11:54:04 +00:00
|
|
|
if ( !$file ) {
|
2008-07-05 15:39:10 +00:00
|
|
|
return false; // No cache file
|
2005-11-08 11:54:04 +00:00
|
|
|
}
|
|
|
|
|
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
// Check to see if the file has the hash specified
|
|
|
|
|
$localHash = fread( $file, 32 );
|
|
|
|
|
if ( $hash === $localHash ) {
|
|
|
|
|
// All good, get the rest of it
|
|
|
|
|
$serialized = '';
|
|
|
|
|
while ( !feof( $file ) ) {
|
|
|
|
|
$serialized .= fread( $file, 100000 );
|
2008-02-02 18:15:57 +00:00
|
|
|
}
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
fclose( $file );
|
|
|
|
|
return unserialize( $serialized );
|
2008-02-10 14:29:17 +00:00
|
|
|
} else {
|
2011-01-26 17:41:18 +00:00
|
|
|
fclose( $file );
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
return false; // Wrong hash
|
2005-11-08 11:54:04 +00:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
2008-07-05 15:39:10 +00:00
|
|
|
* Save the cache to a local file.
|
2005-11-08 11:54:04 +00:00
|
|
|
*/
|
2008-07-05 15:39:10 +00:00
|
|
|
function saveToLocal( $serialized, $hash, $code ) {
|
* Introduced a new system for localisation caching. The system is based around fast fetches of individual messages, minimising memory overhead and startup time in the typical case. It handles both core messages (formerly in Language.php) and extension messages (formerly in MessageCache.php). Profiling indicates a significant win for average throughput.
* The serialized message cache, which would have been redundant, has been removed. Similar performance characteristics can be achieved with $wgLocalisationCacheConf['manualRecache'] = true;
* Added a maintenance script rebuildLocalisationCache.php for offline rebuilding of the localisation cache.
* Extension i18n files can now contain any of the variables which can be set in Messages*.php. It is possible, and recommended, to use this feature instead of the hooks for special page aliases and magic words.
* $wgExtensionAliasesFiles, LanguageGetMagic and LanguageGetSpecialPageAliases are retained for backwards compatibility. $wgMessageCache->addMessages() and related functions have been removed. wfLoadExtensionMessages() is a no-op and can continue to be called for b/c.
* Introduced $wgCacheDirectory as a default location for the various local caches that have accumulated. Suggested $IP/cache as a good place for it in the default LocalSettings.php and created this directory with a deny-all .htaccess.
* Patched Exception.php to avoid using the message cache when an exception is thrown from within LocalisationCache, since this tends to fail horribly.
* Removed Language::getLocalisationArray(), Language::loadLocalisation(), Language::load()
* Fixed FileDependency::__sleep()
* In Cdb.php, fixed newlines in debug messages
In MessageCache::get():
* Replaced calls to $wgContLang capitalisation functions with plain PHP functions, reducing the typical case from 99us to 93us. Message cache keys are already documented as being restricted to ASCII.
* Implemented a more efficient way to filter out bogus language codes, reducing the "foo/en" case from 430us to 101us
* Optimised wfRunHooks() in the typical do-nothing case, from ~30us to ~3us. This reduced MessageCache::get() typical case time from 93us to 38us.
* Removed hook MessageNotInMwNs to save an extra 3us per cache hit. Reimplemented the only user (LocalisationUpdate) using the new hook LocalisationCacheRecache.
2009-06-28 07:11:43 +00:00
|
|
|
global $wgCacheDirectory;
|
2008-05-23 22:14:32 +00:00
|
|
|
|
* Introduced a new system for localisation caching. The system is based around fast fetches of individual messages, minimising memory overhead and startup time in the typical case. It handles both core messages (formerly in Language.php) and extension messages (formerly in MessageCache.php). Profiling indicates a significant win for average throughput.
* The serialized message cache, which would have been redundant, has been removed. Similar performance characteristics can be achieved with $wgLocalisationCacheConf['manualRecache'] = true;
* Added a maintenance script rebuildLocalisationCache.php for offline rebuilding of the localisation cache.
* Extension i18n files can now contain any of the variables which can be set in Messages*.php. It is possible, and recommended, to use this feature instead of the hooks for special page aliases and magic words.
* $wgExtensionAliasesFiles, LanguageGetMagic and LanguageGetSpecialPageAliases are retained for backwards compatibility. $wgMessageCache->addMessages() and related functions have been removed. wfLoadExtensionMessages() is a no-op and can continue to be called for b/c.
* Introduced $wgCacheDirectory as a default location for the various local caches that have accumulated. Suggested $IP/cache as a good place for it in the default LocalSettings.php and created this directory with a deny-all .htaccess.
* Patched Exception.php to avoid using the message cache when an exception is thrown from within LocalisationCache, since this tends to fail horribly.
* Removed Language::getLocalisationArray(), Language::loadLocalisation(), Language::load()
* Fixed FileDependency::__sleep()
* In Cdb.php, fixed newlines in debug messages
In MessageCache::get():
* Replaced calls to $wgContLang capitalisation functions with plain PHP functions, reducing the typical case from 99us to 93us. Message cache keys are already documented as being restricted to ASCII.
* Implemented a more efficient way to filter out bogus language codes, reducing the "foo/en" case from 430us to 101us
* Optimised wfRunHooks() in the typical do-nothing case, from ~30us to ~3us. This reduced MessageCache::get() typical case time from 93us to 38us.
* Removed hook MessageNotInMwNs to save an extra 3us per cache hit. Reimplemented the only user (LocalisationUpdate) using the new hook LocalisationCacheRecache.
2009-06-28 07:11:43 +00:00
|
|
|
$filename = "$wgCacheDirectory/messages-" . wfWikiID() . "-$code";
|
2011-07-25 22:01:19 +00:00
|
|
|
wfMkdirParents( $wgCacheDirectory, null, __METHOD__ ); // might fail
|
2005-11-08 11:54:04 +00:00
|
|
|
|
2008-07-05 15:39:10 +00:00
|
|
|
wfSuppressWarnings();
|
2005-11-08 11:54:04 +00:00
|
|
|
$file = fopen( $filename, 'w' );
|
2008-07-05 15:39:10 +00:00
|
|
|
wfRestoreWarnings();
|
|
|
|
|
|
2005-11-08 11:54:04 +00:00
|
|
|
if ( !$file ) {
|
|
|
|
|
wfDebug( "Unable to open local cache file for writing\n" );
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
fwrite( $file, $hash . $serialized );
|
|
|
|
|
fclose( $file );
|
2011-06-03 10:20:45 +00:00
|
|
|
wfSuppressWarnings();
|
|
|
|
|
chmod( $filename, 0666 );
|
|
|
|
|
wfRestoreWarnings();
|
2005-11-08 11:54:04 +00:00
|
|
|
}
|
|
|
|
|
|
2004-09-02 23:28:24 +00:00
|
|
|
/**
|
2008-07-05 15:39:10 +00:00
|
|
|
* Loads messages from caches or from database in this order:
|
* Introduced a new system for localisation caching. The system is based around fast fetches of individual messages, minimising memory overhead and startup time in the typical case. It handles both core messages (formerly in Language.php) and extension messages (formerly in MessageCache.php). Profiling indicates a significant win for average throughput.
* The serialized message cache, which would have been redundant, has been removed. Similar performance characteristics can be achieved with $wgLocalisationCacheConf['manualRecache'] = true;
* Added a maintenance script rebuildLocalisationCache.php for offline rebuilding of the localisation cache.
* Extension i18n files can now contain any of the variables which can be set in Messages*.php. It is possible, and recommended, to use this feature instead of the hooks for special page aliases and magic words.
* $wgExtensionAliasesFiles, LanguageGetMagic and LanguageGetSpecialPageAliases are retained for backwards compatibility. $wgMessageCache->addMessages() and related functions have been removed. wfLoadExtensionMessages() is a no-op and can continue to be called for b/c.
* Introduced $wgCacheDirectory as a default location for the various local caches that have accumulated. Suggested $IP/cache as a good place for it in the default LocalSettings.php and created this directory with a deny-all .htaccess.
* Patched Exception.php to avoid using the message cache when an exception is thrown from within LocalisationCache, since this tends to fail horribly.
* Removed Language::getLocalisationArray(), Language::loadLocalisation(), Language::load()
* Fixed FileDependency::__sleep()
* In Cdb.php, fixed newlines in debug messages
In MessageCache::get():
* Replaced calls to $wgContLang capitalisation functions with plain PHP functions, reducing the typical case from 99us to 93us. Message cache keys are already documented as being restricted to ASCII.
* Implemented a more efficient way to filter out bogus language codes, reducing the "foo/en" case from 430us to 101us
* Optimised wfRunHooks() in the typical do-nothing case, from ~30us to ~3us. This reduced MessageCache::get() typical case time from 93us to 38us.
* Removed hook MessageNotInMwNs to save an extra 3us per cache hit. Reimplemented the only user (LocalisationUpdate) using the new hook LocalisationCacheRecache.
2009-06-28 07:11:43 +00:00
|
|
|
* (1) local message cache (if $wgUseLocalMessageCache is enabled)
|
2008-07-05 15:39:10 +00:00
|
|
|
* (2) memcached
|
|
|
|
|
* (3) from the database.
|
|
|
|
|
*
|
|
|
|
|
* When succesfully loading from (2) or (3), all higher level caches are
|
|
|
|
|
* updated for the newest version.
|
|
|
|
|
*
|
2010-08-07 23:41:03 +00:00
|
|
|
* Nothing is loaded if member variable mDisable is true, either manually
|
2008-07-05 15:39:10 +00:00
|
|
|
* set by calling code or if message loading fails (is this possible?).
|
|
|
|
|
*
|
|
|
|
|
* Returns true if cache is already populated or it was succesfully populated,
|
|
|
|
|
* or false if populating empty cache fails. Also returns true if MessageCache
|
|
|
|
|
* is disabled.
|
|
|
|
|
*
|
2012-12-09 03:09:48 +00:00
|
|
|
* @param bool|String $code String: language to which load messages
|
|
|
|
|
* @throws MWException
|
2012-02-09 21:33:27 +00:00
|
|
|
* @return bool
|
2004-09-02 23:28:24 +00:00
|
|
|
*/
|
2008-07-05 15:39:10 +00:00
|
|
|
function load( $code = false ) {
|
* Introduced a new system for localisation caching. The system is based around fast fetches of individual messages, minimising memory overhead and startup time in the typical case. It handles both core messages (formerly in Language.php) and extension messages (formerly in MessageCache.php). Profiling indicates a significant win for average throughput.
* The serialized message cache, which would have been redundant, has been removed. Similar performance characteristics can be achieved with $wgLocalisationCacheConf['manualRecache'] = true;
* Added a maintenance script rebuildLocalisationCache.php for offline rebuilding of the localisation cache.
* Extension i18n files can now contain any of the variables which can be set in Messages*.php. It is possible, and recommended, to use this feature instead of the hooks for special page aliases and magic words.
* $wgExtensionAliasesFiles, LanguageGetMagic and LanguageGetSpecialPageAliases are retained for backwards compatibility. $wgMessageCache->addMessages() and related functions have been removed. wfLoadExtensionMessages() is a no-op and can continue to be called for b/c.
* Introduced $wgCacheDirectory as a default location for the various local caches that have accumulated. Suggested $IP/cache as a good place for it in the default LocalSettings.php and created this directory with a deny-all .htaccess.
* Patched Exception.php to avoid using the message cache when an exception is thrown from within LocalisationCache, since this tends to fail horribly.
* Removed Language::getLocalisationArray(), Language::loadLocalisation(), Language::load()
* Fixed FileDependency::__sleep()
* In Cdb.php, fixed newlines in debug messages
In MessageCache::get():
* Replaced calls to $wgContLang capitalisation functions with plain PHP functions, reducing the typical case from 99us to 93us. Message cache keys are already documented as being restricted to ASCII.
* Implemented a more efficient way to filter out bogus language codes, reducing the "foo/en" case from 430us to 101us
* Optimised wfRunHooks() in the typical do-nothing case, from ~30us to ~3us. This reduced MessageCache::get() typical case time from 93us to 38us.
* Removed hook MessageNotInMwNs to save an extra 3us per cache hit. Reimplemented the only user (LocalisationUpdate) using the new hook LocalisationCacheRecache.
2009-06-28 07:11:43 +00:00
|
|
|
global $wgUseLocalMessageCache;
|
2008-07-05 15:39:10 +00:00
|
|
|
|
|
|
|
|
if( !is_string( $code ) ) {
|
|
|
|
|
# This isn't really nice, so at least make a note about it and try to
|
|
|
|
|
# fall back
|
|
|
|
|
wfDebug( __METHOD__ . " called without providing a language code\n" );
|
|
|
|
|
$code = 'en';
|
|
|
|
|
}
|
2008-06-03 20:41:57 +00:00
|
|
|
|
2008-07-05 15:39:10 +00:00
|
|
|
# Don't do double loading...
|
2011-01-26 17:41:18 +00:00
|
|
|
if ( isset( $this->mLoadedLanguages[$code] ) ) {
|
|
|
|
|
return true;
|
|
|
|
|
}
|
2008-07-05 15:39:10 +00:00
|
|
|
|
|
|
|
|
# 8 lines of code just to say (once) that message cache is disabled
|
2003-12-14 14:32:19 +00:00
|
|
|
if ( $this->mDisable ) {
|
2005-05-28 11:07:55 +00:00
|
|
|
static $shownDisabled = false;
|
|
|
|
|
if ( !$shownDisabled ) {
|
2008-07-05 15:39:10 +00:00
|
|
|
wfDebug( __METHOD__ . ": disabled\n" );
|
2005-05-28 11:07:55 +00:00
|
|
|
$shownDisabled = true;
|
|
|
|
|
}
|
2003-12-14 14:32:19 +00:00
|
|
|
return true;
|
|
|
|
|
}
|
2004-09-20 14:55:25 +00:00
|
|
|
|
2008-07-05 15:39:10 +00:00
|
|
|
# Loading code starts
|
|
|
|
|
wfProfileIn( __METHOD__ );
|
|
|
|
|
$success = false; # Keep track of success
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
$staleCache = false; # a cache array with expired data, or false if none has been loaded
|
2008-07-05 15:39:10 +00:00
|
|
|
$where = array(); # Debug info, delayed to avoid spamming debug log too much
|
|
|
|
|
$cacheKey = wfMemcKey( 'messages', $code ); # Key in memc for messages
|
2005-11-08 11:54:04 +00:00
|
|
|
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
# Local cache
|
2008-07-05 15:39:10 +00:00
|
|
|
# Hash of the contents is stored in memcache, to detect if local cache goes
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
# out of date (e.g. due to replace() on some other server)
|
* Introduced a new system for localisation caching. The system is based around fast fetches of individual messages, minimising memory overhead and startup time in the typical case. It handles both core messages (formerly in Language.php) and extension messages (formerly in MessageCache.php). Profiling indicates a significant win for average throughput.
* The serialized message cache, which would have been redundant, has been removed. Similar performance characteristics can be achieved with $wgLocalisationCacheConf['manualRecache'] = true;
* Added a maintenance script rebuildLocalisationCache.php for offline rebuilding of the localisation cache.
* Extension i18n files can now contain any of the variables which can be set in Messages*.php. It is possible, and recommended, to use this feature instead of the hooks for special page aliases and magic words.
* $wgExtensionAliasesFiles, LanguageGetMagic and LanguageGetSpecialPageAliases are retained for backwards compatibility. $wgMessageCache->addMessages() and related functions have been removed. wfLoadExtensionMessages() is a no-op and can continue to be called for b/c.
* Introduced $wgCacheDirectory as a default location for the various local caches that have accumulated. Suggested $IP/cache as a good place for it in the default LocalSettings.php and created this directory with a deny-all .htaccess.
* Patched Exception.php to avoid using the message cache when an exception is thrown from within LocalisationCache, since this tends to fail horribly.
* Removed Language::getLocalisationArray(), Language::loadLocalisation(), Language::load()
* Fixed FileDependency::__sleep()
* In Cdb.php, fixed newlines in debug messages
In MessageCache::get():
* Replaced calls to $wgContLang capitalisation functions with plain PHP functions, reducing the typical case from 99us to 93us. Message cache keys are already documented as being restricted to ASCII.
* Implemented a more efficient way to filter out bogus language codes, reducing the "foo/en" case from 430us to 101us
* Optimised wfRunHooks() in the typical do-nothing case, from ~30us to ~3us. This reduced MessageCache::get() typical case time from 93us to 38us.
* Removed hook MessageNotInMwNs to save an extra 3us per cache hit. Reimplemented the only user (LocalisationUpdate) using the new hook LocalisationCacheRecache.
2009-06-28 07:11:43 +00:00
|
|
|
if ( $wgUseLocalMessageCache ) {
|
2008-07-05 15:39:10 +00:00
|
|
|
wfProfileIn( __METHOD__ . '-fromlocal' );
|
|
|
|
|
|
|
|
|
|
$hash = $this->mMemc->get( wfMemcKey( 'messages', $code, 'hash' ) );
|
2008-02-02 18:15:57 +00:00
|
|
|
if ( $hash ) {
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
$cache = $this->getLocalCache( $hash, $code );
|
|
|
|
|
if ( !$cache ) {
|
|
|
|
|
$where[] = 'local cache is empty or has the wrong hash';
|
|
|
|
|
} elseif ( $this->isCacheExpired( $cache ) ) {
|
|
|
|
|
$where[] = 'local cache is expired';
|
|
|
|
|
$staleCache = $cache;
|
|
|
|
|
} else {
|
|
|
|
|
$where[] = 'got from local cache';
|
|
|
|
|
$success = true;
|
|
|
|
|
$this->mCache[$code] = $cache;
|
|
|
|
|
}
|
2005-11-08 11:54:04 +00:00
|
|
|
}
|
2008-07-05 15:39:10 +00:00
|
|
|
wfProfileOut( __METHOD__ . '-fromlocal' );
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if ( !$success ) {
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
# Try the global cache. If it is empty, try to acquire a lock. If
|
|
|
|
|
# the lock can't be acquired, wait for the other thread to finish
|
|
|
|
|
# and then try the global cache a second time.
|
|
|
|
|
for ( $failedAttempts = 0; $failedAttempts < 2; $failedAttempts++ ) {
|
|
|
|
|
wfProfileIn( __METHOD__ . '-fromcache' );
|
|
|
|
|
$cache = $this->mMemc->get( $cacheKey );
|
|
|
|
|
if ( !$cache ) {
|
|
|
|
|
$where[] = 'global cache is empty';
|
|
|
|
|
} elseif ( $this->isCacheExpired( $cache ) ) {
|
|
|
|
|
$where[] = 'global cache is expired';
|
|
|
|
|
$staleCache = $cache;
|
|
|
|
|
} else {
|
|
|
|
|
$where[] = 'got from global cache';
|
|
|
|
|
$this->mCache[$code] = $cache;
|
|
|
|
|
$this->saveToCaches( $cache, 'local-only', $code );
|
|
|
|
|
$success = true;
|
|
|
|
|
}
|
2008-06-01 03:27:48 +00:00
|
|
|
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
wfProfileOut( __METHOD__ . '-fromcache' );
|
|
|
|
|
|
|
|
|
|
if ( $success ) {
|
|
|
|
|
# Done, no need to retry
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
# We need to call loadFromDB. Limit the concurrency to a single
|
|
|
|
|
# process. This prevents the site from going down when the cache
|
|
|
|
|
# expires.
|
|
|
|
|
$statusKey = wfMemcKey( 'messages', $code, 'status' );
|
|
|
|
|
$acquired = $this->mMemc->add( $statusKey, 'loading', MSG_LOAD_TIMEOUT );
|
|
|
|
|
if ( $acquired ) {
|
|
|
|
|
# Unlock the status key if there is an exception
|
|
|
|
|
$that = $this;
|
|
|
|
|
$statusUnlocker = new ScopedCallback( function() use ( $that, $statusKey ) {
|
|
|
|
|
$that->mMemc->delete( $statusKey );
|
|
|
|
|
} );
|
|
|
|
|
|
|
|
|
|
# Now let's regenerate
|
|
|
|
|
$where[] = 'loading from database';
|
|
|
|
|
|
|
|
|
|
# Lock the cache to prevent conflicting writes
|
|
|
|
|
# If this lock fails, it doesn't really matter, it just means the
|
|
|
|
|
# write is potentially non-atomic, e.g. the results of a replace()
|
|
|
|
|
# may be discarded.
|
|
|
|
|
if ( $this->lock( $cacheKey ) ) {
|
|
|
|
|
$mainUnlocker = new ScopedCallback( function() use ( $that, $cacheKey ) {
|
|
|
|
|
$that->unlock( $cacheKey );
|
|
|
|
|
} );
|
|
|
|
|
} else {
|
|
|
|
|
$mainUnlocker = null;
|
|
|
|
|
$where[] = 'could not acquire main lock';
|
2012-12-19 20:49:54 +00:00
|
|
|
}
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
|
|
|
|
|
$cache = $this->loadFromDB( $code );
|
|
|
|
|
$this->mCache[$code] = $cache;
|
|
|
|
|
$success = true;
|
|
|
|
|
$saveSuccess = $this->saveToCaches( $cache, 'all', $code );
|
|
|
|
|
|
|
|
|
|
# Unlock
|
|
|
|
|
ScopedCallback::consume( $mainUnlocker );
|
|
|
|
|
ScopedCallback::consume( $statusUnlocker );
|
|
|
|
|
|
|
|
|
|
if ( !$saveSuccess ) {
|
|
|
|
|
# Cache save has failed.
|
|
|
|
|
# There are two main scenarios where this could be a problem:
|
|
|
|
|
#
|
|
|
|
|
# - The cache is more than the maximum size (typically
|
|
|
|
|
# 1MB compressed).
|
|
|
|
|
#
|
|
|
|
|
# - Memcached has no space remaining in the relevant slab
|
|
|
|
|
# class. This is unlikely with recent versions of
|
|
|
|
|
# memcached.
|
|
|
|
|
#
|
|
|
|
|
# Either way, if there is a local cache, nothing bad will
|
|
|
|
|
# happen. If there is no local cache, disabling the message
|
|
|
|
|
# cache for all requests avoids incurring a loadFromDB()
|
|
|
|
|
# overhead on every request, and thus saves the wiki from
|
|
|
|
|
# complete downtime under moderate traffic conditions.
|
|
|
|
|
if ( !$wgUseLocalMessageCache ) {
|
|
|
|
|
$this->mMemc->set( $statusKey, 'error', 60 * 5 );
|
|
|
|
|
$where[] = 'could not save cache, disabled globally for 5 minutes';
|
|
|
|
|
} else {
|
|
|
|
|
$where[] = "could not save global cache";
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
# Load from DB complete, no need to retry
|
|
|
|
|
break;
|
|
|
|
|
} elseif ( $staleCache ) {
|
|
|
|
|
# Use the stale cache while some other thread constructs the new one
|
|
|
|
|
$where[] = 'using stale cache';
|
|
|
|
|
$this->mCache[$code] = $staleCache;
|
|
|
|
|
$success = true;
|
|
|
|
|
break;
|
|
|
|
|
} elseif ( $failedAttempts > 0 ) {
|
|
|
|
|
# Already retried once, still failed, so don't do another lock/unlock cycle
|
|
|
|
|
# This case will typically be hit if memcached is down, or if
|
|
|
|
|
# loadFromDB() takes longer than MSG_WAIT_TIMEOUT
|
|
|
|
|
$where[] = "could not acquire status key.";
|
|
|
|
|
break;
|
2008-10-10 13:02:57 +00:00
|
|
|
} else {
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
$status = $this->mMemc->get( $statusKey );
|
|
|
|
|
if ( $status === 'error' ) {
|
|
|
|
|
# Disable cache
|
|
|
|
|
break;
|
|
|
|
|
} else {
|
|
|
|
|
# Wait for the other thread to finish, then retry
|
|
|
|
|
$where[] = 'waited for other thread to complete';
|
|
|
|
|
$this->lock( $cacheKey );
|
|
|
|
|
$this->unlock( $cacheKey );
|
|
|
|
|
}
|
2008-10-10 13:02:57 +00:00
|
|
|
}
|
2003-12-14 14:32:19 +00:00
|
|
|
}
|
2007-01-05 18:08:29 +00:00
|
|
|
}
|
|
|
|
|
|
2008-07-05 15:39:10 +00:00
|
|
|
if ( !$success ) {
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
$where[] = 'loading FAILED - cache is disabled';
|
2007-01-05 18:08:29 +00:00
|
|
|
$this->mDisable = true;
|
|
|
|
|
$this->mCache = false;
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
# This used to throw an exception, but that led to nasty side effects like
|
|
|
|
|
# the whole wiki being instantly down if the memcached server died
|
2008-07-05 15:39:10 +00:00
|
|
|
} else {
|
|
|
|
|
# All good, just record the success
|
|
|
|
|
$this->mLoadedLanguages[$code] = true;
|
2003-12-14 14:32:19 +00:00
|
|
|
}
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
$info = implode( ', ', $where );
|
|
|
|
|
wfDebug( __METHOD__ . ": Loading $code... $info\n" );
|
2008-07-05 15:39:10 +00:00
|
|
|
wfProfileOut( __METHOD__ );
|
2003-12-14 14:32:19 +00:00
|
|
|
return $success;
|
|
|
|
|
}
|
|
|
|
|
|
2004-09-02 23:28:24 +00:00
|
|
|
/**
|
2008-07-05 15:39:10 +00:00
|
|
|
* Loads cacheable messages from the database. Messages bigger than
|
|
|
|
|
* $wgMaxMsgCacheEntrySize are assigned a special value, and are loaded
|
|
|
|
|
* on-demand from the database later.
|
|
|
|
|
*
|
2013-03-11 17:15:01 +00:00
|
|
|
* @param string $code language code.
|
2011-01-26 17:41:18 +00:00
|
|
|
* @return Array: loaded messages for storing in caches.
|
2004-09-02 23:28:24 +00:00
|
|
|
*/
|
2010-08-21 16:41:53 +00:00
|
|
|
function loadFromDB( $code ) {
|
2007-01-05 18:08:29 +00:00
|
|
|
wfProfileIn( __METHOD__ );
|
2010-09-07 22:37:55 +00:00
|
|
|
global $wgMaxMsgCacheEntrySize, $wgLanguageCode, $wgAdaptiveMessageCache;
|
2007-01-22 23:50:42 +00:00
|
|
|
$dbr = wfGetDB( DB_SLAVE );
|
2008-07-05 15:39:10 +00:00
|
|
|
$cache = array();
|
|
|
|
|
|
|
|
|
|
# Common conditions
|
|
|
|
|
$conds = array(
|
|
|
|
|
'page_is_redirect' => 0,
|
|
|
|
|
'page_namespace' => NS_MEDIAWIKI,
|
|
|
|
|
);
|
|
|
|
|
|
2010-08-21 16:41:53 +00:00
|
|
|
$mostused = array();
|
2012-10-24 14:59:37 +00:00
|
|
|
if ( $wgAdaptiveMessageCache && $code !== $wgLanguageCode ) {
|
|
|
|
|
if ( !isset( $this->mCache[$wgLanguageCode] ) ) {
|
|
|
|
|
$this->load( $wgLanguageCode );
|
|
|
|
|
}
|
|
|
|
|
$mostused = array_keys( $this->mCache[$wgLanguageCode] );
|
|
|
|
|
foreach ( $mostused as $key => $value ) {
|
|
|
|
|
$mostused[$key] = "$value/$code";
|
2008-07-05 15:39:10 +00:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
2010-08-21 16:41:53 +00:00
|
|
|
if ( count( $mostused ) ) {
|
|
|
|
|
$conds['page_title'] = $mostused;
|
2010-09-07 22:37:55 +00:00
|
|
|
} elseif ( $code !== $wgLanguageCode ) {
|
2012-10-24 14:59:37 +00:00
|
|
|
$conds[] = 'page_title' . $dbr->buildLike( $dbr->anyString(), '/', $code );
|
2010-08-21 16:41:53 +00:00
|
|
|
} else {
|
|
|
|
|
# Effectively disallows use of '/' character in NS_MEDIAWIKI for uses
|
|
|
|
|
# other than language code.
|
|
|
|
|
$conds[] = 'page_title NOT' . $dbr->buildLike( $dbr->anyString(), '/', $dbr->anyString() );
|
|
|
|
|
}
|
|
|
|
|
|
2008-07-05 15:39:10 +00:00
|
|
|
# Conditions to fetch oversized pages to ignore them
|
|
|
|
|
$bigConds = $conds;
|
|
|
|
|
$bigConds[] = 'page_len > ' . intval( $wgMaxMsgCacheEntrySize );
|
2003-12-14 14:32:19 +00:00
|
|
|
|
2007-01-05 18:08:29 +00:00
|
|
|
# Load titles for all oversized pages in the MediaWiki namespace
|
2010-07-30 08:00:29 +00:00
|
|
|
$res = $dbr->select( 'page', 'page_title', $bigConds, __METHOD__ . "($code)-big" );
|
2010-08-07 23:43:28 +00:00
|
|
|
foreach ( $res as $row ) {
|
2008-07-05 15:39:10 +00:00
|
|
|
$cache[$row->page_title] = '!TOO BIG';
|
2005-03-19 10:40:41 +00:00
|
|
|
}
|
2006-06-25 08:38:17 +00:00
|
|
|
|
2008-07-05 15:39:10 +00:00
|
|
|
# Conditions to load the remaining pages with their contents
|
|
|
|
|
$smallConds = $conds;
|
|
|
|
|
$smallConds[] = 'page_latest=rev_id';
|
|
|
|
|
$smallConds[] = 'rev_text_id=old_id';
|
|
|
|
|
$smallConds[] = 'page_len <= ' . intval( $wgMaxMsgCacheEntrySize );
|
|
|
|
|
|
2011-01-26 17:41:18 +00:00
|
|
|
$res = $dbr->select(
|
|
|
|
|
array( 'page', 'revision', 'text' ),
|
2007-01-05 18:08:29 +00:00
|
|
|
array( 'page_title', 'old_text', 'old_flags' ),
|
2011-01-26 17:41:18 +00:00
|
|
|
$smallConds,
|
|
|
|
|
__METHOD__ . "($code)-small"
|
|
|
|
|
);
|
2006-06-25 08:38:17 +00:00
|
|
|
|
2010-08-07 23:43:28 +00:00
|
|
|
foreach ( $res as $row ) {
|
2011-09-27 00:41:24 +00:00
|
|
|
$text = Revision::getRevisionText( $row );
|
|
|
|
|
if( $text === false ) {
|
|
|
|
|
// Failed to fetch data; possible ES errors?
|
|
|
|
|
// Store a marker to fetch on-demand as a workaround...
|
|
|
|
|
$entry = '!TOO BIG';
|
|
|
|
|
wfDebugLog( 'MessageCache', __METHOD__ . ": failed to load message page text for {$row->page_title} ($code)" );
|
|
|
|
|
} else {
|
|
|
|
|
$entry = ' ' . $text;
|
|
|
|
|
}
|
|
|
|
|
$cache[$row->page_title] = $entry;
|
2006-01-07 13:31:29 +00:00
|
|
|
}
|
2008-07-05 15:39:10 +00:00
|
|
|
|
|
|
|
|
$cache['VERSION'] = MSG_CACHE_VERSION;
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
$cache['EXPIRY'] = wfTimestamp( TS_MW, time() + $this->mExpiry );
|
2007-01-05 18:08:29 +00:00
|
|
|
wfProfileOut( __METHOD__ );
|
2008-07-05 15:39:10 +00:00
|
|
|
return $cache;
|
2003-12-14 14:32:19 +00:00
|
|
|
}
|
2004-09-20 14:55:25 +00:00
|
|
|
|
2008-07-05 15:39:10 +00:00
|
|
|
/**
|
|
|
|
|
* Updates cache as necessary when message page is changed
|
|
|
|
|
*
|
2013-03-11 17:15:01 +00:00
|
|
|
* @param string $title name of the page changed.
|
2008-07-05 15:39:10 +00:00
|
|
|
* @param $text Mixed: new contents of the page.
|
|
|
|
|
*/
|
|
|
|
|
public function replace( $title, $text ) {
|
2008-06-01 03:27:48 +00:00
|
|
|
global $wgMaxMsgCacheEntrySize;
|
2008-06-30 03:02:06 +00:00
|
|
|
wfProfileIn( __METHOD__ );
|
2008-07-05 15:39:10 +00:00
|
|
|
|
2010-08-08 00:28:17 +00:00
|
|
|
if ( $this->mDisable ) {
|
2011-02-10 16:04:19 +00:00
|
|
|
wfProfileOut( __METHOD__ );
|
2010-08-08 00:28:17 +00:00
|
|
|
return;
|
|
|
|
|
}
|
2008-07-05 15:39:10 +00:00
|
|
|
|
2010-09-04 04:00:09 +00:00
|
|
|
list( $msg, $code ) = $this->figureMessage( $title );
|
2008-07-05 15:39:10 +00:00
|
|
|
|
|
|
|
|
$cacheKey = wfMemcKey( 'messages', $code );
|
2010-08-08 00:28:17 +00:00
|
|
|
$this->load( $code );
|
|
|
|
|
$this->lock( $cacheKey );
|
2008-07-05 15:39:10 +00:00
|
|
|
|
2010-08-08 00:28:17 +00:00
|
|
|
$titleKey = wfMemcKey( 'messages', 'individual', $title );
|
2008-07-05 15:39:10 +00:00
|
|
|
|
2010-08-08 00:28:17 +00:00
|
|
|
if ( $text === false ) {
|
|
|
|
|
# Article was deleted
|
|
|
|
|
$this->mCache[$code][$title] = '!NONEXISTENT';
|
|
|
|
|
$this->mMemc->delete( $titleKey );
|
|
|
|
|
} elseif ( strlen( $text ) > $wgMaxMsgCacheEntrySize ) {
|
|
|
|
|
# Check for size
|
|
|
|
|
$this->mCache[$code][$title] = '!TOO BIG';
|
|
|
|
|
$this->mMemc->set( $titleKey, ' ' . $text, $this->mExpiry );
|
|
|
|
|
} else {
|
|
|
|
|
$this->mCache[$code][$title] = ' ' . $text;
|
|
|
|
|
$this->mMemc->delete( $titleKey );
|
2008-07-05 15:39:10 +00:00
|
|
|
}
|
2010-08-08 00:28:17 +00:00
|
|
|
|
|
|
|
|
# Update caches
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
$this->saveToCaches( $this->mCache[$code], 'all', $code );
|
2010-08-08 00:28:17 +00:00
|
|
|
$this->unlock( $cacheKey );
|
2008-07-05 15:39:10 +00:00
|
|
|
|
|
|
|
|
// Also delete cached sidebar... just in case it is affected
|
2010-02-01 04:57:42 +00:00
|
|
|
$codes = array( $code );
|
2013-02-09 22:03:53 +00:00
|
|
|
if ( $code === 'en' ) {
|
2010-02-01 04:57:42 +00:00
|
|
|
// Delete all sidebars, like for example on action=purge on the
|
|
|
|
|
// sidebar messages
|
2012-03-08 20:56:26 +00:00
|
|
|
$codes = array_keys( Language::fetchLanguageNames() );
|
2010-02-01 04:57:42 +00:00
|
|
|
}
|
2009-07-31 07:12:25 +00:00
|
|
|
|
2012-02-07 16:49:34 +00:00
|
|
|
global $wgMemc;
|
2010-02-01 04:57:42 +00:00
|
|
|
foreach ( $codes as $code ) {
|
|
|
|
|
$sidebarKey = wfMemcKey( 'sidebar', $code );
|
2012-02-07 16:49:34 +00:00
|
|
|
$wgMemc->delete( $sidebarKey );
|
2009-07-31 07:12:25 +00:00
|
|
|
}
|
2011-01-26 17:41:18 +00:00
|
|
|
|
2010-09-04 04:00:09 +00:00
|
|
|
// Update the message in the message blob store
|
|
|
|
|
global $wgContLang;
|
|
|
|
|
MessageBlobStore::updateMessage( $wgContLang->lcfirst( $msg ) );
|
2008-07-05 15:39:10 +00:00
|
|
|
|
2011-01-26 17:41:18 +00:00
|
|
|
wfRunHooks( 'MessageCacheReplace', array( $title, $text ) );
|
2009-04-08 14:23:14 +00:00
|
|
|
|
2008-07-05 15:39:10 +00:00
|
|
|
wfProfileOut( __METHOD__ );
|
|
|
|
|
}
|
|
|
|
|
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
/**
|
|
|
|
|
* Is the given cache array expired due to time passing or a version change?
|
|
|
|
|
*/
|
|
|
|
|
protected function isCacheExpired( $cache ) {
|
|
|
|
|
if ( !isset( $cache['VERSION'] ) || !isset( $cache['EXPIRY'] ) ) {
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
if ( $cache['VERSION'] != MSG_CACHE_VERSION ) {
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
if ( wfTimestampNow() >= $cache['EXPIRY'] ) {
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
2008-07-05 15:39:10 +00:00
|
|
|
/**
|
|
|
|
|
* Shortcut to update caches.
|
|
|
|
|
*
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
* @param $cache array: cached messages with a version.
|
|
|
|
|
* @param $dest string: Either "local-only" to save to local caches only
|
|
|
|
|
* or "all" to save to all caches.
|
|
|
|
|
* @param $code string: Language code.
|
2012-02-09 17:42:35 +00:00
|
|
|
* @return bool on somekind of error.
|
2008-07-05 15:39:10 +00:00
|
|
|
*/
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
protected function saveToCaches( $cache, $dest, $code = false ) {
|
2008-07-05 15:39:10 +00:00
|
|
|
wfProfileIn( __METHOD__ );
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
global $wgUseLocalMessageCache;
|
2008-07-05 15:39:10 +00:00
|
|
|
|
|
|
|
|
$cacheKey = wfMemcKey( 'messages', $code );
|
|
|
|
|
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
if ( $dest === 'all' ) {
|
|
|
|
|
$success = $this->mMemc->set( $cacheKey, $cache );
|
* Converted BagOStuff.php from the style of memcached-client.php to the standard MediaWiki style, including camel case, using protected visibility instead of initial underscore, abstract functions instead of stubs, stylize.php.
* In SqlBagOStuff, ignore errors due to a read-only database, per my comments on CR r42796. Same for LocalisationCache.
* Merged SqlBagOStuff and MediaWikiBagOStuff, that proved to be an awkward and unnecessary generalisation. Use the standard quoting wrapper functions instead of $db->query().
* Implemented atomic incr() and decr() functions for SqlBagOStuff.
* Made incr() and decr() generally work roughly the same as it does in memcached, respecting negative steps instead of ignoring such operations. This allows decr() to be implemented in terms of incr().
* Per bug 11533, in MessageCache.php, don't retry 20 times on a cache failure, that's really memcached-specific and won't be useful for other cache types. It's not really very useful for memcached either.
* Moved MySQL-specific implementations of wasDeadlock() and wasErrorReissuable() to DatabaseMysql.
* Briefly tested page views with $wgReadOnly=read_only=1, fixed an error from Article::viewUpdates(). A CentralAuth fix will be in a subsequent commit.
2009-08-15 03:45:19 +00:00
|
|
|
} else {
|
|
|
|
|
$success = true;
|
2008-06-03 20:41:57 +00:00
|
|
|
}
|
2008-07-05 15:39:10 +00:00
|
|
|
|
|
|
|
|
# Save to local cache
|
* Introduced a new system for localisation caching. The system is based around fast fetches of individual messages, minimising memory overhead and startup time in the typical case. It handles both core messages (formerly in Language.php) and extension messages (formerly in MessageCache.php). Profiling indicates a significant win for average throughput.
* The serialized message cache, which would have been redundant, has been removed. Similar performance characteristics can be achieved with $wgLocalisationCacheConf['manualRecache'] = true;
* Added a maintenance script rebuildLocalisationCache.php for offline rebuilding of the localisation cache.
* Extension i18n files can now contain any of the variables which can be set in Messages*.php. It is possible, and recommended, to use this feature instead of the hooks for special page aliases and magic words.
* $wgExtensionAliasesFiles, LanguageGetMagic and LanguageGetSpecialPageAliases are retained for backwards compatibility. $wgMessageCache->addMessages() and related functions have been removed. wfLoadExtensionMessages() is a no-op and can continue to be called for b/c.
* Introduced $wgCacheDirectory as a default location for the various local caches that have accumulated. Suggested $IP/cache as a good place for it in the default LocalSettings.php and created this directory with a deny-all .htaccess.
* Patched Exception.php to avoid using the message cache when an exception is thrown from within LocalisationCache, since this tends to fail horribly.
* Removed Language::getLocalisationArray(), Language::loadLocalisation(), Language::load()
* Fixed FileDependency::__sleep()
* In Cdb.php, fixed newlines in debug messages
In MessageCache::get():
* Replaced calls to $wgContLang capitalisation functions with plain PHP functions, reducing the typical case from 99us to 93us. Message cache keys are already documented as being restricted to ASCII.
* Implemented a more efficient way to filter out bogus language codes, reducing the "foo/en" case from 430us to 101us
* Optimised wfRunHooks() in the typical do-nothing case, from ~30us to ~3us. This reduced MessageCache::get() typical case time from 93us to 38us.
* Removed hook MessageNotInMwNs to save an extra 3us per cache hit. Reimplemented the only user (LocalisationUpdate) using the new hook LocalisationCacheRecache.
2009-06-28 07:11:43 +00:00
|
|
|
if ( $wgUseLocalMessageCache ) {
|
2008-07-05 15:39:10 +00:00
|
|
|
$serialized = serialize( $cache );
|
|
|
|
|
$hash = md5( $serialized );
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
$this->mMemc->set( wfMemcKey( 'messages', $code, 'hash' ), $hash );
|
|
|
|
|
$this->saveToLocal( $serialized, $hash, $code );
|
2008-07-05 15:39:10 +00:00
|
|
|
}
|
|
|
|
|
|
2007-01-05 18:08:29 +00:00
|
|
|
wfProfileOut( __METHOD__ );
|
2008-07-05 15:39:10 +00:00
|
|
|
return $success;
|
2003-12-14 14:32:19 +00:00
|
|
|
}
|
|
|
|
|
|
2004-09-02 23:28:24 +00:00
|
|
|
/**
|
|
|
|
|
* Represents a write lock on the messages key
|
2010-04-09 21:27:23 +00:00
|
|
|
*
|
2011-09-14 15:07:20 +00:00
|
|
|
* @param $key string
|
|
|
|
|
*
|
2010-04-09 21:27:23 +00:00
|
|
|
* @return Boolean: success
|
2004-09-02 23:28:24 +00:00
|
|
|
*/
|
2011-01-26 17:41:18 +00:00
|
|
|
function lock( $key ) {
|
2008-07-05 15:39:10 +00:00
|
|
|
$lockKey = $key . ':lock';
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
$acquired = false;
|
|
|
|
|
$testDone = false;
|
|
|
|
|
for ( $i = 0; $i < MSG_WAIT_TIMEOUT && !$acquired; $i++ ) {
|
|
|
|
|
$acquired = $this->mMemc->add( $lockKey, 1, MSG_LOCK_TIMEOUT );
|
|
|
|
|
if ( $acquired ) {
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
# Fail fast if memcached is totally down
|
|
|
|
|
if ( !$testDone ) {
|
|
|
|
|
$testDone = true;
|
|
|
|
|
if ( !$this->mMemc->set( wfMemcKey( 'test' ), 'test', 1 ) ) {
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
}
|
2011-01-26 17:41:18 +00:00
|
|
|
sleep( 1 );
|
2003-12-14 14:32:19 +00:00
|
|
|
}
|
2004-09-20 14:55:25 +00:00
|
|
|
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
return $acquired;
|
2003-12-14 14:32:19 +00:00
|
|
|
}
|
2004-09-20 14:55:25 +00:00
|
|
|
|
2011-01-26 17:41:18 +00:00
|
|
|
function unlock( $key ) {
|
2008-07-05 15:39:10 +00:00
|
|
|
$lockKey = $key . ':lock';
|
2004-05-09 05:12:55 +00:00
|
|
|
$this->mMemc->delete( $lockKey );
|
2003-12-14 14:32:19 +00:00
|
|
|
}
|
2004-09-20 14:55:25 +00:00
|
|
|
|
2007-01-05 18:08:29 +00:00
|
|
|
/**
|
2013-03-28 10:24:19 +00:00
|
|
|
* Get a message from either the content language or the user language.
|
2007-01-05 18:08:29 +00:00
|
|
|
*
|
2013-03-28 10:24:19 +00:00
|
|
|
* @param $key String: the message cache key
|
|
|
|
|
* @param $useDB Boolean: get the message from the DB, false to use only
|
|
|
|
|
* the localisation
|
|
|
|
|
* @param bool|string $langcode Code of the language to get the message for, if
|
|
|
|
|
* it is a valid code create a language for that language,
|
|
|
|
|
* if it is a string but not a valid code then make a basic
|
|
|
|
|
* language object, if it is a false boolean then use the
|
|
|
|
|
* current users language (as a fallback for the old
|
|
|
|
|
* parameter functionality), or if it is a true boolean
|
|
|
|
|
* then use the wikis content language (also as a
|
|
|
|
|
* fallback).
|
2010-04-09 21:27:23 +00:00
|
|
|
* @param $isFullKey Boolean: specifies whether $key is a two part key
|
|
|
|
|
* "msg/lang".
|
2011-09-14 15:07:20 +00:00
|
|
|
*
|
2012-10-07 23:35:26 +00:00
|
|
|
* @throws MWException
|
2013-03-28 10:24:19 +00:00
|
|
|
* @return string|bool
|
2007-01-05 18:08:29 +00:00
|
|
|
*/
|
2008-04-27 14:50:55 +00:00
|
|
|
function get( $key, $useDB = true, $langcode = true, $isFullKey = false ) {
|
2010-09-07 22:37:55 +00:00
|
|
|
global $wgLanguageCode, $wgContLang;
|
2008-04-24 11:59:32 +00:00
|
|
|
|
2011-09-15 11:12:19 +00:00
|
|
|
if ( is_int( $key ) ) {
|
|
|
|
|
// "Non-string key given" exception sometimes happens for numerical strings that become ints somewhere on their way here
|
|
|
|
|
$key = strval( $key );
|
|
|
|
|
}
|
|
|
|
|
|
2010-08-05 16:40:50 +00:00
|
|
|
if ( !is_string( $key ) ) {
|
2011-01-26 17:41:18 +00:00
|
|
|
throw new MWException( 'Non-string key given' );
|
2010-08-05 16:40:50 +00:00
|
|
|
}
|
|
|
|
|
|
2009-07-10 11:54:32 +00:00
|
|
|
if ( strval( $key ) === '' ) {
|
|
|
|
|
# Shortcut: the empty key is always missing
|
2010-03-25 20:21:31 +00:00
|
|
|
return false;
|
2009-07-03 06:19:39 +00:00
|
|
|
}
|
|
|
|
|
|
2013-03-28 10:24:19 +00:00
|
|
|
$lang = wfGetLangObj( $langcode );
|
|
|
|
|
if ( !$lang ) {
|
2013-01-16 07:28:54 +00:00
|
|
|
throw new MWException( "Bad lang code $langcode given" );
|
|
|
|
|
}
|
2013-03-28 10:24:19 +00:00
|
|
|
|
|
|
|
|
$langcode = $lang->getCode();
|
|
|
|
|
|
|
|
|
|
$message = false;
|
2011-02-16 16:07:58 +00:00
|
|
|
|
Reverted breakage of non-ASCII message keys, Domas says that's not allowed. Optimised Language::lcfirst and Language::ucfirst() instead.
Timings in microseconds for ASCII no-change, ASCII change, non-ASCII no-change, non-ASCII change:
lcfirst: 1.8, 3.6, 21.2, 22.1
ucfirst: 1.5, 2.3, 21.1, 21.7
2009-08-28 17:58:54 +00:00
|
|
|
# Normalise title-case input (with some inlining)
|
* Introduced a new system for localisation caching. The system is based around fast fetches of individual messages, minimising memory overhead and startup time in the typical case. It handles both core messages (formerly in Language.php) and extension messages (formerly in MessageCache.php). Profiling indicates a significant win for average throughput.
* The serialized message cache, which would have been redundant, has been removed. Similar performance characteristics can be achieved with $wgLocalisationCacheConf['manualRecache'] = true;
* Added a maintenance script rebuildLocalisationCache.php for offline rebuilding of the localisation cache.
* Extension i18n files can now contain any of the variables which can be set in Messages*.php. It is possible, and recommended, to use this feature instead of the hooks for special page aliases and magic words.
* $wgExtensionAliasesFiles, LanguageGetMagic and LanguageGetSpecialPageAliases are retained for backwards compatibility. $wgMessageCache->addMessages() and related functions have been removed. wfLoadExtensionMessages() is a no-op and can continue to be called for b/c.
* Introduced $wgCacheDirectory as a default location for the various local caches that have accumulated. Suggested $IP/cache as a good place for it in the default LocalSettings.php and created this directory with a deny-all .htaccess.
* Patched Exception.php to avoid using the message cache when an exception is thrown from within LocalisationCache, since this tends to fail horribly.
* Removed Language::getLocalisationArray(), Language::loadLocalisation(), Language::load()
* Fixed FileDependency::__sleep()
* In Cdb.php, fixed newlines in debug messages
In MessageCache::get():
* Replaced calls to $wgContLang capitalisation functions with plain PHP functions, reducing the typical case from 99us to 93us. Message cache keys are already documented as being restricted to ASCII.
* Implemented a more efficient way to filter out bogus language codes, reducing the "foo/en" case from 430us to 101us
* Optimised wfRunHooks() in the typical do-nothing case, from ~30us to ~3us. This reduced MessageCache::get() typical case time from 93us to 38us.
* Removed hook MessageNotInMwNs to save an extra 3us per cache hit. Reimplemented the only user (LocalisationUpdate) using the new hook LocalisationCacheRecache.
2009-06-28 07:11:43 +00:00
|
|
|
$lckey = str_replace( ' ', '_', $key );
|
Reverted breakage of non-ASCII message keys, Domas says that's not allowed. Optimised Language::lcfirst and Language::ucfirst() instead.
Timings in microseconds for ASCII no-change, ASCII change, non-ASCII no-change, non-ASCII change:
lcfirst: 1.8, 3.6, 21.2, 22.1
ucfirst: 1.5, 2.3, 21.1, 21.7
2009-08-28 17:58:54 +00:00
|
|
|
if ( ord( $key ) < 128 ) {
|
|
|
|
|
$lckey[0] = strtolower( $lckey[0] );
|
|
|
|
|
$uckey = ucfirst( $lckey );
|
|
|
|
|
} else {
|
|
|
|
|
$lckey = $wgContLang->lcfirst( $lckey );
|
|
|
|
|
$uckey = $wgContLang->ucfirst( $lckey );
|
|
|
|
|
}
|
2007-01-07 14:22:32 +00:00
|
|
|
|
2007-01-05 18:08:29 +00:00
|
|
|
# Try the MediaWiki namespace
|
2013-03-28 10:24:19 +00:00
|
|
|
if( !$this->mDisable && $useDB ) {
|
|
|
|
|
$title = $uckey;
|
|
|
|
|
if( !$isFullKey && ( $langcode != $wgLanguageCode ) ) {
|
|
|
|
|
$title .= '/' . $langcode;
|
2004-11-21 13:56:04 +00:00
|
|
|
}
|
2013-03-28 10:24:19 +00:00
|
|
|
$message = $this->getMsgFromNamespace( $title, $langcode );
|
2004-05-15 03:36:39 +00:00
|
|
|
}
|
2003-12-14 14:32:19 +00:00
|
|
|
|
2004-09-24 00:09:42 +00:00
|
|
|
# Try the array in the language object
|
2008-07-05 15:39:10 +00:00
|
|
|
if ( $message === false ) {
|
2013-03-28 10:24:19 +00:00
|
|
|
$message = $lang->getMessage( $lckey );
|
|
|
|
|
if ( is_null( $message ) ) {
|
2005-10-22 20:52:30 +00:00
|
|
|
$message = false;
|
|
|
|
|
}
|
2004-09-20 14:55:25 +00:00
|
|
|
}
|
2003-12-14 14:32:19 +00:00
|
|
|
|
2013-03-28 10:24:19 +00:00
|
|
|
# Try the array of another language
|
* Introduced a new system for localisation caching. The system is based around fast fetches of individual messages, minimising memory overhead and startup time in the typical case. It handles both core messages (formerly in Language.php) and extension messages (formerly in MessageCache.php). Profiling indicates a significant win for average throughput.
* The serialized message cache, which would have been redundant, has been removed. Similar performance characteristics can be achieved with $wgLocalisationCacheConf['manualRecache'] = true;
* Added a maintenance script rebuildLocalisationCache.php for offline rebuilding of the localisation cache.
* Extension i18n files can now contain any of the variables which can be set in Messages*.php. It is possible, and recommended, to use this feature instead of the hooks for special page aliases and magic words.
* $wgExtensionAliasesFiles, LanguageGetMagic and LanguageGetSpecialPageAliases are retained for backwards compatibility. $wgMessageCache->addMessages() and related functions have been removed. wfLoadExtensionMessages() is a no-op and can continue to be called for b/c.
* Introduced $wgCacheDirectory as a default location for the various local caches that have accumulated. Suggested $IP/cache as a good place for it in the default LocalSettings.php and created this directory with a deny-all .htaccess.
* Patched Exception.php to avoid using the message cache when an exception is thrown from within LocalisationCache, since this tends to fail horribly.
* Removed Language::getLocalisationArray(), Language::loadLocalisation(), Language::load()
* Fixed FileDependency::__sleep()
* In Cdb.php, fixed newlines in debug messages
In MessageCache::get():
* Replaced calls to $wgContLang capitalisation functions with plain PHP functions, reducing the typical case from 99us to 93us. Message cache keys are already documented as being restricted to ASCII.
* Implemented a more efficient way to filter out bogus language codes, reducing the "foo/en" case from 430us to 101us
* Optimised wfRunHooks() in the typical do-nothing case, from ~30us to ~3us. This reduced MessageCache::get() typical case time from 93us to 38us.
* Removed hook MessageNotInMwNs to save an extra 3us per cache hit. Reimplemented the only user (LocalisationUpdate) using the new hook LocalisationCacheRecache.
2009-06-28 07:11:43 +00:00
|
|
|
if( $message === false ) {
|
|
|
|
|
$parts = explode( '/', $lckey );
|
|
|
|
|
# We may get calls for things that are http-urls from sidebar
|
|
|
|
|
# Let's not load nonexistent languages for those
|
|
|
|
|
# They usually have more than one slash.
|
|
|
|
|
if ( count( $parts ) == 2 && $parts[1] !== '' ) {
|
|
|
|
|
$message = Language::getMessageFor( $parts[0], $parts[1] );
|
|
|
|
|
if ( is_null( $message ) ) {
|
|
|
|
|
$message = false;
|
2006-08-10 09:10:06 +00:00
|
|
|
}
|
2006-08-07 12:21:06 +00:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
2013-03-28 10:24:19 +00:00
|
|
|
# Is this a custom message? Try the default language in the db...
|
|
|
|
|
if( ( $message === false || $message === '-' ) &&
|
|
|
|
|
!$this->mDisable && $useDB &&
|
|
|
|
|
!$isFullKey && ( $langcode != $wgLanguageCode ) ) {
|
|
|
|
|
$message = $this->getMsgFromNamespace( $uckey, $wgLanguageCode );
|
|
|
|
|
}
|
|
|
|
|
|
2003-12-14 14:32:19 +00:00
|
|
|
# Final fallback
|
2005-09-11 14:50:47 +00:00
|
|
|
if( $message === false ) {
|
2010-03-25 20:21:31 +00:00
|
|
|
return false;
|
2003-12-14 14:32:19 +00:00
|
|
|
}
|
2009-06-04 07:26:46 +00:00
|
|
|
|
|
|
|
|
# Fix whitespace
|
2010-02-05 04:25:30 +00:00
|
|
|
$message = strtr( $message,
|
2009-06-04 07:26:46 +00:00
|
|
|
array(
|
|
|
|
|
# Fix for trailing whitespace, removed by textarea
|
|
|
|
|
' ' => ' ',
|
|
|
|
|
# Fix for NBSP, converted to space by firefox
|
|
|
|
|
' ' => "\xc2\xa0",
|
2010-05-30 17:33:59 +00:00
|
|
|
' ' => "\xc2\xa0",
|
2009-06-04 07:26:46 +00:00
|
|
|
) );
|
|
|
|
|
|
2003-12-14 14:32:19 +00:00
|
|
|
return $message;
|
|
|
|
|
}
|
2005-07-07 03:08:58 +00:00
|
|
|
|
2007-01-05 18:08:29 +00:00
|
|
|
/**
|
2008-04-14 07:45:50 +00:00
|
|
|
* Get a message from the MediaWiki namespace, with caching. The key must
|
2007-01-05 18:08:29 +00:00
|
|
|
* first be converted to two-part lang/msg form if necessary.
|
|
|
|
|
*
|
2013-03-26 15:42:48 +00:00
|
|
|
* Unlike self::get(), this function doesn't resolve fallback chains, and
|
|
|
|
|
* some callers require this behavior. LanguageConverter::parseCachedTable()
|
|
|
|
|
* and self::get() are some examples in core.
|
|
|
|
|
*
|
2013-03-11 17:15:01 +00:00
|
|
|
* @param string $title Message cache key with initial uppercase letter.
|
|
|
|
|
* @param string $code code denoting the language to try.
|
2011-09-14 15:07:20 +00:00
|
|
|
*
|
2012-02-10 15:37:33 +00:00
|
|
|
* @return string|bool False on failure
|
2007-01-05 18:08:29 +00:00
|
|
|
*/
|
2008-07-05 15:39:10 +00:00
|
|
|
function getMsgFromNamespace( $title, $code ) {
|
2010-08-05 18:19:34 +00:00
|
|
|
$this->load( $code );
|
2010-08-08 00:28:17 +00:00
|
|
|
if ( isset( $this->mCache[$code][$title] ) ) {
|
2010-08-05 18:19:34 +00:00
|
|
|
$entry = $this->mCache[$code][$title];
|
2010-08-08 00:28:17 +00:00
|
|
|
if ( substr( $entry, 0, 1 ) === ' ' ) {
|
2010-08-05 18:19:34 +00:00
|
|
|
return substr( $entry, 1 );
|
2010-08-08 00:28:17 +00:00
|
|
|
} elseif ( $entry === '!NONEXISTENT' ) {
|
|
|
|
|
return false;
|
2010-08-21 16:41:53 +00:00
|
|
|
} elseif( $entry === '!TOO BIG' ) {
|
|
|
|
|
// Fall through and try invididual message cache below
|
2010-12-10 13:18:11 +00:00
|
|
|
}
|
|
|
|
|
} else {
|
|
|
|
|
// XXX: This is not cached in process cache, should it?
|
|
|
|
|
$message = false;
|
2011-01-26 17:41:18 +00:00
|
|
|
wfRunHooks( 'MessagesPreLoad', array( $title, &$message ) );
|
2010-12-10 13:18:11 +00:00
|
|
|
if ( $message !== false ) {
|
|
|
|
|
return $message;
|
|
|
|
|
}
|
2007-01-05 18:08:29 +00:00
|
|
|
|
2012-10-24 14:59:37 +00:00
|
|
|
return false;
|
2007-01-05 18:08:29 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
# Try the individual message cache
|
2010-08-08 00:28:17 +00:00
|
|
|
$titleKey = wfMemcKey( 'messages', 'individual', $title );
|
2010-08-05 18:19:34 +00:00
|
|
|
$entry = $this->mMemc->get( $titleKey );
|
|
|
|
|
if ( $entry ) {
|
2010-08-08 00:28:17 +00:00
|
|
|
if ( substr( $entry, 0, 1 ) === ' ' ) {
|
2010-08-05 18:19:34 +00:00
|
|
|
$this->mCache[$code][$title] = $entry;
|
2010-08-08 00:28:17 +00:00
|
|
|
return substr( $entry, 1 );
|
2010-08-05 18:19:34 +00:00
|
|
|
} elseif ( $entry === '!NONEXISTENT' ) {
|
2010-08-08 00:28:17 +00:00
|
|
|
$this->mCache[$code][$title] = '!NONEXISTENT';
|
2010-08-05 18:19:34 +00:00
|
|
|
return false;
|
|
|
|
|
} else {
|
|
|
|
|
# Corrupt/obsolete entry, delete it
|
|
|
|
|
$this->mMemc->delete( $titleKey );
|
2005-03-13 11:50:54 +00:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
2010-08-08 00:28:17 +00:00
|
|
|
# Try loading it from the database
|
2012-09-05 15:50:13 +00:00
|
|
|
$revision = Revision::newFromTitle(
|
|
|
|
|
Title::makeTitle( NS_MEDIAWIKI, $title ), false, Revision::READ_LATEST
|
|
|
|
|
);
|
2010-08-08 00:28:17 +00:00
|
|
|
if ( $revision ) {
|
2012-06-08 07:41:04 +00:00
|
|
|
$content = $revision->getContent();
|
|
|
|
|
if ( !$content ) {
|
2011-09-27 00:41:24 +00:00
|
|
|
// A possibly temporary loading failure.
|
2012-02-16 05:04:17 +00:00
|
|
|
wfDebugLog( 'MessageCache', __METHOD__ . ": failed to load message page text for {$title} ($code)" );
|
2012-06-08 07:41:04 +00:00
|
|
|
$message = null; // no negative caching
|
2011-09-27 00:41:24 +00:00
|
|
|
} else {
|
2012-10-08 15:26:11 +00:00
|
|
|
// XXX: Is this the right way to turn a Content object into a message?
|
|
|
|
|
// NOTE: $content is typically either WikitextContent, JavaScriptContent or CssContent.
|
|
|
|
|
// MessageContent is *not* used for storing messages, it's only used for wrapping them when needed.
|
2012-08-20 19:33:07 +00:00
|
|
|
$message = $content->getWikitextForTransclusion();
|
2012-06-08 07:41:04 +00:00
|
|
|
|
|
|
|
|
if ( $message === false || $message === null ) {
|
2012-08-20 19:33:07 +00:00
|
|
|
wfDebugLog( 'MessageCache', __METHOD__ . ": message content doesn't provide wikitext "
|
2012-10-08 15:26:11 +00:00
|
|
|
. "(content model: " . $content->getContentHandler() . ")" );
|
2012-08-20 19:33:07 +00:00
|
|
|
|
2012-06-08 07:41:04 +00:00
|
|
|
$message = false; // negative caching
|
|
|
|
|
} else {
|
|
|
|
|
$this->mCache[$code][$title] = ' ' . $message;
|
|
|
|
|
$this->mMemc->set( $titleKey, ' ' . $message, $this->mExpiry );
|
|
|
|
|
}
|
2011-09-27 00:41:24 +00:00
|
|
|
}
|
2005-09-11 14:50:47 +00:00
|
|
|
} else {
|
2012-06-08 07:41:04 +00:00
|
|
|
$message = false; // negative caching
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if ( $message === false ) { // negative caching
|
2010-08-08 00:28:17 +00:00
|
|
|
$this->mCache[$code][$title] = '!NONEXISTENT';
|
2008-07-05 15:39:10 +00:00
|
|
|
$this->mMemc->set( $titleKey, '!NONEXISTENT', $this->mExpiry );
|
2005-03-13 11:50:54 +00:00
|
|
|
}
|
2010-08-08 00:28:17 +00:00
|
|
|
|
2005-03-13 11:50:54 +00:00
|
|
|
return $message;
|
|
|
|
|
}
|
2004-03-01 05:51:55 +00:00
|
|
|
|
2011-04-18 12:43:53 +00:00
|
|
|
/**
|
|
|
|
|
* @param $message string
|
|
|
|
|
* @param $interface bool
|
|
|
|
|
* @param $language
|
|
|
|
|
* @param $title Title
|
|
|
|
|
* @return string
|
|
|
|
|
*/
|
2011-02-09 15:19:45 +00:00
|
|
|
function transform( $message, $interface = false, $language = null, $title = null ) {
|
* Introduced a new system for localisation caching. The system is based around fast fetches of individual messages, minimising memory overhead and startup time in the typical case. It handles both core messages (formerly in Language.php) and extension messages (formerly in MessageCache.php). Profiling indicates a significant win for average throughput.
* The serialized message cache, which would have been redundant, has been removed. Similar performance characteristics can be achieved with $wgLocalisationCacheConf['manualRecache'] = true;
* Added a maintenance script rebuildLocalisationCache.php for offline rebuilding of the localisation cache.
* Extension i18n files can now contain any of the variables which can be set in Messages*.php. It is possible, and recommended, to use this feature instead of the hooks for special page aliases and magic words.
* $wgExtensionAliasesFiles, LanguageGetMagic and LanguageGetSpecialPageAliases are retained for backwards compatibility. $wgMessageCache->addMessages() and related functions have been removed. wfLoadExtensionMessages() is a no-op and can continue to be called for b/c.
* Introduced $wgCacheDirectory as a default location for the various local caches that have accumulated. Suggested $IP/cache as a good place for it in the default LocalSettings.php and created this directory with a deny-all .htaccess.
* Patched Exception.php to avoid using the message cache when an exception is thrown from within LocalisationCache, since this tends to fail horribly.
* Removed Language::getLocalisationArray(), Language::loadLocalisation(), Language::load()
* Fixed FileDependency::__sleep()
* In Cdb.php, fixed newlines in debug messages
In MessageCache::get():
* Replaced calls to $wgContLang capitalisation functions with plain PHP functions, reducing the typical case from 99us to 93us. Message cache keys are already documented as being restricted to ASCII.
* Implemented a more efficient way to filter out bogus language codes, reducing the "foo/en" case from 430us to 101us
* Optimised wfRunHooks() in the typical do-nothing case, from ~30us to ~3us. This reduced MessageCache::get() typical case time from 93us to 38us.
* Removed hook MessageNotInMwNs to save an extra 3us per cache hit. Reimplemented the only user (LocalisationUpdate) using the new hook LocalisationCacheRecache.
2009-06-28 07:11:43 +00:00
|
|
|
// Avoid creating parser if nothing to transform
|
2008-07-05 15:39:10 +00:00
|
|
|
if( strpos( $message, '{{' ) === false ) {
|
|
|
|
|
return $message;
|
|
|
|
|
}
|
|
|
|
|
|
2011-04-18 12:43:53 +00:00
|
|
|
if ( $this->mInParser ) {
|
2011-04-18 12:59:50 +00:00
|
|
|
return $message;
|
2011-04-18 12:43:53 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
$parser = $this->getParser();
|
|
|
|
|
if ( $parser ) {
|
|
|
|
|
$popts = $this->getParserOptions();
|
|
|
|
|
$popts->setInterfaceMessage( $interface );
|
|
|
|
|
$popts->setTargetLanguage( $language );
|
|
|
|
|
|
2011-05-15 07:16:25 +00:00
|
|
|
$userlang = $popts->setUserLang( $language );
|
2011-04-18 12:43:53 +00:00
|
|
|
$this->mInParser = true;
|
|
|
|
|
$message = $parser->transformMsg( $message, $popts, $title );
|
|
|
|
|
$this->mInParser = false;
|
2011-05-15 07:16:25 +00:00
|
|
|
$popts->setUserLang( $userlang );
|
2011-04-18 12:43:53 +00:00
|
|
|
}
|
|
|
|
|
return $message;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* @return Parser
|
|
|
|
|
*/
|
|
|
|
|
function getParser() {
|
2008-08-26 14:37:15 +00:00
|
|
|
global $wgParser, $wgParserConf;
|
2006-07-03 11:17:27 +00:00
|
|
|
if ( !$this->mParser && isset( $wgParser ) ) {
|
2006-07-04 09:13:10 +00:00
|
|
|
# Do some initialisation so that we don't have to do it twice
|
|
|
|
|
$wgParser->firstCallInit();
|
|
|
|
|
# Clone it and store it
|
2008-08-26 14:37:15 +00:00
|
|
|
$class = $wgParserConf['class'];
|
|
|
|
|
if ( $class == 'Parser_DiffTest' ) {
|
|
|
|
|
# Uncloneable
|
|
|
|
|
$this->mParser = new $class( $wgParserConf );
|
|
|
|
|
} else {
|
|
|
|
|
$this->mParser = clone $wgParser;
|
|
|
|
|
}
|
2006-07-03 11:17:27 +00:00
|
|
|
}
|
2011-04-18 12:43:53 +00:00
|
|
|
return $this->mParser;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* @param $text string
|
2011-04-22 20:17:21 +00:00
|
|
|
* @param $title Title
|
2011-04-18 12:43:53 +00:00
|
|
|
* @param $linestart bool
|
2011-09-14 15:07:20 +00:00
|
|
|
* @param $interface bool
|
2011-04-18 12:43:53 +00:00
|
|
|
* @param $language
|
2012-11-01 17:13:30 +00:00
|
|
|
* @return ParserOutput|string
|
2011-04-18 12:43:53 +00:00
|
|
|
*/
|
2011-04-22 20:17:21 +00:00
|
|
|
public function parse( $text, $title = null, $linestart = true, $interface = false, $language = null ) {
|
2011-04-18 12:43:53 +00:00
|
|
|
if ( $this->mInParser ) {
|
2011-04-18 12:59:50 +00:00
|
|
|
return htmlspecialchars( $text );
|
2011-04-18 12:43:53 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
$parser = $this->getParser();
|
|
|
|
|
$popts = $this->getParserOptions();
|
2012-03-02 04:00:58 +00:00
|
|
|
$popts->setInterfaceMessage( $interface );
|
|
|
|
|
$popts->setTargetLanguage( $language );
|
2011-04-18 12:43:53 +00:00
|
|
|
|
2011-09-14 12:32:22 +00:00
|
|
|
wfProfileIn( __METHOD__ );
|
2011-04-22 20:17:21 +00:00
|
|
|
if ( !$title || !$title instanceof Title ) {
|
|
|
|
|
global $wgTitle;
|
|
|
|
|
$title = $wgTitle;
|
2011-04-18 14:02:13 +00:00
|
|
|
}
|
2011-09-14 12:32:22 +00:00
|
|
|
// Sometimes $wgTitle isn't set either...
|
|
|
|
|
if ( !$title ) {
|
|
|
|
|
# It's not uncommon having a null $wgTitle in scripts. See r80898
|
|
|
|
|
# Create a ghost title in such case
|
|
|
|
|
$title = Title::newFromText( 'Dwimmerlaik' );
|
|
|
|
|
}
|
2011-04-22 20:17:21 +00:00
|
|
|
|
2011-04-18 12:43:53 +00:00
|
|
|
$this->mInParser = true;
|
|
|
|
|
$res = $parser->parse( $text, $title, $popts, $linestart );
|
|
|
|
|
$this->mInParser = false;
|
|
|
|
|
|
2011-09-14 12:32:22 +00:00
|
|
|
wfProfileOut( __METHOD__ );
|
2011-04-18 12:43:53 +00:00
|
|
|
return $res;
|
2004-04-05 10:38:40 +00:00
|
|
|
}
|
2004-09-20 14:55:25 +00:00
|
|
|
|
2011-01-26 17:41:18 +00:00
|
|
|
function disable() {
|
|
|
|
|
$this->mDisable = true;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
function enable() {
|
|
|
|
|
$this->mDisable = false;
|
|
|
|
|
}
|
2010-02-05 04:25:30 +00:00
|
|
|
|
2004-09-02 23:28:24 +00:00
|
|
|
/**
|
|
|
|
|
* Clear all stored messages. Mainly used after a mass rebuild.
|
|
|
|
|
*/
|
2004-08-11 02:31:47 +00:00
|
|
|
function clear() {
|
2012-03-08 20:56:26 +00:00
|
|
|
$langs = Language::fetchLanguageNames( null, 'mw' );
|
2012-10-24 14:59:37 +00:00
|
|
|
foreach ( array_keys( $langs ) as $code ) {
|
2010-08-05 18:19:34 +00:00
|
|
|
# Global cache
|
|
|
|
|
$this->mMemc->delete( wfMemcKey( 'messages', $code ) );
|
|
|
|
|
# Invalidate all local caches
|
|
|
|
|
$this->mMemc->delete( wfMemcKey( 'messages', $code, 'hash' ) );
|
2004-08-11 02:31:47 +00:00
|
|
|
}
|
2010-09-04 13:27:12 +00:00
|
|
|
$this->mLoadedLanguages = array();
|
2004-08-11 02:31:47 +00:00
|
|
|
}
|
2006-08-24 16:58:44 +00:00
|
|
|
|
2011-09-14 15:07:20 +00:00
|
|
|
/**
|
|
|
|
|
* @param $key
|
|
|
|
|
* @return array
|
|
|
|
|
*/
|
2008-07-05 15:39:10 +00:00
|
|
|
public function figureMessage( $key ) {
|
2010-09-07 22:37:55 +00:00
|
|
|
global $wgLanguageCode;
|
2008-10-07 18:10:08 +00:00
|
|
|
$pieces = explode( '/', $key );
|
2011-01-26 17:41:18 +00:00
|
|
|
if( count( $pieces ) < 2 ) {
|
2010-09-07 22:37:55 +00:00
|
|
|
return array( $key, $wgLanguageCode );
|
2011-01-26 17:41:18 +00:00
|
|
|
}
|
2008-07-05 15:39:10 +00:00
|
|
|
|
2008-10-07 18:10:08 +00:00
|
|
|
$lang = array_pop( $pieces );
|
2012-03-08 20:56:26 +00:00
|
|
|
if( !Language::fetchLanguageName( $lang, null, 'mw' ) ) {
|
2010-09-07 22:37:55 +00:00
|
|
|
return array( $key, $wgLanguageCode );
|
2011-01-26 17:41:18 +00:00
|
|
|
}
|
2008-07-05 15:39:10 +00:00
|
|
|
|
2008-10-07 18:10:08 +00:00
|
|
|
$message = implode( '/', $pieces );
|
|
|
|
|
return array( $message, $lang );
|
2008-07-05 15:39:10 +00:00
|
|
|
}
|
|
|
|
|
|
2011-09-14 19:59:50 +00:00
|
|
|
/**
|
|
|
|
|
* Get all message keys stored in the message cache for a given language.
|
|
|
|
|
* If $code is the content language code, this will return all message keys
|
|
|
|
|
* for which MediaWiki:msgkey exists. If $code is another language code, this
|
|
|
|
|
* will ONLY return message keys for which MediaWiki:msgkey/$code exists.
|
|
|
|
|
* @param $code string
|
|
|
|
|
* @return array of message keys (strings)
|
|
|
|
|
*/
|
|
|
|
|
public function getAllMessageKeys( $code ) {
|
|
|
|
|
global $wgContLang;
|
|
|
|
|
$this->load( $code );
|
|
|
|
|
if ( !isset( $this->mCache[$code] ) ) {
|
|
|
|
|
// Apparently load() failed
|
|
|
|
|
return null;
|
|
|
|
|
}
|
Fix message cache expiry semantics
* Use the stale message cache while the new one is being generated
* Revert I811755d4 (make message cache load failure fatal). This
escalated several very plausible temporary site issues from barely
noticeable to complete downtime -- for example, memcached being down
on a site with only one memcached server.
* Remove $wgLocalMessageCacheSerialized, it's always been pointless
* Clarify a couple of comments.
* Increased lock wait timeout to 30s
* Make lock() fail immediately on memcached connection refused
Tests done:
* With local cache enabled: normal cold refill; refill local from
global cache; use stale local cache during remote refill; use stale
global cache during remote refill; cold cache wait for remote refill;
saveToCaches() failure; memcached connection refused.
* With local cache disabled: saveToCaches() failure; cache disabled due
to "error" status key; memcached connection refused.
Setting a 1-day expiry in memcached, with a ~10s CPU cost to replace, is
not the best idea since it inevitably leads to a cache stampede. Dealing
with the stampede by waiting for a lock is not ideal, even if it were
implemented properly, since it's not necessary to deliver perfectly
fresh message cache data to all clients.
This is especially obvious when you note that barring bugs, expiry and
regeneration always gives you back the exact same data, because we have
incremental updates (MessageCache::replace()). Keeping all clients
waiting for 10s just to give them the data they have already is pretty
pointless.
So, continue to serve the site from the stale message cache while the
new one is being generated.
One caveat: if local caching enabled, when the message cache becomes
stale, a sudden spike in network bandwidth may result due to the full
array (also typically stale) being fetched from the shared cache.
Bug: 43516
Change-Id: Ia145fd90da33956d8aac127634606aaecfaa176b
2013-04-03 10:54:34 +00:00
|
|
|
// Remove administrative keys
|
|
|
|
|
$cache = $this->mCache[$code];
|
|
|
|
|
unset( $cache['VERSION'] );
|
|
|
|
|
unset( $cache['EXPIRY'] );
|
|
|
|
|
// Remove any !NONEXISTENT keys
|
|
|
|
|
$cache = array_diff( $cache, array( '!NONEXISTENT' ) );
|
2011-09-14 19:59:50 +00:00
|
|
|
// Keys may appear with a capital first letter. lcfirst them.
|
|
|
|
|
return array_map( array( $wgContLang, 'lcfirst' ), array_keys( $cache ) );
|
|
|
|
|
}
|
2003-12-14 14:32:19 +00:00
|
|
|
}
|