Thijs/wiki.techinc.nl
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

objectcache: Remove $wgMainWANCache and $wgWANObjectCaches

Browse source 
We always wrap the local cluster cache, and there are no subclasses
of WANObjectCache. It was never documented or recommended how these
would be used. It is a left-over from the original 2015 Multi-DC plan
in which WANObjectCache would work differently. See task for details.

Note that this requires no configuration changes, even in the
theoretical case of these variables being used, as the only
option is to use the main cache, and that's also the default.

* Update WAN overrides to override the underlying main cache
  instead.
* Fix EditPageTest which was previously implicitly using a 'hash'
  as main cache but also relying on wan cache to be 'none'.
  The part that it actually needs is the 'none'. When WAN cache is
  enabled, testUpdateNoMinor fails due to an edit conflict because
  one of the edits it makes is made with a current timestamp whereas
  it expects to simulate wpEdittime in the year 2012 which, when
  caching is enabled, is ignored and becomes the current time instead.
  I don't understand exactly why, but I'm going to conserve that
  behaviour for now.
* Fix TemplateCategoriesTest, which was failing due to an unexpected
  cache hit:
  > [objectcache] fetchOrRegenerate(…:page:10:…): volatile hit
  This could be solved in a more realistic way by splitting the test,
  or by explicitly resetting services half-way the test to clear
  WikiPageFactory, PageStore and WANCache process state.
  For now, keep the prior behaviour of no cache in this test.

Bug: T305093
Bug: T329680
Depends-On: If890622eed0d0f8b4bd73d36ba1815a3d760ea05
Depends-On: Ie1def75208822bdf19bb2cfd7e6edf32c2000e6b
Depends-On: I35cce61dc3ee90dcee3dd6f0b36f84133be029ed
Change-Id: I53781a8c06ebb2583f6ca83dd91bbfe8a5c88b13
This commit is contained in:
Timo Tijhof 2023-02-14 21:43:12 +00:00
parent f1fa0aa64c
commit 39ead04e98
16 changed files with 54 additions and 230 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
.phan/config.php
View file

@ -173,7 +173,6 @@ $cfg['globals_type_map'] = array_merge( $cfg['globals_type_map'], [
'wgLogActionsHandlers' => 'array<string,class-string>',
'wgPasswordPolicy' => 'array<string,array<string,string|array>>',
'wgVirtualRestConfig' => 'array<string,array>',
'wgWANObjectCaches' => 'array[]',
'wgLocalInterwikis' => 'string[]',
'wgDebugLogGroups' => 'string|false|array{destination:string,sample?:int,level:int}',
'wgCookiePrefix' => 'string|false',

6
RELEASE-NOTES-1.40
View file

@ -49,6 +49,12 @@ For notes on 1.39.x and older releases, see HISTORY.
==== Removed configuration ====
* $wgShellboxUrl that was deprecated in 1.37 has now been removed,
set $wgShellboxUrls instead.
* $wgMainWANCache and $wgWANObjectCaches have been removed. To inject
WANObjectCache parameters, use $wgWANObjectCache instead. These variables
were introduced for multi-DC wiki farms to add a separate memcached
proxy for cross-DC relaying of purges. The configuration was never used
because WANObjectCache works based on route prefixes, that can be
transparently handled by the main memcached proxy.
* $wgParserTestFiles which was deprecated in 1.30 has now been removed;
extensions can place their parser test files in `tests/parser` instead.
* (T231412) $wgAutoloadAttemptLowercase, which was deprecated in 1.35,

53
docs/config-schema.yaml
View file

@ -2304,6 +2304,11 @@ config-schema:
- CACHE_ACCEL: APC, APCU or WinCache
- (other): A string may be used which identifies a cache
configuration in $wgObjectCaches.
For a multi-datacenter setup, the underlying service should be configured
to broadcast operations by WANObjectCache using Mcrouter or Dynomite.
See @ref wanobjectcache-deployment "Deploying WANObjectCache".
To configure the `broadcastRoutingPrefix` WANObjectCache parameter,
use $wgWANObjectCache.
@see $wgMessageCacheType, $wgParserCacheType
MessageCacheType:
default: -1
@ -2416,51 +2421,13 @@ config-schema:
For MemcachedPhpBagOStuff parameters see {@link MemcachedPhpBagOStuff::__construct}
For MemcachedPeclBagOStuff parameters see {@link MemcachedPeclBagOStuff::__construct}
For RedisBagOStuff parameters see {@link RedisBagOStuff::__construct}
MainWANCache:
default: false
type:
- integer
- string
- boolean
description: |-
Main Wide-Area-Network cache type.
By default, this will wrap $wgMainCacheType (which is disabled, since the basic
stock default of CACHE_DB is not fast enough to make it worthwhile).
For single server or single datacenter setup, setting $wgMainCacheType
is enough.
For a multiple datacenter setup, WANObjectCache should be configured to
broadcast some if its operations using Mcrouter or Dynomite.
See @ref wanobjectcache-deployment "Deploying WANObjectCache".
The options are:
- false: Configure the cache using $wgMainCacheType, without using
a relayer (only matters if there are multiple datacenters)
- CACHE_NONE: Do not cache
- (other): A string may be used which identifies a cache
configuration in $wgWANObjectCaches
@since 1.26
WANObjectCaches:
default:
- { class: WANObjectCache, cacheId: 0 }
WANObjectCache:
default: { }
type: object
description: |-
Advanced WAN object cache configuration.
The format is an associative array where the key is an identifier
that may be referenced by $wgMainWANCache, and the value is an array of options:
- class: (Required) The class to use (must be WANObjectCache or a subclass).
- cacheId: (Required) A cache identifier from $wgObjectCaches.
- secret: (Optional) Stable secret for hashing long strings in key components.
Default: $wgSecretKey.
Any other options are treated as constructor parameters to WANObjectCache,
except for 'cache', 'logger', 'stats' and 'asyncHandler' which are
unconditionally set by MediaWiki core's ServiceWiring.
**Example:**
```
$wgWANObjectCaches['memcached-php'] => [
'class' => WANObjectCache::class,
'cacheId' => 'memcached-php',
];
```
@since 1.26
Extra parameters to the WANObjectCache constructor.
See @ref wanobjectcache-deployment "Deploying WANObjectCache".
@since 1.40
MainStash:
default: db-replicated
description: |-

12
docs/config-vars.php
View file

@ -1470,16 +1470,10 @@ $wgStatsCacheType = null;
$wgObjectCaches = null;
/**
* Config variable stub for the MainWANCache setting, for use by phpdoc and IDEs.
* @see MediaWiki\MainConfigSchema::MainWANCache
* Config variable stub for the WANObjectCache setting, for use by phpdoc and IDEs.
* @see MediaWiki\MainConfigSchema::WANObjectCache
*/
$wgMainWANCache = null;
/**
* Config variable stub for the WANObjectCaches setting, for use by phpdoc and IDEs.
* @see MediaWiki\MainConfigSchema::WANObjectCaches
*/
$wgWANObjectCaches = null;
$wgWANObjectCache = null;
/**
* Config variable stub for the MainStash setting, for use by phpdoc and IDEs.

12
includes/MainConfigNames.php
View file

@ -1485,16 +1485,10 @@ class MainConfigNames {
public const ObjectCaches = 'ObjectCaches';
/**
* Name constant for the MainWANCache setting, for use with Config::get()
* @see MainConfigSchema::MainWANCache
* Name constant for the WANObjectCache setting, for use with Config::get()
* @see MainConfigSchema::WANObjectCache
*/
public const MainWANCache = 'MainWANCache';
/**
* Name constant for the WANObjectCaches setting, for use with Config::get()
* @see MainConfigSchema::WANObjectCaches
*/
public const WANObjectCaches = 'WANObjectCaches';
public const WANObjectCache = 'WANObjectCache';
/**
* Name constant for the MainStash setting, for use with Config::get()

66
includes/MainConfigSchema.php
View file

@ -79,7 +79,6 @@ use UploadLogFormatter;
use UserEditCountInitJob;
use UserGroupExpiryJob;
use UserOptionsUpdateJob;
use WANObjectCache;
use WatchlistExpiryJob;
use WebRequest;
use WikitextContentHandler;
@ -3746,6 +3745,12 @@ class MainConfigSchema {
* - (other): A string may be used which identifies a cache
* configuration in $wgObjectCaches.
*
* For a multi-datacenter setup, the underlying service should be configured
* to broadcast operations by WANObjectCache using Mcrouter or Dynomite.
* See @ref wanobjectcache-deployment "Deploying WANObjectCache".
* To configure the `broadcastRoutingPrefix` WANObjectCache parameter,
* use $wgWANObjectCache.
*
* @see $wgMessageCacheType, $wgParserCacheType
*/
public const MainCacheType = [
@ -3929,65 +3934,14 @@ class MainConfigSchema {
];
/**
* Main Wide-Area-Network cache type.
* Extra parameters to the WANObjectCache constructor.
*
* By default, this will wrap $wgMainCacheType (which is disabled, since the basic
* stock default of CACHE_DB is not fast enough to make it worthwhile).
*
* For single server or single datacenter setup, setting $wgMainCacheType
* is enough.
*
* For a multiple datacenter setup, WANObjectCache should be configured to
* broadcast some if its operations using Mcrouter or Dynomite.
* See @ref wanobjectcache-deployment "Deploying WANObjectCache".
*
* The options are:
* - false: Configure the cache using $wgMainCacheType, without using
* a relayer (only matters if there are multiple datacenters)
* - CACHE_NONE: Do not cache
* - (other): A string may be used which identifies a cache
* configuration in $wgWANObjectCaches
*
* @since 1.26
* @since 1.40
*/
public const MainWANCache = [
'default' => false,
'type' => 'integer|string|false',
];
/**
* Advanced WAN object cache configuration.
*
* The format is an associative array where the key is an identifier
* that may be referenced by $wgMainWANCache, and the value is an array of options:
*
* - class: (Required) The class to use (must be WANObjectCache or a subclass).
* - cacheId: (Required) A cache identifier from $wgObjectCaches.
* - secret: (Optional) Stable secret for hashing long strings in key components.
* Default: $wgSecretKey.
*
* Any other options are treated as constructor parameters to WANObjectCache,
* except for 'cache', 'logger', 'stats' and 'asyncHandler' which are
* unconditionally set by MediaWiki core's ServiceWiring.
*
* **Example:**
*
* ```
* $wgWANObjectCaches['memcached-php'] => [
* 'class' => WANObjectCache::class,
* 'cacheId' => 'memcached-php',
* ];
* ```
*
* @since 1.26
*/
public const WANObjectCaches = [
'default' => [
CACHE_NONE => [
'class' => WANObjectCache::class,
'cacheId' => CACHE_NONE,
]
],
public const WANObjectCache = [
'default' => [],
'type' => 'map',
];

35
includes/ServiceWiring.php
View file

@ -1018,45 +1018,24 @@ return [
'MainWANObjectCache' => static function ( MediaWikiServices $services ): WANObjectCache {
$mainConfig = $services->getMainConfig();
$wanId = $mainConfig->get( MainConfigNames::MainWANCache );
$wanParams = $mainConfig->get( MainConfigNames::WANObjectCaches )[$wanId] ?? null;
if ( !$wanParams ) {
throw new UnexpectedValueException(
"wgWANObjectCaches must have \"$wanId\" set (via wgMainWANCache)"
);
}
$cacheId = $wanParams['cacheId'];
$wanClass = $wanParams['class'];
unset( $wanParams['cacheId'] );
unset( $wanParams['class'] );
$storeParams = $mainConfig->get( MainConfigNames::ObjectCaches )[$cacheId] ?? null;
if ( !$storeParams ) {
throw new UnexpectedValueException(
"wgObjectCaches must have \"$cacheId\" set (via wgWANObjectCaches)"
);
}
$store = ObjectCache::newFromParams( $storeParams, $services );
$store = $services->get( '_LocalClusterCache' );
$logger = $store->getLogger();
$logger->debug( 'MainWANObjectCache using store {class}', [
'class' => get_class( $store )
] );
$wanParams['cache'] = $store;
$wanParams['logger'] = $logger;
$wanParams['secret'] ??= $mainConfig->get( MainConfigNames::SecretKey );
$wanParams = $mainConfig->get( MainConfigNames::WANObjectCache ) + [
'cache' => $store,
'logger' => $logger,
'secret' => $mainConfig->get( MainConfigNames::SecretKey ),
];
if ( !$GLOBALS[ 'wgCommandLineMode' ] ) {
// Send the statsd data post-send on HTTP requests; avoid in CLI mode (T181385)
$wanParams['stats'] = $services->getStatsdDataFactory();
// Let pre-emptive refreshes happen post-send on HTTP requests
$wanParams['asyncHandler'] = [ DeferredUpdates::class, 'addCallableUpdate' ];
}
$instance = new $wanClass( $wanParams );
'@phan-var WANObjectCache $instance';
return $instance;
return new WANObjectCache( $wanParams );
},
'MediaHandlerFactory' => static function ( MediaWikiServices $services ): MediaHandlerFactory {

9
includes/SetupDynamicConfig.php
View file

@ -14,15 +14,6 @@ if ( $wgLogos !== false && isset( $wgLogos['1x'] ) ) {
$wgLogo = $wgLogos['1x'];
}
if ( $wgMainWANCache === false ) {
// Create a WAN cache from $wgMainCacheType
$wgMainWANCache = 'mediawiki-main-default';
$wgWANObjectCaches[$wgMainWANCache] = [
'class' => WANObjectCache::class,
'cacheId' => $wgMainCacheType,
];
}
// Back-compat
if ( isset( $wgFileBlacklist ) ) {
$wgProhibitedFileExtensions = array_merge( $wgProhibitedFileExtensions, $wgFileBlacklist );

14
includes/config-schema.php
View file

@ -516,12 +516,7 @@ return [
'reportDupes' => false,
],
],
'MainWANCache' => false,
'WANObjectCaches' => [
0 => [
'class' => 'WANObjectCache',
'cacheId' => 0,
],
'WANObjectCache' => [
],
'MainStash' => 'db-replicated',
'ParsoidCacheConfig' => [
@ -2619,12 +2614,7 @@ return [
],
'MultiShardSiteStats' => 'boolean',
'ObjectCaches' => 'object',
'MainWANCache' => [
0 => 'integer',
1 => 'string',
2 => 'boolean',
],
'WANObjectCaches' => 'object',
'WANObjectCache' => 'object',
'ParsoidCacheConfig' => 'object',
'ChronologyProtectorStash' => [
0 => 'string',

3
tests/common/TestSetup.php
View file

@ -28,7 +28,7 @@ class TestSetup {
* of a Maintenance subclass which then gets called via MW_SETUP_CALLBACK in Setup.php.
*/
public static function applyInitialConfig() {
global $wgMainCacheType, $wgMessageCacheType, $wgParserCacheType, $wgMainWANCache, $wgSessionCacheType;
global $wgMainCacheType, $wgMessageCacheType, $wgParserCacheType, $wgSessionCacheType;
global $wgMainStash, $wgChronologyProtectorStash;
global $wgObjectCaches;
global $wgLanguageConverterCacheType, $wgUseDatabaseMessages;
@ -53,7 +53,6 @@ class TestSetup {
// See also MediaWikiIntegrationTestCase::run() which mocks CACHE_DB and APC.
// Disabled per default in MainConfigSchema, override local settings
$wgMainWANCache =
$wgMainCacheType = CACHE_NONE;
// Uses CACHE_ANYTHING per default in MainConfigSchema, use hash instead of db
$wgMessageCacheType =

7
tests/phpunit/includes/EditPageTest.php
View file

@ -12,12 +12,8 @@ use Wikimedia\TestingAccessWrapper;
/**
* @group Editing
*
* @group Database
* ^--- tell jenkins this test needs the database
*
* @group medium
* ^--- tell phpunit that these test cases may take longer than 2 seconds.
*/
class EditPageTest extends MediaWikiLangTestCase {
@ -34,6 +30,9 @@ class EditPageTest extends MediaWikiLangTestCase {
[ 'testing' => 'DummyContentHandlerForTesting' ] +
MainConfigSchema::getDefaultValue( MainConfigNames::ContentHandlers ),
] );
// Disable WAN cache to avoid edit conflicts in testUpdateNoMinor
$this->setMainCache( CACHE_NONE );
}
/**

2
tests/phpunit/includes/ResourceLoader/MessageBlobStoreTest.php
View file

@ -29,8 +29,6 @@ class MessageBlobStoreTest extends \PHPUnit\Framework\TestCase {
protected function setUp(): void {
parent::setUp();
// MediaWiki's test wrapper sets $wgMainWANCache to CACHE_NONE.
// Use HashBagOStuff here so that we can observe caching.
$this->wanCache = new WANObjectCache( [
'cache' => new HashBagOStuff()
] );

7
tests/phpunit/includes/TemplateCategoriesTest.php
View file

@ -7,6 +7,13 @@ use MediaWiki\Title\Title;
*/
class TemplateCategoriesTest extends MediaWikiIntegrationTestCase {
protected function setUp(): void {
parent::setUp();
// Don't let PageStore hit WANObjectCache process cache for revision metadata
$this->setMainCache( CACHE_NONE );
}
/**
* @covers Title::getParentCategories
*/

12
tests/phpunit/includes/cache/MessageCacheTest.php vendored
View file

@ -1,7 +1,6 @@
<?php
use MediaWiki\MainConfigNames;
use MediaWiki\MainConfigSchema;
use MediaWiki\Revision\RevisionRecord;
use MediaWiki\Revision\SlotRecord;
use MediaWiki\Title\Title;
@ -137,17 +136,8 @@ class MessageCacheTest extends MediaWikiLangTestCase {
}
public function testReplaceCache() {
// We need a WAN cache for this.
$this->overrideConfigValues( [
MainConfigNames::MainWANCache => CACHE_HASH,
MainConfigNames::WANObjectCaches =>
MainConfigSchema::getDefaultValue( MainConfigNames::WANObjectCaches ) + [
'hash' => [
'class' => WANObjectCache::class,
'cacheId' => CACHE_HASH,
'channels' => []
]
]
MainConfigNames::MainCacheType => CACHE_HASH,
] );
$messageCache = $this->getServiceContainer()->getMessageCache();

8
tests/phpunit/includes/page/ArticleViewTest.php
View file

@ -265,13 +265,7 @@ class ArticleViewTest extends MediaWikiIntegrationTestCase {
public function testViewOfOldRevisionFromCache() {
$this->overrideConfigValues( [
MainConfigNames::OldRevisionParserCacheExpireTime => 100500,
MainConfigNames::MainWANCache => 'main',
MainConfigNames::WANObjectCaches => [
'main' => [
'class' => WANObjectCache::class,
'cacheId' => 'hash',
],
],
MainConfigNames::MainCacheType => CACHE_HASH,
] );
$revisions = [];

37
tests/phpunit/unit/includes/SetupDynamicConfigTest.php
View file

@ -117,18 +117,6 @@ class SetupDynamicConfigTest extends MediaWikiUnitTestCase {
'SharedPrefix' => '',
'SharedSchema' => null,
'MetaNamespace' => 'MediaWiki',
'MainWANCache' => 'mediawiki-main-default',
'WANObjectCaches' => [
// XXX Is this duplication really intentional? Isn't the first entry unused?
0 => [
'class' => WANObjectCache::class,
'cacheId' => 0,
],
'mediawiki-main-default' => [
'class' => WANObjectCache::class,
'cacheId' => 0,
],
],
'EnableUserEmailMuteList' => false,
'EnableUserEmailBlacklist' => false,
'NamespaceProtection' => [ NS_MEDIAWIKI => 'editinterface' ],
@ -271,31 +259,6 @@ class SetupDynamicConfigTest extends MediaWikiUnitTestCase {
'MetaNamespace' => 'my_site',
],
];
yield '$wgMainCacheType set' => [
[ 'MainCacheType' => 7 ],
[
'MainCacheType' => 7,
'WANObjectCaches' => [
0 => [
'class' => WANObjectCache::class,
'cacheId' => 0,
],
'mediawiki-main-default' => [
'class' => WANObjectCache::class,
'cacheId' => 7,
],
],
],
];
yield '$wgMainWANCache set' => [
[ 'MainWANCache' => 'my-cache' ],
[
'MainWANCache' => 'my-cache',
// XXX Is this intentional? Customizing MainWANCache without adding it to
// WANObjectCaches seems like it will break everything?
'WANObjectCaches' => [ [ 'class' => WANObjectCache::class, 'cacheId' => 0 ] ],
],
];
yield '$wgProhibitedFileExtensions set' => [
[ 'ProhibitedFileExtensions' => [ 'evil' ] ],
[