Add a multi-primary mode option that supports MySQL DB setups
that use circular replication with STATEMENT formatted binlogs.
The `modtoken` column is only used when multi-primary mode is
explicitly enabled in configuration. The column is used by write
queries to determine the "winning" version of keys, with the goal
of approximating "Last-Write-Wins" eventual consistency.
Writes with different timestamps can be handled by picking the
one with the highest timestamp as the "winner". Writes with the
same timestamp, from different primary DBs, can be handled by
picking the one from the primary DB with the highest server_id.
Writes with the same token timestamp from the same primary DB can
be handled by picking the last write to appear in the binlog.
The delete() operation uses tombstones in multi-primary mode,
since there must be a key version to actually compare with the
versions from other operations.
Also:
* Remove "LOCK IN SHARE MODE" that was made obsolete by the
CONN_TRX_AUTOCOMMIT flag. For the SQLite transaction case,
it is serializable anyway.
* Simplified handleWriteError() to match handleReadError()
and merged them into handleDBError().
Changes from d56a686f83:
* Keep using replace() for set() in non-multi-primary mode.
* Fix old encoding issues when incrementing keys with postgres.
Bug: T274174
Change-Id: I25b1278586f99fdace8f2a081f6f460f734e5d83
807 lines
30 KiB
PHP
807 lines
30 KiB
PHP
<?php
|
|
/**
|
|
* Copyright © 2003-2004 Brion Vibber <brion@pobox.com>
|
|
* https://www.mediawiki.org/
|
|
*
|
|
* This program is free software; you can redistribute it and/or modify
|
|
* it under the terms of the GNU General Public License as published by
|
|
* the Free Software Foundation; either version 2 of the License, or
|
|
* (at your option) any later version.
|
|
*
|
|
* This program is distributed in the hope that it will be useful,
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
* GNU General Public License for more details.
|
|
*
|
|
* You should have received a copy of the GNU General Public License along
|
|
* with this program; if not, write to the Free Software Foundation, Inc.,
|
|
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
|
|
* http://www.gnu.org/copyleft/gpl.html
|
|
*
|
|
* @file
|
|
* @ingroup Cache
|
|
*/
|
|
|
|
/**
|
|
* @defgroup Cache Cache
|
|
*/
|
|
|
|
use Liuggio\StatsdClient\Factory\StatsdDataFactoryInterface;
|
|
use Psr\Log\LoggerAwareInterface;
|
|
use Psr\Log\LoggerInterface;
|
|
use Psr\Log\NullLogger;
|
|
use Wikimedia\LightweightObjectStore\ExpirationAwareness;
|
|
use Wikimedia\LightweightObjectStore\StorageAwareness;
|
|
use Wikimedia\ScopedCallback;
|
|
|
|
/**
|
|
* Class representing a cache/ephemeral data store
|
|
*
|
|
* This interface is intended to be more or less compatible with the PHP memcached client.
|
|
*
|
|
* Class instances should be created with an intended access scope for the dataset, such as:
|
|
* - a) A single PHP thread on a server (e.g. stored in a PHP variable)
|
|
* - b) A single application server (e.g. stored in APC or sqlite)
|
|
* - c) All application servers in datacenter (e.g. stored in memcached or mysql)
|
|
* - d) All application servers in all datacenters (e.g. stored via mcrouter or dynomite)
|
|
*
|
|
* Callers should use the proper factory methods that yield BagOStuff instances. Site admins
|
|
* should make sure that the configuration for those factory methods match their access scope.
|
|
* BagOStuff subclasses have widely varying levels of support replication features within and
|
|
* among datacenters.
|
|
*
|
|
* Subclasses should override the default "segmentationSize" field with an appropriate value.
|
|
* The value should not be larger than what the backing store (by default) supports. It also
|
|
* should be roughly informed by common performance bottlenecks (e.g. values over a certain size
|
|
* having poor scalability). The same goes for the "segmentedValueMaxSize" member, which limits
|
|
* the maximum size and chunk count (indirectly) of values.
|
|
*
|
|
* A few notes about data consistency for BagOStuff instances:
|
|
* - Read operation methods, e.g. get(), should be synchronous in the local datacenter.
|
|
* When used with READ_LATEST, such operations should reflect any prior writes originating
|
|
* from the local datacenter (e.g. by avoiding replica DBs or invoking quorom reads).
|
|
* - Write operation methods, e.g. set(), should be synchronous in the local datacenter, with
|
|
* asynchronous cross-datacenter replication. This replication can be either "best effort"
|
|
* or eventually consistent. When used with WRITE_SYNC, such operations will wait until all
|
|
* datacenters are updated or a timeout occurs. If the write succeeded, then any subsequent
|
|
* get() operations with READ_LATEST, regardless of datacenter, should reflect the changes.
|
|
* - Locking operation methods, e.g. lock(), unlock(), and getScopedLock(), should only apply
|
|
* to the local datacenter.
|
|
* - Any set of single-key write operation method calls originating from a single datacenter
|
|
* should observe "best effort" linearizability. Any set of single-key write operations using
|
|
* WRITE_SYNC, regardless of the datacenter, should observe "best effort" linearizability.
|
|
* In this context, "best effort" means that consistency holds as long as connectivity is
|
|
* strong, network latency is low, and there are no relevant storage server failures.
|
|
* Per https://en.wikipedia.org/wiki/PACELC_theorem, the store should act as a PA/EL
|
|
* distributed system for these operations.
|
|
*
|
|
* @stable to extend
|
|
* @ingroup Cache
|
|
*/
|
|
abstract class BagOStuff implements
|
|
ExpirationAwareness,
|
|
StorageAwareness,
|
|
IStoreKeyEncoder,
|
|
LoggerAwareInterface
|
|
{
|
|
/** @var StatsdDataFactoryInterface */
|
|
protected $stats;
|
|
/** @var LoggerInterface */
|
|
protected $logger;
|
|
/** @var callable|null */
|
|
protected $asyncHandler;
|
|
/**
|
|
* @var array<string,array> Cache key processing callbacks and info for metrics
|
|
* @phan-var array<string,array{0:string,1:callable}>
|
|
*/
|
|
protected $wrapperInfoByPrefix = [];
|
|
|
|
/** @var int[] Map of (ATTR_* class constant => QOS_* class constant) */
|
|
protected $attrMap = [];
|
|
|
|
/** @var string Default keyspace; used by makeKey() */
|
|
protected $keyspace;
|
|
|
|
/**
|
|
* @var bool Whether to send debug log entries to the SPI logger instance
|
|
* @deprecated since 1.36 -- unused
|
|
*/
|
|
protected $debugMode = true;
|
|
|
|
/** @var float|null */
|
|
private $wallClockOverride;
|
|
|
|
/** Bitfield constants for get()/getMulti(); these are only advisory */
|
|
public const READ_LATEST = 1; // if supported, avoid reading stale data due to replication
|
|
public const READ_VERIFIED = 2; // promise that the caller handles detection of staleness
|
|
/** Bitfield constants for set()/merge(); these are only advisory */
|
|
public const WRITE_SYNC = 4; // if supported, block until the write is fully replicated
|
|
public const WRITE_CACHE_ONLY = 8; // only change state of the in-memory cache
|
|
public const WRITE_ALLOW_SEGMENTS = 16; // allow partitioning of the value if it is large
|
|
public const WRITE_PRUNE_SEGMENTS = 32; // delete all the segments if the value is partitioned
|
|
public const WRITE_BACKGROUND = 64; // if supported, do not block on completion until the next read
|
|
|
|
/** @var string Global keyspace; used by makeGlobalKey() */
|
|
protected const GLOBAL_KEYSPACE = 'global';
|
|
/** @var string Precomputed global cache key prefix (needs no encoding) */
|
|
protected const GLOBAL_PREFIX = 'global:';
|
|
/** @var string Precomputed global cache key prefix length */
|
|
protected const GLOBAL_PREFIX_LEN = 7;
|
|
|
|
/** @var int Item is a single cache key */
|
|
protected const ARG0_KEY = 0;
|
|
/** @var int Item is an array of cache keys */
|
|
protected const ARG0_KEYARR = 1;
|
|
/** @var int Item is an array indexed by cache keys */
|
|
protected const ARG0_KEYMAP = 2;
|
|
/** @var int Item does not involve any keys */
|
|
protected const ARG0_NONKEY = 3;
|
|
|
|
/** @var int Item is an array indexed by cache keys */
|
|
protected const RES_KEYMAP = 0;
|
|
/** @var int Item does not involve any keys */
|
|
protected const RES_NONKEY = 1;
|
|
|
|
/** Key to the metric group to use for the relevant cache wrapper */
|
|
private const WRAPPER_STATS_GROUP = 0;
|
|
/** Key to the callback that extracts collection names from cache wrapper keys */
|
|
private const WRAPPER_COLLECTION_CALLBACK = 1;
|
|
|
|
/**
|
|
* Parameters include:
|
|
* - keyspace: Keyspace to use for keys in makeKey(). [Default: "local"]
|
|
* - asyncHandler: Callable to use for scheduling tasks after the web request ends.
|
|
* In CLI mode, it should run the task immediately. [Default: null]
|
|
* - stats: IStatsdDataFactory instance. [optional]
|
|
* - logger: Psr\Log\LoggerInterface instance. [optional]
|
|
* @param array $params
|
|
* @phan-param array{keyspace?:string,logger?:Psr\Log\LoggerInterface,asyncHandler?:callable} $params
|
|
*/
|
|
public function __construct( array $params = [] ) {
|
|
$this->keyspace = $params['keyspace'] ?? 'local';
|
|
$this->asyncHandler = $params['asyncHandler'] ?? null;
|
|
$this->stats = $params['stats'] ?? new NullStatsdDataFactory();
|
|
$this->setLogger( $params['logger'] ?? new NullLogger() );
|
|
}
|
|
|
|
/**
|
|
* @param LoggerInterface $logger
|
|
* @return void
|
|
*/
|
|
public function setLogger( LoggerInterface $logger ) {
|
|
$this->logger = $logger;
|
|
}
|
|
|
|
/**
|
|
* @since 1.35
|
|
* @return LoggerInterface
|
|
*/
|
|
public function getLogger(): LoggerInterface {
|
|
return $this->logger;
|
|
}
|
|
|
|
/**
|
|
* @param bool $enabled
|
|
* @deprecated since 1.36, always true
|
|
*/
|
|
public function setDebug( $enabled ) {
|
|
wfDeprecated( __METHOD__, '1.36' );
|
|
}
|
|
|
|
/**
|
|
* Get an item with the given key, regenerating and setting it if not found
|
|
*
|
|
* The callback can take $exptime as argument by reference and modify it.
|
|
* Nothing is stored nor deleted if the callback returns false.
|
|
*
|
|
* @param string $key
|
|
* @param int $exptime Time-to-live (seconds)
|
|
* @param callable $callback Callback that derives the new value
|
|
* @param int $flags Bitfield of BagOStuff::READ_* or BagOStuff::WRITE_* constants [optional]
|
|
* @return mixed The cached value if found or the result of $callback otherwise
|
|
* @since 1.27
|
|
*/
|
|
final public function getWithSetCallback( $key, $exptime, $callback, $flags = 0 ) {
|
|
$value = $this->get( $key, $flags );
|
|
|
|
if ( $value === false ) {
|
|
$value = $callback( $exptime );
|
|
if ( $value !== false && $exptime >= 0 ) {
|
|
$this->set( $key, $value, $exptime, $flags );
|
|
}
|
|
}
|
|
|
|
return $value;
|
|
}
|
|
|
|
/**
|
|
* Get an item with the given key
|
|
*
|
|
* If the key includes a deterministic input hash (e.g. the key can only have
|
|
* the correct value) or complete staleness checks are handled by the caller
|
|
* (e.g. nothing relies on the TTL), then the READ_VERIFIED flag should be set.
|
|
* This lets tiered backends know they can safely upgrade a cached value to
|
|
* higher tiers using standard TTLs.
|
|
*
|
|
* @param string $key
|
|
* @param int $flags Bitfield of BagOStuff::READ_* constants [optional]
|
|
* @return mixed Returns false on failure or if the item does not exist
|
|
*/
|
|
abstract public function get( $key, $flags = 0 );
|
|
|
|
/**
|
|
* Set an item
|
|
*
|
|
* @param string $key
|
|
* @param mixed $value
|
|
* @param int $exptime Either an interval in seconds or a unix timestamp for expiry
|
|
* @param int $flags Bitfield of BagOStuff::WRITE_* constants
|
|
* @return bool Success
|
|
*/
|
|
abstract public function set( $key, $value, $exptime = 0, $flags = 0 );
|
|
|
|
/**
|
|
* Delete an item
|
|
*
|
|
* For large values written using WRITE_ALLOW_SEGMENTS, this only deletes the main
|
|
* segment list key unless WRITE_PRUNE_SEGMENTS is in the flags. While deleting the segment
|
|
* list key has the effect of functionally deleting the key, it leaves unused blobs in cache.
|
|
*
|
|
* @param string $key
|
|
* @return bool True if the item was deleted or not found, false on failure
|
|
* @param int $flags Bitfield of BagOStuff::WRITE_* constants
|
|
*/
|
|
abstract public function delete( $key, $flags = 0 );
|
|
|
|
/**
|
|
* Insert an item if it does not already exist
|
|
*
|
|
* @param string $key
|
|
* @param mixed $value
|
|
* @param int $exptime
|
|
* @param int $flags Bitfield of BagOStuff::WRITE_* constants (since 1.33)
|
|
* @return bool Success
|
|
*/
|
|
abstract public function add( $key, $value, $exptime = 0, $flags = 0 );
|
|
|
|
/**
|
|
* Merge changes into the existing cache value (possibly creating a new one)
|
|
*
|
|
* The callback function returns the new value given the current value
|
|
* (which will be false if not present), and takes the arguments:
|
|
* (this BagOStuff, cache key, current value, TTL).
|
|
* The TTL parameter is reference set to $exptime. It can be overriden in the callback.
|
|
* Nothing is stored nor deleted if the callback returns false.
|
|
*
|
|
* @param string $key
|
|
* @param callable $callback Callback method to be executed
|
|
* @param int $exptime Either an interval in seconds or a unix timestamp for expiry
|
|
* @param int $attempts The amount of times to attempt a merge in case of failure
|
|
* @param int $flags Bitfield of BagOStuff::WRITE_* constants
|
|
* @return bool Success
|
|
* @throws InvalidArgumentException
|
|
*/
|
|
abstract public function merge(
|
|
$key,
|
|
callable $callback,
|
|
$exptime = 0,
|
|
$attempts = 10,
|
|
$flags = 0
|
|
);
|
|
|
|
/**
|
|
* Change the expiration on a key if it exists
|
|
*
|
|
* If an expiry in the past is given then the key will immediately be expired
|
|
*
|
|
* For large values written using WRITE_ALLOW_SEGMENTS, this only changes the TTL of the
|
|
* main segment list key. While lowering the TTL of the segment list key has the effect of
|
|
* functionally lowering the TTL of the key, it might leave unused blobs in cache for longer.
|
|
* Raising the TTL of such keys is not effective, since the expiration of a single segment
|
|
* key effectively expires the entire value.
|
|
*
|
|
* @param string $key
|
|
* @param int $exptime TTL or UNIX timestamp
|
|
* @param int $flags Bitfield of BagOStuff::WRITE_* constants (since 1.33)
|
|
* @return bool Success Returns false on failure or if the item does not exist
|
|
* @since 1.28
|
|
*/
|
|
abstract public function changeTTL( $key, $exptime = 0, $flags = 0 );
|
|
|
|
/**
|
|
* Acquire an advisory lock on a key string, exclusive to the caller
|
|
*
|
|
* @param string $key
|
|
* @param int $timeout Lock wait timeout; 0 for non-blocking [optional]
|
|
* @param int $exptime Lock time-to-live in seconds; 1 day maximum [optional]
|
|
* @param string $rclass If this thread already holds the lock, and the lock was acquired
|
|
* using the same value for this parameter, then return true and use reference counting so
|
|
* that only the unlock() call from the outermost lock() caller actually releases the lock
|
|
* (note that only the outermost time-to-live is used) [optional]
|
|
* @return bool Success
|
|
*/
|
|
abstract public function lock( $key, $timeout = 6, $exptime = 6, $rclass = '' );
|
|
|
|
/**
|
|
* Release an advisory lock on a key string
|
|
*
|
|
* @param string $key
|
|
* @return bool Success
|
|
*/
|
|
abstract public function unlock( $key );
|
|
|
|
/**
|
|
* Get a lightweight exclusive self-unlocking lock
|
|
*
|
|
* Note that the same lock cannot be acquired twice.
|
|
*
|
|
* This is useful for task de-duplication or to avoid obtrusive
|
|
* (though non-corrupting) DB errors like INSERT key conflicts
|
|
* or deadlocks when using LOCK IN SHARE MODE.
|
|
*
|
|
* @param string $key
|
|
* @param int $timeout Lock wait timeout; 0 for non-blocking [optional]
|
|
* @param int $expiry Lock expiry [optional]; 1 day maximum
|
|
* @param string $rclass Allow reentry if set and the current lock used this value
|
|
* @return ScopedCallback|null Returns null on failure
|
|
* @since 1.26
|
|
*/
|
|
final public function getScopedLock( $key, $timeout = 6, $expiry = 30, $rclass = '' ) {
|
|
$expiry = min( $expiry ?: INF, self::TTL_DAY );
|
|
|
|
if ( !$this->lock( $key, $timeout, $expiry, $rclass ) ) {
|
|
return null;
|
|
}
|
|
|
|
return new ScopedCallback( function () use ( $key, $expiry ) {
|
|
$this->unlock( $key );
|
|
} );
|
|
}
|
|
|
|
/**
|
|
* Delete all objects expiring before a certain date
|
|
*
|
|
* @param string|int $timestamp The reference date in MW or TS_UNIX format
|
|
* @param callable|null $progress Optional, a function which will be called
|
|
* regularly during long-running operations with the percentage progress
|
|
* as the first parameter. [optional]
|
|
* @param int $limit Maximum number of keys to delete [default: INF]
|
|
* @param string|null $tag Tag to purge a single shard only.
|
|
* This is only supported when server tags are used in configuration.
|
|
* @return bool Success; false if unimplemented
|
|
*/
|
|
abstract public function deleteObjectsExpiringBefore(
|
|
$timestamp,
|
|
callable $progress = null,
|
|
$limit = INF,
|
|
string $tag = null
|
|
);
|
|
|
|
/**
|
|
* Get an associative array containing the item for each of the keys that have items
|
|
*
|
|
* @param string[] $keys List of keys
|
|
* @param int $flags Bitfield; supports READ_LATEST [optional]
|
|
* @return mixed[] Map of (key => value) for existing keys
|
|
*/
|
|
abstract public function getMulti( array $keys, $flags = 0 );
|
|
|
|
/**
|
|
* Batch insertion/replace
|
|
*
|
|
* This does not support WRITE_ALLOW_SEGMENTS to avoid excessive read I/O
|
|
*
|
|
* WRITE_BACKGROUND can be used for bulk insertion where the response is not vital
|
|
*
|
|
* @param mixed[] $valueByKey Map of (key => value)
|
|
* @param int $exptime Either an interval in seconds or a unix timestamp for expiry
|
|
* @param int $flags Bitfield of BagOStuff::WRITE_* constants (since 1.33)
|
|
* @return bool Success
|
|
* @since 1.24
|
|
*/
|
|
abstract public function setMulti( array $valueByKey, $exptime = 0, $flags = 0 );
|
|
|
|
/**
|
|
* Batch deletion
|
|
*
|
|
* This does not support WRITE_ALLOW_SEGMENTS to avoid excessive read I/O
|
|
*
|
|
* WRITE_BACKGROUND can be used for bulk deletion where the response is not vital
|
|
*
|
|
* @param string[] $keys List of keys
|
|
* @param int $flags Bitfield of BagOStuff::WRITE_* constants
|
|
* @return bool Success
|
|
* @since 1.33
|
|
*/
|
|
abstract public function deleteMulti( array $keys, $flags = 0 );
|
|
|
|
/**
|
|
* Change the expiration of multiple keys that exist
|
|
*
|
|
* @see BagOStuff::changeTTL()
|
|
*
|
|
* @param string[] $keys List of keys
|
|
* @param int $exptime TTL or UNIX timestamp
|
|
* @param int $flags Bitfield of BagOStuff::WRITE_* constants (since 1.33)
|
|
* @return bool Success
|
|
* @since 1.34
|
|
*/
|
|
abstract public function changeTTLMulti( array $keys, $exptime, $flags = 0 );
|
|
|
|
/**
|
|
* Increase stored value of $key by $value while preserving its TTL
|
|
*
|
|
* @param string $key Key to increase
|
|
* @param int $value Value to add to $key (default: 1) [optional]
|
|
* @param int $flags Bit field of class WRITE_* constants [optional]
|
|
* @return int|bool New value or false on failure
|
|
*/
|
|
abstract public function incr( $key, $value = 1, $flags = 0 );
|
|
|
|
/**
|
|
* Decrease stored value of $key by $value while preserving its TTL
|
|
*
|
|
* @param string $key
|
|
* @param int $value Value to subtract from $key (default: 1) [optional]
|
|
* @param int $flags Bit field of class WRITE_* constants [optional]
|
|
* @return int|bool New value or false on failure
|
|
*/
|
|
abstract public function decr( $key, $value = 1, $flags = 0 );
|
|
|
|
/**
|
|
* Increase the value of the given key (no TTL change) if it exists or create it otherwise
|
|
*
|
|
* This will create the key with the value $init and TTL $exptime instead if not present.
|
|
* Callers should make sure that both ($init - $value) and $exptime are invariants for all
|
|
* operations to any given key. The value of $init should be at least that of $value.
|
|
*
|
|
* @param string $key Key built via makeKey() or makeGlobalKey()
|
|
* @param int $exptime Time-to-live (in seconds) or a UNIX timestamp expiration
|
|
* @param int $value Amount to increase the key value by [default: 1]
|
|
* @param int|null $init Value to initialize the key to if it does not exist [default: $value]
|
|
* @param int $flags Bit field of class WRITE_* constants [optional]
|
|
* @return int|bool New value or false on failure
|
|
* @since 1.24
|
|
*/
|
|
abstract public function incrWithInit( $key, $exptime, $value = 1, $init = null, $flags = 0 );
|
|
|
|
/**
|
|
* Get the "last error" registered; clearLastError() should be called manually
|
|
* @return int ERR_* constant for the "last error" registry
|
|
* @since 1.23
|
|
*/
|
|
abstract public function getLastError();
|
|
|
|
/**
|
|
* Clear the "last error" registry
|
|
* @since 1.23
|
|
*/
|
|
abstract public function clearLastError();
|
|
|
|
/**
|
|
* Let a callback be run to avoid wasting time on special blocking calls
|
|
*
|
|
* The callbacks may or may not be called ever, in any particular order.
|
|
* They are likely to be invoked when something WRITE_SYNC is used used.
|
|
* They should follow a caching pattern as shown below, so that any code
|
|
* using the work will get it's result no matter what happens.
|
|
* @code
|
|
* $result = null;
|
|
* $workCallback = function () use ( &$result ) {
|
|
* if ( !$result ) {
|
|
* $result = ....
|
|
* }
|
|
* return $result;
|
|
* }
|
|
* @endcode
|
|
*
|
|
* @param callable $workCallback
|
|
* @since 1.28
|
|
*/
|
|
abstract public function addBusyCallback( callable $workCallback );
|
|
|
|
/**
|
|
* Make a cache key for the given keyspace and components
|
|
*
|
|
* Long components might be converted to respective hashes due to size constraints.
|
|
* In extreme cases, all of them might be combined into a single hash component.
|
|
*
|
|
* @internal This method should not be used outside of BagOStuff (since 1.36)
|
|
*
|
|
* @param string $keyspace Keyspace component
|
|
* @param string[]|int[] $components Key components (key collection name first)
|
|
* @return string Keyspace-prepended list of encoded components as a colon-separated value
|
|
* @since 1.27
|
|
*/
|
|
abstract public function makeKeyInternal( $keyspace, $components );
|
|
|
|
/**
|
|
* Make a cache key for the default keyspace and given components
|
|
*
|
|
* @see IStoreKeyEncoder::makeGlobalKey()
|
|
*
|
|
* @param string $collection Key collection name component
|
|
* @param string|int ...$components Additional, ordered, key components for entity IDs
|
|
* @return string Colon-separated, keyspace-prepended, ordered list of encoded components
|
|
* @since 1.27
|
|
*/
|
|
abstract public function makeGlobalKey( $collection, ...$components );
|
|
|
|
/**
|
|
* Make a cache key for the global keyspace and given components
|
|
*
|
|
* @see IStoreKeyEncoder::makeKey()
|
|
*
|
|
* @param string $collection Key collection name component
|
|
* @param string|int ...$components Additional, ordered, key components for entity IDs
|
|
* @return string Colon-separated, keyspace-prepended, ordered list of encoded components
|
|
* @since 1.27
|
|
*/
|
|
abstract public function makeKey( $collection, ...$components );
|
|
|
|
/**
|
|
* Check whether a cache key is in the global keyspace
|
|
*
|
|
* @param string $key
|
|
* @return bool
|
|
* @since 1.35
|
|
*/
|
|
public function isKeyGlobal( $key ) {
|
|
return ( strncmp( $key, self::GLOBAL_PREFIX, self::GLOBAL_PREFIX_LEN ) === 0 );
|
|
}
|
|
|
|
/**
|
|
* @param int $flag ATTR_* class constant
|
|
* @return int QOS_* class constant
|
|
* @since 1.28
|
|
*/
|
|
public function getQoS( $flag ) {
|
|
return $this->attrMap[$flag] ?? self::QOS_UNKNOWN;
|
|
}
|
|
|
|
/**
|
|
* @return int|float The chunk size, in bytes, of segmented objects (INF for no limit)
|
|
* @since 1.34
|
|
*/
|
|
public function getSegmentationSize() {
|
|
return INF;
|
|
}
|
|
|
|
/**
|
|
* @return int|float Maximum total segmented object size in bytes (INF for no limit)
|
|
* @since 1.34
|
|
*/
|
|
public function getSegmentedValueMaxSize() {
|
|
return INF;
|
|
}
|
|
|
|
/**
|
|
* @param int $field
|
|
* @param int $flags
|
|
* @return bool
|
|
* @since 1.34
|
|
*/
|
|
final protected function fieldHasFlags( $field, $flags ) {
|
|
return ( ( $field & $flags ) === $flags );
|
|
}
|
|
|
|
/**
|
|
* Merge the flag maps of one or more BagOStuff objects into a "lowest common denominator" map
|
|
*
|
|
* @param BagOStuff[] $bags
|
|
* @return int[] Resulting flag map (class ATTR_* constant => class QOS_* constant)
|
|
*/
|
|
final protected function mergeFlagMaps( array $bags ) {
|
|
$map = [];
|
|
foreach ( $bags as $bag ) {
|
|
foreach ( $bag->attrMap as $attr => $rank ) {
|
|
if ( isset( $map[$attr] ) ) {
|
|
$map[$attr] = min( $map[$attr], $rank );
|
|
} else {
|
|
$map[$attr] = $rank;
|
|
}
|
|
}
|
|
}
|
|
|
|
return $map;
|
|
}
|
|
|
|
/**
|
|
* Make a "generic" reversible cache key from the given components
|
|
*
|
|
* All previously prepared values will be cleared. Each of the new prepared values will be
|
|
* individually cleared as they get used by write operations for that key. This is done to
|
|
* avoid unchecked growth in PHP memory usage.
|
|
*
|
|
* Example usage:
|
|
* @code
|
|
* $valueByKey = [ $key1 => $value1, $key2 => $value2, $key3 => $value3 ];
|
|
* $cache->setNewPreparedValues( $valueByKey );
|
|
* $cache->set( $key1, $value1, $cache::TTL_HOUR );
|
|
* $cache->setMulti( [ $key2 => $value2, $key3 => $value3 ], $cache::TTL_HOUR );
|
|
* @endcode
|
|
*
|
|
* This is only useful if the caller needs an estimate of the serialized object sizes.
|
|
* The caller cannot know the serialization format and even if it did, it could be expensive
|
|
* to serialize complex values twice just to get the size information before writing them to
|
|
* cache. This method solves both problems by making the cache instance do the serialization
|
|
* and having it reuse the result when the cache write happens.
|
|
*
|
|
* @param array $valueByKey Map of (cache key => PHP variable value to serialize)
|
|
* @return int[]|null[] Corresponding list of size estimates (null for invalid values)
|
|
* @since 1.35
|
|
*/
|
|
abstract public function setNewPreparedValues( array $valueByKey );
|
|
|
|
/**
|
|
* Register info about a caching layer class that uses BagOStuff as a backing store
|
|
*
|
|
* Object cache wrappers are classes that implement generic caching/storage functionality,
|
|
* use a BagOStuff instance as the backing store, and implement IStoreKeyEncoder with the
|
|
* same "generic" style key encoding as BagOStuff. Such wrappers transform keys before
|
|
* passing them to BagOStuff methods; a wrapper-specific prefix component will be prepended
|
|
* along with other possible additions. Transformed keys still use the "generic" BagOStuff
|
|
* encoding.
|
|
*
|
|
* The provided callback takes a transformed key, having the specified prefix component,
|
|
* and extracts the key collection name. For sanity, the callback must be able to handle
|
|
* keys that bear the prefix (by coincidence) but do not originate from the wrapper class.
|
|
*
|
|
* Calls to this method should be idempotent.
|
|
*
|
|
* @param string $prefixComponent Key prefix component used by the wrapper
|
|
* @param string $statsGroup Stats group to use for metrics from this wrapper
|
|
* @param callable $collectionCallback Static callback that gets the key collection name
|
|
* @internal For use with BagOStuff and WANObjectCache only
|
|
* @since 1.36
|
|
*/
|
|
public function registerWrapperInfoForStats(
|
|
string $prefixComponent,
|
|
string $statsGroup,
|
|
callable $collectionCallback
|
|
) {
|
|
$this->wrapperInfoByPrefix[$prefixComponent] = [
|
|
self::WRAPPER_STATS_GROUP => $statsGroup,
|
|
self::WRAPPER_COLLECTION_CALLBACK => $collectionCallback
|
|
];
|
|
}
|
|
|
|
/**
|
|
* At a minimum, there must be a keyspace and collection name component
|
|
*
|
|
* @param string|int ...$components Key components for keyspace, collection name, and IDs
|
|
* @return string Keyspace-prepended list of encoded components as a colon-separated value
|
|
* @since 1.35
|
|
*/
|
|
final protected function genericKeyFromComponents( ...$components ) {
|
|
if ( count( $components ) < 2 ) {
|
|
throw new InvalidArgumentException( "Missing keyspace or collection name" );
|
|
}
|
|
|
|
$key = '';
|
|
foreach ( $components as $i => $component ) {
|
|
if ( $i > 0 ) {
|
|
$key .= ':';
|
|
}
|
|
// Escape delimiter (":") and escape ("%") characters
|
|
$key .= strtr( $component, [ '%' => '%25', ':' => '%3A' ] );
|
|
}
|
|
|
|
return $key;
|
|
}
|
|
|
|
/**
|
|
* Extract the components from a "generic" reversible cache key
|
|
*
|
|
* @see BagOStuff::genericKeyFromComponents()
|
|
*
|
|
* @param string $key Keyspace-prepended list of encoded components as a colon-separated value
|
|
* @return string[] Key components for keyspace, collection name, and IDs
|
|
* @since 1.35
|
|
*/
|
|
final protected function componentsFromGenericKey( $key ) {
|
|
// Note that the order of each corresponding search/replace pair matters
|
|
return str_replace( [ '%3A', '%25' ], [ ':', '%' ], explode( ':', $key ) );
|
|
}
|
|
|
|
/**
|
|
* Convert a "generic" reversible cache key into one for this cache
|
|
*
|
|
* @see BagOStuff::genericKeyFromComponents()
|
|
*
|
|
* @param string $key Keyspace-prepended list of encoded components as a colon-separated value
|
|
* @return string Keyspace-prepended list of encoded components as a colon-separated value
|
|
*/
|
|
abstract protected function convertGenericKey( $key );
|
|
|
|
/**
|
|
* Call a method on behalf of wrapper BagOStuff instance that uses "generic" keys
|
|
*
|
|
* @param string $method Name of a non-final public method that reads/changes keys
|
|
* @param int $arg0Sig BagOStuff::ARG0_* constant describing argument 0
|
|
* @param int $resSig BagOStuff::RES_* constant describing the return value
|
|
* @param array $genericArgs Method arguments passed to the wrapper instance
|
|
* @return mixed Method result with any resulting cache keys remapped to "generic" keys
|
|
*/
|
|
protected function proxyCall( $method, $arg0Sig, $resSig, array $genericArgs ) {
|
|
// Get the corresponding store-specific cache keys...
|
|
$storeArgs = $genericArgs;
|
|
switch ( $arg0Sig ) {
|
|
case self::ARG0_KEY:
|
|
$storeArgs[0] = $this->convertGenericKey( $genericArgs[0] );
|
|
break;
|
|
case self::ARG0_KEYARR:
|
|
foreach ( $genericArgs[0] as $i => $genericKey ) {
|
|
$storeArgs[0][$i] = $this->convertGenericKey( $genericKey );
|
|
}
|
|
break;
|
|
case self::ARG0_KEYMAP:
|
|
$storeArgs[0] = [];
|
|
foreach ( $genericArgs[0] as $genericKey => $v ) {
|
|
$storeArgs[0][$this->convertGenericKey( $genericKey )] = $v;
|
|
}
|
|
break;
|
|
}
|
|
|
|
// Result of invoking the method with the corresponding store-specific cache keys
|
|
$storeRes = $this->$method( ...$storeArgs );
|
|
|
|
// Convert any store-specific cache keys in the result back to generic cache keys
|
|
if ( $resSig === self::RES_KEYMAP ) {
|
|
// Map of (store-specific cache key => generic cache key)
|
|
$genericKeyByStoreKey = array_combine( $storeArgs[0], $genericArgs[0] );
|
|
|
|
$genericRes = [];
|
|
foreach ( $storeRes as $storeKey => $value ) {
|
|
$genericRes[$genericKeyByStoreKey[$storeKey]] = $value;
|
|
}
|
|
} else {
|
|
$genericRes = $storeRes;
|
|
}
|
|
|
|
return $genericRes;
|
|
}
|
|
|
|
/**
|
|
* @param string $key Key generated by an IStoreKeyEncoder instance
|
|
* @return string A stats prefix to describe this class of key (e.g. "objectcache.file")
|
|
*/
|
|
protected function determineKeyPrefixForStats( $key ) {
|
|
$firstComponent = substr( $key, 0, strcspn( $key, ':' ) );
|
|
|
|
$wrapperInfo = $this->wrapperInfoByPrefix[$firstComponent] ?? null;
|
|
if ( $wrapperInfo ) {
|
|
// Key has the prefix of a cache wrapper class that wraps BagOStuff
|
|
$collection = $wrapperInfo[self::WRAPPER_COLLECTION_CALLBACK]( $key );
|
|
$statsGroup = $wrapperInfo[self::WRAPPER_STATS_GROUP];
|
|
} else {
|
|
// Key came directly from BagOStuff::makeKey() or BagOStuff::makeGlobalKey()
|
|
// and thus has the format of "<scope>:<collection>[:<constant or variable>]..."
|
|
$components = explode( ':', $key, 3 );
|
|
// Handle legacy callers that fail to use the key building methods
|
|
$collection = $components[1] ?? $components[0];
|
|
$statsGroup = 'objectcache';
|
|
}
|
|
|
|
// Replace dots because they are special in StatsD (T232907)
|
|
return $statsGroup . '.' . strtr( $collection, '.', '_' );
|
|
}
|
|
|
|
/**
|
|
* @internal For testing only
|
|
* @return float UNIX timestamp
|
|
* @codeCoverageIgnore
|
|
*/
|
|
public function getCurrentTime() {
|
|
return $this->wallClockOverride ?: microtime( true );
|
|
}
|
|
|
|
/**
|
|
* @internal For testing only
|
|
* @param float|null &$time Mock UNIX timestamp
|
|
* @codeCoverageIgnore
|
|
*/
|
|
public function setMockTime( &$time ) {
|
|
$this->wallClockOverride =& $time;
|
|
}
|
|
}
|