Commit graph

360 commits

Author SHA1 Message Date
Tim Starling
fb91d409d6 Fix SqlBagOStuff exptime uniqueness assumption
The WMF parser cache tables have some 30000 rows with an identical, old
expiry time. So each time deleteObjectsExpiringBefore() is run, the
first batch on each table is correct, but the keys for the second batch
are selected with exptime > X, where X is shared exptime of the many old
rows.

So, use exptime >= X instead.

Change-Id: I6853f64f88f65d4529be2a42c9ed70cfa62cf653
2017-06-10 20:54:23 +10:00
Aaron Schulz
dd359741cc Move DB errors to Rdbms namespace
Change-Id: I463bd86123501abc68fdb78b4cda6110f7af2549
2017-04-15 10:47:41 -07:00
Aaron Schulz
d735dc562d Move Database and subclasses to Rdbms namespace
Change-Id: I52bef87512f9ddd155d1f4cc0052f6b7a0db5b42
2017-04-12 10:43:57 -07:00
Brian Wolff
c5a0fa5bed objectcache: Never use CACHE_NONE for CACHE_ANYTHING
If $wgMainCacheType = CACHE_ACCEL, but there is no APC, then its
possible that CACHE_ANYTHING will default to CACHE_NONE because
that's what CACHE_ACCEL would do.

Possibly also T147161

Bug: T160519
Change-Id: I9ac2d071437b35a0f9cd3678e2279628f7b1931e
2017-03-30 01:38:15 +00:00
Aaron Schulz
488a647831 Move IDatabase/IMaintainableDatabase to Rdbms namespace
Change-Id: If7e8a8ff574661fd827de8bcec11d2c39a687300
2017-03-28 15:32:38 -07:00
Amir Sarabadani
bcf61a7080 Fix mistake in ObjectCache doc
Change-Id: Iafe3c460ca462aa160cfea6769fd78f8b023548c
2017-03-10 13:12:43 +03:30
jenkins-bot
17eda64357 Merge "includes: Replace implicit Bugzilla bug numbers with Phab ones" 2017-02-28 00:51:57 +00:00
Aaron Schulz
64df456b39 Move remaining LoadBalancer classes to Rdbms
The old names are left as aliases.

Change-Id: I52a327f2463a2ba7437324047b5b00d28cd1d758
2017-02-23 20:38:31 -08:00
James D. Forrester
9635dda73a includes: Replace implicit Bugzilla bug numbers with Phab ones
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.

Change-Id: I6f59febaf8fc96e80f8cfc11f4356283f461142a
2017-02-21 18:13:24 +00:00
Aaron Schulz
59053a0716 Move ChronologyProtector/TransactionProfiler to Rdbms namespace
Change-Id: I37a655bd8bd267c9bc32028b55925b2dce527d33
2017-01-26 10:30:57 -08:00
Erik Bernhardson
9053890020 Cleanup static analysis errors
While prepping the code base for using etsy/phan i found a few problems.
A couple are real problems, while others are just places etsy couldn't
figure out the types.

* AuthPlugin referenced UserLoginTemplate which doesn't exist. Tracing
  back usages this accepts classes that extend from BaseTemplate.
* DatabaseSqlite had an invalid @var annotation.
* LoadBalancer had some missing or under-defined annotations. Make them
  more specific where possible.
* SqlBagOStuff didn't have the appropriate use statement for
  WaitConditionLoop and probably doesn't work. Also updated annotations
  that point to non-existent DatabaseBase to use IDatabase.

Change-Id: Iff2270b418ad2f8f97cfdb6df646c435d3283536
2016-12-06 16:10:38 -08:00
Aaron Schulz
f19a4a1566 Clean up newWANCacheFromParams() getInstance() calls
Change-Id: I0daa2531708d0a110e8028c4a5641153a23c0a97
2016-10-23 21:01:03 -07:00
Aaron Schulz
5c59e46a72 Remove deprecated ObjectCache::newAccelerator method
Change-Id: I2be2afe3f2bba830aea4e985005adb72f0e45a82
2016-10-12 15:15:15 -07:00
jenkins-bot
1d95c68977 Merge "ObjectCache::newFromParams fix SqlBagOStuff backcompat" 2016-10-05 21:55:45 +00:00
addshore
b63b3ce036 ObjectCache::newFromParams fix SqlBagOStuff backcompat
Bug: T147407
Change-Id: I718d01b309d96deeb9f7aa1fdcdee9f824657f9d
2016-10-05 22:11:48 +01:00
Aaron Schulz
514d759b58 Add the main stash, WAN, and server caches to MediaWikiServices
Leave getLocalServerInstance() array fallback logic.

Restores 6d99fa5824
Which got reverted by 7ab9283292.

Change-Id: Id1f48e10753d6aaf64eecb4d3c2228740abc1340
2016-10-03 18:36:06 +00:00
Reedy
c214c9a952 Detect/use APCu properly
In PHP 5.5 and above, userland APC caching moved to an extension

Bug: T140587
Change-Id: Ie0871776cd7e67838471a4fe95451cf4164079b7
2016-10-01 23:07:47 +01:00
Hashar
7ab9283292 Revert "Add the main stash, WAN, and server caches to MediaWikiServices"
$ php maintenance/eval.php
 > $wgMessageCacheType = CACHE_ACCEL;

 > wfGetMessageCacheStorage();
 PHP Warning:  Illegal offset type in isset or empty in includes/objectcache/ObjectCache.php on line 93
 PHP Warning:  Illegal offset type in isset or empty in includes/objectcache/ObjectCache.php on line 125
 PHP Notice:  Array to string conversion in includes/objectcache/ObjectCache.php on line 133
 Caught exception InvalidArgumentException: Invalid object cache type "Array" requested. It is not present in $wgObjectCaches.
 #0 includes/objectcache/ObjectCache.php(94): ObjectCache::newFromId(Array)
 #1 includes/objectcache/ObjectCache.php(287): ObjectCache::getInstance(Array)
 #2 [internal function]: ObjectCache::getLocalServerInstance(Array)
 #3 includes/objectcache/ObjectCache.php(181): call_user_func('ObjectCache::ge...', Array)
 #4 includes/objectcache/ObjectCache.php(137): ObjectCache::newFromParams(Array)
 #5 includes/objectcache/ObjectCache.php(94): ObjectCache::newFromId(3)
 #6 includes/GlobalFunctions.php(3430): ObjectCache::getInstance(3)
 #7 maintenance/eval.php(78) : eval()'d code(1): wfGetMessageCacheStorage()
 #8 maintenance/eval.php(78): eval()
 #9 {main}

This reverts commit 6d99fa5824.

Change-Id: I4d756eb6f66318a12d8cbbc1ba546397be8a29bf
2016-09-29 10:46:48 +02:00
Aaron Schulz
6d99fa5824 Add the main stash, WAN, and server caches to MediaWikiServices
Also removed unused getLocalServerInstance() b/c $fallback logic.

Change-Id: Ifa5f798de10783741a7b079f22d283bb9cb7f4c0
2016-09-28 15:35:44 -07:00
Aaron Schulz
5bbac35ed1 Replace DatabaseBase:: with Database:: and update type hints
Change-Id: I3919b04eb2de4fa0bf8a02239fb5bbf17d347511
2016-09-27 04:20:03 +00:00
Aaron Schulz
8729167f13 Move RedisBagOStuff to /libs
Change-Id: Ia49fb044aacb44e53d36c80277783cd806aca439
2016-09-23 20:13:36 -07:00
Aaron Schulz
10593ffaab Move RedisConnectionPool to /libs/redis
Change-Id: Ied4a85d7172ab76b90f6d9ce4d47a83c3fd7d111
2016-09-23 00:20:19 -07:00
Aaron Schulz
64060bea46 Split out new ObjectCache::newWANCacheFromParams() method
Change-Id: Ib2e409dd129bd1e2871fe239e71a4eb8fb42944c
2016-09-21 22:06:37 +00:00
Aaron Schulz
5a3d42dbd8 Use standard exceptions in ObjectCache and always handle CACHE_NONE/"hash"
* If cache CACHE_NONE or "hash" is not set yet or was unset,
  return the proper new object anyway for sanity.
* Also fixed $params typo in newFromParams(). This should not
  have any effect since SqlBagOStuff ignores the key in this case.
* Removed excess "use" statement.

Change-Id: I36108a865273b9c209145828e5e1216b545ef431
2016-09-21 21:49:15 +00:00
Marius Hoch
2d16cb6069 Fix SQLite $wgObjectCaches definition + b/c handling
Follow-up to 47e816c69c

Change-Id: Id94857a7c8726871cf826a34b5cf10cb465b7125
2016-09-19 19:41:23 +00:00
jenkins-bot
d9f412635f Merge "Move PECL memcached class to /libs" 2016-09-14 01:45:02 +00:00
Aaron Schulz
a3dacac90f Support masking the WRITE_SYNC latency from ChronologyProtector
* Use OutputPage::output() as the method to mask latency, since it
  takes a good while to run. By the time it runs, cache replication
  should have caught up, so the reap call will likely not block.
* For redirects emitted after changes in POST, instead of masking
  with OutputPage, add a parameter to the redirect and block on
  the positions appearing. This uses the redirection RTT to mask
  the replication latency.

Change-Id: Ib23690c302e8033610fef9a0ef451dafe8a5803e
2016-09-12 23:58:49 +00:00
Aaron Schulz
05589cad6b Move PECL memcached class to /libs
Remove IP class dependency, which is simple enough here

Change-Id: I3aed6ae6747d8d2b9e75b492e5d5293ba4ce9edd
2016-09-07 22:55:11 -07:00
Subramanya Sastry
fc05a13a2d Trivial: Typo fix in comment
Change-Id: I7ad1ee3ca1e3007e70b5566209e3bbbc9b19201a
2016-09-07 14:44:14 -05:00
Aaron Schulz
57e19b610d Renamed some variables from "slave" to "replica"
Change-Id: I455278294cd7ea344d14a76ac5957ece2e07fbf3
2016-09-05 23:03:01 -07:00
Aaron Schulz
950cf6016c Rename DB_SLAVE constant to DB_REPLICA
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.

The old constant is an alias now.

Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
2016-09-05 22:55:53 -07:00
Aaron Schulz
16266edff3 Change "slave" => "replica DB" in /includes
Change-Id: Icb716219c9335ff8fa447b1733d04b71d9712bf9
2016-09-05 21:01:01 +00:00
Aaron Schulz
e99bb1b7dc objectcache: allow for callbacks to mask SYNC_WRITE latency
Change-Id: I908222ad3788ebe330aa58831cda139da32becd8
2016-09-01 13:44:18 -07:00
Aaron Schulz
e92cefafdf objectcache: Make SqlBagOStuff::waitForSlaves() no-op without slaves
Change-Id: Ibaa4745c18c6f4f66edb4c5f190196b575d3b738
2016-08-30 22:47:04 +00:00
jenkins-bot
d1cb2084b7 Merge "Memcached PECL: Allow using binary protocol" 2016-08-28 21:49:47 +00:00
Aaron Schulz
c387e5cd84 Cleanups to SqlBagOStuff
* Keep track of the custom LoadBalancer when it makes one.
* Use the custom LoadBalancer to wait for slaves if one
  was used, rather than the main singleton.
* Only wait on the slaves in the LoadBalancer if the
  main DBs are being used.

Change-Id: I11de814306c44f27e0c33b08b5921c0fd4cdc24f
2016-08-24 19:38:14 -07:00
Matthew Flaschen
ad7a968fb0 Memcached PECL: Allow using binary protocol
This is required for e.g. the touch() method in older versions of
the Memcached client.

It still defaults to ASCII.

Bug: T143464
Change-Id: Ib6b22b47fddc3dbad885f86c31eb1571df0438ff
2016-08-24 14:57:16 -04:00
Aaron Schulz
21abf22680 Code cleanups to SqlBagOStuff
* Refactor local DB usage check into usesMainDB() method.
* Avoid using the db member of DBError instances.

Change-Id: I7350f5a471c551492094bfaf545ebc222eb6f7dd
2016-08-23 05:10:05 +00:00
Aaron Schulz
1f313389e2 Remove direct rollback() calls from some places
Rely on the mass-rollback logic in MWExceptionHandler instead.
This results in a better chance of atomicity.

Change-Id: I2eb5661d4acc105a1323d69c5463268c234bd745
2016-08-23 01:19:21 +00:00
Aaron Schulz
ab0aaef7ad objectcache: Optimize changeTTL() for SqlBagOStuff
Change-Id: I539dd83175a8ea922221f194e23ca47aea7539cf
2016-08-17 19:14:09 +00:00
Amir Sarabadani
efa0d7af44 Clean up array() syntax in docs, part III
Also fixing some typos here and there

Change-Id: I29c29acf87f84ba9993ba75ebf2ad8091d981574
2016-08-13 05:40:40 +04:30
Aaron Schulz
55f6639c2e objectcache: Add changeTTL() method
This can change the TTL without fetching the object
so that keys can be renewed or set to expire soon.

Change-Id: Id1c2c9f89b3445cfa34263057dc5029cbe170833
2016-08-11 19:55:32 -07:00
Bartosz Dziewoński
a49b3216fc Canonicalise '@deprecated since' doc comments
Change-Id: I440041512feb47a998efb45e76e3fd00e0655752
2016-08-02 22:21:57 +00:00
Aaron Schulz
5f921702d2 Avoid stack overflow in LoadBalancer with CACHE_DB WAN/server cache
* Add the ability to expose key BagOStuff attributes like whether the
  emulation SQL cache is being used. Several callers use hacks to detect
  this and can be updated later.
* Fallback to a safe empty WAN cache in the CACHE_DB case. This fixes
  a regression from f4bf52e843.
* Also add this protection to the server cache, which could break
  similarly before too.
* Also fix CACHE_ANYTHING by avoid the recursion risk from
  fc1d4d7960 by just checking the disabled service map.

Bug: T123829
Bug: T141804
Change-Id: I17ee26138f69e01ec1aaddb55ab27caa4d542193
2016-08-02 13:06:49 +00:00
Kunal Mehta
4a8ae0a0b4 Use callable type-hint in a few places
Mostly places which immediately had a:
 if ( !is_callable( $callback ) ) {
  throw new Exception(...);
 }
check at the beginning of the function.

Change-Id: Ia78663b2231629010816bd1cda8814b996968d1d
2016-06-07 16:39:22 -07:00
daniel
fc1d4d7960 Fix installer issues introduces by MediaWikiServices
This fixes three issues with the installer:

1) Make sure LocalizationCache can find the installer's i18n files.
2) Make sure we don't try to use an SqlBagOStuff for caching before we have
   a functioning database.
3) Don't try to output HTML when redirecting (this is unrelated to
   MediaWikiServices, but came up during testing)

Bug: T135169
Change-Id: I7caa932024cd771d6fa226a3ac6001c3148ecc9c
2016-05-17 15:24:31 +00:00
Brad Jorsch
cdc93a62bf RedisBagOStuff: Fix unserialization of negative numbers
ctype_digit( $data ) doesn't return true if $data is
a negative integer.

Bug: T134923
Change-Id: Ie8a23fc6354a15210e010062e3da3058f4c463bb
2016-05-11 20:51:08 -07:00
Stanislav Malyshev
4f95365877 Convert EventRelayerGroup to services infrastructure
Change-Id: If55ddf441de69189c7fcdd3440f899c6b14f690f
2016-04-25 14:22:31 -07:00
Aaron Schulz
09a0a5e4f6 Make WAN cache config use $wgEventRelayerConfig
This makes the channels more explicit and defined in a less
ad-hoc way. Systems like Kafka would prefer explicit channel
definitions anyway, so the channel prefix just obscures things.

Change-Id: I5631eb1b1382083396a0f08904d9273cc92601e8
2016-04-25 12:08:09 -07:00
Bryan Davis
450bb6723d BagOStuff: Optionally log duplicate key lookups
Add new 'reportDupes' parameter to BagOStuff. This parameter enables scheduling
of a callback after the current web request ends. This callback will emit warning
log messages for all keys that were requested more than once.

The default ObjectCache factory for MediaWiki enables this option by default.
Not by default for plain BagOStuff instances, however. E.g. `new HashBagOStuff()`.

It also set 'asyncHandler' for all classes now (not just MultiWriteBagOStuff).

Bug: T128125
Co-Authored-By: Timo Tijhof <krinklemail@gmail.com>
Change-Id: I8a2b06cf54d2acf5950eed71756ecdf50e224be1
2016-04-22 04:21:20 +00:00