Commit graph

15 commits

Author SHA1 Message Date
Tim Starling
4aecdad647 HashRing: use bisection instead of guess and refine
Use bisection to find the position within the ring, as in ketama.
According to my benchmarking this is faster than the guess/refine
algorithm for rings with 100 locations, i.e. 16000 ring nodes requiring
at most 14 bisection iterations.

Change-Id: I6042f382de093081971b5ec210eda8dfa74988f2
2018-06-08 02:36:49 -07:00
Aaron Schulz
f2e6543c46 Rewrite HashRing to use consistent hashing
This scheme avoids disruptive re-hashing when a node is
ejected; only the adjectent clockwise sector is affected,
the later taking responsibility for the former. Addition
of new nodes is likewise less disruptive.

When used with MD5, this maps keys the same as libketama.

Also:
* Make HashRing::getLocations() faster by guessing the
  sector list index and moving clockwise/counter-clockwise.
* Made HashRing Serializable for more robust storage.
* Allow for different hash functions to be specified.

Note:
* This makes PoolCounterRedis use the new hash scheme.
* Likewise for JobQueueFederated (for job de-duplication).
  Switching from one mode to the other will cause some of
  duplicated effort from old jobs only temporarily.

Change-Id: Iff432a4e5dc3fb40b2dda70fb3dbe9dda0b74933
2018-06-07 03:36:22 -07:00
Aaron Schulz
0bc629fefc Remove unused HashRing::newWithoutLocation() method
Change-Id: I6f149b2ef2e7e9fe2e7ddbda6c0c7d795c24720e
2018-03-17 23:49:21 -07:00
Aaron Schulz
45a6a7ad67 Set visiblity of some HashRing methods
* getLiveRing() is now protected
* getLocation() is now final

Change-Id: I16002fe7187d8bbb6e36d4dd9a1302ee1d46a4bb
2018-03-17 23:49:03 -07:00
Aaron Schulz
e0ff77f130 Convert @var Array => array
Change-Id: Ie5c2d7b8e73cce74af982a5c54b0d487a7f6981d
2017-11-09 22:40:40 -08:00
Brad Jorsch
2c34fd6e0e Replace uses of each()
It's deprecated in PHP 7.2, may as well replace it now.

I note that, contrary to claims at
https://wiki.php.net/rfc/deprecations_php_7_2#each, none of our uses
were trivially replaceable with foreach.

* wfArrayDiff2_cmp() is processing two arrays by value in parallel.
* MagicWordArray::parseMatch() is doing something funky with the data
  structure returned by preg_match().
* HashRing was using it like "nextKey()", replaced with calls to key()
  and next().
* FormatMetadata and IndexPager were both using it as a shorter way to
  get both key() and current() for the first element in the array. I
  suppose a foreach(){ break; } would do the same, but that's confusing.

Bug: T174354
Change-Id: I36169a04c764fdf1bfd6603395111c6fe0aae5eb
2017-09-20 09:51:28 -04:00
Umherirrender
3f1a52805e Use short type bool/int in param documentation
Enable the phpcs sniffs for this and used phpcbf

Change-Id: Iaa36687154ddd2bf663b9dd519f5c99409d37925
2017-08-20 13:20:59 +02:00
Aaron Schulz
82e2c924e4 Remove "@author Aaron Schulz" annotations
Bug: T139301
Change-Id: Ib5248e8e27d60611c7373bce4b29dd5e85aa3489
2017-06-27 15:24:14 -07:00
Umherirrender
be42e09aa8 build: Prepare for mediawiki/mediawiki-codesniffer to 0.9.0
The used phpcs has a bug, so the version 0.9.0 could not be enforced at the moment.
Will be fixed in next version, see T167168

Changed:
- Remove duplicate newline at end of file
- Add space between function and ( for closures
- and -> &&, or -> ||

Change-Id: I4172fb08861729bccd55aecbd07e029e2638d311
2017-06-26 17:14:31 +00:00
Erik Bernhardson
d67197fa11 Cleanup some incorrect return annotations
Most of these are simply changing annotations to reflect
reality. If a function can return false to indicate failure
the @return should indicate it.

Some are fixing preg_match calls, preg match returns 1, 0 or false,
but the functions all claim to return booleans.

This is far from all the incorrect return types in mediawiki, there
are around 250 detected by phan, but have to start somewhere.

Change-Id: I1bbdfee6190747bde460f8a7084212ccafe169ef
2016-12-12 10:15:05 -08:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Aaron Schulz
9442f8c196 Fixed HashRing IDE errors
Change-Id: Ia3fa0df6f13adc4788d1414a9bc965fdc4ec1915
2015-08-24 04:57:26 -07:00
Aaron Schulz
3993b6a4de Fixed broken exception code
Change-Id: Id399cb22f183d75acc006e698feb161bd12096bd
2014-07-03 16:37:25 -07:00
Aaron Schulz
f7f710287b Improved HashRing usage in JobQueueFederated
* Added proper ejection and caching in HashRing to avoid rebuilding
  the hash all the time (or doing manual caching).
* Made JobQueueFederated blacklist failing servers for a few seconds.
* Also made the JobQueueFederated root job methods properly respect
  the weights when they fail over.

Change-Id: Ifa4c03272c1777cfff2523ab21f780074ddcf359
2014-04-18 20:28:48 +00:00
Aaron Schulz
bb519a91df Moved HashRing to /libs
Change-Id: I0b74b386f7459f550816f99aa7e00970c3cff4c7
2014-01-07 22:18:09 +00:00
Renamed from includes/utils/HashRing.php (Browse further)