Commit graph

60 commits

Author SHA1 Message Date
Aaron Schulz
720ba67b5f Added a quick test for getAllQueuedJobs().
Change-Id: Ia39461bc1e2356da1a694bf1898b8ca8e4200ff8
2013-04-09 14:32:20 -07:00
Siebrand Mazeland
6da93fc6f6 Update code formatting
Also update some previous inconsistencies pointed out by Krinkle in change IDs:
* Ide20743a2e84ff68549286120e6cff9d9f396f54
* I811ca957b6588085d67606ebc0cd4033a1e53839

Change-Id: Ife33b931870d0d7e04fcb40974997436d27f528f
2013-03-27 14:15:11 +01:00
jenkins-bot
b3d9004342 Merge "Tests: Use more setMwGlobals" 2013-03-22 19:15:03 +00:00
Timo Tijhof
b36d883017 Tests: Make phpunit providers "public static".
Follows-up I9d2b148e57 (including phpunit/languages this time).

Bug: 46434
Change-Id: I30e5efcd88c516121c454676bd7a18f9b7c8fca6
2013-03-22 03:12:37 +01:00
umherirrender
ff3485ec99 Tests: Use more setMwGlobals
Change some tests to use setMwGlobals to have restoring of globals after
the test.
This also removes some save/restore code, which is not needed, due to
the automatically restoring on tearDown with setMwGlobals.

Change-Id: I8d2ac9f6cc14f0bd4ee8eb851c09f2e71babc6e0
2013-03-21 20:35:44 +01:00
Aaron Schulz
cc3ac23c4f [JobQueue] Optimized redis queue to use Lua scripts.
* Cleaned up some data structures into hashes, which get better
  compression and play well with the KEYS parameter in Lua scripts.
  The claim list is now a sorted set with O(logN) removal in ack()
  and O(log(N)+M) searching in recycleAndDeleteStaleJobs().
* Made the class itself control object serialization, so that lua
  scripts have an easy time. Only the job data itself needs to be
  serialized, where as other things just get bloated.
* Used Lua scripts to get push(), pop() and ack() down to 1 RTT.
* Likewise rewrote recycleAndDeleteStaleJobs() to use a script.
* Fixed bug where claimed duplicate jobs removed the data on ack(),
  which meant that claimed duplicated jobs could no-op newer ones.
  De-duplication should only apply to unclaimed jobs like for the
  JobQueueDB class, so that unfinished jobs don't no-op new ones.
* Removed locking in recycleAndDeleteStaleJobs(), which would not do
  much since the exclusive set request would serialize on the lua
  script anyway. The lua script will finish quickly the next times
  if done more than once in a row due to sorted set usage.
  Also made recycleAndDeleteStaleJobs() run randomly to reduce the
  chance of a single calling tying up the server.
* Removed useless hDel() call in getJobFromUidInternal().
* Changed unit tests to handle the different supported orders better.
  Added tests for the 'timestamp' ordering.

Change-Id: Ib2d7aff18753195248ab856afd4a46e18b301db9
2013-03-20 01:04:53 +00:00
umherirrender
8764b3aa7c Remove spaces in function signature
Change-Id: I45aea7a7af88cd913b2f485913620a8af0ab2fed
2013-03-18 20:44:43 +01:00
Aaron Schulz
9cc7e6a39a [JobQueue] Cleanups for JobQueueRedis.
* Cleaned up 'server' option to not fragment the pool.
  Also made it actually match the documentation.
* Made it use doGetPeriodicTasks() for job recycling.
* Made it so that other job queue classes can be tested.
* Renamed "redisConf" => "redisConfig".
* Tweaked comments about the "random" order option.

Change-Id: I7823d90010e6bc9d581435c3be92830c5ba68480
2013-02-21 05:38:01 +00:00
Siebrand Mazeland
de04f37ef1 Update formatting
5 of n.

Change-Id: I811ca957b6588085d67606ebc0cd4033a1e53839
2013-02-15 12:35:50 +00:00
Aaron Schulz
5e81bffce0 [JobQueue] Added unit tests for job queue code.
Change-Id: I13f2dd555ea807b5d13ea9fbee6626d6a8f5aab3
2013-02-07 09:18:10 -08:00