Commit graph

40 commits

Author SHA1 Message Date
Aaron Schulz
fc5d51f129 jobqueue: add GenericParameterJob and RunnableJob interface
Simplify the code of jobs that do not care about titles and removes
the direct Title dependency from JobQueue. Remove getTitle() from
IJobSpecification itself. Move all the Job::factory calls into a
single JobQueue::factoryJob() method.

Depends-on: Iee78f4baeca0c0b4d6db073f2fbcc56855114ab0
Change-Id: I9c9d0726d4066bb0aa937665847ad6042ade13ec
2019-04-08 11:05:23 -07:00
Aaron Schulz
a51ea350be Add Job::getMetadata() and Job::setMetadata() accessors
Change-Id: I3a97008d324f600a1c9f6005673073277ee564fa
2019-04-03 16:34:32 +00:00
Kunal Mehta
cc5d9a92a2 build: Updating mediawiki/mediawiki-codesniffer to 24.0.0
Change-Id: I66b1775b7c1d36076d9ca78cbeb42787a743f2aa
2019-02-07 18:39:42 +00:00
Aaron Schulz
51945dbca3 Use DB domain in JobQueueGroup and make WikiMap domain ID methods stricter
Using domains means thats JobQueueDB has the right value to use for calls
like LoadBalancer::getConnection(). The full domain includes the schema in
the case of Postgres. This makes calls to getConnection() less awkward by
not relying on the fallback logic in reallyOpenConnection() for null schemas.

Make getWikiIdFromDomain/isCurrentWikiDomain account for the schema if it
is both defined and is not simply the generic "mediawiki" schema MediaWiki
uses by default. If all wikis use the default schema, the wiki IDs can get
by with DB/prefix alone, which various config and methods may be built around.
Otherwise, the config callbacks must account for schema and the config must
include it in various wiki domain ID lists to properly disambiguate wikis.

Also, clean up SiteConfiguration::siteFromDB() since it is not meant
to handle schemas unless the callback method was taylored to do so.

Finally, add more comments to DefaultSettings.php about already existing
limitations of wiki domain IDs and their components.

Change-Id: I8d94a650e5c99a19ee50551c5be9544318eb05b1
2018-11-07 04:46:56 +00:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
Aaron Schulz
45a6a7ad67 Set visiblity of some HashRing methods
* getLiveRing() is now protected
* getLocation() is now final

Change-Id: I16002fe7187d8bbb6e36d4dd9a1302ee1d46a4bb
2018-03-17 23:49:03 -07:00
Umherirrender
255d76f2a1 build: Updating mediawiki/mediawiki-codesniffer to 15.0.0
Clean up use of @codingStandardsIgnore
- @codingStandardsIgnoreFile -> phpcs:ignoreFile
- @codingStandardsIgnoreLine -> phpcs:ignore
- @codingStandardsIgnoreStart -> phpcs:disable
- @codingStandardsIgnoreEnd -> phpcs:enable

For phpcs:disable always the necessary sniffs are provided.
Some start/end pairs are changed to line ignore

Change-Id: I92ef235849bcc349c69e53504e664a155dd162c8
2018-01-01 14:10:16 +01:00
Umherirrender
a9007e8baf Add missing & to @param documentation to match functon call
Change-Id: I81e68310abcbc59964b22e0e74842d509f6b1fb9
2017-08-11 18:47:46 +02:00
Petr Pchelko
7fd1f6a17b JobQueue: Create a debugging queue
As a part of creating the kafka-based JobQueue implementation,
we need to transition to the new queue smoothly and be able to
first debug the event production side, then try executing a no-op
job in parallel with each of the real jobs (to be able to check
deduplication and all the jpb management code) and only then
try switching jobs one-by-one to the new backend queue.

In order to achieve that at first we need to temporary produce
both to the old job pipeline and to the new one - thus creating
a queue 'splitter' - it delegates all the methods to the main
queue while the produce requests also go to the EventBus.

This class it temporary, so it's marked deprecated right away.
It will not be needed once the queue transition is over.

Bug: T163380
Change-Id: I483b8f00f1c593ff3af6ba6826868b553339525f
2017-07-25 18:13:38 +00:00
Aaron Schulz
82e2c924e4 Remove "@author Aaron Schulz" annotations
Bug: T139301
Change-Id: Ib5248e8e27d60611c7373bce4b29dd5e85aa3489
2017-06-27 15:24:14 -07:00
Aaron Schulz
b5d8f3aed1 Fix some JobQueue IDEA errors
Change-Id: I624f65ee1ca97c1acae9b54ca36d910eb4e42a70
2016-03-08 03:23:13 -08:00
Bartosz Dziewoński
c161c46d26 Improve code suffering from PHP 5.3's lack of support for foo()[]
I searched for /\$(\S+) = (.+?\(.*?\);)\n.*?\$\1\[/, ignored
everything involving isset(), unset() or array assigments, then
skimmed through the remaining results and changed things where they
made sense. These changes were not automated, so please review them.

Change-Id: Ib37b4c66fc57648470f151ad412210b3629c2538
2016-02-28 22:49:20 +01:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Aaron Schulz
ec351986e5 Remove JobQueue::setTestingPrefix() hack
The tests are only run on dev install and only touch the
null queue anyway.

Change-Id: I441a2a4605a9e2984142485b022dd524ff819360
2015-12-10 17:20:10 -08:00
Aaron Schulz
de58577c2d Make sure job queue partitions use the null aggregator
* Fixes problem from 37042262e3
* The main class already handles the calls.
* Partitions call notifyQueueNonEmpty() on pop() even
  when the other partitions have jobs, so it is also wrong
  in addition to redundant.

Bug: T101427
Change-Id: Ic3235e1f2038053fabf92f97c663479a21d75317
2015-06-05 00:02:45 +00:00
Aaron Schulz
22734b3c0f Let deduplicateRootJob() accept JobSpecification for consistency
Change-Id: I872c2af40028e918fe6d9a8cd32ac97c70bffdae
2015-05-23 12:49:44 -07:00
jenkins-bot
2e89500994 Merge "Removed executeReadyPeriodicTasks() method" 2015-05-12 20:03:02 +00:00
Aaron Schulz
2f99b38f5f Removed unused variable in doFlushCaches()
Change-Id: I599fc2e8575cf5e8242bcc15be93e54b74ede435
2015-05-07 18:21:12 +00:00
Aaron Schulz
ec3da97659 Removed executeReadyPeriodicTasks() method
* Moved all these hacks to JobQueueDB, which is the only queue that
  should need this (for stock installs). Newer queues should always
  have the queue store manage stuff like this, not MediaWiki.
* This also avoids expensive object construction that does nothing
  when non-DB queues are used.

Change-Id: Id718cda25750be73044a049b39958cca55aa3172
2015-05-06 20:20:40 -07:00
Aaron Schulz
6c9e517a70 Made showJobs.php show claimed jobs too
Change-Id: Ifc5eeff10fdcdbbe4ade7b32e5388fee4eb94bcf
2015-04-17 12:35:39 -07:00
Aaron Schulz
9720c6c410 Made showJobs.php include abandoned jobs in --list
Change-Id: I7ae8dd2470d5e15fd66c6c06f3feb6d70527daa3
2015-03-26 12:17:10 -07:00
jenkins-bot
4f1067e27e Merge "Removed odd "partitionsNoPush" setting to simplify the code" 2015-03-18 23:06:50 +00:00
Aaron Schulz
ce33c484ed Made JobQueueFederated::supportsDelayedJobs() check all the partition queues
Change-Id: Id451faedd6286a84f993fa183e67e0bf35d21c41
2015-03-11 11:10:12 -07:00
Aaron Schulz
6846ff953f Removed odd "partitionsNoPush" setting to simplify the code
* A maintenance script already exist for copying jobs to other queues

Change-Id: Id105e7e60fae4b8a30cda74a45597a2a2e548a91
2015-03-03 22:00:43 -08:00
jenkins-bot
ea3646b476 Merge "Removed overzealous caching from JobQueueFederated" 2015-03-04 05:48:38 +00:00
Aaron Schulz
32684df25f Moved JobQueueFederated partition exceptions to their own log
Change-Id: If495e9b1a631b587cb3b8d83974837eef1d2d4c6
2015-03-02 11:15:40 -08:00
Aaron Schulz
d8a12160a2 Removed overzealous caching from JobQueueFederated
Change-Id: Id3a80dc89e5b46d6d95180e73e753b38a476eb36
2015-02-16 14:33:44 -08:00
Ricordisamoa
fc5fd5c37a Typo fixes and non-code tweaks
Skipped replacements:
* prefered → preferred
* prolly → probably

Skipped files:
* resources/lib/jquery.ui/jquery.ui.mouse.js
* resources/lib/jquery/jquery.form.js

Change-Id: Ib7923f362ddfca1b892bf5d601785d6b5aa5d44c
2014-12-12 18:31:15 +00:00
Aaron Schulz
bc5e7d037a Fixed use of wrong class constant
Change-Id: I6c7e49d16685315f9e84b61ba11d1a3b08730597
2014-07-28 12:03:58 -07:00
Aaron Schulz
77ea812c9e Removed use of cache in JobQueueFederated pop() method
* This does not really buy anything for a number of reason and
  can cause bugs. For example, if the aggregator is notified due
  to undelayed jobs but this cache is not updated, then the runners
  will not see the jobs and will de-list aggregator entry.

bug: 68506
Change-Id: I1eacca108df59fff9c0fcc846aafaf1616cf5895
2014-07-28 17:16:08 +00:00
umherirrender
44d14fc684 Doc: Reformat @params declaration
I have not found documentation about the @params command, so this seems
not valid.
I have moved the text to the corresponding @param

Change-Id: I443866d602b60570278289e6e24a77d37314cede
2014-07-24 13:12:18 +02:00
Aaron Schulz
8052dc9ffc Fixed job miscounting bug when a string digit is returned
Change-Id: I857b56a6e78f715157c7f648929d6bf38eb74828
2014-05-19 10:12:14 -07:00
Thiemo Mättig
e292bc972d Finish removing boolean return values from JobQueue code
This is a follow-up for patch
Ia706ac0122a7dd7f418e2dc2d3bd36e9a0252c25.

Change-Id: I19fe58a939706d3f7594d937e0bcad6d97c52a50
2014-04-28 22:21:58 +02:00
Siebrand Mazeland
f994817f6b Pass phpcs-strict on various files in includes/
These files have all had treatment before, and these occurrences have either
been missed or have been introduced after.

Change-Id: I06cdab4616b5bff47c85152df28f18c861730a23
2014-04-24 21:50:01 +02:00
umherirrender
dcf6955e5c Fixed some @params documentation (includes/*)
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.

Change-Id: Ifbb1da2a6278b0bde2a6f6ce2e7bd383ee3fb28a
2014-04-20 23:33:05 +02:00
Aaron Schulz
f7f710287b Improved HashRing usage in JobQueueFederated
* Added proper ejection and caching in HashRing to avoid rebuilding
  the hash all the time (or doing manual caching).
* Made JobQueueFederated blacklist failing servers for a few seconds.
* Also made the JobQueueFederated root job methods properly respect
  the weights when they fail over.

Change-Id: Ifa4c03272c1777cfff2523ab21f780074ddcf359
2014-04-18 20:28:48 +00:00
Aaron Schulz
94c37ffb96 Revert "Revert "Removed useless JobQueue return values""
Made the obvious update to a caller missed in the change.

This reverts commit c76d5a95c1.

Change-Id: I67400ba5b9fc7de16c9f9d5075c488c5e58cea9e
2014-04-16 11:22:31 -07:00
Reedy
c76d5a95c1 Revert "Removed useless JobQueue return values"
This reverts commit bc8c89d2df.

Bug: 64007

Change-Id: I4b4dbe4637dc50cd4630ef19d54f01efba10e138
2014-04-16 17:41:53 +00:00
Aaron Schulz
bc8c89d2df Removed useless JobQueue return values
* This are already totally redundant to the use of exceptions

Change-Id: Ia706ac0122a7dd7f418e2dc2d3bd36e9a0252c25
2014-04-02 17:56:05 -07:00
Aaron Schulz
9ffd4f085d Renamed /job to /jobqueue
Change-Id: I4c8a2b42140630838867c77a70d45ba14b5d95e2
2014-03-14 13:42:04 -07:00
Renamed from includes/job/JobQueueFederated.php (Browse further)