Commit graph

69 commits

Author SHA1 Message Date
Umherirrender
e0dfa35032 Remove JobQueueGroup::singleton()/destroySingletons()
Bug: T301016
Change-Id: Ia059bff9a9e0c61dcd803c5a69261827b1fc0fa3
2022-09-26 18:10:44 +02:00
Thiemo Kreuz
e23b070b45 Make use of ?? and ?: operators where it makes sense
Change-Id: I1d9d62d80b17d1a05222c6e82f631cb801c311a8
2022-08-04 21:43:12 +02:00
Umherirrender
049c6f198d Hard deprecate JobQueueGroup::singleton()/destroySingletons()
Bug: T301016
Change-Id: I7873224af5cb40a17eb0c6b06cbc39f0d8c6e7c1
2022-06-24 18:48:31 +02:00
Reedy
11cc2d6993 Consistently use @deprecated since rather @deprecated
Change-Id: I301786f35cd6eb9c63fb72e0b64cffa9efd7b86b
2022-03-06 02:55:54 +00:00
Timo Tijhof
4646ba7a59 jobqueue: Fix reference to $wgConf
Partially reverts 47adb6d65a (I1a691f01cd82e60). This isn't a configuration
variable.

Change-Id: I7ce7c5529db6a463efa080225f0923ab866177c0
2022-01-12 22:14:54 +00:00
TChin
47adb6d65a Refactor global variables to use MediaWikiServices instead
Automatically refactors wg prefixed globals to use MediaWikiServices config using Rector. Doesn't include files that set globals or files that fail CI.

Rector Gist: https://gist.github.com/tchin25/7cc54f6d23aedef010b22e4dfbead228

* This patch uses a modified source code rector library for our specific use case and the rector will have different effects without it.

A writeup for future reference is here: https://meta.wikimedia.org/wiki/User:TChin_(WMF)/Using_Rector_On_MediaWiki

Change-Id: I1a691f01cd82e60bf41207d32501edb4b9835e37
2022-01-10 13:55:53 -05:00
Reedy
7bf779524a Remove or replace usages of "sanity"
Bug: T254646
Change-Id: I2b120f0b9c9e1dc1a6c216bfefa3f2463efe1001
2021-11-19 23:19:42 +00:00
Amir Sarabadani
419c14b013 jobqueue: Batch jobs that will end up in the default queue
JobQueueGroup::get( $type ) returns a new object for each job type even
if it's the same class (default class). Also it builds different queues
and push them separelty a couple lines lower limiting the ability to
batch push them in the default job queue object.

Given that JobQueue cares about job types being the same as the class
type (which is important for JobQueueDB implementation), adding a config
option called typeAgnostic to ignore type mismatch in JobQueue
implementations that don't care about the type (like JobQueueEventBus)

Bug: T292048
Change-Id: I151f067ca94a985c816446b545921c387b083911
2021-10-04 09:04:11 +02:00
Umherirrender
240eaed9df Improve exception message for JobQueueGroup::assertValidJobs
Bug: T287623
Change-Id: I2108fe719a256c5d1d613055fdcb2890341bc0ee
2021-08-19 22:05:49 +02:00
Lucas Werkmeister
a8c775f112 JobQueueGroup: Fix method name in deprecation notice
Bug: T277648
Change-Id: I4d9375fc439d9f76a451024437755166fab4246b
2021-06-23 10:43:44 +00:00
mainframe98
ae55be3000 Add a JobQueueGroupFactory service
This replaces JobQueueGroup::singleton and ::destroySingletons.
A JobQueueGroup service has been added to serve as convenience
getter.

Bug: T277648
Change-Id: Ic8709119773ab788a07531193dc65d418536eda2
2021-05-07 06:50:29 +00:00
Reedy
bbe130b242 Rename some variables to use more neutral language
Bug: T254646
Change-Id: I997625b8201dce2c257d20f96d00089e995c2b0d
2021-03-19 16:18:49 +00:00
Umherirrender
8de3b7d324 Use static closures where safe to use
This is micro-optimization of closure code to avoid binding the closure
to $this where it is not needed.

Created by I25a17fb22b6b669e817317a0f45051ae9c608208

Change-Id: I0ffc6200f6c6693d78a3151cb8cea7dce7c21653
2021-02-11 00:13:52 +00:00
DannyS712
f4e21014ce Deprecate a bunch of global functions
* wfAcceptToPrefs
* wfClearOutputBuffers
* wfConfiguredReadOnlyReason
* wfDebugMem
* wfGetPrecompiledData
* wfNegotiateType

Bug: T264976
Bug: T264979
Bug: T264981
Bug: T264983
Bug: T264984
Bug: T264985
Change-Id: Ia05bc84e4d1be7c8a02472f32e2c009e4bb32032
2020-12-18 21:34:50 +00:00
Thiemo Kreuz
b0130ca649 Update a lot of unspecific "array" types in PHPDocs
This includes fixing some mistakes, as well as removing
redundant text that doesn't add new information, either because
it literally repeats what the code already says, or is actually
duplicated.

Change-Id: I3a8dd8ce57192deda8916cc444c87d7ab1a36515
2020-10-28 11:01:33 +01:00
Umherirrender
ba216e52e7 includes: Use expression assignment operator += or |= where possible
It is easier to read.

Change-Id: Ia3965b80153d64f95b415c6c30f526efa252f554
2020-07-31 22:26:42 +00:00
Reedy
d53e91d272 Fix more PSR12.Properties.ConstantVisibility.NotFound
Change-Id: I94520b10d78a17ea8e965633dd475ea711f25c99
2020-05-15 00:33:32 +01:00
Niklas Laxström
1ccac4d9b9 Fix typo in JobQueueGroup
Change-Id: I0d55eec426486549c04d3f7c3b10305d94c2f779
2020-04-09 07:05:36 +00:00
Aaron Schulz
6b12696452 Move UIDGenerator code to a service and put it under /libs
All MediaWiki dependencies have been removed or injected.

Change-Id: I01c9e96edd6b03496c1595670967ffa5a4069c9d
2020-02-18 00:20:40 +00:00
Daimona Eaytoy
e2e543f7c2 Unsuppress more phan issues (part 5)
Bug: T231636
Depends-On: I6e5fba7bd273219b1206559420b5bdb78734aa84
Change-Id: I50377746f01749b058c39fd8229f9d566224cc43
2019-09-01 09:48:31 +00:00
Aaron Schulz
43c78e83d7 jobqueue: migrate root job deduplication to the WAN cache
If the root job timestamp keys are lost or otherwise unknown, they
will now be deductively recached with the best known values as jobs
are popped and executed. This means the running any of many child
jobs of a root job can restore the root timestamp if it was lost.
This does not need to use the main stash given this fact.

Bug: T227376
Change-Id: Iae0f3af15803af048ff49f3bf281b2bde18c87f2
2019-07-13 15:38:17 -07:00
Umherirrender
cb82a52adf Fix type hints in jobqueue related classes
JobQueueGroup is giving RunnableJob on pop(), so it should take the same
type for ack() and deduplicateRootJob()
JobQueue::ack alsready accept the interface

Also change to RunnableJob in JobRunner to work with the type from the
job queue

Change-Id: I7b09586cff8affabe807ee16e80d04f5137dce45
2019-07-05 22:20:56 +02:00
Umherirrender
4b489fa2a7 Fix type hint for properties holding MapCacheLRU
Change-Id: Ia22b1dbcde3b96b3f437a213da95b3556fa42d70
2019-06-05 20:32:16 +02:00
Derick Alangi
c9e365df68 jobqueue: Remove deprecated unused method pushLazyJobs()
Was deprecated in 1.33 and is no longer used anywhere. See usage
below;

Usage
=====

https://codesearch.wmflabs.org/search/?q=%5CbpushLazyJobs%5Cb&i=nope&files=&repos=

Bug: T220656
Change-Id: I4d3f6bc019b8ebc7e504d841fc75acdb08061cb4
2019-05-13 10:37:28 +00:00
Aaron Schulz
fc5d51f129 jobqueue: add GenericParameterJob and RunnableJob interface
Simplify the code of jobs that do not care about titles and removes
the direct Title dependency from JobQueue. Remove getTitle() from
IJobSpecification itself. Move all the Job::factory calls into a
single JobQueue::factoryJob() method.

Depends-on: Iee78f4baeca0c0b4d6db073f2fbcc56855114ab0
Change-Id: I9c9d0726d4066bb0aa937665847ad6042ade13ec
2019-04-08 11:05:23 -07:00
Aaron Schulz
b48dcf28d6 jobqueue: dependency inject more objects into JobQueue
Also moved some WikiMap/$wgJobClasses checks to JobQueueGroup::pop
which is the method callers are supposed to use.

Change-Id: I2ab82d8adc4ae1f54697d2935afa2053539cf2db
2019-04-04 01:32:59 -07:00
jenkins-bot
f28f7a0d0f Merge "jobqueue: remove entire unused JobQueueAggregator class hierarchy" 2019-04-03 17:43:31 +00:00
Aaron Schulz
1ca3cc260f jobqueue: change MWException throw statements to using JobQueueError
Also fix some minor phpstorm warnings in JobQueue

Change-Id: Ie4970f3e93b504fae509ec755b0e0afad71a7749
2019-03-29 21:29:11 -07:00
Aaron Schulz
196ef348c5 jobqueue: remove entire unused JobQueueAggregator class hierarchy
Change-Id: I3d56d5db54fcb4278c7cf2a8d08933c27773c3f7
2019-03-29 20:25:05 -07:00
Aaron Schulz
18dd14a05a Avoid use of deprecated wikiId field in JobQueue
Change-Id: I9e30519e7985c20525572b685234ad239d62c214
2019-03-29 15:26:25 -07:00
Aaron Schulz
5196ac32c6 Rename WikiMap DB domain ID methods to reduce confusion with web domains
Those added in this same release do not need alias methods.

Change-Id: I05feeb9b0b13afe43aea1f95551965489cdbe094
2019-02-06 12:28:45 -08:00
Thiemo Kreuz
734a969d55 Safe replacement of a lot of !count() with === []
This was originally a global search and replace. I manually checked all
replacements and reverted them if (due to the lack of type hints) either
null (that would be 0 when counted) or a Countable object can end in the
variable or property in question.

Now this patch only touches places where I'm sure nothing can break.

For the sanity of the honorable reviewers this patch is exclusively touching
negated counts. You should not find a single `!== []` in this patch, that
would be a mistake.

Change-Id: I5eafd4d8fccdb53a668be8e6f25a566f9c3a0a95
2019-01-15 17:28:49 +01:00
Aaron Schulz
51945dbca3 Use DB domain in JobQueueGroup and make WikiMap domain ID methods stricter
Using domains means thats JobQueueDB has the right value to use for calls
like LoadBalancer::getConnection(). The full domain includes the schema in
the case of Postgres. This makes calls to getConnection() less awkward by
not relying on the fallback logic in reallyOpenConnection() for null schemas.

Make getWikiIdFromDomain/isCurrentWikiDomain account for the schema if it
is both defined and is not simply the generic "mediawiki" schema MediaWiki
uses by default. If all wikis use the default schema, the wiki IDs can get
by with DB/prefix alone, which various config and methods may be built around.
Otherwise, the config callbacks must account for schema and the config must
include it in various wiki domain ID lists to properly disambiguate wikis.

Also, clean up SiteConfiguration::siteFromDB() since it is not meant
to handle schemas unless the callback method was taylored to do so.

Finally, add more comments to DefaultSettings.php about already existing
limitations of wiki domain IDs and their components.

Change-Id: I8d94a650e5c99a19ee50551c5be9544318eb05b1
2018-11-07 04:46:56 +00:00
jenkins-bot
c6ad2554f4 Merge "Add isCurrentWikiId()/isCurrentWikiDomain()/getCurrentWikiDomain() to WikiMap" 2018-10-29 22:27:15 +00:00
Aaron Schulz
dcd0a3d534 Add isCurrentWikiId()/isCurrentWikiDomain()/getCurrentWikiDomain() to WikiMap
Use these in place of various wfWikiID() calls.

Also cleanup UserRightsProxy wiki ID variable names and removed unused
and poorly named getDBname() method.

Change-Id: Ib28889663989382d845511f8d34712b08317f60e
2018-10-29 14:53:37 -07:00
jenkins-bot
dea8f5344b Merge "Create JobQueueEnqueueUpdate class to call JobQueueGroup::pushLazyJobs()" 2018-10-28 23:10:53 +00:00
Aaron Schulz
6030e9cf2c Create JobQueueEnqueueUpdate class to call JobQueueGroup::pushLazyJobs()
This assures that MergeableUpdate tasks that lazy push job will actually
have those jobs run instead of being added after the lone callback update
to call JobQueueGroup::pushLazyJobs() already ran.

This also makes it more obvious that push will happen, since a mergeable
update is added each time lazyPush() is called and a job is buffered,
rather than rely on some magic callback enqueued into DeferredUpdates at
just the right point in multiple entry points.

Bug: T207809
Change-Id: I13382ef4a17a9ba0fd3f9964b8c62f564e47e42d
2018-10-28 22:19:06 +00:00
Fomafix
43244db9a2 Use PHP 7 '??' operator instead of if-then-else
Change-Id: If9d4be5d88c8927f63cbb84dfc8181baf62ea3eb
2018-10-21 21:46:46 +02:00
Marko Obrovac
a5aa44567b JobQueueGroup: Allow readOnlyReason to be specified per JQ type
We use $wgReadOnly for various reasons, one of which is to disallow
writes in the currently-non-active DC. However, we should allow the
readOnlyReason configuration variable to be available per JobQueue type
and have it the code respect that.

In our current set-up, we use JobQueueEventBus which ever only uses the
enqueue execution path and does not depend upon which DC it is executed
in, so this will allow us to enqueue jobs in both DCs.

Note that this is an alternative approach to the one outlined in
Ibbad6063b6b154d7f7d172c79f7be324bf80eb7e

Bug: T199594
Change-Id: I8f1a57a81ea11c1c587c0057fa8bb3454b0e0b56
2018-07-19 18:13:38 +02:00
Aaron Schulz
e5d71f2552 Switch callers from ProcessCacheLRU to MapCacheLRU
Change-Id: Iefe5c25e952079420b3241bc26efe1d2d086e6fb
2018-07-17 23:17:17 +00:00
Chad
bfbc44648d Revert "JobQueueGroup: Improve failed-to-insert error"
I made a useless commit.

This reverts commit a82cb5aa18.

Change-Id: I70328bcbcf9af5480c7a0b8e297ed41a463fd457
2018-02-23 08:05:13 +00:00
Chad Horohoe
a82cb5aa18 JobQueueGroup: Improve failed-to-insert error
This could possibly use LoggerFactory & friends to use them as
parameters, but either way let's sort the entries so they're
at least remotely capable of self-grouping.

Change-Id: Iaf435093d70add02e2f82e3037c64fec11870979
2018-02-23 01:57:11 +00:00
Aaron Schulz
df7ec19a12 Disallow job pushes from JobQueueGroup to bogus wikis
Bug: T171371
Change-Id: I03a5dbd18cf6b5bcacb3ec07cef9e0b051bc147c
2017-08-11 14:56:47 -07:00
Aaron Schulz
82e2c924e4 Remove "@author Aaron Schulz" annotations
Bug: T139301
Change-Id: Ib5248e8e27d60611c7373bce4b29dd5e85aa3489
2017-06-27 15:24:14 -07:00
Seb35
24842cfac0 Use AutoCommitUpdate instead of Database->onTransactionIdle
This is needed for deferred updates LinksDeletionUpdate and LinksUpdate, else
callbacks registered with onTransactionIdle prevent other transactions from
being executed, at least in this case.

Bug: T154425
Bug: T154438
Bug: T157679
Change-Id: Iecd396d584a62ac936cd963915339159467b44cd
2017-06-06 14:23:37 +02:00
Seb35
e00f7180b3 Prevent PHPDBG from issuing notices in unit tests
Although issuing PHP notices in unit tests is only a nice-to-have, the cause
is PHPDBG has no special treatment in JobQueueGroup at the contrary of the
PHP SAPI 'cli', and it may be desirable both 'command line SAPIs' have the
same behaviour.

This is the most apparent difference between cli and phpdbg but there could
be a dozen of other occurrences (see bug), potentially creating differences
in unit tests depending on the SAPI.

Bug: T162591
Change-Id: Idf9c14db72f1f768c5a17b49ed689a05922c57d3
2017-04-10 13:32:15 +02:00
Aaron Schulz
83e8d972fe Avoid triggering SiteConfiguration lookup in JobQueueGroup::push()
Just use $wgJobTypesExcludedFromDefaultQueue directly

Bug: T145819
Change-Id: I0d978cdbef40ced98880c1a7d7524fd2b7438bfc
2016-09-16 11:16:19 -07:00
Aaron Schulz
6f9a246d25 Make JobQueueGroup::push() update the queuesHaveJobs() cache
This makes it so jobs can run sooner with $wgJobRunRate

Bug: T145206
Change-Id: Ia7673f51312f90a696fa056bd3daf7f3f440782b
2016-09-11 04:57:12 -07:00
Aaron Schulz
16266edff3 Change "slave" => "replica DB" in /includes
Change-Id: Icb716219c9335ff8fa447b1733d04b71d9712bf9
2016-09-05 21:01:01 +00:00
Aaron Schulz
ea1f4e8a53 Add TTL_PROC_* constants for clarity
This makes the reason for the cache times more obvious

Change-Id: Ie75df6be77c513feeb6cff3417ccbc124a6a1141
2016-05-06 23:45:41 +00:00