Commit graph

769 commits

Author SHA1 Message Date
Reedy
11cc2d6993 Consistently use @deprecated since rather @deprecated
Change-Id: I301786f35cd6eb9c63fb72e0b64cffa9efd7b86b
2022-03-06 02:55:54 +00:00
Umherirrender
9efd9ca45e Add explicit casts between scalar types
* Some functions accept only string, cast ints and floats to string
* After preg_matches or explode() casts numbers to int to do maths
* Cast unix timestamps to int to do maths
* Cast return values from timestamp format function to int
* Cast bitwise operator to bool when needed as bool

* php internal functions like floor/round/ceil documented to return
  float, most cases the result is used as int, added casts

Found by phan strict checks

Change-Id: Icb2de32107f43817acc45fe296fb77acf65c1786
2022-03-01 18:19:33 +01:00
jenkins-bot
9a03e73ab2 Merge "jobqueue: JobQueueDB::jobFromRow cannot return null" 2022-02-26 21:08:46 +00:00
Umherirrender
b126dbe3f2 Fix various documentation related to null types
The functions returning null or the class property is set explict null

Found by phan strict checks

Change-Id: I4a271093fb6526564d8083a08249c64cb21f2453
2022-02-26 10:31:24 +01:00
Umherirrender
25188a6ae8 jobqueue: JobQueueDB::jobFromRow cannot return null
Found by phan strict checks

Change-Id: Ic7b2b8d262b8cdb5954222944e0d5a8613d6123d
2022-02-26 08:16:03 +00:00
jenkins-bot
eaf9570872 Merge "Make runJobs.php die peacefully if the DB server goes away" 2022-02-15 07:14:02 +00:00
jenkins-bot
7db80245b7 Merge "RefreshLinksJob: Restructure refreshlinks.* metrics and improve docs" 2022-02-10 11:36:53 +00:00
Tim Starling
6ce4044fe1 Make runJobs.php die peacefully if the DB server goes away
Connection loss with a failure to reconnect led to a variety of failure
modes, such as an assertion from getBindingHandle() when addQuotes() is
next called. And Maintenance::shutdown() was calling
commitPrimaryChanges() three times, each call leading to an assertion
due to the round stage being broken.

So, try to exit promptly if a DBConnectionError is caught, by
introducing a new "reached" category for fatal exceptions. Guard the
calls to commitPrimaryChanges(), and skip the call prior to shutdown()
since shutdown() calls it already.

Change-Id: I81ee9017a58adac674a9495802f6bd4bcc22ee61
2022-02-10 15:17:43 +11:00
Timo Tijhof
8eee142af2 RefreshLinksJob: Restructure refreshlinks.* metrics and improve docs
Change the metric names such that they do not overlap between
logically unrelated categories.

One non-overlapping metric for outcomes.

One misc metric for warnings and other unusual observations which
may overlap with outcomes.

Within outcomes, don't require metric queries to the names and/or
manually group them as positive or negative. Use a prefix to
distinguish between early exists that are bad (logged as failure and
will retry), and those that are considered good (no problem,
job superseded, no retry needed).

Bug: T301432
Change-Id: I2d5f3cc7cc2229cd910f341cf1f51d1e26be6e58
2022-02-10 00:37:27 +00:00
daniel
b1e3100feb RefreshLinksJob: assert proper title
If a RefreshLinksJob is created with an improper/invalid title,
executing the job later will fail with a PageAssertionException
while trying to create a WikiPage for that title. Better check
the title when constructing the job, so the offending code is on the
stack when it happens.

Bug: T293291
Depends-On: I7dcfac0eacdd5b22bdf443e88f8e6ddb883b92cc
Change-Id: I77a622591836873415f097453da01ca7e61c41be
2022-02-09 20:39:08 +01:00
Tim Starling
ca71e69fc6 Try not to discard Excimer timeout exceptions
Don't catch and discard exceptions from the RequestTimeout library,
except when the exception is properly handled and the code seems to be
trying to wrap things up.

In most cases the exception is rethrown. Ideally it should instead be
done by narrowing the catch, and this was feasible in a few cases. But
sometimes the exception being caught is an instance of the base class
(notably DateTime::__construct()). Often Exception is the root of the
hierarchy of exceptions being thrown and so is the obvious catch-all.

Notes on specific callers:

* In the case of ResourceLoader::respond(), exceptions were caught for API
  correctness, but processing continued. I added an outer try block for
  timeout handling so that termination would be more prompt.
* In LCStoreCDB the Exception being caught was Cdb\Exception not
  \Exception. I added an alias to avoid confusion.
* In ImageGallery I added a special exception class.
* In Message::__toString() the rationale for catching disappears
  in PHP 7.4.0+, so I added a PHP version check.
* In PoolCounterRedis, let the shutdown function do its thing, but
  rethrow the exception for logging.

Change-Id: I4c3770b9efc76a1ce42ed9f59329c36de04d657c
2022-02-02 16:27:44 +11:00
Umherirrender
68808e5832 Replace deprecated JobQueueGroup::singleton()
Change-Id: Icdb301d352d302f70fefba9b40df2368cb217fd2
2022-01-27 21:37:50 +01:00
Umherirrender
5173a4ac48 ActivityUpdateJob: Use primary key for db update on watchlist table
Bug: T204561
Change-Id: I4c86e112b5a55c45948eca2105c2f8b22e36f669
2022-01-27 04:00:15 +00:00
jenkins-bot
3cf9b2e64d Merge "DeletePage: add option to delete the associated talk page" 2022-01-16 03:00:37 +00:00
Timo Tijhof
4646ba7a59 jobqueue: Fix reference to $wgConf
Partially reverts 47adb6d65a (I1a691f01cd82e60). This isn't a configuration
variable.

Change-Id: I7ce7c5529db6a463efa080225f0923ab866177c0
2022-01-12 22:14:54 +00:00
Umherirrender
95852cb22b Explicit cast TS_UNIX to int for arithmetic operations and int args
When the called function has a doc of int, it should be cast to be
explicit here.
Also cast for arithmetic operations to be explicit about the number

Change-Id: I905b78dfb66e66443e0e3203488bab5b548db543
2022-01-11 20:13:13 +01:00
Daimona Eaytoy
f976775201 DeletePage: add option to delete the associated talk page
Currently this is implemented internally as a second method call. In the
future, it might be possible to optimize the implementation, e.g. to
reduce the amount of DB queries/jobs etc. without changing anything for
callers.

Since the implementation of e.g. the result getters is generic, a future
commit may add an option to delete subpages with relatively low effort.

The revision limit for big deletions is now using the total number of
revisions (base page + talk page). This is because both deletions would
happen in the same main transaction, and we presumably want to optimize
the deletion as a whole, not as two separated steps. This would become
even more important if the aforementioned improvements are ever
implemented. Note, the whole concept of "big deletion" might have been
superseded by batched deletions in the author's opinion, but as long as
such a limit exists, it should serve its purpose.

The current interpretation of the associated talk option is that the
request is validated, and an exception is raised if we cannot delete the
associated talk, as opposed to silently ignoring the parameter if the
talk cannot be deleted. The only exception to this is that it will not
fail hard if the talk page turns out not to exist by the time the
deletion is attempted, as that's hard to foresee due to race conditions.

Note that the summary for the talk deletion is prefixed with the
delete-talk-summary-prefix message. The main reason behind this is that
the delete UI can sometimes offer an auto-generated summary like "the
only contributor was XXX" or "the content was YYY", and since this may
not apply to the talk page as well, the log entry might look confusing.

Bug: T263209
Bug: T27471
Change-Id: Ife1f4e8258a69528dd1ce8fef8ae809761aa6f1d
2022-01-11 14:48:56 +01:00
jenkins-bot
057d6f4392 Merge "Refactor global variables to use MediaWikiServices instead" 2022-01-10 20:22:20 +00:00
TChin
47adb6d65a Refactor global variables to use MediaWikiServices instead
Automatically refactors wg prefixed globals to use MediaWikiServices config using Rector. Doesn't include files that set globals or files that fail CI.

Rector Gist: https://gist.github.com/tchin25/7cc54f6d23aedef010b22e4dfbead228

* This patch uses a modified source code rector library for our specific use case and the rector will have different effects without it.

A writeup for future reference is here: https://meta.wikimedia.org/wiki/User:TChin_(WMF)/Using_Rector_On_MediaWiki

Change-Id: I1a691f01cd82e60bf41207d32501edb4b9835e37
2022-01-10 13:55:53 -05:00
Tim Starling
682aad7557 Move LinksUpdate and LinksDeletionUpdate into the new namespace
Change-Id: I5cf7a08324d08aa89c23540222ba8eddc1ae2647
2022-01-04 15:35:57 +11:00
Siddharth VP
60faebb725 Fix typos in comments (I-J)
Change-Id: Icaea2b6665cfc3b811d94f70c93452237f5e72bf
2021-12-30 20:38:04 +05:30
Daimona Eaytoy
57bf1f67bc DeletePageJob: add fallback for page role
Might cause issues with queued jobs, possibly something else, as per
comments at I32679b7cacc638ec3e9dc5b8dfe9bcc794b22ecf.

Change-Id: I851aeb9af068d31a48f8f6a3ba98751374e88b6e
2021-12-07 02:10:08 +01:00
jenkins-bot
b085cd60cf Merge "Change return format of DeletePage again" 2021-12-06 22:20:15 +00:00
Daimona Eaytoy
0e558da7b8 Change return format of DeletePage again
In retrospect, I rushed the previous patch: we really need a way to tell
which deletion an ID belong to (and whether it was scheduled). So make
both result getters return an array with known keys that can be used
programmatically. Right now the class can only delete a single page, and
thus there's a single constant and this change is effectively a noop.

The deletionWasScheduled() method, introduced in 1.37, was
hard-deprecated out of an abundance of caution. There are no known uses
on codesearch, so it can probably just be removed in the next release.

Also reorder constructor params to DeletePage -- BacklinkCacheFactory is
a service injected by the factory, hence it shouldn't be grouped
together with value objects injected by the caller.

Change-Id: I32679b7cacc638ec3e9dc5b8dfe9bcc794b22ecf
2021-11-29 15:09:42 +00:00
Derick Alangi
11fbdc01bf jobqueue: Fix max no. of double redirect jobs in job queue
Introduce a constant to track the maximum number of double redirects
jobs in the job queue. Before we allow 10000 to spread in the code
base, we can just change the constant now and references of this value
will be changed.

Does this qualify to be a configuration or a regular constant will do?

Change-Id: I10c45a8c1cd5b3382edebf9a05f4b3f94f5327df
2021-11-26 19:59:41 +01:00
Reedy
7bf779524a Remove or replace usages of "sanity"
Bug: T254646
Change-Id: I2b120f0b9c9e1dc1a6c216bfefa3f2463efe1001
2021-11-19 23:19:42 +00:00
Thiemo Kreuz
41ec072d16 Add missing null to string casts when using rd_fragment and such
These columns are nullable, but the method doesn't accept null.

Found in Logstash.

Change-Id: I3eaafe31005c9319fdd03f8f4b54e7b51ef430f9
2021-11-02 07:14:32 +00:00
Paladox
bc931b1253 JobQueueRedis: Replace deprecated zSize with zCard
https://github.com/phpredis/phpredis/issues/1669

Change-Id: I15f1e398d45d699c3e683164e42ea6c0dc974d96
2021-10-26 01:28:27 +00:00
Umherirrender
66d4e6a3d0 Use correct default value when calling Status::getWikiText
Both parameter have a default of false, not null.

Change-Id: I5754895347db244eee155f8cf5965df5f4fd2d1a
2021-10-16 22:35:01 +02:00
jenkins-bot
92b52bfc07 Merge "jobqueue: Batch jobs that will end up in the default queue" 2021-10-06 17:32:37 +00:00
Daimona Eaytoy
ab6de6305b Use DeletePage in ApiDelete and DeletePageJob
Bug: T288758
Change-Id: I1cd9b18214df93bdcc0ff1e9dbd937368841e538
2021-10-05 13:25:36 +00:00
Amir Sarabadani
419c14b013 jobqueue: Batch jobs that will end up in the default queue
JobQueueGroup::get( $type ) returns a new object for each job type even
if it's the same class (default class). Also it builds different queues
and push them separelty a couple lines lower limiting the ability to
batch push them in the default job queue object.

Given that JobQueue cares about job types being the same as the class
type (which is important for JobQueueDB implementation), adding a config
option called typeAgnostic to ignore type mismatch in JobQueue
implementations that don't care about the type (like JobQueueEventBus)

Bug: T292048
Change-Id: I151f067ca94a985c816446b545921c387b083911
2021-10-04 09:04:11 +02:00
TChin
fd5915823f Update callers of deprecated BacklinkCache methods
Bug: T290015
Change-Id: I73a111d064f7778bbeb90fd4ea85d8bec201427b
2021-09-29 18:56:17 +00:00
Alexander Vorwerk
04dfdc3653 Hard deprecate User::setOption()
deprecated since 1.35

Bug: T277818
Change-Id: Ic251d624e5d6fa857aa92f9c5dd3df44714ac610
2021-09-26 17:18:54 +02:00
DannyS712
db49026745 ActivityUpdateJob: accept PageReference
For both LinkTarget and PageReference, just extract
namespace and dbkey and pass those in the array
of parameters to Job::__construct().

Allows a bunch of simplification to WatchedItemStore.

Bug: T291531
Change-Id: Id150d0c62af38d4b3d17e5698866127c6e04717e
2021-09-21 23:30:51 +00:00
DannyS712
a2b20b63d7 Emit deprecation warnings reading from $wgUser
With this patch deprecation warnings will be emitted
if $wgUser is accessed or written into. The only pattern
of usage still allowed is

$oldUser = $wgUser;
$wgUser = $newUser;
// Do something
$wgUser = $oldUser;

Once there is no deprecation warnings, we know that nothing
legitimately depends on $wgUser being set, so we can safely
remove the code that's still allowed as well.

Bug: T267861
Change-Id: Ia1c42b3a32acd0e2bb9b0e93f1dc3c82640dcb22
2021-09-15 20:17:04 -07:00
Derick Alangi
db43511e6a Title: Make use of BacklinkCacheFactory service
Change-Id: I48161585de6f329ec4037156234e0b07b3b837e6
2021-09-09 14:04:02 +01:00
James D. Forrester
3407458ea0 MWExceptionHandler: Rename rollbackMasterChangesAndLog to rollbackPrimaryChangesAndLog
Change-Id: I9a90b4f74eb65cd9e20ae9faa6d1949be96543c0
2021-09-03 17:36:34 -07:00
Umherirrender
2e4ee47c3d Cleanup mixed space/tab line indent
Change-Id: I833052a656b1ce419c0929f6f0514f2a33c2c4cc
2021-09-04 00:52:31 +02:00
James D. Forrester
8426aacd96 ILoadBalancer: Rename waitForMasterPos to waitForPrimaryPos
Hard-deprecating immediately as no known users in git outside of this repo.

Bug: T282894
Change-Id: I5fa312ae89660a9c6de3c24e13d48de92893e5b0
2021-09-03 20:11:05 +00:00
James D. Forrester
86f3270bda Tag various new 'primary' methods as @since 1.37
Change-Id: Icbbbc84154174da1bd9398990679857898f418d5
2021-09-02 11:58:19 -07:00
jenkins-bot
9957bc2f84 Merge "ILoadBalancer/ILBFactory: Rename commitMasterChanges() to commitPrimaryChanges()" 2021-09-02 18:50:43 +00:00
jenkins-bot
5f30d77f4d Merge "ILoadBalancer/ILBFactory: Rename beginMasterChanges() to beginPrimaryChanges()" 2021-09-02 18:30:07 +00:00
James D. Forrester
5ad7ca7eba ILoadBalancer/ILBFactory: Rename commitMasterChanges() to commitPrimaryChanges()
Bug: T282894
Change-Id: I0d80be56e683924254c4e38d05e1109ea74eeab5
2021-09-02 11:27:10 -07:00
James D. Forrester
577e1c3710 ILoadBalancer/ILBFactory: Rename beginMasterChanges() to beginPrimaryChanges()
Bug: T282894
Change-Id: I10e607215e6772c48670659719948f7135472a7f
2021-09-01 22:15:45 +00:00
Umherirrender
4d42b5e86e docs: Change wording master to primary in comments and log text
Bug: T254646
Change-Id: I5379dc79be60c99f0a30f74e5d624f81fe6f921b
2021-09-01 23:04:40 +02:00
Daimona Eaytoy
b9a82b3209 jobs: Remove unused param from DoubleRedirectJob::fixRedirects
It's an optional parameter at the end, so it can be removed without
breaking anything.

Change-Id: I987e678846f1a204efb7018575226299f716309e
2021-09-01 01:59:11 +00:00
jenkins-bot
ae8522b34c Merge "Same code style and formatting for exists-style SQL queries" 2021-08-24 17:44:17 +00:00
Umherirrender
240eaed9df Improve exception message for JobQueueGroup::assertValidJobs
Bug: T287623
Change-Id: I2108fe719a256c5d1d613055fdcb2890341bc0ee
2021-08-19 22:05:49 +02:00
Thiemo Kreuz
46f26dff03 Same code style and formatting for exists-style SQL queries
Queries that query a value "1" are expected to return either
that "1" (possibly as a string), or false. It's safe to simply
cast this to bool and use it as it is.

Queries for COUNT(*) are expected to return that number,
possibly as a string. It's not possible for such a query to
return false. And even if, casting to 0 is fine.

I found an existing code style where the table name and the
"1" are on the same line as the selectField() method name, and
applied it to all similar queries.

Change-Id: I9453196281871c03ef03f653f43762eb9284342f
2021-08-12 11:37:58 +02:00