While RefreshLinksJob is de-duplicated by page-id, it is possible
for two jobs to run for the same page ID if the second one was queued
after the first one started running. In that case they the newer
one must not be skipped or ignored because it will have newer
information to record to the database, but it also has no way
to stop the old one, and we can't run them concurrently.
Instead of letting the lock exception mark the job as error,
making it implicitly retry, do this more explicitly, which avoids
logspam.
Bug: T170596
Co-Authored-By: Aaron Schulz <aschulz@wikimedia.org>
Change-Id: Id2852d73d00daf83f72cf5ff778c638083f5fc73
In some functions MediaWikiServices::getInstance() was called twices or
in loops. Extract the variable to reduce calls.
Change-Id: I2705db11d7a9ea73efb9b5a5c40747ab0b3ea36f
The invalid UTF-8 could cause incorrect sorting of affected pages in
category lists on wikis using UCA collations. On my local testing
wiki, the generated cl_sortkey was just 0x30 regardless of the value
of cl_sortkey_prefix.
This doesn't fix existing bad data in the database. It will only be
updated when the affected page is edited (or null-edited).
The cl_timestamp field will also be updated when that happens, which
apparently may affect Wikinews' DynamicPageList extension, according
to comments on T27254. This is not easily avoidable.
Bug: T200623
Change-Id: I4baa9ea3c7f831ff3c9c51e6b8e5d66e7da42a91
This method returns the value used as cl_type for category links that
are "from" pages within the namespace, and is added to avoid duplication
of code across a few classes.
Change-Id: I4e55932a5a27858cfedb12009b455fcd02f9b5df
Uses new PHP 5.6 syntax like ...parameter unpacking and
calling anything looking like a callback to make the code more readable.
There are much more occurrences but this commit is intentionally limited
to an easily reviewable size.
Change-Id: Idcec077ef3fdf029b632cceafd0150851ad723e3
Adds a maintenance script to populate the field, has that be
automatically run during update.php, and drops the no-longer-needed
default value on the column (where possible: mssql has some sort of
constraint thing going on that I have no idea how it works).
Bug: T59176
Change-Id: I971edf013a1a39466aca3b6e34c915cb24fd3aa7
The hook handlers are likely to write to secondary databases, in which
case it is better to wrap the callback in its own transaction round.
This lowers the chance of pending write warnings happening in
runMasterTransactionIdleCallbacks() as well as DBTransactionError
exceptions in LBFactory due to recursion during commit.
Bug: T191282
Bug: T193668
Change-Id: Ie207ca312888b6bb076f783d41f05b701f70a52e
This is a stop-gap solution for the problem described in T190178.
A better solution would probably include changing the behavior
of LegacyLogger.
Bug: T190178
Change-Id: I433be04b8ee725becd174e567270aa674d2661df
The replaces the hacky use of onTransactionIdle(), which no longer runs
immediately in explicit transaction rounds since d4c31cf841.
Also clarified TransactionRoundDefiningUpdate comment about rounds.
Change-Id: Ie17eacdcaea4e47019cc94e1c7beed9d7fec5cf2
Storing the user name or IP in every row in large tables like revision
and logging takes up space and makes operations on these tables slower.
This patch begins the process of moving those into one "actor" table
which other tables can reference with a single integer field.
A subsequent patch will remove the old columns.
Bug: T167246
Depends-On: I9293fd6e0f958d87e52965de925046f1bb8f8a50
Change-Id: I8d825eb02c69cc66d90bd41325133fd3f99f0226
This pattern is already used elsewhere and seems like a more efficient
way to acquire locks in a non-blocking way.
Change-Id: Idb369e7cb03b793d5f8295e956fecd8d1f849e17
* Mark method visibility
* Remove unused code/methods
* Use WAN cache process cache in pagesInNs()
* Remove "m" prefix from SiteStatsInit fields
* Avoid use of $wg variables
Change-Id: Iab4001f02c9b2e8667ca4bac033fd4f6ef272148
* Remove some creation dates, they are not protected by GPL
* Remove duplicate @defgroup API
* Remove @ingroup from some @file doc comments on class files. It is not
useful to list class files alongside classes in the doxygen module menu.
Add @ingroup to some more class files that had @ingroup on their file,
that was probably the author's intent.
* In PackedOverlayImageGallery, use the file comment as a class comment
* Don't put @defgroup and @file in the same comment. @defgroup makes the
whole doc comment describe the group.
* Instead of putting AnsiTermColorer in two groups, use hierarchical
groups.
Change-Id: If54f6e0b2bc1ea6de42045885cf836ee67b8e961
This allows scheduling of updates that need to start their own
transaction round. Specifically for cases where the ability to
commit early is not enough (which is already possible via LBFactory
getEmptyTransactionTicket and commitAndWaitForReplication).
Change-Id: I0910587b61c8ddf825f91e92c2f93582cc7ebd80
From 1% of lines to 12% in deferred/.
From 6% of lines to 68% in DeferredUpdates.php.
* Adding relevant @covers tags to existing tests.
* Add coverage for MWCallableUpdate.
* Add coverage for MergeableUpdate.
Also:
* Make MergeableUpdate extend DeferrableUpdate.
1. Because PHPUnit doesn't support implementing multiple interfaces
in a mock, and would make the mock fail the typehint at run-time.
2. It DeferredUpdates doesn't support having a MergeableUpdate that isn't
itself a DeferrableUpdate given the only way to reach that code is past
methods that are type-hinted with DeferrableUpdate.
3. Making the interface extend DeferrableUpdate helps produce better and
earlier errors. Instead of run-time error:
> Argument 1 passed to addUpdate() must implement interface DeferrableUpdate
> instance of MergeableUpdate given
We get:
> Fatal error: Class .. contains 1 abstract method and must therefore be
> declared abstract or implement the remaining methods (doUpdate)
Change-Id: Ie384bf849a96bb37dc3e4a4154da2b02889e9fc8
We already exclude external (mostly Wikidata) actions from counting on
Special:ActiveUsers as of 56524c05. Should be consistent.
Change-Id: Ib69a614a65246a0f4f47365751f9bef130f47475
Xenon shows a significant amount of time is spent individually fetching
the content model for titles being purged. It's probably not a huge
improvement but lets fetch them all in one go instead of lazily as
needed.
Change-Id: I71af6f13c84e740c1367617d7a51210501363b33
Having such comments is worse than not having them. They add zero
information. But you must read the text to understand there is
nothing you don't already know from the class and the method name.
This is similar to I994d11e. Even more trivial, because this here is
about comments that don't say anything but "constructor".
Change-Id: I474dcdb5997bea3aafd11c0760ee072dfaff124c
This avoids putting updates in the PRESEND queue at a point where they
may never get run later in the request. The peculiarity lead to a
regression in 24842cfac.
Move "enqueue" logic to runUpdate() to simplify execute(). If job
insertion batching is strongly desired for a class, then it can use
MergeableUpdate.
Removed unused "update" field in $executeContext.
Bug: T168723
Change-Id: I40d16f6cd0adc8583797b99d859b76a836d362a8
Partially revert I61dc536 that broke phrase search support.
Fix phrase search by making explicit that there are two
kind of legalSearchChars() usecases :
- the chars allowed to be part of the search query (including special
syntax chars such as " and *). Used by SearchDatabase::filter() to
cleanup the whole query string (the default).
- the chars allowed to be part of a search term (excluding special
syntax chars) Used by search engine implementaions when parsing with
a regex.
For future reference:
Originally this distinction was made "explicit" by calling directly
SearchEngine::legalSearchChars() during the parsing stage. This was
broken by Iaabc10c by enabling inheritance.
This patch adds a new optional param to legalSearchChars to make this
more explicit.
Also remove the function I introduced in I61dc536 (I wrongly assumed
that the disctinction made between legalSearchChars usecases was due
to a difference in behavior between indexing and searching).
Added more tests to prevent this from happening in the future.
Bug: T167798
Change-Id: Ibdc796bb2881a2ed8194099d8c9f491980010f0f
I think the bug was introduced during a cleanup in Iaabc10c.
I don't think that " should be part of the legalSearchChars at query
time, it seems to break the regex.
The strategy here is to distinguish legalSearchChars used query time vs
the ones used at index time by introducing:
SearchEngine::legalSearchCharsForUpdate()
Bug: T167798
Change-Id: I61dc53665e26d3c6c48caed78dd3bbde9a33def7
Previously, tryOpportunisticExecute() tried to nest transaction rounds,
which would fail. Added LBFactory::hasTransactionRound() as needed.
Also cleaned up some unqualified class names in callbacks and set the
PRESEND flag for the JobQueueDB AutoCommitUpdate callback. Use the
proper getMasterDB() method while at it. These follow up 24842cfac.
Bug: T154425
Change-Id: Ib1d38f68bd217903d1a7d46fb15b7d7d9620daa6
In the postprocessing, some jobs can be executed but given the deferred
updates were already "closed", any new DeferredUpdate were directly called
(as explained by Krinkle on T165714), and the transactions opened by
classical jobs are badly mixed with transactions (directly) executed by
DeferredUpdates jobs, issuing a DBError, avoiding the job, which stays
in a 'claimed' status even if failed.
Quite similarly, some DeferredUpdates callables use JobQueueGroup::lazyPush
so it is needed to really push the generated jobs.
This change removes the run-immediately-deferred-updates behaviour even
in the post-connection shutdown, and given there is a call to
DeferredUpdates::doUpdates in JobRunner::execute it is not necessary to
add another one and hence execution of Web jobs is more similar to execution
of CLI jobs. In the same spirit to reconcile Web jobs and CLI jobs, the
call to JobQueueGroup::pushLazyJobs is done in JobRunner::execute.
Bug: T165714
Bug: T100085
Change-Id: I721e7167eca5b0b6227234fe516005243ab22388
