When fetching a key from an array, PHP converts numeric strings to
integers. This led to an incorrect category update query.
So:
* In LinksTable subclasses, cast all string-like array keys to string.
* Add a unit test which confirms that no integers leak into link IDs.
* Add a fully integrated regression test for T301433.
The integration test and 7 of the link ID cases are confirmed to fail
if Amir's patch is reverted.
Bug: T301433
Change-Id: I8d19443607121b3efcafb82096bcff18c41035df
* In PagePropsTable, compare values after converting to string. In the
old code this was implicitly done by array_diff_assoc().
* Also convert to string before inserting, since
ParserOutput::setPageProperty() is documented as allowing null, but
the field is not nullable, so the query failed.
* Add a regression test which confirms that no write queries are done on
the second invocation of LinksUpdate.
Bug: T299662
Change-Id: I3f7d5800bf27a47cb57be80bdba13792fe3caae6
Running cat_pages + 1 and cat_pages - 1 for the same rows/categories is
uneeded.
Cannot skip the whole update on move, because by use of parser functions
there could be new categories after cross namespace page moves.
Follow-Up: Ifaa4051a736b01cda82fbd890325e0751ce041f8
Change-Id: I188d0224f1a1d43746f657159e0ed8c80b8646cd
Since LinksUpdate was already run after a page move, it was just a
matter of propagating a move flag down to it, and resolving the
differences between the move case and the edit case.
This should be faster since it avoids writing to the links tables twice
on a page move.
Change-Id: Ifaa4051a736b01cda82fbd890325e0751ce041f8
* In LinksUpdate, put each table in its own class, so that the
special code for each table has a more elegant place to go.
* Use generators to abstract and flatten out the underlying ParserOutput
data structures, without requiring the data to be copied. This allows
the incremental logic which was previously duplicated for each table,
i.e. computing insertions and deletions from the old and new state,
to be factored out. The per-table batch classes act as container
abstractions for the underlying data.
Bug: T263437
Depends-On: I472f4a023969bfde6298eb56112c16d2ae842199
Change-Id: Id40be19a8caa7b4669911927964cfd6755c25238
Each table at least needs an added, deleted and unmodified link in order
to fully exercise the incremental update code.
Disable INSERT IGNORE during testing, since this can hide an omitted
delete query.
Change-Id: Iaa9f32ea5171da4f983379d7ff313bfdf1cb0b2f
The ::getProperty() naming is too generic and doesn't clearly indicate
that these are "page properties" (which have their own table in the DB).
As part of refactoring a clean API out of ParserOutput which can be used
by Parsoid, clean up the naming here.
Soft-deprecation in this patch, there are a handful of external users
which need to be cleaned up before we hard-deprecate.
Bug: T287216
Change-Id: Ie963eea5aa0f0e984ced7c4dfa0fd65d57313cfa
Results in passing a user where previously the fallback
to $wgUser was being used, mostly in tests.
Bug: T255507
Change-Id: Iabe24315b23c0ad1272353186425e71974528d23
This allows deferred update tasks to be constructed with a PageIdentity
rather than a Title instance.
Bug: T278459
Change-Id: Ifa2888fdef9ecc97f5bf09ae0b3fa7be48a0e9ef
JobRunner catches all exceptions and hides them in the status array,
meaning that it is not obvious when a job fails during a test case.
So, introduce MediaWikiIntegrationTestCase::runJobs(), which runs jobs
and asserts various things about the returned status array.
Depends-On: I4f4790c5d16a0767790eeff202e0be8fcdaeda93
Depends-On: I118f9e3f8950fd82d7b02baed6705b29fd6ab7d5
Change-Id: I63603aa158f77df4b40add096cb248f3b24979f4
This logic is not needed to run on every PHP process and was making
it difficult to run offline maintenance scripts without additional
complexity based on Maintenance::getDbType and DB_NONE.
Instead of skipping this only for DB_NONE, and establishing a pattern
that may spread to other ad-hoc places throughout the codebase, instead
remove this entirely from the eager set up code for all PHP processes
and move it to the service wiring and dependency injection.
That way, it naturally doesn't happen until and unless the DB service
is actually called upon. Scripts and entry point that need to disable
the DB service, can continue to use
MediaWikiServices::disableStorageBackend.
== Impact on SiteStatsUpdate ==
With wgCommandLineMode no longer being read at run-time from a global,
but in service wiring, this means SiteStatsUpdateTest can't change
the behaviour between CLI-like and Web-like, unless it e.g. resets
the 'DBLoadBalancerFactory' service first. Unfortunately, while most
any reset is supported, a reset of the 'DBLoadBalancerFactory' would
be unsupported as that would lose the temporary db clone context and
such, bringing us to either the developer's live db, or a broken set
up altogether. If there is a strong need for toggling oppertunistic
updates off and on at run-time, this could be supported in the
DeferredUpdates class perhaps, but we already have numerous methods
there (incl db begin/commit being a good proxy already), which this
test already used, so for now I've just removed the extra assertion
for this as it wasn't essential to that test.
Bug: T228895
Bug: T238436
Change-Id: Icf29bc484c155f52b6d8f61e5902233a15ba0c6d
Ended up using
grep -Prl '\->setMethods\(' . | xargs sed -r -i 's/setMethods\(/onlyMethods\(/g'
special-casing setMethods( null ) -> onlyMethods( [] )
and then manual fix of failing test (from PS2 onwards).
Bug: T278010
Change-Id: I012dca7ae774bb430c1c44d50991ba0b633353f1
This is no longer useful. The schema update it supports
has been removed: patch-pp_sortkey.sql/(I76b4235)
Bug: T273978
Change-Id: Ida469a5401f98a028b7a69242105dc1595af3613
Follow-up ba6490aa1e.
That patch worked around $executeContext iteration errors due to the
lack of recursion support in doUpdates()/handleUpdateQueue(). Errors
were triggered by MediaWiki::schedulePostSendJobs() enqueueing deferred
updates that invoke JobRunner::run(), which, in turn, invoke doUpdates()
for each job via JobRunner:: doExecuteJob(). The doUpdates() method was
changed to operate on the sub-queue for the in-progress DeferrableUpdate.
Further improve the recursion logic:
* Use classes for the scope stack and scope queues.
* Put *all* updates added during a DeferrableUpdate::doUpdate() call into
the subqueue, regardless of "defer until" stage or whether it implements
MergeableUpdate. Now, doUpdate() can run *everything* it enqueued instead
of a non-obvious subset. Note that the TransactionRoundDefiningUpdate
that invokes JobRunner was already a POSTSEND update before ba6490aa1eb;
the only effect of this change is that MergeableUpdate instances from jobs
will once again run in doExecuteJob().
* Make recursive DeferredUpdates::doUpdate() calls error out immediately
unless the DeferrableUpdate responsible is a TransactionRoundAwareUpdate
with the TRX_ROUND_ABSENT flag. This covers the schedulePostSendJobs()
scenario and only prohibits insane call patterns. Failing early avoids
the risk of handleUpdateQueue() dropping all the updates due to the same
DBTransactionError error in DeferredUpdates::attemptUpdate().
* Avoid recursion loop in tryOpportunisticExecute() when JobQueueDB is in
use and a large number of tasks are pending. This happened due to methods
like onTransactionPreCommitOrIdle() being used within JobQueueDB.
Mark DeferredUpdates::doUpdates()/tryOpportunisticExecute() as @internal
and create a Maintenance::shutdown() method to avoid a direct call in the
doMaintenance.php file.
Bug: T249069
Bug: T268840
Change-Id: Ib369f0e74243a48ababdb9cd83b155c9a0f5e741
The name change happened some time ago, and I think its
about time to start using the name name!
(Done with a find and replace)
My personal motivation for doing this is that I have started
trying out vscode as an IDE for mediawiki development, and
right now it doesn't appear to handle php aliases very well
or at all.
Change-Id: I412235d91ae26e4c1c6a62e0dbb7e7cf3c5ed4a6
Done with `composer fix` and suppressing the rest (i.e. sniffs for
global variables, which for core should be suppressed anyway).
Additionally, add `-p` to `phpcbf`, as otherwise it just seems stuck.
Change-Id: Ide8d6cdd083655891b6d654e78440fbda81ab2bc
Sometimes, an edit is done with a Title object that has gone
out of sync with the database after a page move. In this case,
we should re-load the current page ID from the database,
instead of failing the update hard.
Bug: T246720
Bug: T204793
Bug: T221763
Bug: T225366
Change-Id: If7701205ec2bf4d4495349d3e67cf53d32ee8357
This is a re-submit of 35da1bbd7c, which was accidentally merged before
CR (and reverted with aa4da3c2e8).
The purge() method handles purging of both file cache and CDN, using
a PRESEND deferred update. This avoids code duplication and missing
file cache purge calls.
Also:
* Migrate HTMLCacheUpdate callers to just directly using HTMLCacheUpdateJob
* Add HtmlFileCacheUpdate class and defer such updates just like with CDN
* Simplify HTMLCacheUpdate constructor parameters
* Remove BacklinkCache::clear() calls which do nothing since the backlink
query does not actually happen until the job runs
Bug: T230025
Change-Id: Ic1005e70e2c22d5bd1ca36dcdb618108ebe290f3
The call to `$this->updater->getSecondaryDataUpdates()` probably started
an implicit transaction, and a sub-update may have tried to register a
callback to watch for a rollback which would make that implicit
transaction non-empty.
Bug: T248003
Change-Id: Ib9db8ec9c43c8b2871f283733ed6a05d2dec6dd1
The code is appropriately logging when it encounters an exception.
However, the CI interprets that as an actual exception being thrown.
Setting the logger to null allows the CI to complete.
Bug: T248189
Change-Id: I9ce7c451b9951892e509afa048213e7691498e47
The class was already documented as "given a list of URLs or Title
instances", this makes that work.
Title objects will have ->getCdnUrls() called when the update is
resolved, which avoids problems like those encountered in T240083 where
that was being called too early.
Bug: T240083
Change-Id: I30b29a7359a8f393fb19ffc199211a421d3ea4d9
This class implements EnqueueableDataUpdate and can be pushed as a
job if it fails to run via DeferredUpdates.
Unlike a1f7fd3ada, make RefreshSecondaryDataUpdate skip failing
updates in doUpdate(). Instead of throwing the first exception from
any update, log any exceptions that occur and try all the other
updates. The first error will be re-thrown afterwards.
Also, make sure that each DataUpdate still has outer transaction
scope. This property is documented at mediawiki.org and should not
be changed.
Add integration tests for RefreshSecondaryDataUpdateTest.
Bug: T218456
Bug: T206283
Change-Id: I7c6554a4d4cd76dfe7cd2967afe30b3aa1069fcb
Add public, protected or private to function missing a visibility
Enable the tests folder for the phpcs sniff
Change-Id: Ibefce76ea9984c47e08c94889ea2eafca7565e2c
