Commit graph

336 commits

Author SHA1 Message Date
Aaron Schulz
da9e4b52a5 rdbms: Add multi-statement query support to Database
Add Database::queryMulti(), which will execute an array of
queries as a batch with minimal roundtrips.

SQLite fallbacks to looping through each statement and
invoking doQuery().

Add QueryStatus class to reduce complexity in Database.

Rewrite doQuery() as doSingleStatementQuery().

Change-Id: I3d51083e36ab06fcc1d94558e51b38e106f71bb9
2022-06-09 11:45:38 +10:00
jenkins-bot
b494330aa7 Merge "ParserCache: always use JSON" 2022-06-07 14:12:29 +00:00
daniel
697f28df32 ParserCache: always use JSON
When JSON support was introduced into ParserCache in 1.36, it was
controlled by a feature flag, $wgParserCacheUseJson. The feature flag
was "born deprecated" in 1.36. It can now be removed.

This means that ParserCache will always store entries as JSON.
Support for reading old non-JSON entries remains intact.
This is needed when updating wikis from a version older than 1.36
to the current version.

Change-Id: Id04e42bfb458d98414bac50e0d6c505e8878e5c0
2022-06-07 15:19:45 +02:00
Reedy
41c42d5435 Tests: Cleanup some unnecessary nested function calls
Replace ->will( ->return with ->willReturn(

Change-Id: Ia2dfafa03cac8169d86d6fa5a30b73bfad1fe9fa
2022-06-06 01:02:34 +01:00
Umherirrender
cada8a827f RollbackPage: Include patrol status of revisions with same timestamp
In case two revision have the same timestamp, the higher rev_id is the
later revision and needs to be patrolled as well.

This also fix the tests, where the clock was faked to get higher
timestamps on the revisions

The WHERE changed from:
rc_cur_id = 2 AND (rc_timestamp > '20220603122124') AND rc_actor = 10
to
rc_cur_id = 2 AND ((rc_timestamp > '20220603122124') OR (rc_timestamp =
'20220603122124' AND (rc_this_oldid > 2))) AND rc_actor = 10

Bug: T309817
Bug: T304766
Bug: T260798
Follow-Up: I66bb6575dad4a970339e93b91fe486727111092a
Change-Id: I3cfcca4f0405f1d701c2883109c53b15854d6e64
2022-06-03 20:49:51 +00:00
Subramanya Sastry
d3c3abcd22 Unbreak ParsoidHTMLHelperTest: Fix call to ParsoidHTMLHelper constructor
Change-Id: I17af4240ac81c6234c213c7a92465354a9942201
2022-06-02 17:26:24 -05:00
Matěj Suchánek
5e5c879be2 RollbackPage: Make rollback not overwrite manual RC patrol status
When performing a rollback, we marks the reverted edits as
patrolled (on UseRCPatrol wikis) and also pretend they were by a bot
(discussion at T304766).

The way it is marked as patrolled is by setting the patrol status to
autopatrolled. But, if (one of) the edits was already patrolled before
the rollback, the manual patrol status should not be overwritten.

* Fix this.
* Also improve query performance by setting via rc_id instead of
  through more complex conditional clauses in the update query
  (should help with T260798).
* Test as much of this patrolling stuff as possible.

Bug: T304766
Bug: T260798
Change-Id: I66bb6575dad4a970339e93b91fe486727111092a
2022-06-01 09:36:55 +02:00
daniel
7f8e3c5360 Docs: remove references to DefaultSettings.php
Bug: T300129
Change-Id: I1e83a0dde6235bed91d28d7d43691c6a5d979f2f
2022-05-30 21:27:09 +02:00
Derick Alangi
141b42c7ca Rest: Collect stats on Cache & Stash usage
As a means of understanding the usage of the stash FEAT for
/page/html & /revision/html endpoints used by VE extension,
this patch introduces the collection of stats using the
StatsDataFactory.

Bug: T309017
Change-Id: I4e17d50e79da263637bdd55ab62e993df441fe38
2022-05-30 09:51:55 +01:00
Umherirrender
f01a391f0d tests: Fix types in documentation by adding use statements
Some types used in @param/@return are not correct,
because the class is not used with namespace

Change-Id: Ic2643687378a6addabc2c9cd402224e442257f86
2022-05-29 07:11:15 +00:00
Umherirrender
b0398654c5 tests: Do not use class alias in @covers
Also fix case for the XMLTypeCheck test

Change-Id: I809510c8085a36e20fd0eefb5e77d2671b3148f2
2022-05-28 00:37:43 +02:00
Umherirrender
770f905900 tests: Use namespaced IDatabase class
Change-Id: I7171ff26faee00d9eaabc33c2f3d91049ea0b40d
2022-05-28 00:09:55 +02:00
Derick Alangi
d62f97d5e0 Rest: Return different eTags for different output modes
This patch enables the response from PageHTMLHandler and
RevisionHTMLHandler to have different eTags for different
output modes and varying flavors.

Before, the only difference we got was when the stashing
option is set or not, but we need more flavors.

Bug: T308744
Change-Id: I2e9679e46a31955a2106a52af4eb612b32799c8c
2022-05-25 11:15:47 +00:00
Derick Alangi
13f6ec9e1b Rest: Migrate parsoid stashing logic from RESTbase
Add stash option to /page/html & /revision/html endpoints.
When this option is set, the PageBundle returned by Parsoid is
stashed and an etag is returned that can later be used to
make use of the stashed PageBundle.

The stash is for now backed by the BagOStuff returned by
ObjectCache::getLocalClusterInstance().

This patch adds additional data to the ParserOutput stored in ParserCache.
Old entries lacking that data will be ignored.

Bug: T267990
Co-Authored-by: Nikki <nnikkhoui@wikimedia.org>
Change-Id: Id35f1423a69e3ff63e4f9883b3f7e3f9521d81d5
2022-05-23 17:28:29 +01:00
jenkins-bot
6411cc09c5 Merge "Allow REST API handlers to require csrf-safe session providers" 2022-05-23 10:06:52 +00:00
Bill Pirkle
7295100773 Allow REST API handlers to require csrf-safe session providers
Bug: T305043
Depends-On: Ic7c1b19b86e8a151e2d42aaec00ef0e89db77f08
Change-Id: Ic6bd48b400ecd839ef99b518ef955781470cd05c
2022-05-20 16:52:54 +00:00
jenkins-bot
9efc36c652 Merge "Fix visibility of setUp methods in tests" 2022-05-17 14:15:24 +00:00
Thiemo Kreuz
29aca66975 Fix visibility of setUp methods in tests
Change-Id: Iefc05b4f4a9c258a385a22fc8d87b29648900e44
2022-05-17 14:11:22 +02:00
Derick Alangi
1618bbd671 Add data-parsoid data to ParserOutput for caching
NOTE: This changes the HTML returned by the endpoint!
It will now include the id="mwXYZ" attributes needed to
later map to data-parsoid entries.

Bug: T268205
Change-Id: I0a29434b996cc289eb67083e62bd6f1ad750cb4d
2022-05-16 15:06:15 +00:00
jenkins-bot
fa97f5405c Merge "Make SpecialPageAliasTest more strict" 2022-05-15 10:37:25 +00:00
Amir Sarabadani
244127fc5d RestrictionStore: Add support for templatelinks migration
For checking cascade protection.

Bug: T308207
Change-Id: Ia7a3ec84e8de11a77f3addc05e3de0bab2cb51a6
2022-05-13 23:45:24 +00:00
Thiemo Kreuz
5d08c94ef6 Make SpecialPageAliasTest more strict
The underscores are not strictly necessary, but appear to be good
practice. The vast majority of special page aliases we have already
use underscores (94% according to my test).

I made this a warning in the test for now, not an error.

Change-Id: I18827eb795f84bdb7895793d00bc3c895abebc7a
2022-05-13 18:06:00 +02:00
Umherirrender
d79fd02d4b Use injection for LinksMigration on LinkBatch, action, api, special page
Even the service does not long stay in that classes,
it should be injected to avoid global state

Bug: T304780
Change-Id: Ib488037f5a6966ab61042ed3cd889ddc50f1ba8e
2022-05-10 20:19:58 +00:00
Derick Alangi
362380f67d Avoid accessing MW services in data provider
Change-Id: Ifce325abc72ac7c05636c350c82334a166009f17
2022-05-09 16:14:05 +00:00
Derick Alangi
4c1107fb47
phpunit: Add tests for SearchSuggestionSet::fromTitles()
Change-Id: Icda5f3ce5d804c11414220cbe035648cfa7017b1
2022-05-06 01:39:28 +01:00
Aryeh Gregor
4851f512b3 IResultWrapper::next() now returns void
As required by the Iterator interface. In PHP 8.1 we can suppress the
warning with #[\ReturnTypeWillChange], but it's wrong and should change
anyway.

Depends-On: I576109808755b054e7876556f244ee8eafd12f9d
Change-Id: Ibaa7b4c81d8f114783db45c9100200ba14547c8f
2022-05-02 10:43:51 +03:00
Aryeh Gregor
62e5146a13 GlobalIdGeneratorTest is not a unit test
It calls the Shell class, which uses MediaWikiServices. Why was this not
causing test failures on CI?

Change-Id: Id953c1425659ede158034988dd652b59a0e9811b
2022-04-24 15:49:43 +03:00
Thiemo Kreuz
312566d156 Use more specific assertions in HTMLTitleTextField test
I would like to argue that it's useful to know if an error message is
returned, and if it's the correct one.

Bug: T306568
Change-Id: I865fa4c8a5b2d2d88d26c190352c852e63913a56
2022-04-23 22:10:26 +02:00
Umherirrender
2909d06a08 Use new namespace for revision related classes
All revision related classes are namespaced MediaWiki\Revision
instead of MediaWiki\Storage since 1.32. The old namespaced
class names are deprecated and only kept for backwards-compatibility.

Bug: T305784
Change-Id: I34e492d84d9fc4bc78481667202716d93b3c43cb
2022-04-14 23:03:43 +02:00
jenkins-bot
559af34cc9 Merge "Promote experimental revision rest endpoints to official" 2022-04-14 19:13:14 +00:00
jenkins-bot
bd8691818a Merge "TempUser infrastructure and services" 2022-04-14 15:33:50 +00:00
jenkins-bot
d2e7be31d8 Merge "rdbms: make automatic connection recovery more robust" 2022-04-14 01:33:46 +00:00
Aaron Schulz
db36853718 rdbms: make automatic connection recovery more robust
Rename canRecoverFromDisconnect() in order to better describe
its function. Make it use the transaction ID and query walltime
as arguments and return an ERR_* class constant instead of a bool.
Avoid retries of slow queries that yield lost connection errors.

Track session state errors caused by the loss of named locks or
temp tables (e.g. during connection loss). Such errors will prevent
further queries except for rollback() and flushSession(), which must
be issued to resolve the error.

Add flushPrimarySessions() methods to LBFactory/LoadBalancer
and use it in places where session state loss is meant to be
safely aknowledged.

Change-Id: I60532f86e629c83b357d4832d1963eca17752944
2022-04-14 11:09:31 +10:00
Amir Sarabadani
280c2ed0d3 rdbms: Fold MaintainableDBConnRef into DBConnRef
We really don't need this complexity and it prevents us from improving
connection management.

MaintainableDatabase should stay but the connection ref shouldn't.

Bug: T255493
Change-Id: I867301dc7fa07cac298f8faba9cf82ca4617f50e
2022-04-14 01:57:11 +02:00
Tim Starling
e8dbf5f80c TempUser infrastructure and services
Add services and utilities for automatic creation of temporary user
accounts on page save, in order to avoid exposing the user's IP
address.

* Add $wgAutoCreateTempUser, for configuring the system
* Add TempUserConfig service, which interprets the config.
* Add TempUserCreator service, which creates users during page save as
  requested by EditPage. With proxy methods to TempUserConfig for
  convenience.
* Add table user_autocreate_serial. Table creation is necessary before
  the feature is enabled but is not necessary before deployment of this
  commit.

Bug: T300263
Change-Id: Ib14a352490fc42039106523118e8d021844e3dfb
2022-04-14 09:23:55 +10:00
Bill Pirkle
fc7acbee64 Promote experimental revision rest endpoints to official
These three endpoints have been experimental for many months:
  revision/{id}
  revision/{id}/html
  revision/{id}/with_html
Promote them to officially released. This completes the
basic "revision" endpoint support, and helps clear out
the coreDevelopmentRoutes.json file for unrelated
experiments.

This also modifies the existing revision/{id}/bare endpoint
response, which previously pointed callers to the experimental
endpoint for html. It now points callers to the official one.

Bug: T305506
Change-Id: Iee8d1723e98dd3e3e389a0514dde28799914b2fd
2022-04-13 11:30:53 -05:00
Alexander Vorwerk
62a70ec7c7 Use new namespace for revision related classes
All revision related classes are namespaced MediaWiki\Revision
instead of MediaWiki\Storage since 1.32. The old namespaced
class names are deprecated and only kept for backwards-compatibility.

Bug: T305784
Change-Id: Ia0030814ce2176d06e2898acffe533d31633fccb
2022-04-09 20:22:36 +02:00
Tim Starling
1b0b40016c Fix UserFactory::newAnonymous() performance
When newAnonymous() was called with a specified IP address, it called
newFromName(), leading to a DB query on User::load() as if the IP
address could be in the database.

Instead, use User::setName() which is documented to work for this
purpose and indeed appears to do so.

Change-Id: I8e8be719c89b1d1e4ae2cf9063db68b929bb3651
2022-03-30 17:31:38 +11:00
jenkins-bot
b9a52d28b3 Merge "Improve error messages in EditFilterMergedContentHookConstraint" 2022-03-26 06:23:15 +00:00
Daimona Eaytoy
2012fe3b2b Improve error messages in EditFilterMergedContentHookConstraint
Use multiple error boxes instead of a wikitext list of error messages,
as the latter has several issues (as reported on phab). Also, remove the
edit-error-* messages as they don't seem to add any value. Also, don't
use the passed-in context, inject what the class actually needs instead
(but keep the context for the hook).

Bug: T303060
Bug: T230229
Change-Id: I2806e57826e7ab062b45dd1d9972b05fd6baeafb
2022-03-22 14:53:02 +01:00
Tim Starling
1155648ecf Fix core newtalk 304 suppression and update deferral
Deferring newtalk updates caused the skin to show the notification on
the same user talk page view request in which it is cleared. Even if the
user refreshed the page, it would still show the notification since a
304 response was given.

So, move the newtalk part of WatchlistManager
::clearTitleUserNotifications down to TalkPageNotificationManager, so
that the in-process updates can be done immediately, with the DB updates
being deferred. TalkPageNotificationManager is now responsible for
deferring the relevant parts.

Originally (2002) the newtalk status was set by calling
User::saveSettings(), which had the side-effect of updating
user_touched. Restore this behaviour, which is fast now due to the
"quick touched" cache. Also update the touched timestamp when the
notification is cleared.

Change-Id: Ic3c14bca7bed04f46afffa971fa0a57224c666ce
2022-03-18 16:52:12 +11:00
Ladsgroup
31c1ca8658 Revert "rdbms: make automatic connection recovery apply to more cases"
This reverts commit 4cac31de4e.

Reason for revert: Blocking the train, reverting the chain.

Change-Id: I7f275b3a25379c6f3256e90947c8eed4b232c0f4
2022-03-17 20:11:10 +01:00
Ladsgroup
27ac381fd2 Revert "rdbms: Followups to automatic connection recovery patch"
This reverts commit 89cfcb3cfd.

Reason for revert: Blocking the train, reverting the change.

Change-Id: Ibb5611491f50dea5ceb7b1d8661505f081c51af4
2022-03-17 18:29:16 +00:00
Ladsgroup
a8c5232702 Revert "rdbms: fix owner id and RELEASE_ALL_LOCKS query in session flushing methods"
This reverts commit eed58f2f61.

Reason for revert: Blocking the train, going to revert the whole chain

Change-Id: I64f9e5a9dde106671783f958a686ca697182077b
2022-03-17 18:25:48 +00:00
Aaron Schulz
eed58f2f61 rdbms: fix owner id and RELEASE_ALL_LOCKS query in session flushing methods
Use the LoadBalancer id in flushPrimarySessions(), not the LBFactory one,
and use assertOwnership() to check $owner, similar to other methods.

In DatabaseMysqlBase::doFlushSession(), change RELEASE_ALL_LOCKS() query
to use RELEASE_LOCK(), since only newer MariaDB versions (>=10.5.2) support
it. No errors were thrown in the method since they are suppressed, but the
syntax error would cause the transaction to be placed in an error state.

Add assertion to testTransactionCallbackChains() that would otherwise fail.

Randomize lock names in lock() tests to avoid contention.

Bug: T292239
Bug: T303887
Follow-Up: ee3c65d541
Follow-Up: 4cac31de4e
Change-Id: I414d737028338cfd5369eee24576df4aa26a2f6f
2022-03-17 02:55:43 +00:00
daniel
e239b02a5e Add convenience methods for asserting status.
This ensures that assertions work in a uniform way,
and provides meaningful messages in cause of failure.

Change-Id: Ic01715b9a55444d3df6b5d4097e78cb8ac082b3e
2022-03-16 22:44:25 +01:00
Alexander Vorwerk
d66fe8b7c7 Bump minimum required version for upgrade to 1.31
The policy allows this and since 1.39 is going to be the next LTS
release, I think it is fine to do this now.

Change-Id: If426e0ee349252ccc0ba9c4222c7d6865ab57fa2
2022-03-15 22:12:59 +01:00
Alexander Vorwerk
71a092e513 Add 1.38 snapshot to sqlite db tests
Change-Id: I72e703473679dbe982c8d851a6c357b17de415eb
2022-03-15 17:05:42 +01:00
jenkins-bot
98f06217ff Merge "Fix error message in test assertion" 2022-03-14 21:53:42 +00:00
Aaron Schulz
277d34818b tests: Split out DatabaseSqliteUpgradeTest class
The testUpgrades() method is slow and is easier to
parallelize via paratest if it lies in a separate class/file

Also make the test use a data provider

Bug: T50217
Change-Id: I5c8df6ae0ff34941f4cf7275680b309e79565925
2022-03-10 23:29:29 -08:00