Commit graph

271 commits

Author SHA1 Message Date
MusikAnimal
72566c80b1 Allow MediaWikiTestCase::insertPage to use given User
Sometimes you need to create pages with a non-sysop.
Example: I31aedcb9af7584fa5504916c67ca10f205ec9910

Change-Id: I9e48f7c00efbce8c2de3f5db3a74462d47ae8f64
2018-06-29 17:04:33 -04:00
jenkins-bot
84fa176c9c Merge "Avoid deprecated LinkCache::singleton()" 2018-06-14 23:48:54 +00:00
addshore
0ef66de3cf MCR RevisionStore, multi content mode..
Bug: T174024
Change-Id: Ifabf39e12ba843eb754ad0c029b7d16a311047a5
2018-06-14 17:30:33 +02:00
addshore
125bf7e44f [MCR] RevisionStore, enable insertions for new schema
Enable inserts to the new MCR db schema in single slot mode only.

TODO:
 - RELEASE NOTES

Notes:
 - When in MIGRATION_WRITE_NEW or greater, deleting and then
 restoring a page will result in different data in the revision table.
 For example, if you delete a page that has text_ids present in the
 revision table and restore it, the text_ids will be blank after.
 - When in MIGRATION_WRITE_BOTH or greater the archive table will
 start to ar_content_model entries where previously it would have been
 given NULL. This is due to the old content schema having NULL in the db
 when the default content model is used, but the new schema will always
 have a value, taken from the content_models table

Note: If259b1e1c49ce was squashed into this change.

Bug: T183488
Bug: T174024
Change-Id: Ic2221da30c8f6ac2ba42720fcd568f2d0ed70534
2018-06-14 13:36:08 +00:00
addshore
129c992ff1 Add @since tags to come MediaWikiTestCase methods
Follow up to:
 - I86e140ec981dfa4e904822b1600399c533f9e3d6 /
 e4b775acf1

Change-Id: Ib45f8ba30dfd382318dd87bc232a529c47c9a307
2018-06-14 08:46:18 +01:00
daniel
e4b775acf1 Suppress addCoreDBData() in tests overriding the revision schema.
Some unit tests for the MCR schema migration change the DB schema
in the test setup. However, addCoreDBData() will not work with the
modified schema. Since these tests don't actually need addCoreDBData()
to do anything, they can simply override it to do nothing.

Without this change, tests for Ic2221da30c and Ifabf39e12ba843
fail with $wgMultiContentRevisionSchemaMigrationStage = MIGRATION_WRITE_BOTH.

Change-Id: I86e140ec981dfa4e904822b1600399c533f9e3d6
2018-06-14 07:41:28 +00:00
Marius Hoch
291440ba49 Fix typehint-typo in MediaWikiTestCase
Change-Id: I3817d586cfaaaf38439820741e68eff044e9bf76
2018-06-12 20:20:49 +02:00
Kunal Mehta
c4e5a9dd97 Avoid deprecated LinkCache::singleton()
Change-Id: Ie0e5c4ef0fe6ec896378bb2433af0898655dd907
2018-06-10 23:55:11 -07:00
jenkins-bot
21ca5a3471 Merge "tests: Reset mwuser/pagecontent tables if necessary for postgres" 2018-06-07 14:39:15 +00:00
jenkins-bot
c3de9b16fb Merge "Reset relevant DB tables before the first test of a suite." 2018-06-07 13:46:56 +00:00
daniel
96a964631b Reset relevant DB tables before the first test of a suite.
MediaWikiTestCase truncates all the tables in the tablesUsed field
after each test. It should also do that before the first test of the
class.

Change-Id: I8c33be7b1bdd83559a9ea7803471a1f39e0eb870
2018-06-07 14:53:26 +02:00
Kunal Mehta
fda6e09a4e tests: Reset mwuser/pagecontent tables if necessary for postgres
As of 556c5cf464, postgres renames the `user` and `text` tables to
`mwuser` and `pagecontent` respectively. Have resetDB() explicitly
reset those tables when using postgres.

Bug: T195807
Change-Id: I5052dd663a4fb16389611cd2985b712fb9a15069
2018-06-06 20:31:57 -07:00
jenkins-bot
46cdecb05e Merge "Avoid "unittest_imagelinks" already exists error in tests" 2018-06-06 18:21:21 +00:00
Brad Jorsch
c84083e413 Make archive.ar_rev_id unique
To follow up I39b0825c, this change replaces the existing non-unique
index on the column with a unique index, to help avoid some of these
sort of bugs in the future.

Bug: T193180
Change-Id: I932478c9c6a13210bc9dff75286d0f08da56682c
2018-06-04 08:52:06 +00:00
Aaron Schulz
b1eaf294e1 Avoid "unittest_imagelinks" already exists error in tests
This occured in MediaWikiTestCase::setUpSchema if the original wiki prefix was not empty.

Change-Id: I6b0a21473986334bcfbb85ef416759aed44f79a3
2018-06-02 17:17:27 -07:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
Bartosz Dziewoński
b191e5e860 Use PHP 7 '<=>' operator in 'sort()' callbacks
`$a <=> $b` returns `-1` if `$a` is lesser, `1` if `$b` is lesser,
and `0` if they are equal, which are exactly the values 'sort()'
callbacks are supposed to return.

It also enables the neat idiom `$a[x] <=> $b[x] ?: $a[y] <=> $b[y]`
to sort arrays of objects first by 'x', and by 'y' if they are equal.

* Replace a common pattern like `return $a < $b ? -1 : 1` with the
  new operator (and similar patterns with the variables, the numbers
  or the comparison inverted). Some of the uses were previously not
  correctly handling the variables being equal; this is now
  automatically fixed.
* Also replace `return $a - $b`, which is equivalent to `return
  $a <=> $b` if both variables are integers but less intuitive.
* (Do not replace `return strcmp( $a, $b )`. It is also equivalent
  when both variables are strings, but if any of the variables is not,
  'strcmp()' converts it to a string before comparison, which could
  give different results than '<=>', so changing this would require
  careful review and isn't worth it.)
* Also replace `return $a > $b`, which presumably sort of works most
  of the time (returns `1` if `$b` is lesser, and `0` if they are
  equal or `$a` is lesser) but is erroneous.

Change-Id: I19a3d2fc8fcdb208c10330bd7a42c4e05d7f5cf3
2018-05-30 18:05:20 -07:00
Timo Tijhof
45660456cf phpunit: Clear job queues in MediaWikiTestCase::setUp()
This should fix the flaky unit test 'SiteStatsTest::testJobsCountGetCached',
which fails locally as follows, when run in isolation.

> 1) SiteStatsTest::testJobsCountGetCached
> A single job enqueued bumps jobscount stat to 1
> Failed asserting that 2 matches expected 1.
>
> /var/www/mediawiki/tests/phpunit/includes/SiteStatsTest.php:22
> /var/www/mediawiki/tests/phpunit/MediaWikiTestCase.php:421
> /var/www/mediawiki/maintenance/doMaintenance.php:94

Instrumenting JobQueueMemory::doBatchPush reveals the following
jobs to have been queued.

- MediaWikiTestCase->run/->addCoreDBData/::getTestSysop/..
  ../User->addGroup/UserGroupMembership->insert/..
  > UserGroupExpiryJob (2)
- MediaWikiTestCase->run/->addCoreDBData/WikiPage->doEditContent/..
  ../WikiPage->{closure}/WikiPage->doEditUpdates/JobQueueGroup->lazyPush/..
  > CategoryMembershipChangeJob
  > HTMLCacheUpdateJob (2)

Fix this by adding clearing of job queues to doLightweightServiceReset()
in MediaWikiTestCase.

Also:
- Move the call to doLightweightServiceReset() from run() to setUp(),
  where it is easier to understand in context. It still runs at the same
  logical point because PHPUnit calls setUp() right before run().

- Remove redundant reset for WANObjectCache->clearProcessCache that
  was both in setUp() and in doLightweightServiceReset().

- Simplify SiteStatsTest by removing the hardcoded delete() calls.
  An alternative fix for the flaky unit test would've been to add
  a delete() call to categoryMembershipChange, but rather than hardcoding
  all possible jobs that TestCase or another test could make, it's easier
  to just reset/delete them all between tests.

- Simplify SiteStatsTest by using the $cache reference directly instead
  of roundtripping through MediaWikiServices. If for some reason
  setService() didn't work, the test will fail either way because it must
  match the one used by JobQueueGroup (TODO: Use injection!), and besides
  the setService() method already has its own unit test.

Change-Id: Ia4b7871221c76c65eacf31915b515705a36940d5
2018-04-24 00:46:19 +01:00
Aaron Schulz
40b2e059c0 Reset table sequences and skip some test assertions for sqlite
Various revision storage tests assume the sequences are reset
and fail otherwise.

Also disable some failing tests that are not applicable to sqlite
as well as postgres.

Change-Id: Ibdb034121a44e16bb35059a92baafb1867951ea8
2018-04-13 10:36:08 +00:00
James D. Forrester
0da97e7a03 Immediately drop wgValidateAllHtml and related code
Bug: T191670
Change-Id: If13d02ee1b30fec1c701226af9d363c6e08b3737
2018-04-10 10:51:28 -07:00
jenkins-bot
579319d960 Merge "Improve test coverage for ApiMain.php" 2018-04-10 17:03:47 +00:00
Aryeh Gregor
be391449ae Improve test coverage for ApiMain.php
One bug fixed: if ApiCheckCanExecute returned false but didn't set
$message, we would try to output a message of false, which would throw
an exception.

Change-Id: Ib06970e280d750ff57d81672f1b365167b93aa3e
2018-04-08 21:15:37 +03:00
jenkins-bot
f1d64e2fc6 Merge "Provide PHPUnit 4 and 6 compatibility layer" 2018-04-06 23:04:48 +00:00
Kunal Mehta
b34260f379 Provide PHPUnit 4 and 6 compatibility layer
PHPUnit 6 removed some functions that were heavily used in version 4. To
be able to support both versions for a short time, we'll use a trait to
fill in the missing methods until we drop PHPUnit 4 support.

This trait is included in MediaWikiTestCase so most tests will be able
to benefit from it by default. Otherwise, anything that calls
setExpectedException() or getMock() will need to use it.

Change-Id: I707129e471e960e034e2aa994a467b9dc0239b69
2018-04-06 14:33:25 -07:00
Brad Jorsch
501b1fb7df tests: Reset Postgres sequences when cloning and truncating
This improves the repeatability of the unit tests by making the ID
values generated depend less on what previous tests might have done.

It also prevents tests from using up sequence numbers for the live DB's
tables.

Change-Id: Iaa8ae1e5cef4b9099bd1b4b8fc806f5af372a7ff
2018-04-05 23:57:24 +00:00
daniel
891eda0931 Allow schema overrides to drop tables.
This allows unit tests to use schema overrides that drop tables,
in addition to overrides that create or modify tables.

Change-Id: I59761c7db7f83698749324ca6b9ffced86ab1249
2018-03-29 21:44:47 +02:00
Brad Jorsch
27c61fb1e9 Add actor table and code to start using it
Storing the user name or IP in every row in large tables like revision
and logging takes up space and makes operations on these tables slower.
This patch begins the process of moving those into one "actor" table
which other tables can reference with a single integer field.

A subsequent patch will remove the old columns.

Bug: T167246
Depends-On: I9293fd6e0f958d87e52965de925046f1bb8f8a50
Change-Id: I8d825eb02c69cc66d90bd41325133fd3f99f0226
2018-02-23 10:06:20 -08:00
Umherirrender
63d96c15fd build: Updating mediawiki/mediawiki-codesniffer to 16.0.0
Change-Id: I59b59f79bbf3ce4feff3b3a20c1c31bc16370531
2018-02-17 13:29:13 +01:00
Reedy
39f0f919c5 Update suppressWarning()/restoreWarning() calls
Bug: T182273
Change-Id: I9e1b628fe5949ca54258424c2e45b2fb6d491d0f
2018-02-10 08:50:12 +00:00
Chad Horohoe
5c93fabafd Drop XCache support
It was never super popular anyway, APC was always the best option.
The project has no plans to move to PHP7, so it's quickly reaching
its end of life. Oh, and Fedora dropped it from their repos 2 years
ago.

Change-Id: Ia3257e86a6323d8943f04a5c05c72c0bd4c4b0a9
2018-02-07 13:45:40 -08:00
Aaron Schulz
3f05f346a8 SiteStats row initialization cleanups
* Remove unreachable code (field cannot be both null and -1)
* Avoid PHP warnings during test runs in miser mode due to
  the $row field being "false".
* Init the site_stats row in unit tests to unbreak miser mode.

Change-Id: Ieb45cb3068b71e01aa12c674e0bfcf13a7f2493c
2018-02-02 08:22:03 +00:00
Umherirrender
45da581551 Use ::class to resolve class names in tests
This helps to find renamed or misspelled classes earlier.
Phan will check the class names

Change-Id: Ie541a7baae10ab6f5c13f95ac2ff6598b8f8950c
2018-01-26 22:49:13 +01:00
jenkins-bot
b9ecb8e067 Merge "Add options and join conds to MediaWikiTestCase::assertSelect" 2018-01-06 21:28:23 +00:00
daniel
969b97f129 Fix dropping of temp tables on SQLite and PG.
Also addresses other compatibility issues with PG mentioned
in comments on I7a4071072.

Bug: T184333
Change-Id: I478aa1aee15fdef99630c65a37b1af5f3f8cce14
2018-01-06 11:16:13 +01:00
Brad Jorsch
1b9f454629 Add options and join conds to MediaWikiTestCase::assertSelect
Because selects sometimes need to specify these.

Change-Id: I853e8210bbafe16a62060b9075384afb9cdb03c0
2018-01-05 11:27:40 -05:00
daniel
047151c692 Introduce DB schema overrides for unit tests.
This introduces MediaWikiTestCase::getSchemaOverrides,  which can be overwritten
to return information about which tables are going to be altered, and which SQL
files should be used to set up the target schema. This allows tests for a class
that interacts with the database can have a subclass for each supported database
schema.

NOTE: this has only been tested with MySQL.

Bug: T180705
Change-Id: I7a4071072d802a82ecf7d16fbf8882ff8c79287f
2018-01-05 16:23:55 +00:00
Aaron Schulz
336454104d Try to opportunistically flush statsd data in maintenance scripts
This helps to avoid OOMs from buffer build-ups in the statsd
factory object. This piggy-backs on to the same checks used
for deferred update runs. In addition, the output() method
checks if the data size is getting large and emits if needed.

Bug: T181385
Change-Id: I598be98a5770f8358975815e51380c4b8f63a79e
2017-12-30 05:01:21 +00:00
Kunal Mehta
dafe195c28 Verify that all @covers tags are sane when running tests
PHPUnit only verifies that @covers tags are correct when actually
running coverage - but that's too slow to do on every commit. And if
even a single tag is incorrect, the entire coverage job will fail. It
also has some different requirements compared to normal tags, like all
namespaced classes must use their absolute name. All of those things
combined make it easy for developers to accidentally break the coverage
job.

There are some external tools that also do this, but those tools are
incompatible with our unusual PHPUnit setup (the phpunit.php wrapper
script).

The MediaWikiCoversValidator trait just calls the same method that
PHPUnit does to validate @covers tags. It is implemented as a trait so
that test cases that implement PHPUnit_Framework_TestCase directly can
still use this.

Bug: T171899
Change-Id: I1d564bcae2bfbedb004c440b90db6341148ed4ba
2017-12-29 20:19:12 +00:00
Timo Tijhof
cd9f2f44e5 phpunit: Remove outdated comment about calling of LinkCache::clear()
This is actually being called now, from resetDB() between tests.

Change-Id: I5aa15ce3fd57b6fc90b424fde8691ff33c7ddf45
2017-12-27 21:52:44 +01:00
addshore
d849eb16ff Clarify the use of @group Database in MediaWikiTestCase
Only the class level doc comment is checked.
Adding @group Database to a method doc comment will do
nothing.

Change-Id: Ic0fff3636d6c44e7a27df8806c9712155ce81d8c
2017-11-23 19:19:14 +00:00
Umherirrender
f739a8f368 Improve some parameter docs
Add missing @return and @param to function docs and fixed some @param

Change-Id: I810727961057cfdcc274428b239af5975c57468d
2017-09-10 20:32:31 +02:00
MusikAnimal
d09554b6ef Add basic IP range support to Special:Contributions
This works by using the new table introduced with T156318.

The only thing that differs from normal Special:Contribs is we are
showing the IP address next to each entry. This is it how it is
displayed if you request to see newbie contributions:
https://en.wikipedia.org/wiki/Special:Contributions?contribs=newbie

For the time being, Special:DeletedContributions does not support
IP ranges. Various other irrelevant links such as Uploads and Logs
are also hidden.

Refer to P4725 for a way to automate creation of edits by random
IPs in your dev environment.

IP::isValidBlock() has been deprecated with this dependent change:
https://gerrit.wikimedia.org/r/#/c/373165/

Bug: T163562
Change-Id: Ice1bdae3d16cf365da14c6df0e8d91d2b914e064
2017-09-05 16:15:33 -04:00
Brad Jorsch
11cf01dd9a Add comment table and code to start using it
A subsequent patch will remove the old columns.

Bug: T166732
Change-Id: Ic3a434c061ed6e443ea072bc62dda09acbeeed7f
2017-08-30 15:05:00 +10:00
jenkins-bot
a8ec960e9d Merge "Produce RDF dump of all categories and subcategories in a wiki." 2017-08-28 11:01:58 +00:00
Stanislav Malyshev
d9bb673b72 Produce RDF dump of all categories and subcategories in a wiki.
Example:
http://en.wiki.local.wmftest.net:8080/wiki/Category:Ducks> a mediawiki:Category ;
        rdfs:label "Ducks" ;
        mediawiki:isInCategory <http://en.wiki.local.wmftest.net:8080/wiki/Category:Birds> .

Bug: T157676
Change-Id: I59b9603581b37af59d17dd6c38247c85aee44911
2017-08-28 00:30:35 -07:00
Umherirrender
9b8b314992 Fix spacing for @param and indent of function comments
In phpcs.xml rename renamed sniffs and add the failing sniffs,
because now the whole sniff is no longer excluded.

Change-Id: If5b0bd16028761abc2c47ace9e97d37ad14bb36f
2017-08-15 14:33:29 +00:00
WMDE-Fisch
6df9ed1ad6 update mediawiki-codesniffer to 0.11.0 and fix issues
- mostly auto fixes
- some too long lines fixed
- ignore amp space in one case  passing by reference

Change-Id: I6472f83bc3cbf4bd629d83050cc3319b19ec465c
2017-08-11 22:27:51 +02:00
Bartosz Dziewoński
8785e4a5b3 Replace remaining uses of deprecated DB_SLAVE with DB_REPLICA
Change 950cf6016c took care of the most,
but a few remain, either outside of includes/ and maintenance/
directories (which that change was limited to), or in code introduced
afterwards.

Change-Id: I9c363d0219ea7e71cde520faba39406949a36d27
2017-08-05 10:10:26 +00:00
jenkins-bot
313dde5d06 Merge "Add WAN Cache to SiteStats::jobs" 2017-07-25 18:30:58 +00:00
Antoine Musso
29aaac5c63 Add WAN Cache to SiteStats::jobs
The method hits the jobrunner backend to find out how many jobs are
enqueued in each of the JobQueue.  It is publicly available via the
MediaWiki API request:
    /w/api.php?action=query&meta=siteinfo&siprop=statistics

That is often used by bots when querying recent changes among other and
with fast bot cause useless queries toward the jobrunner backend.

Wrap SiteStats::jobs() with a WAN cache under key SiteStats:jobscount.
Drop SiteStats::$jobs private variable that was used for in process
cache. The WAN Cache does it for us via 'pcTTL'.

That is similar to SiteStats::numberingroup().
Set TTL to one minute, which should still give fresh enough results for
public uses.

Cover that behavior with a test.

When writing tests I noticed MediaWikiTestCase generates a few jobs due
to the creation of the UTPage page:

* HTMLCacheUpdateJob to refresh backlinks (eg: history)
* RecentChangesUpdateJob which happens randomly

Pass EDIT_SUPPRESS_RC to doEditContent to prevent the first and blindly
delete entries in the recentChangesUpdate jobqueue for the second.

Change-Id: I95a272d0691d779bfee9e7a671cbab66a113dfa1
2017-07-25 18:17:34 +00:00