Commit graph

36 commits

Author SHA1 Message Date
Timo Tijhof
0075ebd83b objectcache,resourceloader,rdbms,jobqueue: Widen @covers annotations
Follows-up I4c7d826c7ec654b, I1287f3979aba1bf1.

We lose useful coverage and spend valuable time keeping these accurate
through refactors (or worse, forget to do so). The theoretically "bad"
accidental coverage is almost never actually bad.

Having said that, I'm not removing them wholesale (yet). I've audited
each of these specific files to confirm it is a general test of the
specified subject class, and also kept it limited to those specified
classes. That's imho more than 100% of the benefit for less than 1%
of the cost (more because `@covers` is more valuable than the fragile
and corrosive individual private method tracking in tests that
inevitably get out of date with no local incentive to keep them up to
date).

Cases like structure tests keep `@coversNothing` etc and we still don't
count coverage of other classes. There may be a handful of large
legacy classes where some methods are effectively class-like in
complexity and that's why it's good for PHPUnit to offer the precision
instrument but that doesn't meant we have to use that by-default for
everything.

I think best practice is to write good narrow unit tests, that reflect
how the code should be used in practice. Not to write bad tests and
hide part of its coverage within the same class or even namespace.
Fortunately, that's generally what we do already it's just that we
also kept these annotations still in many cases.

This wastes time to keep methods in sync, time to realize (and fix)
when other people inevitably didn't keep them in sync, time to find
uncovered code only to realize it is already covered, time for a less
experienced engineer to feel obligate to and do write a low quality
test to cover the "missing" branch in an unrealistic way, time wasted
in on-boarding by using such "bad" tests as example for how to use
the code and then having to unlearn it months/years later, loss of
telemetry in knowing what code actually isn't propertly tested due to
being masked by a bad test, and lost oppertunities to find actually
ununused/unreachable code and to think about how to instead structure
the code such that maybe that code can be removed.

------

Especially cases like LBFactoryTest.php were getting out of hand,
and in GlobalIdGeneratorTest.php we even resorted to reminding people
with inline comments to keep tags in sync.

Change-Id: I69b5385868cc6b451e5f2ebec9539694968bf58c
2023-04-10 22:00:17 +00:00
Tim Starling
5e30a927bc tests: Make some PHPUnit data providers static
Just methods where adding "static" to the declaration was enough, I
didn't do anything with providers that used $this.

Initially by search and replace. There were many mistakes which I
found mostly by running the PHPStorm inspection which searches for
$this usage in a static method. Later I used the PHPStorm "make static"
action which avoids the more obvious mistakes.

Bug: T332865
Change-Id: I47ed6692945607dfa5c139d42edbd934fa4f3a36
2023-03-24 02:53:57 +00:00
jenkins-bot
81ab58ed82 Merge "Protect HistoryBlob storage against malicious class injection" 2022-12-07 20:54:34 +00:00
Tim Starling
540bddfb1f When content is marked bad, show an error, don't pretend it is empty
It misrepresents the users contribution to show empty text for a
revision when in fact the revision contained some text which we later
lost.

Also, errors from SqlBlobStore::fetchBlobs() did not stop a cache entry
from being written, so a subsequent cache hit would show the bad
revision as empty.

So, in Storage:
* Add BadBlobException, which is thrown by the Storage layer to
  indicate that a revision is marked as bad.
* Have SqlBlobStore::getBlobStore() return an error for bad blobs
  instead of an empty string.
* Duplicate the check for flags=error into SqlBlobStore::expandBlob().
  This avoids an unnecessary cache fetch, and avoids making
  decompressData() throw on error, which would be a b/c break.
* In SqlBlobStore::getBlob(), suppress the cache when there was an
  error.

In Revision:
* Add BadRevisionException, to wrap BadBlobException in the Revision
  layer.
* Return null from RevisionRecord::getContent() on a broader set of
  errors. Make it mostly non-throwing.
* Add RevisionRecord::getContentOrThrow() which returns a non-nullable
  Content.
* Note that SlotRecord::getContent() returns a non-nullable Content so
  now throws in more cases.

In the UI:
* In Article::view(), catch the exception and show an error message.
* In DifferenceEngine, catch the exception and make a suitable error
  message available via getRevisionLoadErrors(). In the diff page, show
  the error message in a box.
* In ApiComparePages and the legacy rvdiffto, show a warning.
* In RawAction, show a 404 by analogy with other error cases.
* In EditPage, there was already handling for $content=null with an
  appropriate error message (missing-revision-content). But having
  $this->textbox1 = null caused PHP 8.1 deprecation warnings, so I fixed
  that.
* In EditPage undo, there was already handling for null content, but I
  improved the error message: "does not exist or was deleted" seems more
  appropriate than "conflicting intermediate edits".

Change-Id: Idd1278d6d756ef37d64addb7b5f3be30747ea603
2022-12-05 22:03:45 +00:00
Tim Starling
21352255ad Protect HistoryBlob storage against malicious class injection
* Add a safe unserialize() wrapper for HistoryBlob classes
* Add a safe unserialize() wrapper for plain array data as used for
  compressed internal storage by ConcatenatedGzipHistoryBlob and
  DiffHistoryBlob.
* Fix tests broken by this.
* Fix unnecessary call to uncompress(), __wakeup() does this already.
  Was a phan error now that we have more information about the type of
  $obj.
* Add tests for successful unserialize and wakeup of WMF production
  data.

Change-Id: Ic995dda16d9c6045b33f2fdae7f6575ac8329976
2022-12-02 00:26:11 +00:00
Kevin Israel
0ae29bbb69 tests: Replace checkPHPExtension() with @requires annotations
PHPUnit 3.7.0 was the first version to support @requires extension. At
the time checkPHPExtension() was added in a7901801b4, MediaWiki
still supported PHPUnit 3.6.7.

MediaWiki now requires intl and xml, so I removed checks for those
extensions rather than converting them to annotations.

checkPHPExtension() is removed without deprecation; it does not appear
to have ever been used (and is not likely to be used) in MW extensions.
This is explicitly permitted under the stable interface policy. Even if
it were not, only tests are affected, and they are supposed to fail
anyway if hard deprecated code is used.

Change-Id: I45f9b4c0e120683103cead916f4d4ef58bd11530
2022-10-23 20:47:14 -04:00
Thiemo Kreuz
61ae7504df Replace trivial usa of mock builder with createMock() shortcut
createMock() does the same, but is much easier to read.

A small difference is that some of the replacements made in this
patch didn't use disableOriginalConstructor() before. In case this
was relevant we should see the respective test fail. If not we can
save some CPU cycles and skip these constructors.

Change-Id: Ib98fb06e0fe753b7a53cb087a47e1159515a8ad5
2022-07-15 16:43:48 +00:00
Matěj Suchánek
3d66fdc3b4 Remove SqlBlobStore::getLegacyEncodingConversionLang
It has been deprecated since 1.34 and it is unused.

Change-Id: Ib08d77ef1d35b4436cdb5532c0d5d50c08a5f4f9
2022-06-21 19:42:14 +02:00
Timo Tijhof
128debb64b tests: Change use of AtEase to at operator
Follows-up I361fde0de7f4406bce6ed075ed397effa5be3359.

Per T253461, not mass-changing source code, but the use of the native
error silencing operator (@) is especially useful in tests because:

1. It requires any/all statements to be explicitly marked. The
   suppressWarnings/restoreWarnings sections encourage developers to
   be "lazy" and thus encapsulate more than needed if there are multiple
   ones near each other, which would ignore potentially important
   warnings in a test case, which is generally exactly the time when
   it is really useful to get warnings etc.

2. It avoids leaking state, for example in LBFactoryTest the
   assertFalse call would throw a PHPUnit assertion error (not meant
   to be caught by the local catch), and thus won't reach
   AtEase::restoreWarnings. This then causes later code to end up
   in a mismatching state and creates a confusing error_reporting
   state.

See .phpcs.xml, where the at operator is allowed for all test code.

Change-Id: I68d1725d685e0a7586468bc9de6dc29ceea31b8a
2022-02-24 21:29:51 +00:00
Alexander Vorwerk
decbaf4f38 phpunit: use ->getServiceContainer() in integration tests
Change-Id: I38299cb65eeaadfdc0eb05db4e8c0b0119cfb37d
2022-01-27 22:04:16 +01:00
Umherirrender
977f0c0c19 Remove passing markTestSkippedIfDbType or add comment
It seems some tests are fixed in the meantime without removing the
skipped part for this.

Bug: T195807
Follow-Up: I58261dd70eea3581803987a4a7739c7d55558f42
Change-Id: Ie883b7056cdd0cb6bec81c2f7e4ed54177b4ecb2
2022-01-22 17:04:53 +00:00
Umherirrender
bcee220ad1 tests: Avoid non-namespaced aliases for Wikimedia\Rdbms namespace
Change-Id: I49cb151f64c578f41dc1236b2a8906694ca97b93
2021-09-18 20:24:08 +02:00
TChin
077b9af960 Replace Assert::parameterType with typehints
Bug: T287530
Change-Id: I6060e194339614b53e3a9c036ff3a3ac2e68f8df
2021-08-03 10:03:48 -04:00
DannyS712
7855f60aa3 SqlBlobStoreTest: test against tt:10000 instead of tt:1000
The removal of the Revision class led to this test failing, since
tt:1000 started to have content - not sure why but I assume something
in the Revision tests was clearing the store - the text
was "UTContent" which is used in many tests via helper
method MediaWikiIntegrationTestCase::getExistingTestPage()
which can create pages with that text.

Change-Id: I04c669ad8c8a7166138eabc973140205b60536a5
2021-05-10 19:00:29 +00:00
Thiemo Kreuz
b655f382db Remove broken/outdated @param/@throws tags from @dataProviders
My personal best practice is to not document @params when there
is a @dataProvider. I mean, these test…() functions are not
meant to be called from anywhere. They do not really need
documentation. @param tags don't do much but duplicate what the
@dataProvider does. This is error-prone, as demonstrated by the
examples in this patch.

This patch also removes @throws tags from tests. A test…() can
never throw an exception. Otherwise the test would fail.

Most of these are found by the not yet released I10559d8.

Change-Id: I3782bca43f875687cd2be972144a7ab6b298454e
2021-01-21 03:42:42 +00:00
Umherirrender
0347fd0631 Improve some function documentation in tests
Also fix some whitespaces

Change-Id: Ibed50a4f07442d3f299cf545c16f5dbb5f27a411
2021-01-14 22:13:55 +01:00
daniel
353d985620 Revision: Point to findBadBlobs.php in exception messages
Advertize the fact that production errors resulting from corruption in
the content storage layer can be remedied using findBadBlobs.php.

Change-Id: I4eab773d0d2da57a545f211c2fef36146331c570
2020-08-23 00:50:32 +00:00
addshore
959bc315f2 MediaWikiTestCase to MediaWikiIntegrationTestCase
The name change happened some time ago, and I think its
about time to start using the name name!
(Done with a find and replace)

My personal motivation for doing this is that I have started
trying out vscode as an IDE for mediawiki development, and
right now it doesn't appear to handle php aliases very well
or at all.

Change-Id: I412235d91ae26e4c1c6a62e0dbb7e7cf3c5ed4a6
2020-06-30 17:02:22 +01:00
daniel
495323c063 Remove compat code for pre-MCR schema.
NOTE: This was manually tested to ensure it does not
break updates, see T242764.

Bug: T198557
Change-Id: I1d9f5465018bae10124514bc38169e23e0e613e6
2020-03-24 19:45:47 +01:00
Thiemo Kreuz
6b2c9deef5 Replace all new stdClass() with identical (object)[]
This should be the exact same. Its more a style change than anything.
So why do it then?
* I believe this is much less confusing than code mentioning a weird
"standard class". Barely anybody knows what this is, and what the
difference between "object" and "stdClass" is.
* The code is shorter.
* It's even faster. In my micro benchmark it's twice as fast.

Change-Id: I7ee0e8ae6d9264a89b6cd1dd861f0466ae620ccc
2020-03-04 21:18:30 +00:00
daniel
292decc425 BlobStore: support "known bad" addresses.
This adds support for addresses starting with the prefix "bad:",
which will always resolve to an empty string.
The syntax for addresses supported by SqlBlobStore is relaxed
to be closer to RFC 3986 and make it easier to include meaningful
information after the "bad:" prefix.

Bug: T240686
Change-Id: I175b1880ce94ff8a9079bc50440cde188e57d95d
2020-01-03 16:38:13 +01:00
Max Semenik
e14fa40ecd More backwards compatible changes for PHPUnit
The change in XmlTest is because PHPUnit 8 messes with serialization
settings.

Bug: T192167
Change-Id: I5c9358d7cfc451845e01fca67c8d15add4039dae
2019-11-12 17:07:23 -08:00
daniel
e1f8a8148d SqlBlobStore HOT FIX: remove caching from getBlobBatch
This is a TEMPORARY HACK to fix a problem with cached values
getting mixed up when using getBlobBatch(). As of the creation of this
patch, the actual cause is unknown. Caching should be put back once the
cause of the problem has been found.

Note that getBlobBatch is only used in maintenance scripts.
Removal of caching in that method has no effect on web requests.

getBlobBatch() was introduced in I56306c50a661 and is used by
RevisionStore::getContentBlobsForBatch(), which in turn is needed
by the Translate extension to remove its dependency on the deprecated
rev_text_id field.

Bug: T235188
Change-Id: I94c6f9ba7b9caeebaa9b055916f15f7bbdcd8fb6
2019-10-16 23:02:52 +00:00
daniel
46bb49dbfd SqlBlobStore: test caching behavior.
Something is going wrong with the way we use WANObjectCache in
getBlobBatch. This patch introduces a test case that should help narrow
down the cause.

Bug: T235188
Change-Id: I2d06c09609874440992e6c90d3ebfa15400ac302
2019-10-14 14:01:06 +00:00
Daimona Eaytoy
98fcdd8f7d Replace setExpectedException with single arg
Find: (\$this->)setExpected(Exception\( \\?[a-z\\]+::class \);)
Replace: $1expect$2

Bug: T192167
Change-Id: I33a24d42b6dc1e126f32d5dbf41daa0bccb1414f
2019-10-05 17:39:46 +02:00
Timo Tijhof
01095f8075 Storage: SqlBlobStore no longer needs Lanugage object
Constructing a Language object in order to initialize the
BlobStoreFactory service causes a circular dependency
(see T231866).

SqlBlobStore was using the Language object to all iconv.
But nothing language specific is done in Language::iconv,
so we can just inline the call.

Bug: T231866
Change-Id: I90c25decbcff10ea762a2c7474a12fd2041b3abc
2019-10-02 09:26:34 -07:00
Petr Pchelko
fa36773e58 Introduce BlobStore::getBlobBatch method
Bug: T230834
Change-Id: I56306c50a6617dc91e4eb362ef010703ac25d951
2019-08-30 09:40:49 +00:00
Aaron Schulz
baafb5adb4 Make ExternalStore wrap ExternalStoreFactory and create access class
* Inject settings and global instances as dependencies to the
  ExternalStoreMedium instances. This includes the local wiki
  domain, so that wfWikiId() calls are not scattered around.
* Create ExternalStoreAccess service for read/write logic.
* Deprecate the ExternalStore wrapper methods.
* Add some exception cases for bogus store URLs are used instead
  of just giving PHP warnings and failing later.
* Make moveToExternal.php require the type/protocol to decide
  which ExternalStoreMedium to use instead of assuming "DB".
* Convert logging calls to use LoggerInterface.

Change-Id: I40c3b5534fc8a31116c4c5eb64ee6e4903a6197a
2019-06-28 14:31:44 -07:00
Fomafix
3ee1560232 No yoda conditions
Replace
  if ( 42 === $foo )
by
  if ( $foo === 42 )

Change-Id: Ice320ef1ae64a59ed035c20134326b35d454f943
2018-11-21 17:54:39 +01:00
Kunal Mehta
b4ea2d8c81 Disable PHPUnit tests that fail under postgres
So we can make the job voting, preventing other tests from regressing.
These tests can be re-enabled whenever they're made to pass.

Bug: T195807
Change-Id: I58261dd70eea3581803987a4a7739c7d55558f42
2018-08-01 22:22:15 +00:00
daniel
564257dc50 Blob can't be false
Change-Id: Ic06fcfaac71128c0ff7e9079685eac18206f2004
2018-06-20 17:32:35 +02:00
daniel
1b115fd022 Introduce SqlBlobStore::makeAddressFromTextId
This introduces a convenience method for constructing a blob address
from a text ID. It's the inverse of SqlBlobStore::getTextIdFromAddress

Change-Id: I31b3ee5e40185c754fb2c119eb5edc50b903f5dc
2018-04-19 17:23:59 +00:00
addshore
4b928c87dc Extra tests for SqlBlobStore with 'windows-1252' legacy encoding
Bug: T184749
Change-Id: Ida717dbe6ae742b3b61f0f09dc60712307c53a96
2018-01-24 22:08:46 +00:00
Umherirrender
84b9564e50 SqlBlobStoreTest: Use utf-8 as name in iconv
utf8 is an alias that doesn't work with all PHP installations.

Change-Id: I6ce9c5b4ece1a8c263afb1d4895dacd3c790a51a
2017-12-15 19:56:48 +01:00
Kunal Mehta
471ec51c93 Fix @covers tags to use absolute class names in Storage tests
PHPUnit doesn't support use statements when evaluating @covers tags.

Change-Id: I7f649afd3e417a6b71b2f61c6ff59be5f404be8f
2017-12-14 19:20:51 -08:00
addshore
06127159e8 [MCR] Tests for RevisionStore & Related classes
Code introduced in:
I140f43a6fb443b38483f41f268c906b9cea64cf7

Change-Id: Iefad870baf2d16f12e9901b303246c64d6431ca6
2017-12-14 13:44:55 +00:00