Using a data provider for a dynamic or otherwise potentially large amount
of cases tends to come with significant overhead due to all the setup
and test abstraction involved.
Especially when the host class has expensive setUp() or tearDown(),
logic such as the case for API integration tests and other test cases
that have `group Database`. Each of the test methods invokes the
DB reset logic for no good reason.
As an example, ApiQueryRecentChangesIntegrationTest takes about
55-60 seconds to run locally for me on master, but takes only 35-40s
after this patch.
Before:
> slow tests (>50ms):
> …
> 44. 107ms to run ApiQueryRecentChangesIn…t:testValidCovers with d… #25
> 45. 107ms to run ApiQueryRecentChangesIn…t:testValidCovers with d… #2
> 46. 107ms to run ApiQueryRecentChangesIn…t:testValidCovers with d… #7
> 47. 107ms to run ApiQueryRecentChangesIn…t:testValidCovers with d… #8
> 48. 107ms to run ApiQueryRecentChangesIn…t:testValidCovers with d… #14
> 49. 107ms to run ApiQueryRecentChangesIn…t:testValidCovers with d… #17
> 50. 106ms to run ApiQueryRecentChangesIn…t:testValidCovers with d… #18
> ...and there are 3 more above your threshold hidden from view
>
> Time: 59.13 seconds
> OK (53 tests)
After:
> slow tests (>50ms):
> …
> 26. 122ms to run ApiQueryRecentChangesIntegrationTest:testValidCovers
> 27. 108ms to run ApiQueryRecentChangesIntegrationTest:testApiTestGroup
>
> Time: 40.8 seconds
> OK (27 tests)
And that's just for running ApiQueryRecentChangesIntegrationTest.php
in isolation. I expect the CI jobs overall to come down by much more
than that.
Bug: T225730
Change-Id: Idae95f7c6f44c02371e2abc6e41d79b9eaced425
… and make use of yield. I believe the code is easier to read and
understand this way.
Depends-On: I898c09ec16521e6f8c60eb589843ccfc48d63025
Depends-On: Iaf212c8cf665ecae04e76600bc7a9f1d40a6c3c0
Change-Id: I4fc06bf76364cd15be81ab6c8532b9dac9ef012e
Just relying on assertSame() between two strings results in
"Failed asserting that two strings are identical." and a diff,
which is not needed. Make it just use the error string as... an
error string.
Change-Id: Ia9f3faa96105f6d91facfe9e9d9bca77c67b1a5b
assertSame() is guaranteed to never do any magic type conversion.
This can be critical when accidentially comparing empty strings (a
value PHP considers to be "falsy") to false, 0, 0.0, null, and such.
Change-Id: I2e2685c5992cae252f629a68ffe1a049f2e5ed1b
I notice that in CI, the inherited "testValidCovers" often shows
multiple times for test classes as being slow (50ms+).
E.g. from a quibble/php72 job:
> …
> 85ms to run ResourceLoaderWikiModuleTest:testValidCovers
> …
Bug: T225730
Change-Id: I524df134c52173c49ebb2b263a90bb5732255865
PHPUnit only verifies that @covers tags are correct when actually
running coverage - but that's too slow to do on every commit. And if
even a single tag is incorrect, the entire coverage job will fail. It
also has some different requirements compared to normal tags, like all
namespaced classes must use their absolute name. All of those things
combined make it easy for developers to accidentally break the coverage
job.
There are some external tools that also do this, but those tools are
incompatible with our unusual PHPUnit setup (the phpunit.php wrapper
script).
The MediaWikiCoversValidator trait just calls the same method that
PHPUnit does to validate @covers tags. It is implemented as a trait so
that test cases that implement PHPUnit_Framework_TestCase directly can
still use this.
Bug: T171899
Change-Id: I1d564bcae2bfbedb004c440b90db6341148ed4ba
