Commit graph

577 commits

Author SHA1 Message Date
jenkins-bot
40077e0971 Merge "rdbms: Add (soft) type hints to DatabaseMysqlBaseTest" 2019-10-07 20:12:15 +00:00
jenkins-bot
348416e94d Merge "objectcache: Stricter assertions for WANObjectCache tests" 2019-10-06 15:13:59 +00:00
Max Semenik
fb06e4cd85 Update tests to use PHPUnit 6 class names
Bug: T192167
Change-Id: I42b0c8908b4968b95b08f861a40af18dc79fa0a1
2019-10-06 01:01:28 -07:00
Timo Tijhof
28c470adc7 objectcache: Stricter assertions for WANObjectCache tests
Use the stricter comparisons from assertSame instead of
assertEquals.

In some cases this revealed a test that was asserting a value
different from the one actually returned:

* testSetAndGet:
  The test failed with the test case (null, 3), where it tries
  to store the value `null`, but gets back `false`, which is wrong.
  This appears to be a genuine bug, filed as T234583.

Change-Id: Ic3dcc7fa2e8749b0f2d68917a8ac728dda26b6ca
2019-10-06 02:10:24 +00:00
Max Semenik
dc3fba4619 tests: Nuke everything HHVM-related
Change-Id: Ieccce5ebba25132b2433fc13a575b65a5bd538f2
2019-10-05 12:51:07 -07:00
Max Semenik
bc3878e33a Begin cleaning up PHPUnit 4 code from tests
This process will be broken up into several parts for reviewability.

Bug: T192167
Change-Id: Ie415fd3308384a5ca2b3de24ba037785f8a3a714
2019-10-04 14:19:05 -07:00
Timo Tijhof
acda466803 resourceloader: Remove data URI fallback for IE6 and IE7
It's a bit late given that just earlier this week we removed
almost all uses of `@embed` (T121730). So as an optimisation, this
won't have much impact now.

Bug: T234582
Bug: T121730
Change-Id: I5729c8b059504244ac7fb0b515df9c29909b9d5e
2019-10-04 03:15:37 +01:00
Thiemo Kreuz
e4272518f7 tests: Replace PHPUnit's loose assertEquals(false) with assertFalse()
assertEquals( false, … ) still succeeds when the actual value is 0, null,
an empty string, even an empty array. All these should be reported as a
failure, I would argue.

Note this patch previously also touched assertSame( false ). I reverted
these. The only benefit would have been consistency within this codebase,
but there is no strict reason to prefer one over the other. assertFalse()
and assertSame( false ) are functionally identical.

Change-Id: Ic5f1c7d504e7249002d3184520012e03313137b4
2019-10-04 00:30:36 +00:00
Daimona Eaytoy
c12b32c9a3 HHVM removal: Kill HHVM_VERSION checks in PHPUnit tests
Change-Id: I1d18b2d6956d326e5f9e50cd01ce2bb683a5b653
2019-10-03 15:35:21 +00:00
Thiemo Kreuz
cff596804a tests: Replace assertions on count() == 0 with strict === []
The benefit of using count() is that the test would still succeed if
the return vfalue is not an array, but an iterable object. It seems
this is not needed.

Change-Id: I23529f6990aebe0cce86e236a21820fe74993204
2019-09-30 16:20:34 +02:00
Daimona Eaytoy
efbfa0a727 StringUtils: Clarify that isValidRegex is for PCRE regexps
As suggested in c14571ba26 (I257a096319f1e).

Change-Id: Ia91d037f2f4bf8a1bad4eac65fecb1d3e2679d2d
2019-09-28 19:05:29 +00:00
jenkins-bot
864c4df8b9 Merge "tests: Replace PHPUnit's loose assertEquals(null) with assertNull()" 2019-09-27 22:43:09 +00:00
Thiemo Kreuz
c2211946f7 tests: Replace PHPUnit's loose assertEquals(null) with assertNull()
assertEquals( null, … ) still succeeds when the actual value is 0, false,
an empty string, even an empty array. All these should be reported as a
failure, I would argue.

Note this patch previously also touched assertSame( null ). I reverted
these. The only benefit would have been consistency within this codebase,
but there is no strict reason to prefer one over the other. assertNull()
and assertSame( null ) are functionally identical.

Change-Id: I92102e833a8bc6af90b9516826abf111e2b79aac
2019-09-27 19:15:38 +00:00
jenkins-bot
e75daacd8e Merge "objectcache: fully respect "pcTTL" in WANObjectCache instead of using INF when >= 0" 2019-09-27 00:13:37 +00:00
Aaron Schulz
8f20122c18 rdbms: fix active GTID filtering in DatabaseMysqlBase
In masterPosWait(), only $pos will have the known active domain/server set
since it usually comes from getMasterPos(). However, the reference position,
from getReplicaDB(), does not have the active domains set since querying
gtid_domain_id on the replica would be incorrect and getting a connection
to the master could be expensive.

Remove obsolete hacks for jobs that used to store master positions.

Also, use the regular Database::query() method for stylistic consistency.

Bug: T224422
Change-Id: I41bbb9f337e46451aa17788dbd446db4a213a5a7
2019-09-25 11:59:05 +00:00
Aaron Schulz
223b7a3717 objectcache: fully respect "pcTTL" in WANObjectCache instead of using INF when >= 0
This was broken since 611e2d5596

Change-Id: I612eaf211ff698d5ab1c911aa58195b7bc44f00c
2019-09-24 02:56:39 -07:00
Thiemo Kreuz
a6b600b0d4 rdbms: Add (soft) type hints to DatabaseMysqlBaseTest
These type hints make it easier to follow the code paths and understand
what code is actually under test here. A few of these tests appear to
be misplaced, as they don't cover any code in the DatabaseMysqlBase
class.

This also makes all @covers tags absolute so more tools can understand
them (e.g. PHPStorm doesn't).

Change-Id: I43c69be994d6094239954ea662801c1e7e8e2e41
2019-09-23 11:10:48 +02:00
jenkins-bot
095c50ade1 Merge "tests: Prefer assertSame() when comparing the integer 0" 2019-09-20 15:33:02 +00:00
Thiemo Kreuz
32a429e8c4 tests: Prefer assertSame() when comparing the integer 0
assertSame() is guaranteed to not do any type conversion. This can be
critical when acciden tially comparing, for example, 0 to 0.0.

Change-Id: Iffcc9bda69573623ba14af655dcd697d0fcce525
2019-09-19 15:35:23 +00:00
jenkins-bot
a1ef77b2d8 Merge "StringUtils: Add a utility for checking if a string is a valid regex" 2019-09-19 08:03:33 +00:00
Daimona Eaytoy
c14571ba26 StringUtils: Add a utility for checking if a string is a valid regex
There's no function to check if a string is a valid regex, and doing it
via preg_match requires error suppression, often performed with @. This
isn't however a good practice, and suppressing errors via functions is
not a one line solution. Adding an utility function would probably be a
benefit for several use cases (try to grep '@preg_match' for a few of
them).

Change-Id: I257a096319f1ec13441e9f745dcd22545fdd5cc6
2019-09-18 09:46:29 +00:00
Timo Tijhof
9244cdb977 objectcache: Escape dots from cache keys in StatsD metrics
Bug: T232907
Change-Id: Ia61256e24fe35803e69a83a1ca235e18297c75cd
2019-09-14 18:16:55 +00:00
jenkins-bot
509b2f7fe2 Merge "libs/Message: Improve documentation" 2019-09-06 01:45:24 +00:00
Aaron Schulz
1fbb16c712 rdbms: expand on LoadBalancer ownership concept
Enforce this pattern for the remaining LoadBalancer methods.
Carry this over into Database::close() to decide how loud the
error handling should be.

In LBFactory, clean up ownership of newMainLB()/newExternalLB().
The should have a null owner if called from outside the class
since the LBFactory does not track nor care about them anymore
in that case. Disable newMainLB() for LBFactorySingle as it
makes no sense and was broken.

Also remove some redundant abstract LBFactory methods that
just duplciate ILBFactory.

Bug: T231443
Bug: T217819
Depends-On: I7ed5c799320430c196a9a8e81af901999e2de7d0
Change-Id: I90b30a79754cfcc290277d302052e60df4fbc649
2019-09-05 12:08:13 -07:00
jenkins-bot
f2d3caf5bc Merge "Optimize HashRing to avoid hashing for the common single-location case" 2019-08-31 17:59:08 +00:00
Aaron Schulz
cfff40e900 Optimize HashRing to avoid hashing for the common single-location case
Change-Id: I3423ac8bd5c56db60630257ac04706838a832d36
2019-08-29 19:30:27 -07:00
Brad Jorsch
ce79d607c7 libs/Message: Improve documentation
Among other things, this removed mention of MediaWiki classes from the
library and adds a README.md that attempts to define constraints for
interoperability between libraries producing MessageValues and formatter
implementations that are expected to handle them.

This also renames "TextParam" to "ScalarParam", as that seems a more
accurate name for the class.

Change-Id: I264dd4de394d734a87929cf4740779e7b7d0e04a
2019-08-29 17:04:01 -04:00
Tim Starling
09cd8eb080 MessageFormatterFactory
An injectable service interface for message formatting, somewhat
narrowed compared to Message.

Only the text format is implemented in this framework so far, with
getTextFormatter() returning a formatter that converts to the text
format. Other formatters could be added to MessageFormatterFactory.

Bug: T226598
Change-Id: Id053074c1dbcb692e8309fdca602f94a385bca0c
2019-08-28 12:28:05 +10:00
jenkins-bot
a34c284b1a Merge "Revert "rdbms: make LoadBalancer::reallyOpenConnection() handle setting DBO_TRX"" 2019-08-25 17:09:21 +00:00
Urbanecm
6ded91313c Revert "rdbms: make LoadBalancer::reallyOpenConnection() handle setting DBO_TRX"
This reverts commit 45831e619c.

Reason for revert: Caused beta not work at all.

Bug: T231162
Change-Id: Icc5c1fa0dc01082a622641ad96c22c939cd56d48
2019-08-25 16:39:33 +00:00
Aaron Schulz
bee2a20903 objectcache: make WANObjectCache::relayPurge() actually use $holdoff
Fixes a regression introduced by 70547f3fa3 in 2016.

Change-Id: I5996b63c31ac3b382e838a6858b8585bdc8c585c
2019-08-24 22:41:39 +00:00
jenkins-bot
11a26311da Merge "rdbms: make LoadBalancer::reallyOpenConnection() handle setting DBO_TRX" 2019-08-24 20:55:48 +00:00
Aaron Schulz
45831e619c rdbms: make LoadBalancer::reallyOpenConnection() handle setting DBO_TRX
Make LoadBalancer::reallyOpenConnection() handle initializing DBO_TRX
instead of Database::__construct().

Also:
* Avoid having the "catch" block appear like it returns a
  half-constructed Database.
* Use the variable name $conn instead of $db to be consistent
  throughout the class. Only send Database::__construct() parameters
  that it recognizes instead of mixing in setLBInfo() data.

Change-Id: Iffc3d1d0713051a164adb51a4c4ee12e4ac887c3
2019-08-24 20:06:17 +00:00
Aaron Schulz
94eb0c946f objectcache: improve BagOStuff arithmetic method signatures
Make the default $init value for incrWithInit() be $value.
This is far less suprising and also makes the operation
easier to replicate without conflicts.

Make decr() definitions more explicit since various cache
drivers do not handle negative incr() values (e.g. memcached).

Change-Id: I2b8d642656cc91c841abbd7a55d97eba101b027a
2019-08-24 12:50:37 -07:00
Aryeh Gregor
a83b33582a TempFSFileFactory service
This replaces TempFSFile::factory(), which is now deprecated.

Change-Id: I9e65c3867e26c16687560dccc7d9f3e195a8bdd6
2019-08-21 16:26:05 +03:00
jenkins-bot
90d6f495bd Merge "objectcache: fix WRITE_ALLOW_SEGMENTS in BagOStuff cas() and add() methods" 2019-08-20 15:50:58 +00:00
Aaron Schulz
942f708b61 objectcache: fix WRITE_ALLOW_SEGMENTS in BagOStuff cas() and add() methods
Add MediumSpecificBagOStuff::getValueOrSegmentList() helper method.

Also:
* Use $keysMissing variable correctly in CachedBagOStuff::getMulti()
  to avoid extra overhead.
* Optimize mergeViaCas() when the current value matches the new one.

Change-Id: I5c4bd74379bc459216ac0278150ce3aecff3b851
2019-08-20 10:55:42 -04:00
jenkins-bot
609091451b Merge "objectcache: make newFromId()/newWANcacheFromId()/getDefaultKeyspace() private" 2019-08-20 13:40:26 +00:00
Aaron Schulz
d421f22eb6 objectcache: make newFromId()/newWANcacheFromId()/getDefaultKeyspace() private
Change-Id: I30b99f23dbd9637ca8178d3a3650b4c38ec43e7d
2019-08-20 13:11:27 +00:00
Aryeh Gregor
d7d5fb81b5 Tests for TempFSFile
These are in preparation for making a TempFSFileFactory service, thus
the odd break-up into two files. I split it into a separate commit so
that we could verify that the same tests pass before and after the
conversion to service.

Tests cover everything except getUsableTempDirectory() (which I don't
see how to test), and register_shutdown_function()-related stuff (which
seems actually impossible to test without starting a new PHP process).

Change-Id: If61b7ea3e332adc2bceefc8e6879a9e9443c99dd
2019-08-20 11:25:02 +03:00
jenkins-bot
22a0bde74b Merge "Add recursion check to createService()" 2019-08-19 15:36:40 +00:00
Aryeh Gregor
a995d9be1d Add recursion check to createService()
This will throw when trying to create a service while already in the
process of creating that same service, i.e., if there's a circular
service dependency. This would have saved me a whole bunch of debugging
time. :)

Change-Id: Id148d4f221f35f4069f3e0ab0069d13ca271df3d
2019-08-18 10:51:24 +03:00
Amir Sarabadani
4d10bb14e8 Drop Oracle and Mssql
After approval of RFC T191231, we are going to drop oracle and mssql
and it will be possible to bring back the support using the abstract schema

Adding to release notes will be done in a follow-up

Bug: T230418
Change-Id: I90bd5cfcc3e18011b193c965fdb1fa54675040b5
2019-08-14 11:31:41 +00:00
Aaron Schulz
49025f52b2 objectcache: clean up MemcachedBagOStuff expiry handling
Partly a follow-up to 88640fd902.

Use real time in changeTTL() tests to fix all remaining
failures for BagOStuff sub-classes.

Change-Id: I537d665d6c8770a68a5a79233a913f1714881dfb
2019-08-10 12:12:12 -07:00
Aaron Schulz
88640fd902 objectcache: fix failing tests for non-HashBagOStuff backends
Use real time for testing absolute expirations with changeTTL().
Otherwise, backends like memcached or redis will fail since
they do not use the mock time.

Also:
* Make SqlBagOStuff actually override changeTTLMulti() by
  using the right method name
* Check TTL_INDEFINITE more explicitly for clarity
* Rename TTL conversion methods for clarity
* Use isRelativeExpiration() in MemcachedBagOStuff

Change-Id: I9365ceb31d4e7bef65906363d42b8c3020a66346
2019-08-07 14:07:22 +00:00
Umherirrender
2664eeb632 Clean up spacing of doc comments
Align the doc stars and normalize start and end tokens

Change-Id: Ib0d92e128e7b882bb5b838bd00c74fc16ef14303
2019-08-05 22:29:50 +00:00
Aaron Schulz
47aa48f073 objectcache: make "busyValue" stricter to avoid callback ambigiuity
Change-Id: I01a1503ff5b37d65ef148fef79270505d8eb3146
2019-07-27 02:21:56 -04:00
Aaron Schulz
023c73f612 rdbms: normalize Database open() code and error handling
Mainly:
* Use oci_new_connect() for Oracle to avoid broken connection reuse
  similar to the PGSQL_CONNECT_FORCE_NEW flag in DatabasePostgres
* Set 'client_min_messages' unconditionally for PostgreSQL
* Factor out Database::getConnectExceptionAndLog() helper method
* Use the same style of query() calls in DatabaseOracle::open() as
  the other subclasses
* Make sure the Database driver handle field is null on failure
  instead of false for sanity

Also:
* Disallow changing of Database handle DBO_* flags after construction
  where it does not make sense to change them
* Do not mention DBO_* flags meant for non-config use in $wgDBservers
* Ignore DBO_PERSISTENT for SQLite if DBO_TRX is also set for sanity
* Remove $wgDBOracleDRCP variable to discourage careless automatic
  setting of DBO_PERSISTENT that breaks LoadBalancer assumptions

Change-Id: Iea948f7f872294ea8fc5d897fc10c9d29b7141d5
2019-07-26 15:24:28 +01:00
jenkins-bot
f577358f43 Merge "objectcache: let BagOStuff::getWithSetCallback() callbacks modify the TTL" 2019-07-25 23:26:05 +00:00
Tim Starling
d8e06a46a8 MimeAnalyzer: fix ZIP parsing failure
unpack() actually returns an array with indexes starting from 1, not
zero, so unpack(...)[0] gives a notice and always returns null. It is
lucky that ZIPs normally have zero-length comments, so this would have
had little impact on file type detection aside from log spam.

Also, add a check to make sure the unpack() will not read beyond
the end of the file. Without this, unpack() could generate a warning.

The bug was introduced by me in f12db38048.

Add tests. The test files were generated by appending an EOCDR signature
and some extra bytes to 1bit-png.png.

Bug: T223728
Change-Id: I6fab63102d1d8eea92cdcce5ab6d1eb747a0a890
2019-07-25 13:40:18 +10:00