* Make sure cancelled onTransactionPreCommitOrIdle() callbacks do not
run if a transaction round is rolled back and then a second round is
committed. LoadBalancer::rollbackMasterChanges() now always calls
rollback(), which in turn always cleans up such callbacks.
* Remove error logging for rollback() calls when trxLevel = 0; this is
harmless and is sometimes hard to avoid in error handling anyway.
* Add more related unit tests.
Change-Id: I6bdefe8bf8b6630fc252b5bbafe4808758ba1684
SPDX released version 3 of their license list (<https://spdx.org/licenses/>),
which changed the FSF licenses to explicitly end in -only or -or-later
instead of relying on an easy to miss + symbol.
Bug: T183858
Change-Id: I4cbb98ea2bbacf46a57401dbba2bdb076fca1d3a
If the $var argument is provided, then it will make the resulting
count exclude rows where the value for that column is NULL.
Also add buildSelectSubquery() method and Subquery
wrapper class for use with select() for calculated tables.
Change-Id: I549d629af99afdf370602de095f7fba6d1546c37
A use declaration for a non-namespaced class in a non-namespaced context
causes a PHP warning.
Bug: T189302
Change-Id: I023e64c8194dd03cc3a1098e2d60c73f99bb02e3
* Database::factory() supports a $connect parameter, that defaults
to NEW_CONNECTED (current behavior) but can also be NEW_UNCONNECTED.
* Add tests asserting the type of various instances returned from
Database::factory().
* Clean up sqlite "conn" field handling to handle cases of it
not being set, just as other classes do.
* Add some comments about the return type of doQuery().
Change-Id: Ic0837cfdb35326c2045133d664abd29043d48c03
Follows-up b4eb1feed0, which inadvertendly replaced the setFlag()
test with the clearFlag() test.
Also move the test to the DatabaseTest.php file given it's only
executing and covering base class.
Change-Id: I2f4ed6c4eeba845eb67013e1ab7d2b2bde863119
This fixes an issue that arises because empty( "0" ) is true in PHP.
The new behavior rejects any conditions that are not strings or arrays,
and lets $conds = "0" be passed to the databases as WHERE 0.
Some databases may reject this as invalid syntax, which is the expected
behavior here, instead of silently ignoring the 0, causing no condition to
be applied to the query.
Bug: T188314
Change-Id: I5bc4d7f41221a886c85e54d9da67c4c095a7d9ce
An INSERT SELECT in MySQL/MariaDB is unsafe for replication if a column
is getting values from auto-increment, statement-based replication is in
use, and the default innodb_autoinc_lock_mode is set.
I9173f655 added checks to force non-native insertSelect for the
statement-based replication and innodb_autoinc_lock_mode != 2 case, but
determining whether a column is getting values from auto-increment is
too hard to do automatically there.
Instead, let's add a flag to let the caller hint that the query isn't
getting any auto-increment values. And use it in MysqlUpdater when
appropriate.
Bug: T160993
Change-Id: If70450a64aa3bcbf763c62838bb21306d124ae3d
It would be easy for a call to nonNativeInsertSelect() to generate an
INSERT that's too big for the database to actually process. Add batching
to try to avoid that.
Bug: T160993
Change-Id: I1de994208d95926f0d75c0d7cab7b5fe1dd565c3
ChronologyProtector uses these classes to briefly store positions
and everytime the fields change then errors can happen when old
values are unserialized and used. Use a simple two-element map
format for serialized positions. The fields are recomputed back
from the data map.
Values from before this change will issue the warning
"Erroneous data format for unserializing". To avoid that, bump
the ChronologyProtector key version. Future field changes will
not require this.
This change should be deployed on all wikis at once.
Bug: T187942
Change-Id: I71bbbc9b9d4c7e02ac02f1d8750b70bda08d4db1
* Trim whitespaces after opening and before closing parentheses and
brackets.
* Ensure by test case that the invalid "url (" will not become the
valid "url(" by minification.
This change also prevents the parsing problem in Firefox 57.0
https://bugzilla.mozilla.org/1418152 which is fixed in Firefox 57.0.1.
Change-Id: I804477ba7c6363f0e964fc8c7c0bc74d2d4c1a0d
Named locks are session-level constructs and this transaction agnostic.
Also make lockIsFree() a bit more consistent when the thread has the
lock itself.
Change-Id: Ief51196161bbc50c798740f3c738fd0e39880508
The binlog file/pos where only being used in __toString() for the GTID
case. Make that method use the GTID set instead and avoid querying the
old-fashioned binlog fields all together in that case. The STATUS
queries involve some global lock contention.
Bug: T180918
Change-Id: I18123a702e4f554b87bf5f90017b248062e73049
Deprecate the unnamespaced version and move it to includes/compat.
Bug: T147167
Depends-On: I39c805bfb98b32f32f3d0dc1eee9e823afe1c21a
Change-Id: I3780c7adf51683f3f7adb35a88f9a25a0a2e2530
* Update replace()/upsert() to combine the affected row
count for the non-native case
* Also make replace() atomic in the non-native case,
similar to how upsert() already works
Change-Id: I6c9bcba54eca6bcf4a93a9b230aaedf7f36aa877
* Make sure all unique keys specified have all their values
provided to avoid large bogus DELETEs. Do not ignore them
in such cases either, as that would cause inconsistencies
between the native and non-native case. Use an exception.
* Make ChangeTags caller clearer that the list of indexes
is not a list of fields for a single index. Also, avoid
mentioning indexes for values not defined in the new
records, as this causes errors or inconsistencies with
the native vs non-native case.
* This also fixes the "Undefined index: ts_log_id" error
when running unit tests on postgres.
Change-Id: I30263df22066bd6d4836202b1bcad5d1aa1e7383
This PHP extension was deprecated in PHP 5.5 and removed in PHP 7.0.
The newer MySQL driver 'mysqli' is the default driver since MediaWiki
1.22 and 'mysql' was deprecated in MediaWiki 1.30.
Bug: T120333
Change-Id: Icff7a63ab47ae1483de87ba6486945974699362b
Redundant given this is the project-wide license already,
especially in file headers that already include the GPL license
header.
This and other minor fixups based on feedback from Ie0cea0ef5027c7e5.
* Add @file where missing.
* Move @ingroup and @deprecated from file to class doc where needed.
Change-Id: I7067abb7abee1f0c238cb2536e16192e946d8daa
Removing unnecessary XML declaration prolog when embedding SVGs
as data URI and adding more complex elements & attributes to test.
Bug: T175318
Change-Id: Ibec8c2ab498554a6ad33b4420ae6258dcc80fcdf
When parsing an incomplete string literal or regex literal at the end of a file,
$end would be set to an offset higher than $length, because the code
speculatively increases $end without correcting for this scenario.
This is due to the assumption that the strcspn() search will end because
an end character was seen, instead of simply ending because the string
doesn't have any further characters.
Bug: T75556
Change-Id: I2325c9aff33293c13ff414699c2d47306182aaa6
Convert newlines and tabs to spaces (which can be unencoded),
and consolidate runs of multiple spaces into a single space.
Also remove any leading and trailing spaces that might result
(most files end in a newline, for example).
Follow-up to Iddc98332d. This was reverted as dependency for the
original issues causing patch, but works perfectly fine.
Bug: T175318
Change-Id: Ic25d73caeb42ba7a32ef304806d401047b5a1997
Clean up use of @codingStandardsIgnore
- @codingStandardsIgnoreFile -> phpcs:ignoreFile
- @codingStandardsIgnoreLine -> phpcs:ignore
- @codingStandardsIgnoreStart -> phpcs:disable
- @codingStandardsIgnoreEnd -> phpcs:enable
For phpcs:disable always the necessary sniffs are provided.
Some start/end pairs are changed to line ignore
Change-Id: I92ef235849bcc349c69e53504e664a155dd162c8
> There was 1 error:
>
> 1) HashBagOStuffTest::testEvictionAdd
> A non-numeric value encountered
>
> tests/phpunit/includes/libs/objectcache/HashBagOStuffTest.php:106
This warning is new in PHP 7.1, however the behaviour is not new.
This code was already not behaving as it should have. Concatenation
preceeds addition/multiplication. As such, this was producing
(int)-10 each time instead of (str)"key0" through (str)"key10".
Change-Id: Ibb1a59e373740772f02dfec77ee7ebd9d181d852
Also removed useless line from testLockTSE(). That would have needed
to be using $this->internalCache and those locks are freed immediately.
Bug: T180035
Change-Id: Ida1a923f779aaf8410da76643457d2200da6cb20
In the off chance something called this, it would break all
sorts of code that expects that either query result functions
either succeed or throw an error.
Callers are not expected to have to check if the result of
a query is meaningful or false due to an error.
Change-Id: I0b4fe1403f55a399ffd40817ed12f857087d6f83
The SQL standard supports parenthesized joins like
a JOIN (b JOIN c ON (...)) ON (...)
But it doesn't support parenthesizing a single table name, i.e. a
one-table "join", like
a JOIN (b) ON (...)
Detect the degenerate single-table case and omit the parentheses.
Bug: T181674
Change-Id: I82cacd80465092aa67ff19bdcfd6682001bf12ab
Callback code that happens to make use of $oldValue might not be able
to handle missing, new, or changed fields due to key version changes.
Overhaul testGetWithSetCallback_versions() to be cleaner and cover the
case of unversioned => versioned keys.
Change-Id: If108a73078c530c985d30bdadcbfa9ddd53dc2be
Use of microtime() is now just for baselines, and it is no longer
assumed to be increasing with each call. Such an assumption is
particuliarly bad on Windows.
I've done 100X run rounds with now failures on Windows.
Change-Id: Ica2a47982495bc95b10ca507414972744ea9507e
This simply involves passing it through to the set() call
Also added some related commons to adaptiveTTL() involving
usage of this option.
Change-Id: Id5833a5d4efb6cad2eb646832e5b0188e86e12fc
This stores values for very short times while the main
value is a tombstone in case of particularly high traffic.
Also make mutex keys expire immediately on unlock.
Change-Id: I4ec5cf7f8b49239fdd2518e5d955534877a0f7ee
SQL supports parentheses for grouping in the FROM clause.[1] This is
useful when you want to left-join against a join of other tables.
For example, say you have tables 'a', 'b', and 'c'. You want all rows
from 'a', along with rows from 'b' + 'c' only where both of those
exist.
SELECT * FROM a LEFT JOIN b ON (a_b = b_id) JOIN c ON (b_c = c_id)
doesn't work, it'll only give you the rows where 'c' exists.
SELECT * FROM a LEFT JOIN b ON (a_b = b_id) LEFT JOIN c ON (b_c = c_id)
doesn't work either, it'll give you rows from 'b' without a
corresponding row in 'c'. What you need to do is
SELECT * FROM a LEFT JOIN (b JOIN c ON (b_c = c_id)) ON (a_b = b_id)
This patch implements this by extending the syntax for the $table
parameter to IDatabase::select(). When passing an array of tables, if a
value in the array is itself an array that is interpreted as a request
for a parenthesized join. To produce the example above, you'd do
something like
$db->select(
[ 'a', 'nest' => [ 'b', 'c' ] ],
'*',
[],
__METHOD__,
[],
[
'c' => [ 'JOIN', 'b_c = c_id ],
'nest' => [ 'LEFT JOIN', 'a_b = b_id' ],
]
);
[1]: In standards as far back as SQL-1992 (I couldn't find an earlier
version), and it seems to be supported by at least MySQL 5.6, MariaDB
10.1.28, PostgreSQL 9.3, PostgreSQL 10.0, Oracle 11g R2, SQLite 3.20.1,
and MSSQL 2014 (from local testing and sqlfiddle.com).
Change-Id: I1e0a77381e06d885650a94f53847fb82f01c2694
class WANObjectCache (methods: 40% -> 67%, lines: 65% -> 88%)
* Allow indirect coverage of protected/private utility methods from
existing tests.
* Add basic test for setLogger().
* Add basic test for newEmpty().
* Add basic test for getQoS().
Change-Id: Ifb79ed2ff1febbd2f5477b8ed6319992ce88eb29
Convert newlines and tabs to spaces (which can be unencoded),
and consolidate runs of multiple spaces into a single space.
Also remove any leading and trailing spaces that might result
(most files end in a newline, for example).
Bug: T175318
Change-Id: Ic66c6acb37079cae84dd80ab2d5f2c829cf2df96
When a URL (data: or otherwise) contains quotes, don't wrap it in
double quotes (") but in single quotes ('). This then allows
us to unencode double quotes (") in the data URI embedding of SVGs.
Bug: T175318
Change-Id: I3e7eab64e1c3e82066014fb594f82d786983ce90
This is to remove confusion with the MediaWiki Block class.
All instances of isValidBlock within MediaWiki core have been updated.
Usage of this function will be more widespread with this patch:
https://gerrit.wikimedia.org/r/#/c/349457/
Change-Id: Ice1bdae3d16cf365da14c6df0e8d91d2b914e067
Reenable MediaWiki.WhiteSpace.SpaceBeforeClassBrace.NoSpaceBeforeBrace,
because the mentioned bug is fixed
Bug: T172933
Change-Id: I1593bdba2295ebed401b921f2beabed69dba7638
Only missing cases where 'USE INDEX' and 'IGNORE INDEX'.
The test doesn't do much since the underlying methods are no-ops
by default, but at least it ensures there are no PHP errors from
these branches.
We can later re-use some of these test cases in tests specific
to one backend.
Change-Id: Id004a2ae41efaa7a367f964013e25d98ecc591ff
If no writes started a transaction yet, the callback would run
but not commit (by design, joining the request round). Later
writes will then pile on top of it.
The point of this method is to avoid such cases, so this edge
case has been fixed.
Change-Id: I9b44b19261d679de4aff6e44a9cfeb4f684ce02e
* Add case for `$tables[0] == ' '`.
* Add case for `$tables == ''`.
* Add case for 'DISTINCT' option.
* Add case for 'FOR UPDATE' option.
* Add case for 'LOCK IN SHARE MODE' option.
* Add case for 'EXPLAIN' option.
Change-Id: I4a5f4754bc30d31ec35a085f39321fd358b6aa49
Many of the main methods here have, over the years, been split up
into several protected/private methods.
Change-Id: I1b8489b1c61c0294288442a0a0cd28c9fa77f82e
If someone writes an object into a BagOStuff, they typically expect that
later changes to the object will not affect the value stored.
MultiWriteBagOStuff's async write handling was violating this
expectation, which is potentially causing T168040.
Bug: T168040
Change-Id: Ie897b900befdc8998614af06f9339cd07665703e
* Move DatabaseTest and DatabaseSQLTest to libs,
and remove MediaWikiTestCase dependency.
* Refactor DatabaseTest to be a test of the Database abstract class,
not of whatever current DB backend is configured by LocalSettings.
- Remove most switches/conditionals and other tests for specific
database backends. Move those to individual test classes for
those backends instead.
- Some tests appear to have been integration tests for the PHP driver
and/or the db backend itself. Moved to a new DatabaseIntegrationTest.
- Now that only the abstract Database is invoked, the test runs a bit
faster (no real connections/queries).
* Add missing @covers tags, and remove or fix broken ones
(follows-up 26e52f0c49).
Change-Id: I9dc4a558e701d00e95789e7eb8e02926783b65ad
Beyond giving a more complete picture, this can be used to adjust
the PostSend criteria to log master queries under the restInPeace()
entry point.
Bug: T166199
Change-Id: If5cf704ba10291d0eba63e66b388666fde3ca649
Undo traces of a practice we carried over from past projects and
existing examples that is neither universal nor actively encouraged in
the MediaWiki codebase.
Bug: T139301
Change-Id: I5c9c89b72a45a44aa4264a5e57b003c1a86cdf6e
Co-Authored-By: Brad Jorsch <bjorsch@wikimedia.org>
The used phpcs has a bug, so the version 0.9.0 could not be enforced at the moment.
Will be fixed in next version, see T167168
Changed:
- Remove duplicate newline at end of file
- Add space between function and ( for closures
- and -> &&, or -> ||
Change-Id: I4172fb08861729bccd55aecbd07e029e2638d311
Similar to what WANObjectCache and CachedBagOStuff are already doing.
Also add missing tests for WANObjectCache (similar to those for CachedBagOStuff).
Bug: T167465
Change-Id: I1a0c9324726aa6a1b221def985773b1b819181fd
Backend cache queries are now avoided if they are fresh in memory.
Also factor out some code into private methods for clarity.
Change-Id: Ib343fde3dbf63f39e8cb09eca6278811a7d9738b
The warmup cache was not properly prefixed and was also using the entity
IDs instead of the cache keys. Thus, it effectively just wasted a
getMulti() query and resulted in the usual separate GETs anyway.
Added some unit tests for this.
Change-Id: I75b7a31214b515511856f9d95db32e8881d80ccc
Previously they were often being cut short due to the url pattern
ending at the first single quote, double quote or closing parenthesis
regardless of which of those started the url match.
Running benchmarkCSSMin.php before and after the change doesn't seem
produce consistent improvement or regression. Repeated runs with count=100
with and without this change both have a median between 2.6ms and 2.9ms
using PHP 5.6, and between 2.6ms and 2.8ms using HHVM 3.12.
Bug: T60473
Change-Id: I6d6a077ad76588f3ed81b1901a26b7e56d2157ee
Replaces \TestingAccessWrapper (defined in core) with
\Wikimedia\TestingAccessWrapper (defined in the composer package
wikimedia/testing-access-wrapper).
See https://gerrit.wikimedia.org/r/#/q/topic:librarize-testing-access-wrapper
for downstream patches.
The core version of the class is kept around for a while to avoid
circular dependency problems.
Bug: T163434
Change-Id: I52cc257e593da3d6c3b01a909e554a950225aec8
The default will remain PHPUnit 4.x due to PHP 5.5 support.
But, we should allow developers to run tests with newer PHPUnit
versions which are noticably faster (especially for code coverage
reports).
* <https://github.com/sebastianbergmann/phpunit/wiki/Release-Announcement-for-PHPUnit-5.4.0>
PHPUnit 5 deprecates the getMock() shortcut for getMockBuilder()->getMock().
It instead introduces the shortcut createMock() which has better defaults
than getMockBuilder(). For example, it sets 'disableArgumentCloning' and
other things by default.
Going forward, code should either use getMockBuilder directly and configure
it using the setter methods (instead of the confusing variadic arguments
of getMock) or simply use the new minimalistic createMock method. This patch
backports the createMock method to MediaWikiTestCase so that we can start
using it.
Change-Id: I091c0289b21d2b1c876adba89529dc3e72b99af2
Direct follow up for I1768646. The tests are copied from my patch
If985f33, with one important difference: Now the file extension always
wins. Mismatches are not fixed any more by this code. This is intended.
See the discussion at I1768646.
Change-Id: I06b873a808d58ae7e53272765b156ac0fa293c2c
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.
Change-Id: I46261416f7603558dceb76ebe695a5cac274e417
* This fixes keys based on some sort of change log.
Updates are wrapped in a mutex and keep track of the
last known good position.
* Make WANObjectReapUpdate class that cleans up title
related keys using the recentchanges table. This triggers
as a deferred updates on RC view.
Change-Id: I7f14b9ca2533032147e62b1a3cc004a23da86579
Test case for Opus file check so we don't regress!
Confirms the fix for T151352 that Opus files return
type 'audio' and not 'multimedia'.
Bug: T151352
Change-Id: Ib868793d90b60609ccec25726e6fbaa28bffebaf
This moves and refactors the ConsistentReadConnectionManager
from Wikibase into the core rdbms lib.
The refactoring also creates a generic ConnectionManager.
This relates to Iff20a22f9f2bc7ceefd6defc0ed9a494a6fe62c0
which introduced a DB factory / connection manager in
an extension revealing the need for this in multiple places.
Change-Id: I0c58e15aed5bed88323d18cb95e5008f8d3381c5
Causes HHVM 3.12.7 to segfault and or:
Fatal error: Stack overflow in includes/libs/objectcache/BagOStuff.php on line 754
This reverts commit b47ce21cec.
Bug: T150833
Change-Id: Ia52fdf65f5e386c0eaa1046328680fb5c8a9081a
This works by setting a callback to return the cache set
options. The callback will watch DB reads and create a
merged result from said usage.
This handles callers that are missing getCacheSetOptions().
Change-Id: Ia264f011e45e8cf105480955dad7e2c4c2357b73
The check for file existence before checking its MIME type was
building the path incorrectly, causing this piece of code to always
think the file does not exist. (The code to actually embed the file
was correct.)
This was not caught by the tests before of a bug in the test code,
which cancelled out this one.
Bug: T100958
Change-Id: I4469079f41d48052d0a02f98bb1ed02407abd54c
Because the process cache can be lagged by virtue of blind TTL,
the HOLDOFF_TTL might not be enough to account for it, so avoid
using it when already inside a callback.
Also split of the tests from the MediaWiki test class, so this
does not require DB access anymore.
Change-Id: I743a1233a5efc7f036fad140a9ff8a30b32f8f27
Upstream composer has replaced the 'hash' with a smarter 'content-hash',
but instead of re-implementing (or copy-pasting) all of that in MediaWiki
we can just compare the dependencies themselves, since that's all we
care about.
Bug: T147189
Change-Id: Ic2f22a82699e2b707b6ccb355605999a183a56a0
* The later resides in /libs with related files.
* Explose MimeAnalyzer as a service.
* Keep MimeMagic::singleton() as a b/c alias.
* MimeMagic::applyDefaultConfig() will bootstrap the service
with all of the old config, extension hook handler, and
detector command shell-out behavior.
Change-Id: Ie2695a52e7a3bcfda9f7fa83659a9ff31b372bc3
Follows-up 24200e8, 37df4f8. Test coverage still broken.
> Trying to @cover not existing method
> "WANObjectCache::getMultitWithSetCallback".
Change-Id: Ie4c9a998ea352f93f3c7dcdea25ea5c017ffbab5
