This lets callers check whether a user recently change a DB and
possibly try harder to reflect certain changes or refresh caches.
Also simplified some logging code a bit.
Change-Id: Ia1168cf0d46cfdee046838ce4c5a6294e4d81760
* This is more consistent with the name flushReplicaSnapshots().
* Also add it to IDatabase, which defines the modern OLTP methods
of DatabaseBase. This methods is useful for callers that get
locks or wait for a replica DB to catch up before querying it.
Change-Id: Ia31e480bb9ccf461bf05ede4278920918eec4f16
This avoids slave lag and makes query time account easier.
It also avoids table-level autoinc locking and slave drift
with statement-based replication in some setups.
Also refactored the use of $wgCommandLine mode in
DatabaseBase slightly, so that it can be injected.
Change-Id: I2dba6024ecf32c9ee24a3080cce3b02568c1458b
* Fixed bug in LBFactory::beginMasterChanges where
untracked DBs that then get written to where not in the
transaction round. The call to onTransactionResolution()
right after commit() was also problematic, so a flag was
added to handle that.
* Add LBFactory::setWaitForReplicationListener(), which
fires at what is often a good point to run deferred updates.
* Add DatabaseBase::clearSnapshot() for committing no-op
transactions.
* Add IDatabase::setTransactionListener() for persistent
callbacks. Monitoring commits can be used to find a point
where deferred updates can run.
* Follow-up commit will address DeferredUpdates.
Change-Id: I4589e6f3ae40b1c362601416db48857fb89840bf
This is in intended to replace the DataUpdate transaction round logic.
It could also be useful for doing transaction rounds in maintenance
scripts.
Also renamed $db => $conn in a few LB methods for consistency.
Change-Id: If21c2ba5e8bac48c250b96137279e7edaa8289f7
Previously it would commit() since the __destruct() call happens
as an exception is thrown but before it reached MWExceptionHandler.
Change-Id: I3d4186eb9ec02cf4d42ac84590869e2cf29b30b4
* Disallow $ignoreErrors in query() on deadlocks, since that would otherwise
silently rollback all changes from any other callers.
* Move recoverability checks for disconnects to canRecoverFromDisconnect().
* The first write of a DBO_TRX transaction is now considered recoverable.
* Run onTransactionResolution() callbacks on disconnect/deadlock rollback.
Some DeferrableUpdate need this to know to abort.
* Disallow $ignoreErrors on disconnects considered unrecoverable. This
makes it so that query() callers cannot cause writes from other callers
to be silently lost, which is hard to reason about.
* Moved ping() logic to simple reconnect() method and ping() simply do
a dummy SELECT, which triggeres reconnection if safe. Previously,
ping() might cause subtle partial transaction loss.
* Remove ping() from strencode(), which would cause partial transaction
loss where it actually reached.
* Remove mysqlPing() per https://bugs.php.net/bug.php?id=52561.
Bug: T142079
Change-Id: Ifb7f772ae849d67c0d92240a115c3f392e252937
The IDs will be included in MySQLMasterPos objects and,
if specified by config, in slave lag wait methods.
Bug: T135027
Change-Id: I1dfc0210b715b449ec07760c712d0267763f2697
* Added onTransactionResolution() DB method.
* Use this method so that file unlocks fire on unlockAndRollback()
as well as on DB errors (via MWExceptionHandler::handleException).
This prevents locks from getting stuck for minutes when deadlocks
happen, since the LockManager::destruct() method is not reliable.
* Fix broken reference counting which always released locks on the
first unlock() call, even if there were 2+ lock() calls.
* Added some type hints to IDatabase methods.
* Fixed DatabaseBase::__destruct() logging to include all callbacks.
Bug: T132921
Change-Id: I684706957f4d794cb6fe61505b0d26b7893de706
This reduces the runtime of database-bound tests by about 40%
(on my system, from 4:55 to 2:47; results from Jenkins are
inconclusive).
The basic idea is to call addCoreDBData() only once, and have
a addDBDataOnce() that is called once per test class, not for
every test method lie addDBData() is. Most tests could be
trivially be changed to implement addDBDataOnce() instead of
addDBData(). The ones for which this did not work immediately
were left out for now. A closer look at the tests that still
implement addDBData() may reveal additional potential for
improvement.
TODO: Once this is merged, try to change addDBData() to
addDBDataOnce() where possible in extensions.
Change-Id: Iec4ed4c8419fb4ad87e6710de808863ede9998b7
* Clean up return value types and docs.
* Handle master switch-over better w.r.t the job queue due
to binlog name changes (the host portion). Previously the method
would fail and trigger read-only mode when waiting on former
master positions. Assume the the switch-over was done properly
and thus return immediately.
Bug: T126436
Change-Id: Ib8c05a5c72d03a5c98e41b23c5653fc194b6d130
Before, it just used the immediate master entry, which could be another
slave. In that case, it may not even exists at all.
Bug: T119648
Change-Id: Iea970b81ad2c9855aafcccf0bb0662fc0b3a8d4d
If a callback set DBO_TRX, make sure it gets unset if that
is not what the DB was set to before the callbacks ran.
Change-Id: I79b649de02e821494d7f88e8979764ec8a9d0c43
* It now works for users without using sessions.
Sessions should not be cluttered with things
unrelated to authentication and tokens.
* Public services doing api.php requests on behalf
of a users only need to set XFF headers (as normal)
for position wait logic to trigger. They can opt out
of ChronologyProtector via a new HTTP header
"ChronologyProtection: false".
* Requests across subdomains, such as the SUL2 handshake
for CentralAuth on account creation, now have position
wait logic applied. This helps avoid anomolies were a
row just written in the last request may not be seen.
* Use merge() to avoid rolling back master positions if
the user has multiple tabs open and doing writes at once.
* $_SESSION global state is gone from ChronologyProtector.
* Cleaned up post-send LBFactory::shutdown() logic for
avoiding master position writes with an explicit flag.
* Use 'replication' debug log group in more places.
Bug: T111264
Change-Id: Ib25d05994d62b25c2f89e67b7f51009c54f4bca8
* This avoids bothering with MASTER_POS_WAIT() if an equal/higher
position was already successfully waited on by another process.
* Add DBMasterPos toString() and hasReached() methods, which
MySQLMasterPos already implemented and used.
* Moved more wfDebug() statements to the 'replication' log.
Change-Id: I423b5fe2da8d97889a6d204a635e351342de7649
Failed with "Catchable fatal error: Object of class
PDO could not be converted to string in".
The message I used was a rather arbitrary choice, but
I think it makes sense.
Bug: T114709
Change-Id: I0023fae3fa2a0c2b37cb3c34751706fe0d481d19
* wfGetDB( DB_MASTER )->getLBInfo() now shows the master flag
* A corresponding slave flag was also added
* Added a few badly needed LBFactory/LoadBalancer tests
Change-Id: I9254b12cff63af7d754a3a14c5db44276f58d280
* This should *only* ever be used internal for error suppression,
such as in the exception reporting methods. Having it public
means callers have to worry (in theory) about whether the
DB handles errors one way or a totally different way even
though there is really only meant to be one.
Change-Id: I5916830d1bd53ee948308f394e55c17dd515ad33
* Remove redundant @licence/@license from test suite files.
They already have full licence headers. And @licence raises a
warning in Doxygen.
* Fix weird messes of comments inside comments and other things.
Change-Id: I38da8ca76330f72b8dc22b0ecf1ea69d5ea55ede
* The data directory can now be set via the construction params
* A standalone factory method now replaces the subclass
* Also made mDatabaseFile protected
Change-Id: I1791fd4f630e5c121fa7f68f473411a7c12d0c97
Amends to Database::makeList() to check if 'NULL' is present in array while
building IN clause. Presently a NULL in array intended to build IN clause, is
treated as a usual value. This creates
problem:https://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#function_in
Therefore null is appended at the end of the IN clause as " $field IS NULL"
rather than within the IN clause.
DatabaseSQLTest.php: Some test cases added for the modified behaviour.
Bug: T50853
Change-Id: Ic255c37e9702fd44d8173027ab8036ddc48f9d2e
- Added/removed spaces around parenthesis
- Added newline in empty blocks
- Added space after switch/foreach/function
- Use tabs at begin of line
- Add newline at end of file
Change-Id: I244cdb2c333489e1020931bf4ac5266a87439f0d
Was not completely removed and the remaining bits were mostly
broken. $wgProfiler['output'] = 'db' is the proper way.
Fixes T75917
Change-Id: I36565e2372db2ed49b219cf533ec433e8111c52f
This was used in 2 special classes, the logger classes and spread
to a few other random classes.
Afaik this has no meaning. Is for something we don't use, and
goes against the meaning of '@section' in Doxygen, which we do
use.
In Doxygen output, all LICENSE references became links to
ProfilerXhprof (the one Doxygen encoutered first).
Bug: T72328
Change-Id: Icc7c443245c70bc0f549bee7d105eef5691c864d
The "@group Database" is only effective when it is in the
comment immediately before the class definition.
Merge the two comments into one.
Bug: 69612
Change-Id: Icfc7c9587534115154d90e20d510c854359fa8f0
Internal calls to this method show up in my XDebug profiling as
the second most called method (after array_key_exists). Called
10000 times. Sure, it's very cheap and clearly not a bottleneck.
But just not necesarry to have a method for that. The benefit of
having a method is to have a dynamic prefix that can change
depending on other things in an object. But I think this is not
a good idea for a prefix.
Since this is protected it is part of the contract of this class.
I already cleaned known subclasses. Should be @deprecated for
some time and can be removed later.
Change-Id: I46a0d7e072d0a69e9aef5f77e92044b62e3d0ce7
MediaWiki default is "@return type Description", so set a type after
return and start the description with a capital letter. Also use the
more common spelling of boolean.
See http://phpdoc.org/docs/latest/references/phpdoc/tags/return.html for
more about @return
Change-Id: I4e5198822fe92836f9cef9918a9fc1a1a1e0a043
- Added/removed spaces around parenthesis
- Added space after switch/if/foreach
- changed else if to elseif
Change-Id: I99cda543e0e077320091addd75c188cb6e3a42c2
Currently the first row of the result
is used to return the number of fields.
This fix corrects that calculation, which has returned
twice the number of fields before. The reason for that
was that the resulting data array has the following form:
array() {
[0] => value_0
["field_name0"] =>value_0
[1] => value_1
["field_name1"] =>value_1
...
Furthermore the constant 0 (int) is returned if the result
set is empty. This issue should be corrected in the future,
since the number or fields should be answered in a metadata context
independent of the data stored in the relation.
Bug: 65578
Change-Id: I399c7c857dcbd774cc4eb6102bbfe8a8764b07c9
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.
Change-Id: Ic8aaf0a93796b97d0fa4617c1f86ff59f4b36131
Instead test that LBFactory::getLBFactoryClass() does the right thing.
In LBFactory::getLBFactoryClass(), the method now requires a configuration
array as a parameter, and returns the class name as a string.
LBFactory::singleton() now passes in $wgLBFactoryConf into getLBFactoryClass(),
while tests can use test configuration.
Only use of LBFactory::getLBFactoryClass() in core is in LBFactory::singleton().
No extension in gerrit uses getLBFactoryClass() and I can find no use of it
when searching places like Github and Google, so it should be safe to make
such change in LBFactory.
Bug: 59105
Change-Id: I71ae7df16bc5c214b9389125140bca5ce68d274c
DatabaseMysqlBase::fieldType() is used in the maintenance script
convertLinks.php for upgrading from MediaWiki 1.2 or earlier. This
script was failing when the mysqli extension was enabled.
Follows-up b1cc3511fd, b2d64432ee, and 00299ecb95.
Change-Id: I5d888f4dd344ecba15506023d35f6d2d6e24cfd8
- Removed double spaces
- Added space after if/switch/foreach
- Removed space on elseif
- Added space around parentheses
- Added newline at end of file
- Removed space before semicolon at end of line
Change-Id: Id40b87e04786c6111e6686d7f7eea1e588bdf37d
According to the PHP manual, it is the recommended way to set the charset.
- mysql extension has it for MySQL >= 5.0.7
- mysqli extension has it for MySQL >= 5.0.6
or if using mysqlnd.
Change-Id: I8cd2f97fcad4b045c6f99ff894254847b13c6878
We were missing a method to list out views defined in a database. This
patch adds in MysqlBase::isView() and MysqlBse::listViews().
Since listViews() cache its result in DatabaseBase::$allViews, we also
introduce a final DatabaseBase::clearViewsCache() to let us clear the
per process cache.
Finally, fixed fatal error when duplicating VIEWs in MySQL.
bug: 43571
Change-Id: I8650baa4b721fe69ea3e1d557dd76745c0c7754e
DROP INDEX is database-wide (and not table-specific) in SQLite,
so we need to add a rule in our MySQL to SQLite schema convert.
"DROP INDEX foo ON bar" becomes "DROP INDEX foo".
Change-Id: Ie10784d166ed5a7f6ea52cf4f3a812fb7878a744
Previously mysql_real_escape_string() was improperly used which escapes
for inclusion in a plain string rather than an identifier.
Also adds basic test support for the DatabaseMysqlBase class.
Co-Authored by: Antoine Musso <hashar@free.fr>
Bug: 55427
Change-Id: Ic045e195c89d8d5d0f0edbda0cd1df781de7025c
Follow up Id7fc6652
Pass function name to begin/rollback/commit
Use __METHOD__ as default param (see I86cbdeab)
Change-Id: I9eb326c035d4a604db5b3492f090d8dd9d21c920
* All callers are calling it this way and it breaks
hhvm when declared statically and called dynamically.
Change-Id: I894e615fd828615384aa8457a16a759c8aa416ef
Created a DatabaseTestHelper class, which extends DatabaseBase and
implements STUBs for the interface methods and abstract methods
Change-Id: I9965b3604e78b2722077a35a7b4ce62a5bcb370e
Changed some gettype == 'array', 'object' and similar to is_array,
is_object or similar
Output of gettype must not stable across versions and it is slow
Change-Id: I07bfc063b03be1200989dd6facee66b35ab51d77
using a type cast instead, otherwise an empty string gets converted to
true when set with bool type.
Change-Id: I4161de32151a649161ab4084e80cafe4a462c7d5
Fix almost all occurences of the following sniffs:
Generic.CodeAnalysis.UselessOverridingMethod.Found
Generic.Formatting.NoSpaceAfterCast.SpaceFound
Generic.Functions.FunctionCallArgumentSpacing.SpaceBeforeComma
Generic.Functions.OpeningFunctionBraceKernighanRitchie.BraceOnNewLine
Generic.PHP.LowerCaseConstant.Found
PSR2.Classes.PropertyDeclaration.ScopeMissing
PSR2.Files.EndFileNewline.TooMany
PSR2.Methods.MethodDeclaration.StaticBeforeVisibility
Change-Id: I96aacef5bafe5a2bca659744fba1380999cfc37d
Database::select will return false if a query fails and the DB object
is set to "ignore errors" mode. ORMTable now detect this and throw
a DBQueryError, overriding the ignore errors mode.
This change also adds a dummy implementation for ORMTable to ORMTableTest,
so tests can be run on the base implementation.
Change-Id: I5d87295626c0800c38f807d19becb312ba7cc956
The 'profiling' table is missing from expected tables when
$wgProfileToDatabase is set to true, since patch-profiling.sql
is applied while updating in this case.
Change-Id: Iedbe72dca1deda728304fb95fe19ef3b4bdad8de
1) DatabaseSQLTest::testMediaWikiTestCaseParentSetupCalled
DatabaseSQLTest::setUp() must call parent::setUp()
Failed asserting that an array has the key 'setUp'.
Change-Id: I6aa5cb09999e0cee9b36fedd9193cb5aa08914ad
- this is more generally useful than to be buried inside Wikibase client.
- it was in Wikibase/client/tests/phpunit/includes/store/EntityCacheTableTest.php
- patchset 2: add to test autoloader
Change-Id: I5bc41d6d205f28bcc5bf0c8a78b782c8888c18eb
This change provides a workaround and test case for a problem
in the sqlite library:
SQLite truncates strings at ASCII value 00 aka \0. Strings
containing \0 need to be represented in hexadecimal form.
Reported to PHP as bug 63419
https://bugs.php.net/bug.php?id=63419
Change-Id: I2bbc445ffebd41e181edfc3201e6e5514de06142
ORMRow shouldn't apply the "IGNORE" modifier for insertions by default.
IGNORE means pretending the insert was successfull even if it wasn't.
That's not a good default, and for some databases (like sqlite), may
actually hide errors beyond key conflicts.
Change-Id: I8b00cd03a459419441195ed25091385371b027a5
Some class extending MediaWikiTestCase did not call its setUp method. We
most probably always want to do it since MediaWikiTestCase::setUp() does
garbage collection and might do more in the future.
Change-Id: I68dde370a62c8f4a779836ca0c4ad06844fdc916
This commit depends on the introduction of
MediaWikiTestCase::setMwGlobals in change Iccf6ea81f4.
Various tests already set their globals, but forgot to restore
them afterwards, or forgot to call the parent setUp, tearDown...
Either way they won't have to anymore with setMwGlobals.
Consistent use of function characteristics:
* protected function setUp
* protected function tearDown
* public static function (provide..)
(Matching the function signature with PHPUnit/Framework/TestCase.php)
Replaces:
* public function (setUp|tearDown)\(
* protected function $1(
* \tfunction (setUp|tearDown)\(
* \tprotected function $1(
* \tfunction (data|provide)\(
* \tpublic static function $1\(
Also renamed a few "data#", "provider#" and "provides#" functions
to "provide#" for consistency. This also removes confusion where
the /media tests had a few private methods called dataFile(),
which were sometimes expected to be data providers.
Fixes:
TimestampTest often failed due to a previous test setting a
different language (it tests "1 hour ago" so need to make sure
it is set to English).
MWNamespaceTest became a lot cleaner now that it executes with
a known context. Though the now-redundant code that was removed
didn't work anyway because wgContentNamespaces isn't keyed by
namespace id, it had them was values...
FileBackendTest:
* Fixed: "PHP Fatal: Using $this when not in object context"
HttpTest
* Added comment about:
"PHP Fatal: Call to protected MWHttpRequest::__construct()"
(too much unrelated code to fix in this commit)
ExternalStoreTest
* Add an assertTrue as well, without it the test is useless
because regardless of whether wgExternalStores is true or false
it only uses it if it is an array.
Change-Id: I9d2b148e57bada64afeb7d5a99bec0e58f8e1561
We can now do this since we finally switched to PHP 5.3 for MW 1.20 and get rid of the silly dirname(__FILE__) stuff :)
Change-Id: Id9b2c9cd2e678197aa81c78adced5d1d31ff57b1
This introduce the syntax from aliased table names for aliased field
names into the abstract database layer:
array( 'alias' => 'field' ) gives 'field AS alias'
This patch also includes changes to query pages, api and some more
places to show, how the new syntax looks in "production".
This allow us to remove the "AS" for Non-PostgreSQL databases, if we
want that.
Change-Id: I5f0de1c2f29092c173aec3de93ffdef436799e8d
* Removed unused and obsolete set() and safeQuery() functions.
* Removed unused deprecated constructor functions.
* Removed unused limitResultForUpdate() cruft function.
* Removed unused standardSelectDistinct(), it's better to just follow the standard for all queries.
* Removed other cruft functions unused by core/extensions.
* Made some internal functions protected.
Change-Id: I90be88ea740834a417a17d7751f1be7bac4eae4e
Currently the logic is the following:
$database = ( $format == 'quoted' || $this->isQuotedIdentifier( $database ) ? $database : $this->addIdentifierQuotes( $database ) );
This means that $database will *not* be quoted if $format is set to 'quoted',
which is the opposite of what we want here. Instead I just copied the code
used to quote $table just below and replace the variable.
This causes SQL syntax errors when the shared table name contain hyphens
since it is not quoted.
No RELEASE-NOTES entry since this should be backported to 1.19 once merged.
Patchset #2: added some tests
Patchset #3: fix tests for SQLite
Change-Id: Idda844cb470454a9a22001a31b6692de49892ca2
Patchset 2: attempt to fix sql error when using sqlite
Patchset 3: for great justice
Patchset 4: sqlite, not postgres... /facepalm
Patchset 5: joy, more sql divergence...
Patchset 7: handle inconsistencies in MWs database abstraction layer
Change-Id: I1948c4ad815008321801c93584eb249c1f597560
This is reverting the work done by MaxSem to support stored procedures
and stored function in MySQL. The reasons are:
- it is not needed yet
- tests are not functionals
- alter the stable include/db/Database.php and drop support for ';;'
So please create a branch to work on it and merge it back in trunk
once we have branched 1.19 :-)
I have opened bug 33654 to track this enhancement request.
Reverts r107376, r107994.
* Refactored DatabaseBase::sourceStream(), made it possible for descendant classes to alter its behaviour w/o having to redo it completely like Oracle does.
* MySQL class now supports specifying DELIMITER.
* Thrown away the mess of catering for double semicolon. If it's a problem, fix your .sql files!
* Haven't actually touched Oracle.
* Tests!
An existing db is not needed, since it uses a memory one.
Having this in Database group fails in DatabaseSqlite->listTables
since it is working with a MockDatabaseSqlite whose query() return true.
Following a live discussion with Catrope.
When using Database::makeList() in LIST_AND or LIST_OR modes, you can now
suffix the field name with an exclamation mark. It will negate the logical
boolean.
Example:
$db->makeList( array( 'field!' => array( 1,2,3 ) );
outputs:
'field' NOT IN ('1', '2', '3' );
$db->makeList( array( 'foo!' => array( 777 ) ) );
outputs:
'foo' =! 777
(note: tests not ran, please run them and ammend them)
