Commit graph

76 commits

Author SHA1 Message Date
Thiemo Kreuz
9c6473dc1d Reduce nesting by turning big if-else into guard clauses
Change-Id: If3d2f18af0cc7d0f8fe9f764bd4d0e4bae70e440
2020-07-24 11:55:31 +02:00
Umherirrender
fd1ed0c092 Pass function name to database functions
Useful for logging

Change-Id: Ia2160fb6be5fc93f28ab51f7ae23d7f078247481
2020-06-07 14:16:52 +02:00
Tim Starling
47a1619027 Remove terminating line breaks from debug messages
A terminating line break has not been required in wfDebug() since 2014,
however no migration was done. Some of these line breaks found their way
into LoggerInterface::debug() calls, where they mess up the formatting
of the debug log.

So, remove terminating line breaks from wfDebug() and
LoggerInterface::debug() calls.

Also:
* Fix the stripping of leading line breaks from the log header emitted
  by Setup.php. This feature, accidentally broken in 2014, allows
  requests to be distinguished in the log file.
* Avoid using the global variable $self.
* Move the logging of the client IP back to Setup.php. It was moved to
  WebRequest in the hopes that it would not always be needed, however
  $wgRequest->getIP() is now called unconditionally a few lines up in
  Setup.php. This means that it is put in its proper place after the
  "start request" message.
* Wrap the log header code in a closure so that variables like $name do
  not leak into global scope.
* In Linker.php, remove a few instances of an unnecessary second
  parameter to wfDebug().

Change-Id: I96651d3044a95b9d210b51cb8368edc76bebbb9e
2020-06-03 12:01:16 +10:00
Reedy
d53e91d272 Fix more PSR12.Properties.ConstantVisibility.NotFound
Change-Id: I94520b10d78a17ea8e965633dd475ea711f25c99
2020-05-15 00:33:32 +01:00
James D. Forrester
4f2d1efdda Coding style: Auto-fix MediaWiki.Classes.UnsortedUseStatements.UnsortedUse
Change-Id: I94a0ae83c65e8ee419bbd1ae1e86ab21ed4d8210
2020-01-10 09:32:25 -08:00
Brad Jorsch
b26248235b Add string casts when using array_keys() with SQL query conditions
Until I70473280, integer literals were always quoted as strings, because
the databases we support all have no problem with casting
string-literals for comparisons and such.

But it turned out that gave MySQL/MariaDB's planner problems in some
queries, so we changed it to not quote actual PHP integers.

But then we run into the fact that PHP associative arrays don't preserve
the types of keys, it converts integer-like strings into actual
integers. And when those are passed to the DB unquoted for comparison
with a string-typed column, MySQL/MariaDB screws up the comparison while
PostgreSQL simply throws an error. Sigh.

This patch adds string casting to direct uses of array_keys() to supply
values for such query conditions. It doesn't change uses where the field
being compared is a numeric field.

If anyone knows of a good way to find indirect uses of array_keys() for
passing as $conds to IDatabase methods, please do so!

Change-Id: Ie72ee33437d492904e1495b3f4ebb1fcf0118f49
2019-12-16 16:05:18 -05:00
Daimona Eaytoy
f18af0b61f Remove more Oracle and Mssql leftovers
Follows-up 4d10bb14e8 and 807d793ab9.

According to codesearch [0], these were the last usages. Note that this
patch leaves two constants in places, IDatabase::DBO_SYSDBA and
DBO_DDLMODE. These are public constants used "mostly for oracle" according
to the docs, but maybe we could find other use cases in the future (?).

[0] - https://codesearch.wmflabs.org/core/?q=oracle%7Cmssql&i=fosho&files=%5C.%5B%5Ej%5Cd%5D%7Cen%5C.json&repos=

Bug: T230418
Change-Id: Ibfb748b4b23b885a77f4de161af4bf2ab9649a89
2019-08-25 17:21:49 +00:00
Aaron Schulz
97cbaa349e Cleanup JobQueueDB::recycleAndDeleteStaleJobs() use of IDatabase::affectedRows()
Bug: T229456
Change-Id: Ie22085ddba66f122e59e93baaf9b53c76b5ce448
2019-08-08 10:59:26 +00:00
Aaron Schulz
43c78e83d7 jobqueue: migrate root job deduplication to the WAN cache
If the root job timestamp keys are lost or otherwise unknown, they
will now be deductively recached with the best known values as jobs
are popped and executed. This means the running any of many child
jobs of a root job can restore the root timestamp if it was lost.
This does not need to use the main stash given this fact.

Bug: T227376
Change-Id: Iae0f3af15803af048ff49f3bf281b2bde18c87f2
2019-07-13 15:38:17 -07:00
Aaron Schulz
aa4f4d6b53 jobqueue: make JobQueueDB stricter about broken job_params fields
Also rename throwDBException() to getDBException() and make the
callers throw the result to avoid phpstorm warnings. Remove $row
assignment that is always overridden as well.

Change-Id: I84bc4b11f10152eada6dd6f4788c4f79dcb4a2fb
2019-04-27 00:14:12 +00:00
Aaron Schulz
fc5d51f129 jobqueue: add GenericParameterJob and RunnableJob interface
Simplify the code of jobs that do not care about titles and removes
the direct Title dependency from JobQueue. Remove getTitle() from
IJobSpecification itself. Move all the Job::factory calls into a
single JobQueue::factoryJob() method.

Depends-on: Iee78f4baeca0c0b4d6db073f2fbcc56855114ab0
Change-Id: I9c9d0726d4066bb0aa937665847ad6042ade13ec
2019-04-08 11:05:23 -07:00
Aaron Schulz
b48dcf28d6 jobqueue: dependency inject more objects into JobQueue
Also moved some WikiMap/$wgJobClasses checks to JobQueueGroup::pop
which is the method callers are supposed to use.

Change-Id: I2ab82d8adc4ae1f54697d2935afa2053539cf2db
2019-04-04 01:32:59 -07:00
jenkins-bot
a0f9d54f11 Merge "Add Job::getMetadata() and Job::setMetadata() accessors" 2019-04-03 17:43:40 +00:00
jenkins-bot
f28f7a0d0f Merge "jobqueue: remove entire unused JobQueueAggregator class hierarchy" 2019-04-03 17:43:31 +00:00
Aaron Schulz
a51ea350be Add Job::getMetadata() and Job::setMetadata() accessors
Change-Id: I3a97008d324f600a1c9f6005673073277ee564fa
2019-04-03 16:34:32 +00:00
Aaron Schulz
4122e53845 jobqueue: fix DBO_TRX logic in JobQueueDB for avoiding transactions
Various methods were missing the flag setting logic and tests could
fail or have "outer scope" warnings in the logs for sqlite.

Change-Id: Ia0607d189a307667297f06109a34363c92e37d92
2019-04-01 22:29:31 +00:00
Aaron Schulz
196ef348c5 jobqueue: remove entire unused JobQueueAggregator class hierarchy
Change-Id: I3d56d5db54fcb4278c7cf2a8d08933c27773c3f7
2019-03-29 20:25:05 -07:00
Aaron Schulz
236e6ae11e jobqueue: allow direct server configuration arrays to JobQueueDB
This is useful for using a separate sqlite DB.

Change-Id: I6a7bb1d4bddedca34d68b0d8460c96c6ef5bbcf6
2019-03-09 03:40:20 +00:00
Aaron Schulz
a26eee40e0 Avoid use of deprecated ObjectCache::getMainWANInstance method
Change-Id: I429e68c8642a00afa812f5ae900c1fb4f8308528
2019-03-01 02:38:55 +00:00
Brian Wolff
286d49011f Various fixes for phan-taint-check
Bug: T216348
Change-Id: I0adafdc680dae0e930f38f08fe926645c57be06c
2019-02-17 11:41:11 +00:00
Thiemo Kreuz
734a969d55 Safe replacement of a lot of !count() with === []
This was originally a global search and replace. I manually checked all
replacements and reverted them if (due to the lack of type hints) either
null (that would be 0 when counted) or a Countable object can end in the
variable or property in question.

Now this patch only touches places where I'm sure nothing can break.

For the sanity of the honorable reviewers this patch is exclusively touching
negated counts. You should not find a single `!== []` in this patch, that
would be a mistake.

Change-Id: I5eafd4d8fccdb53a668be8e6f25a566f9c3a0a95
2019-01-15 17:28:49 +01:00
Jakub Vrana
547312ae39 Do not pass unused parameter
Found by PHPStan.

Change-Id: I48a613d94624ef4f8c533b98b8c128b4cc9c47f5
2018-12-01 18:08:04 +01:00
Aaron Schulz
51945dbca3 Use DB domain in JobQueueGroup and make WikiMap domain ID methods stricter
Using domains means thats JobQueueDB has the right value to use for calls
like LoadBalancer::getConnection(). The full domain includes the schema in
the case of Postgres. This makes calls to getConnection() less awkward by
not relying on the fallback logic in reallyOpenConnection() for null schemas.

Make getWikiIdFromDomain/isCurrentWikiDomain account for the schema if it
is both defined and is not simply the generic "mediawiki" schema MediaWiki
uses by default. If all wikis use the default schema, the wiki IDs can get
by with DB/prefix alone, which various config and methods may be built around.
Otherwise, the config callbacks must account for schema and the config must
include it in various wiki domain ID lists to properly disambiguate wikis.

Also, clean up SiteConfiguration::siteFromDB() since it is not meant
to handle schemas unless the callback method was taylored to do so.

Finally, add more comments to DefaultSettings.php about already existing
limitations of wiki domain IDs and their components.

Change-Id: I8d94a650e5c99a19ee50551c5be9544318eb05b1
2018-11-07 04:46:56 +00:00
Aaron Schulz
c947f3288d jobqueue: clean up JobQueueDB::getCacheKey() to use makeGlobalKey()
Change-Id: I341720fda0d242c7822f78048cb49f63538d08bc
2018-10-13 00:14:38 -07:00
Umherirrender
a4caa4d0c6 build: Updating mediawiki/mediawiki-codesniffer to 22.0.0
Added spaces around .
Removed empty return statement which are not required
Removed return after phpunit markTestIncomplete,
which is throwing to exit the test, no need for a return

Change-Id: I2c80b965ee52ba09949e70ea9e7adfc58a1d89ce
2018-09-16 15:51:11 +00:00
Aaron Schulz
5938a6efdc rdbms: add more comments and sanity checks for CONN_TRX_AUTOCOMMIT
Change-Id: I69992cf2e2ae3ef62125b0bc733a0cb7274f814e
2018-08-09 01:39:07 +00:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
James D. Forrester
f8ac266287 Job: Remove insert() and batchInsert(), deprecated since 1.21
Bug: T179624
Change-Id: I0d78a926c8c513fd652a424e8a51ded20e524a4a
2018-05-21 20:10:09 +01:00
Aaron Schulz
b6cd5421b9 rdbms: rename onTransactionIdle() to onTransactionCommitOrIdle()
This is clearer and is consistent with onTransactionPreCommitOrIdle()

Change-Id: I3a34a0e9adea69ec55ed6ddfef47703e31e7c3b5
2018-05-09 21:07:06 +00:00
Aaron Schulz
c8085ad43f rdbms: make IDatabase::onTransaction* methods pass the DB handle for convenience
Change-Id: Ia45a26830d62326b103593268fbf34c907783c90
2018-04-24 16:45:11 -07:00
Aaron Schulz
f8c2486d15 rdbms: rename CONN_TRX_AUTO constant to CONN_TRX_AUTOCOMMIT
The "AUTO" means AUTOCOMMIT, not "automatic transactions"/DBO_TRX,
which is basically the opposite concept. The new name does not
suffer from that ambiguity.

Keep the old constant as an alias for backwards compatibility.

Also remove LoadBalancer comment about non-existing field

Change-Id: I63beeb061fc9be73f320308e4d6393b58628b8c8
2018-04-12 13:01:52 -07:00
Brad Jorsch
01a10dba5a Remove reference to deprecated IDatabase->nextSequenceValue()
The method was deprecated and made unnecessary in Ib308190c.

Change-Id: I1729ac0b3a88270a4c2f064187a2472112aaeb1e
2017-09-01 12:28:39 -04:00
Aaron Schulz
99c80a8fc7 rdbms: Support secondary autocommit connections in LoadBalancer
This is useful for things like SqlBagOStuff and JobQueueDB
being able to write in auto-commit mode while the main
transaction round still goes on.

SqlBagOStuff already had this sort of logic.

JobQueueDB now also takes this approach so it does not have to
defer insertion except for sqlite. This makes callers more likely
to know when insertion failed or not.

Make sure LoadBalancer sets "master"/"replica" LB info itself;
this fixes a bug found in the new tests.

Bug: T140338
Bug: T42451
Change-Id: I4199a9598d7afbf976a6efa8ed84b85b56da02bd
2017-08-18 01:28:34 +00:00
Aaron Schulz
c9544ab6b2 Remove excess getExternalLB() argument
Change-Id: I097b1ba46806649f014ca624d15d9064ee0f9ba7
2017-07-17 16:32:53 -07:00
Aaron Schulz
82e2c924e4 Remove "@author Aaron Schulz" annotations
Bug: T139301
Change-Id: Ib5248e8e27d60611c7373bce4b29dd5e85aa3489
2017-06-27 15:24:14 -07:00
Aaron Schulz
95fdff36c2 Make DeferredUpdates detect LBFactory transaction rounds
Previously, tryOpportunisticExecute() tried to nest transaction rounds,
which would fail. Added LBFactory::hasTransactionRound() as needed.

Also cleaned up some unqualified class names in callbacks and set the
PRESEND flag for the JobQueueDB AutoCommitUpdate callback. Use the
proper getMasterDB() method while at it. These follow up 24842cfac.

Bug: T154425
Change-Id: Ib1d38f68bd217903d1a7d46fb15b7d7d9620daa6
2017-06-10 15:22:32 +00:00
Seb35
24842cfac0 Use AutoCommitUpdate instead of Database->onTransactionIdle
This is needed for deferred updates LinksDeletionUpdate and LinksUpdate, else
callbacks registered with onTransactionIdle prevent other transactions from
being executed, at least in this case.

Bug: T154425
Bug: T154438
Bug: T157679
Change-Id: Iecd396d584a62ac936cd963915339159467b44cd
2017-06-06 14:23:37 +02:00
Aaron Schulz
dd359741cc Move DB errors to Rdbms namespace
Change-Id: I463bd86123501abc68fdb78b4cda6110f7af2549
2017-04-15 10:47:41 -07:00
Aaron Schulz
3c2a172c3f Move DBConnRef/MaintainableDBConnRef to Rdbms namespace
Change-Id: I7ca6b40f973b2aae83747de92e280ded222c87e3
2017-03-29 11:04:47 -07:00
Aaron Schulz
488a647831 Move IDatabase/IMaintainableDatabase to Rdbms namespace
Change-Id: If7e8a8ff574661fd827de8bcec11d2c39a687300
2017-03-28 15:32:38 -07:00
James D. Forrester
9635dda73a includes: Replace implicit Bugzilla bug numbers with Phab ones
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.

Change-Id: I6f59febaf8fc96e80f8cfc11f4356283f461142a
2017-02-21 18:13:24 +00:00
jenkins-bot
69ae945e8d Merge "Update weblinks in comments from HTTP to HTTPS" 2016-11-08 21:32:00 +00:00
Fomafix
202f695f67 Update weblinks in comments from HTTP to HTTPS
Use HTTPS instead of HTTP where the HTTP link is a redirect to the HTTPS link.

Also update some defect links.

Change-Id: Ic3a5eac910d098ed5c2a21e9f47c9b6ee06b2643
2016-11-07 15:24:46 +01:00
Aaron Schulz
1eb86e0e4d Rename JobQueueDB::getSlaveDB() to getReplicaDB()
Change-Id: I7a88c4876e76836c92cc0b45b4e72124a087121c
2016-10-29 00:04:39 -07:00
Kunal Mehta
61adc1e146 Use namespaced ScopedCallback
The un-namespaced \ScopedCallback is deprecated.

Change-Id: Ie014d5a775ead66335a24acac9d339915884d1a4
2016-10-17 15:46:05 -07:00
Aaron Schulz
a2790b1b80 Remove wfGetCaller() dependency from DatabaseBase
Change-Id: I3e240b2eb5c1f6a21f1bc974c3d28f5755c7451a
2016-09-15 14:40:00 -07:00
Aaron Schulz
fca4beb302 Avoid using deprecated methods in JobQueueDB
Change-Id: Ib35b8792e3e4902b52c3e708d8c6e756f35986bd
2016-09-12 19:56:37 -07:00
Aaron Schulz
950cf6016c Rename DB_SLAVE constant to DB_REPLICA
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.

The old constant is an alias now.

Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
2016-09-05 22:55:53 -07:00
Aaron Schulz
16266edff3 Change "slave" => "replica DB" in /includes
Change-Id: Icb716219c9335ff8fa447b1733d04b71d9712bf9
2016-09-05 21:01:01 +00:00
Aaron Schulz
f3cfdf0baa Remove commit() calls from JobQueueDB
These are not safe for the common case where the local DB
handle is used for the queue (and other table writes).

Change-Id: Ic24a05c18bf31e49bf7e9a3c058deb5d35271511
2016-08-23 17:24:58 +00:00