Commit graph

96 commits

Author SHA1 Message Date
Amir Sarabadani
7ffd35ecc6 rdbms: Remove Database::attributesFromType and hard-deprecate ::factory
They have been soft-deprecated since 1.39 and the first one is not used
anywhere, ::factory is used in some third party code but not in
Wikimedia-deployed ones

Change-Id: Icd0f743d9c76554dd02471485ee732e25b9fb932
2023-02-23 17:59:54 +00:00
Amir Sarabadani
a4f296facf jobqueue: Migrate all IDatabase::select to SelectQueryBuilder
And some clean ups while we are at it like using buildComparison instead
of raw SQL.

Bug: T311866
Change-Id: Ib1bc73a2987763fcad667d393138a64c529f53d1
2023-02-23 15:01:57 +00:00
jenkins-bot
0d1dc069f6 Merge "jobqueue: Add documentation about specific type to Iterator type" 2023-02-16 21:55:32 +00:00
Adam Wight
5ea23a5b6e Document exceptions
Some of these can probably be removed.  In the meantime, documenting
makes it easier to understand their purpose.

Change-Id: I5bc407f56ce4eb915c7a3f2868fe66675769f723
2022-11-28 17:04:25 +00:00
Matěj Suchánek
a90a065586 Remove deprecated getMasterDB methods
They have been deprecated since 1.37 and are unused.

Change-Id: I626525eead68e3ba50564ac629a86c7c8dc9dd73
2022-11-17 14:30:47 +01:00
Umherirrender
3a7bcb9c77 jobqueue: Add documentation about specific type to Iterator type
In manageJobs.php check for Job class to skip the check for RunnableJob
interface, which does not implement the method

Change-Id: I2a42bfc78140310cf3df9c882ffb089719af5466
2022-11-14 19:51:17 +01:00
Umherirrender
1b342a8893 Various doc fixes about false and null on method arguments/return types
Doc-only changes

Change-Id: Ice974b3ba41708859dfe646e94b31c5ebbf26410
2022-11-03 18:55:47 +01:00
Umherirrender
b9a5d69d26 JobQueueDB: Always use IDatabase::addQuotes for quoting
Use addQuotes to quote the empty string instead of hard-coding '',
for better rdbms support and is always used in the file

Change-Id: I92d6481b725ccf27da187196c2cc7feb423cf828
2022-07-09 23:47:54 +02:00
Umherirrender
25188a6ae8 jobqueue: JobQueueDB::jobFromRow cannot return null
Found by phan strict checks

Change-Id: Ic7b2b8d262b8cdb5954222944e0d5a8613d6123d
2022-02-26 08:16:03 +00:00
Siddharth VP
60faebb725 Fix typos in comments (I-J)
Change-Id: Icaea2b6665cfc3b811d94f70c93452237f5e72bf
2021-12-30 20:38:04 +05:30
James D. Forrester
86f3270bda Tag various new 'primary' methods as @since 1.37
Change-Id: Icbbbc84154174da1bd9398990679857898f418d5
2021-09-02 11:58:19 -07:00
Thiemo Kreuz
46f26dff03 Same code style and formatting for exists-style SQL queries
Queries that query a value "1" are expected to return either
that "1" (possibly as a string), or false. It's safe to simply
cast this to bool and use it as it is.

Queries for COUNT(*) are expected to return that number,
possibly as a string. It's not possible for such a query to
return false. And even if, casting to 0 is fine.

I found an existing code style where the table name and the
"1" are on the same line as the selectField() method name, and
applied it to all similar queries.

Change-Id: I9453196281871c03ef03f653f43762eb9284342f
2021-08-12 11:37:58 +02:00
James D. Forrester
9fe4138f3d Follow-up I68198bc3: Hard-deprecate getMasterDB() methods
Also switch one use that we missed in the rebase.

Bug: T254646
Depends-On: I632610dec1aa6bb59af5423e6f86164fc791aadd
Change-Id: I9873c889f943fcbd7d867d534a7857f20ef0c5dc
2021-07-15 09:12:17 +01:00
James D. Forrester
597376e12c Replace getMasterDB methods with getPrimaryDB
Bug: T254646
Change-Id: I68198bc39b174ea1920b4acc2617cb6c6ce406e9
2021-07-14 16:21:04 -07:00
James D. Forrester
f2f9345e39 Replace uses of DB_MASTER with DB_PRIMARY in documentation and local variables
This is just a start.

Bug: T254646
Change-Id: I9213aad4660e27afe7ff9e5d2e730cbf03911068
2021-05-14 12:40:34 -07:00
James D. Forrester
df5eb22f83 Replace uses of DB_MASTER with DB_PRIMARY
Just an auto-replace from codesniffer for now.

Change-Id: I5240dc9ac5929d291b0ef1c743ea2bfd3f428266
2021-04-29 09:24:31 -07:00
Reedy
4c2d7e87b8 JobQueueDB: Remove unused getReplicaDB() call
Change-Id: I9e33fee718281a63d0271ad8b898ee893b10b7c2
2021-04-21 14:43:17 +01:00
Sébastien Beyou
dec13460ea Implement JobQueueDB::getAllAbandonedJobs
Bug: T276945
Change-Id: I918356733409dd69ac438b0311ad7120d0b37d40
2021-04-05 21:53:22 +02:00
Umherirrender
8de3b7d324 Use static closures where safe to use
This is micro-optimization of closure code to avoid binding the closure
to $this where it is not needed.

Created by I25a17fb22b6b669e817317a0f45051ae9c608208

Change-Id: I0ffc6200f6c6693d78a3151cb8cea7dce7c21653
2021-02-11 00:13:52 +00:00
Umherirrender
a5ccb3e549 Avoid hard coded AS for column aliases
Use 'alias' => 'column' feature of the abstract database layer

Wikipage old code:
SELECT cl_to AS page_title, 14 AS page_namespace FROM `categorylinks`
WHERE cl_from = 651

Wikipage new code:
SELECT cl_to AS `page_title`,14 AS `page_namespace` FROM `categorylinks`
WHERE cl_from = 651

Change-Id: Ibd2fb22bd556f331b955378eb144153f2fdb9dc6
2020-09-21 21:07:14 +02:00
Thiemo Kreuz
9c6473dc1d Reduce nesting by turning big if-else into guard clauses
Change-Id: If3d2f18af0cc7d0f8fe9f764bd4d0e4bae70e440
2020-07-24 11:55:31 +02:00
Umherirrender
fd1ed0c092 Pass function name to database functions
Useful for logging

Change-Id: Ia2160fb6be5fc93f28ab51f7ae23d7f078247481
2020-06-07 14:16:52 +02:00
Tim Starling
47a1619027 Remove terminating line breaks from debug messages
A terminating line break has not been required in wfDebug() since 2014,
however no migration was done. Some of these line breaks found their way
into LoggerInterface::debug() calls, where they mess up the formatting
of the debug log.

So, remove terminating line breaks from wfDebug() and
LoggerInterface::debug() calls.

Also:
* Fix the stripping of leading line breaks from the log header emitted
  by Setup.php. This feature, accidentally broken in 2014, allows
  requests to be distinguished in the log file.
* Avoid using the global variable $self.
* Move the logging of the client IP back to Setup.php. It was moved to
  WebRequest in the hopes that it would not always be needed, however
  $wgRequest->getIP() is now called unconditionally a few lines up in
  Setup.php. This means that it is put in its proper place after the
  "start request" message.
* Wrap the log header code in a closure so that variables like $name do
  not leak into global scope.
* In Linker.php, remove a few instances of an unnecessary second
  parameter to wfDebug().

Change-Id: I96651d3044a95b9d210b51cb8368edc76bebbb9e
2020-06-03 12:01:16 +10:00
Reedy
d53e91d272 Fix more PSR12.Properties.ConstantVisibility.NotFound
Change-Id: I94520b10d78a17ea8e965633dd475ea711f25c99
2020-05-15 00:33:32 +01:00
James D. Forrester
4f2d1efdda Coding style: Auto-fix MediaWiki.Classes.UnsortedUseStatements.UnsortedUse
Change-Id: I94a0ae83c65e8ee419bbd1ae1e86ab21ed4d8210
2020-01-10 09:32:25 -08:00
Brad Jorsch
b26248235b Add string casts when using array_keys() with SQL query conditions
Until I70473280, integer literals were always quoted as strings, because
the databases we support all have no problem with casting
string-literals for comparisons and such.

But it turned out that gave MySQL/MariaDB's planner problems in some
queries, so we changed it to not quote actual PHP integers.

But then we run into the fact that PHP associative arrays don't preserve
the types of keys, it converts integer-like strings into actual
integers. And when those are passed to the DB unquoted for comparison
with a string-typed column, MySQL/MariaDB screws up the comparison while
PostgreSQL simply throws an error. Sigh.

This patch adds string casting to direct uses of array_keys() to supply
values for such query conditions. It doesn't change uses where the field
being compared is a numeric field.

If anyone knows of a good way to find indirect uses of array_keys() for
passing as $conds to IDatabase methods, please do so!

Change-Id: Ie72ee33437d492904e1495b3f4ebb1fcf0118f49
2019-12-16 16:05:18 -05:00
Daimona Eaytoy
f18af0b61f Remove more Oracle and Mssql leftovers
Follows-up 4d10bb14e8 and 807d793ab9.

According to codesearch [0], these were the last usages. Note that this
patch leaves two constants in places, IDatabase::DBO_SYSDBA and
DBO_DDLMODE. These are public constants used "mostly for oracle" according
to the docs, but maybe we could find other use cases in the future (?).

[0] - https://codesearch.wmflabs.org/core/?q=oracle%7Cmssql&i=fosho&files=%5C.%5B%5Ej%5Cd%5D%7Cen%5C.json&repos=

Bug: T230418
Change-Id: Ibfb748b4b23b885a77f4de161af4bf2ab9649a89
2019-08-25 17:21:49 +00:00
Aaron Schulz
97cbaa349e Cleanup JobQueueDB::recycleAndDeleteStaleJobs() use of IDatabase::affectedRows()
Bug: T229456
Change-Id: Ie22085ddba66f122e59e93baaf9b53c76b5ce448
2019-08-08 10:59:26 +00:00
Aaron Schulz
43c78e83d7 jobqueue: migrate root job deduplication to the WAN cache
If the root job timestamp keys are lost or otherwise unknown, they
will now be deductively recached with the best known values as jobs
are popped and executed. This means the running any of many child
jobs of a root job can restore the root timestamp if it was lost.
This does not need to use the main stash given this fact.

Bug: T227376
Change-Id: Iae0f3af15803af048ff49f3bf281b2bde18c87f2
2019-07-13 15:38:17 -07:00
Aaron Schulz
aa4f4d6b53 jobqueue: make JobQueueDB stricter about broken job_params fields
Also rename throwDBException() to getDBException() and make the
callers throw the result to avoid phpstorm warnings. Remove $row
assignment that is always overridden as well.

Change-Id: I84bc4b11f10152eada6dd6f4788c4f79dcb4a2fb
2019-04-27 00:14:12 +00:00
Aaron Schulz
fc5d51f129 jobqueue: add GenericParameterJob and RunnableJob interface
Simplify the code of jobs that do not care about titles and removes
the direct Title dependency from JobQueue. Remove getTitle() from
IJobSpecification itself. Move all the Job::factory calls into a
single JobQueue::factoryJob() method.

Depends-on: Iee78f4baeca0c0b4d6db073f2fbcc56855114ab0
Change-Id: I9c9d0726d4066bb0aa937665847ad6042ade13ec
2019-04-08 11:05:23 -07:00
Aaron Schulz
b48dcf28d6 jobqueue: dependency inject more objects into JobQueue
Also moved some WikiMap/$wgJobClasses checks to JobQueueGroup::pop
which is the method callers are supposed to use.

Change-Id: I2ab82d8adc4ae1f54697d2935afa2053539cf2db
2019-04-04 01:32:59 -07:00
jenkins-bot
a0f9d54f11 Merge "Add Job::getMetadata() and Job::setMetadata() accessors" 2019-04-03 17:43:40 +00:00
jenkins-bot
f28f7a0d0f Merge "jobqueue: remove entire unused JobQueueAggregator class hierarchy" 2019-04-03 17:43:31 +00:00
Aaron Schulz
a51ea350be Add Job::getMetadata() and Job::setMetadata() accessors
Change-Id: I3a97008d324f600a1c9f6005673073277ee564fa
2019-04-03 16:34:32 +00:00
Aaron Schulz
4122e53845 jobqueue: fix DBO_TRX logic in JobQueueDB for avoiding transactions
Various methods were missing the flag setting logic and tests could
fail or have "outer scope" warnings in the logs for sqlite.

Change-Id: Ia0607d189a307667297f06109a34363c92e37d92
2019-04-01 22:29:31 +00:00
Aaron Schulz
196ef348c5 jobqueue: remove entire unused JobQueueAggregator class hierarchy
Change-Id: I3d56d5db54fcb4278c7cf2a8d08933c27773c3f7
2019-03-29 20:25:05 -07:00
Aaron Schulz
236e6ae11e jobqueue: allow direct server configuration arrays to JobQueueDB
This is useful for using a separate sqlite DB.

Change-Id: I6a7bb1d4bddedca34d68b0d8460c96c6ef5bbcf6
2019-03-09 03:40:20 +00:00
Aaron Schulz
a26eee40e0 Avoid use of deprecated ObjectCache::getMainWANInstance method
Change-Id: I429e68c8642a00afa812f5ae900c1fb4f8308528
2019-03-01 02:38:55 +00:00
Brian Wolff
286d49011f Various fixes for phan-taint-check
Bug: T216348
Change-Id: I0adafdc680dae0e930f38f08fe926645c57be06c
2019-02-17 11:41:11 +00:00
Thiemo Kreuz
734a969d55 Safe replacement of a lot of !count() with === []
This was originally a global search and replace. I manually checked all
replacements and reverted them if (due to the lack of type hints) either
null (that would be 0 when counted) or a Countable object can end in the
variable or property in question.

Now this patch only touches places where I'm sure nothing can break.

For the sanity of the honorable reviewers this patch is exclusively touching
negated counts. You should not find a single `!== []` in this patch, that
would be a mistake.

Change-Id: I5eafd4d8fccdb53a668be8e6f25a566f9c3a0a95
2019-01-15 17:28:49 +01:00
Jakub Vrana
547312ae39 Do not pass unused parameter
Found by PHPStan.

Change-Id: I48a613d94624ef4f8c533b98b8c128b4cc9c47f5
2018-12-01 18:08:04 +01:00
Aaron Schulz
51945dbca3 Use DB domain in JobQueueGroup and make WikiMap domain ID methods stricter
Using domains means thats JobQueueDB has the right value to use for calls
like LoadBalancer::getConnection(). The full domain includes the schema in
the case of Postgres. This makes calls to getConnection() less awkward by
not relying on the fallback logic in reallyOpenConnection() for null schemas.

Make getWikiIdFromDomain/isCurrentWikiDomain account for the schema if it
is both defined and is not simply the generic "mediawiki" schema MediaWiki
uses by default. If all wikis use the default schema, the wiki IDs can get
by with DB/prefix alone, which various config and methods may be built around.
Otherwise, the config callbacks must account for schema and the config must
include it in various wiki domain ID lists to properly disambiguate wikis.

Also, clean up SiteConfiguration::siteFromDB() since it is not meant
to handle schemas unless the callback method was taylored to do so.

Finally, add more comments to DefaultSettings.php about already existing
limitations of wiki domain IDs and their components.

Change-Id: I8d94a650e5c99a19ee50551c5be9544318eb05b1
2018-11-07 04:46:56 +00:00
Aaron Schulz
c947f3288d jobqueue: clean up JobQueueDB::getCacheKey() to use makeGlobalKey()
Change-Id: I341720fda0d242c7822f78048cb49f63538d08bc
2018-10-13 00:14:38 -07:00
Umherirrender
a4caa4d0c6 build: Updating mediawiki/mediawiki-codesniffer to 22.0.0
Added spaces around .
Removed empty return statement which are not required
Removed return after phpunit markTestIncomplete,
which is throwing to exit the test, no need for a return

Change-Id: I2c80b965ee52ba09949e70ea9e7adfc58a1d89ce
2018-09-16 15:51:11 +00:00
Aaron Schulz
5938a6efdc rdbms: add more comments and sanity checks for CONN_TRX_AUTOCOMMIT
Change-Id: I69992cf2e2ae3ef62125b0bc733a0cb7274f814e
2018-08-09 01:39:07 +00:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
James D. Forrester
f8ac266287 Job: Remove insert() and batchInsert(), deprecated since 1.21
Bug: T179624
Change-Id: I0d78a926c8c513fd652a424e8a51ded20e524a4a
2018-05-21 20:10:09 +01:00
Aaron Schulz
b6cd5421b9 rdbms: rename onTransactionIdle() to onTransactionCommitOrIdle()
This is clearer and is consistent with onTransactionPreCommitOrIdle()

Change-Id: I3a34a0e9adea69ec55ed6ddfef47703e31e7c3b5
2018-05-09 21:07:06 +00:00
Aaron Schulz
c8085ad43f rdbms: make IDatabase::onTransaction* methods pass the DB handle for convenience
Change-Id: Ia45a26830d62326b103593268fbf34c907783c90
2018-04-24 16:45:11 -07:00