Commit graph

36 commits

Author SHA1 Message Date
Reedy
499dbfb4cb maintenance: Use more of namespaced Maintenance class
Change-Id: I53f2e32c73c92cc3a0deee48ebe6d13329a7a0cf
2024-10-16 01:09:19 +00:00
Umherirrender
c8ec25a961 maintenance: Add missing documentation to class properties
Add doc-typehints to class properties found by the PropertyDocumentation
sniff to improve the documentation.

Once the sniff is enabled it avoids that new code is missing type
declarations. This is focused on documentation and does not change code.

Change-Id: I7dec01892a987a87b1b79374a1c28f97d055e8fa
2024-09-13 19:29:24 +02:00
Dreamy Jazz
e7393b3cc7 Exclude boilerplate maintenance code from code coverage reports
Why:
* Maintenance scripts in core have bolierplate code that is
  added before and after the class to allow directly running
  the maintenance script.
* Running the maintenance script directly has been deprecated
  since 1.40, so this boilerplate code is only to support a now
  deprecated method of running maintenance scripts.
* This code cannot also be marked as covered, due to PHPUnit
  not recognising code coverage for files.
* Therefore, it is best to ignore this boilerplate code in code
  coverage reports as it cannot be marked as covered and also
  is for deprecated code.

What:
* Wrap the boilerplate code (requiring Maintenance.php and then
  later defining the maintenance script class and running if the
  maintenance script was called directly) with @codeCoverageIgnore
  comments.
* Some files use a different boilerplate code, however, these
  should also be marked as ignored for coverage for the same
  reason that coverage is not properly reported for files.

Bug: T371167
Change-Id: I32f5c6362dfb354149a48ce9c28da9a7fc494f7c
2024-08-27 13:22:29 +01:00
James D. Forrester
35b2542895 Namespace includes/cache
Bug: T353458
Change-Id: Ic3f3168ef17113f5fb3ec11e0a47f52d65eefba9
2024-02-20 10:28:03 -05:00
Amir Sarabadani
d9370003fb maintenance: Introduce getReplicaDB() and getPrimaryDB()
And start using them instead of wfGetDB(), LB/LBF connection methods or
worse, $this->getDB().

$this->getDB() reuses the database object regardless of whether you're
calling a replica or primary, leading to returning a replica on a
primary and other way around.

Bug: T330641
Change-Id: I9e2cf85ca277022284fc26b9f37db57bd12aaa81
2024-01-18 15:12:04 +01:00
Amir Sarabadani
69cabb628c maintenance: Migrate to expression builders
This was somehow left out

Bug: T210206
Change-Id: I70851b5b99fa865dbfd629caf2c1866c85418350
2024-01-17 20:27:08 +01:00
Derick Alangi
74033c50cd maintenance: Begin using Maintenance::getServiceContainer()
Maintenance class provides a method for getting a fresh reference
of the MW services container instance. Let's make use of these in
maintenance scripts now that we have it.

NOTE: There are still some static methods like in refreshLinks.php
that makes use of services that we can't use this method for now.

Change-Id: Idba744057577896fc97c9ecf4724db27542bf01c
2023-09-04 10:39:58 +00:00
Amir Sarabadani
7432b21816 Migrate more calls of Database::select* to SelectQueryBuilder
Using a php parser written on top of ANTLR4, done semi-automatically.

I checked everything and made adjustments.

Bug: T311866
Change-Id: I6150c6909bce8f3dbd745a26380cc0af9d9c547f
2023-07-26 13:01:28 +02:00
Umherirrender
aa3079a4d2 Replace deprecated Hooks::register in benchmarkParse.php
Change-Id: Id68d2f0731f9d3aae934e7fe7d5f602fd71ea1ec
2023-05-18 17:21:26 +02:00
Bartosz Dziewoński
6ba47296d9 Fix Phan suppressions related to Title::castFrom*() and friends
There is no way to express that Title::castFromPageIdentity(),
Title::castFromPageReference() and Title::castFromLinkTarget()
can only return null when the parameter is null. We need to add
Phan suppressions or explicit types almost everywhere that these
methods are used with parameters that are known to not be null.

Instead, introduce new methods Title::newFromPageIdentity() and
Title::newFromPageReference() (Title::newFromLinkTarget() already
exists), without the null-coalescing behavior, and use them when
the parameter is not null. This lets static analysis tools, and
humans, easily understand where nulls can't appear.

Do the same with the corresponding TitleFactory methods.

Change the obvious uses of castFrom*() to newFrom*() (if there is
a Phan suppression, a type check, or a method call on the result).

Change-Id: Ida4da75953cf3bca372a40dc88022443109ca0cb
2023-04-22 16:45:09 +02:00
James D. Forrester
ad06527fb4 Reorg: Namespace the Title class
This is moderately messy.

Process was principally:

* xargs rg --files-with-matches '^use Title;' | grep 'php$' | \
  xargs -P 1 -n 1 sed -i -z 's/use Title;/use MediaWiki\\Title\\Title;/1'
* rg --files-without-match 'MediaWiki\\Title\\Title;' . | grep 'php$' | \
  xargs rg --files-with-matches 'Title\b' | \
  xargs -P 1 -n 1 sed -i -z 's/\nuse /\nuse MediaWiki\\Title\\Title;\nuse /1'
* composer fix

Then manual fix-ups for a few files that don't have any use statements.

Bug: T166010
Follows-Up: Ia5d8cb759dc3bc9e9bbe217d0fb109e2f8c4101a
Change-Id: If8fc9d0d95fc1a114021e282a706fc3e7da3524b
2023-03-02 08:46:53 -05:00
Umherirrender
1f71eccf63 phan: Disable null_casts_as_any_type setting
Make phan stricter about null types by setting null_casts_as_any_type to
false (the default in mediawiki-phan-config)
Remaining false positive issues are suppressed.
The suppression and the setting change can only be done together

Bug: T242536
Bug: T301991
Change-Id: I0f295382b96fb3be8037a01c10487d9d591e7e01
2022-03-21 18:25:07 +00:00
Umherirrender
6ea3d6ac2c Add explicit casts between scalar types
php internal functions like floor/round/ceil documented to return
float, most cases the result is used as int, added casts

Found by phan strict checks

Change-Id: I92daeb0f7be8a0566fd9258f66ed3aced9a7b792
2022-03-08 16:59:01 +00:00
Roman Stolar
fa5237eb48 Replace Content::getParserOutput call to ContentRenderer::getParserOutput
Bug: T287158
Change-Id: I8a13f45027e08e2d8ddefa140dd47a0c55094934
2021-10-20 12:11:24 +03:00
Umherirrender
94792e97be maintenance: Cleanup use of exit in Maintenance::execute implementation
Returning void from execute() is success.
Returning true is success, false is failure with exit(1)
Using fatalError also using exit(1)

Change-Id: I1d40430ad6226e4aab8f0810b03ee1213282d123
2021-08-30 19:48:27 +00:00
Umherirrender
21e3caa033 Remove explicit LIMIT 1 when using IDatabase::selectRow/selectField
The limit is the reason why there is an explicit function to use

Change-Id: Ic85cb80b10f233fad4f8ffa214848d76092aee91
2021-08-06 00:54:47 +02:00
DannyS712
47d70dbfba Post Revision-removal cleanup
Updates for the removal of the Revision class itself
and the various methods/hooks/variables removed in the
process, including:

- Update some documentation removing most references
to the Revision class and updating the MCR migration
notes to reflect the past tense for Revision methods.

- Change some capitalization from "Revision" to "revision"
to make it clear comments are about revisions in general,
not the Revision class in particular.

- Minor code tweaks including removing unused variables that
were around for the old hooks that were removed, and
removing the use of DeprecatablePropertyArray where no
longer needed for anything.

- Fix incorrect documentation for PageUpdater::getStatus(),
the status value changed a while ago to have revision-record
in addition to revision, and recently to only have the
revision-record, but ironically PageUpdater was never updated.

- Removed Parser::$mRevisionObject, used to be a Revision object
and was deprecated in 1.35, missed earlier because it was no
longer being set to Revision objects, always null.

- Add RevisionRecord typehints in DummyLinker to match those
in the corresponding Linker methods

This should be a no-op in terms of functionality.

Bug: T247143
Change-Id: I03bbb94fc29085855448780b1a5ad9063911ecc4
2021-06-24 00:32:39 +00:00
C. Scott Ananian
0676256ccc parser: new BeforeParserFetchTemplateRevisionRecord hook
This new hook provides for the use case in T47096 (allowing the
Translate extension to transclude a page from another language) by
adding a new hook which would let us deprecate and replace two awkward
legacy hooks (one with an embarrassing capitalization issue).  The new
hook is a little more tightly scoped in terms of what it allows and
gives access to, and it uses the new RevisionRecord API.

In addition, the new hook uses LinkTarget instead of Title per
current best practices. (PageIdentity is not appropriate for
reasons documented at the hook invocation site.)

The original BeforeParserFetchTemplateAndtitle (sic) hook allowed
redirecting the revision id of a template inclusion, but not the
title. The only known current use is Extension:ApprovedRevs; the
FlaggedRevs extension replaces the entire function using
ParserOptions::setCurrentRevisionRecordCallback().

Extension:Translate would like to redirect the title as well, possibly
recursively (for a limited number of hops) to handle fallback
languages.  That is, when invoked on Foo/fr, including Template:Bar
would redirect to Template:Bar/fr -- and, if that doesn't exist, then
Template:Bar/fr would redirect to its fallback language, say
Template:Bar/en. It uses the top-level page title as context to set
the desired page language.  This would require 2 invocations of the
hook; we've set the recursion limit to 3 to provide a little bit
of future-proofing.

The hook added in this patch uses RevisionRecord instead of int
$rev_id, and thus can handle the case where the redirect is to a page
which doesn't exist (by setting the RevisionRecord to a
MutableRevisionRecord with the correct title and no main slot content)
in the fallback language case above.

The new hook deprecates BeforeParserFetchTemplateAndtitle and replaces
ParserFetchTemplate as well (deprecated in 1.35).  Code search:

https://codesearch.wmcloud.org/search/?q=BeforeParserFetchTemplateAndtitle&i=nope&files=&repos=

Bug: T47096
Change-Id: Ia5b5d339706ce4084c16948300e0e3418b11792e
2021-02-05 09:37:32 -05:00
Tim Starling
20d06b34bb Safer autoloading with respect to file-scope code
Many files were in the autoloader despite having potentially harmful
file-scope code.

* Exclude all CommandLineInc maintenance scripts from the autoloader.
* Introduce  "NO_AUTOLOAD" tag which excludes the file containing it
  from the autoloader. Use it on CommandLineInc.php and a few
  suspicious-looking files without classes in case they are refactored
  to add classes in the future.
* Add a test which parses all non-PSR4 class files and confirms that
  they do not contain dangerous file-scope code. It's slow (15s) but
  its results were enlightening.
* Several maintenance scripts define constants in the file scope,
  intending to modify the behaviour of MediaWiki. Either move the
  define() to a later setup function, or protect with NO_AUTOLOAD.
* Use require_once consistently with Maintenance.php and
  doMaintenance.php, per the original convention which is supposed to
  allow one maintenance script to use the class of another maintenance
  script. Using require breaks autoloading of these maintenance class
  files.
* When Maintenance.php is included, check if MediaWiki has already
  started, and if so, return early. Revert the fix for T250003 which
  is incompatible with this safety measure. Hopefully it was superseded
  by splitting out the class file.
* In runScript.php add a redundant PHP_SAPI check since it does some
  things in file-scope code before any other check will be run.
* Change the if(false) class_alias(...) to something more hackish and
  more compatible with the new test.
* Some site-related scripts found Maintenance.php in a non-standard way.
  Use the standard way.
* fileOpPerfTest.php called error_reporting(). Probably debugging code
  left in; removed.
* Moved mediawiki.compress.7z registration from the class file to the
  caller.

Change-Id: I1b1be90343a5ab678df6f1b1bdd03319dcf6537f
2021-01-11 11:59:36 +11:00
Tim Starling
df65f09669 Revert "Correct capitalization of BeforeParserFetchTemplateAndTitle hook"
In the new hook system, it is invalid to have two hooks that differ only
in case.

This reverts commit afb5b38e15.

Change-Id: I160ece0a7bc68c748037b383137364b787be86a0
2020-04-30 07:50:24 +10:00
DannyS712
0f826d1f73 Replace uses and hard deprecate Revision::newFromTitle
Also fixed a use of ::newFromId in benchmarkParse and Parser

Bug: T249183
Change-Id: I3219a72f1085836205dc497e2236b52db24d7c16
2020-04-18 04:47:10 +00:00
C. Scott Ananian
afb5b38e15 Correct capitalization of BeforeParserFetchTemplateAndTitle hook
We deprecate the incorrect capitalization and add a new hook with the
correct capitalization.  There are very few users of this hook, and none
in WMF-deployed code.

WMF-deployed code search:
https://codesearch.wmflabs.org/deployed/?q=BeforeParserFetchTemplateAndtitle&i=nope&files=&repos=

Everything code search:
https://codesearch.wmflabs.org/search/?q=BeforeParserFetchTemplateAndtitle&i=nope&files=&repos=

It would be tempting to remove this hook entirely, but it seems to
serve a valuable purpose in allowing page and template fetches at a
specific "old" timestamp.  (Extension:Memento and Extension:TimeTravel
don't seem to have implemented this feature, however.)

Change-Id: I501b0ee35b81fee424d45084088a6ed63c3dfd54
2020-04-17 17:19:44 -04:00
Umherirrender
b4fe9c4bcc Set method visibility on maintenance scripts
Change-Id: I44c82fbe65e1d002803ce065df6563f06dd39cd4
2019-11-16 22:54:17 +00:00
Umherirrender
7363e38ddf Set public for override of Maintenance functions
The following function are set to public in the parent class and cannot
have another visibility in subclasses
Maintenance::__construct
Maintenance::execute
Maintenance::getDbType
Maintenance::validateParamsAndArgs
Maintenance::setDB

Change-Id: I0cd6514642d479aca20f1221bf673b0713c21631
2019-10-09 20:41:33 +02:00
Thiemo Kreuz
06c1a5976c maintenance: Deprecate Maintenance::hasArg/getArg with no param
Benefit of keeping the parameter optional:
- In maintenance scripts that really only have one parameter, it's a
  little more convenient to be able to ask for *the* parameter via an
  empty getArg().

Disadvantages:
- It's unclear what getArg() means when there is no indication *which*
  argument the code asks for. This might as well return the last
  argument, or an array of all arguments.
- In scripts with two or more arguments, it's confusing to see
  getArg( 1 ) next to an empty getArg().
- The methods are more complex and a bit more complicated to use with
  the extra feature of this parameter being optional. Users need to
  look up what the default is to be able to use it safely.

Change-Id: I22a43bfdfc0f0c9ffdb468c13aba73b888d1f15e
2019-03-29 14:37:46 +01:00
Aaron Schulz
cb15755e92 Normalize use of "INNER JOIN" to "JOIN" in database queries
The ANSI SQL default join type is INNER and this might save
some line breaks here and there.

Change-Id: Ibd39976f46ca3f9b71190d3b60b76ca085787a00
2019-03-06 09:17:30 -08:00
Umherirrender
ad776c7d5f Use ::class to resolve class names in maintenance scripts
This helps to find renamed or misspelled classes earlier.
Phan will check the class names

Change-Id: I1d4567f47f93eb1436cb98558388e48d35258666
2018-01-23 17:40:16 +00:00
Bryan Davis
9e34eeff23 Maintenance: add fatalError() method
Deprecate the second argument to Maintenance::error() in favor of a new
Maintenance::fatalError() method. This is intended to make it easier to
review flow control in maintenance scripts.

Change-Id: I75699008638f7e99b11210c7bb9e2e131fca7c9e
2017-11-21 21:34:16 -07:00
Kunal Mehta
0efe030cd9 benchmarkParse: Add option to clear link cache
For performance testing the lookup of links (e.g. LinkHolderArray),
allow resetting the LinkCache after every parse.

Change-Id: I75c2cad9a8cf1b41d192708773194f799673ce83
2016-09-14 23:57:16 -07:00
Aaron Schulz
950cf6016c Rename DB_SLAVE constant to DB_REPLICA
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.

The old constant is an alias now.

Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
2016-09-05 22:55:53 -07:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Reedy
44cebea941 Update wfGetDB calls in Maintenance scripts to use getDB()
Change-Id: I9ad6745d84506b736dae94747256caac89715899
2016-01-02 16:58:23 +00:00
Max Semenik
1120bbe202 benchmarkParse.php: add options for HHVM testing
--warmup makes a specified number of loops before measuring;
--loops repeats parsing given number of times

Change-Id: I18bff702fce1f97c4afc6c9fd618fdc1f3272732
2014-07-15 17:23:27 -07:00
Siebrand Mazeland
f2f31fadc7 Update formatting in maintenance/ (2/4)
Change-Id: I2b791d3bff0de464b6bdaaeae0622c065389c31c
2014-04-23 20:08:42 +02:00
Tyler Anthony Romeo
58a5641415 Code, style, and doc fixes for benchmarkParse.php
Fixes database query semantics, various documentation
issues, and unused variables.

Change-Id: Ic2c2d4d3912a9cd6703f7690f9d08df8ea08cd5c
Follows-up: I3bf3366a5ff58942 (dfd481e113)
2014-02-04 22:00:37 -05:00
Tim Starling
dfd481e113 Maintenance script for benchmarking parse operations
Including a feature allowing templates from some other time to be used,
which allows comparative benchmarking before and after the migration of
a template to Lua.

Change-Id: I3bf3366a5ff589421f6c52e9186e0cea05e6cff3
2014-02-05 00:40:21 +00:00