Commit graph

11 commits

Author SHA1 Message Date
Umherirrender
06947613d7 maintenance: Use namespaced classes
This does not include use of MediaWiki\Maintenance\Maintenance,
assuming the maintenance scripts going into the same namespace

Change-Id: I488f95b537ce86eb5e463be7bce3653610dd13d9
2024-10-21 19:08:08 +02:00
Dreamy Jazz
e7393b3cc7 Exclude boilerplate maintenance code from code coverage reports
Why:
* Maintenance scripts in core have bolierplate code that is
  added before and after the class to allow directly running
  the maintenance script.
* Running the maintenance script directly has been deprecated
  since 1.40, so this boilerplate code is only to support a now
  deprecated method of running maintenance scripts.
* This code cannot also be marked as covered, due to PHPUnit
  not recognising code coverage for files.
* Therefore, it is best to ignore this boilerplate code in code
  coverage reports as it cannot be marked as covered and also
  is for deprecated code.

What:
* Wrap the boilerplate code (requiring Maintenance.php and then
  later defining the maintenance script class and running if the
  maintenance script was called directly) with @codeCoverageIgnore
  comments.
* Some files use a different boilerplate code, however, these
  should also be marked as ignored for coverage for the same
  reason that coverage is not properly reported for files.

Bug: T371167
Change-Id: I32f5c6362dfb354149a48ce9c28da9a7fc494f7c
2024-08-27 13:22:29 +01:00
Umherirrender
81c6df6a46 maintenance: Use expression builder instead of raw sql
Bug: T361023
Change-Id: Ieb229d8088cb1ff3f03e44f7ac99eb612f48bc7b
2024-07-22 22:29:20 +02:00
C. Scott Ananian
105bb58ae2 [ParsoidCachePrewarmJob] Use ParserOutputAccess
One more step in gradually replacing uses of ParsoidOutputAccess.  This
one was pretty easy, as ParsoidOutputAccess was pretty much directly
calling ParserOutputAccess when provided with a ExistingPageRecord
and RevisionRecord.

Bug: T367074
Change-Id: I96161a64952e1809c0aec773d5a3dd4c71105657
2024-06-17 13:24:39 +00:00
C. Scott Ananian
a565e388f9 Move ParsoidOutputAccess::supportsContentModel() into Parsoid SiteConfig
The `supportsContentModel` method is really querying Parsoid for the
set of content models it supports, so it makes sense to put it in the
Parsoid-specific SiteConfig service.

This is part of the work to deprecate and remove ParsoidOutputAccess.

Change-Id: I81eb2df8cef93ede95361a4e03185b3d58e5b84b
2024-05-22 10:57:37 -04:00
Amir Sarabadani
d9370003fb maintenance: Introduce getReplicaDB() and getPrimaryDB()
And start using them instead of wfGetDB(), LB/LBF connection methods or
worse, $this->getDB().

$this->getDB() reuses the database object regardless of whether you're
calling a replica or primary, leading to returning a replica on a
primary and other way around.

Bug: T330641
Change-Id: I9e2cf85ca277022284fc26b9f37db57bd12aaa81
2024-01-18 15:12:04 +01:00
Subramanya Sastry
c8d0470f4b Make ParsoidOutputAccess a wrapper over ParserOutputAccess
* Updated ParserOutput to set Parsoid render ids that REST API
  functionality expects in ParserOutput objects.
* CacheThresholdTime functionality no longer exists since it was
  implemented in ParsoidOutputAccess and ParserOutputAccess doesn't
  support it. This is tracked in T346765.
* Enforce the constraint that uncacheable parses are only for fake or
  mutable revisions. Updated tests that violated this constraint to
  use 'getParseOutput' instead of calling the parse method directly.
* Had to make some changes in ParsoidParser around use of preferredVariant
  passed to Parsoid. I also left some TODO comments for future fixes.
  T267067 is also relevant here.

PARSOID-SPECIFIC OPTIONS:
* logLinterData: linter data is always logged by default -- removed
  support to disable it. Linter extension handles stale lints properly
  and it is better to let it handle it rather than add special cases
  to the API.
* offsetType: Moved this support to ParsoidHandler as a post-processing
  of byte-offset output. This eliminates the need to support this
  Parsoid-specific options in the ContentHandler hierarchies.
* body_only / wrapSections: Handled this in HtmlOutputRendererHelper
  as a post-processing of regular output by removing sections and
  returning the body content only. This does result in some useless
  section-wrapping work with Parsoid, but the simplification is probably
  worth it. If in the future, we support Parsoid-specific options in
  the ContentHandler hierarchy, we could re-introduce this. But, in any
  case, this "fragment" flavor options is likely to get moved out of
  core into the VisualEditor extension code.

DEPLOYMENT:
* This patch changes the cache key by setting the useParsoid option
  in ParserOptions. The parent patch handles this to ensure we don't
  encounter a cold cache on deploy.

TESTS:
* Updated tests and mocks to reflect new reality.
* Do we need any new tests?

Bug: T332931
Change-Id: Ic9b7cc0fcf365e772b7d080d76a065e3fd585f80
2023-10-13 15:03:03 -05:00
Derick Alangi
74033c50cd maintenance: Begin using Maintenance::getServiceContainer()
Maintenance class provides a method for getting a fresh reference
of the MW services container instance. Let's make use of these in
maintenance scripts now that we have it.

NOTE: There are still some static methods like in refreshLinks.php
that makes use of services that we can't use this method for now.

Change-Id: Idba744057577896fc97c9ecf4724db27542bf01c
2023-09-04 10:39:58 +00:00
rrana-wmf
017949e4cc Extend script to allow filtering by namespace
This patch introduces the --namespace option to filter
down the pages to warm PC with by namespace.

Bug: T338922
Change-Id: I60c721488e8976709700c4843344d7c515a8c6ce
2023-06-26 21:57:55 +00:00
Derick Alangi
8477516416 Process the cache warming activity in batches of 100 pages
In order not to potentially try running cache warming for million
of pages on large sites all at once, do this in batches of 100 pages
iteratively until we finish running for all pages.

Bug: T338922
Change-Id: I046b32675e186d5862a2b93eac1fe3a073ed9cc5
2023-06-27 00:56:54 +03:00
rrana-wmf
5488772c5f Populate parser cache with parsoid output.
The --force option is supported. NOTE: Without supplying any
arguments/options, the script will run for all pages in the wiki.

TODO: Run for all pages in batch sizes, so things don't explode
      if we have so many pages.

Bug: T338922
Change-Id: Id894f4769ad6f20137b13143178f50f227d7481a
2023-06-20 14:45:42 +03:00