Commit graph

177 commits

Author SHA1 Message Date
Ebrahim Byagowi
4c270a72ac Add namespace to WikitextContent
It adds MediaWiki\Content namespace to WikitextContent
and two classes related.

Change-Id: Ib74e4c5b3edac6aa0e35d3b2093ce1d0b794cb6d
2024-08-06 17:42:51 +03:30
Bartosz Dziewoński
df4cbf5ac6 Replace gettype() with get_debug_type() in debug/log/test output
get_debug_type() does the same thing but better (spelling type names
in the same way as in type declarations, and including names of
object classes and resource types). It was added in PHP 8, but the
symfony/polyfill-php80 package provides it while we still support 7.4.

Also remove uses of get_class() and get_resource_type() where the new
method already provides the same information.

For reference:
https://www.php.net/manual/en/function.get-debug-type.php
https://www.php.net/manual/en/function.gettype.php

In this commit I'm only changing code where it looks like the result
is used only for some king of debug, log, or test output. This
probably won't break anything important, but I'm not sure whether
anything might depend on the exact values.

Change-Id: I7c1f0a8f669228643e86f8e511c0e26a2edb2948
2024-07-31 19:33:57 +02:00
thiemowmde
75e9cbf5ca Stop using LBFactory in UserEditCountInitJobTest
I believe almost all other code was already updated to use the much
more narrow ConnectionProvider instead. I believe this was just
forgotten.

Change-Id: I07ef12bed522eea263806b09c9b4c8cb56723334
2024-07-18 15:20:12 +02:00
jenkins-bot
82d91ed739 Merge "Use MainConfigNames constants in tests where possible" 2024-07-11 12:46:04 +00:00
thiemowmde
4bd95cd96b Use MainConfigNames constants in tests where possible
I believe this makes the code less brittle, and also makes it a bit
more obvious what these strings are meant to represent.

Change-Id: Ia39b5c80af4b495931d0a68fd091b783645dd709
2024-07-10 10:11:22 +00:00
Ebrahim Byagowi
fab78547ad Add namespace to the root classes of ObjectCache
And deprecated aliases for the the no namespaced classes.

ReplicatedBagOStuff that already is deprecated isn't moved.

Bug: T353458
Change-Id: Ie01962517e5b53e59b9721e9996d4f1ea95abb51
2024-07-10 00:14:54 +03:30
jenkins-bot
68cebdfbb0 Merge "[ParsoidCachePrewarmJob] Use ParserOutputAccess" 2024-06-28 11:44:24 +00:00
Wandji69
c257e2276c Replace db with getDb for Tests
Bug: T316841
Change-Id: I29e535e8ee9b5641a4546d53b98cd5060d39681d
2024-06-23 23:47:56 +01:00
Amir Sarabadani
bb6c6e4174 schema: Drop old pagelinks columns
It has been dropped in production already.

Bug: T299947
Change-Id: I8ec1e7d9224c81d6494c39c78df9e4bdac38d377
2024-06-18 21:13:38 +02:00
C. Scott Ananian
105bb58ae2 [ParsoidCachePrewarmJob] Use ParserOutputAccess
One more step in gradually replacing uses of ParsoidOutputAccess.  This
one was pretty easy, as ParsoidOutputAccess was pretty much directly
calling ParserOutputAccess when provided with a ExistingPageRecord
and RevisionRecord.

Bug: T367074
Change-Id: I96161a64952e1809c0aec773d5a3dd4c71105657
2024-06-17 13:24:39 +00:00
C. Scott Ananian
a565e388f9 Move ParsoidOutputAccess::supportsContentModel() into Parsoid SiteConfig
The `supportsContentModel` method is really querying Parsoid for the
set of content models it supports, so it makes sense to put it in the
Parsoid-specific SiteConfig service.

This is part of the work to deprecate and remove ParsoidOutputAccess.

Change-Id: I81eb2df8cef93ede95361a4e03185b3d58e5b84b
2024-05-22 10:57:37 -04:00
Timo Tijhof
b432fd2f20 Move various job classes to relevant component directories
Motivation:

* Remove need to hardcode out-of-component files in various places.
  Especially for Gerrit queries, this not only reduces effort but
  also makes these files more likely to show up in the first place.
* Clarify component boundaries and ownership, e.g. when filing tasks
  with a stack trace.
* Now rather than later, to avoid need for multiple class renames and
  aliases when PSR-4 namespaces are applied later (T166010).

Background:

Similar to how jobs in extensions belong to that extension, the same
is true within core. There is a distinction between the JobQueue system,
and individual jobs belonging to a specific feature or component,
likewise for deferred updates, and special pages.

To make a begin with this, I propose moving the job classes to their
respective components for cases where component boundaries are already
clear. E.g. due to exclusively being queued by that same component,
being named/introduced by the same commit, being already documented
as such in component descriptions on mediawiki.org.

Bug: T364652
Change-Id: I73d6b97c6ca9f5abd29f4277d0f845426aa23bd0
2024-05-10 20:14:00 +01:00
thiemowmde
1f7886a6af Remove auto-generated "Class ClassName" comments
This is most certainly auto-generated by some IDEs. Unfortunately
there is nothing to learn from such comments. It's just noise.
Especially in tests.

Change-Id: Idf59332d96ca4718b6ce9d17b4da79a88641d4fd
2024-05-06 10:03:41 +02:00
Máté Szabó
95ad873d05 RefreshLinksJob: Migrate to StatsFactory
Switch metrics emitted by RefreshLinksJob to use StatsFactory and add
relevant tests.

Bug: T359365
Change-Id: I8677d72ed92ccf21968353cdff942017d5962511
2024-05-05 09:41:36 +00:00
Amir Sarabadani
8e183495e1 Stop using LoadBalancer::getConnectionRef() so it can be hard-deprecated
Bug: T326274
Change-Id: I90493d7cd4c21fdc022bcc19765fc04d986a9c8f
2024-04-30 13:31:08 +01:00
Taavi Väänänen
7c535b254e
tests: Fix PSR2.Classes.PropertyDeclaration.Multiple errors
Change-Id: Id4ac60a713391cf6360ff1995f20da94012ac1a0
2024-04-22 12:48:49 +03:00
Umherirrender
82134d5837 tests: Migrate to IDatabase::newUpdateQueryBuilder
Bug: T353219
Change-Id: Icecc444e6b4d6d2e9f4b13cda2931b10bb753318
2024-04-14 18:48:21 +00:00
Umherirrender
523372df32 Migrate to IDatabase::newDeleteQueryBuilder
Change-Id: Idf42d67c1b0b311f81a1d60a7a8a6f875f99e864
2024-04-12 20:12:05 +02:00
Reedy
85396a9c99 tests: Fix @covers and @coversDefaultClass to have leading \
Change-Id: I5629f91387f2ac453ee4341bfe4bba310bd52f03
2024-02-16 22:43:56 +00:00
Daimona Eaytoy
7acfa6a0a5 Replace more instances of unchecked MWException
Most (all?) of the remaining usages are caught somewhere and will be
migrated later.

Bug: T328220
Change-Id: I5c36693a5361dd75b4f1e7a0bab5ad48626ed75c
2024-01-23 16:20:53 +00:00
Umherirrender
388b0374fa tests: Use namespaced classes
Changes to the use statements done automatically via script
Addition of missing use statements and changes to docs done manually

Change-Id: Ib326ae1e5c8409a98398c721e8b8ce42c73bd012
2023-12-11 15:59:55 +01:00
James D. Forrester
4ed5ca48b1 Follow-up 71ff05267: Stop writing to tablesUsed in tests, now unnecessary
Bug: T342301
Change-Id: I5ea01f7ee103570165261bde0965c5b65e04c369
2023-11-21 09:02:48 -05:00
Subramanya Sastry
c8d0470f4b Make ParsoidOutputAccess a wrapper over ParserOutputAccess
* Updated ParserOutput to set Parsoid render ids that REST API
  functionality expects in ParserOutput objects.
* CacheThresholdTime functionality no longer exists since it was
  implemented in ParsoidOutputAccess and ParserOutputAccess doesn't
  support it. This is tracked in T346765.
* Enforce the constraint that uncacheable parses are only for fake or
  mutable revisions. Updated tests that violated this constraint to
  use 'getParseOutput' instead of calling the parse method directly.
* Had to make some changes in ParsoidParser around use of preferredVariant
  passed to Parsoid. I also left some TODO comments for future fixes.
  T267067 is also relevant here.

PARSOID-SPECIFIC OPTIONS:
* logLinterData: linter data is always logged by default -- removed
  support to disable it. Linter extension handles stale lints properly
  and it is better to let it handle it rather than add special cases
  to the API.
* offsetType: Moved this support to ParsoidHandler as a post-processing
  of byte-offset output. This eliminates the need to support this
  Parsoid-specific options in the ContentHandler hierarchies.
* body_only / wrapSections: Handled this in HtmlOutputRendererHelper
  as a post-processing of regular output by removing sections and
  returning the body content only. This does result in some useless
  section-wrapping work with Parsoid, but the simplification is probably
  worth it. If in the future, we support Parsoid-specific options in
  the ContentHandler hierarchy, we could re-introduce this. But, in any
  case, this "fragment" flavor options is likely to get moved out of
  core into the VisualEditor extension code.

DEPLOYMENT:
* This patch changes the cache key by setting the useParsoid option
  in ParserOptions. The parent patch handles this to ensure we don't
  encounter a cold cache on deploy.

TESTS:
* Updated tests and mocks to reflect new reality.
* Do we need any new tests?

Bug: T332931
Change-Id: Ic9b7cc0fcf365e772b7d080d76a065e3fd585f80
2023-10-13 15:03:03 -05:00
Amir Sarabadani
abbb4281cb tests: Migrate more cases to SQB
Bug: T344971
Change-Id: Ia69d82d6a6e623b9032240dc910fb47ff5887661
2023-09-21 18:30:37 +00:00
James D. Forrester
94ece673b2 Namespace TitleValue under \MediaWiki\Title
One of the big ones, so doing this alone.

Bug: T166010
Change-Id: I4c901d5c32696d8334ec30cede7d9b6f3d8d645e
2023-09-18 18:24:39 +01:00
Amir Sarabadani
5bd33d46ef Reorg: Move WebRequest to includes\Request
This has been approved as part of RFC T166010

Bug: T321882
Change-Id: I6bbdbbe6ea48cc1f50bc568bb8780fc7c5361a6f
2023-09-11 21:44:34 +01:00
James D. Forrester
7c85ac3c7c JobQueue: Drop getWiki(), deprecated in 1.33
Change-Id: I38fdaf0cd8ca98534dcbf64e13925138f8174a0d
2023-08-24 20:53:17 +00:00
Amir Sarabadani
15a278189f Reorg: Move MWTimestamp to MediaWiki\Utils
Bug: T321882
Change-Id: I48c10343295c4eb3d9ef8037343b0070e928f040
2023-08-19 05:53:40 +02:00
Daimona Eaytoy
4c72c161d8 Deprecate MediaWikiIntegrationTestCase::$users
This property shouldn't be needed in modern code, where most things can
use Authority. So much so because TestUser adds a dependency on the
database, but many tests that use TestUser don't even need the database.
ApiTestCase, in particular, sets this property in setUp, thus adding a
database dependency to all API tests, including those that don't need
users or the database at all.

Deprecate the property and replace existing usages in core. The one in
ApiTestCase is much harder to migrate, but this patch replaces the array
with an anonymous ArrayAccess class to allow lazy initialization and
remove DB dependencies.

Bug: T155147
Change-Id: I59c4ed1f6a7572d3a92387b15b8e56625bc376a2
2023-07-25 12:12:43 +00:00
daniel
b424671726 ParsoidCachePrewarmJob: enable deduplication
ParsoidCachePrewarmJob should be deduplicated by revision ID.
Note that this is not 100% accurate: the same revision may requrie
different renderings. However, we don't expect this to be relevant
within the short period of time the job spec is in the queue.

Bug: T341123
Change-Id: I56a9480132a01a158804a0fccd13b020c7103d08
2023-07-12 21:59:52 +02:00
Tim Starling
5e665d8cdb Migrate assertSelect() to SelectQueryBuilder
* Add MediaWikiIntegrationTestCase::newSelectQueryBuilder(), which
  creates a subclass of SelectQueryBuilder with assert methods.
* Migrate most callers of assertSelect() to this new query builder
  interface.

Bug: T311866
Change-Id: I7392b37988067020d5f684276320dae0a474631a
2023-06-28 09:03:58 +10:00
Daimona Eaytoy
1c6e872bd4 Replace usages of deprecated MWException
Use SPL exceptions instead when the exception is unchecked.

Bug: T328220
Change-Id: Ia1e5edc2ef3269a44b670262b78b305d07559829
2023-06-06 17:05:32 +02:00
Timo Tijhof
0075ebd83b objectcache,resourceloader,rdbms,jobqueue: Widen @covers annotations
Follows-up I4c7d826c7ec654b, I1287f3979aba1bf1.

We lose useful coverage and spend valuable time keeping these accurate
through refactors (or worse, forget to do so). The theoretically "bad"
accidental coverage is almost never actually bad.

Having said that, I'm not removing them wholesale (yet). I've audited
each of these specific files to confirm it is a general test of the
specified subject class, and also kept it limited to those specified
classes. That's imho more than 100% of the benefit for less than 1%
of the cost (more because `@covers` is more valuable than the fragile
and corrosive individual private method tracking in tests that
inevitably get out of date with no local incentive to keep them up to
date).

Cases like structure tests keep `@coversNothing` etc and we still don't
count coverage of other classes. There may be a handful of large
legacy classes where some methods are effectively class-like in
complexity and that's why it's good for PHPUnit to offer the precision
instrument but that doesn't meant we have to use that by-default for
everything.

I think best practice is to write good narrow unit tests, that reflect
how the code should be used in practice. Not to write bad tests and
hide part of its coverage within the same class or even namespace.
Fortunately, that's generally what we do already it's just that we
also kept these annotations still in many cases.

This wastes time to keep methods in sync, time to realize (and fix)
when other people inevitably didn't keep them in sync, time to find
uncovered code only to realize it is already covered, time for a less
experienced engineer to feel obligate to and do write a low quality
test to cover the "missing" branch in an unrealistic way, time wasted
in on-boarding by using such "bad" tests as example for how to use
the code and then having to unlearn it months/years later, loss of
telemetry in knowing what code actually isn't propertly tested due to
being masked by a bad test, and lost oppertunities to find actually
ununused/unreachable code and to think about how to instead structure
the code such that maybe that code can be removed.

------

Especially cases like LBFactoryTest.php were getting out of hand,
and in GlobalIdGeneratorTest.php we even resorted to reminding people
with inline comments to keep tags in sync.

Change-Id: I69b5385868cc6b451e5f2ebec9539694968bf58c
2023-04-10 22:00:17 +00:00
C. Scott Ananian
cfd9c516e1 Allow setting a ParserOption to generate Parsoid HTML
This is an initial quick-and-dirty implementation.  The
ParsoidParser class will eventually inherit from \Parser,
but this is an initial placeholder to unblock other Parsoid
read views work.

Currently Parsoid does not fully implement all the ParserOutput
metadata set by the legacy parser, but we're working on it.

This patch also addresses T300325 by ensuring the the Page HTML
APIs use ParserOutput::getRawText(), which will return the entire
Parsoid HTML document without post-processing.  This is what
the Parsoid team refers to as "edit mode" HTML. The
ParserOutput::getText() method returns only the <body> contents
of the HTML, and applies several transformations, including
inserting Table of Contents and style deduplication; this is
the "read views" flavor of the Parsoid HTML.

We need to be careful of the interaction of the `useParsoid` flag with
the ParserCacheMetadata.  Effectively `useParsoid` should *always* be
marked as "used" or else the ParserCache will assume its value doesn't
matter and will serve legacy content for parsoid requests and
vice-versa.  T330677 is a follow up to address this more thoroughly by
splitting the parser cache in ParserOutputAccess; the stop gap in this
patch is fragile and, because it doesn't fork the ParserCacheMetadata
cache, may corrupt the ParserCacheMetadata in the case when Parsoid
and the legacy parser consult different sets of options to render a
page.

Bug: T300191
Bug: T330677
Bug: T300325
Change-Id: Ica09a4284c00d7917f8b6249e946232b2fb38011
2023-03-26 21:46:05 -04:00
Tim Starling
5e30a927bc tests: Make some PHPUnit data providers static
Just methods where adding "static" to the declaration was enough, I
didn't do anything with providers that used $this.

Initially by search and replace. There were many mistakes which I
found mostly by running the PHPStorm inspection which searches for
$this usage in a static method. Later I used the PHPStorm "make static"
action which avoids the more obvious mistakes.

Bug: T332865
Change-Id: I47ed6692945607dfa5c139d42edbd934fa4f3a36
2023-03-24 02:53:57 +00:00
James D. Forrester
ad06527fb4 Reorg: Namespace the Title class
This is moderately messy.

Process was principally:

* xargs rg --files-with-matches '^use Title;' | grep 'php$' | \
  xargs -P 1 -n 1 sed -i -z 's/use Title;/use MediaWiki\\Title\\Title;/1'
* rg --files-without-match 'MediaWiki\\Title\\Title;' . | grep 'php$' | \
  xargs rg --files-with-matches 'Title\b' | \
  xargs -P 1 -n 1 sed -i -z 's/\nuse /\nuse MediaWiki\\Title\\Title;\nuse /1'
* composer fix

Then manual fix-ups for a few files that don't have any use statements.

Bug: T166010
Follows-Up: Ia5d8cb759dc3bc9e9bbe217d0fb109e2f8c4101a
Change-Id: If8fc9d0d95fc1a114021e282a706fc3e7da3524b
2023-03-02 08:46:53 -05:00
Amir Sarabadani
4bb2886562 Reorg: Migrate WikiMap to WikiMap/ out of includes
And WikiReference

Bug: T321882
Change-Id: I60cf4b9ef02b9d58118caa39172677ddfe03d787
2023-02-27 05:19:46 +01:00
Timo Tijhof
2249870a0f BacklinkCache: Remove unused internal clear() method
Follows-up Ife77661308e042.

Remove test call, which hasn't been needed in a little while because
it's not a static cache, and services are already reset between
tests.

Change-Id: I3992e5ae27d8b28a3289d071e8d3f45286b6abf7
2023-02-21 15:59:42 +00:00
Derick Alangi
83cdb69ed2 Avoid deprecated usage of Job::factory()
Change-Id: Ia2b8b71159ed8d387ae31f921ddcfd0d9136118c
2023-01-25 13:25:40 +02:00
daniel
025cc3d2be Parsoid: cache warming job: add render reason
We want to be able to track what activity causes renders and cache
writes. To achieve this, we need to plumb causeAgent and causeAction
from DerivedPageDataUpdater through ParsoidCachePrewarmJob to
ParserOptions.

Change-Id: I0274ec3976a8ef48ccb99156fb4fbeec85048189
2023-01-24 14:08:01 +01:00
daniel
d9003dd275 Add dependency injection for Job classes
Introduces JobFactory

Bug: T245900
Change-Id: I094cc9325e5eaf2717ae510ef988a72407195224
2023-01-19 15:44:11 +00:00
jenkins-bot
25572c28f8 Merge "tests: Handle dynamic created properties on test classes" 2023-01-13 21:50:49 +00:00
Umherirrender
abf2ea44e8 tests: Handle dynamic created properties on test classes
- Declare missing properties
- Remove unused or avoid set of properties
- Add AllowDynamicProperties

Bug: T314099
Change-Id: Ie631894f2ffbb616f6dd09851405d681f3eb8fcd
2023-01-13 21:03:42 +00:00
Derick Alangi
99d989087d jobqueue: Warm parsoid parser cache using a job
This patch introduces ParsoidParserCachePrewarm job
that is used to warm PC with parsoid outputs in order
to speed up page reads on large wikis.

Bug: T322427
Change-Id: Ib63a02d3cf5348b36f4f166ff6939f4d2e7fef6f
2023-01-11 15:15:15 +01:00
Kevin Israel
d2ffc38ff8 tests: Set serialize_precision=-1 in TestSetup
Starting in PHP 7.1, this is now the default[1][2], and results in the
"shortest string that yields d when read in and rounded to nearest."[3]

As such, it would generally be the best choice for tests. The existing
value of 17 has been used only because it was the default at the time
(October 2015, see commit 71443707ad), and -1 for "dtoa mode 0"
simply was not an option yet.

[1]: https://wiki.php.net/rfc/precise_float_value
[2]: https://www.php.net/manual/en/migration71.other-changes.php#migration71.other-changes.ini-file-handling
[3]: https://github.com/php/php-src/blob/php-8.1.13/Zend/zend_strtod.c#L3713-L3714

Change-Id: I95c3c36c90be5aa0af22c3c77ce67641b81786d7
2022-12-14 23:18:13 -05:00
Reedy
0cb2c3c106 Fix casing of class and function name usages
Bug: T253628
Change-Id: I5c64f436d3cf757390b751ce3e34bfc7872bc176
2022-12-04 19:09:30 +00:00
jenkins-bot
1d045146d0 Merge "Make use of the new PageUpdateStatus" 2022-12-01 10:45:32 +00:00
daniel
166b569444 Make use of the new PageUpdateStatus
PageUpdateStatus provides clean access the the newly created
RevisionRecord.

Depends-On: Ia08c586198082ea47e8313d0d41835f9830fb29e
Change-Id: Id6963842321c4eaa3d7d029ad0b769f73433c103
2022-11-30 17:56:58 +00:00
Tim Starling
28750729bd Miscellaneous PHP version related cleanups
MediaWikiTitleCodec: I removed the comment about dbkey being
"conveniently nullified" since that is no longer correct. The first
preg_replace() can return null, and that is guarded. The second
preg_replace() hopefully can't return null, because if it does, trim()
will generate a deprecation notice on PHP 8.1.

Some other self-explanatory changes.

Change-Id: Iad0ace821eba782c3033ec8abfeac461ac4e8ace
2022-11-21 10:11:53 +11:00
Zabe
f6b9381d7f Revert "Reorg: Move some of request related classes to MediaWiki/Request"
This reverts commit 2bdc0b2b72.

Reason for revert: T166010#8349431

Bug: T166010
Change-Id: Idcd3025647aec99532f5d69b9c1718c531761283
2022-10-27 13:14:16 +00:00