Commit graph

1746 commits

Author SHA1 Message Date
C. Scott Ananian
84d0dff968 HtmlHelper: Add "legacy compatibility" mode
When using HtmlHelper for style deduplication, slight differences in
the serialization format used by the legacy parser caused test failures.
Add a "compatibility" mode which tries to better match legacy parser
behavior for void elements, character escapes, and other details.

Parsoid HTML has always been serialized using an HTML5 serializer,
so this compatibility mode will be disabled when processing Parsoid
HTML.

Change-Id: I0441aa3e44f6562e05e95a18cc282c53fe446788
2023-12-11 21:04:50 +01:00
Tim Starling
0b143b0e26 rdbms: Simplify TRUNCATE handling
Tracking of "pristine" status in Database supported omission of TRUNCATE
queries for temporary tables that are untouched by unit tests. Since we
now detect the used tables, this is no longer necessary.

Also, make the temp table info into a class.

Change-Id: I57ff5b43cc7551ca32130f6987edd5f7c4f79910
2023-12-11 14:29:52 +11:00
jenkins-bot
6b7b92c72b Merge "Remove support for extensions requiring a MW version via Composer" 2023-12-10 05:06:15 +00:00
Isabelle Hurbain-Palatin
d50deb23e8 Introduce ContentDOMTransformStage
This subclass is used for transformations that apply to the DOM tree.
For now, we load the document from the string in ParserOutput and store
it back; ultimately, this will be replaced by a direct access to the DOM
content of the ContentHolder/HtmlHolder.

Change-Id: Ibc46c34072eadb3e84f69df49a3a424f3864c952
2023-12-08 18:27:30 -05:00
Isabelle Hurbain-Palatin
a3f51c732d Refactor DefaultOutputTransform into a pipeline of transforms
Bug: T348253
Change-Id: I53551ec6d6471569709c71c1155729e550f64de8
2023-12-08 18:06:19 -05:00
jenkins-bot
3dcc6006ce Merge "Output: Add IframeSandbox class" 2023-12-08 22:29:59 +00:00
Max Semenik
d0e0113648 Remove support for extensions requiring a MW version via Composer
Dependency of an extension upon particular MediaWiki versions should
be expressed in extension.json. As brought up on wikitech-l, we still
have the code that injects a virtual mediawiki/mediawiki package into
Composer so that extensions can express a dependency upon particular
MediaWiki versions in composer.json. This is duplicate information
potentially introducing inconsistency. This patch removes support for
this capability. Per codesearch, there are no remaining extensions
found using this feature.

The classes in includes/composer are removed immediately as they've
never been intended for extensions to use, while MediaWikiVersionFetcher
is marked for standard deprecation.

Bug: T467
Bug: T249573
Change-Id: I62bacca45e4680812dd42f4e061ca7da17bfcdbb
2023-12-07 05:10:13 -08:00
Daniel Kinzler
b73cc87dd1 Re-apply "REST: Emit swagger spec"
This reverts commit 890558f1fa.
This restores Id584208d9b67d877606a0add1d71c9b1784cdb1b with some fixes.

Bug: T323786
Bug: T352742
Change-Id: Ib31c451ddd75b06c95a544c8a3d2a64b32264126
2023-12-06 11:20:11 +01:00
Urbanecm
890558f1fa Revert "REST: Emit swagger spec"
This reverts commit bb4b5c5f87.

Reason for revert: patch breaks CI

Bug: T352742
Change-Id: Iaf57bad945f5cbd01508a513f0d219ec4b510ce8
2023-12-05 10:37:48 +00:00
daniel
bb4b5c5f87 REST: Emit swagger spec
This is only enabled in development mode for now.
It's intended as a baseline for further development,
the feature is not ready for production.

Bug: T323786
Change-Id: Id584208d9b67d877606a0add1d71c9b1784cdb1b
Co-authored-by: Atieno <pnjira@wikimedia.org>
2023-12-04 16:28:32 +03:00
C. Scott Ananian
cb09a27cc2
Output: Add IframeSandbox class
Add a class for wrapping arbitrary HTML content in an iframe,
with an interface similar to that of OutputPage. The iframe
uses 'sandbox="allow-scripts"' (which disallows various actions
for scripts executed in the iframe, most notably access to any
browser data for the website embedding the iframe), making it
suitable for running untrusted scrips.

This is a minimal patch to get the a feature in a state where
it can be tested in staging. It's not intended for real-world
use yet.

See P53299 for hooking up the code for manual testing.

Soft-depends on  I3ce24a1b1d2635ba46d1af1c51dccb292dfdbc55.

Bug: T222807
Bug: T169027
Co-Authored-By: Gergő Tisza <tgr.huwiki@gmail.com>
Co-Authored-By: C. Scott Ananian <cscott@cscott.net>
Change-Id: I658c7f3c751505ce4045047a19f11da1d236cd88
2023-12-03 22:33:17 -08:00
Umherirrender
094f0b4fe6 Remove Hooks class, all contents deprecated in 1.35 or 1.41
This includes the deprecated static functions:
- Hooks::register()
- Hooks::clear()
- Hooks::isRegistered()
- Hooks::getHandlers()
- Hooks::run()
- Hooks::runWithoutAbort()
- Hooks::runner()

Bug: T335536
Change-Id: I77b8cccefe50868a087a20fc67a3d8f8d5fb1a63
2023-11-30 17:14:00 -05:00
jenkins-bot
b7fc1b2f43 Merge "Only cache expensive renderings" 2023-11-30 21:24:34 +00:00
daniel
e3fb964439 Only cache expensive renderings
Pages that are fast to render can be omitted from the parser cache
to preserve disk space and cache write operations.

The threshold is configurable per namespace, so the tradeoff can
be evaluated based on different access patterns. For example, pages
that are accessed rarely, like file description pages on commons,
may have a high threshold configured, while pages that are read
frequently, like wikipedia articles, may be configured to be always
cached, using a 0 threshold.

Filtering is based on a time profile recorded in the ParserOutput.
A generic mechanism for capturing the timing profile is implemented
in the ContentHandler base class. Subclasses may implement a more
rigorous capture mechanism.

Bug: T346765
Change-Id: I38a6f3ef064f98f3ad6a7c60856b0248a94fe9ac
2023-11-30 20:56:12 +00:00
Martin Urbanec
6ec01553c5 Move StaticUserOptionsLookup to the MediaWiki\User\Options namespace
Follow-up to I9822eb1553870b876d0b8a927e4e86c27d83bd52,
I missed the Static class in that patch.

Bug: T352284
Change-Id: I42653491c19dde5de99e0661770e2c81df5d7e84
2023-11-29 14:08:40 -05:00
Martin Urbanec
29af4dd074 Move user options related classes into its own namespace
There are a couple of user options related classes already,
and the T321527 work on dynamic defaults is going to add
even more. Let's move them into a separate namespace
to make core a bit more organized.

Old name is kept as an alias for compatibility purposes.

Bug: T321527
Bug: T352284
Change-Id: I9822eb1553870b876d0b8a927e4e86c27d83bd52
2023-11-29 13:27:13 +01:00
Tim Starling
d6727856fd Support new block schema
Support migration stages when reading and writing blocks.

I tried to set it up for an easy next stage, in which support for the
old schema is removed. I tried to avoid factoring out of shared code
between the two schemas, so that the old schema cases can simply be
deleted without the need to revert unnecessary abstractions.

However, I added HideUserUtils to factor out ipb_deleted queries. Code
review showed that this was already quite complex, with multiple
approaches to the problem, so it benefits from refactoring even without
the schema abstraction.

HideUserUtils is a service rather than a standalone class to support
unit tests, since unit tests do not allow global config access. When
the migration stage config is removed, it will be a service with no
constructor parameters -- an unnecessary abstraction which should
ideally be resolved at that time.

When interpreting result rows, it is possible to share code by using
field aliases. But when constructing WHERE conditions, the actual field
names need to be used, so the migration is more intrusive in
ApiQueryBlocks and SpecialBlockList, where complex conditions are used.

Bug: T346293
Bug: T51504
Bug: T349883
Change-Id: I408acf7a57b0100fe18c455fc13141277a598925
2023-11-29 13:31:42 +11:00
jenkins-bot
77b3df20da Merge "title: Drop TitleArray in favor of TitleArrayFromResult" 2023-11-28 18:02:04 +00:00
James D. Forrester
67217d08df Namespace remaining files under includes/deferred
Bug: T166010
Change-Id: Ibd40734b96fd2900e3ce12239d09becfb4150059
2023-11-22 10:08:53 -05:00
Daniel Kinzler
32c12be6fe Re-apply "Rest: replace use of deprecated pingLimiter method"
This introduces RestAuthorizeTrait to ensure proper error reporting
after calls to Authorizer methods to avoid misleading error reports,
see T350117 and T350202.

This reverts commit e047668d9f.
This restores change 701ff30193.

Change-Id: I617cb7ba24a1614c39e2b1072888f0ee7b3127e3
2023-11-22 13:57:06 +11:00
Daimona Eaytoy
71ff052677 Replace MediaWikiIntegrationTestCase::$tablesUsed with automatic query tracking
This patch deprecates the $tablesUsed property, and introduces a new
utility, ChangedTablesTracker, that automatically keeps track of all
tables changed in a test. Every table used is now guaranteed to be reset
after the test.

Note that tables changed in addDBDataOnce are only cleared at the end of
the test class, or the data would be lost.

Fix a test in SpecialBlockTest which would fail with this patch.

$tablesUsed is now a no-op and can be removed from all tests that
declare it.

Bug: T342301
Change-Id: Ie2f1809dac243ef06ba0c34f039ce4e62cbf99cf
2023-11-21 10:35:59 +11:00
Amir Sarabadani
beb3261b8d Remove language coverter for Kazakh
This has been constantly mentioned as buggy and broken and there is no
official version of latin or Arabic (see the ticket for more details).

This can be turned back as an extension if needed by third party users.

Bug: T350684
Bug: T268143
Depends-On: I6180dca2c49b3119751766268acc56087aaf8414
Change-Id: Ifbf3c8954d885daf891f8d9efc11743d898302f0
2023-11-20 10:31:16 -05:00
jenkins-bot
1a1ffc4ad6 Merge "Edit Recovery: add new special page to list unsaved changes" 2023-11-13 12:06:57 +00:00
Sam Wilson
fb2f0d003c Edit Recovery: add new special page to list unsaved changes
Add Special:EditRecovery to dispaly a basic list of all pages
with locally-saved edit recovery data. This change just sets up
the initial work for this, and the actual design and improved
UX will come in subsequent changes.

Bug: T347673
Change-Id: I8edbfd21258fcb2e4fc9f3e4ded9876d6635d752
2023-11-12 15:28:25 +08:00
jenkins-bot
a9a163f62f Merge "Change EditPage 'edit' rights check to do per page instead of general" 2023-11-10 01:12:33 +00:00
Martin Urbanec
c9908da103 IP Masking: Expire temporary accounts in 1 year
Why:
Temporary accounts (introduced as part of IP Masking)
are supposed to expire 1 year after their registration.
Automatic account expiration can be done via a maintenance
script, which would be periodically executed via cron / systemd.

Make it possible for extensions to provide their own logic
for generating a list of temporary accounts to invalidate.
This is used in CentralAuth to base registration timestamp
on the global registration timestamp.

The default behavior is "temporary accounts do not expire",
given the feature requires a maintenance script to run
periodically, which will not be the case on third party
instances.

What:
* Add `expireAfterDays` to $wgAutoCreateTempUser, controlling
  how many days temporary accounts have.
* Add UserSelectQueryBuilder::whereRegisteredTimestamp(),
  filtering accounts based on user_registration.
* Add ExpireTemporaryAccounts maintenance script, which is
  @stable to extend.

Bug: T344695
Change-Id: If17bf84ee6620c8eb784b7d835682ad5e7afdfcc
2023-11-09 15:11:48 +01:00
Brian Wolff
d3f38054a9 Change EditPage 'edit' rights check to do per page instead of general
Previously the edit rights check would fail if you didn't have edit
rights in general. This made it difficult to write custom permission
extensions that want to give users edit rights to a few specific pages
while not giving rights in general.

This check goes back to 2005 when the edit rights system was first
introduced a2a28a34c1. Presumably per-page permissions didn't exist
yet.

Bug: T342515
Change-Id: Ibcf0e18c44912abdd02a676a551da3cc7ed22ee0
2023-11-04 21:46:03 +06:00
jenkins-bot
69b6191820 Merge "Remove cleanupUsersWithNoId.php" 2023-11-03 22:56:56 +00:00
jenkins-bot
fc41681088 Merge "Move DatabaseBlock read query methods to DatabaseBlockStore" 2023-11-03 17:54:30 +00:00
Amir Sarabadani
f60e576c69 rdbms: Add support for LIKE in expression builder
Bug: T210206
Change-Id: Iec33a64bb1ec1485ce91b8b05e660f8c1723182b
2023-11-03 02:03:44 +01:00
Tim Starling
d06ac1a3cd Move DatabaseBlock read query methods to DatabaseBlockStore
* Move to DatabaseBlockStore the DatabaseBlock methods newFromID,
  getQueryInfo, getRangeCond, newFromRow, isExemptedFromAutoblocks,
  doAutoblock, updateTimestamp, getAutoblockExpiry, newFromTarget,
  newListFromTarget.
* Split DatabaseBlock::getBlocksForIPList. Now
  BlockManager::getBlocksForIPList() is responsible for XFF header
  validation and trusted proxy handling. DatabaseBlockStore::
  newListFromIPs() just does the queries and constructs the Block
  objects.
* In DatabaseBlockStore::newFromRow() and doAutoblock(), use the
  DatabaseBlock constructor instead of calling many setter methods. Add
  constructor options decodedExpiry, decodedTimestamp, id,
  parentBlockId and restrictions to support this.
* Move isExemptedFromAutoblocks() to its own service. Remove the cache
  since in my testing with production eval.php, the WAN cache fetch is
  10 times slower than just using the message cache, contradicting the
  comment written in 2008.
* Fix AuthManagerTest which was previously passing an unrecognised
  "restrictions" option to DatabaseBlock. Now that the option actually
  works, we have to use the right type.

Bug: T255433
Change-Id: I5049e60be1681f67fcca133e569e315792dc42dd
2023-11-03 11:00:57 +11:00
Tim Starling
cbe6714a99 Remove cleanupUsersWithNoId.php
All of the fields it's cleaning up have been deleted. The script cleans
up denormalized username fields which were deleted in the actor
migration.

It skips tables for which the name fields don't exist, which is all of
them.

Change-Id: I407a75c85cbd5ff6ab0d6d48d2bf07793e7c0c3e
2023-11-03 10:51:30 +11:00
jenkins-bot
13c7f7d56f Merge "Add ApiCreateTempUserTrait for APIs that create temporary users" 2023-11-02 19:55:31 +00:00
Thalia
9396cd6eed Add ApiCreateTempUserTrait for APIs that create temporary users
Refactor EditPage's logic for getting a redirect URL into a trait
that can be shared between classes that exteend ApiBase.

Bug: T349223
Change-Id: I7cafa0bfc17a75d44e1d83b9561b4b2778b5f132
2023-11-01 13:29:02 +00:00
Amir Sarabadani
f64db42f62 rdbms: Drop some of class aliases of DatabaseMySQL
They were supposed to be dropped before 1.41 release.

Depends-On: Icd972535725e65e6eaed25607f1fd1c69d6b3a5c
Change-Id: Ieeaf39ec07407a55daa51c350cd57b66283a27bb
2023-10-31 23:18:21 +00:00
Derick Alangi
e54665dccb title: Drop TitleArray in favor of TitleArrayFromResult
This patch kills TitleArray and makes TitleArrayFromResult the way
forward.

Change-Id: I056edecbee69114c77d14ecc9e7fe6212262a970
2023-10-31 12:01:51 +00:00
daniel
04197cb13f Introduce StatusFormatter
This takes us one step closer to deprecating Status,
so we can isolate StatusValue from presentation logic.

FormatterFactory is introduced as a mechanism for
getting instance of formatters that need access to the user interface
language and other request dependent information.

Usage is demonstrated in thumb.php, SpecialCreateAccount, and
SearchHandler. The examples indicates that there is no work do
be done around ErrorPageError and LocalizedHttpException.

Change-Id: I7fe5fee24cadf934e578c36856cc5d45fb9d0981
2023-10-27 14:56:51 +02:00
Derick Alangi
5bca452be3
lib/virtualrest: Drop the virtualrest library
Code is no longer used and all previous usage has been replaced
with the appropriate HTTP logic and cleanup has been taken care
of.

As a follow-up to this patch, I'll be finally removing the
`$wgVirtualRestConfig` setting from core and extensions.

Bug: T338921
Change-Id: I7e1fff996033ba26e1b3dd229dd299a64c610707
2023-10-23 11:02:59 +01:00
jenkins-bot
49859c6bd4 Merge "rdbms: Introduce expression builder" 2023-10-20 20:49:27 +00:00
Amir Sarabadani
3ef6662f31 rdbms: Introduce expression builder
To avoid accepting raw SQL in conditions and avoid the need to call to
addQuotes() which might get forgotten and open a SQL injection vulnerability.

A migration path would be something like the following:

Before:
'rev_timestamp > ' . $db->addQuotes( $timestamp )
After:
$db->expr( 'rev_timestamp', '>', $timestamp )

See the ticket for more information and reasoning.

Bug: T210206
Change-Id: I72431410c21ab97e797c7df07bf0dedd19307e65
2023-10-20 20:15:55 +00:00
Tim Starling
69e8a152d5 Remove UserRightsProxy
And have UserGroupManagerFactory cache instances, per the TODO.

I'm doing this because SpecialUserRightsTest was flaky when there are
two local instances of UserGroupManager floating around. Resolving the
todo is the simplest way to ensure there is only a single
UserGroupManager. There are no remaining usages of UserRightsProxy in
codesearch.

Change-Id: Icb3395dfc53a4441b667f001ac1227f36d1f2e8d
2023-10-20 10:10:24 +11:00
Isabelle Hurbain-Palatin
36b4ab44f6 Refactor ParserOutput::getText into DefaultOutputTransform service
This also introduces the ephemeral field "$mTransformedText" to store
the result of transformation in ParserOutput.

This is a first step before the transformation uses HtmlHolder as input
and output.

Bug: T348253
Change-Id: I312f3748ebfb0373ee3542ba0abdeefe7db1d488
2023-10-16 13:11:38 +02:00
Tim Starling
03ceb372b3 Add cache to BlockManager
Add a cache to BlockManager, so that we can get rid of the block cache in
User.

The cache stores up to three blocks -- a request block, a user block and
an anonymous (IP) block. Using object identity to compare keys means
that we can avoid serializing the WebRequest and UserIdentity.

Bug: T345683
Change-Id: I7639766519861690a759629144c8680767539293
2023-10-12 13:42:58 +11:00
jenkins-bot
e8340a75c7 Merge "Remove deprecated interface ParserTestTablesHook" 2023-10-03 20:11:04 +00:00
jenkins-bot
aa5008f24e Merge "ResourceLoader: Improve and generalize validateScriptFile test coverage" 2023-10-03 19:33:52 +00:00
Umherirrender
42152f5fb4 Remove deprecated interface ParserTestTablesHook
The hook is no longer called, so the deprecation warning of the
HookContainer (as mention in DeprecatedHooks) cannot work as expected.

Change-Id: Idb586e9c2d8ba87411acc137537db6d6c75485a7
2023-10-03 19:52:49 +02:00
jenkins-bot
c7fe8a109d Merge "Revert "Revert "Remove unused class_alias deprecated in MW 1.40""" 2023-10-03 17:03:31 +00:00
Timo Tijhof
8f778a8b23 ResourceLoader: Improve and generalize validateScriptFile test coverage
In preparation for the underlying library changing, generalize the
tests for this code so that test changes are kept to a minimum in
the actual commit. Rather than test the upstream directly, move
our custom test cases from JsMinPlusTest to ResourceLoader/ModuleTest.

Fix benchmarkJSMinPlus.php to not emit warnings on PHP 8.2 due to
use of dynamic property creation:

> PHP Deprecated:  Creation of dynamic property JSNode::$readOnly
> is deprecated in /mediawiki/includes/libs/jsminplus.php:1728

Bug: T75714
Change-Id: I0515be8229f51de48c1f0c92f09e2d827e1b1add
2023-10-02 18:46:25 -07:00
Fomafix
19d4adbe1c Remove deprecated class alias GetBlockErrorMessageKey
Deprecated since 033e75cb26 included in MediaWiki 1.40.
Use GetBlockErrorMessageKeyHook instead.

Change-Id: I346c7ec9ed7d57e3d1e1aa9f761d24e87c4ec513
2023-10-02 21:44:19 +00:00
Isabelle Hurbain-Palatin
33908cebed Revert "Revert "Remove unused class_alias deprecated in MW 1.40""
This reverts commit cbde6b69de to re-apply
the initial patch. This should only be merged once
I2acfd0b7a1e48aec107ded3bbe4963e2df24f4d3 is deployed.

Change-Id: If12ab65b1d773946fca6c8601ff51290136549c8
2023-09-29 13:08:50 +02:00