Commit graph

297 commits

Author SHA1 Message Date
Atieno
da8d30e0c6 [REST Sandbox] Remove SwaggerUI from MediaWiki Releases
Bug: T397470
Change-Id: I4e25792e347e8eeeb1dee4db1d25128948c1cdd0
2025-06-30 16:05:49 +01:00
Reedy
30c8f812ac DnsBlacklistUrls: Remove sorbs.net
Service is no longer running, so it's a default that doesn't do anything

Bug: T382987
Change-Id: I3a21c12ba689928d38e410cbe2547ab7e616ac8a
(cherry picked from commit 4a6fac8b7dadfdffe6e0e239b8a551436e147d97)
2025-01-06 22:57:01 +00:00
Kevin Israel
7d250d2109 Remove CryptHKDF and MWCryptHKDF
MWCryptHKDF was added ten years ago (in af66c04d39), and as far as
I can tell, it was never used anywhere. It seems unlikely that CryptHKDF
will be used in the future, at least in its current form, for several
reasons:

* PHP 7.1.2+ has hash_hkdf(), so HKDF() would not be needed.

* At the time MWCryptHKDF was created, access to a CSPRNG was dependent
  on server configuration: operating system, enabled PHP extensions,
  open_basedir, etc. The "clock drift" RNG used as a last resort was not
  considered to be secure or fast enough for generating large amounts of
  output.[1] random_bytes(), added in PHP 7, changed the situation.

* Depleting the input pool of Linux's RNG is no longer a concern; there
  is no more blocking output pool for /dev/random.[2][3] In 2022, this
  change and others, including some that improved performance,[4] were
  backported to stable kernels as old as 4.9.[5]

* $wgAuthenticationTokenVersion obviated the primary use case of
  quickly resetting the user_token field for all users, assuming all
  the existing tokens are unique.

* CryptHKDF seems to perform much slower than random_bytes(), at least
  on Linux, making it pointless to use given that the other reasons for
  its existence no longer apply.

[1]: https://bots.wmflabs.org/logs/%23mediawiki-core/20161004.txt
[2]: https://lwn.net/Articles/808575/
[3]: https://lore.kernel.org/all/cover.1577088521.git.luto@kernel.org/
[4]: https://www.zx2c4.com/projects/linux-rng-5.17-5.18/
[5]: https://lore.kernel.org/all/Yo3pmh9hiUFtQz77@zx2c4.com/T/

Change-Id: I29136fad826341d21728671aa30285d5551f1162
2024-11-10 22:49:37 -05:00
jenkins-bot
7390cb8993 Merge "Introduce minimal OTEL tracing library" 2024-10-10 17:33:42 +00:00
Máté Szabó
16ec1a3703 Introduce minimal OTEL tracing library
In T340552, the official PHP OpenTelemetry client was effectively
rejected for inclusion in MediaWiki due to its size. Implement a minimal
tracing library instead that eschews conformance with the OTEL client
specification in favor of simplicity, while remaining capable of
emitting trace data in OTLP format and thus retaining compatibility with
any ingestion endpoint capable of handling OTLP.

In its current state, the library supports a basic feature set that
should be sufficient for basic tracing integration:

* Span creation, inclusive span activation and automatic parent span
  assignment,
* Span attributes and span kinds,
* Basic resource (process/request)-level metadata generation,
* Data export over OTLP.

Additional functionality, such as trace propagation, can then be
incrementally added to the library.

Bug: T340552
Change-Id: Ibc3910058cd7ed064cad293a3cdc091344e66b86
2024-10-09 15:55:31 +02:00
Timo Tijhof
b18121d98c rdbms,objectcache: Replace wgChronologyProtectorStash with MicroStash
Bug: T336004
Change-Id: I2f769aa703ce98b15fa0fe98eda092ff19c27d0a
2024-10-09 01:52:55 +01:00
Alexander Vorwerk
71d332c2c3 Make revision-slots expiry configurable
Bug: T183490
Change-Id: I08e35ee64eab081de85ed8745361ebc4c9025d91
2024-09-30 10:24:30 +00:00
Bartosz Dziewoński
943d089008 Replace raw HTML copyright footer message with wikitext one
Replaces 'copyright' with 'copyright-footer' and 'history_copyright'
with 'copyright-footer-history' (the original still takes precedence
if set). Adds SkinCopyrightFooterMessage hook which works the same
way as SkinCopyrightFooter for the new messages. Allows disabling
the old messages by setting $wgAllowRawHtmlCopyrightMessages = false.

Co-Authored-By: Gergő Tisza <tgr.huwiki@gmail.com>
Bug: T45646
Change-Id: I5fd5607f8d43b6e934c8d4d35097cec430c56043
2024-09-26 16:18:52 +02:00
C. Scott Ananian
92ca7f68a4 Randomly sample statistics for Parsoid Selective Update
Controlled by $wgParsoidSelectiveUpdateSampleRate (which defaults to off)
randomly sample 1 in N parses to collect statistics to inform the design
of Parsoid selective update:

* For both legacy parses and Parsoid, count how many times a previous
  parse is in the cache when a new parse is requested.  This needs to
  sample the legacy parser as well as Parsoid because Parsoid is not
  yet invoked from the RefreshLinksJob.  We also count the relative
  number of parses from the different
  RevisionRenderer::getRenderedRevision() call sites to determine
  which pathways might account for the most opportunities for
  optimized selective update.

* For sampled parses using the Parsoid parser where a previous parse
  result is available, also fetch the previous wikitext source from the
  database.

Bug: T371713
Change-Id: I208aeac1b315a96bdb9669427cd03de461b914b4
2024-09-13 19:29:18 -04:00
dvorapa
10ab0e40a9 parser: Add a new {{USERLANGUAGE}} magic word for use in wikitext
Depending on configuration, this returns either the interface language
code of the current user or the current page language.

Bug: T4085
Change-Id: Iab7fda272ec81af88c74612727ff6bed014d4a81
2024-09-07 19:16:32 +00:00
Máté Szabó
f89aa38f69 objectcache: Remove WinCache support
WinCache is an APCu equivalent for use with Microsoft IIS, but in recent
years has been unmaintained and lacks support for PHP 8 and newer.[1]
So, remove support for it as MediaWiki will be raising the minimum
supported PHP version to 8.1.

[1] https://www.php.net/manual/en/install.windows.recommended.php

Bug: T365691
Change-Id: I4d2dc01a9119bb1f858132f0146b894750c1e86d
2024-09-05 17:59:26 +00:00
jenkins-bot
01abfbfcbd Merge "Add option to sort categories in OutputPage" 2024-08-29 17:03:13 +00:00
C. Scott Ananian
493df826b9 Add option to sort categories in OutputPage
Some wikis treat the category list from ParserOutput as a /set/, others
as an /ordered list/.  For those who don't care about the order of
categories, provide the option for wikis to sort the categories
in OutputPage.

This can also be activated with a query parameter, `&sortcat=1`, which
is useful to the Parsoid team when doing visual diff testing to avoid
false positives caused by differences in category ordering.

Bug: T373480
Change-Id: Idd14650a1898c6a49c88441ef024ce3012903bbe
2024-08-29 12:20:19 -04:00
MusikAnimal
ef14cd41c3 PasswordReset: remove $wgAllowRequiringEmailForResets feature flag
Update a few tests that relied on the feature flag to ignore
the 'requireemail' preference on "User1" to instead use "User2",
who doesn't have the preference set.

Bug: T242406
Change-Id: I996d3996272d704a071d1d2094c3568247b80f98
2024-08-28 00:55:43 +02:00
jenkins-bot
516450947b Merge "Codex: Allow a local development version to be used" 2024-08-23 21:16:16 +00:00
Roan Kattouw
8a39d83175 Codex: Allow a local development version to be used
Developers can use this to test their local version of Codex with
MediaWiki by pointing $wgCodexDevelopmentDir to their local clone of the
Codex repo, e.g. $wgCodexDevelopmentDir = '/home/yourname/git/codex';

Setting $wgCodexDevelopmentDir affects where the following things come
from:
- Codex JS/CSS files for the full library
- Codex JS/CSS files for code-split chunks, and the manifest.json file
  that points to them
- Icons retrieved by CodexModule::getIcons()
- CSS-only icons imported in Less
- Design tokens imported in Less

Other changes in this patch:
- Add CodexModule::makeFilePath() to centralize the repeated path
  concatenation. This makes it easier to switch out the regular path for
  the dev mode path.
- Replace all uses of $IP (which is deprecated) and MW_INSTALL_PATH in
  CodexModule with the BaseDirectory config setting.
- Make CodexModule::getIcons() reset its static cache if the path to the
  icons file changes. Without this, it's impossible to make the unit
  tests pass.
- Move the i18n messages code from the CodexModule constructor to
  getMessages(). It can't be in the constructor because makeFilePath()
  doesn't work there (it fails because the Config object hasn't been set
  up yet).
- Add a 'mediawiki.skin.codex' import path so that we can stop
  hard-coding the path to the Codex mixins file. Without this, we can't
  make the Codex mixins come from the right place in development mode.
- Consider $wgCodexDevelopmentDir in setting the cache key for compiled
  Less code, since changing this setting can change the output of Less
  compilation (by changing design tokens, icons or mixins).
- Add unit tests for (the non-dev mode behavior of)
  CodexModule::getIcons() and the i18n message key handling.

Bug: T314507
Change-Id: I11c6a81a1ba34fe10f4b1c98bf76f0db40c1ce98
2024-08-22 17:20:24 -07:00
Hannah Okwelum
393fa39db5 ResourceLoader: Turn on ResourceLoaderUseObjectCacheForDeps by default
Bug: T343492
Change-Id: I1e7dd95a1255437894e4bd9a1feb9ab354d952ca
2024-08-15 19:28:03 +00:00
jenkins-bot
6c2aa36829 Merge "Stats: Remove $wgSamplingStatsdClient, deprecate SamplingStatsdClient class" 2024-08-08 21:00:34 +00:00
Timo Tijhof
9d439fe64e Stats: Remove $wgSamplingStatsdClient, deprecate SamplingStatsdClient class
== Motivation ==

* Reduce amount of configuration switches relating to MW Stats component,
  to reduce noise for MW sysadmins and devs.

* Reduce amount of code in the legacy Statsd implementation.

== Background ==

* commit e56f7b6c63 (2015, MW 1.26): Add statsd sampling.
  Refers to T106457 under T106450 ("hook usage counts"), which used it
  in https://gerrit.wikimedia.org/r/226640 which was reverted shortly
  after in https://gerrit.wikimedia.org/r/233045 due to too high
  overhead of stats code for something as hot as hooks, regardless
  of actual sampling/stats sending.

* commit b203ec5fcd (2016, MW 1.29): Make statsd sampling configurable.
  Use case unknown, but it is not used today in wmf-config, and
  appears to have never been set in the past either:

  ```
  operations-mediawiki-config (master)
          $ git log -p wmf-config/ | grep wgStatsdSamplingRates
  24s ec=1$
  ```

  Looking for unmerged patches, I do find an abandoned patch for
  the Wikibase extension that attempted to use to instrument database
  load in 2020. It was abandoned in favour of dedicated DB tooling
  to inspect database queries, which offer more detail as well.
  https://gerrit.wikimedia.org/r/q/message:wgStatsdSamplingRates
  https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/593335

== Future ==

In 10 years, we've not needed sampling anywhere even at our scale.
StatsD absorbs high traffic very well, with controlled flushing to
Graphite as backend. Prometheus exporters are similarly buffered
close to the producer with controlled scraping intervals. This
should make sampling unlikely to be needed.

We did have one experiment where sampling was attempted, which led
to the developments this commit removes. That instrumentation was
removed because the code was far too frequently called and hot that
it posed a latency problem, even before we factor in whether it
sends the metric. The overhead of the stats code itself added too
much overhead to Hook::run / HookContainer.

The new StatsFactory service class does feature per-metric sampling,
although this is similarly without use case at the moment:
https://codesearch.wmcloud.org/search/?q=%3EsetSampleRate&excludeFiles=test

New instrumentations should use StatsFactory, and if sampling were
needed in the future, it could be experimented by calling setSampleRate
on Metric objects (as part of the instrumentation, rather than in
site config, unless the feature in question makes it configurable).
That seems enough upfront investment for a theoretical future need.

Change-Id: I5f68e48d6d2c880a43f83915234d3c9f32a2d1ef
2024-08-07 20:50:30 +00:00
Ebrahim Byagowi
4c270a72ac Add namespace to WikitextContent
It adds MediaWiki\Content namespace to WikitextContent
and two classes related.

Change-Id: Ib74e4c5b3edac6aa0e35d3b2093ce1d0b794cb6d
2024-08-06 17:42:51 +03:30
Reedy
b52a8addf5 Drop writeapi userright
Bug: T294397
Depends-On: Ib34228a18917e404517d45e539bd786419d9c401
Change-Id: Ifad2edc782b36d21c8c67fecde7f011dce02c11b
2024-07-31 11:10:50 +00:00
Aaron Schulz
161ec6333a api: deprecate and deactivate $wgAPIRequestLog
Make update.php warn if this setting is still used in config.

Bug: T276975
Change-Id: I8909e5c92f002949c0d77cfd7aa3febaf707f57f
2024-07-29 14:42:00 +00:00
Arlo Breault
44580945ed Add OutputPipelineStages from extensions
Adds an experimental configuration to allow extensions to define
OutputPipelineStages to include in the DefaultOutputPipeline.

There are a lot of open questions about this api, like ordering of
execution, but adding it @experimental will help surface the
requirements.

Bug: T370541
Needed-By: I6dc92af0611c680b6e55605a7c9ff8a3fc1dfa26
Change-Id: I64baea40a1687c7a06fbcda9efe9f9a159b0ae8d
2024-07-25 11:44:17 -04:00
Ebrahim Byagowi
fab78547ad Add namespace to the root classes of ObjectCache
And deprecated aliases for the the no namespaced classes.

ReplicatedBagOStuff that already is deprecated isn't moved.

Bug: T353458
Change-Id: Ie01962517e5b53e59b9721e9996d4f1ea95abb51
2024-07-10 00:14:54 +03:30
jenkins-bot
68cebdfbb0 Merge "[ParsoidCachePrewarmJob] Use ParserOutputAccess" 2024-06-28 11:44:24 +00:00
MusikAnimal
7326c8e534 MainConfigSchema: add 'pagelang' as a permission for 'editpage'
This permission isn't currently available in other grant, making it
impossible to use action=setpagelanguage via BotPasswords or OAuth.

This commit adds 'pagelang' to the 'editpage' grant. Note that this
doesn't automatically give any consumer with the 'editpage' grant the
'pagelang' right -- they must actually have such rights on the user
account as well (for most wikis this is sysop, translationadmin).

Bug: T368578
Bug: T365558
Change-Id: I7048c49dc0428016cd798622552c446bbb10d99f
2024-06-27 00:21:48 +00:00
Kosta Harlan
499277a67a
[temp accounts] Set expiration to 90 days
Why:

- We are unlikely to see good faith editing patterns with temp accounts
  that would require a full year

What:

- Set the default expiry for temp accounts to 90 days

Bug: T359653
Change-Id: Iae9dd0f73aceecfc9935b2b6019b035b1057eeb3
2024-06-20 10:43:23 +01:00
jenkins-bot
4fd9bdeb83 Merge "Revert "Add option to show experimental login popup links"" 2024-06-19 19:53:38 +00:00
jenkins-bot
3078fe7eb9 Merge "Add protection indicators to mediawiki/core" 2024-06-19 18:54:26 +00:00
jenkins-bot
990fd9441c Merge "schema: Drop old pagelinks columns" 2024-06-18 20:28:09 +00:00
Amir Sarabadani
bb6c6e4174 schema: Drop old pagelinks columns
It has been dropped in production already.

Bug: T299947
Change-Id: I8ec1e7d9224c81d6494c39c78df9e4bdac38d377
2024-06-18 21:13:38 +02:00
Bartosz Dziewoński
43c6ae92f7 Revert "Add option to show experimental login popup links"
This reverts commit 9b8d4fc6e1.

Bug: T367891
Change-Id: I1b51a56755f7c6123c0b065abef15efd935169fa
2024-06-18 16:13:54 +00:00
C. Scott Ananian
105bb58ae2 [ParsoidCachePrewarmJob] Use ParserOutputAccess
One more step in gradually replacing uses of ParsoidOutputAccess.  This
one was pretty easy, as ParsoidOutputAccess was pretty much directly
calling ParserOutputAccess when provided with a ExistingPageRecord
and RevisionRecord.

Bug: T367074
Change-Id: I96161a64952e1809c0aec773d5a3dd4c71105657
2024-06-17 13:24:39 +00:00
jenkins-bot
9cb183acd7 Merge "[temp accounts] Introduce 'known' config flag" 2024-06-14 14:32:10 +00:00
jenkins-bot
286f49b678 Merge "Add Special:RestSandbox for exploring REST API" 2024-06-14 01:19:30 +00:00
Thalia
2c3b456ffc
[temp accounts] Introduce 'known' config flag
Why:

- We want to distinguish temp account creation being enabled from a
  configuration state where MediaWiki knows about temp accounts
  existing (and can identify and manage them accordingly) while temp
  account creation is disabled

What:

- Introduce a 'known' configuration flag to TempUserConfig
- If 'enabled' is set to true, then 'known' is automatically overridden
  to true
- If an administrator wishes to disable temp account creation after
  temporary accounts have been created, the administrator should set
  'enabled=false' and 'known=true'

Co-authored-by: Tchanders <thalia.e.chan@googlemail.com>
Co-authored-by: Kosta Harlan <kharlan@wikimedia.org>
Bug: T356524
Change-Id: I4ce534a847461230f7fa276a565bdc1d6c9857e1
2024-06-13 22:11:27 +02:00
daniel
c01b7c7b4b Add Special:RestSandbox for exploring REST API
Special:RestSandbox presents a Swagger-UI interface for exploring REST APIs. The available APIs can be configured using RestSandboxSpecs.

For now, the default is to support no APIs, so the feature is disabled in production. In the future, it would make sense to expose the wiki's own REST API per default. The corresponding entry in $wgRestSandboxSpecs in LocalSettings.php would look like this:

	'mw' => [
		'url' => $wgScriptPath . '/rest.php/',
		'name' => 'MediaWiki REST API',
	]

Note that the spec URL may still change.

To also explore the endpoints exposed through RESTbase, we might add:

	'wmf-restbase' => [
		'url' => $wgServer . '/api/rest_v1/',
		'name' => 'Wikimedia RESTbase API',
	]

Similarly, we could expose a spec for endpoints on api.wikimedia.org, which could then be explored using the new special page.

NOTE: This adds a dependency on the swagger-ui npm library. See T325558 for the security review.

Bug: T362006
Change-Id: I1dd5ed82680a28f9c15136b446a2de0398525061
2024-06-13 21:40:36 +02:00
Kevin Israel
71f27d46f1 password: Remove automatic fallback to hash_pbkdf2()
The criteria for doing so have now been met:

* PHP 8.1+ is now the documented minimum, and the 1.42 branch already
  enforces this in PHPVersionCheck. (T359868)
* OpenSSL support is also now required. (e4127e5864)

As stated in AbstractPbkdf2Password::canUseOpenSSL(), the version check
is no longer needed because PHP 8.1 requires OpenSSL >= 1.0.2. While the
the master branch may still work on PHP 7.4 for now, it is unlikely that
a site using it would still have a version of OpenSSL older than 1.0.1f.
(For example, WMF stopped using Ubuntu 14.04 "Trusty", which has exactly
that minimum version of OpenSSL, once Canonical started charging for
security updates in 2019.)

The reasons for the version check were:

* Old versions of OpenSSL appeared to perform at least as well as PHP
  for reasonably long passwords (up to 128 bytes for SHA-512 hashes);
  however, they had the same DoS issue that our own implementation for
  PHP 5.3 had (see T64685). hash_pbkdf2() never had that problem.

* If PHP were to incorporate the major optimization of hashing the HMAC
  key blocks only once, then the old OpenSSL versions would actually be
  slower. So far, this has not happened.

Change-Id: I47eb1aabf3d0ae4792624f9ba1c392880d52d0b7
2024-06-08 01:06:22 -04:00
Bartosz Dziewoński
9b8d4fc6e1 Add option to show experimental login popup links
Bug: T366486
Change-Id: Ie08089ada5461cd631687b197015725aca97711c
2024-06-03 18:29:46 +02:00
Sohom
bfb2d1d920 Add protection indicators to mediawiki/core
- Add a disabled by default feature flag 'EnableProtectionIndicators'
- When the config flag is enabled, show a lock indicator at the top
of the page.
- The lock icon should be overridable by the content of the page
- The indicator has a predictable ID which could be potentially used
 to style the icon using the onwiki Common.css file.
- The lock icon by default links to https://www.mediawiki.org/wiki/Help:Protection. However
this link can be customized per wiki per protection level using a
empty message (for example: `protection-sysop-helppage`)

Bug: T12347
Change-Id: I3e36d98edfe54a9c138b1fe8e5057e107ded281b
2024-05-28 16:09:27 -04:00
jenkins-bot
ed10e9dbfb Merge "Remove TemplateLinksSchemaMigrationStage config" 2024-05-28 11:45:51 +00:00
jenkins-bot
07ece77e44 Merge "Add MediaWiki\Watchlist namespace to the related classes" 2024-05-23 23:12:10 +00:00
jenkins-bot
a17833c271 Merge "Add MediaWiki\Content namespace to FallbackContent{,Handler}" 2024-05-23 15:19:50 +00:00
C. Scott Ananian
a565e388f9 Move ParsoidOutputAccess::supportsContentModel() into Parsoid SiteConfig
The `supportsContentModel` method is really querying Parsoid for the
set of content models it supports, so it makes sense to put it in the
Parsoid-specific SiteConfig service.

This is part of the work to deprecate and remove ParsoidOutputAccess.

Change-Id: I81eb2df8cef93ede95361a4e03185b3d58e5b84b
2024-05-22 10:57:37 -04:00
Ebrahim Byagowi
12660db261 Add MediaWiki\Watchlist namespace to the related classes
This adds MediaWiki\Watchlist namespace to the classes of watchlist
directory and adds deprecation notice since 1.43 to the just created
unnamespaced aliases of the classes.

Bug: T353458
Change-Id: I4234f8fe62bb3bde6f5271c7ba31a2420b0f4b90
2024-05-22 01:23:10 +03:30
Ebrahim Byagowi
c556eda906 Add MediaWiki\Content namespace to FallbackContent{,Handler}
This adds MediaWiki\Content namespace to FallbackContent
and FallbackContentHandler and declares the unnamespaced version
as deprecated since version 1.43.

Bug: T353458
Change-Id: I3ee80aea379788b71539cc1c7a4ec216b753e042
2024-05-21 17:05:28 -04:00
Ebrahim Byagowi
656c7fac3b Add namespace and deprecation alias to JsonContentHandler
This patch introduces a namespace declaration for the
MediaWiki\Content to JsonContentHandler and establishes a class
alias marked as deprecated since version 1.43.

Bug: T353458
Change-Id: Ia4ba6d3eddcb7b3f3d9f41a5ff80f724dbd01b22
2024-05-21 17:10:20 +03:30
jenkins-bot
cd40801020 Merge "Add namespace and deprecation alias to UDPRCFeedEngine" 2024-05-20 14:13:04 +00:00
Ebrahim Byagowi
b557846a09 Add namespace and deprecation alias to UDPRCFeedEngine
This patch introduces a namespace declaration for the
MediaWiki\RCFeed to UDPRCFeedEngine and establishes a class
alias marked as deprecated since version 1.43.

Bug: T353458
Change-Id: I27437c2417984f21e29237b011add20cad9f4389
2024-05-20 06:38:25 +03:30
Ebrahim Byagowi
16197f7d76 Add namespace and deprecation alias to TextContentHandler
This patch introduces a namespace declaration for the
MediaWiki\Content to TextContentHandler and establishes a class
alias marked as deprecated since version 1.43.

Bug: T353458
Change-Id: I2c72dacf28ee72fb70b15acdd81d0eb717ea949a
2024-05-20 05:34:31 +03:30