Commit graph

1144 commits

Author SHA1 Message Date
Aaron Schulz
892c3383dd objectcache: clarify usage of getMultiWithUnionSetCallback()
The examples and comments were misleading with regard to the handling
of missing entities.

Bug: T303092
Change-Id: Ib672fd18d1270c35253683eb54b48be056e6dc3c
2022-03-17 17:40:45 +00:00
daniel
e239b02a5e Add convenience methods for asserting status.
This ensures that assertions work in a uniform way,
and provides meaningful messages in cause of failure.

Change-Id: Ic01715b9a55444d3df6b5d4097e78cb8ac082b3e
2022-03-16 22:44:25 +01:00
jenkins-bot
5ec380a8f6 Merge "Integration test for extension registration" 2022-03-16 17:05:36 +00:00
jenkins-bot
b5dd025fcb Merge "ParamValidator: Allow to use User:0 in UserDef" 2022-03-15 23:25:24 +00:00
Umherirrender
67f6d2c8da ParamValidator: Allow to use User:0 in UserDef
UserNameUtils returns string|false. Check explicit for false to take the
string '0' as valid, which was a falsy check before.

Change-Id: I04c6b5303f9c2356ca85855686694b48f6448d56
2022-03-15 22:25:35 +01:00
jenkins-bot
f426a261ff Merge "rdbms: Remove deprecated IDatabase functions" 2022-03-15 20:28:56 +00:00
Amir Sarabadani
46374a8e3e rdbms: Remove deprecated IDatabase functions
Soft-deprecated in 1.37, hard-deprecated in 1.38

Bug: T286694
Change-Id: Icd23271149bba3e4632d595c057a733d13707503
2022-03-15 19:33:32 +01:00
Aaron Schulz
ee3c65d541 rdbms: provide $owner argument in LoadBalancer::flushPrimarySessions()
This avoids DBUnexpectedError in LBFactory::flushPrimarySessions()

Follow-up: 4cac31de4e
Change-Id: I6b6e274135d842f379d71fb9efb42758984afc78
2022-03-15 10:42:21 -07:00
jenkins-bot
6fd4c95f5e Merge "Use updated ObjectFactory namespace" 2022-03-12 00:19:53 +00:00
daniel
1c7ffcd46a Integration test for extension registration
This adds full integration tests for extension registration, ensuring
that ExtensionProcessor and ExtensionRegistry work together as expected.

The existing tests for ExtensionProcessor and ExtensionRegistry are
converted to "pure" unit tests.

This change is intended to protect against bugs while refactoring
ExtensionProcessor and ExtensionRegistry to better integrate with the
new SettingsBuilder class.

Bug: T297166
Change-Id: I2d9cc373b8a4aa014aef93b0a8e8d83758851783
2022-03-11 21:43:41 +00:00
jenkins-bot
f55817d497 Merge "Allow ContentHandler to "override" non-existent actions" 2022-03-10 03:08:09 +00:00
Alexander Vorwerk
9b09bf3112 Use updated ObjectFactory namespace
Depends-On: I99c5e5664d2401c36a9890f148eba7c25e6e8324
Depends-On: I48ab818b2965da14af15ef370aa83ad9455badd9
Depends-On: I018371e4b77911e56152ca7b2df734afc73f58a5
Change-Id: I04ebdb52102f6191d49a9cc70b1f98308299e72f
2022-03-09 23:04:51 +00:00
jenkins-bot
4acd28a17c Merge "rdbms: Hard-deprecate LoadBalancer::getLazyConnectionRef" 2022-03-09 15:14:53 +00:00
Amir Sarabadani
49a9d44b18 rdbms: Hard-deprecate LoadBalancer::getLazyConnectionRef
All deployed usages have been migrated.

Bug: T255493
Change-Id: I82683596d644bb61a109e79dde6b0cb22c72b399
2022-03-09 10:45:00 +01:00
Timo Tijhof
f8ecea1e5c rcfeed: Deprecate $wgRCEngines and RCFeedEngine
Follows-up 39a6e3dc4d (I8be497c623c5d92).

* Improve documentation all around and advertise 'class'
  everywhere instead of 'uri'.

* Add test coverage for RCFeed::factory().

* Deprecate the $wgRCEngines "uri to class" mapping in favour
  of specifying "class" directly in $wgRCFeeds.

* Deprecate RCFeedEngine in favour of FormattedRCFeed.
  Convert to class_alias so that UDPRCFeedEngine no longer has
  to extend the deprecated class name explicitly (for instanceof compat).

* Hard-deprecate RecentChange::getEngine.

Bug: T250628
Depends-On: Ie939e1d06b9ee2d841ec7256c8d24cc4e7e386dd
Change-Id: Ib6758d724c7200404c89c7ab157aa55f1cad9763
2022-03-08 19:50:19 +00:00
jenkins-bot
4420102e99 Merge "Fix edge cases in schema validation." 2022-03-08 17:48:26 +00:00
daniel
404f4a8512 Fix edge cases in schema validation.
Since PHP arrays make no clear distinction between lists (JSON arrays)
and maps (JSON objects), some edge case handling is needed to make
validation work reliably when we declare types for all arrays:

1) Allow array keys to be ignored, so an associative PHP array validates
as a JSON array. This is needed for the SessionProviders setting.

2) Allow associative arrays with numeric keys to validate as JSON
objects. This is done by ignoring the type validation when numeric keys
are detected. A warning is returned in the status object.

3) Work around validation failing on float values that are expected to
be integers. All numbers come from the yaml parser as floats, and the
"integer" type in JSON schema should accept floats with if the
fractional part is 0. But that doesn't seem to work, we need to cast the
values to integers explicitly.

Also, this fixes some mistakes in the schema: LockManagers is a list,
so it should use the JSON type "array". NamespacesToBeSearchedDefault
is a map (JSON object), even though it uses numeric keys. The Actions
registry is also a map.

Change-Id: I9d0453d740c377b7cce574df743536c39a0ec619
2022-03-08 13:59:19 +01:00
Tim Starling
1ae3b0ca86 Allow ContentHandler to "override" non-existent actions
Flow defines a lot of custom actions which are only implemented for its
own content type. This use case is not what we had in mind when
getActionOverrides() was introduced, but allowing it in ActionFactory
removes the need for Flow to write dummy ignored config to $wgActions.

Bug: T303237
Change-Id: I67c1ae95344b0004eb93023fae7e93de4e47888b
2022-03-08 14:17:22 +11:00
Bartosz Dziewoński
ba96351f46 Only apply transformations for &uselang=qqx when actually using it
Depends-On: I4f3b01eccf571475f1248db838a8aa6cba18b93e
Depends-On: I0a0681b332abb623fc6871ab78da9e64a79fad84
Bug: T300069
Bug: T302469
Change-Id: I45f58034117e9041fe5403dbfcf83ba25f20518d
2022-03-08 01:50:44 +00:00
jenkins-bot
fd2c1795a9 Merge "Add Sanitizer::removeSomeTags() which uses Remex to tokenize" 2022-03-06 08:35:40 +00:00
C. Scott Ananian
9f14fbd002 Add Sanitizer::removeSomeTags() which uses Remex to tokenize
The existing Sanitizer::removeHTMLtags() method, in addition to having
dodgy capitalization, uses regular expressions to parse the HTML.
That produces corner cases like T298401 and T67747 and is not guaranteed
to yield balanced or well-formed HTML.

Instead, introduce and use a new Sanitizer::removeSomeTags() method
which is guaranteed to always return balanced and well-formed HTML.

Note that Sanitizer::removeHTMLtags()/::removeSomeTags() take a callback
argument which (as far as I can tell) is never used outside core. Mark
that argument as @internal, and clean up the version used by
::removeSomeTags().

Use the new ::removeSomeTags() method in the two places where
DISPLAYTITLE is handled (following up on T67747).  The use by the
legacy parser is more difficult to replace (and would have a
performace cost), so leave the old ::removeHTMLtags() method in place
for that call site for now: when the legacy parser is replaced by
Parsoid the need for the old ::removeHTMLtags() will go away.  In a
follow-up patch we'll rename ::removeHTMLtags() and mark it @internal
so that we can deprecate ::removeHTMLtags() for external use.

Some benchmarking code added.  On my machine, with PHP 7.4, the new
method tidies short 30-character title strings at a rate of about
6764/s while the tidy-based method being replaced here managed 6384/s.
Sanitizer::removeHTMLtags blazes through short strings 20x faster
(120,915/s); some of this difference is due to the set up cost of
creating the tag whitelist and the Remex pipeline, so further
optimizations could doubtless be done if Sanitizer::removeSomeTags()
is more widely used.

Bug: T299722
Bug: T67747
Change-Id: Ic864c01471c292f11799c4fbdac4d7d30b8bc50f
2022-03-04 14:06:02 -05:00
jenkins-bot
73a422bcf9 Merge "rdbms: Migrate TransactionProfiler calls to TransactionManager" 2022-03-03 20:20:56 +00:00
Timo Tijhof
cd7ef12c4c skins: Fix non-replacing skippable state bug in SkinFactory
As found via the test added with I3609cc71de48d3f5, which fails
without this change, and is enabled now in this patch.

Change-Id: Ie536644084b8a8b5386eb013cfd54621cfabc52d
2022-03-01 20:00:42 +00:00
jenkins-bot
2d3d4a8d69 Merge "skins: Increase SkinFactory test coverage, improve various class docs" 2022-03-01 19:50:51 +00:00
Timo Tijhof
0cc93dfc24 skins: Increase SkinFactory test coverage, improve various class docs
* Skin constructor docs were unreadable on doc.wm.o and in other
  places and IDEs that display docs due to lack of formatting (line
  are just whitespace in Markdown, same as HTML).

* Make some of the method briefs (first line separated by new line)
  more useful, especially where this brief was missing and thus
  rendered unhelpful text like "Returns array of:" with no more.

* Misc grammar and typo fixes, consistent phrasing, and other minor
  points from code conventions. Changed a few "fetch" to "get" in
  trivial methods.

* Found a bug in SkinFactory::register() regarding skipskin handling,
  will fix in follow-up.

* Found a bug in Skin::buildSidebar() regarding insufficient cache
  fragmentation. Will fix in a follow-up.

Change-Id: I3609cc71de48d3f5c8404ea834d42c0cec5cba59
2022-02-28 14:15:09 +00:00
Matthias Mullie
b0bd09c939 Remove change tags for media additions/removals
This reverts commit dab7b16741.

Reason for revert: This was inaccurate/incomplete &
looks like we will not need this after all

Bug: T286362
Change-Id: I6d63d69f180d193f99986ce6878617d63cfb2b6d
2022-02-28 11:08:09 +01:00
Amir Sarabadani
0543fc9906 rdbms: Migrate TransactionProfiler calls to TransactionManager
Bug: T299698
Change-Id: I4bf6ab7768909233a7bf75fd6eb842449852c333
2022-02-26 00:46:50 +01:00
Timo Tijhof
128debb64b tests: Change use of AtEase to at operator
Follows-up I361fde0de7f4406bce6ed075ed397effa5be3359.

Per T253461, not mass-changing source code, but the use of the native
error silencing operator (@) is especially useful in tests because:

1. It requires any/all statements to be explicitly marked. The
   suppressWarnings/restoreWarnings sections encourage developers to
   be "lazy" and thus encapsulate more than needed if there are multiple
   ones near each other, which would ignore potentially important
   warnings in a test case, which is generally exactly the time when
   it is really useful to get warnings etc.

2. It avoids leaking state, for example in LBFactoryTest the
   assertFalse call would throw a PHPUnit assertion error (not meant
   to be caught by the local catch), and thus won't reach
   AtEase::restoreWarnings. This then causes later code to end up
   in a mismatching state and creates a confusing error_reporting
   state.

See .phpcs.xml, where the at operator is allowed for all test code.

Change-Id: I68d1725d685e0a7586468bc9de6dc29ceea31b8a
2022-02-24 21:29:51 +00:00
Reedy
12aae45101 tests: Replace some usages of Wikimedia\(suppress|restore)Warnings()
Change-Id: I361fde0de7f4406bce6ed075ed397effa5be3359
2022-02-24 12:55:25 +00:00
daniel
f9b589f556 config-schema: Define types for all arrays.
This patch ensures that we know which arrays are lists (JsonSchema type
"array") and which are maps (JsonSchema type "object"). We can then
default to array_merge for lists and to array_plus for maps. This seems
clearer than requiring an explicit merge strategy to be declared for all
arrays.

This patch specified a mergeTrategy for some config variables that need
behavior different from the default.

This patch also changes the merging behavior to allow non-array values
to replace arrays and vice versa. It also changes the behavior of
defaults to allow falsy values to override non-falsy defaults.

Bug: T300129
Change-Id: Ia7b0c0250af6a957eac1efb554fb47511f5e093f
2022-02-23 14:09:41 +01:00
jenkins-bot
176194cef8 Merge "Remove unnecessary TestingAccessWrapper from SkinComponentTableOfContentsTest" 2022-02-23 07:59:06 +00:00
jenkins-bot
670652e1b5 Merge "resourceloader: Add regression test for long strings and regexes" 2022-02-23 07:10:45 +00:00
Nicholas Ray
f0f860a5ea Remove unnecessary TestingAccessWrapper from SkinComponentTableOfContentsTest
This was an artifact left over from when we were testing private
methods. Since we only test the public method now it can be removed.

Change-Id: I1666c5a041e11c9ecc59a8a9cc3688e010dfae6b
2022-02-22 20:06:13 -07:00
Timo Tijhof
71e80770a4 resourceloader: Add regression test for long strings and regexes
Bug: T299537
Change-Id: I9b6f4f9ad11b201f095ca1e58eb0f86fb74ee143
2022-02-23 11:04:37 +11:00
jenkins-bot
33783a1ed3 Merge "Allow config merge strategies to be bypassed." 2022-02-22 23:33:01 +00:00
daniel
cf499e8491 Allow config merge strategies to be bypassed.
Sometimes, we need to force an exact value and bypass the default
behavior of merging config variables.

This also renames setConfigValue to putConfigValue, to avoid confusion
about the behavior of that method.

Change-Id: I82c606632b94f974e655a44a0b63394de7a0804b
2022-02-22 22:59:25 +00:00
jenkins-bot
e9e4e41b8f Merge "Add validation for abstract schema" 2022-02-22 18:06:42 +00:00
mainframe98
de0c4819d1 Add validation for abstract schema
This adds an option to the schema generating maintenance scripts to
validate abstract schemas and schema changes and a structure test to
validate exisiting schemas and schema changes. Schemas are also
validated when generating.

The validation for the schema doesn't impose limits on table, index or
column names as I couldn't find any reliable conventions for them.

The structure tests only cover MediaWiki itself as there is no
convention on where extensions store their abstract schema.
Ideally, auto detection would be possible for sql/, but for now
extensions have to define their own (thankfully trivial) tests.

A couple of invalid definitions were fixed thanks to these tests.

I aimed to be thorough, but not all parts of the abstract schema
are completely clear, and Doctrine's documentation is not complete.
As a result, not everything has a description field.

Bug: T298320
Change-Id: I681d265317d4d1584869142ebb23d4098c06885f
2022-02-22 17:41:08 +00:00
jenkins-bot
9329dd71be Merge "Add number-section-count to data returned by SkinComponentTableOfContents" 2022-02-21 22:51:15 +00:00
Nicholas Ray
89287735b0 Add number-section-count to data returned by SkinComponentTableOfContents
The data returned by SkinComponentTableOfContents::getTemplateData is a
nested array which makes it somewhat arduous for the client to know how
many sections are in the array. Add `number-section-count` to make it
easier on the client.

Additionally:

* Add validation in SkinMustacheTest for keys that contain numbers.
* Revise unit tests to only test public method (`getTemplateData`) and
  not private methods.
* Remove `testGetTOCData` test as these test cases will now be covered
  by the `testGetTemplateData` unit test.

Bug: T300973
Change-Id: Ifaee451e1903f2accd0ada2f2ed6dfa3f83037b6
2022-02-21 14:41:01 -07:00
Ammarpad
c06dd6f667 UserNameUtils: Always strip subject namespace in canonical username.
Username with namespace prefix is not valid username. It's neither
creatable nor usable.

If the rigor validation passed to getCanonical() is not RIGOR_NONE,
this namespace prefix will always be removed by TitleParser.

We will now remove it even if the validation rigor is RIGOR_NONE
since the prefix can never appear in legal username, and its presence
will cause issues later even if the strict validation is not requested.

The validation is turned off usually when it's not known whether the
username is IP address or not, as is the case for T283915.

Bug: T283915
Change-Id: I34aa1d257f8bd90c80f40a76190a756d1e93e254
2022-02-21 22:37:44 +01:00
jenkins-bot
24aa34d06c Merge "phpcs: Disable Generic.Files.LineLength for test files" 2022-02-21 15:51:29 +00:00
jenkins-bot
26190f6e41 Merge "Remove deprecated LoadBalancer::openConnection() method" 2022-02-21 06:03:00 +00:00
Timo Tijhof
8d406bbcd6 phpcs: Disable Generic.Files.LineLength for test files
There is a common and reasonable need for longer lines in tests.
The nudge for shorter lines doesn't seem valuable here. The natural
breaks will likely still fall in 80-100 given the enforced practice
for non-test code, e.g. whether through habit, or 80-100 column markers
in text editors, or the finite width of diff and code review
interfaces.

Change-Id: I879479e13551789a67624ce66f0946d2f185e6ee
2022-02-18 18:32:05 +00:00
bwang
24c9728334 Add template data to support TOC collapsible section arrows in Vector
Bug: T299361
Change-Id: I8ab5c0543b898d1df9399a1cb39672c45daf2acd
2022-02-18 15:58:33 +00:00
Clare Ming
e70633b3b2 Introduce SkinComponentTableOfContents
- Moves Skin::getSectionsData and related methods into new TOC component.
- Update SkinMustacheTest for toc getTemplateData method.
- Add test for new TOC component.

Bug: T301523
Change-Id: I29dda96f1e91da6892840d38a80c6102d425d0f7
2022-02-17 17:43:19 -07:00
jenkins-bot
4021d419fd Merge "Simplify DnsSrvDiscoverer interface" 2022-02-17 15:16:44 +00:00
Dan Duvall
aa5fdf9e6f Simplify DnsSrvDiscoverer interface
Refactor DnsSrvDiscoverer constructor to accept service, protocol, and
domain as discrete arguments to better match the specifications outlined
in RFC 2782 and make clearer the concerns and contract between the
discoverer and the caller. The latter should not need to know about
discovery details but only the service to be discovered and the scope of
that discovery (the domain).

Domain is now optional where an omitted value results in discovery
relative to the host's configured search domain, supporting
installations where full DNS based discovery is desired over explicit
configuration. This makes the simplest use case:

    $servers = ( new DnsSrvDiscoverer( 'service' ) )->getServers();

Overall use of the interface is simplified by changing `getServers()` to
unconditionally sort results by priority according to RFC 2872 and
return a simplified result of host/port tuples (the only data relevant
in primary discovery use cases). This makes the handling of discovery
fallback simpler since the caller can construct a default without having
to populate all response record details.

Example

    $dsd = new DnsSrvDiscoverer( $service, $proto, $host );
    $servers = $dsd->getServers() ?: [ [ $host, $defaultPort ] ];

A `getRecords()` method is provided as public to allow a caller to
obtain details on the response (and do its own sorting) should that be
needed.

Added an optional `$resolver` argument to the constructor (which
defaults to `dns_get_record`) to make the class implementation more
testable, and wrote functional tests.

EtcdConfig has been refactored to reflect the new usage of
DnsSrvDiscoverer and now sets the discoverer as a property in the
constructor to allow for mocking in tests.

Implemented tests for etcd DNS SRV discovery within EtcdConfig.

Bug: T296771
Change-Id: Idbd60049853439f96ff6045e01aa03014b4e587f
2022-02-16 15:00:45 +00:00
jenkins-bot
5def3665f0 Merge "rdbms: Move more code from Database to TransactionManager" 2022-02-16 12:57:57 +00:00
Amir Sarabadani
5c6c7a8568 rdbms: Move more code from Database to TransactionManager
Bug: T299698
Change-Id: I063d76243271b1291a38cc5d722af91cfcd63ed7
2022-02-16 04:45:40 +01:00