Commit graph

133 commits

Author SHA1 Message Date
Kunal Mehta
674e486cb5 Recognize CSS/JS redirects with non-ASCII targets
The regex in JavaScriptContent and CssContent was trying to pass
urlencoded stuff to Title::newFromText(), which would fail. Make sure we
urldecode it first.

Bug: T208264
Change-Id: I189c4c308da2875839ad8c1061332500f0e6d244
2018-10-31 23:09:50 -07:00
Brad Jorsch
dff469a408 Re-namespace RevisionStore and RevisionRecord classes
During development a lot of classes were placed in MediaWiki\Storage\.
The precedent set would mean that every class relating to something
stored in a database table, plus all related value classes and such,
would go into that namespace.

Let's put them into MediaWiki\Revision\ instead. Then future classes
related to the 'page' table can go into MediaWiki\Page\, future classes
related to the 'user' table can go into MediaWiki\User\, and so on.

Note I didn't move DerivedPageDataUpdater, PageUpdateException,
PageUpdater, or RevisionSlotsUpdate in this patch. If these are kept
long-term, they probably belong in MediaWiki\Page\ or MediaWiki\Edit\
instead.

Bug: T204158
Change-Id: I16bea8927566a3c73c07e4f4afb3537e05aa04a5
2018-10-09 10:22:48 -04:00
jenkins-bot
6871d732a1 Merge "Use Remex for TextContentTest subclasses" 2018-10-04 16:38:46 +00:00
jenkins-bot
67dc908246 Merge "Preserve whitespace in search index text content" 2018-10-01 18:22:48 +00:00
C. Scott Ananian
2cb60f4204 Use Remex for TextContentTest subclasses
We are deprecating untidy configurations of the parser.

Bug: T198214
Change-Id: I16529879a3b3aed960c5dc006e2af513d7e91fcd
2018-09-25 18:42:56 +00:00
Gergő Tisza
6e8d39c6e7
Add constant for the name of the 'main' slot for MCR
Bug: T202142
Change-Id: I97a74e5a029b014f3c2195188936d5c8233c1b7f
2018-09-24 16:52:12 -07:00
C. Scott Ananian
6f8431c4f5 Hard-deprecate the $wgUseTidy option
This has been soft-deprecated since MW 1.26; this hard-deprecation
sets the stage for future removal of this old cruft.

Bug: T198214
Depends-On: Idf246d05d116f63a73105b50a1929a7721fbe7b9
Change-Id: I2e7d990da1da378eb6e828d4b3c0f5a41791dd92
2018-09-20 11:08:40 -04:00
jenkins-bot
b7625d63a6 Merge "Introduce ContentHandler::getSecondaryDataUpdates." 2018-09-17 14:06:19 +00:00
Erik Bernhardson
0d779c1ac6 Preserve whitespace in search index text content
Certain html tags imply a word break, but our html stripping doesn't
understand that at all. Adjust the html stripping to inject whitespace
for all block level tags (per MDN) along with the <br> element.

Bug: T195389
Change-Id: I9fbfac765ea88628e4f9b2794fb54e1cd0060203
2018-09-14 11:10:35 -07:00
daniel
7960d5385f Introduce ContentHandler::getSecondaryDataUpdates.
This adds getSecondaryDataUpdates and getDeletionUpdates
to ContentHandler, and updates WikiPage and DerivedPageDataUpdates
to handle DataUpdates from all slots.

Bug: T194038
Bug: T194037
Change-Id: I75c96318f58a5cdda48484f7040ae41e6f42392a
2018-09-14 16:18:18 +02:00
daniel
9bd162dce2 [MCR] Set MCR migration stage to write-both/read-new.
This patch exists to see if CI passes with
$wgMultiContentRevisionSchemaMigrationStage set to
SCHEMA_COMPAT_WRITE_BOTH | SCHEMA_COMPAT_READ_NEW.

NOTE: verify that $wgMultiContentRevisionSchemaMigrationStage
is explicitly set toSCHEMA_COMPAT_WRITE_BOTH | SCHEMA_COMPAT_READ_OLD
in production config (T197816) before merging this.

Bug: T198561
Depends-On: Ib718868d2c768b6ea851355eef047bc0e6593495
Change-Id: I15989adae2b5916577d164c50d7da88774e49324
2018-09-11 12:27:42 +02:00
Gergő Tisza
d31580eeb0
[MCR] Render multi-slot diffs
Move logic for rendering a diff between two content objects out of
DifferenceEngine, into a new SlotDiffRenderer class. Make
DifferenceEngine use multiple SlotDiffRenderers, one per slot.

This separates the class tree for changing high-level diff properties
such as the header or the revision selection method (same as before:
subclass DifferenceEngine and override ContentHandler::getDiffEngineClass
or implement GetDifferenceEngine) and the one for changing the actual
diff rendering for a given content type (subclass SlotDiffRenderer and
override ContentHandler::getSlotDiffRenderer or implement
GetSlotDiffRenderer). To keep B/C, when SlotDiffRenderer is not overridden
for a given content type but DifferenceEngine is, that DifferenceEngine
will be used instead.
The weak point of the scheme is overriding the DifferenceEngine methods
passing control to the SlotDiffRenderers (the ones calling
getDifferenceEngines), without calling the parent. These are:
showDiffStyle, getDiffBody, getDiffBodyCacheKeyParams. Extensions doing
that will probably break in unpredictable ways (most likely, only
showing the main slot diff). Nothing in gerrit does it, at least.

A new GetSlotDiffRenderer hook is added to modify rendering for content
models not owned by the extension, much like how GetDifferenceEngine
works.

Also deprecates public access to mNewRev/mOldRev and creates public
getters instead. DifferenceEngine never supported external changes to
those properties, this just acknowledges it.

Bug: T194731
Change-Id: I2f8a9dbebd2290b7feafb20e2bb2a2693e18ba11
Depends-On: I04e885a33bfce5bccc807b9bcfe1eaa577a9fd47
Depends-On: I203d8895bf436b7fee53fe4718dede8a3b1768bc
2018-08-20 15:39:12 +02:00
jenkins-bot
3e5be2f38a Merge "Give pages with ~~~~ a different cache TTL" 2018-08-12 06:43:52 +00:00
Aryeh Gregor
f2e3c4d09f Give pages with ~~~~ a different cache TTL
This was supposed to already be the case, but it wasn't. The flag that
was set got cleared and never did anything.

Change-Id: Ide960f8cb9228f9a9d68c540369f122ada0a2a6f
2018-08-12 06:19:51 +00:00
Aryeh Gregor
90d4f56fe4 Mass conversion of $wgContLang to service
Brought to you by vim macros.

Bug: T200246
Change-Id: I79e919f4553e3bd3eb714073fed7a43051b4fb2a
2018-08-11 22:44:29 -06:00
jenkins-bot
07beacf802 Merge "Automatically reset namespace caches when needed" 2018-08-02 02:26:33 +00:00
Aryeh Gregor
63d7f2ad13 Automatically reset namespace caches when needed
This avoids error-prone code written separately in every test.  In
addition to no existing tests resetting the TitleFormatter (more
services probably need to be reset as well), they mostly reset only the
namespace cache on $wgContLang, which wouldn't help for any other
language.

The parser test runner still doesn't do this, but maybe it should.

Change-Id: I44b7a1aec48f14b0950907fa14bd0df80f674296
2018-08-01 16:30:08 +03:00
Aryeh Gregor
4bdae1c9d2 Convert remaining MagicWord:: calls to MagicWordFactory
Bug: T200247
Depends-On: Ie061fe90f9b9eca0cbf7e8199d9ca325c464867a
Change-Id: I49c507f3875e46a8e15fd2c28d61c17188aabffc
2018-08-01 10:47:43 +03:00
Aryeh Gregor
5189333c39 MagicWordFactory to replace MagicWord static members/methods
Static members of MagicWord have been removed.

Static methods are soft-deprecated and forward to the factory.  They
will be hard-deprecated when all callers are removed from core.

MagicWord::clearCache() has been removed.  Instead, call
resetServiceForTesting( 'MagicWordFactory' ) on your MediaWikiServices
object.

Change-Id: Ie061fe90f9b9eca0cbf7e8199d9ca325c464867a
Bug: T200247
2018-07-30 21:20:43 +03:00
Aryeh Gregor
355e21590a Use setContentLang() instead of setMwGlobals()
This changes behavior in some tests by making them set $wgLanguageCode
as well as $wgContLang, but that seems like a good thing.

Bug: T200246
Change-Id: I936888f46ff9fefe2707efba837e2ce3a7ca5e3f
2018-07-26 11:35:58 +00:00
Brad Jorsch
c6810d74d1 Deprecate ContentHandler::makeParserOptions()
Having a different ParserOptions for each content model isn't feasible
in an MCR world. And the only thing using this was Wikibase, which has
been fixed to do what it needs in a different way.

Bug: T194263
Change-Id: I01373b29ee25fa9346c6b0317155be4ccdc8c515
2018-07-13 14:32:59 -04:00
Kunal Mehta
c4e5a9dd97 Avoid deprecated LinkCache::singleton()
Change-Id: Ie0e5c4ef0fe6ec896378bb2433af0898655dd907
2018-06-10 23:55:11 -07:00
Subramanya Sastry
30495ea1f9 RFC T157418: Trim whitespace in table cells, list items, headings
* Matmarex had implemented this for wikitext headings in b3dd3881.
* This patch extends this to wikitext list items and wikitext table cells.
* Updated RELEASE NOTES.

tests/parser/parserTests.txt:
* All whitespace removed in output of list items, table cells, and
  headings. Removed corresponding whitespace in the input wikitext
  except for a few tests where the whitespace is significant "| +"
  or "| -", for example.
* Updated output of html/parsoid sections as well.
* Added new tests to spec white-space trimming behavior.

tests/phpunit/*:
* Fixed a few tests that used whitespace in list items and table cells.

Bug: T157418
Change-Id: I8ea34c7ab893c0c125c81d810feeb3c581e4bba1
2018-03-16 13:42:55 -05:00
James D. Forrester
326d655fc9 Drop 'comma' value for wgArticleCountMethod
We have three methods for page counting currently supported for wikitext
non-redirect pages; 'any' counts any page that exists, 'link' counts any
page that has any outbound links, and 'comma' which searches for any ','
in the text having loaded it. This last option is much slower than these
other two, and is only used on a very small number of installations. Now
by dropping support for this method we can simplify this code and so run
it more often. Note that non-wikitext pages already did not support this
count method.

Installations with this setting set to 'comma', or any other string will
now work as if it was configured with 'any'.

Bug: T188472
Change-Id: I965927edcd2485ec4b49b2d80fdf216dbf19520b
2018-03-01 16:37:23 -08:00
Umherirrender
45da581551 Use ::class to resolve class names in tests
This helps to find renamed or misspelled classes earlier.
Phan will check the class names

Change-Id: Ie541a7baae10ab6f5c13f95ac2ff6598b8f8950c
2018-01-26 22:49:13 +01:00
jenkins-bot
495e8ddf7e Merge "Remove very, very old disabled test cases and todos" 2018-01-25 17:01:54 +00:00
Umherirrender
255d76f2a1 build: Updating mediawiki/mediawiki-codesniffer to 15.0.0
Clean up use of @codingStandardsIgnore
- @codingStandardsIgnoreFile -> phpcs:ignoreFile
- @codingStandardsIgnoreLine -> phpcs:ignore
- @codingStandardsIgnoreStart -> phpcs:disable
- @codingStandardsIgnoreEnd -> phpcs:enable

For phpcs:disable always the necessary sniffs are provided.
Some start/end pairs are changed to line ignore

Change-Id: I92ef235849bcc349c69e53504e664a155dd162c8
2018-01-01 14:10:16 +01:00
Thiemo Mättig
29eebc6773 Remove very, very old disabled test cases and todos
Some of this is from 2011.

Change-Id: I3cf78e6b451af92063746af87c25c1d352ce94da
2017-12-28 18:44:31 +00:00
Kunal Mehta
4de7b0a542 Add @covers tags to ContentHandler tests
Change-Id: I4d0994e91f63a0921343f34ef81a82463b4ab969
2017-12-27 17:48:56 +00:00
jenkins-bot
fc5dced1ce Merge "Add new core tags" 2017-11-30 18:43:28 +00:00
petarpetkovic
62713be990 Add new core tags
Add tags to types of edits that get automatic edit summaries:
- Making a page a redirect
- Changing redirect target
- Changing an existing redirect into a non-redirect
- Blanking of the page
- Removing nearly all (more than 90%) content
- Rolling back an edit

Bug: T167656
Bug: T73236
Change-Id: Ie7f637fcec5ee659c1086e28e8ba21f470c45160
2017-11-30 10:29:58 -08:00
addshore
e6e59bb425 Remove method level @group Database tags
MessageTest did not have the group at class level
either so that is added in this patch.

Change-Id: Idb6213282d6854044f374c9df51ea32d0a5523e1
2017-11-23 19:20:44 +00:00
Brad Jorsch
2d9f29a846 Improve namespace handling in tests
MWNamespace has three internal caches, only one of which can be cleared
(and that somewhat oddly by passing a boolean to
MWNamespace::getCanonicalNamespaces()).

This change introduces a MWNamespace::clearCaches() method to clear all
three caches. It also adds some resetting in tests that seemed to be
missing it.

Change-Id: I1dcfcd8713888b3ff8fc75e95329ba72bd95d0c9
2017-09-29 05:21:47 +00:00
Umherirrender
5544cef16b Add missing type to @param documentation
Change-Id: I6b2c9c7af9a281fe457099cc3a336a60a25e74aa
2017-08-11 20:37:35 +02:00
Kunal Mehta
d1cf48a397 build: Update mediawiki/mediawiki-codesniffer to 0.10.1
And auto-fix all errors.

The `<exclude-pattern>` stanzas are now included in the default ruleset
and don't need to be repeated.

Change-Id: I928af549dc88ac2c6cb82058f64c7c7f3111598a
2017-07-22 18:24:09 -07:00
Umherirrender
be42e09aa8 build: Prepare for mediawiki/mediawiki-codesniffer to 0.9.0
The used phpcs has a bug, so the version 0.9.0 could not be enforced at the moment.
Will be fixed in next version, see T167168

Changed:
- Remove duplicate newline at end of file
- Add space between function and ( for closures
- and -> &&, or -> ||

Change-Id: I4172fb08861729bccd55aecbd07e029e2638d311
2017-06-26 17:14:31 +00:00
Brad Jorsch
1aac0a2992 Wrap parser output in <div class="mw-parser-output">
This will allow CSS to target just the parser output, without also
accidentally targeting the edit form, diff tables, and so on.

Bug: T37247
Change-Id: If4eb5bf71f94fa366ec4eddb6964e8f4df6b824a
Depends-On: I330c6aa4aaee045614b1801ed34bc9e03be69650
Depends-On: I52a518fa44e017841fe78474012cd69823e0a41d
2017-05-08 05:32:03 +00:00
Timo Tijhof
447ce7e39a phpunit: Avoid use of deprecated getMock for PHPUnit 5 compat
The default will remain PHPUnit 4.x due to PHP 5.5 support.

But, we should allow developers to run tests with newer PHPUnit
versions which are noticably faster (especially for code coverage
reports).

* <https://github.com/sebastianbergmann/phpunit/wiki/Release-Announcement-for-PHPUnit-5.4.0>
  PHPUnit 5 deprecates the getMock() shortcut for getMockBuilder()->getMock().
  It instead introduces the shortcut createMock() which has better defaults
  than getMockBuilder(). For example, it sets 'disableArgumentCloning' and
  other things by default.

  Going forward, code should either use getMockBuilder directly and configure
  it using the setter methods (instead of the confusing variadic arguments
  of getMock) or simply use the new minimalistic createMock method. This patch
  backports the createMock method to MediaWikiTestCase so that we can start
  using it.

Change-Id: I091c0289b21d2b1c876adba89529dc3e72b99af2
2017-04-06 00:44:32 +00:00
Bartosz Dziewoński
ecdef925bb Miscellaneous indentation tweaks
I was bored. What? Don't look at me that way.

I mostly targetted mixed tabs and spaces, but others were not spared.
Note that some of the whitespace changes are inside HTML output,
extended regexps or SQL snippets.

Change-Id: Ie206cc946459f6befcfc2d520e35ad3ea3c0f1e0
2017-02-27 19:23:54 +01:00
Stanislav Malyshev
fb44ca9094 Add GetContentModels hook to allow extensions to enumerate dynamic content models.
Bug: T155139
Change-Id: Icb41c470dfa4638676eb3ba0e74f437e85acc792
2017-02-02 12:00:42 -08:00
Stanislav Malyshev
8de54f8349 Add content model indexing
Bug: T156371
Change-Id: I1514f9db70870ec05b30153cc56558af0a28b8c9
2017-01-26 15:42:52 -08:00
rlot
99800a9e03 Removed deprecated ContentHandler hooks.
Plus removed unused "ContentHandler::runLegacyHooks"

Bug: T154498
Change-Id: Ie398fd4e06d3e286fe8e24112d0c8b4ac7d883dc
2017-01-18 21:19:17 +00:00
Stanislav Malyshev
2a395370fc Create fields & data for image/file data indexing
Bug: T145558
Change-Id: I23d4c8235d0e4150eefec31cea4b2cfdd32bf32a
2016-09-26 23:42:06 -07:00
Kunal Mehta
bb8fff662a JsonContentHandler: Make sure makeEmptyContent() is valid JSON
The default TextContentHandler::makeEmptyContent() implementation
is an empty string, which is not valid JSON.

Change-Id: I8bc6ec647c5e8cd01c5cd63031525619c4ef44d0
2016-09-08 12:52:56 -07:00
jenkins-bot
dc36560cdf Merge "Add DEFAULTSORT to search index field data" 2016-09-01 14:51:41 +00:00
dcausse
7c09f09432 Add DEFAULTSORT to search index field data
Added FLAG_SOURCE_DATA to support additional data that is not supposed to be
part of the default mapping.

Should merged with I1484c2e62788bedb57a42869a5fb25cd8f64482f, otherwize rebuilding
an index may add an extra field to CirrusSearch mapping.

Bug: T134978
Change-Id: Ia41f8eeb9dd4f764543bdd4d71b7a50de8101101
2016-08-29 16:51:57 +02:00
Kunal Mehta
85034abca5 content: Refactor normalization of line endings code
The code that normalizes line endings ("\r\n" and "\r" to "\n") and
trims trailing whitespace is buried in Parser::preSaveTransform(), and
was duplicated to TextContent in 96b6afb31d, as non-wikitext content
models should still be normalizing line endings.

This splits the duplicated code into
TextContent::normalizeLineEndings(), and utilize it in the Parser.
Additionally, expand the documentation of
TextContent::preSaveTransform() to document that subclasses should make
sure they normalize line endings during the PST stage.

And remove a useless rtrim() call from WikitextContent that did nothing.

Change-Id: I9094c671d4bbd23d75436f8f1d682d6dd6e6d2fc
2016-08-23 11:09:59 -07:00
jenkins-bot
4b70bc2b28 Merge "Extract ParserOutput search index data fields from WikiTextContentHandler" 2016-08-19 18:40:17 +00:00
aude
64ee3d3269 Extract ParserOutput search index data fields from WikiTextContentHandler
Bug: T142491
Change-Id: I69b010b893135e53fac7f16f4b927b8fbcba06d2
2016-08-19 09:26:17 -04:00
Kunal Mehta
4b5c1745ec Allow marking legacy ContentHandler hooks as deprecated
Instead of having a single development flag that controls whether all
the hooks emit warnings, use the standard hook deprecation method to
mark individual hooks as deprecated once we have determined that their
usage is minimal.

Change-Id: I9a00ae131cb7f609c1d44926d1f6547089f70c34
2016-08-18 17:43:49 +00:00