Commit graph

90 commits

Author SHA1 Message Date
Kunal Mehta
bb8fff662a JsonContentHandler: Make sure makeEmptyContent() is valid JSON
The default TextContentHandler::makeEmptyContent() implementation
is an empty string, which is not valid JSON.

Change-Id: I8bc6ec647c5e8cd01c5cd63031525619c4ef44d0
2016-09-08 12:52:56 -07:00
jenkins-bot
dc36560cdf Merge "Add DEFAULTSORT to search index field data" 2016-09-01 14:51:41 +00:00
dcausse
7c09f09432 Add DEFAULTSORT to search index field data
Added FLAG_SOURCE_DATA to support additional data that is not supposed to be
part of the default mapping.

Should merged with I1484c2e62788bedb57a42869a5fb25cd8f64482f, otherwize rebuilding
an index may add an extra field to CirrusSearch mapping.

Bug: T134978
Change-Id: Ia41f8eeb9dd4f764543bdd4d71b7a50de8101101
2016-08-29 16:51:57 +02:00
Kunal Mehta
85034abca5 content: Refactor normalization of line endings code
The code that normalizes line endings ("\r\n" and "\r" to "\n") and
trims trailing whitespace is buried in Parser::preSaveTransform(), and
was duplicated to TextContent in 96b6afb31d, as non-wikitext content
models should still be normalizing line endings.

This splits the duplicated code into
TextContent::normalizeLineEndings(), and utilize it in the Parser.
Additionally, expand the documentation of
TextContent::preSaveTransform() to document that subclasses should make
sure they normalize line endings during the PST stage.

And remove a useless rtrim() call from WikitextContent that did nothing.

Change-Id: I9094c671d4bbd23d75436f8f1d682d6dd6e6d2fc
2016-08-23 11:09:59 -07:00
jenkins-bot
4b70bc2b28 Merge "Extract ParserOutput search index data fields from WikiTextContentHandler" 2016-08-19 18:40:17 +00:00
aude
64ee3d3269 Extract ParserOutput search index data fields from WikiTextContentHandler
Bug: T142491
Change-Id: I69b010b893135e53fac7f16f4b927b8fbcba06d2
2016-08-19 09:26:17 -04:00
Kunal Mehta
4b5c1745ec Allow marking legacy ContentHandler hooks as deprecated
Instead of having a single development flag that controls whether all
the hooks emit warnings, use the standard hook deprecation method to
mark individual hooks as deprecated once we have determined that their
usage is minimal.

Change-Id: I9a00ae131cb7f609c1d44926d1f6547089f70c34
2016-08-18 17:43:49 +00:00
Brad Jorsch
96b6afb31d TextContent: Normalize newlines in preSaveTransform()
This does the same normalization of newlines that
Parser::preSaveTransform() does. This should be appropriate for any text
content type, especially considering that EditPage uses
WebRequest::getText() which does a less-strict version of this same
transformation.

This also cleans up the code for doing that newline replacement
to be a bit less verbose.

Bug: T142805
Change-Id: I462afcda502f031a8b0360d982ce2398a0383a96
2016-08-16 10:21:32 -04:00
Stanislav Malyshev
723bb18646 Add search-ignored-headings string, copied from cirrus-search-ignored-headings.
Change-Id: I4178f872996800379843301f7119840f4a4551df
2016-08-04 11:12:26 -07:00
Stanislav Malyshev
add1ebe2ab Make content handlers assemble content for search
Bug: T89733
Change-Id: Ie45de496ecc826211d98eea3a410c7639b4be0a4
2016-07-26 13:08:45 -07:00
umherirrender
eac06f65ce Use more short array syntax in comments (/tests/)
Change-Id: I86c73cb9447ac562a73348b4030e24ebf49a90dc
2016-07-10 17:23:29 +02:00
Stanislav Malyshev
86b55ad03c Create API to allow content handlers to handle structured data definitions
Change-Id: Ia1738803c42f6114575587c1c838fec62b6f54aa
Bug: T89733
2016-07-06 13:41:20 -07:00
jenkins-bot
0583d261bb Merge "Remove support for $wgWellFormedXml=false" 2016-05-13 21:52:23 +00:00
Kunal Mehta
449084ec4e Add LinkCache to MediaWikiServices
LinkCache::singleton() is now deprecated, and the destroySingleton() and
setSingleton() methods were removed. They were not used in extensions,
and the usage in core was updated to use MediaWikiServices.

Change-Id: I08bb4f7913b03f71331ff683d0197c948aad6790
2016-05-12 15:44:33 -07:00
Brian Wolff
ee4d5c6eed Remove support for $wgWellFormedXml=false
tl;dr: Having unnessary complexity in security critical code is bad.

* Extra options add extra complexity and maintenance burden
** Thus we should only have one html output mode. well formed = false
     was already vetoed in T52040, so lets go with WellFormed=true.
* Options which are used by very few people tend to get tested less
* Escaping is an area of code where we should be very conservative
* Having escaping rules depend on making assumptions about which
    characters various browsers consider "whitespace" is scary
* $wgWellFormedXml=false has had a negative security impact in the
    past (Usually not directly its fault, but has made other bugs
    more exploitable)
* Saving a couple bytes (even less bytes after gzip taken into
    account) is really not worth it in this context (imho).

Change-Id: I5c922e0980d3f9eb39adb5bb5833e158afda42ed
2016-05-12 17:40:01 -04:00
Florian
b91cbed2ed Remove Title::newFromRedirectRecurse()
Bug: T122754
Change-Id: I8b957eff7f355493ef6850f7a83dcb619ab6be78
2016-04-03 13:31:57 +01:00
Florian
876ae95e63 Remove Title::newFromRedirectArray
Bug: T122754
Change-Id: I2cdeb98c81d4b98ea986bd2f7b23cd93c62c5930
2016-04-02 13:28:37 +00:00
Reedy
83fb19cb13 Swap the rest of array() -> []
Change-Id: I76a7259ed952a0673a1941f08b39b545211fba07
2016-03-30 22:04:58 +00:00
aude
437f60f358 Add ContentHandler::supportsCategories method
and check for this in WikiPage::doEditUpdates before
inserting a new CategoryMembershipChangeJob.

Some content models like the Wikibase ones do not
have categories and it's wasteful to add these jobs
for all Wikibase edits.

Bug: T126977
Change-Id: I2c54a4ba1546445dc41101e15cb83a2c6cc2b1c9
2016-03-02 15:20:56 +01:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Bene
9e3e3306b0 Allow callbacks to be passed to $wgContentHandlers
Change-Id: Icf980313a6e7fcc83f5183c450b0a824353596b8
2016-02-06 09:48:11 +01:00
addshore
96e1167d92 Give TestCase::checkHasDiff3 a better name
This method actually marks the test it is called
from as skipped if Diff3 is not confugured.

The new name "markTestSkippedIfNoDiff3" better
reflects that.

Change-Id: I1dffeba0aceb312b3a82216f0b55227e24bc2e34
2016-01-28 12:46:11 +01:00
addshore
044f6418b1 Switch name to username in @author tags
From now on I will simply use addshore
everywhere to keep things uniform...

Change-Id: Iaf441b2d7a67a12c20529f0e9c7b47819f4abfae
2016-01-27 10:59:31 +01:00
umherirrender
e386a46eb3 Fix some language depending phpunit tests by using MediaWikiLangTestCase
Some tests fails on wikis with $wgLanguageCode not = 'en', fix this by
setting the correct language context.

Change-Id: Ia72017df59d8beb5c508832584b2431ffab0f77f
2015-12-07 17:26:16 +01:00
saper
8fa1555c78 JavaScriptContentTest: $wgScript != $wgScriptPath
Set $wgScript, $wgScriptPath and $wgResourceBasePath
to avoid failures when running on a wiki with
a non-standard $wgScriptPath.

Reported-on:

https://lists.wikimedia.org/pipermail/wikitech-l/2015-October/083675.html

Change-Id: I8acbcca5449060ff5604bf275f690b53343e706e
2015-10-26 23:05:15 +01:00
Amir E. Aharoni
c50cbe48ab Fix remaining Generic.Files.LineLength failures
Except the ones for which there is no clear way to fix
because of T114213.

Bug: T102614
Change-Id: I94a1fc7c890c82a36f0b51e42d311585c275775c
2015-10-03 17:23:07 +00:00
Amir E. Aharoni
d461d5c8af Make lines short to pass phpcs in WikitextContentTest.php
Bug: T102614
Change-Id: Id5f12d4cca1d952580d2bb68fdbcb3cfb2467085
2015-10-03 12:30:38 +00:00
Amir E. Aharoni
e77af93e83 Add @codingStandardsIgnoreStart to 5 files under tests/phpunit/includes
To pass phpcs.

Bug: T102614
Change-Id: I4c7d60d26112c7ba67b66f1923ce945e96175d15
2015-10-03 12:30:29 +00:00
Amir E. Aharoni
c369defa97 Add @codingStandardsIgnoreStart to pass phpcs in 2 files in tests/phpunit/includes
Change-Id: I7f551dafcf437cc23aa9aed24f9e4d385816a0a0
2015-10-01 18:45:28 +00:00
Vivek Ghaisas
c54766586a Fix issues identified by SpaceBeforeSingleLineComment sniff
Change-Id: I048ccb1fa260e4b7152ca5f09b053defdd72d8f9
2015-09-26 23:06:52 +00:00
jenkins-bot
c3a5d76d19 Merge "Re-enable MediaWiki.WhiteSpace.SpaceyParenthesis.SingleSpaceBeforeCloseParenthesis" 2015-09-26 16:53:03 +00:00
Reedy
8e8368ca65 Re-enable MediaWiki.WhiteSpace.SpaceyParenthesis.SingleSpaceBeforeCloseParenthesis
Change-Id: I8482f5dd9d79e4946e862b0ac03bd027d62e8646
2015-09-26 17:38:35 +01:00
Reedy
3a6fc5e96c Re-enable Squiz.WhiteSpace.ScopeClosingBrace.Indent
Change-Id: I743c4ce019321d631c0a11ee2112fe9aa8ba4c5d
2015-09-26 16:09:54 +00:00
Tim Starling
2c6c954e23 Abstract and refactor Tidy support
* Split tidy implementations into a class hierarchy
* Bring all tidy configuration into a single associative array and
  deprecate the old configuration.
* Remove $wgAlwaysUseTidy

This is preparatory to replacement of Tidy (T89331). I used the name
"Raggett" for things relating to Dave Raggett's Tidy, since if we use
"tidy" to mean the new abstract system as well as Raggett's tidy, it
gets confusing.

Change-Id: I77af1a16cbbb47fc226d05fb9aad56c58e8910b5
2015-09-10 20:18:52 -07:00
Kunal Mehta
71b3e7a4b7 Implement redirects in CssContent
Just like ad9f14d662 which was for JavaScript. The redirect will be
of the form "/* #REDIRECT */@import url(...);".

Bug: T73201
Bug: T35973
Change-Id: I10bae44af4b4923f8797172702974cd45dc25ab4
2015-07-28 17:53:47 +00:00
Kunal Mehta
ad9f14d662 Support redirects in JavaScriptContent
When a JavaScript page is moved, a "redirect" in the form of
mw.loader.load(...) will be left behind, so any other
JavaScript loading the page that way will still work, albeit
with an extra HTTP request.

This also implements Content::getRedirectTarget(), so redirects
are marked properly in the database, and users viewing them
are redirected properly. A magic "/* #REDIRECT */" comment
must be in front of the mw.loader.load call. This is done so
that pages which currently are just one mw.loader.load call
aren't turned into redirects.

Bug: 71200
Bug: 33973
Change-Id: I10fdff087a901da56fad64531f0e382f90ebcf37
2015-07-20 15:36:49 +00:00
Vivek Ghaisas
9f5b6f5aeb Fix whitespace issues around parentheses
Fix issues found by MediaWiki.WhiteSpace.SpaceyParenthesis sniff.

Bug: T102617
Change-Id: Iec7f71e64081659fba373ec20d9d2006306a98f4
2015-06-16 22:14:02 +03:00
jenkins-bot
4e90d55fdc Merge "content: Recognise .json as JsonContent in User and MediaWiki namespace" 2015-06-03 16:25:09 +00:00
aude
2513086ec1 Add supportsDirectEditing methods to ContentHandler
This adds supportsDirectApiEditing and
supportsDirectEditing methods to ContentHandler. Both
return false by default for the ContentHandler base
class, and true for TextContentHandler and it's
derivatives. (everything in core)

Extension content types that directly extend
AbstractContent / ContentHandler, often / generally don't
support direct editing. EntityContent in Wikibase
and Flow boards are the two such content types currently
in gerrit-hosted extensions.

The use and direct settings of the allowNonTextContent
member variable is replaced by enableApiEditOverride and
a setter for that. The only place allowNonTextContent is
used in all of Wikimedia-hosted git repos is core itself
(EditPage and ApiEditPage), so should be safe to make
this change.

With this change, Wikibase can remove its ApiCheckCanExecute
hook handler that disallows editing there, and MobileFrontend
could check if direct editing is allowed before enabling it's
editing features, instead of Wikibase having to add
MobileFrontend hook handlers to disable the features.

Bug: T96382
Change-Id: I276cd6ecedf38108f1f2be16b38e699e8c5d2d0c
2015-04-17 15:55:20 +00:00
Kunal Mehta
95928eaefd Don't require database access for ContentHandlerTest
We can just mark all the titles as non-existent in the LinkCache

Change-Id: Ifd6fdfd0f8318dac8af7f7509705cf30d44c8a4e
2015-03-26 17:01:13 -07:00
Timo Tijhof
7c340aed29 JsonContent: Support non-object values as root structure
* Remove is_object restriction from isValid.
* Implement rootValueTable to support rendering of other
  root structures.

Also:
* Minor method documentation tweaks.
* Document why quotes are not escaped for strings in value cells.

Bug: T86270
Change-Id: Ic1d10393912fcefa22d675fd4aa2baf437d2a05a
2015-01-16 20:57:07 +00:00
Timo Tijhof
3b44da3261 content: Recognise .json as JsonContent in User and MediaWiki namespace
Bug: T76554
Change-Id: I45200f4651d7200cfad541cdd21cc625c711e81f
2015-01-05 20:05:55 +00:00
Christian Aistleitner
58e23da40f Make PHPCS pass on JsonContentTest
Change-Id: I1054d7d0e903e3be7cb815035dddb0f34bcd6334
2014-12-18 14:13:07 +01:00
Timo Tijhof
e4f84af980 content: Refactor and fix various bugs in JsonContent
Follows-up d2a82fcb60. These issues weren't previously exposed
as nothing uses JsonContent by default in core, and the extensions
using it (e.g. EventLogging) lock it down very early. As are
most of these methods were never really put to use (they were
called after the extension does its superset of checking, or
too early and WikiPage ignores it).

Bug fixes

* Empty JSON object was converted to an array by PST conversion.
  The beautifyJSON method is intended for prettify purposes but
  actually modified the content stored in the database and made
  it no longer roundtrip ({} != []).

  We can't change getJsonData to return an object since it's
  a public method and people use it as an array. So we can't cast
  it to a PHP object as that would break back-compat.

  Turns out the class doesn't even support non-objects anyway (a
  primitive in JSON can trivially cause a fatal as it wasn't
  consistently considered invalid, though it didn't actually fatal
  due to some lucky spaghetti code in WikiPage).

* Fix beautifyJSON by checking for empty objects to prevent
  implicit {} to [] conversion.

* Add isValid() check to fillParserOutput() as it's called early
  on. Otherwise it throws a warning that 'foreach' (in objectTable)
  iterates over null. In practice it doesn't matter since the
  entire parser output is rejected when WikiPage eventually
  checks isValid (through Content::prepareSave).

* Consider all non- (PHP) array values invalid instead of just
  non-null values.

Enhancements

* Display message "Empty object" instead of a completely blank page
  for an empty object.

* Display message "Empty object" or "Empty array" instead of an
  empty table cell.

* Render arrays as a list of values (without indices).

* Remove italics from table cells for values. The monospace font
  should be enough. It also offsets it from the "Empty"
  placeholders (which are italicised).

Refactoring and clean up

* Use FormatJson::parse so that we can use Status to distinguish
  between null parse result and thus reliably cache it.

  Ideally we wouldn't need to cache it, but right now this code
  is pulled apart and called in so many strange ways that we end
  up calling this several times.

* Improve fairly meaningless test (testBeautifyJson) that was
  calling FormatJson in its data provider, exactly what the method
  being tested did. It also provided the test with data that could
  never end up in normal usage (a PHP-style associated array with
  implied numerical indices).

* Document that this class rejects non-array values.

* Document the problem with WikiPage assumming PST can run on any
  content. WikiPage fundamentally still assumes wikitext, in that
  there's no concept of invalid content.

* Fix incorrect documentation for getJsonData's return value
  (It may return null.)

* Fix incorrect documentation for beautifyJSON's return value.
  (It never returned boolean.)

Bug: T76553
Change-Id: Ifed379ba4674a8289b554a95953951886bf2cbfd
2014-12-17 23:04:24 +00:00
Timo Tijhof
b7a0f77800 content: Re-arange tests to declare data provider first
For consistency with other test suites.

Change-Id: Ic4b4d8e228a9ce82a42a07d88d690a9c932960c3
2014-12-03 03:38:43 +00:00
umherirrender
cb82a1d798 Fix bypassing hooks in TextContentTest
Save the whole global $wgHooks with the default way instead of just a
hook in a own variable.

Change-Id: Ie796f7a53ee38e0dc07f5b026134bbdcb8c9fd53
2014-10-23 17:46:55 +00:00
umherirrender
10e87016a4 Set content language for ContentHandlerTest::testGetAutosummary
This test depends on content language when comparing to a english
message text.

Change-Id: Ic50c0b6a821fb1d51d70119fbf712a18b73ce4c8
2014-10-01 19:38:32 +00:00
Timo Tijhof
96771e3a65 test: Clean up data providers that should be static
Follows-up b36d883.

By far most data providers are static (and PHPUnit expects them
to be static and calls them that way).

Most of these classes already had their data providers static
but additional commits sloppily introduced non-static ones.

* ResourceLoaderWikiModuleTest, 8968d8787f.
* TitleTest, 545f1d3a73.
  Odd unused method 'dataTestIsValidMoveOperation' was introduced
  in 550b878e63.
* GlobalVarConfigTest, a3e18c3670.

Change-Id: I5da99f7cd3da68c550ae507ffe1f725d31e7666f
2014-09-18 12:52:44 -07:00
Brad Jorsch
bba2bc6ca0 Record redirect target in ParserOptions
Since Id44d566a, the text passed to the parser when parsing a
&redirect=no page no longer contains the #REDIRECT directive. For the
benefit of extensions that want to know the redirect target from various
parser hooks, record the target on the ParserOptions object associated
with the parse.

Bug: 62856
Change-Id: Icd1da9911a43eabacbd9e9a369a8326f67f270ff
2014-09-18 06:19:31 +00:00
Yuri Astrakhan
3a28ee5acb CSS/JSON/JavaScript ContentHandler refactoring
* All content handlers that deal with code/data tend to have
English as their page language & pageview language, so moved common
code to the abstract CodeContentHandler class.

* Renamed JSONContent & JSONContentHandler into JsonContent*

Change-Id: I46819a0572ef5becc211d0d82471ff7102edaa3c
2014-09-15 08:24:15 +00:00