Commit graph

608 commits

Author SHA1 Message Date
Timo Tijhof
60e4f3fd90 tests: Ensure main RequestContext reflects custom $wgLang
Call RequestContext::resetMain() so that any subsequent call
to getMain() and getLanguage() is aware of the latest state
as set up by the setUp() and setupGlobals() methods.

The MediaWikiTestCase class for PHPUnit did this already, but
the parserTest suite and UploadFromUrlTestSuite didn't yet.

Change-Id: I6481176228944004091078704d0346c8f3fc0cf1
2016-03-03 01:39:51 +00:00
Arlo Breault
d13a933fee Fix parserTests: Set scriptPath in setupGlobals() as well
Follows-up to 2eeda9bc7, which only set it in setUp().

Change-Id: Ifafaeaab679f51d452419bea223a955afcc63b51
2016-02-23 22:05:34 +00:00
Arlo Breault
88d7413ddf Sync up with Parsoid parserTests.
This now aligns with Parsoid commit 239d2e60d82c736905a935525db748ebf6b45e23

Change-Id: Iec62b6824ab7d4c5d660999af130c777f8621571
2016-02-23 12:24:58 -08:00
Timo Tijhof
2eeda9bc7d parserTest: Make $wgResourceBasePath match $wgScriptPath
Right now it forgets to reset $wgResourceBasePath, which means it
is inherited from the wikis's (or Jenkins') default settings which
is typically '/w'. That caused parser tests to behave as if pointers
to /extensions were outside /w.

Also update wgScriptPath to be '' instead of '/'. Otherwise this
can cause double-slash prefixed urls.

Change-Id: Ic455d62fca8fcac2c4ecc055cc0d7e311b70a94a
2016-02-23 02:23:05 +00:00
Arlo Breault
87f43fd803 Match html5 unquoted attribute parsing
* Brings us closer to the html5 attribute parsing algorithm described
   in http://www.w3.org/TR/html5/syntax.html#before-attribute-value-state

 * There's a similar patch for the Parsoid in,
   I2160a23b2a3c914eb369347bbf5d58328440041d

 * The spec says <div class=  style="123">hi</div> should parse as
   <div class="style=\"123\"">hi</div>, which it now does, whereas it
   used to yield <div class="" style="123">hi</div>.

 * Merge with caution. This is going to break pages like,
   frwikisource/La_Mirlitantouille_(Lenotre)?oldid=4669681

Bug: T108134
Change-Id: Ic2fc1b573a55a847e6c05707678b58c1189ecc52
2016-02-22 00:50:06 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Bartosz Dziewoński
0fd1a6b62b Merge "Revert "Preprocessor: Don't allow unclosed extension tags (matching until end of input)"" 2016-02-04 01:10:58 +00:00
Legoktm
543f46e9c0 Revert "Preprocessor: Don't allow unclosed extension tags (matching until end of input)"
This reverts commit f51d0d9a81.

Breaks templates with non-closed </noinclude> tags, which
were previously acceptable.

Bug: T125754
Change-Id: I8bafb15eefac4e1d3e727c1c84782636d8b82c2b
2016-02-04 00:38:35 +00:00
Arlo Breault
55313f4eaf Remove use of explodeMarkup
* At that point, element attributes are already escaped so it serves no
   purpose. Before `doTableStuff` is called, `Sanitizer::removeHTMLtags`
   has been invoked which calls `Sanitizer::fixTagAttributes` which
   calls `Sanitizer::safeEncodeTagAttributes` and finally gets down to
   `Sanitizer::safeEncodeAttribute`, with the goal of "extra armoring
   against further wiki processing."

Change-Id: Ieeb9b21148c2909eb839d13195d7d10012b48e3b
2016-01-29 11:15:29 -08:00
Arlo Breault
67b0c669a5 Sync up with Parsoid parserTests.
This now aligns with Parsoid commit b04ce02432166128a295c0f00ac1d64c6a469a81

Change-Id: I9f27dc299b87aa265039c0e25084489810c0f7c8
2016-01-25 12:14:31 -08:00
Arlo Breault
63aeabeff1 Last caption wins in gallery
* Currently, for images:

   [[File:Foobar.jpg|hi|alt=100|ho]]

   caption: ho

   but for galleries:

   <gallery>
   File:Foobar.jpg|hi|alt=100|ho
   </gallery>

   caption: hi|ho

 * This patch brings some consistency to them.

Change-Id: I3b73189b27cc35fade4809477cf18779b953aa3b
2016-01-22 11:22:06 -08:00
Bartosz Dziewoński
f51d0d9a81 Preprocessor: Don't allow unclosed extension tags (matching until end of input)
I think it's saner to treat this as invalid syntax, and output the
mismatched tag code verbatim. The current behavior is particularly
annoying for <ref> tags, which often swallow everything afterwards.

This does not affect HTML tags, though. Assuming Tidy is enabled, they
are still auto-closed at the end of the page content.

Related to T17712 and T58306. I think this brings the PHP parser closer
to Parsoid's interpretation.

It reduces performance somewhat in the worst case, though. Testing with
https://phabricator.wikimedia.org/F3245989 (a 1 MB page starting with
3000 opening tags of 15 different types), parsing time rises from
~0.2 seconds to ~1.1 seconds on my setup. We go from O(N) to O(kN),
where N is bytes of input and k is the number of types of tags present
on the page. Maximum k shouldn't exceed 30 or so in reasonable setups
(depends on installed extensions, it's 20 on English Wikipedia).

To consider:
* Should we keep previous behavior for unclosed <includeonly> /
  <noinclude>? This would be particularly disruptive for these if
  someone relied on the old behavior, and they're already
  special-cased in places.
* Unclosed <pre> tags are now treated as HTML tags, and are still
  displayed as preformatted text, but without suppressing wikitext
  formatting.

Change-Id: Ia2f24dbfb3567c4b0778761585e6c0303d11ddd0
2016-01-21 04:22:34 +00:00
jenkins-bot
3092b9bf8e Merge "ParserTests: Add a video file to parser test suite" 2016-01-09 07:30:54 +00:00
Derk-Jan Hartman
175e95a909 ParserTests: Add a video file to parser test suite
Introduce an ogv video file to the parser file testsuite, so that we
can use it later in TimedMediaHandler parsertests.

Change-Id: I6a3b307ad9c82e9df0aeec025934d736eec8375f
2016-01-07 13:24:43 -08:00
Derk-Jan Hartman
8ddb11ad1c Print which file we are running parsertests for
I spent half an hour searching which extension was causing a crash in
my parser tests. Printing the filename seems like a handy plan here...

Change-Id: I09558101be15e92eeb3841f8f4198c2bc01f3e9d
2016-01-07 00:36:17 -08:00
This, that and the other
e2278426ea Fix parserTests after f66ff60244
Change-Id: If3691cb022211e78093e1cbd36dbb76b6d5f674b
2016-01-07 15:55:53 +11:00
Arlo Breault
edf52ea076 Sync up with Parsoid parserTests.
This now aligns with Parsoid commit cc0dfb3271495d764e471d18927b0b3ee8ce1cdb

Change-Id: I3fd6f8cc31a5536471a04d2f6a52de92893f9911
2015-12-29 16:31:14 -08:00
Subramanya Sastry
013540cfc9 Revert "Sync up with Parsoid parserTests."
Something is broken in the syncing worfklow. I deleted a
test that cscott added in 1888bf560.

Reverting this sync so we can figure out what is broken.

This reverts commit b55a729f74.

Change-Id: Idd32daa95bb44fd073262a1c37ded236bf724e5d
2015-12-29 23:33:19 +00:00
Subramanya Sastry
b55a729f74 Sync up with Parsoid parserTests.
This now aligns with Parsoid commit cc0dfb3271495d764e471d18927b0b3ee8ce1cdb

Change-Id: Iea3694b397c4e4b8ae87b1e61418767e2109f61e
2015-12-29 15:50:50 -06:00
C. Scott Ananian
1888bf5601 Don't allow atttributes whose namespace starts with data-.
Change-Id: Ia76c74941b09e3ad131fe2fee31ffec3e540170b
2015-12-22 18:35:09 -05:00
Timo Tijhof
b1e3f9e38d Don't trigger 'wikipage.diff' hook for content that looks like a diff
Follows-up 9cfb9cb9fb, and b62f0e9156.

Add data-mw="interface" to elements created by the interface
(e.g. not user-generated content) and use this to narrow down
scope of elements eligible for JavaScript binding.

This avoids bugs where e.g. the diff hook triggers on a wiki page about diffs.
This isn't a security issue per-se, but causing odd behaviour.

Also add missing tests for data-ooui filtering (follows-up aa9a52da).

Change-Id: I9a0c86c92d411538bd9e203ec6ae54616fdf49b8
2015-12-15 16:29:58 -08:00
Timo Tijhof
baf1721491 resourceloader: Remove obsolete msg_resource database table
No longer used as of 5d5b269e0e.

Change-Id: Ieb8448185cf39aa396e902e025e673f0886ac3c3
2015-12-14 01:24:27 +00:00
Timo Tijhof
cbf5bacae5 resourceloader: Remove obsolete msg_resource_links table
No longer used as of Ia9131f570.

Change-Id: If009e2620e59002e127d21b90a96bdd29e5d2a9d
2015-12-14 01:23:19 +00:00
jenkins-bot
9ab6887c8c Merge "Reserve data-mw and data-parsoid attribute prefix for trusted values" 2015-12-09 06:54:56 +00:00
Brian Wolff
b62f0e9156 Reserve data-mw and data-parsoid attribute prefix for trusted values
Don't let users set attributes starting with data-mw or data-parsoid.
The main idea is to allow MediaWiki to use data-mw-<something>
attributes for trusted input to client side scripts. There have
been a couple security vulnerabilities in the past based on users
being able to manipulate a data attribute, which client side was
assuming was trusted.

Also include data-mw and data-parsoid as both are used by Parsoid
currently.

See https://lists.wikimedia.org/pipermail/wikitech-l/2015-November/083811.html

A corresponding change will also have to be made in Parsoid.

Change-Id: I06585380bde3bc57b17ad76740c5acc2056d7c44
2015-12-09 06:47:49 +00:00
Reedy
00c426e3c2 Replace wfBaseConvert with Wikimedia\base_convert
Change-Id: Iadab3d018c3559daf79be90edb23d131729bdb68
2015-11-24 22:51:42 +00:00
Arlo Breault
1f08236b97 Sync up with Parsoid parserTests.
This now aligns with Parsoid commit b869b084be641d0e2150648ea926ca974590e6e3

Change-Id: I3e8ebd578db8b6630d23ebbdedb5036fa5a092e5
2015-11-10 01:05:20 +00:00
Glaisher
0968cce3f7 Don't apply CSS columns if less than 3 results were found on AllPages & PrefixIndex
If there are less than 3 entries, the browser still tries to render as it would
appear as if there are 3 columns, so the final rendering is broken. So don't apply
CSS columns and just show a normal list if there are less than 3 results. This also
matches the behavior of CategoryViewer.

Bug: T117887
Change-Id: Ie6ac0e1174ff8cc14008f39a91c95bcd6f616353
2015-11-09 22:28:37 +05:00
Tim Starling
eb40eb0f18 Client-side migration for empty li preservation
It is desirable in terms of user-friendly syntax to display an empty
list item if the user adds one to the source. However, we suspect that
this change will break the rendering of existing templates. So, preserve
the empty <li> element, but style it with display:none so that there is
no user-visible change. Changes can then be observed with a user script,
then eventually the CSS can be removed so that the desired behaviour will
be user visible.

This is imagined as a staged deployment of T89331, i.e. it is better to
resolve differences with Html5Depurate one at a time instead of
deploying it all at once.

The CSS module is specified in parser/MWTidy.php since the tidy driver
hierarchy is not meant to be so closely tied to the MW environment.

Bug: T49673
Change-Id: Ifb44b782c617240e3de73dcdf76c8737c7307d94
2015-10-28 23:35:18 +00:00
Subramanya Sastry
e97dca8cbd Sync up with Parsoid parserTests.
This now aligns with Parsoid commit 31012bb740fea4fe6ce44464dd940d48576ed4ef

Change-Id: I525622766d702c53009d59c1a2a80a52966774fe
2015-10-20 21:33:39 +00:00
Glaisher
3b3a119c46 Use responsive CSS columns on Special:PrefixIndex and Special:AllPages
Stop using table on Special:PrefixIndex and use CSS columns instead and
removed 'column' parameter from this page.
This will also change the weird ordering (side to side) to the standard
top to bottom instead.

The pagination links hide at the corner on small screens. This can be fixed
later.

Bug: T32965
Change-Id: I720d34e21950c18de3c6e0b1d6d8a4461db495cb
2015-10-18 21:11:04 +05:00
Tim Starling
75d7731dde In parserTests.php, fix upload directory handling
Instead of having two conflicting filerepo configurations, one at
/tmp/test-repo and another at a randomized path, and populating the one
that isn't used with files, let's just have a single upload directory,
populate it with files, and then actually use those files.

This fixes a set of confusing system-dependent parser test failures. In
the failure scenario, the file upload would be recorded, but then
invalid JPEG metadata would trigger LocalFile::loadFromFile(), which
would look for the file in the wrong place and mark it missing. Then
parser tests would fail due to image links being broken (red).

This is probably not as nice as the fake in-memory repo used by
NewParserTest, but this approach does provide a richer integration test.
This is a conservative fix that just fixes the things that are terribly
broken rather than making new things.

Also:

* Clear the RepoGroup singleton, for completeness, since it doesn't
  make sense to set $wgLocalRepo without clearing it. Since we now set up
  a similar repo at initialisation to the one set up with each test, it
  probably doesn't have any effect.

* Warn if gd is not present since this causes 49 test failures.

* Use glob patterns in teardownUploadDir() instead of requiring every
  file to be individually listed. This fixes an error due to failure to
  delete a 240px file.

Change-Id: I56a0e0d1cb363b40cf19c735e00cbb8929c1401a
2015-09-28 15:48:32 +10:00
Vivek Ghaisas
c54766586a Fix issues identified by SpaceBeforeSingleLineComment sniff
Change-Id: I048ccb1fa260e4b7152ca5f09b053defdd72d8f9
2015-09-26 23:06:52 +00:00
Reedy
58f0a7ee4e Wrap some long strings in tests/
Change-Id: I89d53c5051e5ee4bd8624df8ee2b25993090a7df
2015-09-26 21:01:59 +01:00
Arlo Breault
cc062f0e76 Sync up with Parsoid parserTests.
This now aligns with Parsoid commit 6619409e60a3208144836970773f191e7a6d147f

Change-Id: Iea6199d2d6a314f118febdc2308be1723a9970d2
2015-09-24 10:26:13 +02:00
C. Scott Ananian
a05971dfc7 Terminate free external link on &nbsp; (and numeric versions of <>)
Bug: T84937
Change-Id: Ic74d8d069e08c0597c7b26755e0d942bf3a510cc
2015-09-23 16:00:52 -04:00
Tim Starling
2c6c954e23 Abstract and refactor Tidy support
* Split tidy implementations into a class hierarchy
* Bring all tidy configuration into a single associative array and
  deprecate the old configuration.
* Remove $wgAlwaysUseTidy

This is preparatory to replacement of Tidy (T89331). I used the name
"Raggett" for things relating to Dave Raggett's Tidy, since if we use
"tidy" to mean the new abstract system as well as Raggett's tidy, it
gets confusing.

Change-Id: I77af1a16cbbb47fc226d05fb9aad56c58e8910b5
2015-09-10 20:18:52 -07:00
Arlo Breault
bef1ba5c71 Sync up with Parsoid parserTests.
This now aligns with Parsoid commit 45ef2301bfc549c890851de811e5b34f68d5646a

Change-Id: I9394b326bc370ea6240dd6504aabc2678e346d2a
2015-09-10 09:17:51 -07:00
jenkins-bot
804cc6885a Merge "Revert "Do not encode "'" as %27 (redirect loop in Opera 12)"" 2015-09-10 15:36:10 +00:00
Bartosz Dziewoński
8477b1b277 Revert "Do not encode "'" as %27 (redirect loop in Opera 12)"
This seems to cause redirect loops in current Firefox instead.

This reverts commit a89a21990e.

Bug: T106793
Change-Id: I18fac8ab0f94e2df8476131b132c9866902a02c4
2015-09-10 15:23:25 +00:00
Subramanya Sastry
23e2f64ec5 Sync up with Parsoid parserTests.
This now aligns with Parsoid commit c8c7b5efa5a1754ec17294ee9e0269b3dfc9b0b1

Change-Id: I006cc95fb354c1b4ad40f0999938c5c4afaf099d
2015-09-07 20:28:23 -05:00
Bartosz Dziewoński
a89a21990e Do not encode "'" as %27 (redirect loop in Opera 12)
Similar to 7b4df0e12e,
but for apostrophe instead of tilde and for Opera
instead of Chrome.

Bug: T106793
Change-Id: Ic54390434cebcc76a6c8ab49acc164d36e0cdff6
2015-09-05 21:17:33 +01:00
Subramanya Sastry
cd3ddb1f59 Sync up with Parsoid parserTests.
This now aligns with Parsoid commit 5f2fae6c7e601b706403621924b769e87c5ed554

Change-Id: Ied3e5b2ac4df059c37f8e617ae55e47e8d15c1ee
2015-09-02 13:23:01 -05:00
Arlo Breault
174b09c184 Allow stop characters as quoted attribute delimiters
* Matches Parsoid in Ibdaa51f94eadc640278594a3eb5dd43356c286ea

Change-Id: I51b6861d7232d857b75881966091ae1e662c13a9
2015-08-26 14:33:12 -07:00
mjbmr
04f93370bb Replace 'emailpage' with 'emailuser'
Bug: T109614
Change-Id: I345d0091b17ef61499dc7d8c170308fd8194dc00
2015-08-20 10:49:03 +00:00
jenkins-bot
586effc0b7 Merge "Escape return of {{int:}} if message not exists" 2015-08-19 17:31:48 +00:00
C. Scott Ananian
bb281c0317 Sync up with Parsoid parserTests.
This now aligns with Parsoid commit e2aa4e4159b9c7b5350567f26f5f14780843a368

Change-Id: I64b4e901ece0e66011b5110312885fab9c8b9cb8
2015-08-18 19:24:34 -04:00
C. Scott Ananian
bc75784cbb T106578: Update Sanitizer to match legal HTML5 character entities.
Invalid HTML5 character entities become instances of UTF8_REPLACEMENT,
so we also ensure that checkCSS notices this and emits the proper
human-friendly sanitization notice.

Change-Id: I76cef7c772b1e3eba0af8dab6403e9100beab03a
2015-08-18 23:05:10 +00:00
C. Scott Ananian
87eebf8dd5 Support IPv6 URLs in bracketed and auto links.
The corresponding patch for Parsoid is
Ibb33188cdfe2004e469c3f6ee6f30d34d1923283.

Task: T23261
Change-Id: Iff077bf31168b431febb243e2e62f2c6502616bc
2015-08-18 22:50:58 +00:00
C. Scott Ananian
59e8147b67 Sync up with Parsoid parserTests.
This now aligns with Parsoid commit 48617c40c2eb9c7a421bb9c5b3ba0907fdec9c21

Change-Id: I565886079e64985844fc0a7c67ab74340b42f84a
2015-08-17 16:52:48 -04:00