Commit graph

929 commits

Author SHA1 Message Date
Umherirrender
52338150c8 Fix return type for html strings
Change-Id: Ifc1ae7740ad1b130186b4b970d3d84651b016177
2018-04-06 13:07:01 +02:00
Subramanya Sastry
87c7ccd9bc Fix whitespace trimming in headings
* b3dd3881 was trimming whitespace in wikitext as well as HTML headings
  whereas the whitespace-trimming proposal was going to leave HTML tags
  untouched.

* 30495ea1 missed this because coincidentally, the test I added there
  for HTML headings had a typo and used <h2>...<h2> instead of
  <h2>...</h2> which caused the test to magically pass.

* This patch trims whitespace in
  doHeadings (which deals with wikitext headings) instead of
  formatHeadings (which deals with all headings).

* Updated parser tests to account for this.

Change-Id: I854f20b4c39a0a8e03d70155b269de77acf02cae
2018-03-23 11:42:01 -05:00
Subramanya Sastry
30495ea1f9 RFC T157418: Trim whitespace in table cells, list items, headings
* Matmarex had implemented this for wikitext headings in b3dd3881.
* This patch extends this to wikitext list items and wikitext table cells.
* Updated RELEASE NOTES.

tests/parser/parserTests.txt:
* All whitespace removed in output of list items, table cells, and
  headings. Removed corresponding whitespace in the input wikitext
  except for a few tests where the whitespace is significant "| +"
  or "| -", for example.
* Updated output of html/parsoid sections as well.
* Added new tests to spec white-space trimming behavior.

tests/phpunit/*:
* Fixed a few tests that used whitespace in list items and table cells.

Bug: T157418
Change-Id: I8ea34c7ab893c0c125c81d810feeb3c581e4bba1
2018-03-16 13:42:55 -05:00
C. Scott Ananian
65fcb7a945 Use class="free external" only on unbracketed URLs
The ability for URLs to be marked free even if they use bracketed syntax
but "sorta look free" (aka unbracketed) was added 13 years ago in
2d71cb3080 (r7074).

It seemed like a reasonable idea at the time: make printed output a little
prettier by marking "sorta free" URLs as free.  But this complicates the
semantics of wikitext, and introduces all sorts of strange corner cases,
for example:

  [http://example.com/&amp; http://example.com/&]

isn't marked as free, even though the parser output is:

  <a rel="nofollow" class="external text" href="http://example.com/&amp;">http://example.com/&amp;</a>

This functionality isn't actually needed: if you want the pretty printed
output of an unbracketed URL, then actually use an unbracketed URL.

In recent years we're more concerned with simplifying the semantics of
wikitext and eliminating corner cases, such that the content of our wikis
can be effectively archived.  The "effectively free" URLs are low-hanging
fruit in this quest.

Change-Id: I339e8698786c60c96a37a73443cb9a04362662c4
2018-03-07 00:20:09 -05:00
Tim Starling
f0247e05bd StripState testing and cleanup
* Added StripState unit tests
* Deprecated unmaintained "half-parsed" serialization experiment
* Renamed some variables for brevity and removed unused "prefix"

Change-Id: I838d7ac7f9a2189e13d39c6939dba5d70e74a6b7
2018-03-05 16:43:58 +11:00
Tim Starling
3dfda8c155 Limit total expansion size in StripState and improve limit handling
* Add a new limit to the parser which limits the size of the output
  generated by StripState. The relevant bug shows exponential blowup in
  output size.
* Remove the $prefix parameter from the StripState constructor. Used by
  no Gerrit-hosted extensions, hard-deprecated since 1.26.
* Convert the existing unstrip recursion depth limit to a normal parser
  limit with limit report row, warning and tracking category. Provide
  the same features in the new limit.
* Add an optional $parser parameter to the StripState constructor so
  that warnings and tracking categories can be added.

Bug: T187833
Change-Id: Ie5f6081177610dc7830de4a0a40705c0c8cb82f1
2018-03-05 05:16:04 +00:00
Arlo Breault
ee1787dd51 Ensure abort link parsing on xmlish tag in link title position
This shouldn't be dependent on the current definition of legal title
chars and strip marker.

See the test "<nowiki> inside a link"

Change-Id: I0d87aca1bb0adf4ec5ac480e0373a65fcd150a72
2018-03-01 14:08:24 -05:00
Brad Jorsch
2791fb0861 Hard-deprecate ParserOutput stateful transform methods
This also removes all the in-core calls that had been kept for the
benefit of extensions, and causes them to not have any effect since
anything that had been calling them was already either a no-op or will
probably be broken now that nothing in core is setting or checking the
flags.

Change-Id: Id22c1a5a6d6a249debb14063ae3f8838d105b634
2018-02-13 12:28:36 -05:00
Umherirrender
3124a990a2 Use ::class to resolve class names in includes files
This helps to find renamed or misspelled classes earlier.
Phan will check the class names

Change-Id: I07a925c2a9404b0865e8a8703864ded9d14aa769
2018-01-27 20:34:29 +01:00
Prateek Saxena
60a64e8912 Gallery: Use Parser::parseWidthParam() for gallery dimensions
Used by the `setWidths` and `setHeights` methods to make sure we are
using correct values.

Makes `parseWidthParam` static to be used in the gallery class.

Bug: T129372
Change-Id: I38b9ef0ea26e3748ad5d5458fadd2545f677ef93
2018-01-25 17:35:40 -05:00
jenkins-bot
a18476eab3 Merge "Remove @param comments that literally repeat what the code says" 2018-01-11 23:48:03 +00:00
Thiemo Mättig
ef470ebf7f Remove @param comments that literally repeat what the code says
These comments do not add anything. I argue they are worse than having
no comments, because I have to read them first to understand they
actually don't explain anything. Removing them makes room for actual
improvements in the future (if needed).

Change-Id: Iee70aad681b3385e9af282d5581c10addbb91ac4
2018-01-10 14:14:26 +01:00
Roan Kattouw
7f68220db6 Follow-up 6f07389ef2: fix variable name
Caused Notice: Undefined variable: text

Bug: T184123
Change-Id: I950a02134b145a2928af33995ca37a6965f265e4
2018-01-04 21:31:41 +00:00
Umherirrender
255d76f2a1 build: Updating mediawiki/mediawiki-codesniffer to 15.0.0
Clean up use of @codingStandardsIgnore
- @codingStandardsIgnoreFile -> phpcs:ignoreFile
- @codingStandardsIgnoreLine -> phpcs:ignore
- @codingStandardsIgnoreStart -> phpcs:disable
- @codingStandardsIgnoreEnd -> phpcs:enable

For phpcs:disable always the necessary sniffs are provided.
Some start/end pairs are changed to line ignore

Change-Id: I92ef235849bcc349c69e53504e664a155dd162c8
2018-01-01 14:10:16 +01:00
Kunal Mehta
37480222fb Parser: extract $title, follow-up 3d560be428
In the conversion away from extract(), the $title variable was missed. This
broke LabeledSectionTransclusion.

Change-Id: If4c140aedf16fc16a4ae2361f465798055748255
2017-12-30 18:50:06 +00:00
jenkins-bot
1a40e0cc86 Merge "Change php extract() to explicit code" 2017-12-28 09:44:59 +00:00
daniel
6af796f3e0 MCR: Deprecate and gut Revision class
This is a re-submission of I4f24e7fbb68.

As a first major step towards Multi-Content-Revisions (MCR),
this patch turns the Revision class into a legacy proxy for
the new RevisionRecord and RevisionStore classes.

Backwards compatibility is maintained for all but some
rare edge cases, like constructing a completely empty
Revision object.

For more information on MCR, see
<https://www.mediawiki.org/wiki/Requests_for_comment/Multi-Content_Revisions>.

NOTE: once this is merged, verify create/delete/restore cycle on beta,
      ideally with emulated replication lag.

Bug: T174025
Change-Id: Ia4c20a91e98df0b9b14b138eb4825c55e5200384
2017-12-21 18:08:54 +00:00
Daniel Kinzler
09bf4f5bb2 Revert "[MCR] Turn Revision into a proxy to new code."
This reverts commit 9dcc56b3c9.

With this patch applied, newly created revisions are sometimes not found
just after submitting an edit, until replicas have caught up.

Our best theory is that it somehow interfere with ChronologyProtector,
but we don't have a good idea how.

Also, as legoktm mentioned, the commit message is terrible and needs fixing.

Change-Id: Idf3404f3fa8f8d08a7fb2ab8268726e2c1edecfe
2017-12-19 12:38:48 +00:00
jenkins-bot
3d95da4952 Merge "Require indentation of CASE statements in PHP code" 2017-12-19 12:21:59 +00:00
daniel
9dcc56b3c9 [MCR] Turn Revision into a proxy to new code.
Change-Id: I4f24e7fbb683cb51f3fd8b250732bae9c7541ba2
2017-12-18 14:37:29 +00:00
jenkins-bot
47818f1b44 Merge "Split limit report out of Parser::parse()" 2017-12-15 05:04:01 +00:00
jenkins-bot
3844fd9d63 Merge "Parser: Add guessSectionNameFromStrippedText() and refactor" 2017-12-12 13:10:55 +00:00
Huji Lee
e74bfe13f6 Require indentation of CASE statements in PHP code
Bug: T182546
Change-Id: I91a9555893a08e4ec58da97c6cc4d1e70000ff6b
2017-12-10 22:07:50 -05:00
Phantom42
6c3a9662b2 Add quotes to comment based strip markers
Bug: T180159
Change-Id: Ic9dbb8ef3948fe751d16c3963769b616b5db2fc7
2017-12-08 17:00:26 +02:00
Umherirrender
3d560be428 Change php extract() to explicit code
Avoid php magic and make var settings more visible

Change-Id: I223874fd871104b0ac6a80d7f39c6dd997d0551d
2017-12-08 14:46:33 +01:00
Tim Starling
6a2a43f285 Split limit report out of Parser::parse()
It was 100 lines. Also update a few nearby comments. The one about just
handling <nowiki> sections was actually written by Lee, and is
hilariously outdated now.

Change-Id: I12ee2a7e488a3c787b36d3a457c6166bbbb46aff
2017-12-08 16:33:05 +11:00
Roan Kattouw
6f07389ef2 Parser: Add guessSectionNameFromStrippedText() and refactor
Split up guessSectionNameFromWikiText() into pieces to reduce code
duplication, and provide guessSectionNameFromStrippedText() which
doesn't do link stripping.

Really these should be named guessSection*ANCHOR*From... because they
return an anchor (with encoding and a '#' prefix) instead of a section
name, but I didn't want to rename the existing one.

Also make normalizeSectionName static (it doesn't use $this) so that
guessSectionNameFromStrippedText() can be static as well.

Change-Id: I56b9dda805a51517549c5ed709f4bd747ca04577
2017-12-07 10:22:45 -08:00
Max Semenik
129067c907 Remove nbsp and similar characters from section IDs
Bug: T90902
Change-Id: I71bdb7dd43c3e532287290e3c691d9739da45475
2017-11-02 19:35:11 -07:00
Santhosh Thottingal
f07b32a7dd Parser: Disable commafy for magic variables for month and day
In Parser#getVariableValue for the following magic variables
Language#formatNum was called without commafy parameter:

currentmonth, currentmonth1, currentday, currentday2, localmonth,
localmonth1, localday, localday2

The default value for formatNum nocommafy is false, meaning formatNum
will do commafication. For the above context, commafy is not needed
since the passed values are often month values like 02, 03 etc.
Commafy is noop on this values.

Explicitly pass false value for formatNum's nocommafy argument.
Language#formatNum method documentation for nocommafy also recommends
setting it true in case of dates.

Change-Id: I3233d5458af8cef583e5d1d599d9408542ba08c9
2017-10-16 08:35:26 +00:00
jenkins-bot
079d61fb79 Merge "Remove "only newlines in trailer" special case for category/language links" 2017-09-29 22:20:52 +00:00
jenkins-bot
a5b41a26b6 Merge "Fix link prefix/suffixes around Category and Language links (take 2)." 2017-09-19 16:59:58 +00:00
Fomafix
b6c895ddc5 Do not double decode HTML entities for IDs
* in links (T103714)
* in indicators (T104196)

This change removes the automatic Sanitizer::decodeCharReferences from
Sanitizer::escapeId and Sanitizer::escapeIdInternal. Where decoding of
HTML entities are wanted an explicit call to
Sanitizer::decodeCharReferences is added.

Explicit decode HTML entities in non local autocomments. (T104311)

Bug: T103714
Bug: T104196
Bug: T104311
Change-Id: I88e8e2077e6f5eec2b232391f7818370894a62dc
2017-09-12 15:42:17 +02:00
jenkins-bot
2480aae0c9 Merge "Show a warning in edit preview when a template loop is detected" 2017-09-11 18:26:11 +00:00
C. Scott Ananian
5676481c6d Remove "only newlines in trailer" special case for category/language links
This special case complicates wikitext semantics and ought to be
unnecessary.  Parsoid doesn't include this special case; if this patch
to the PHP parser isn't merged, we should write one for Parsoid to
implement the missing special case logic.

Bug: T175416
Change-Id: I3865c51b21de9d63ac5d06dcc3a3fa9108129d6c
2017-09-08 23:44:42 -04:00
C. Scott Ananian
6d5fd8077f Fix link prefix/suffixes around Category and Language links (take 2).
Previous attempt was I943cd9bec0855d9a326b0b50739d686a29995370, reverted in
e687f2da3e due to T174639.

There's still a weird behavior with newline stripping between links, which
I'll try to tackle in a follow-on patch (T175416).

Bug: T2087
Bug: T10897
Bug: T87753
Bug: T174639
Change-Id: I8228cdd3b80faf899000adb511a983edc454bc76
2017-09-08 16:12:21 -04:00
jenkins-bot
c15f569fce Merge "Revert "Fix link prefix/suffixes around Category and Language links."" 2017-09-01 00:57:54 +00:00
Tim Starling
e687f2da3e Revert "Fix link prefix/suffixes around Category and Language links."
This reverts commit c66c9aa535.

Bug: T174639
Change-Id: Ibf6d3780f384ba8edc80bf28c893f1aee8ce28a8
2017-09-01 00:47:32 +00:00
Kunal Mehta
844d724621 Avoid using deprecated Title::canTalk()
Change-Id: Ibd224f9de595435524e683262882c9ebf2761abf
2017-08-29 12:36:33 -07:00
Matthew Bowker
a296331541 Remove two deprecated functions and one depreciated variable in a function call within Parser.php
* Parser::getRandomString() (deprecated in 1.26) was removed.
* Parser::uniqPrefix() (deprecated in 1.26) was removed.
* Parser::extractTagsAndParams() now only accepts three arguments.  The fourth, $uniq_prefix was deprecated in 1.26 and has now been removed.

Bug: T61113
Change-Id: I7333fff4eb8b9a754b4596992f2a69bbdaac664d
2017-08-22 18:14:14 -05:00
C. Scott Ananian
c66c9aa535 Fix link prefix/suffixes around Category and Language links.
Bug: T2087
Bug: T10897
Bug: T87753
Change-Id: I943cd9bec0855d9a326b0b50739d686a29995370
2017-08-15 13:13:12 -04:00
WMDE-Fisch
6df9ed1ad6 update mediawiki-codesniffer to 0.11.0 and fix issues
- mostly auto fixes
- some too long lines fixed
- ignore amp space in one case  passing by reference

Change-Id: I6472f83bc3cbf4bd629d83050cc3319b19ec465c
2017-08-11 22:27:51 +02:00
Umherirrender
a9007e8baf Add missing & to @param documentation to match functon call
Change-Id: I81e68310abcbc59964b22e0e74842d509f6b1fb9
2017-08-11 18:47:46 +02:00
Max Semenik
fd6e9ef2d4 Human-readable section ID support
It adds the ability to replace the current section ID escaping
schema (.C0.DE) with a HTML5-compliant escaping schema that is
displayed as Unicode in many modern browsers.

See the linked bug for discussion of various options that were
considered before the implementation. A few remarks:
* Because Sanitizer::escapeId() is used in a bunch of places without
  escaping, I'm deprecating it without altering its behavior.
* The bug described in comments for Parser::guessLegacySectionNameFromWikiText()
  is still there in some Edge versions that display mojibake.

Bug: T152540
Change-Id: Id304010a0342efbb7ef2d56c5b8b244f2e4fb2c5
2017-08-01 20:32:20 -07:00
Kunal Mehta
d1cf48a397 build: Update mediawiki/mediawiki-codesniffer to 0.10.1
And auto-fix all errors.

The `<exclude-pattern>` stanzas are now included in the default ruleset
and don't need to be repeated.

Change-Id: I928af549dc88ac2c6cb82058f64c7c7f3111598a
2017-07-22 18:24:09 -07:00
jenkins-bot
3b5b239e85 Merge "Make multiple colons escaping interlanguage links invalid, consistently" 2017-07-11 15:19:17 +00:00
Aaron Schulz
7ff8529984 Avoid high edit stash TTLs when a user signature was used
This adds a new ParserOuput user-signature tracking flag.

Bug: T84843
Change-Id: I77de05849c15e17ee2b9b31b34172f4b6a49a38e
2017-07-06 16:34:26 -07:00
Arlo Breault
0e1b52a40e Make multiple colons escaping interlanguage links invalid, consistently
* Right now, one or two are permitted.  This patch limits it to one.
   The current behaviour seems more a byproduct of refactoring than an
   explicit goal.

 * Note that this will break links on a handful of pages surfaced in
   Parsoid's roundtrip testing.

Change-Id: Icabd34bbf15781bb891bd8e0c079d1a65eb28595
2017-07-06 17:09:25 -04:00
Umherirrender
b5cddfb27b Remove empty lines at begin of function, if, foreach, switch
Organize phpcs.xml a bit

Change-Id: Ifb767729b481b4b686e6d6444cf48b1f580cc478
2017-07-01 11:34:16 +00:00
Umherirrender
be42e09aa8 build: Prepare for mediawiki/mediawiki-codesniffer to 0.9.0
The used phpcs has a bug, so the version 0.9.0 could not be enforced at the moment.
Will be fixed in next version, see T167168

Changed:
- Remove duplicate newline at end of file
- Add space between function and ( for closures
- and -> &&, or -> ||

Change-Id: I4172fb08861729bccd55aecbd07e029e2638d311
2017-06-26 17:14:31 +00:00
jenkins-bot
3f849d695f Merge "Parser: Emit deprecation warnings for ParserLimitReport hook" 2017-06-26 15:05:22 +00:00