Commit graph

973 commits

Author SHA1 Message Date
Tim Starling
a6a017cea4 Fix use of non-existent variable Parser::$config
Fix bug from Ib4394f370cb561ccf195338a1c2e9e465dcb3dc3

Add test.

Bug: T208000
Change-Id: Ia81cca1b64afef2af3cb8dff19719a7f0de9d306
2018-10-25 16:27:55 -07:00
jenkins-bot
b3f03fd75e Merge "Inject Config into Parser instead of using globals" 2018-10-17 14:34:31 +00:00
James D. Forrester
976c50c21a Drop ParserLimitReport, deprecated in 1.22
Change-Id: I4898d92569bd823f09c12f68fa186e2e139790a7
2018-10-10 16:20:18 -07:00
Aryeh Gregor
5173d5ee60 Inject Config into Parser instead of using globals
Change-Id: Ib4394f370cb561ccf195338a1c2e9e465dcb3dc3
2018-10-02 21:26:01 +03:00
jenkins-bot
50f6b24ee6 Merge "Parser: Refactor parsing of [[File:...|link=...]] syntax for reusability" 2018-09-26 19:18:34 +00:00
Bartosz Dziewoński
1c9664d18a Parser: Refactor parsing of [[File:...|link=...]] syntax for reusability
Change-Id: I91467297de4b7c532448a4c20b9a0dc8216c7200
2018-09-26 13:36:32 +02:00
C. Scott Ananian
327f0f92fa Use wfIsHHVM() instead of a HipHop-specific environment variable
Change-Id: I5bbf3e4f65d9b6a0d7419f67e3931e77e92b7e6c
2018-09-20 09:23:54 -04:00
daniel
465954aa23 Provide new, unsaved revision to PST to fix magic words.
This injects the new, unsaved RevisionRecord object into the Parser used
for Pre-Save Transform, and sets the user and timestamp on that revision,
to allow {{subst:REVISIONUSER}} and {{subst:REVISIONTIMESTAMP}} to function.

Bug: T203583
Change-Id: I31a97d0168ac22346b2dad6b88bf7f6f8a0dd9d0
2018-09-06 18:33:44 +02:00
Brian Wolff
13e5700b23 Use annotations for taint in Parser & ParserOutput.
This replaces the builtin taints that are removed in
Ic1e1983a51c. Additionally, parse will no longer warn about
double escaping - there's many situations where such warnings
are wrong (e.g. Using Html::rawElement()). However this also
means that Parser::parse( wfMessage( 'foo' )->parse() ); will
no longer give a double escaping warning, which is unfortunate.

Bug: T202380
Change-Id: Ia52d37411beb62b112c6ff102438063c3d750769
2018-08-31 15:55:44 +00:00
jenkins-bot
a3357744c3 Merge "[MCR] Introduce RevisionRenderer" 2018-08-31 11:25:15 +00:00
Kunal Mehta
eb7150b029 Set @param-taint for Parser::internalParse()
This is not strictly accurate, because Parser::internalParse() actually
returns half-parsed HTML, which is not safe for output. But it is safe for
output from a parser tag.

Maybe phan-taint-check plugin needs to learn about half-parsed HTML as an
extra taint type, and make that an acceptable thing for parser tags to return,
but not other things.

But this fixes the failures for the Listings extension, so I think it's
worthwhile in the meantime.

Change-Id: Idf87f5c3dcf81dd210de73a4ff15e3b1aabd9f89
2018-08-30 21:46:10 -07:00
daniel
e9f71517f7 [MCR] Introduce RevisionRenderer
RevisionRenderer is the MCR replacement for Content::getParserOutput,
as outlined in <https://www.mediawiki.org/wiki/User:Daniel_Kinzler_(WMDE)/MCR-PageUpdater>.

Note: This change also introduces quite a bit of code for
merging ParserOutput objects.

Bug: T194048
Change-Id: I871978bf79f67c9e7954fb3fc8528d6e365f2cc1
2018-08-30 19:15:12 +02:00
jenkins-bot
ba6c827485 Merge "Apply content wrapping in ParserOutput::getText()" 2018-08-29 16:25:22 +00:00
daniel
0dc7ba02b4 Apply content wrapping in ParserOutput::getText()
Instead of applying wrapping the the parser and unwrapping in
ParserOutput::getText(), turn this around and apply wrapping in getText(),
and only if desired.

This avoids search&replace logic for unwrapping, and it also makes it a lot
easier to merge the output of multiple slots for MCR output.

This changes behavior in two hopefully irrelevant ways:
1) the limit report comments will be inside the wrapper div, instead of
following it.
2) if HTML with a wrapper div is explicitly injected into a ParserOutput
object, it will not be possible to unwrap the text.

Bug: T174035
Change-Id: I1641b7995af9bd297f1acd610d583fbf874f34e0
2018-08-29 16:46:25 +02:00
jenkins-bot
e548e0f35c Merge "Make interwiki transclusion use the WAN cache" 2018-08-27 21:17:04 +00:00
Aaron Schulz
6504e23074 Make interwiki transclusion use the WAN cache
This means that now:
* Entries actually get deleted when expired
* The transclusion cache is shared across wikis
* Large blobs that do not fit in cache no longer cause DB errors
* DB writes are not triggered on GET requests
* Keys are hashed and no longer need to be so restrictive

Also, add a "check key" based purge system and process cache the
text/html values similar to how regular revision text is cached.

Bug: T189702
Change-Id: I8ac12b53c02bb26857175dd5a4af29d49e03dc33
2018-08-27 19:32:04 +00:00
Kunal Mehta
2852255186 Inject SpecialPageFactory into Parser
Change-Id: I6a6a94cbdafdc724ce02408cd9e744e7b3eda92b
2018-08-17 12:03:13 -07:00
Kunal Mehta
b7b8f214bb Parser: Call firstCallInit() in getTags/getFunctionHooks
So callers don't need to do this manually. Pointed out by Tim in T201799.

Depends-On: Ia6c36d5a650095e35093bf47e275e081e89b3daf
Change-Id: Ida62767f3ca53f99609cae01d3a20051bb92ccab
2018-08-14 14:16:42 -07:00
Kunal Mehta
e8370d6977 Parser: Add accessors needed by CodeMirror
Change-Id: Ia2d98baf6caed2cd779cb00aceba5785cf13d633
2018-08-13 22:44:48 -07:00
Aryeh Gregor
90d4f56fe4 Mass conversion of $wgContLang to service
Brought to you by vim macros.

Bug: T200246
Change-Id: I79e919f4553e3bd3eb714073fed7a43051b4fb2a
2018-08-11 22:44:29 -06:00
Aryeh Gregor
bca6085920 Use ParserFactory in a bunch of places
I wasn't sure how to convert the rest of the occurrences in core (there
are a significant number).

Bug: T200881
Change-Id: I114bba946cd3ea8a293121e275588c3c4d174f94
2018-08-11 00:16:17 -06:00
Aryeh Gregor
62515f7b15 Introduce ParserFactory service
Bug: T200881
Change-Id: I257e78200983cb10afb76de1f07dd1b9d531c52a
2018-08-11 00:15:52 -06:00
jenkins-bot
8e7f352610 Merge "Update Parser to use ContentLanguage" 2018-08-03 04:52:25 +00:00
Aryeh Gregor
c2f29a2efc Update Parser to use ContentLanguage
Bug: T200246
Change-Id: Ie54677706ec175189c3ff52342a9d8ac2f5d90d8
2018-08-03 04:29:39 +00:00
Kunal Mehta
612f6e5690 Document Parser::$mFirstCall
And don't bother checking its value in clearState(), since firstCallInit()
will do that anyways.

Change-Id: Ibc5e809daa614e99be91d65a363de4f697e6afa5
2018-08-02 01:57:53 -07:00
Aryeh Gregor
5a16d92e04 Update MagicWordArray to use MagicWordFactory
Bug: T200247
Change-Id: Ie5a60b81382d7299beadc691fe4d27e931ebe0ed
2018-07-31 21:40:21 +03:00
Aryeh Gregor
72ab013be0 Update CoreParserFunctions to use MagicWordFactory
Bug: T200247
Change-Id: I122d8acf601581b18756a5b8d65e50953b28c21d
2018-07-31 15:36:57 +03:00
Aryeh Gregor
c3ee073d62 Update Parser to use MagicWordFactory
Bug: T200247
Depends-On: Ie061fe90f9b9eca0cbf7e8199d9ca325c464867a
Change-Id: Iab6a4cb32c491caf2685cdd68f9465ef1dfa3c4c
2018-07-30 21:20:43 +03:00
Brad Jorsch
d469f53847 Parser: Remove style and script tags' content from TOC
We don't want to display the stylesheet as part of the TOC entry if
someone uses TemplateStyles in a heading.

Bug: T198618
Change-Id: I2f7316daaba0cce662b6a4702ab87322e6783655
2018-07-16 22:52:51 -04:00
Umherirrender
130ec2523d Fix PhanTypeMismatchDeclaredParam
Auto fix MediaWiki.Commenting.FunctionComment.DefaultNullTypeParam sniff

Change-Id: I865323fd0295aabd06f3e3c75e0e5043fb31069e
2018-07-07 00:34:30 +00:00
Fomafix
a60dcdc2e3 Armor against French spaces detection in HTML attributes
This change also solves T13874 in a generic way.

Bug: T5158
Change-Id: Id8cdb887182f346acab2d108836ce201626848af
2018-06-21 19:24:07 +02:00
jenkins-bot
84fa176c9c Merge "Avoid deprecated LinkCache::singleton()" 2018-06-14 23:48:54 +00:00
C. Scott Ananian
7de2c566dd Deprecate Language::markNoConversion, which confuses readers
Language::markNoConversion is used only within Parser.php and differs
from LanguageConverter::markNoConversion in that, contrary to its name
and its namesake, it only protects *things which look like URLs* from
language conversion.

This wasted several days of my time before I realized what was going on.
It's needless; just hoist the "looks like a URL" special casing inline
to the single place where that functionality is used.  (And I wonder
if the "looks like a URL" case is actually needed at all any more,
since most of those cases are probably free external links, which
go through a different code path, not bracketed external links.)

This is a clean-up to the clean-up that liangent performed in 2012
with e01adbfc0b.

Change-Id: I80479600f34170651732b032e8881855aa1204d8
2018-06-13 13:26:58 -04:00
C. Scott Ananian
dbda7cdfb0 Remove unnecessary Parser::getConverterLanguage() indirection
The getConverterLanguage() method was added in March 2012 in commit
561424c266 as a workaround for a regression
in mediawiki 1.19.  It was an indirection which checked the global variable
$wgBug34832TransitionalRollback to return a different converter language
for Chinese wikis.

When this temporary bugfix was reverted in January 2013 in commit
a3fbdaaa2c, the temporary global variable
was removed, but not the getConverterLanguage() indirection.  Since then,
new code in the parser seems to have faithfully used getConverterLanguage()
instead of getTargetLanguage(), even though they are identical and the
need for getConverterLanguage() has long since passed.

Strike a small blow for elegant minimalism by removing the completely
unnecessary Parser::getConverterLanguage() indirection.  Well, sort
of: since this blight has been slowly growing inside Parser.php for
so long, we need to deprecate getConverterLanguage() first just in
case any external dependency has been infected.  Next release we
can finally excise the unnecessary method.

Change-Id: I567c29c9c7699020955699b76cbe8578d02e2fe6
2018-06-12 23:33:03 +00:00
Fomafix
e1630b6a53 PHP: Use short ternary operator (?:) where possible
Change-Id: Idcc7e4fcdd4d8302ceda44bf6d294fa8c2219381
2018-06-11 11:26:35 +02:00
Kunal Mehta
c4e5a9dd97 Avoid deprecated LinkCache::singleton()
Change-Id: Ie0e5c4ef0fe6ec896378bb2433af0898655dd907
2018-06-10 23:55:11 -07:00
Max Semenik
6e956d55aa Replace call_user_func_array(), part 2
Uses new PHP 5.6 syntax like ...parameter unpacking and
calling anything looking like a callback to make the code more readable.
There are much more occurrences but this commit is intentionally limited
to an easily reviewable size.

In one occurrence, a simple conditional instead of trickery was much more readable.

This patch finishes all the easy stuf in the core, the remainder is either unobvious
or would result in smaller readability gains. It will be carefully dealt with in
further commits.

Change-Id: I79a16c48bfb98b75e5b99f2f6f4fa07b3ae02c5b
2018-06-07 20:19:26 -07:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
Bartosz Dziewoński
b191e5e860 Use PHP 7 '<=>' operator in 'sort()' callbacks
`$a <=> $b` returns `-1` if `$a` is lesser, `1` if `$b` is lesser,
and `0` if they are equal, which are exactly the values 'sort()'
callbacks are supposed to return.

It also enables the neat idiom `$a[x] <=> $b[x] ?: $a[y] <=> $b[y]`
to sort arrays of objects first by 'x', and by 'y' if they are equal.

* Replace a common pattern like `return $a < $b ? -1 : 1` with the
  new operator (and similar patterns with the variables, the numbers
  or the comparison inverted). Some of the uses were previously not
  correctly handling the variables being equal; this is now
  automatically fixed.
* Also replace `return $a - $b`, which is equivalent to `return
  $a <=> $b` if both variables are integers but less intuitive.
* (Do not replace `return strcmp( $a, $b )`. It is also equivalent
  when both variables are strings, but if any of the variables is not,
  'strcmp()' converts it to a string before comparison, which could
  give different results than '<=>', so changing this would require
  careful review and isn't worth it.)
* Also replace `return $a > $b`, which presumably sort of works most
  of the time (returns `1` if `$b` is lesser, and `0` if they are
  equal or `$a` is lesser) but is erroneous.

Change-Id: I19a3d2fc8fcdb208c10330bd7a42c4e05d7f5cf3
2018-05-30 18:05:20 -07:00
Kunal Mehta
5952d84307 Deprecate Parser::fetchFile() since it's unused
Change-Id: Ic2bc3dd0479a373159a22da5f0a6961e212352ff
2018-05-27 23:17:50 -07:00
Kunal Mehta
e2ed410492 Parser: Don't catch exception just to rethrow it
This is left over from 4ff813680.

Change-Id: I624c2c22b7736af249647997565fe06f52d40fe2
2018-05-26 18:34:43 -07:00
Aaron Schulz
ec46c46787 Reduce impact of revision day/month/year variables on edit stashing
Change-Id: I0ddebdfa8a13844ab003aad577624e89daba7d6b
2018-05-20 08:11:22 +00:00
Kunal Mehta
230958d97c Autofix MediaWiki.Commenting.FunctionComment.SpacingDoc* errors
Change-Id: I63761ebce04c03b9b13237919c27cc10180f198f
2018-05-19 14:07:03 -07:00
Arlo Breault
5970ecec86 parser: Don't unnecessarily add and remove a pipe
Change-Id: I884ab88f9e8ac6f402cd4b3a54e33ccbd30637a2
2018-05-16 16:39:48 +00:00
Umherirrender
52338150c8 Fix return type for html strings
Change-Id: Ifc1ae7740ad1b130186b4b970d3d84651b016177
2018-04-06 13:07:01 +02:00
Subramanya Sastry
87c7ccd9bc Fix whitespace trimming in headings
* b3dd3881 was trimming whitespace in wikitext as well as HTML headings
  whereas the whitespace-trimming proposal was going to leave HTML tags
  untouched.

* 30495ea1 missed this because coincidentally, the test I added there
  for HTML headings had a typo and used <h2>...<h2> instead of
  <h2>...</h2> which caused the test to magically pass.

* This patch trims whitespace in
  doHeadings (which deals with wikitext headings) instead of
  formatHeadings (which deals with all headings).

* Updated parser tests to account for this.

Change-Id: I854f20b4c39a0a8e03d70155b269de77acf02cae
2018-03-23 11:42:01 -05:00
Subramanya Sastry
30495ea1f9 RFC T157418: Trim whitespace in table cells, list items, headings
* Matmarex had implemented this for wikitext headings in b3dd3881.
* This patch extends this to wikitext list items and wikitext table cells.
* Updated RELEASE NOTES.

tests/parser/parserTests.txt:
* All whitespace removed in output of list items, table cells, and
  headings. Removed corresponding whitespace in the input wikitext
  except for a few tests where the whitespace is significant "| +"
  or "| -", for example.
* Updated output of html/parsoid sections as well.
* Added new tests to spec white-space trimming behavior.

tests/phpunit/*:
* Fixed a few tests that used whitespace in list items and table cells.

Bug: T157418
Change-Id: I8ea34c7ab893c0c125c81d810feeb3c581e4bba1
2018-03-16 13:42:55 -05:00
C. Scott Ananian
65fcb7a945 Use class="free external" only on unbracketed URLs
The ability for URLs to be marked free even if they use bracketed syntax
but "sorta look free" (aka unbracketed) was added 13 years ago in
2d71cb3080 (r7074).

It seemed like a reasonable idea at the time: make printed output a little
prettier by marking "sorta free" URLs as free.  But this complicates the
semantics of wikitext, and introduces all sorts of strange corner cases,
for example:

  [http://example.com/&amp; http://example.com/&]

isn't marked as free, even though the parser output is:

  <a rel="nofollow" class="external text" href="http://example.com/&amp;">http://example.com/&amp;</a>

This functionality isn't actually needed: if you want the pretty printed
output of an unbracketed URL, then actually use an unbracketed URL.

In recent years we're more concerned with simplifying the semantics of
wikitext and eliminating corner cases, such that the content of our wikis
can be effectively archived.  The "effectively free" URLs are low-hanging
fruit in this quest.

Change-Id: I339e8698786c60c96a37a73443cb9a04362662c4
2018-03-07 00:20:09 -05:00
Tim Starling
f0247e05bd StripState testing and cleanup
* Added StripState unit tests
* Deprecated unmaintained "half-parsed" serialization experiment
* Renamed some variables for brevity and removed unused "prefix"

Change-Id: I838d7ac7f9a2189e13d39c6939dba5d70e74a6b7
2018-03-05 16:43:58 +11:00
Tim Starling
3dfda8c155 Limit total expansion size in StripState and improve limit handling
* Add a new limit to the parser which limits the size of the output
  generated by StripState. The relevant bug shows exponential blowup in
  output size.
* Remove the $prefix parameter from the StripState constructor. Used by
  no Gerrit-hosted extensions, hard-deprecated since 1.26.
* Convert the existing unstrip recursion depth limit to a normal parser
  limit with limit report row, warning and tracking category. Provide
  the same features in the new limit.
* Add an optional $parser parameter to the StripState constructor so
  that warnings and tracking categories can be added.

Bug: T187833
Change-Id: Ie5f6081177610dc7830de4a0a40705c0c8cb82f1
2018-03-05 05:16:04 +00:00