Commit graph

83 commits

Author SHA1 Message Date
Jackmcbarn
368aa5dc67 Make RECOVER_ORIG preserve extension tags
Add PPFrame::NO_TAGS, set by PPFrame::RECOVER_ORIG, to preserve extension
tags rather than expanding them.

Bug: 22683
Change-Id: I427333a20d32eb711a7b5d5ac8b780ef89c752a1
2014-06-19 18:12:14 +00:00
Jackmcbarn
18d15fa138 Add PPFrame::getTTL() and setTTL()
Add functions to frames to control the TTL of their output, and expose
this via expandtemplates in the API.

Bug: 49803
Change-Id: I412febf3469503bf4839fb1ef4dca098a8c79457
2014-06-09 20:40:22 +00:00
Brad Jorsch
d18ba4e9df Add PPFrame::isVolatile and PPFrame::setVolatile
Most wikitext is safe to parse once and then cache for when that same
wikitext is used again, such as for multiple transclusions of the same
template within a page. There are occasions, though, where some piece of
wikitext has side effects and so should not be cached; a prominent
example of such wikitext is the <ref> and <references> tags in Cite.php.

This change adds PPFrame::setVolatile so parser hooks such as <ref> and
<references> can indicate that they have done something that should not
be cached, and PPFrame::isVolatile so that callers of PPFrame::expand
can know when to avoid caching.

Bug: 46815
Bug: 31834
Change-Id: I95b3cf8781cf047cdb63da221cef45f3e7d1632e
2014-05-30 14:07:06 -04:00
Jackmcbarn
2094e578b4 Restrict empty-frame cache entries to their parent
Remove the parser's global $mTplExpandCache, and replace it with an
alternative that is separated by parent frame. This allows the integrity
of the empty-frame expansion cache to be maintained while also allowing
parent frame access.

A page with 3 copies of 
http://ja.wikipedia.org/wiki/%E4%B8%AD%E5%A4%AE%E7%B7%9A_(%E9%9F%93%E5%9B%BD) 
has the following statistics: Without this change, there are 4625 cache hits
on this page, and a sample of 3 parses took 16.6, 16.9, and 16.8 seconds.
With this change, there are 2588 cache hits, and a sample of 3 parses took
16.7, 16.7, and 17.0 seconds.

Change-Id: I621e9075e0f136ac188a4d2f53418b7cc957408d
2014-05-30 01:38:15 +00:00
Ori.livneh
df983f6642 Revert "Declare visibility on class properties of includes/parser/"
See https://bugzilla.wikimedia.org/65375#c4

This reverts commit f359cdf614.

Bug: 65375
Change-Id: I12a60b5cc52a07a6deabcbf47c7c99cd2faac3c3
2014-05-16 00:52:24 +00:00
jenkins-bot
3564c521a1 Merge "PPAccum_Hash -> PPDAccum_Hash" 2014-05-11 20:46:34 +00:00
Reedy
e9109eee11 PPAccum_Hash -> PPDAccum_Hash
Change-Id: I79eac91cc3ff4ca515d15d8fa08d492b5cdcaa1b
2014-05-11 17:27:21 +01:00
Siebrand Mazeland
dfc7416fbe Various documentation updates for includes/parser/
Change-Id: I16dd3a792cc83f8c80b3652d42c055730f6d177a
2014-05-11 18:18:26 +02:00
Siebrand Mazeland
2527cca6de Fix most CodeSniffer issues in includes/parser/
Remaining are the classes containing underscores and possibly a few other
issues that will be addressed soonish.

Change-Id: Icf56374c71afc134420ebbcfecf12dcb29dc9564
2014-05-11 08:44:52 +00:00
Siebrand Mazeland
f359cdf614 Declare visibility on class properties of includes/parser/
Change-Id: If03a9bd5eb83be4d15f54e73f49f42540fb7d5fc
2014-05-11 02:25:00 +02:00
umherirrender
7f9fd63901 Fixed some @params documentation (includes/parser)
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.

Change-Id: I49f8f48b521878de7abd9cc40efdeff6cf9a37e0
2014-04-22 01:38:39 +02:00
umherirrender
23fab68274 Fix spacing after @param and friends in comments
Searched for:
\@(param|return|throws|since|deprecated|access|todo|var)[ \t]{2,}

Change-Id: Icce22ba9fe0635455691ca58d9872d618151f346
2014-04-05 20:02:29 +00:00
umherirrender
047c86f26e Fix spacing between two functions
Added and removed some new lines to have one new line between two
functions

Change-Id: I1ccfbd575dd26b160396ef3d3e2e079f5cdbe196
2014-03-15 20:57:23 +00:00
Alexandre Emsenhuber
c29d513deb Put the "else" (or "elseif") on the same line as the previous closing brace
Per https://www.mediawiki.org/wiki/Manual:Coding_conventions#Indenting_and_alignment

Change-Id: I208981db0a866524156bad18cb687f010afeac2c
2014-03-15 13:54:53 +01:00
umherirrender
0bc583af2c Move closing parenthesis from multi line if and function to own line
The Line continuation Coding conventions prefers the closing parenthesis
on the same line than the beginning curly braces. This is done for ifs
and functions.
Also move some boolean operator from the end of a line to the beginning
and changed some indentation to make the condition hopefully better
readable.

Change-Id: Id0437b06bde86eb5a75bc59eefa19e7edb624426
2013-12-01 21:39:00 +01:00
umherirrender
24bfde2710 Fix spacing and break some lines
Change-Id: Ia57685d8858e02e399ad5c75ce64d12609d340ac
2013-08-24 17:06:25 +02:00
C. Scott Ananian
35422cadf2 Allow lines empty but for tabs and comments to be ignored.
We originally allowed only spaces around comments.  Now allow tabs as
well.  This ought to affect very few pages, but it helps predictability
and to maintain consistency between the PHP preprocessor and parsoid.

Change-Id: Icb3ff6eec08aaa83ae332d03c910c13995c9c9ee
2013-08-13 15:36:57 -04:00
C. Scott Ananian
f089e20bc0 Preprocessor: Don't treat a line containing multiple comments as a blank line.
After this patch, 'a', 'b', and 'c' are all treated as members of the
same list in the following wikitext:

*a
 <!--x-->
*b
 <!--x--> <!--y-->
*c

The old comment-removal rule was "trim a comment which is both
preceded and followed by a newline (ignoring spaces)".  This only works
if there is a single comment on the line, and was often surprising
to users.  The new rule allows any number of whitespace-separated
comments on the line.

Bug: 41756
Change-Id: I6030086226e1eeece59643c29dbb4361668b4bd6
2013-08-09 00:28:28 +00:00
umherirrender
6c38a5eb72 Fixed spacing in logging/parser/profiler/rl/revdel/search folder
Added spaces before if, foreach
Added some braces for one line statements

Change-Id: I11bbcfa351e945b7bde10c2105d61a3cf5622205
2013-04-20 17:38:24 +02:00
umherirrender
15abcf71ca Added/Removed spaces around string concatenation
And added/removed spaces around some other tokens,
like +, -, *, /, <, >, =, !

Fixed windows newline style

Change-Id: I0b9c8c408f3f6bfc0d685a074d7ec468fb848fc8
2013-04-13 13:36:24 +02:00
umherirrender
978bb31c5e Add missing wfProfileOut before throwing an exception
Change-Id: I1d830da0597f19efd0b2ae48642389975e736e23
2013-04-08 18:37:24 +00:00
umherirrender
6c278b6d7e fix some spacing
* Removed spaces around array index
* Removed double spaces or added spaces to begin or end of function
  calls, method signature, conditions or foreachs
* Added braces to one-line ifs
* Changed multi line conditions to one line conditions
* Realigned some arrays

Change-Id: Ia04d2a99d663b07101013c2d53b3b2e872fd9cc3
2013-03-25 22:22:46 +00:00
Tyler Anthony Romeo
4dcc7961df Fixed @param tags to conform with Doxygen format.
Doxygen expects parameter types to come before the
parameter name in @param tags. Used a quick regex
to switch everything around where possible. This
only fixes cases where a primitve variable (or a
primitive followed by other types) is the variable
type. Other cases will need to be fixed manually.

Change-Id: Ic59fd20856eb0489d70f3469a56ebce0efb3db13
2013-03-11 13:15:01 -04:00
umherirrender
d63121016d fix some spacing
Added/removed spaces around logical/arithmetic operator
Reduced multiple empty lines to one empty line
Removed wrong tabs before comments at end of line
Removed too many spaces in assigments

Change-Id: I2bba4e72f9b5f88c53324d7b70e6042f1aad8f6b
2013-03-07 17:53:21 +01:00
Siebrand Mazeland
9b7889b84b Use American English spelling for behavior
Spotted in ipbreason-dropdown by Shirayuki.

Change-Id: I576ed4bc0abe5ab980aaee3fb9f9e4b43087311f
2013-03-04 10:24:57 +01:00
Dan Collins
9846e2558c (bug 27936) Improve parser handling of nested braces.
Parser tests also included, test case and original patch supplied by
Bergi on bugzilla. Tested against the current version.

Change-Id: Id7ec4e694783dd0f682f65f39d8b9e59f82e58aa
2013-02-06 03:39:07 +00:00
umherirrender
570dda0455 fix some spacing
Change-Id: I88e73d47a552918880514d88a876296a6cb80d88
2013-02-04 20:09:18 +00:00
Siebrand Mazeland
d4b046a893 Update docs for return and exception info
* Removed some inline tabs in the process.
* IDE fixed some incorrect leading spaces, too.

Change-Id: Ic9303eff6db4424ac3f1fa2816839692b43e6190
2012-10-09 09:41:58 +00:00
Aaron Schulz
5f5ba94b06 Merge "store text length in local var in Preprocess::processToObj/Xml" 2012-08-15 00:22:10 +00:00
umherirrender
80cc857fc1 store text length in local var in Preprocess::processToObj/Xml
The $text is constant and that means, the length of $text is also
constant, store it in a local var is easy than.

Change-Id: I9631b862f40eef7f8b18559ffd474a0037077d18
2012-08-06 13:41:28 +02:00
umherirrender
91c3d5c6c5 Fix wrong return value in Preprocessor::getChildrenOfType
I am not sure, but this looks wrong, because it adds the type to the
array and not the child.
This method is unused in core and wmf extensions, maybe
removing/deprecating is a better idea, but I am not sure, if that is
possible.
I have only see this possible error, while looking through the
preprocessor.

Change-Id: I5b7492d455989a8a3e71b5db6d31091b986c502a
2012-08-05 22:14:45 +02:00
Antoine Musso
aab43dd495 escape tags and entity in doxygen comments
When inserting XML elements inline <such as this one>, doxygen chokes
about it not being known. Simply enclosing the tag in double quotes
prevents doxygen from emitting a warning.

Also enclosed a few invalid functions calls such as \. and double quoted
the HTML entities such as &foobar;

Change-Id: I4019637145e683c2bec3d17b2fd98b0c50a932f1
2012-07-10 17:08:32 +02:00
Tim Starling
7fc5234cbe Scribunto parser support
Add $indexOffset parameter to PPFrame::newChild(). This makes it
possible to use newChild() for interpreting named parameters to
invoke in Scribunto -- otherwise I would have had to duplicate its
functionality, which would have been tricky given that I wanted to
make a real frame with an expand() method. Setting $indexOffset allows
newChild() to start counting numbered parameters from somewhere other
than the first pipe character, leaving room for the Scribunto function
name.

Fixed PPCustomFrame_*::getArguments(), was missing for no apparent
reason. I didn't end up using it in Scribunto, but there's no harm in
adding it anyway.

Change-Id: I0c761aab8a7f1ae74e8d151a1346febb5c466e18
2012-05-22 13:53:41 +10:00
Reedy
2df7fae395 Add/update function level parameter documentation
Change-Id: I148b84e3b81c29c63585f3bf524fc445fd7683a7
2012-05-15 00:44:25 +01:00
umherirrender
e5614cea2c Show Highest expansion depth in limit report
With 1.20wmf2 we get a tracking category with all the problem pages,
seeing the limit for a page is a helpful information than

Change-Id: I1916e5fa6de06b923a01cf1f0ca9362287a9fd70
2012-05-04 22:44:14 +02:00
Alexandre Emsenhuber
0fc8c8e14e Added missing GPLv2 headers in some places.
Also made file/class documentation more consistent.

Change-Id: I10c077f27a2077a266a64048fa137f7b1f8e226c
2012-05-01 09:05:48 +02:00
umherirrender
069a4b2280 (bug 22887) Add warning and tracking category for preprocessor errors
I have only add things and not change the current error strings to
messages, because bug 21521 is WONTFIX

Change of Preprocessor_HipHop.php is not tested

Change-Id: I7a7243b8ba010dbb395bdbbb3e00e3217088038e
2012-04-27 16:16:35 +01:00
Christian Aistleitner
0954fc366f Sanitizing already existing use of PHP's assert 2012-03-19 11:14:43 +00:00
Sam Reed
85bbb0b080 Fixing some of the "@return true" or "@return false", need to be "@return bool" and then the metadata can say true if foo, false if bar
Other documentation improvements
2012-02-09 18:01:10 +00:00
Brion Vibber
d60063eac2 Revert r103795 -- adds <dws> pseudotag which modifies preprocessor behavior in some way to eat whitespace
Not sure whether this is something we want, but it should probably be talked over first.
2011-11-23 00:30:13 +00:00
Mark A. Hershberger
5bd4f88169 Fixes Bug 31865 - Tag <dws> for discarding whitespaces.
Patch with parser tests from Van de Bugger
2011-11-21 01:45:23 +00:00
Sam Reed
2849f384d4 Bug 31995 - Addition of PPFrame::getTitle
Applied patch by Van de Bugger

Added documentation to go with

Added code to Preprocess_HipHop too
2011-11-09 20:52:24 +00:00
Sam Reed
937fc3bcfd Back out r95396 and friends 2011-09-29 22:08:00 +00:00
Aaron Schulz
23d5a27e2e Reverted r86072, r86419 per CR. Lots of conflicts resolved here. Removes lineStart from r87346 code as well for preprocess parser tests. 2011-09-12 19:16:22 +00:00
Sam Reed
44020e254c Merge the iwtransclusion branch back into trunk
Hexmode fixed broken unit tests in revisions after last time
2011-08-24 13:03:03 +00:00
Sam Reed
183d2dab37 Back out r93533, r93531, r93530
Breaks unit tests as below, not going to be able to fix them before I disappear for the evening, so might aswell leave trunk clean

ArticleTablesTest testbug14404

Error:
ArticleTablesTest::testbug14404
Undefined offset: 0

/home/ci/cruisecontrol-bin-2.8.3/projects/mw/source/tests/phpunit/includes/ArticleTablesTest.php:31
/home/ci/cruisecontrol-bin-2.8.3/projects/mw/source/tests/phpunit/MediaWikiTestCase.php:60
/home/ci/cruisecontrol-bin-2.8.3/projects/mw/source/tests/phpunit/MediaWikiPHPUnitCommand.php:20
/home/ci/cruisecontrol-bin-2.8.3/projects/mw/source/tests/phpunit/phpunit.php:60

ParserTests testParserTest #552 - testParserTest with data set #551

Failure:
ParserTests::testParserTest with data set #551 ('RAW magic word', '{{RAW:QUERTY}}', '<p><a href="/index.php?title=Template:QUERTY&amp;action=edit&amp;redlink=1" class="new" title="Template:QUERTY (page does not exist)">Template:QUERTY</a>
</p>', '', '')
RAW magic word
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-<p><a href="/index.php?title=Template:QUERTY&amp;action=edit&amp;redlink=1" class="new" title="Template:QUERTY (page does not exist)">Template:QUERTY</a>
+<p><a href="/index.php?title=Template:RAW:QUERTY&amp;action=edit&amp;redlink=1" class="new" title="Template:RAW:QUERTY (page does not exist)">Template:RAW:QUERTY</a>
 </p>

/home/ci/cruisecontrol-bin-2.8.3/projects/mw/source/tests/phpunit/includes/parser/NewParserTest.php:545
/home/ci/cruisecontrol-bin-2.8.3/projects/mw/source/tests/phpunit/MediaWikiTestCase.php:60
/home/ci/cruisecontrol-bin-2.8.3/projects/mw/source/tests/phpunit/MediaWikiPHPUnitCommand.php:20
/home/ci/cruisecontrol-bin-2.8.3/projects/mw/source/tests/phpunit/phpunit.php:60
2011-07-30 15:56:54 +00:00
Sam Reed
dc9d39887e Merge iwtransclusion branch into trunk 2011-07-30 15:30:01 +00:00
Brion Vibber
b9995d5583 Provisional revert of r89648: "Another try at fixing bug 93 "tilde signatures inside nowiki tags sometimes get expanded (<includeonly><nowiki>~~~~</nowiki></includeonly>)""
This seems to disable PST for <includeonly>...</includeonly> in addition to the <includeonly><nowiki>...</nowiki></includeonly> that the bug is about.
This was caught by a parser test case, but the test case was changed to match... seems like the wrong change?

Also failed to update Preprocessor_HipHop.hphp to match the other two preprocessor classes.
2011-06-07 17:34:20 +00:00
Paul Copperman
27b5141563 Another try at fixing bug 93 "tilde signatures inside nowiki tags sometimes get expanded (<includeonly><nowiki>~~~~</nowiki></includeonly>)"
* Change the preprocessor to insert strip items for <ignore> nodes during pre-save-transform, just like <comment> nodes are handled already. This effectively disables all pre-save-transform steps inside <includeonly> tags.
* Adapt parser tests to the new behavior.
2011-06-07 15:12:26 +00:00
Brion Vibber
96a947fc7a Revert r88997: fix for bug 93 caused some additional problems.
Follow-up to r89191: disable test case for bug 93 as it currently fails (bug left open).
2011-06-02 00:32:43 +00:00