Doxygen was unable to parse the file past validateSig().
> Parser.php:6397: warning: reached end of file while inside a ~~~ block!
> The command that should end the block seems to be missing!
Change-Id: I3d1b547968302611d2bd78a7c11dd0738b40d23a
* This makes use of the injected new revision object used elsewhere
in Parser to solve this problem.
Bug: T94407
Change-Id: I7881583cf7cb2bc799c89ffaa2a344a2d4ca3a4e
Currently, the parser adds a "_2" to the second of two identical headlines to
avoid collisions, but there's still a collision if another headline actually
ends in "_2". This change causes the new headline to also be checked for a
collision, and advances to "_3" or beyond if there is one.
Bug: T26787
Change-Id: Id0a55aa4c1917bac2f8f0d4863fcb85bd3dff1ca
* Sanitizer: dev.w3.org/html5/spec-preview
Follows-up 8e8b15afc6.
Use stable reference to www.w3.org/TR/html5 instead (currently
from October 2014) instead of an old preview branch from 2012.
* parserTests: dev.w3.org/html5
Follows-up 959aa336a1.
Url is now a dead end. Replaced with link to a draft from around
that time. The relevant section no longer exists in the curent
spec as it got split off into a separate spec. Maybe this one:
https://url.spec.whatwg.org/#percent-encoded-bytes
* Parser, HTMLIntField: dev.w3.org/html5
Use stable reference to www.w3.org/TR/html5 instead.
* HTMLFloatField.php: dev.w3.org/html5
Url is now a dead end. Draft from around that time:
http://www.w3.org/TR/2011/WD-html5-20110525/common-microsyntaxes.html#real-numbers
The section "Real numbers" no longer exists in the current spec,
but the Infrastructure chapter has a section on floating point
numbers that describes the same sequence now.
Change-Id: I7dcd49b6cd39785fb1b294e4eeaf39bda52337b2
Xhprof generates this data now. Custom profiling of various
sub-function units are kept.
Calls to profiler represented about 3% of page execution
time on Special:BlankPage (1.5% in/out); after this change
it's down to about 0.98% of page execution time.
Change-Id: Id9a1dc9d8f80bbd52e42226b724a1e1213d07af7
This continues the work started in T67278 to make magic link parsing
more consistent with wiki text parsing in general, and closes two
long-standing bugs.
Bug: T30950
Bug: T31025
Change-Id: I71f8b337543163569c64bbfdec154eb9b69d7264
Autolinking free external links is clever about making sure that trailing
punctuation isn't included in the link. But if an HTML entity happens to
terminate the URL, the semicolon from the entity is stripped from the url,
breaking it.
Fix this corner case. This also unifies autolink parsing with Parsoid.
See: I5ae8435322c78dd1df170d7a3543fff3642759b1
Change-Id: I5482782c25e12283030b0fd2150ac55092f7979b
The behavior of the different preprocessors differs when given \r or
\r\n newlines. We already normalize the latter here, so may as well do
the former here too.
Bug: T78488
Change-Id: Id6390f64a73ea01088729f25d79103388c1fe7e8
Ensure that there is a \b boundary before and after RFC, PMID, and ISBN
links. (Previously we enforced \b boundaries only before free external
links and after ISBN links.) Consistency is a good thing!
In addition:
* \b is not a PHP escape sequence, so you don't need to write \\b inside
a string.
* \b before the numeric part of an ISBN is pointless: by the structure
of the regexp there will always be a space on the left and a word
character (a digit) on the right.
Bug: 65278
Change-Id: Ic315b988091a5c7530a8285b9249804db72e55db
- Added/removed spaces around parenthesis
- Added newline in empty blocks
- Added space after switch/foreach/function
- Use tabs at begin of line
- Add newline at end of file
Change-Id: I244cdb2c333489e1020931bf4ac5266a87439f0d
* Added a standard getFunctionStats() method for Profilers to return
per function data as maps. This is not toolbar specific like getRawData().
* Cleaned up the interface of SectionProfiler::getFunctionStats() a bit.
* Removed unused cpu_sq, real_sq fields from profiler UDP output.
* Moved getTime/getInitialTime to ProfilerStandard.
Co-Authored-By: Aaron Schulz <aschulz@wikimedia.org>
Change-Id: I266ed82031a434465f64896eb327f3872fdf1db1
includes/parser/Parser.php
* Pull out a chunk of code we need to reuse from parse() to
internalParseHalfParsed(). This is a fully backwards-compatible
change.
Code changes:
* Add a guard for running ParserBeforeTidy and ParserAfterTidy
hooks, as extensions might not expect them to be called for
snippets, only full page content.
* Change $options to $this->mOptions.
The bulk of parsing work is now done in internalParse() and
internalParseHalfParsed(), parse() only handles four things:
* Resetting parser state when a parse starts/finishes
* Page title language conversion
* Outputting limit report and limitation warnings
* Running ParserAfterParse hook (dunno why, but it's documented)
* Expand documentation for recursiveTagParse(), with some uppercase
warnings so that no one does the stupid thing I did ever again.
* Add new public method recursiveTagParseFully(), which is a
recursive parser entry point that produces fully parsed HTML ready
for inclusion in HTML output. Compared to Parser::parse(), it
doesn't produce limit reports and doesn't run the ParserAfterParse
hook.
includes/parser/CoreTagHooks.php
* Use the new recursiveTagParseFully() method.
* Use Parser::stripOuterParagraph() to remove silly tags.
Bug: 72887
Change-Id: I89ae9a50b82245f9a9e4a903563aeb1c51b6103e
Breaks extensions, doesn't entirely fix the problem it was meant to fix.
This reverts commit 6da3f169ac.
Change-Id: Ic193abcff8c72b0c8b434fcac514f88603a45beb
The JIT compiler in newer versions of PCRE experiences lock contention
when multithreaded applications perform a high rate of concurrent
compilations. We are seeing some performance impact on HHVM under normal
production traffic.
The random part of the strip marker is just there to protect against
deliberate insertion of strip markers into the source text, which is
very rare. So use a generic regex to find strip markers, and check in
the callback whether the random state ID is correct.
StripState::killMarkers() will be slower when it has to remove many
strip markers, but most calls to it will not match any strip markers, so
overall performance should be improved due to reduced JIT compilation.
Bug: 72205
Change-Id: I8d37ae929a8c669c9e39adc8096b89e5732b68d0
addTrackingCategory is more in line with ParserOutput's functionality
(addLink, addCategory etc), and tracking categories are useful even for
content types which do not use the parser at all. There is no reason to
require the caller to obtain a Parser object just to be able to add
tracking categories.
Change-Id: I89d9ea1db3a4e6486e77eee940bd438f7753b776
If you have a reference *to* an object field (anywhere in the call
stack) when you clone the object, the field will be cloned as a
reference rather than as a value.
So we have to break those unexpected references in the cloned object
manually, which is easy enough by making a non-reference copy and then
rebinding the cloned object's reference to this copy.
Bug: 56226
Change-Id: I9c600e9c0845b4fde0366126ce3809d74e2240b4
Add Parser::fetchCurrentRevisionOfTitle(). By default, this just calls
Revision::newFromTitle, but a callback can be set in ParserOptions that
will override it. Anything that runs as part of a parse should use this
wherever possible.
Bug: 70495
Change-Id: I521f1f68ad819cf0f37e63240806f10c1cceef9c
The previous implementation would unescape '&', '=', '+', and '%'. The
first three will break the URL when unescaped in the query string, and
the last will break when unescaped anywhere.
The code is now changed to treat the path, query, and fragment parts of
the URL separately when unescaping. We also escape any unsafe characters
and ensure all percent-encodings use uppercase hexits.
And since the old name is no longer accurate,
Parser::replaceUnusualEscapes is deprecated in favor of
Parser::normalizeLinkUrl.
Bug: 57909
Change-Id: I77dc308d0d016c395ad737c08cf10a7711e25bbd
In Parser.php an array was built and then the elements of that array
were used, replaced this by local vars.
In ParserOutput.php also use local vars to make the code more readable.
Also inlined a private callback by using an anonymous function.
Change-Id: I1c31c9e4855f93a8fb65e1c21faba46fcdcb1f4b
This regex looked something like /^(?i)bitcoin:|ftp://|ftps://|.../, which
meant the anchoring ^ only applied to the first name. This meant that any
link= value that happened to contain a URL protocol anywhere within it
(e.g. wikinews:Foo containing "news:") got incorrectly matched by this
regex.
Bug: 69317
Change-Id: Ide1c4f64137666db99f8e3b6816df01ef5099c8e
This solution is somewhat imperfect, as the logic being added here to
MediaWikiTitleCodec really belongs in the parser. However, given the
current state of this code, this is the cleanest possible solution at
the moment.
Modified the existing release note for this.
Bug: 68802
Change-Id: I38309186bdcad23f49e23beb26daaf3ef5bceea1
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling
Change-Id: I8ebfbcea0e2ae2670553822acedde49c1aa7e98d
This was noticed on enwiki after w: was marked as a local interwiki prefix
there. Links like [[w🇩🇪Foo]] ought to act like [[🇩🇪Foo]], not
[[de:Foo]].
Also adding a number of additional parser tests related to interwiki links.
Bug: 68085
Change-Id: If39af06edb4af2da85c9bcf43df7088181809fcf
It is needed for PageImages to collect information about galleries, improving results
for Commons mainspace.
Bug: 66510
Change-Id: I3136d648ef2c1841767db0ab33855cd168e3de3e
Add {{!}} as a magic word that expands to a pipe. Parsoid already does
this, so we know it isn't going to cause major breakage.
Change-Id: I1f857417d224d6443504074a5add852df3975b89
