Commit graph

1562 commits

Author SHA1 Message Date
WMDE-Fisch
6df9ed1ad6 update mediawiki-codesniffer to 0.11.0 and fix issues
- mostly auto fixes
- some too long lines fixed
- ignore amp space in one case  passing by reference

Change-Id: I6472f83bc3cbf4bd629d83050cc3319b19ec465c
2017-08-11 22:27:51 +02:00
Umherirrender
ace44e2064 Use correct variable name in @param documentation
For some varargs a variable name is added with suffix ,... as seen for
many other varargs

Some @param are swapped, because there are in the wrong order

Enable Sniff MediaWiki.Commenting.FunctionComment.ParamNameNoMatch

Change-Id: I60fec6025bce824d5c67563ab7b65ad6cd628ad8
2017-08-11 19:27:19 +02:00
Umherirrender
a9007e8baf Add missing & to @param documentation to match functon call
Change-Id: I81e68310abcbc59964b22e0e74842d509f6b1fb9
2017-08-11 18:47:46 +02:00
Max Semenik
fd6e9ef2d4 Human-readable section ID support
It adds the ability to replace the current section ID escaping
schema (.C0.DE) with a HTML5-compliant escaping schema that is
displayed as Unicode in many modern browsers.

See the linked bug for discussion of various options that were
considered before the implementation. A few remarks:
* Because Sanitizer::escapeId() is used in a bunch of places without
  escaping, I'm deprecating it without altering its behavior.
* The bug described in comments for Parser::guessLegacySectionNameFromWikiText()
  is still there in some Edge versions that display mojibake.

Bug: T152540
Change-Id: Id304010a0342efbb7ef2d56c5b8b244f2e4fb2c5
2017-08-01 20:32:20 -07:00
Kunal Mehta
d1cf48a397 build: Update mediawiki/mediawiki-codesniffer to 0.10.1
And auto-fix all errors.

The `<exclude-pattern>` stanzas are now included in the default ruleset
and don't need to be repeated.

Change-Id: I928af549dc88ac2c6cb82058f64c7c7f3111598a
2017-07-22 18:24:09 -07:00
jenkins-bot
e72303c9f3 Merge "Remove auto-generated "Constructor" documentation on constructors" 2017-07-21 13:19:44 +00:00
Thiemo Mättig
91a920fd85 Remove auto-generated "Constructor" documentation on constructors
Having such comments is worse than not having them. They add zero
information. But you must read the text to understand there is
nothing you don't already know from the class and the method name.

This is similar to I994d11e. Even more trivial, because this here is
about comments that don't say anything but "constructor".

Change-Id: I474dcdb5997bea3aafd11c0760ee072dfaff124c
2017-07-21 12:19:30 +02:00
jenkins-bot
3b5b239e85 Merge "Make multiple colons escaping interlanguage links invalid, consistently" 2017-07-11 15:19:17 +00:00
Aaron Schulz
7ff8529984 Avoid high edit stash TTLs when a user signature was used
This adds a new ParserOuput user-signature tracking flag.

Bug: T84843
Change-Id: I77de05849c15e17ee2b9b31b34172f4b6a49a38e
2017-07-06 16:34:26 -07:00
Arlo Breault
0e1b52a40e Make multiple colons escaping interlanguage links invalid, consistently
* Right now, one or two are permitted.  This patch limits it to one.
   The current behaviour seems more a byproduct of refactoring than an
   explicit goal.

 * Note that this will break links on a handful of pages surfaced in
   Parsoid's roundtrip testing.

Change-Id: Icabd34bbf15781bb891bd8e0c079d1a65eb28595
2017-07-06 17:09:25 -04:00
Kunal Mehta
399adec9ad Turn ParserCache into a service, deprecate $parserMemc
ParserCache is already a singleton, making it a good candidate for a
service. $parserMemc is an odd global (it lacks the "wg" prefix) and is
ripe for deprecation.

The following are now deprecated:
* $parserMemc global
* ParserCache::singleton()
* wfGetParserCacheStorage()

A ParserCache::getCacheStorage() method was added for cases where direct
access to the underlying BagOStuff object is necessary.

Usage of $parserMemc will emit deprecation warnings through the
DeprecatedGlobal class mechanism. All usage in core was migrated.

Also take this opportunity to inject the $wgCacheEpoch global value into
ParserCache. This will require an update to the FlaggedRevs extension.

Change-Id: I2ac7afff0d8522214329248c3d1cdccd0f72bbd4
2017-07-05 19:56:49 -07:00
Brad Jorsch
84694a9d59 Remove ParserOptions::legacyOptions() and cleanup related code
ParserOptions::legacyOptions() has been sitting around since 1.17.
Originally it seems to have been intended as a way to avoid a mass cache
invalidation (similar to optionsHashPre30() from I7fb9ffca9). That code
was mostly removed in 1.23, but legacyOptions() was left behind because
it was also being used in a few places as "all cache-varying options"
(despite it not being documented for that purpose) where we'd rather
have any key than no key at all.

This patch creates an actual ParserOptions::allCacheVaryingOptions()
method for those use cases and deprecates the long-obsolete
legacyOptions().

It also makes more explicit the use of the "all cache-varying options"
fallback in ParserCache::getKey(), and doesn't bother trying to use that
fallback in ParserCache::get() where it no longer makes sense.

Change-Id: Ife1e54744155136a570210c03fe907f18f8e8ece
2017-07-04 01:28:57 +00:00
Umherirrender
b5cddfb27b Remove empty lines at begin of function, if, foreach, switch
Organize phpcs.xml a bit

Change-Id: Ifb767729b481b4b686e6d6444cf48b1f580cc478
2017-07-01 11:34:16 +00:00
Brad Jorsch
27fd0920a1 Remove ParserOptions::optionsHashPre30()
The pre-1.30 version of ParserOptions::optionsHash() was kept
temporarily as ParserOptions::optionsHashPre30() to prevent a cache
stampede on WMF sites when the hash format was changed in I7fb9ffca9.

Now that the cache has been rebuilt, it's no longer needed and we should
clean it up instead of leaving it forever to bitrot.

Change-Id: I037d8dfdefe72a295547bd331bc1454e69cb418d
2017-06-28 00:18:59 +00:00
Umherirrender
be42e09aa8 build: Prepare for mediawiki/mediawiki-codesniffer to 0.9.0
The used phpcs has a bug, so the version 0.9.0 could not be enforced at the moment.
Will be fixed in next version, see T167168

Changed:
- Remove duplicate newline at end of file
- Add space between function and ( for closures
- and -> &&, or -> ||

Change-Id: I4172fb08861729bccd55aecbd07e029e2638d311
2017-06-26 17:14:31 +00:00
jenkins-bot
3f849d695f Merge "Parser: Emit deprecation warnings for ParserLimitReport hook" 2017-06-26 15:05:22 +00:00
jenkins-bot
325e344774 Merge "Parser: Ensure function and tag hook callbacks are callable upon registration" 2017-06-25 10:24:58 +00:00
daniel
5d528f39d8 Parser: Ensure function and tag hook callbacks are callable upon registration
This is to safeguard against issues like T167238, where the callback was invalid,
but was only noticed when the magic word was used in an article.

In general, type checks should be applied when things get
registered for later use. Fail early, stay sane.

Change-Id: Ifb7005ee214829c98cec534261c0db7d13f50f35
2017-06-25 10:11:12 +00:00
Kunal Mehta
613ae1c3f2 Parser: Emit deprecation warnings for ParserLimitReport hook
Change-Id: I351f0867132acf8d5cc9ee51b575744445b28b77
2017-06-23 17:37:31 -07:00
jenkins-bot
3cb14f56bd Merge "Show dimensions in TraditionalImageGallery" 2017-06-15 17:16:13 +00:00
Brad Jorsch
da43a0ae34 ParserCache: Delete old-style key when saving
It was noticed that disk usage on the parser cache machines was
increasing since shortly after wmf.4 was redeployed everywhere on the
9th. One theory is that I7fb9ffca9 causes this by making reparses for an
existing old-style cache entry start writing the new-style key where
they would previously have overwritten the old-style key. On that
theory, let's delete that old-style key (that should now be useless) on
save.

I'm assuming here that firing a blind delete for keys that probably
don't exist in the cache (i.e. every new edit) isn't going to hurt
anything. If that's not the case, we'd need to check existence before
deleting.

Bug: T167784
Change-Id: Ie5efb05722cb7da2a90da195a1f244468177175d
2017-06-14 13:42:36 +00:00
Brad Jorsch
46c0c39514 ParserOptions: Fix handling of 'editsection'
The handling of the 'editsection' option prior to I7fb9ffca9 was
unusual: it was included in the cache key, but the getter didn't ever
flag it as "used". This was overlooked in I7fb9ffca9.

This fixes the handling to restore that behavior. It's no longer
considered to be a real parser option, so changing it won't make
isSafeToCache() fail while reading it won't flag it as 'used'.

But to keep Wikibase working (see T85252), if 'editsection' is supplied
in $forOptions optionsHash() will still include it in the hash so
whatever Wikibase is doing by forcing that doesn't break. The hash when
it is included is the same as was used in I7fb9ffca9 to reuse keys.

Once optionsHashPre30() is removed, Wikibase should be changed to use
some other method to fix T85252 so we can remove that hack from
optionsHash().

Change-Id: I77b5519c5a1122a1fafbfc523b77b2268c0efeb1
2017-06-14 04:52:36 +00:00
Brad Jorsch
0facbe3e3d Try harder to avoid parser cache pollution
* ParserOptions is reorganized so it knows all the options and their
  defaults, and can report whether the non-key options are at their
  defaults.
* Definition of the "canonical" ParserOptions (which is unfortunately
  different from the "default" ParserOptions) is moved from
  ContentHandler to ParserOptions.
* WikiPage uses this to throw an exception if it's asked to cache
  with options that aren't used in the cache key.
* ParserCache gets some temporary code to try to avoid a massive cache
  stampede on upgrade.

Bug: T110269
Change-Id: I7fb9ffca96e6bd04db44d2d5f2509ec96ad9371f
Depends-On: I4070a8f51927121f690469716625db4a1064dea5
2017-06-05 14:17:28 +00:00
jenkins-bot
c5fa9b4e64 Merge "Parser: Better debugging of lock errors ("Did you call Parser::parse recursively?")" 2017-05-30 19:01:45 +00:00
Kunal Mehta
ff8a0c788b parser: Avoid deprecated wfMemcKey()
Tested that parser cache keys stay the same, before and after this
change.

Also use the more obvious ObjectCache::getLocalClusterInstance() instead
of looking up the main cache type in config and using
ObjectCache::getInstance().

Change-Id: Icef646b3c05e732ef4079d6900e6bce111debf2b
2017-05-25 12:05:49 -07:00
jenkins-bot
5f764ec180 Merge "Move loading of mediawiki.toc from Parser to Skin" 2017-05-23 18:42:44 +00:00
C. Scott Ananian
186a182a15 Protect language converter markup in the preprocessor (take 2).
This revises 2877402276, which was
reverted in master due to unexpected issues with `-{{...}} ` markup
on translatewiki and enwiki.  Test cases are added to ensure that this
is parsed as a template, not as language converter markup.

https://www.mediawiki.org/wiki/Preprocessor_ABNF is the canonical
documentation for the preprocessor; this will be updated after this
patch is merged.  The basic principles described in that page are
maintained in this patch:

* Rightmost opening structure has precedence: `-{{` is parsed as a
dash followed by template opening.

* `{{{` has precedence over `{{` and `-{`: `-{{{{` is parsed as
`-{` `{{{` since we first grab the rightmost `{{{`.

A bunch of test cases were added to verify the "ideal precedence"
order described on that wiki page.

This patch introduced some minor incompatibilities in existing
markup, in particular with chemical formulae in templates.
Fixes for these are being tracked at
https://www.mediawiki.org/wiki/Parsoid/Language_conversion/Preprocessor_fixups

Bug: T146304
Bug: T153761
Change-Id: I2f0c186c75e392c95e1a3d89266cae2586349150
2017-05-23 15:43:49 +01:00
Bartosz Dziewoński
baab085b32 Parser: Better debugging of lock errors ("Did you call Parser::parse recursively?")
Save the backtrace when locking, so that if some code tries locking again,
we can print the lock owner's backtrace for easier debugging.

Change-Id: I6e352b4aa5e7cb35825a66592f6c066d9e8b95c9
2017-05-23 14:50:13 +02:00
Timo Tijhof
c7e00974c7 Move loading of mediawiki.toc from Parser to Skin
This was the only addModules() call ever to be inside Parser.
Introduced in a54ef1a203. Prior to that, mediawiki.toc had always been loaded
by OutputPage (via mediawiki.util; and before that, via wikibits).

This patch restores that, and also fixes T130632 by making OutputPage get
it from the Skin, instead of hardcoding this somewhere in addParserOutput().

* Remove deprecated method OutputPage::enableTOC().
* Move mEnableTOC to addParserOutputText().

Bug: T130632
Change-Id: Iaad84d241a4c4348c712ac1087a664b8c9c46da4
2017-05-21 19:06:43 +02:00
Brion Vibber
33e4ac5b22 Add \b to regexes in BlockLevelPass to avoid confusing tr & track
With TimedMediaHandler in video.js mode, videos can be inline,
without a wrapper div.

Previously, in this mode two paragraphs where one contained a
video would end up merged into one paragraph, due to BlockLevelPass
matching "<track .../>" against "<tr" in its regexes.

Added \b to a couple of the regexes to protect against such errors,
and corrected a parser test case that had bad output listed, where
"<link .../>" matched against "<li".

Bug: T165817
Change-Id: I06e82b881f5ebddae5e7df7fb940adfa54f6b659
2017-05-20 00:53:05 +02:00
Brad Jorsch
ecb4c0e3fe ParserOptions: Include wrapping class in options hash
Avoids polluting the cache when things take advantage of the option.

Bug: T165115
Bug: T165161
Change-Id: I5be25c6de68012df58b6a0cbf92e2f972be2b68a
2017-05-15 05:55:51 +00:00
Brad Jorsch
1aac0a2992 Wrap parser output in <div class="mw-parser-output">
This will allow CSS to target just the parser output, without also
accidentally targeting the edit form, diff tables, and so on.

Bug: T37247
Change-Id: If4eb5bf71f94fa366ec4eddb6964e8f4df6b824a
Depends-On: I330c6aa4aaee045614b1801ed34bc9e03be69650
Depends-On: I52a518fa44e017841fe78474012cd69823e0a41d
2017-05-08 05:32:03 +00:00
Paladox
54c56da85a Fix php code style
Preparation change for updating mediawiki code sniffer to 0.8.0

Change-Id: Ib0b3fe4afea9096ffa3a1347b4f7e07d3398b0b2
2017-05-05 12:03:54 +00:00
Umherirrender
84132a2613 Remove unused var assign in Parser::getTemplateDom
Change-Id: If11d7a2568d4235df6888e4001500bdf45f58eae
2017-05-02 21:18:54 +02:00
Fomafix
5c41b29993 Use isSpecialPage() where possible
Change-Id: Ie4d0838acf96a7ed4a1fe4cfdc901c77d3312174
2017-04-29 22:31:42 +02:00
Tim Starling
448be2ed3e Add benchmarkTidy.php, to benchmark tidy drivers
Plus representative input file

Change-Id: I254793fc55c57a98c07ae1e4c27e6005965c9a20
2017-04-21 01:02:22 +00:00
Umherirrender
bf51e3542b Add grep infos to Parser::getImageParams
Comments for grep makes searching easier

Change-Id: I98e93baf6bd89df36185d535d6e63c51c6f65bc9
2017-04-14 23:42:15 +02:00
Brian Wolff
17e7bc0235 SECURITY: Always normalize link url before adding to ParserOutput
Move link normalization directly into addExternalLink() method,
since you always need to do it - having it separate is just
inviting people to forget to normalize a link.

Additionally, links weren't properly registered for <gallery>.
This was somewhat unnoticed, as the call to recursiveTagParse()
would register free links, but it wouldn't work for example with
protocol relative links.

Issue originally reported by MZMcBride.

Bug: T48143
Change-Id: I557fb3b433ef9d618097b6ba4eacc6bada250ca2
2017-04-06 13:44:44 -07:00
jenkins-bot
4f43a9608f Merge "Valid tags on a gallery should correspond to unordered list" 2017-04-03 18:25:37 +00:00
Arlo Breault
aea349488c Valid tags on a gallery should correspond to unordered list
* This was introduced in 4d3446a8e3 when galleries were tables.
   However, in 05579cf0e6, it switched to ul's, but missed updating the
   sanitization.

 * As an example, the test shows that summary is currently wrongly
   permitted.

Change-Id: I8c52477dc65499d0c8a1ee5cc661a5f9ae78cc07
2017-04-01 09:59:21 -04:00
Brian Wolff
1c7889446d SECURITY: Disable <html> tag on system messages despite $wgRawHtml = true;
System messages may take parameters from untrusted sources. This
may include taking parameters from urls given by unauthenticated
users even if the wiki is a read-only wiki. Allowing <html> tags
in such a context seems like an accident waiting to happen.

Bug: T156184
Change-Id: I661f482986d319cf41da1d3e7b20a0f028a42e90
2017-03-28 21:51:44 +00:00
Matěj Suchánek
ec25c79139 Add a tracking category when a template loop is detected
Bug: T160743
Change-Id: Ib888634af281fc2347eaa389db4141782a98c15c
2017-03-17 11:52:38 +00:00
jenkins-bot
980c688c2b Merge "Add RemexHtml to the list of available Tidy drivers" 2017-03-08 23:33:17 +00:00
Tim Starling
50fe941457 Add RemexHtml to the list of available Tidy drivers
Change-Id: I5a87a6ed24ca3ef7c5fdb21e74f9eb410bf74b4c
2017-03-09 10:19:23 +11:00
Matthias Mullie
ebb1680359 Show dimensions in TraditionalImageGallery
Bug: T121869
Change-Id: Ie2cb3f1594302f1726ae3d9d2d668c81b7e6b0f1
2017-03-07 13:09:00 +01:00
C. Scott Ananian
3e32d21210 Strip U+0000 in wikitext
U+0000 is not allowed in HTML5, there's no reason to allow it in wikitext.

It simplifies our code if we can just strip them at the start.  Strip in
PST as well so they don't sneak into our database either.

Tweaked the EXT_LINK URLs to account for the fact that invalid characters
get transformed into U+FFFD when using Preprocessor_DOM.  See 73649741ed
(r65967) for context on that change.

Bug: T159174
Change-Id: I3f67e92b61aacc87a40c3662085c84d1dac08bfb
2017-03-06 22:23:38 +00:00
jenkins-bot
aa3319c4c0 Merge "Miscellaneous indentation tweaks" 2017-02-28 18:38:36 +00:00
Bartosz Dziewoński
ecdef925bb Miscellaneous indentation tweaks
I was bored. What? Don't look at me that way.

I mostly targetted mixed tabs and spaces, but others were not spared.
Note that some of the whitespace changes are inside HTML output,
extended regexps or SQL snippets.

Change-Id: Ie206cc946459f6befcfc2d520e35ad3ea3c0f1e0
2017-02-27 19:23:54 +01:00
James D. Forrester
9635dda73a includes: Replace implicit Bugzilla bug numbers with Phab ones
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.

Change-Id: I6f59febaf8fc96e80f8cfc11f4356283f461142a
2017-02-21 18:13:24 +00:00
Eddie Greiner-Petter
635040c91d Fixed documentation on Parser::getVariableValue
$index is definitely not a int here, see the big switch( $index )-case
statement below. It switches for strings, not numbers. Also, note that
this is lowercase, one might expect it to be uppercase as this is how
magic words are written in wikitext.

Bug: T96633
Change-Id: Iea93c3796fdee4ed7abbb7608e89b627ca95aead
2017-02-18 22:41:12 +00:00