Commit graph

993 commits

Author SHA1 Message Date
DannyS712
7c74e448aa Revert "Deprecate parserTests format v1"
This reverts commit a7242a2eaa.

Reason for revert: Not all extension tests included in gated jobs
have been updated (specifically Scribunto and SyntaxHighlight tests)

Bug: T174199
Bug: T249194
Change-Id: I02997eda14e392e042df02dc8f64912bd4a54320
2020-04-02 08:17:08 +00:00
C. Scott Ananian
a7242a2eaa Deprecate parserTests format v1
Parser tests should be tidy by default; we will
shortly stop running untidy tests.

Bug: T174199
Depends-On: I7c8a08c4e6eff7caf4539a26fae475a4133f9a0c
Depends-On: I9c8acb05fdf2c79f618f0f317d511df77f26ec3b
Depends-On: Ibb23da480ac25cdee2ab24492e6eaebb51851bf9
Depends-On: Ifb13838c32748b1d52612da0cdb0e0a44c6e0689
Depends-On: I663c3919e08688354d68ced6945f6eb40265d2ac
Depends-On: I5f6b82d24754e438a4df9e2b1f6b4c9d6eca1be9
Depends-On: I746ee2b985537dc6bbb8a114e21cd11a683e0f74
Change-Id: Id78d04a56f83d8c554cb0a8e395fcaf75a7a9a06
2020-04-01 21:21:44 +00:00
C. Scott Ananian
65dd29ea12 Update extraParserTests.txt to parserTests v2 (tidy by default)
Bug: T174199
Change-Id: I8b4f6865772176667d3072f5a87c26885850194f
2020-04-01 13:15:32 -04:00
Ammar Abdulhamid
61ff46196e Use class keyword for classname resolution
Change-Id: I149abf837c6bd79f05648b1acbfaf45dc900125e
2020-03-27 09:33:01 +01:00
C. Scott Ananian
0030cb525b Sync up with Parsoid parserTests.txt
This now aligns with Parsoid commit a9a4390b71060edbeeb7fc80a7d7696ebabac1c6

Change-Id: I56587b7b1d58124501937e33178d4c0346311bf2
2020-03-19 10:57:36 -04:00
Brian Wolff
b186b20d9f Allow users to set tabindex="0" on elements
Important for keyboard focusability of elements in order to ensure for 
example users with motoric impairments to reach those elements.
This patch does not allow setting tabindex="-1" or tabindex > 0.
tabindex > 1 seems like a terrible idea to allow users to do.
I don't see any valid reason for tabindex="-1" in wikitext, so
lets not allow that for now either.

Bug: T247910
Change-Id: I5065b2deeb14bdb3682dd176b87f254ac6f2cf88
2020-03-18 01:17:30 +00:00
jenkins-bot
c6fab7b740 Merge "Use MediaWikiServices::getMessageCache" 2020-03-14 18:27:12 +00:00
Umherirrender
3161311c5a Use MediaWikiServices::getMessageCache
Change-Id: I07fcc9529991adc634c10e5ed8498ac138a1c2b7
2020-03-14 14:25:03 +01:00
Umherirrender
878330ae30 Use MediaWikiServices::getRepoGroup
Change-Id: Ibcef425c2e0c95d6d2f98b8d889f85ebcb33f20a
2020-03-14 13:16:12 +01:00
C. Scott Ananian
0c04f57be8 Sync up with Parsoid parserTests.txt
This now aligns with Parsoid commit 4158f68529527e60303f705cea16ea478c57a193

Change-Id: Iac5479118be2a1d31418e7183b95c135bde9cf1f
2020-03-06 12:24:16 -05:00
Brian Wolff
0bdce21381 Make id attributes not include ascii whitespace per spec
HTML5 says id attributes should not have whitespace, where
whitespace is defined as LF, CR, FF, TAB or SPACE (oddly enough
VT does not count). Firefox in my testing actually was fine with
these except CR. Nonetheless we should follow the spec, so this converts
these whitespace characters to _. I don't think this will
cause any back-compat issues, since its very hard to make these
characters in wikitext (other than space which was already
being converted) and basically requires either Lua or html entities
to make these (with FF seeming to be impossible).

Bug: T238385
Depends-On: Ie6fa40798f06a358f6082110b4d8cc0028c80321
Change-Id: Ie2b7c9429691e2c491c3359d5b400d8f078aa789
2020-02-25 05:27:33 -08:00
Brian Wolff
28d44262aa Escape % sign if form valid percent-encoding in fragment identifiers
Currently if you combine a valid percent encoding and a non
escaped character that is reserved in urls in a headline, the toc
link does not work. E.g. ==`%41== needs #`%2541 but we currently
generate #`%41 which matches ==`A== instead.

Tested in firefox and chrome

Bug: T238385
Change-Id: Ice2bbf79bed612d488ed6feb7510035e9dfb33af
2020-02-15 02:54:32 -08:00
Matt Fitzpatrick
8a1f19d726 Add ARIA navigation role and label to table of contents div
Adds accessibility context to the automatically generated table
of contents div for a page. The div's role is navigation, and
its label is the heading "Contents".

JAWS, for example, can navigate to the table of contents by
region, and announces it as "Contents navigation region".

Bug: T139221
Change-Id: Ie9b291301f683d2be2cb2134f9d6ae3f7a6d831b
2020-02-05 16:24:46 -08:00
Peter Ovchyn
baa0e2a425 languages: Decrease visibility of public variables in LanguageConverter class
Bug: T243461
Change-Id: I461ce7daba3a6a85464a69f4de76b1740472702d
2020-02-04 16:53:15 +02:00
Peter Ovchyn
61e0908fa2 languages: Introduce LanguageConverterFactory
Done:
* Replace LanguageConverter::newConverter by LanguageConverterFactory::getLanguageConverter
* Remove LanguageConverter::newConverter from all subclasses
* Add LanguageConverterFactory integration tests which covers all languages by their code.
* Caching of LanguageConverters in factory
* Make all tests running (hope that's would be enough)
* Uncomment  the deprecated functions.
* Rename FakeConverter to TrivialLanguageConverter
* Create ILanguageConverter to have shared ancestor
* Make the LanguageConverter class abstract.
* Create table with mapping between lang code and converter instead of using name convention
* ILanguageConverter @internal
* Clean up code

Change-Id: I0e4d77de0f44e18c19956a1ffd69d30e63cf51bf
Bug: T226833, T243332
2020-02-03 11:38:03 +02:00
C. Scott Ananian
f5bf2f23b7 Parser: remove leftover "transparent hook" code
Followup-To: I8384f47bbcde7604efa41b83db0dbc4f429219a8
Change-Id: I6da8cefad8bc70631277f9b7bd5d1af5474259ab
2020-01-28 20:10:20 -05:00
C. Scott Ananian
19bb8ac172 Add parserTests file line number to failing tests output
Change-Id: I57d8aa889ce7dbb3156c6635fb324516bb936257
2020-01-29 00:17:36 +00:00
C. Scott Ananian
12c70a513c Sync up with Parsoid parserTests.txt
This now aligns with Parsoid commit 9c52e78989bed9c4b87997ea9764f340010d9510

Change-Id: I14a618e45004dc46d822d7784303cfb02b691708
2020-01-28 17:35:39 -05:00
C. Scott Ananian
897e073735 Update parserTests to v2 format (default html/php tests to tidy)
The following were the most common cleanups required by switching to tidy:
* Inserting <tbody> in <table> output
* Inserting a space before the closing `/>` in <br>/<link>/<img>/<input> tags
* Entity encoding &gt; and &nbsp; in the output
* Adding `class="mw-empty-elt"` to empty elements in the output, and
  preserving some empty elements which the untidy mode's hacky "regex tidy"
  would have removed
* Fix `title` attributes in section edit links (seems to have been a bug
  in the non-tidy path, but no one noticed because that's not run in
  production)

All tests now have a `php+tidy` clause.  I've left some `php+untidy`
clauses in place to aid review; it makes it easier to see what tidy
did to the output.  Eventually all the untidy clauses will be removed.

Bug: T174199
Bug: T198214
Change-Id: I40363339c615d915a5519650315c1a2682fb9786
2020-01-28 21:33:05 +00:00
jenkins-bot
5e462e5efe Merge "Remove all traces of preprocessorClass from $wgParserConf" 2020-01-26 21:48:23 +00:00
jenkins-bot
03d44b77ad Merge "Support html+untidy tests and version flag in parserTests file" 2020-01-26 20:34:24 +00:00
C. Scott Ananian
0eb9f18d8d Remove all traces of preprocessorClass from $wgParserConf
Bug: T204945
Followup-To: I727f003f9a42d0c92bcbcce8a8289d5af6cd1298
Change-Id: I23821686ad666728d7d196e089f40be4b76b7390
2020-01-25 13:15:16 -05:00
C. Scott Ananian
5cbb64f56a Remove Preprocessor_DOM, deprecated in 1.34
Remove the deprecated Preprocessor_DOM class, which was hard-deprecated
in 1.34.  This begins to simplify parser configuration and reduce redundant
code paths, but I've left two things for cleanup in a future patch:

1. The `preprocessorClass` configuration option to the parser, exposed
in `$wgParserConf`, ServiceWiring, ParserFactory, etc.  There is no reason
for this to be exposed as configurable, but I've left this clean up to a
future patch.

2. The `$wgMaxGeneratedPPNodeCount` configuration, exposed also in
ParserOptions.  Only Preprocessor_DOM calculated this count, and since
we are only using Preprocessor_Hash now, this configuration has no effect.
But since this value was exposed in ParserOptions and elsewhere, I've
deprecated where needed but left this clean up to a future patch.

Bug: T204945
Change-Id: I727f003f9a42d0c92bcbcce8a8289d5af6cd1298
2020-01-25 11:22:45 -05:00
C. Scott Ananian
e7281ada17 Support html+untidy tests and version flag in parserTests file
This is step 1 of deprecating and then removing tests without a "tidy"
section.  We're first adding support for marking test clauses explicitly
as 'untidy'.  Then in a subsequent patch we're going to change the
default for otherwise-unmarked tests from 'untidy' to 'tidy'.

Bug: T174199
Bug: T198214
Change-Id: I1e2cbbd46282cb618178662ceaf8b1c36490d546
2020-01-22 21:24:46 -05:00
C. Scott Ananian
3f9cd0e097 Sync up with Parsoid parserTests.txt
This now aligns with Parsoid commit 815c48ec950e0494217badc4852d491e2572043e

Change-Id: I5eb99e9779d08ce9add8f310cab44a2400f34e77
2020-01-15 13:54:18 -05:00
James D. Forrester
0958a0bce4 Coding style: Auto-fix MediaWiki.Usage.IsNull.IsNull
Change-Id: I90cfe8366c0245c9c67e598d17800684897a4e27
2020-01-10 14:17:13 -08:00
James D. Forrester
4f2d1efdda Coding style: Auto-fix MediaWiki.Classes.UnsortedUseStatements.UnsortedUse
Change-Id: I94a0ae83c65e8ee419bbd1ae1e86ab21ed4d8210
2020-01-10 09:32:25 -08:00
James D. Forrester
7898e4c6db Coding style: Auto-fix MediaWiki.Comment.FunctionComment.*
Change-Id: Ia1b9b527f177d96c8b4a2dd658921b8557076f4a
2020-01-10 10:39:00 +00:00
Subramanya Sastry
f8f31ba170 Sync up with Parsoid parserTests.txt
This now aligns with Parsoid commit ca588b2507926531d1b9c3d4936d641893ec0084

However, "Links 16. Link prefix using unicode \x{...} escapes (T235563)"
was altered (s/1923/Nineteentwentythree/) to account for Postgres not
being able to handle a page_title as an int.

Bug: T239645
Change-Id: I5c4a45d535965353f6b7ef33bfd0b858c91e36a3
2019-12-02 13:58:15 -05:00
Tim Starling
164a3ac1f0 Remove IE 6 security features from server-side code
* Deprecate WebRequest::checkUrlExtension() and have it always return
  true. This reverts the security fixes made for T30235.
* Remove IEUrlExtension. This is a helper for checkUrlExtension() which
  is not used in any extensions.
* Remove CSS sanitization code which is specific to IE6. This reverts
  the changes made to fix T57332, and related followups. I confirmed
  that the relevant test cases do not result in XSS on IE8.
* Remove related tests.

Bug: T232563
Change-Id: I7318ea4a63210252ebc64968691d4f62d79a63e9
2019-11-28 15:11:56 +11:00
jenkins-bot
a36f831054 Merge "Rewrite confusing loop in TestFileEditor helper class" 2019-11-21 03:53:00 +00:00
Thiemo Kreuz
a4b1014ef3 tests: Don't expect the Parser to be passed by ref
Change-Id: I6a64428e729d8675cec1978795fc2eef455432ec
2019-11-18 17:43:35 +01:00
Umherirrender
97ed35e2e8 Add missing @param and @return to documentation
Change-Id: I48ee838a7ebf4f25a4883d4c7886b39c2d3916af
2019-11-16 13:45:55 +01:00
Thiemo Kreuz
a0770d278b Rewrite confusing loop in TestFileEditor helper class
I had to write a little test to understand what this loop is supposed
to do: All it really does is counting the number of consecutive empty
lines in the $this->lines array, starting at the offset $this->pos.

The only edge-case is when the array does not contain anything but empty
lines. In this case, the last empty line is skipped. This is because
explode( …, "\n\n\n" ) returns 4 elements, but we want to count the
number of newline characters, which is 3.

Change-Id: I74ae1ed6ba0a8d65a92a1dd07f681fd7ae4221b4
2019-11-11 10:29:18 +01:00
Max Semenik
036b4add6f Fix ParserTestRunner::checkSetupDone()
Not only the logic was wrong, it also completely failed to include
the second parameter both in doc comment and exception message,
leading to a lot of confusion.

Change-Id: Icc585214e94c82142ac585c748d1038769cd4db2
2019-11-02 22:26:15 -07:00
jenkins-bot
89910e9164 Merge "Deprecate Parser implementation methods (will be private in next release)" 2019-10-29 21:47:42 +00:00
C. Scott Ananian
045ec0a408 Deprecate Parser implementation methods (will be private in next release)
The following public methods were renamed and made private; the old name
is hard-deprecated and calls the new renamed private method:

Parser::doMagicLinks() => handleMagicLinks()
Parser::doDoubleUnderscore() => handleMagicLinks()
Parser::doHeadings() => handleHeadings()
Parser::doAllQuotes() => handleAllQuotes()
Parser::replaceExternalLinks() => handleExternalLinks()
Parser::replaceInternalLinks() => handleInternalLinks()
Parser::replaceInternalLinks2() => handleInternalLinks2()
Parser::getVariableValue() => expandMagicVariable()
Parser::initialiseVariables() => initializeVariables()
Parser::formatHeadings() => finalizeHeadings()
Parser::test{Pst,Preprocess,Srvus}() => fuzzTest{Pst,Preprocess,Srvus}()

Additionally, the following methods are not used externally, but are
used outside the Parser class by core code.  They have been marked
@internal:

Parser::doQuotes() (used by {{#displaytitle}}),
Parser::getExternalLink{Rel,Attribs}() (used by Linker),
Parser::normalizeLinkUrl() (used by Special:LinkSearch and elsewhere).
Parser::{brace,arg,extension}Substitution() (used by PPFrame)

Code search query:
https://codesearch.wmflabs.org/deployed/?q=do%28MagicLinks%7CDoubleUnderscore%7CHeadings%7CAllQuotes%29%7Creplace%28ExternalLinks%7CInternalLinks%28%7C2%29%29%7CgetVariableValue%7CinitialiseVariables%7CformatHeadings%7Ctest%28Pst%7CPreprocess%7CSrvus%29%7CdoQuotes%7CgetExternalLink%28Rel%7CAttribs%29%7CnormalizeLinkUrl%7C%28brace%2Carg%2Cextension%29Substitution&i=nope&files=&repos=

Bug: T236810
Change-Id: I19a43ffc5dcfdd2981b51079c33422c964acb076
2019-10-29 14:00:29 -04:00
Aryeh Gregor
0de9c47b50 Remove Language::factory and getParentLanguage use
Change-Id: I11f8801ef47ec1a1f63d840116e69667e6f3ae3c
2019-10-27 12:34:28 +02:00
James D. Forrester
e64dfb4799 Parser fuzzTest: Stop trying to call hphpd_break, we don't run on HHVM any more
Change-Id: I8c7c6e2c7327811d225ac1287490ce4b92421886
2019-10-11 17:15:44 +00:00
Umherirrender
5bd311b1a2 Add public as visibility in tests folder
Add public, protected or private to function missing a visibility
Enable the tests folder for the phpcs sniff

Change-Id: Ibefce76ea9984c47e08c94889ea2eafca7565e2c
2019-10-10 21:55:37 +02:00
C. Scott Ananian
a8f20adc42 Sync up with Parsoid parserTests.txt
This now aligns with Parsoid commit 4d7cacfedc69ecac6e65f8d62f54583b900a54d2

Change-Id: I14d74032d09da1c6abf478e058e67c4040c1fcd0
2019-09-18 19:46:21 -04:00
Bartosz Dziewoński
18974098ef Remove unused localisation messages for old edit toolbar
Not used since cfa70ea6d0. Also not used
by any extensions (WikiEditor and VisualEditor have their own messages).

The message 'sig-text' used to also be part of the old edit toolbar,
but it is currently still used by WikiEditor.

Bug: T224429
Change-Id: I342351aaa2c69b5a7dde865edd38f1f2f94cfb8c
2019-09-18 14:53:48 +00:00
Brad Jorsch
c29909e59f Mostly drop old pre-actor user schemas
This removes most of the pre-actor user and user_text columns, and the
$wgActorTableSchemaMigrationStage setting that used to determine
whether the columns were used.

rev_user and rev_user_text remain in the code, as on Wikimedia wikis the
revision table is too large to alter at this time. A future change will
combine that with the removal of rev_comment, rev_content_model, and
rev_content_format (and the addition of rev_comment_id and rev_actor).

ActorMigration's constructor continues to take a $stage parameter, and
continues to have the logic for handling it, for the benefit of
extensions that might need their own migration process. Code using
ActorMigration for accessing the core fields should be updated to use
the new actor fields directly. That will be done for in a followup.

Bug: T188327
Change-Id: Id35544b879af1cd708f3efd303fce8d9a1b9eb02
2019-09-09 11:38:36 -04:00
Arlo Breault
00db615f85 Sync up with Parsoid parserTests.txt
This now aligns with Parsoid commit 06a41a99d7811a361446b894da7c5c8224398ad1

Change-Id: I9d324b3e6e9167683c15b7fee2a53b598d79a67c
2019-08-28 12:05:24 -04:00
jenkins-bot
786524a94a Merge "Let Html::element do the HTML encoding" 2019-08-28 06:57:34 +00:00
jenkins-bot
7b69a2b340 Merge "parserTests: Add parser test with filename containing single quotes" 2019-08-28 06:57:28 +00:00
Aryeh Gregor
5a6c18a086 LockManagerGroupFactory to replace singletons
100% test coverage of code that appears to be working and used, in both
LockManagerGroupFactory and also LockManagerGroup. Where possible I
wrote it as unit tests. One preexisting code path seems to be broken and
I marked the test as skipped. Two methods look unused and perhaps not
especially helpful, so I didn't write tests for them yet in case we want
to just get rid of them instead.

Change-Id: Iaa7354f31c451b87773468609c674a3bf1d4382f
2019-08-27 13:19:15 +03:00
Aryeh Gregor
752e7dd707 Convert MessageCache to service
Depends-On: Ia70e6c75f6e8a533f20cd44ebb05e013678e9951
Depends-On: I546eda0377f3a50843144b1450d3fbe8e4e02a8a
Change-Id: I305539a8598535a73e5cd280b2becdafa740ef97
2019-08-18 12:11:36 +03:00
Amir Sarabadani
807d793ab9 More clean up of oracle/mssql
Bug: T230418
Change-Id: I1ff01548e55f01f9419668e31d5aa7dd03d23258
2019-08-14 17:04:50 +02:00
Thalia
5f89f7040f Fix grammatical error in several comments
Change-Id: Ibd8ad594950d53f5feb337c30f184ecaf218bc81
2019-08-08 14:02:13 +01:00