Enforce underscores as used by the most languages in the namespace
aliases and special page aliases
Change-Id: I53d895f4b1d241954bd31973ad4996beed91bfb2
The changes are in namespaces, special page aliases, and link trails.
The orthography was changed in 2016, and letters with apostrophes were
changed into letters with acute accents.
The old namespace names and special page aliases are kept for
backwards compatibility.
There are also a few other changes:
* The namespaces "Help" and "Template" are updated
to be consistent with translations.
* The namespace "File" is updated from "Image".
* The translation of the word "redirect" in special page aliases
is made consistent with the message translations.
* Spelling fix in the alias for Special:Shortpages.
Discussed at
https://kaa.wikipedia.org/w/index.php?oldid=53261#Namespaces
Change-Id: Icbe80c572de40f48cb065df11316cc30c2000756
Follows-up I301f471f86ba2.
For ease of navigation, move Converter subclasses to a group called
"Languages", which for documentation purposes is a subgroup of
"Language". The next commit does the same for Messages* files,
and Language subclasses (done separately for ease of review).
Change-Id: If1cef9aa15f536ebaedd4477ad7453426e7f3b85
The underscores are not strictly necessary, but appear to be good
practice. The vast majority of special page aliases we have already
use underscores (94% according to my test).
I made this a warning in the test for now, not an error.
Change-Id: I18827eb795f84bdb7895793d00bc3c895abebc7a
In cases where we're operating on text data (and not binary data),
use e.g. "\u{00A0}" to refer directly to the Unicode character
'NO-BREAK SPACE' instead of "\xc2\xa0" to specify the bytes C2h A0h
(which correspond to the UTF-8 encoding of that character). This
makes it easier to look up those mysterious sequences, as not all
are as recognizable as the no-break space.
This is not enforced by PHP, but I think we should write those in
uppercase and zero-padded to at least four characters, like the
Unicode standard does.
Note that not all "\xNN" escapes can be automatically replaced:
* We can't use Unicode escapes for binary data that is not UTF-8
(e.g. in code converting from legacy encodings or testing the
handling of invalid UTF-8 byte sequences).
* '\xNN' escapes in regular expressions in single-quoted strings
are actually handled by PCRE and have to be dealt with carefully
(those regexps should probably be changed to use the /u modifier).
* "\xNN" referring to ASCII characters ("\x7F" and lower) should
probably be left as-is.
The replacements in this commit were done semi-manually by piping
the existing "\xNN" escapes through the following terrible Ruby
script I devised:
chars = eval('"' + ARGV[0] + '"').force_encoding('utf-8')
puts chars.split('').map{|char|
'\\u{' + char.ord.to_s(16).upcase.rjust(4, '0') + '}'
}.join('')
Change-Id: Idc3dee3a7fb5ebfaef395754d8859b18f1f8769a
In some languages it's conventional not to insert a thousands
separator in numbers that are four digits long (1000-9999).
Rather than copy-paste the custom code to do this between 13 files,
introduce another option and have the base Language class handle it.
This also fixes an issue in several languages where this logic
previously would not work for negative or fractional numbers.
To implement this, a new option is added to MessagesXx.php files,
`$minimumGroupingDigits = 2;`, with the meaning as defined in
<http://unicode.org/reports/tr35/tr35-numbers.html>. It is a little
roundabout, but it could allow us to migrate the number formatting
(currently all custom code) to some generic library easily.
Bug: T177846
Change-Id: Iedd8de5648cf2de1c94044918626de2f96365d48
* Interface strings are now elsewhere
* MessagesQQQ no longer exists
* Prefer https for translatewiki.net
Change-Id: I76652ea94cca80441cd5d978029e4707ee41c4fd
The existing "linkprefix" message is unlikely to be accurately
customized by message translators (as shown by the fact that, of the 10
distinct customizations prior to Iaa7eaa44 (which made them even more
complicated), 3 were broken or entirely ineffective, 1 was half
ineffective, and 2 more seem to have included the Latin-1 Supplement by
accident) or by local wiki admins. So, like linktrail before it, let's
move it out of the system messages and into a separate language
variable.
At the same time, let's make it a simple character set (like
$wgLegalTitleChars) rather than a complicated regular expression. The
complicated regex now lives in the parser.
This also adjusts the output of the API's action=query&meta=siteinfo and
adds an accessor parallel to the linkTrail accessor to Language.
Note the following changes that are not simply extracting the existing
charset from the linkprefix message for $linkPrefixCharset:
* The En message matched all non-ASCII UTF-8 characters by matching the
component bytes (\\x80-\\xff). The new character set is equivalent.
* Various languages were identical to En and so have no $linkPrefixCharset
set. These are: Ary Az Ce Ga Id Ka Kiu Km Ltg Mk Ms Ne Nn Ro Roa_tara Sc Si
Sr_ec Sr_el Tl Tt_cyrl Tt_latn Ug_arab War
* Cu, Uk, and Udm are changed to match any number of „ or « in the prefix.
* Cv tried to include "«" that was redundant to the range \\x80-\\xff
(see En comment). This was removed.
* Diq was entirely bogus, and so was removed.
* Gu included many additional UTF-8 characters that are redundant to the
range \\x80-\\xff (see En comment). These were removed, and the
resulting character set is equivalent to En.
* Mt has been broken since it was introduced in r37242. The charset used is
equivalent to the broken regex.
Bug: 56031
Change-Id: I3369851b33113fc118a1bace38f3ac310cdd9725
The regular expression in the linkprefix message is run against the
entire page up to each wikilink, and is expected to capture one group
having everything except the prefix and another having only the prefix.
For long pages this winds up being a lot of text, so inefficient regular
expressions are going to cause problems.
The current regex is this:
/^(.*?)([a-zA-Z\\x80-\\xff]+)$/sD
This is not efficient: it will scan through the string trying to match
against every run of one or more letters/non-ASCII characters,
backtracking at every one except possibly the last. The only reason this
hasn't been a huge problem everywhere is because only a few languages
have this feature enabled.
This change replaces this with this regex:
/^((?>.*(?<![a-zA-Z\\x80-\\xff])))(.+)$/sD
This is rather more efficient: it will grab the whole string (which is
actually fast even for huge strings), then back off character by
character until it finds one that isn't a letter/non-ASCII.
Note that the above could be simplified somewhat:
/^((?>.*[^a-zA-Z\\x80-\\xff]|))(.+)$/sD
The performance improvement here is minor, and Gujarati, Church Slavic,
Udmurt, and Ukrainian would still need the other style for their current
implementations.
For Gujarati, we also use another regex trick: a look-behind assertion
in PCRE must be fixed length, so something like (?<!a|bb) won't work.
But that regex fragment is equivalent to (?<!a)(?<!bb) which is allowed,
so we use that instead.
Bug: 52865
Change-Id: Iaa7eaa446b3f045a9ce970affcb2a889f44bdefd
Format the three messages in header as one paragraph with three
sentences, instead of a paragraph and two split unordered lists with
one item each and inconsistent full stops.
Message changes: In 'wlheader-enotif' and 'wlheader-showupdated',
remove initial bullet point if present and add final full stop if
missing. First used the regexes below, then went through each language
file and manually changed the messages if applicable (e.g., Thai not
using full stops at all, Asian languages using '。', Devanagari
languages using '।' etc.)
Find: ('wlheader-[^']+'\s*=>\s*)(['"])(?:\*\s*)?([\s\S]+?)\.?\2,\n
Replace with: $1$2$3.$2,\n
Bug: 48615
Change-Id: I856f71f36d7f4b4baff5e968d88e4d3f7aeecce2
