In cases where we're operating on text data (and not binary data),
use e.g. "\u{00A0}" to refer directly to the Unicode character
'NO-BREAK SPACE' instead of "\xc2\xa0" to specify the bytes C2h A0h
(which correspond to the UTF-8 encoding of that character). This
makes it easier to look up those mysterious sequences, as not all
are as recognizable as the no-break space.
This is not enforced by PHP, but I think we should write those in
uppercase and zero-padded to at least four characters, like the
Unicode standard does.
Note that not all "\xNN" escapes can be automatically replaced:
* We can't use Unicode escapes for binary data that is not UTF-8
(e.g. in code converting from legacy encodings or testing the
handling of invalid UTF-8 byte sequences).
* '\xNN' escapes in regular expressions in single-quoted strings
are actually handled by PCRE and have to be dealt with carefully
(those regexps should probably be changed to use the /u modifier).
* "\xNN" referring to ASCII characters ("\x7F" and lower) should
probably be left as-is.
The replacements in this commit were done semi-manually by piping
the existing "\xNN" escapes through the following terrible Ruby
script I devised:
chars = eval('"' + ARGV[0] + '"').force_encoding('utf-8')
puts chars.split('').map{|char|
'\\u{' + char.ord.to_s(16).upcase.rjust(4, '0') + '}'
}.join('')
Change-Id: Idc3dee3a7fb5ebfaef395754d8859b18f1f8769a
`$a <=> $b` returns `-1` if `$a` is lesser, `1` if `$b` is lesser,
and `0` if they are equal, which are exactly the values 'sort()'
callbacks are supposed to return.
It also enables the neat idiom `$a[x] <=> $b[x] ?: $a[y] <=> $b[y]`
to sort arrays of objects first by 'x', and by 'y' if they are equal.
* Replace a common pattern like `return $a < $b ? -1 : 1` with the
new operator (and similar patterns with the variables, the numbers
or the comparison inverted). Some of the uses were previously not
correctly handling the variables being equal; this is now
automatically fixed.
* Also replace `return $a - $b`, which is equivalent to `return
$a <=> $b` if both variables are integers but less intuitive.
* (Do not replace `return strcmp( $a, $b )`. It is also equivalent
when both variables are strings, but if any of the variables is not,
'strcmp()' converts it to a string before comparison, which could
give different results than '<=>', so changing this would require
careful review and isn't worth it.)
* Also replace `return $a > $b`, which presumably sort of works most
of the time (returns `1` if `$b` is lesser, and `0` if they are
equal or `$a` is lesser) but is erroneous.
Change-Id: I19a3d2fc8fcdb208c10330bd7a42c4e05d7f5cf3
This workaround was needed when ICU in production was broken
but after T189295 this is not needed anymore and we switched off
this collation from all Persian Wikis already
Bug: T139110
Change-Id: Ifad89555b6ac96a3eb36ca24b55e1f8ee57a1f05
This prevents unexpected cuneiform digits from acting as headings for
2 and 3 on category pages.
Bug: T187645
Change-Id: I0424a24769899cb23b28704f97e1002fa44999fd
* Adding new class AbkhazUppercaseCollation, mapped to 'uppercase-ab'.
* Extended CustomUppercaseCollation with support for sorting digraphs
and for alphabets larger than 64 letters (up to 4096).
Bug: T183430
Change-Id: I16d44568e44d7ef5b39c38b1a6257b9fe10a34d4
This commit adds a custom collation order for
Northern Sami ('se'). Northern Sami exists in ICU,
but the version of ICU that Wikimedia uses is a
few years old, and does *not* include Northern
Sami. It could be years before Wikimedia's production
servers use the one that includes Northern Sami (see
bug), so this is a temporary workaround to amend this
issue.
Bug: T181503
Change-Id: Ib8a48b8db99bef8ec4b05144aace5dbdcacfeded
For some varargs a variable name is added with suffix ,... as seen for
many other varargs
Some @param are swapped, because there are in the wrong order
Enable Sniff MediaWiki.Commenting.FunctionComment.ParamNameNoMatch
Change-Id: I60fec6025bce824d5c67563ab7b65ad6cd628ad8
And auto-fix all errors.
The `<exclude-pattern>` stanzas are now included in the default ruleset
and don't need to be repeated.
Change-Id: I928af549dc88ac2c6cb82058f64c7c7f3111598a
Having such comments is worse than not having them. They add zero
information. But you must read the text to understand there is
nothing you don't already know from the class and the method name.
This is similar to I994d11e. Even more trivial, because this here is
about comments that don't say anything but "constructor".
Change-Id: I474dcdb5997bea3aafd11c0760ee072dfaff124c
They should be Ș, Ț (comma-below) and instead they were cedilla-below (Ş, Ţ).
Same as for Romanian (ro) in 486f64f283.
Both of these languages are unsupported by libicu and so the collations
are unlikely to have been used in practice.
Bug: T171043
Bug: T171044
Change-Id: Idd0d593e73cd784fbef7b75e8985f988f5555e26
Can't just clear cache on production, as this
now uses per-server apc instance.
Follow-up 486f64f283
Change-Id: I88df6d5a91c86ef687543d1a6988e0ec050bbfce
This is based on a numeric uppercase collation. Bashkir characters
will be remapped to the private use area for the purpose of sorting.
Bug: T162823
Change-Id: I65f1af0b57ff6ded7d464e39efd401f178a3519e
We have several such collations defined in IcuCollation:
* bs-Cyrl
* de-AT@collation=phonebook
* fr-CA
* sr-Latn
They couldn't actually be used.
Change-Id: I3a62073583c49d3e90910aa8240fe9fcc0682386
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.
Change-Id: I6f59febaf8fc96e80f8cfc11f4356283f461142a
We have to use a tertiary sortkey for everything with the primary
sortkey of 2627. Otherwise, the "Remove duplicate prefixes" logic
in IcuCollation would remove them.
The following characters will now be considered separate letters in
the 'xx-uca-fa' collation for the purpose of displaying the headings
on category pages: ء ئ ا و ٲ ٳ
Bug: T139110
Change-Id: Ibbea5d76348e4cdc38b74cba44286910b2ed592f
Instances of subclasses of IcuCollation with customizations for
specific languages probably shouldn't share this cache with instances
of IcuCollation with the same language.
Change-Id: I06d66d199c99448a3375381baef0366c4d99c8c4
This is based solely on looking at the bn.txt collation data
file. It has not been tested by native speakers.
Bug: T148885
Change-Id: Ide926bc5ee8752269ef6a1bfe972e19b7188d193
At this point I think it's safe to assume that these mostly work well,
and the split makes maintenance of the alphabetical list more difficult
(some entries were already in wrong order). We've been enabling these
collations for more and more Wikimedia wikis and not hearing about any
problems. Mistakes, if any are present, should be treated like any
other bug.
Also made some comments consistent.
Change-Id: I4b5fbcf4dbbdd4dc194ed821341296171fa64bb0
Based on CLDR 29 data files.
This did the relatively easy languages in CLDR 29 (Which is most
of them). I skipped languages with complicated tailoring files.
Change-Id: I8367604f7d3a1cdef9cb4e15813893c8cbfff1ff
This will cause the numeric collation to sort localized digits
for the current content language the same as how 0-9 are.
This only deals with the localized digit numbers, commas
and other number formatting are still not handled. Weird
"numerical" unicode characters are also not handled.
I was unsure if to make a "family" of numeric collations
where you specify numeric-<lang code>, or if it should
just use $wgContLang. Given that $wgContLang effectively
never changes, and also affects all other digit handling,
I opted to just use $wgContLang.
Any wikis currently using the 'numeric' collation will
have to have updateCollation.php --force run after this
change is deployed. At the moment that includes:
bnwiki, bnwikisource and hewiki
Bug: T148873
Change-Id: I9eda52a8a9752a91134d1118546b0a80d3980ccf
A few more languages marked as "Verified by native speakers",
based on which collations we've been using in production
on Wikimedia wikis.
(I'm not sure if this makes sense now that we're fairly confident
that these are good in general, but since it's already here...)
Change-Id: I8e1f31fa61509eca8c76a2df4e18638005e68b77
This collation orders text with numbers "naturally", so that
'Foo 1' < 'Foo 2' < 'Foo 12'.
Note that this only works in terms of sequences of digits, and the
behavior for decimal fractions or pretty-formatted numbers may be
unexpected.
This is only expected to work mostly correctly for English-language
text. Consider it a proof of concept. You probably want to use
an UCA collation with '-u-kn' suffix rather than this.
Bug: T8948
Change-Id: Ie268f2d92c5c75d0aaecf54ede2bdda1af3b309d
To use, add '-u-kn' to the end of a collation name and set it as
the value for $wgCategoryCollation.
Bug: T8948
Change-Id: Ica7908daf80624fa2648127114d01665e96234c0
